Tuesday, July 1, 2014

ChecksumException in HDFS


Came across this exception a couple of times.

/mnt/softwares/hadoop/bin/hadoop fs -copyFromLocal /mnt/folder1/folder2/file.tsv /mnt/folder1/folder2/file.tsv
14/07/01 09:47:58 INFO fs.FSInputChecker: Found checksum error: b[0, 0]=
org.apache.hadoop.fs.ChecksumException: Checksum error: /mnt/folder1/folder2/file.tsv at 0
        at org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSInputChecker.readChunk(ChecksumFileSystem.java:219)
        at org.apache.hadoop.fs.FSInputChecker.readChecksumChunk(FSInputChecker.java:237)
        at org.apache.hadoop.fs.FSInputChecker.read1(FSInputChecker.java:189)
        at org.apache.hadoop.fs.FSInputChecker.read(FSInputChecker.java:158)
        at java.io.DataInputStream.read(DataInputStream.java:83)
        at org.apache.hadoop.io.IOUtils.copyBytes(IOUtils.java:68)
        at org.apache.hadoop.io.IOUtils.copyBytes(IOUtils.java:47)
        at org.apache.hadoop.io.IOUtils.copyBytes(IOUtils.java:100)
        at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:230)
        at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:176)
        at org.apache.hadoop.fs.FileSystem.copyFromLocalFile(FileSystem.java:1209)
        at org.apache.hadoop.fs.FsShell.copyFromLocal(FsShell.java:138)
        at org.apache.hadoop.fs.FsShell.run(FsShell.java:1950)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
        at org.apache.hadoop.fs.FsShell.main(FsShell.java:2090)
copyFromLocal: Checksum error: /mnt/folder1/folder2/file.tsv at 0

Seems like it is a bug reported into hadoop

Simple solution: Remove the crc file from the folder from where you are copying the file.

Cheers.

39 comments :

angela michael said...

horizon security services inc I would like to thank you for your nicely written post, its informative and your writing style encouraged me to read it till end. Thanks

Rasool Bevi said...

The expansion of internet and intelligence in business process lead the way to huge volume of data. It is important to maintain and process these data to be efficient in data handling.
Hadoop Training in Chennai

geethu said...

Updating with the latest technology and implementing it is the only way to survive in our niche. Thanks for making me this article. You have done a great job by sharing this content in here. Keep writing article like this.
SAS Training in Chennai | SAS Course in Chennai

Mark Smith said...

Rolex Watches Authentic Mens & Ladies Rolex Datejust, President Watches for Sale at JavyEstrella.com.

efloral Gift said...

online birthday flowers delivery eFloralgifts is one of the premier floral and gifting retailer, offering gifts for all occasions like Birthdays, Anniversaries , Thank you , Weddings, Sympathy and many others.

jyoti roy said...

Thank you for sharing such a wonderful post. Keep rocking!! CCNA Training Institute in Chennai | CCNP Training Institute in Chennai.

karuma said...

Thanks for your great information..keep updating...
MCSE Certification Exam Center in Chennai | MCSE Certification Training in Chennai | MCSE Exam Center in Chennai | MCSE Training Institute in Chennai

sivanesan said...

The best thing is that your blog really informative thanks for your great information!
erp in chennai

anitha a said...

stuff The best thing is that your blog really informative thanks for your great information!
Web designing course in chennai

vini vinitha said...

I have read this blog, it is interesting and informative.thanks for sharing...
campus events | college paper presentaion

prabash said...

Excellent and useful post. Thanks for taking a time to share Web Designing Training Institute in Chennai | Web Designing Training Institute in Velachery

devo said...

Good post.Thank you for sharing such a wonderful blog...Dot Net Training Institute in Chennai | Dot Net Training Institute in Velachery

srihariparu said...

This was so useful and informative. The article helped me to learn something new.
Six Sigma Certification Training in Chennai | No.1 Six Sigma Certification Exam Center in Chennai

deepti said...

Excellent post! Thank you for Sharing..Android Training Institute in Chennai | Android Training Institute in Velachery

rupiya rupi said...

Thanks for sharing this unique and informative content..keep updating..
Dot Net Training in Chennai | No.1 Dot Net Training Institute in Chennai | Best Dot Net Training Center in Chennai

nancy said...

Really an amazing post..! By reading your blog post i gained more information.
Final Year Project Center in Chennai | IEEE Project Center in Chennai | Diploma Project Center in Chennai

ASO Services said...

Thanks for your marvelous posting! It is very useful and good. Come on. I want to introduce the best aso services for you, I try it and I feel it is so good to rank app to top in app store search results, have you ever heard it?

JamesWilliam said...
This comment has been removed by the author.
JamesWilliam said...

Informative blog and it was up to the point describing the information very effectively. Thanks to blog author for wonderful and informative post. Also great with all of the valuable information on mobile app and you are doing well.
Mobile application developers in Chennai | Android application developers in Chennai

JamesWilliam said...


Interesting post! This is really helpful for me. I like it! Thanks for sharing!
PHP developers Chennai | PHP developers in Chennai

PLC Training in Chennai said...

thanks for sharing

PLC Training Chennai said...

Thanks a lot for providing this with us. Keep on updating this with us regularly...


PLC Training in Chennai | PLC Training Institute in Chennai | PLC Training Center in Chennai | PLC SCADA Training in Chennai | Best PLC Training in Chennai | Best PLC Training Institute in Chennai | PLC Training in Kerala | AutoCAD Training in Chennai | Embedded Systems Training in Chennai | VLSI Training in Chennai | Inplant Training in Chennai

JamesWilliam said...

Informative blog and it was up to the point describing the information very effectively. Thanks to blog author for wonderful and informative post. Also great with all of the
valuable information on mobile app and you are doing well.
Mobile application developers in Chennai | Android application developers in Chennai | Android app developers Chennai

Barath BK said...

It’s nice post of getting strongest link from Google Plus, i will share it to my seo circle. Keep updated.
CorelDraw Training Institute in Chennai | CorelDraw Training Institute in Velachery

sherin angel said...

Yet another excellent post! I will be bookmarking your blog. I shared it on a few social media sites :) Thanks for sharing! Keep these great articles coming!
MCA Project Center in Chennai | MCA Project Center in Velachery

DAVIS MILLER said...

I wish to show thanks to you just for bailing me out of this particular
trouble.As a result of checking through the net and meeting
techniques that were not productive, I thought my life was done.

java training in chennai

Gowri S said...

Expected to form you a next to no word to thank you once more with respect to the decent recommendations you've contributed here.Big Data Training in Marathahalli

Embedded training in chennai said...

Nice post... thank for sharing..its really interesting..

PLC training institute in chennai | Embedded training in Chennai

praveen said...

Articles that you have provided contains a very interesting useful information....VLSI Project Center in Chennai | VLSI Project Center in Velachery

devo said...

Very good and informative article. Thanks for sharing such nice article, keep on updating..
Best Microsoft Azure Training Institute in Chennai | Best Microsoft Azure Training Institute in Velachery

Barath BK said...

There was very wonderful information and that's great one. I really appreciate the kind words, thanks for sharing that valuable information.
Best Graphics Designing Training Institute in Chennai | No.1 Graphics Designing Training Institute in Velachery

karthik said...


Nice post.I am really enjoying reading your well written articles. thanks for sharing..keep on updating..
Best VMware Training Institute in Chennai | Best VMware Training Institute in Velachery

muthu said...

This is a great stuff and keeps on sharing a more informative post like this.
Best Adobe Photoshop Training Institute in Chennai | No.1 Photoshop Training Institute in Chennai

Sarath K said...

Very inspiring article for us Thanks for the great article. …. It’s definitely a good idea for me and my friends to learn the things that’ll expand their skill set your ideas and presentation is very effective and useful for all. I am loving all of the in turn you are sharing with each one…
Best CorelDraw Training Institute in Chennai | No.1 Photoshop Training Institute in Chennai

kalai hai said...

Thanks for sharing this information and keep updating us. This information is very useful to me.
Best Graphics Designing Training Institute in Chennai | No.1 CorelDraw Training Institute in Chennai| Excellent Photoshop Training Institute in Velachery

Jahan Murugassan said...

There was very wonderful information and that's great one. I really appreciate the kind words, thanks for sharing that valuable information.

white label website builder

JamesWilliam said...


Thanks for sharing this niche useful informative post to our knowledge.
brochure designers in chennai | brochure design company in chennai

deepika ranjith said...

Excellent post! Thank you for Sharing.

Apache Spark Training in Chennai
| Apache Spark Online Training

Anoushka Sakthi said...

Energetic blog with loads and loads of info. Keep doing more.


Selenium training in Chennai
Selenium Courses in Chennai
best ios training in chennai
Digital Marketing Training in Chennai
JAVA J2EE Training Institutes in Chennai
Loadrunner course in Chennai
Qtp training institutes in chennai