[
https://issues.apache.org/jira/browse/HADOOP-12260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14638820#comment-14638820
]
Marius commented on HADOOP-12260:
---------------------------------
I was able to recreate this error on fresh machine with a single node setup
running Linux Mint 17.2. The error occured in the same was as before even with
a different avro file (took this:
https://cwiki.apache.org/confluence/display/AVRO/Interoperability+Testing but
wrote a 100MB file by looping over append DATUM).
The error occured with pydoop as well as with the normal streaming command.
Marius
> BlockSender.sendChunks() exception
> ----------------------------------
>
> Key: HADOOP-12260
> URL: https://issues.apache.org/jira/browse/HADOOP-12260
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 2.6.0, 2.7.1
> Environment: OS: CentOS Linux release 7.1.1503 (Core)
> Kernel: 3.10.0-229.1.2.el7.x86_64
> Reporter: Marius
>
> Hi
> I was running some streaming jobs with avro files from my hadoop cluster.
> They performed poorly so i checked the logs of my datanodes and found this:
> http://pastebin.com/DXKJJ55z
> The cluster is running on CentOS machines:
> CentOS Linux release 7.1.1503 (Core)
> This is the Kernel:
> 3.10.0-229.1.2.el7.x86_64
> No one on the userlist replied and i could not find anything helpful on the
> internet despite disk failure which is unlikely to cause this because here
> are several machines and its not very likely that all of their disks fail at
> the same time.
> This error is not reported on the console when running a job and the error
> occurs from time to time and then dissapears and comes back again.
> The block size of the cluster is the default value.
> This is my command:
> hadoop jar hadoop-streaming-2.7.1.jar -files mapper.py,reducer.py,avro-1.
> 7.7.jar,avro-mapred-1.7.7-hadoop2.jar -D mapreduce.job.reduces=15 -libjars
> avro-1.7.7.jar,avro-mapred-1.7.7-hadoop2.jar -input /Y/Y1.avro -output
> /htest/output -mapper mapper.py -reducer reducer.py -inputformat
> org.apache.avro.mapred.AvroAsTextInputFormat
> Marius
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
