[
https://issues.apache.org/jira/browse/MAPREDUCE-2584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13508585#comment-13508585
]
Harsh J commented on MAPREDUCE-2584:
------------------------------------
Problem is coming from the fact that when we lookup the IO_SERIALIZATIONS_KEY,
we don't pass a default classes set.
We could add that in, but it may get harder to maintain it if we extend the
list. In any case, I'll upload another patch.
{code}
java.io.IOException: Couldn't find a serializer for the Map-Output Key class:
'class org.apache.hadoop.io.LongWritable'. If custom serialization is being
used, ensure that the 'io.serializations' property is appropriately configured
for the Job.
at
org.apache.hadoop.mapreduce.JobSubmitter.checkSerializerSpecs(JobSubmitter.java:462)
at
org.apache.hadoop.mapreduce.JobSubmitter.checkSpecs(JobSubmitter.java:424)
at
org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:338)
at org.apache.hadoop.mapreduce.Job$11.run(Job.java:1218)
at org.apache.hadoop.mapreduce.Job$11.run(Job.java:1215)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1437)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:1215)
at org.apache.hadoop.mapred.JobClient$1.run(JobClient.java:617)
at org.apache.hadoop.mapred.JobClient$1.run(JobClient.java:612)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1437)
at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:612)
at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:890)
at
org.apache.hadoop.conf.TestNoDefaultsJobConf.testNoDefaults(TestNoDefaultsJobConf.java:82)
{code}
> Check for serializers early, and give out more information regarding missing
> serializers
> ----------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-2584
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2584
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: task
> Affects Versions: 0.20.2
> Reporter: Harsh J
> Assignee: Harsh J
> Labels: serializers, tasks
> Attachments: MAPREDUCE-2584.r2.diff, MAPREDUCE-2584.r3.diff,
> MAPREDUCE-2584.r4.diff, MAPREDUCE-2584.r5.diff, MAPREDUCE-2584.r6.diff,
> MAPREDUCE-2584.r7.diff, MAPREDUCE-2584.r7.diff, MAPREDUCE-2584.r8.diff,
> MAPREDUCE-2584.r9.diff
>
>
> As discussed on HADOOP-7328, MapReduce can handle serializers in a much
> better way in case of bad configuration, improper imports (Some odd Text
> class instead of the Writable Text set as key), etc..
> This issue covers the MapReduce parts of the improvements (made to IFile,
> MapOutputBuffer, etc. and possible early-check of serializer availability
> pre-submit) that provide more information than just an NPE as is the current
> case.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
