[
https://issues.apache.org/jira/browse/HAMA-784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ikhtiyor Ahmedov updated HAMA-784:
----------------------------------
Description:
Example code which fails:
{code}KeyValueTextInputFormat.addInputPaths(job,
"/tmp/test.seq,/tmp/test2.seq,/tmp/test3.seq");{code}
Exception:
{code}
java.lang.ClassCastException: org.apache.hadoop.io.Text cannot be cast to
org.apache.hadoop.io.LongWritable
at org.apache.hama.bsp.LineRecordReader.next(LineRecordReader.java:1)
at
org.apache.hama.bsp.TrackedRecordReader.moveToNext(TrackedRecordReader.java:63)
at
org.apache.hama.bsp.TrackedRecordReader.next(TrackedRecordReader.java:49)
at org.apache.hama.bsp.BSPPeerImpl.readNext(BSPPeerImpl.java:598)
at org.apache.hama.examples.OnlineCF$OnlineCFBSP.bsp(OnlineCF.java:146)
at
org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:256)
at
org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
at
org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:1)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
{code}
Reason: below is part of implemented function of abstract class which is not
overridden by extended classes (unfortunately static method)
{code:title=FileInputFormat.java}
public static void addInputPaths(BSPJob conf, String commaSeparatedPaths) {
for (String str : getPathStrings(commaSeparatedPaths)) {
addInputPath(conf, new Path(str));
}
}
{code}
Quick fix:
anyway set input format
{code}
job.setInputFormat(KeyValueTextInputFormat.class);//set explicitly
KeyValueTextInputFormat.addInputPaths(job,
"/tmp/test.seq,/tmp/test2.seq,/tmp/test3.seq");
{code}
or add modification to the documentation in link
(http://wiki.apache.org/hama/BSPModel), so that new developers will explicitly
set input format even they use static methods of InputFormat.
was:
Example code which fails:
{code}KeyValueTextInputFormat.addInputPaths(job,
"/tmp/test.seq,/tmp/test2.seq,/tmp/test3.seq");{code}
Exception:
{code}
java.lang.ClassCastException: org.apache.hadoop.io.Text cannot be cast to
org.apache.hadoop.io.LongWritable
at org.apache.hama.bsp.LineRecordReader.next(LineRecordReader.java:1)
at
org.apache.hama.bsp.TrackedRecordReader.moveToNext(TrackedRecordReader.java:63)
at
org.apache.hama.bsp.TrackedRecordReader.next(TrackedRecordReader.java:49)
at org.apache.hama.bsp.BSPPeerImpl.readNext(BSPPeerImpl.java:598)
at org.apache.hama.examples.OnlineCF$OnlineCFBSP.bsp(OnlineCF.java:146)
at
org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:256)
at
org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
at
org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:1)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
{code}
Reason: below is part of implemented function of abstract class which is not
overridden by extended classes
{code:title=FileInputFormat.java}
public static void addInputPaths(BSPJob conf, String commaSeparatedPaths) {
for (String str : getPathStrings(commaSeparatedPaths)) {
addInputPath(conf, new Path(str));
}
}
{code}
Quick fix:
anyway set input format
{code}
job.setInputFormat(KeyValueTextInputFormat.class);//set explicitly
KeyValueTextInputFormat.addInputPaths(job,
"/tmp/test.seq,/tmp/test2.seq,/tmp/test3.seq");
{code}
or add modification to the documentation in link
(http://wiki.apache.org/hama/BSPModel), so that new developers will explicitly
set input format even they use static methods of InputFormat.
> Input format class type is not considered when given multiple input paths
> -------------------------------------------------------------------------
>
> Key: HAMA-784
> URL: https://issues.apache.org/jira/browse/HAMA-784
> Project: Hama
> Issue Type: Bug
> Components: bsp core
> Reporter: Ikhtiyor Ahmedov
> Labels: inputformat
>
> Example code which fails:
> {code}KeyValueTextInputFormat.addInputPaths(job,
> "/tmp/test.seq,/tmp/test2.seq,/tmp/test3.seq");{code}
> Exception:
> {code}
> java.lang.ClassCastException: org.apache.hadoop.io.Text cannot be cast to
> org.apache.hadoop.io.LongWritable
> at org.apache.hama.bsp.LineRecordReader.next(LineRecordReader.java:1)
> at
> org.apache.hama.bsp.TrackedRecordReader.moveToNext(TrackedRecordReader.java:63)
> at
> org.apache.hama.bsp.TrackedRecordReader.next(TrackedRecordReader.java:49)
> at org.apache.hama.bsp.BSPPeerImpl.readNext(BSPPeerImpl.java:598)
> at org.apache.hama.examples.OnlineCF$OnlineCFBSP.bsp(OnlineCF.java:146)
> at
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:256)
> at
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
> at
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:1)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> {code}
> Reason: below is part of implemented function of abstract class which is not
> overridden by extended classes (unfortunately static method)
> {code:title=FileInputFormat.java}
> public static void addInputPaths(BSPJob conf, String commaSeparatedPaths) {
> for (String str : getPathStrings(commaSeparatedPaths)) {
> addInputPath(conf, new Path(str));
> }
> }
> {code}
> Quick fix:
> anyway set input format
> {code}
> job.setInputFormat(KeyValueTextInputFormat.class);//set explicitly
> KeyValueTextInputFormat.addInputPaths(job,
> "/tmp/test.seq,/tmp/test2.seq,/tmp/test3.seq");
> {code}
> or add modification to the documentation in link
> (http://wiki.apache.org/hama/BSPModel), so that new developers will
> explicitly set input format even they use static methods of InputFormat.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
