I traced into the c++ recordreader code:
WordCountReader(HadoopPipes::MapContext& context) {
std::string filename;
HadoopUtils::StringInStream stream(context.getInputSplit());
HadoopUtils::deserializeString(filename, stream);
struct stat statResult;
stat(filename.c_str(), &statResult);
bytesTotal = statResult.st_size;
bytesRead = 0;
cout << filename<<endl;
file = fopen(filename.c_str(), "rt");
HADOOP_ASSERT(file != NULL, "failed to open " + filename);
}
I got nothing for the filename virable, which showed the InputSplit is
empty.
2008/3/4, 11 Nov. <[EMAIL PROTECTED]>:
>
> hi colleagues,
> I have set up the single node cluster to test pipes examples.
> wordcount-simple and wordcount-part work just fine. but
> wordcount-nopipe can't run. Here is my commnad line:
>
> bin/hadoop pipes -conf src/examples/pipes/conf/word-nopipe.xml -input
> input/ -output out-dir-nopipe1
>
> and here is the error message printed on my console:
>
> 08/03/03 23:23:06 WARN mapred.JobClient: No job jar file set. User
> classes may not be found. See JobConf(Class) or JobConf#setJar(String).
> 08/03/03 23:23:06 INFO mapred.FileInputFormat: Total input paths to
> process : 1
> 08/03/03 23:23:07 INFO mapred.JobClient: Running job:
> job_200803032218_0004
> 08/03/03 23:23:08 INFO mapred.JobClient: map 0% reduce 0%
> 08/03/03 23:23:11 INFO mapred.JobClient: Task Id :
> task_200803032218_0004_m_000000_0, Status : FAILED
> java.io.IOException: pipe child exception
> at org.apache.hadoop.mapred.pipes.Application.abort(
> Application.java:138)
> at org.apache.hadoop.mapred.pipes.PipesMapRunner.run(
> PipesMapRunner.java:83)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:192)
> at org.apache.hadoop.mapred.TaskTracker$Child.main(
> TaskTracker.java:1787)
> Caused by: java.io.EOFException
> at java.io.DataInputStream.readByte(DataInputStream.java:250)
> at org.apache.hadoop.io.WritableUtils.readVLong(WritableUtils.java
> :313)
> at org.apache.hadoop.io.WritableUtils.readVInt(WritableUtils.java
> :335)
> at
> org.apache.hadoop.mapred.pipes.BinaryProtocol$UplinkReaderThread.run(
> BinaryProtocol.java:112)
>
> task_200803032218_0004_m_000000_0:
> task_200803032218_0004_m_000000_0:
> task_200803032218_0004_m_000000_0:
> task_200803032218_0004_m_000000_0: Hadoop Pipes Exception: failed to open
> at /home/hadoop/hadoop-0.15.2-single-cluster
> /src/examples/pipes/impl/wordcount-nopipe.cc:67 in
> WordCountReader::WordCountReader(HadoopPipes::MapContext&)
>
>
> Could anybody tell me how to fix this? That will be appreciated.
> Thanks a lot!
>
