Bae, Jae Hyeon
Mon, 15 Mar 2010 23:42:36 -0700
I tested MyStorage with local-installation of hadoop, it worked successfully!!! OMG, what's wrong with my implementation? could you guess? 2010/3/16 Bae, Jae Hyeon <metac...@gmail.com>: > I am sorry that the following error messages are generated from > taskdetails.jsp. > > I tried to investigate what caused mapreduce job to fail with reading > pig source codes, but I couldn't... > > Please let me know how I can fix it... > > I attached console output of pig execution: > > 2010-03-16 15:15:59,092 [main] INFO org.apache.pig.Main - Logging > error messages to: /home1/irteam/work/jaybae/pig/pig_1268720159091.log > 2010-03-16 15:16:00,931 [main] INFO > org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - > Connecting to hadoop file system at: hdfs://h7b611:9000/ > 2010-03-16 15:16:01,348 [main] INFO > org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - > Connecting to map-reduce job tracker at: hdfs://h7b612:9001/ > 2010-03-16 15:16:03,487 [main] INFO > org.apache.pig.impl.logicalLayer.optimizer.PruneColumns - No column > pruned for a > 2010-03-16 15:16:03,487 [main] INFO > org.apache.pig.impl.logicalLayer.optimizer.PruneColumns - No map keys > pruned for a > 2010-03-16 15:16:05,765 [main] INFO > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer > - MR plan size before optimization: 1 > 2010-03-16 15:16:05,765 [main] INFO > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer > - MR plan size after optimization: 1 > 2010-03-16 15:16:08,856 [main] INFO > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler > - Setting up single store job > 2010-03-16 15:16:10,046 [main] INFO > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher > - Cannot get jobid for this job > 2010-03-16 15:16:11,746 [Thread-9] WARN > org.apache.hadoop.mapred.JobClient - Use GenericOptionsParser for > parsing the arguments. Applications should implement Tool for the > same. > 2010-03-16 15:16:20,002 [main] INFO > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher > - 0% complete > 2010-03-16 15:16:26,138 [main] INFO > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher > - 100% complete > 2010-03-16 15:16:26,427 [main] ERROR > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher > - 1 map reduce job(s) failed! > 2010-03-16 15:16:26,437 [main] ERROR > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher > - Failed to produce result in: > "hdfs://h7b611:9000/tmp/temp-1085687982/tmp1226074556" > 2010-03-16 15:16:26,439 [main] INFO > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher > - Failed! > 2010-03-16 15:16:27,271 [main] ERROR org.apache.pig.tools.grunt.Grunt > - ERROR 1066: Unable to open iterator for alias b > Details at logfile: pig_1268720159091.log > > > Here is the log file > > Pig Stack Trace > --------------- > ERROR 1066: Unable to open iterator for alias b > > org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1066: Unable > to open iterator for alias b > at org.apache.pig.PigServer.openIterator(PigServer.java:482) > at org.apache.pig.tools.grunt.GruntParser.processDump(GruntParser.java:539) > at > org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:241) > at > org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:168) > at > org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:144) > at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:89) > at org.apache.pig.Main.main(Main.java:386) > Caused by: java.io.IOException: Job terminated with anomalous status FAILED > at org.apache.pig.PigServer.openIterator(PigServer.java:476) > ... 6 more > ================================================================================ > > > 2010/3/16 Bae, Jae Hyeon <metac...@gmail.com>: >> Hi everyone. >> >> I implemented LoadFunc and Slicer interface and I tried to use it with >> the following pig script: >> >> a = LOAD '/data/part-00000.anc.gz' USING MyStorage() AS (source:chararray); >> b = FILTER a BY (source == 'http://www.sonyfinance.co.jp/'); >> DUMP b; >> >> MapReduce job is dead immediately with the following error stack trace: >> >> java.lang.ArrayIndexOutOfBoundsException: 0 >> at >> org.apache.hadoop.mapred.JobInProgress.getTaskInProgress(JobInProgress.java:2523) >> at >> org.apache.hadoop.mapred.taskdetails_jsp._jspService(taskdetails_jsp.java:113) >> at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:97) >> at javax.servlet.http.HttpServlet.service(HttpServlet.java:820) >> at >> org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:502) >> at >> org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:363) >> at >> org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216) >> at >> org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:181) >> at >> org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766) >> at >> org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:417) >> at >> org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230) >> at >> org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152) >> at org.mortbay.jetty.Server.handle(Server.java:324) >> at >> org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:534) >> at >> org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:864) >> at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:533) >> at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:207) >> at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:403) >> at >> org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:409) >> at >> org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:522) >> >> What should I do to solve this problem? I am using hadoop-0.20.1 and >> pig-0.6.0. >> >