Bae, Jae Hyeon
Tue, 16 Mar 2010 02:30:00 -0700
I found a solution... Implementation of MySlice.getLocations() had a bug :) Sorry for disturbing you... 2010/3/16 Bae, Jae Hyeon <metac...@gmail.com>: > I tested MyStorage with local-installation of hadoop, it worked > successfully!!! > > OMG, what's wrong with my implementation? could you guess? > > 2010/3/16 Bae, Jae Hyeon <metac...@gmail.com>: >> I am sorry that the following error messages are generated from >> taskdetails.jsp. >> >> I tried to investigate what caused mapreduce job to fail with reading >> pig source codes, but I couldn't... >> >> Please let me know how I can fix it... >> >> I attached console output of pig execution: >> >> 2010-03-16 15:15:59,092 [main] INFO org.apache.pig.Main - Logging >> error messages to: /home1/irteam/work/jaybae/pig/pig_1268720159091.log >> 2010-03-16 15:16:00,931 [main] INFO >> org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - >> Connecting to hadoop file system at: hdfs://h7b611:9000/ >> 2010-03-16 15:16:01,348 [main] INFO >> org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - >> Connecting to map-reduce job tracker at: hdfs://h7b612:9001/ >> 2010-03-16 15:16:03,487 [main] INFO >> org.apache.pig.impl.logicalLayer.optimizer.PruneColumns - No column >> pruned for a >> 2010-03-16 15:16:03,487 [main] INFO >> org.apache.pig.impl.logicalLayer.optimizer.PruneColumns - No map keys >> pruned for a >> 2010-03-16 15:16:05,765 [main] INFO >> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer >> - MR plan size before optimization: 1 >> 2010-03-16 15:16:05,765 [main] INFO >> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer >> - MR plan size after optimization: 1 >> 2010-03-16 15:16:08,856 [main] INFO >> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler >> - Setting up single store job >> 2010-03-16 15:16:10,046 [main] INFO >> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher >> - Cannot get jobid for this job >> 2010-03-16 15:16:11,746 [Thread-9] WARN >> org.apache.hadoop.mapred.JobClient - Use GenericOptionsParser for >> parsing the arguments. Applications should implement Tool for the >> same. >> 2010-03-16 15:16:20,002 [main] INFO >> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher >> - 0% complete >> 2010-03-16 15:16:26,138 [main] INFO >> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher >> - 100% complete >> 2010-03-16 15:16:26,427 [main] ERROR >> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher >> - 1 map reduce job(s) failed! >> 2010-03-16 15:16:26,437 [main] ERROR >> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher >> - Failed to produce result in: >> "hdfs://h7b611:9000/tmp/temp-1085687982/tmp1226074556" >> 2010-03-16 15:16:26,439 [main] INFO >> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher >> - Failed! >> 2010-03-16 15:16:27,271 [main] ERROR org.apache.pig.tools.grunt.Grunt >> - ERROR 1066: Unable to open iterator for alias b >> Details at logfile: pig_1268720159091.log >> >> >> Here is the log file >> >> Pig Stack Trace >> --------------- >> ERROR 1066: Unable to open iterator for alias b >> >> org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1066: Unable >> to open iterator for alias b >> at org.apache.pig.PigServer.openIterator(PigServer.java:482) >> at >> org.apache.pig.tools.grunt.GruntParser.processDump(GruntParser.java:539) >> at >> org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:241) >> at >> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:168) >> at >> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:144) >> at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:89) >> at org.apache.pig.Main.main(Main.java:386) >> Caused by: java.io.IOException: Job terminated with anomalous status FAILED >> at org.apache.pig.PigServer.openIterator(PigServer.java:476) >> ... 6 more >> ================================================================================ >> >> >> 2010/3/16 Bae, Jae Hyeon <metac...@gmail.com>: >>> Hi everyone. >>> >>> I implemented LoadFunc and Slicer interface and I tried to use it with >>> the following pig script: >>> >>> a = LOAD '/data/part-00000.anc.gz' USING MyStorage() AS (source:chararray); >>> b = FILTER a BY (source == 'http://www.sonyfinance.co.jp/'); >>> DUMP b; >>> >>> MapReduce job is dead immediately with the following error stack trace: >>> >>> java.lang.ArrayIndexOutOfBoundsException: 0 >>> at >>> org.apache.hadoop.mapred.JobInProgress.getTaskInProgress(JobInProgress.java:2523) >>> at >>> org.apache.hadoop.mapred.taskdetails_jsp._jspService(taskdetails_jsp.java:113) >>> at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:97) >>> at javax.servlet.http.HttpServlet.service(HttpServlet.java:820) >>> at >>> org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:502) >>> at >>> org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:363) >>> at >>> org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216) >>> at >>> org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:181) >>> at >>> org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766) >>> at >>> org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:417) >>> at >>> org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230) >>> at >>> org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152) >>> at org.mortbay.jetty.Server.handle(Server.java:324) >>> at >>> org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:534) >>> at >>> org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:864) >>> at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:533) >>> at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:207) >>> at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:403) >>> at >>> org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:409) >>> at >>> org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:522) >>> >>> What should I do to solve this problem? I am using hadoop-0.20.1 and >>> pig-0.6.0. >>> >> >