Hi Daniel I saw there is a line using "ordered by" in the script1-local.pig file.
"ordered_uniq_frequency = ORDER filtered_uniq_frequency BY hour, score;" Regards! Chen On Fri, Jun 6, 2014 at 12:17 AM, Chen He <airb...@gmail.com> wrote: > Thank you for the reply, Daniel. I will check tomorrow. Actually, i just > use tutorial script1-local. > > Regards! > > Chen > On Jun 5, 2014 11:42 PM, "Daniel Dai" <da...@hortonworks.com> wrote: > >> Hi, Chen, >> Are you seeing this issue only when running a query containing "order >> by"? It seems a Pig bug on this side, I can take a look tomorrow. But >> in worst case, we can disable ObjectRegistry in local mode. >> >> Thanks, >> Daniel >> >> On Thu, Jun 5, 2014 at 9:14 PM, Chen He <airb...@gmail.com> wrote: >> > Hi Hitesh >> > >> > ObjectRegistry is used for broadcastShuffle. If a vertex has more than >> one >> > task. Pig uses ObjectRegistry to save objects that can be used for all >> > tasks in a vertex. The ObjectRegistryImpl's annotation is singleton. I >> > suspect it is a race condition between constructor and caller. >> > >> > Regards! >> > >> > Chen >> > >> > >> > On Thu, Jun 5, 2014 at 8:49 PM, Hitesh Shah <hit...@apache.org> wrote: >> > >> >> @Chen, I am not sure asking for the user code to change is an option. >> We >> >> should address the underlying issue which is fixing ObjectRegistry to >> work >> >> in local mode. >> >> >> >> thanks >> >> -- Hitesh >> >> >> >> >> >> On Thu, Jun 5, 2014 at 1:34 PM, Chen He <airb...@gmail.com> wrote: >> >> >> >> > Hi Hitesh >> >> > >> >> > In my code, I did not touch any ObjectRegistry related code. I >> suspect >> >> the >> >> > instantiation process of ObjectRegistry has problem so that it >> failed to >> >> > create an instance when it is called. >> >> > >> >> > I didn't know why PigProcessor collectSample vertex. But looks like >> it is >> >> > not a must for TezLocalMode. >> >> > >> >> > Chen >> >> > >> >> > >> >> > On Thu, Jun 5, 2014 at 2:59 PM, Chen He <airb...@gmail.com> wrote: >> >> > >> >> > > Hi Hitesh >> >> > > >> >> > > Yes, objectRegistry returns NPE. I saw it is a inject static >> variable. >> >> > > >> >> > > Regards! >> >> > > >> >> > > Chen >> >> > > >> >> > > >> >> > > On Thu, Jun 5, 2014 at 9:15 AM, Hitesh Shah <hit...@apache.org> >> wrote: >> >> > > >> >> > >> @Chen, this might be a bug in the tez local execution code. Can >> you >> >> > >> confirm >> >> > >> that ObjectRegistryFactory::getObjectRegistry returns a non-null >> >> value? >> >> > >> >> >> > >> thanks >> >> > >> — Hitesh >> >> > >> >> >> > >> >> >> > >> On Wed, Jun 4, 2014 at 4:21 PM, Chen He <airb...@gmail.com> >> wrote: >> >> > >> >> >> > >> > I am using pig -x tez_local to test tez local mode and met >> following >> >> > >> > problems (use tutorial.jar) >> >> > >> > >> >> > >> > 2014-06-03 11:06:35,964 [pool-1-thread-1] INFO >> >> > >> > org.apache.hadoop.mapred.YarnTezDagChild - Running task, >> >> > >> > taskAttemptId=attempt_1401811588093_0000_1_05_000000_0 >> >> > >> > 2014-06-03 11:06:35,965 [pool-1-thread-1] ERROR >> >> > >> > org.apache.pig.backend.hadoop.executionengine.tez.PigProcessor - >> >> > >> > Encountered exception while processing: >> >> > >> > java.lang.NullPointerException >> >> > >> > at >> >> > >> > >> >> > >> > >> >> > >> >> >> > >> >> >> org.apache.pig.backend.hadoop.executionengine.tez.ObjectCache.retrieve(ObjectCache.java:47) >> >> > >> > at >> >> > >> > >> >> > >> > >> >> > >> >> >> > >> >> >> org.apache.pig.backend.hadoop.executionengine.tez.PigProcessor.collectSample(PigProcessor.java:307) >> >> > >> > at >> >> > >> > >> >> > >> > >> >> > >> >> >> > >> >> >> org.apache.pig.backend.hadoop.executionengine.tez.PigProcessor.initializeInputs(PigProcessor.java:218) >> >> > >> > at >> >> > >> > >> >> > >> > >> >> > >> >> >> > >> >> >> org.apache.pig.backend.hadoop.executionengine.tez.PigProcessor.run(PigProcessor.java:162) >> >> > >> > at >> >> > >> > >> >> > >> > >> >> > >> >> >> > >> >> >> org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:317) >> >> > >> > at >> >> > >> > >> >> > >> org.apache.hadoop.mapred.YarnTezDagChild$3.run(YarnTezDagChild.java:602) >> >> > >> > at java.security.AccessController.doPrivileged(Native >> >> Method) >> >> > >> > at javax.security.auth.Subject.doAs(Subject.java:415) >> >> > >> > at >> >> > >> > >> >> > >> > >> >> > >> >> >> > >> >> >> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1491) >> >> > >> > at >> >> > >> > >> >> > >> >> >> > >> >> >> org.apache.hadoop.mapred.YarnTezDagChild.pollTask(YarnTezDagChild.java:591) >> >> > >> > at >> >> > >> > >> >> > >> >> >> > >> >> >> org.apache.hadoop.mapred.YarnTezDagChild.runTask(YarnTezDagChild.java:332) >> >> > >> > at >> >> > >> > >> >> > >> > >> >> > >> >> >> > >> >> >> org.apache.tez.dag.app.launcher.LocalContainerLauncher$1.run(LocalContainerLauncher.java:250) >> >> > >> > at java.lang.Thread.run(Thread.java:744) >> >> > >> > at >> >> > >> > >> >> > >> > >> >> > >> >> >> > >> >> >> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) >> >> > >> > at >> >> > >> > >> >> > >> > >> >> > >> >> >> > >> >> >> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) >> >> > >> > at java.lang.Thread.run(Thread.java:744) >> >> > >> > >> >> > >> > code checking implies that the ObjectCache is trying to get >> >> something >> >> > >> from >> >> > >> > a Map but it is not there. >> >> > >> > But what is the purpose of using ObjectCache in PIG on TEZ? >> >> > >> > Why pig needs sampleVertex in the PigProcessor? >> >> > >> > >> >> > >> > Any reply will be appreciated. >> >> > >> > >> >> > >> > Regards! >> >> > >> > >> >> > >> > Chen >> >> > >> > >> >> > >> >> >> > > >> >> > > >> >> > >> >> >> >> -- >> CONFIDENTIALITY NOTICE >> NOTICE: This message is intended for the use of the individual or entity >> to >> which it is addressed and may contain information that is confidential, >> privileged and exempt from disclosure under applicable law. If the reader >> of this message is not the intended recipient, you are hereby notified >> that >> any printing, copying, dissemination, distribution, disclosure or >> forwarding of this communication is strictly prohibited. If you have >> received this communication in error, please contact the sender >> immediately >> and delete it from your system. Thank You. >> >
