Hi Niels, This is probably caused by a discrepancy between the tez version you run on your cluster (0.4.x) and the version that is in your pig release (0.14 is compiled with tez 0.5.2). Tez 0.4 creates a file called tez.session.local-resources.pb.file-name while tez 0.5 creates a file called tez.session.local-resources.pb. This name change was introduced as part of https://issues.apache.org/jira/browse/TEZ-1106 . I suspect upgrading your cluster to tez 0.5 would solve the issue.
Regards, Jan van Bemmelen > On 26 Feb 2015, at 17:24, Niels Basjes <ni...@basjes.nl> wrote: > > Hi, > > > I have installed tez on an existing hadoop cluster (HDP 2.1.2 with > kerberos installed). > > When I run a very basic pig script I get the error you see below. > > > It effectively fails with > > > java.io.FileNotFoundException: > tez.session.local-resources.pb.file-name (No such file or directory) > > > I have done some googleing on that term and I was unable to figure out > what went wrong. > > > Do you guys have any suggestions what I have done wrong? > > > Thanks. > > > Niels Basjes > > > > 2015-02-26 17:15:53,310 INFO [main] > org.apache.hadoop.conf.Configuration.deprecation: fs.default.name is > deprecated. Instead, use fs.defaultFS > 2015-02-26 17:15:53,364 INFO [main] > org.apache.tez.dag.app.DAGAppMaster: Created DAGAppMaster for > application appattempt_1423554480020_30727_000002 > 2015-02-26 17:15:53,534 WARN [main] > org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop > library for your platform... using builtin-java classes where > applicable > 2015-02-26 17:15:53,661 INFO [main] > org.apache.tez.dag.app.DAGAppMaster: Adding session token to > jobTokenSecretManager for application > 2015-02-26 17:15:53,667 INFO [main] > org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class > org.apache.tez.dag.app.rm.container.AMContainerEventType for class > org.apache.tez.dag.app.rm.container.AMContainerMap > 2015-02-26 17:15:53,668 INFO [main] > org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class > org.apache.tez.dag.app.rm.node.AMNodeEventType for class > org.apache.tez.dag.app.rm.node.AMNodeMap > 2015-02-26 17:15:53,669 INFO [main] > org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class > org.apache.tez.dag.app.dag.event.DAGAppMasterEventType for class > org.apache.tez.dag.app.DAGAppMaster$DAGAppMasterEventHandler > 2015-02-26 17:15:53,670 INFO [main] > org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class > org.apache.tez.dag.app.dag.event.DAGEventType for class > org.apache.tez.dag.app.DAGAppMaster$DagEventDispatcher > 2015-02-26 17:15:53,670 INFO [main] > org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class > org.apache.tez.dag.app.dag.event.VertexEventType for class > org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher > 2015-02-26 17:15:53,671 INFO [main] > org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class > org.apache.tez.dag.app.dag.event.TaskEventType for class > org.apache.tez.dag.app.DAGAppMaster$TaskEventDispatcher > 2015-02-26 17:15:53,672 INFO [main] > org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class > org.apache.tez.dag.app.dag.event.TaskAttemptEventType for class > org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher > 2015-02-26 17:15:53,673 INFO [main] > org.apache.tez.dag.app.DAGAppMaster: AM will unregister as this is the > last attempt, currentAttempt=2, maxAttempts=2 > 2015-02-26 17:15:53,674 INFO [main] > org.apache.tez.dag.app.rm.TaskSchedulerEventHandler: TaskScheduler > notified that it should unregister from RM > 2015-02-26 17:15:53,674 INFO [main] > org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class > org.apache.tez.dag.app.rm.AMSchedulerEventType for class > org.apache.tez.dag.app.rm.TaskSchedulerEventHandler > 2015-02-26 17:15:53,675 INFO [main] > org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class > org.apache.tez.dag.app.rm.NMCommunicatorEventType for class > org.apache.tez.dag.app.launcher.ContainerLauncherImpl > 2015-02-26 17:15:54,586 WARN [main] > org.apache.hadoop.hdfs.BlockReaderLocal: The short-circuit local reads > feature cannot be used because libhadoop cannot be loaded. > 2015-02-26 17:15:54,588 INFO [main] > org.apache.hadoop.service.AbstractService: Service > org.apache.tez.dag.app.DAGAppMaster failed in state INITED; cause: > java.io.FileNotFoundException: > tez.session.local-resources.pb.file-name (No such file or directory) > java.io.FileNotFoundException: > tez.session.local-resources.pb.file-name (No such file or directory) > at java.io.FileInputStream.open(Native Method) > at java.io.FileInputStream.<init>(FileInputStream.java:146) > at java.io.FileInputStream.<init>(FileInputStream.java:101) > at > org.apache.tez.dag.app.DAGAppMaster.serviceInit(DAGAppMaster.java:386) > at > org.apache.hadoop.service.AbstractService.init(AbstractService.java:163) > at org.apache.tez.dag.app.DAGAppMaster$5.run(DAGAppMaster.java:1954) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:415) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1557) > at > org.apache.tez.dag.app.DAGAppMaster.initAndStartAppMaster(DAGAppMaster.java:1951) > at org.apache.tez.dag.app.DAGAppMaster.main(DAGAppMaster.java:1790) > 2015-02-26 17:15:54,591 INFO [main] > org.apache.tez.dag.history.HistoryEventHandler: Stopping > HistoryEventHandler > 2015-02-26 17:15:54,592 FATAL [main] > org.apache.tez.dag.app.DAGAppMaster: Error starting DAGAppMaster > org.apache.hadoop.service.ServiceStateException: > java.io.FileNotFoundException: > tez.session.local-resources.pb.file-name (No such file or directory) > at > org.apache.hadoop.service.ServiceStateException.convert(ServiceStateException.java:59) > at > org.apache.hadoop.service.AbstractService.init(AbstractService.java:172) > at org.apache.tez.dag.app.DAGAppMaster$5.run(DAGAppMaster.java:1954) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:415) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1557) > at > org.apache.tez.dag.app.DAGAppMaster.initAndStartAppMaster(DAGAppMaster.java:1951) > at org.apache.tez.dag.app.DAGAppMaster.main(DAGAppMaster.java:1790) > Caused by: java.io.FileNotFoundException: > tez.session.local-resources.pb.file-name (No such file or directory) > at java.io.FileInputStream.open(Native Method) > at java.io.FileInputStream.<init>(FileInputStream.java:146) > at java.io.FileInputStream.<init>(FileInputStream.java:101) > at > org.apache.tez.dag.app.DAGAppMaster.serviceInit(DAGAppMaster.java:386) > at > org.apache.hadoop.service.AbstractService.init(AbstractService.java:163) > ... 6 more > > > -- > Best regards / Met vriendelijke groeten, > > Niels Basjes
