Thanks for digging into the issue. Would you mind filing a Hadoop jira for this? Throwing an IndexOutOfBounds for a user that has no groups seems quite wrong.
— Hitesh On Jul 21, 2015, at 8:07 PM, [email protected] wrote: > I found the log on hiveserver2 . > > 8.784 PM WARN org.apache.hadoop.security.UserGroupInformation > No groups available for user tangjijun > I found the code in ViewFileSystem: > > public FileStatus getFileStatus(Path f) throws IOException { > checkPathIsSlash(f); > return new FileStatus(0, true, 0, 0, creationTime, creationTime, > PERMISSION_555, ugi.getUserName(), ugi.getGroupNames()[0], > > new Path(theInternalDir.fullPath).makeQualified( > myUri, ROOT_PATH)); > } > Then I create user tangjijun on hiveserver2 node. The exception gone. > From: [email protected] > Date: 2015-07-21 14:29 > To: user > Subject: Re: Re: tez throw error when enable hdfs federation > I apply the patch.But throw other error. > > INFO : Tez session hasn't been created yet. Opening session > ERROR : Failed to execute tez graph. > java.lang.ArrayIndexOutOfBoundsException: 0 > at > org.apache.hadoop.fs.viewfs.ViewFileSystem$InternalDirOfViewFs.getFileStatus(ViewFileSystem.java:771) > > at > org.apache.hadoop.fs.viewfs.ViewFileSystem.getFileStatus(ViewFileSystem.java:359) > > at > org.apache.tez.client.TezClientUtils.checkAncestorPermissionsForAllUsers(TezClientUtils.java:955) > > at > org.apache.tez.client.TezClientUtils.setupTezJarsLocalResources(TezClientUtils.java:184) > > at org.apache.tez.client.TezClient.getTezJarResources(TezClient.java:787) > at org.apache.tez.client.TezClient.start(TezClient.java:337) > at > org.apache.hadoop.hive.ql.exec.tez.TezSessionState.open(TezSessionState.java:191) > > at org.apache.hadoop.hive.ql.exec.tez.TezTask.updateSession(TezTask.java:234) > at org.apache.hadoop.hive.ql.exec.tez.TezTask.execute(TezTask.java:136) > at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:160) > at > org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:88) > at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1640) > at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1399) > at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1183) > at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1049) > at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1044) > at > org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:144) > > at > org.apache.hive.service.cli.operation.SQLOperation.access$100(SQLOperation.java:69) > > at > org.apache.hive.service.cli.operation.SQLOperation$1$1.run(SQLOperation.java:196) > > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:415) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1671) > > at > org.apache.hive.service.cli.operation.SQLOperation$1.run(SQLOperation.java:208) > > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > at java.util.concurrent.FutureTask.run(FutureTask.java:262) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > > at java.lang.Thread.run(Thread.java:745) > Error: Error while processing statement: FAILED: Execution Error, return code > 1 from org.apache.hadoop.hive.ql.exec.tez.TezTask (state=08S01,code=1) > > [email protected] > > From: [email protected] > Date: 2015-07-21 08:33 > To: user > Subject: Re: Re: tez throw error when enable hdfs federation > Thanks > > [email protected] > > From: Rajesh Balamohan > Date: 2015-07-20 18:59 > To: user > Subject: Re: tez throw error when enable hdfs federation > https://issues.apache.org/jira/browse/TEZ-2600 > > ~Rajesh.B > > On Mon, Jul 20, 2015 at 2:45 PM, jjtang <[email protected]> wrote: > INFO : Tez session hasn't been created yet. Opening session > ERROR : Failed to execute tez graph. > java.io.IOException: Incomplete HDFS URI, no host: hdfs:///user > at > org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:142) > > at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2596) > at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:91) > at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2630) > at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2612) > at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:370) > at org.apache.hadoop.fs.Path.getFileSystem(Path.java:296) > at > org.apache.hadoop.hive.ql.exec.tez.DagUtils.getDefaultDestDir(DagUtils.java:767) > > at > org.apache.hadoop.hive.ql.exec.tez.DagUtils.getHiveJarDirectory(DagUtils.java:874) > > at > org.apache.hadoop.hive.ql.exec.tez.TezSessionState.createJarLocalResource(TezSessionState.java:337) > > at > org.apache.hadoop.hive.ql.exec.tez.TezSessionState.open(TezSessionState.java:158) > > at org.apache.hadoop.hive.ql.exec.tez.TezTask.updateSession(TezTask.java:234) > at org.apache.hadoop.hive.ql.exec.tez.TezTask.execute(TezTask.java:136) > at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:160) > at > org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:88) > at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1640) > at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1399) > at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1183) > at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1049) > at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1044) > at > org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:144) > > at > org.apache.hive.service.cli.operation.SQLOperation.access$100(SQLOperation.java:69) > > at > org.apache.hive.service.cli.operation.SQLOperation$1$1.run(SQLOperation.java:196) > > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:415) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1671) > > at > org.apache.hive.service.cli.operation.SQLOperation$1.run(SQLOperation.java:208) > > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > at java.util.concurrent.FutureTask.run(FutureTask.java:262) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > > at java.lang.Thread.run(Thread.java:745) > Error: Error while processing statement: FAILED: Execution Error, return code > 1 from org.apache.hadoop.hive.ql.exec.tez.TezTask (state=08S01,code=1) > > Then I set hive.user.install.directory=viewfs:///user/; throw another error: > INFO : Tez session hasn't been created yet. Opening session > ERROR : Failed to execute tez graph. > java.io.IOException: Incomplete HDFS URI, no host: > hdfs:///user/tez/tez-0.7.0.tar.gz > at > org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:142) > > at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2596) > at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:91) > at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2630) > at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2612) > at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:370) > at org.apache.hadoop.fs.Path.getFileSystem(Path.java:296) > at > org.apache.tez.client.TezClientUtils.getLRFileStatus(TezClientUtils.java:130) > at > org.apache.tez.client.TezClientUtils.setupTezJarsLocalResources(TezClientUtils.java:182) > > at org.apache.tez.client.TezClient.getTezJarResources(TezClient.java:787) > at org.apache.tez.client.TezClient.start(TezClient.java:337) > at > org.apache.hadoop.hive.ql.exec.tez.TezSessionState.open(TezSessionState.java:191) > > at org.apache.hadoop.hive.ql.exec.tez.TezTask.updateSession(TezTask.java:234) > at org.apache.hadoop.hive.ql.exec.tez.TezTask.execute(TezTask.java:136) > at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:160) > at > org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:88) > at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1640) > at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1399) > at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1183) > at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1049) > at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1044) > at > org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:144) > > at > org.apache.hive.service.cli.operation.SQLOperation.access$100(SQLOperation.java:69) > > at > org.apache.hive.service.cli.operation.SQLOperation$1$1.run(SQLOperation.java:196) > > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:415) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1671) > > at > org.apache.hive.service.cli.operation.SQLOperation$1.run(SQLOperation.java:208) > > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > at java.util.concurrent.FutureTask.run(FutureTask.java:262) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > > at java.lang.Thread.run(Thread.java:745) > Error: Error while processing statement: FAILED: Execution Error, return code > 1 from org.apache.hadoop.hive.ql.exec.tez.TezTask (state=08S01,code=1) > > My hadoop version is hadoop2.6.0-cdh5.4.3 ,hive version is hive > 1.1.0-cdh5.4.3,tez version is 0.7. > > > [email protected]
