[ 
https://issues.apache.org/jira/browse/HIVE-14411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15405763#comment-15405763
 ] 

Rudd Chen commented on HIVE-14411:
----------------------------------

when we are selecting a non native table, this method would return a strange 
path like "-mr-10004default.hbase_t_hive{}",  and it would throw an exception 
like below when submitting the MapReduce Job.

ava.io.FileNotFoundException: File does not exist: 
/user/admin/-mr-10004default.hbase_t_hive{}
        at 
org.apache.hadoop.hdfs.server.namenode.FSDirStatAndListingOp.getContentSummaryInt(FSDirStatAndListingOp.java:509)
        at 
org.apache.hadoop.hdfs.server.namenode.FSDirStatAndListingOp.getContentSummary(FSDirStatAndListingOp.java:140)
        at 
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getContentSummary(FSNamesystem.java:3862)
        at 
org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.getContentSummary(NameNodeRpcServer.java:1230)
        at 
org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.getContentSummary(ClientNamenodeProtocolServerSideTranslatorPB.java:887)
        at 
org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
        at 
org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:616)
        at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:973)
        at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2089)
        at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2085)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:422)
        at 
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1711)
        at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2083)

        at sun.reflect.GeneratedConstructorAccessor83.newInstance(Unknown 
Source)
        at 
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
        at 
org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:106)
        at 
org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:73)
        at 
org.apache.hadoop.hdfs.DFSClient.getContentSummary(DFSClient.java:2731)
        at 
org.apache.hadoop.hdfs.DistributedFileSystem$18.doCall(DistributedFileSystem.java:816)
        at 
org.apache.hadoop.hdfs.DistributedFileSystem$18.doCall(DistributedFileSystem.java:812)
        at 
org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
        at 
org.apache.hadoop.hdfs.DistributedFileSystem.getContentSummary(DistributedFileSystem.java:812)
        at 
org.apache.hadoop.hive.ql.exec.mr.ExecDriver.getInputFileNum(ExecDriver.java:555)
        at 
org.apache.hadoop.hive.ql.exec.mr.ExecDriver.checkFileNum(ExecDriver.java:535)
        at 
org.apache.hadoop.hive.ql.exec.mr.ExecDriver.execute(ExecDriver.java:430)
        at 
org.apache.hadoop.hive.ql.exec.mr.MapRedTask.execute(MapRedTask.java:137)

> selecting Hive on Hbase table may cause FileNotFoundException
> -------------------------------------------------------------
>
>                 Key: HIVE-14411
>                 URL: https://issues.apache.org/jira/browse/HIVE-14411
>             Project: Hive
>          Issue Type: Bug
>          Components: Physical Optimizer
>    Affects Versions: 1.3.0
>            Reporter: Rudd Chen
>
> 1. create a Hbase table hbase_table
> 2. create a external Hive table test_table mapping to the hbase table 
> example: 
> create 'hbase_t' 
> ,{NAME=>'cf',COMPRESSION=>'snappy'},{NUMREGIONS=>15,SPLITALGO=>'HexStringSplit'}
> create external table hbase_t_hive(key1 string,cf_train string,cf_flight 
> string,cf_wbsw string,cf_wbxw string,cf_bgrz string,cf_bgtf string) 
> stored by 'org.apache.hadoop.hive.hbase.HBaseStorageHandler' 
> with 
> serdeproperties("hbase.columns.mapping"=":key,cf:train,cf:flight,cf:wbsw,cf:wbxw,cf:bgrz,cf:bgtf")
>  tblproperties("hbase.table.name"="hbase_t");
> create table test3 as select * from hbase_t_hive where 1=2;
> ====
> if hive.optimize.null.scan=true, it will return an FileNotFoundException



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Reply via email to