Bulk load will leave some files under /user/hbase/bulkload/hfile, e.g. _tmp, even after successfuly load all hfiles into HBase. So, at the end of my code, I removed the entire directory. Thus, it is impossible for me to find whether the file exist or not.
I think the splitted hfiles are on HDFS, but I am not very sure. Thanks Weihua 2011/6/14 Jean-Daniel Cryans <[email protected]>: > My first guess would be a configuration issue where some files would > be in the local filesystem while now it's trying to read from HDFS. > Hard to tell with just the stack trace. > > J-D > > On Mon, Jun 13, 2011 at 11:06 PM, Weihua JIANG <[email protected]> wrote: >> Hi all, >> >> I am trying to use bulk load to import large data into HBase. the M/R >> works perfectly. But, when importing the hfiles into HBase, I get lots >> of following error message. What is the problem here? >> >> [java] 11/06/14 11:50:41 ERROR hdfs.DFSClient: Exception closing file >> /user/hbase/bulkload/hfile/family/_tmp/CMCC_Detail_ReversePhoneMonth__DateCat_NONE,8.top >> : org.apache.hadoop.ipc.RemoteException: >> org.apache.hadoop.hdfs.server.namenode.LeaseExpiredException: No lease >> on >> /user/hbase/bulkload/hfile/family/_tmp/CMCC_Detail_ReversePhoneMonth__DateCat_NONE,8.top >> File does not exist. Holder DFSClient_1454850411 does not have any >> open files. >> [java] at >> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkLease(FSNamesystem.java:1542) >> [java] at >> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkLease(FSNamesystem.java:1533) >> [java] at >> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.completeFileInternal(FSNamesystem.java:1588) >> [java] at >> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.completeFile(FSNamesystem.java:1576) >> [java] at >> org.apache.hadoop.hdfs.server.namenode.NameNode.complete(NameNode.java:712) >> [java] at sun.reflect.GeneratedMethodAccessor22.invoke(Unknown >> Source) >> [java] at >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) >> [java] at java.lang.reflect.Method.invoke(Method.java:597) >> [java] at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:557) >> [java] at >> org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1415) >> [java] at >> org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1411) >> [java] at java.security.AccessController.doPrivileged(Native Method) >> [java] at javax.security.auth.Subject.doAs(Subject.java:396) >> [java] at >> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1115) >> [java] at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1409) >> [java] >> [java] org.apache.hadoop.ipc.RemoteException: >> org.apache.hadoop.hdfs.server.namenode.LeaseExpiredException: No lease >> on >> /user/hbase/bulkload/hfile/family/_tmp/CMCC_Detail_ReversePhoneMonth__DateCat_NONE,8.top >> File does not exist. Holder DFSClient_1454850411 does not have any >> open files. >> [java] at >> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkLease(FSNamesystem.java:1542) >> [java] at >> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkLease(FSNamesystem.java:1533) >> [java] at >> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.completeFileInternal(FSNamesystem.java:1588) >> [java] at >> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.completeFile(FSNamesystem.java:1576) >> [java] at >> org.apache.hadoop.hdfs.server.namenode.NameNode.complete(NameNode.java:712) >> [java] at sun.reflect.GeneratedMethodAccessor22.invoke(Unknown >> Source) >> [java] at >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) >> [java] at java.lang.reflect.Method.invoke(Method.java:597) >> [java] at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:557) >> [java] at >> org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1415) >> [java] at >> org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1411) >> [java] at java.security.AccessController.doPrivileged(Native Method) >> [java] at javax.security.auth.Subject.doAs(Subject.java:396) >> [java] at >> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1115) >> [java] at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1409) >> [java] >> [java] at org.apache.hadoop.ipc.Client.call(Client.java:1104) >> [java] at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:226) >> [java] at $Proxy6.complete(Unknown Source) >> [java] at sun.reflect.GeneratedMethodAccessor6.invoke(Unknown Source) >> [java] at >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >> [java] at java.lang.reflect.Method.invoke(Method.java:616) >> [java] at >> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82) >> [java] at >> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59) >> [java] at $Proxy6.complete(Unknown Source) >> [java] at >> org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.closeInternal(DFSClient.java:3631) >> [java] at >> org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.close(DFSClient.java:3545) >> [java] at >> org.apache.hadoop.hdfs.DFSClient$LeaseChecker.close(DFSClient.java:1129) >> [java] at org.apache.hadoop.hdfs.DFSClient.close(DFSClient.java:244) >> [java] at >> org.apache.hadoop.hdfs.DistributedFileSystem.close(DistributedFileSystem.java:349) >> [java] at >> org.apache.hadoop.fs.FileSystem$Cache.closeAll(FileSystem.java:1616) >> [java] at >> org.apache.hadoop.fs.FileSystem$Cache$ClientFinalizer.run(FileSystem.java:1586) >> >
