Hi Joshi, You can use Flume + AsyncHbaseSink / HBasesink to move data from local file sytem to HBase.
Thanks, Surendra M -- Surendra Manchikanti On Wed, Apr 17, 2013 at 10:01 AM, Omkar Joshi <omkar.jo...@lntinfotech.com>wrote: > The background thread is here : > > > http://mail-archives.apache.org/mod_mbox/hbase-user/201304.mbox/%3ce689a42b73c5a545ad77332a4fc75d8c1efbe84...@vshinmsmbx01.vshodc.lntinfotech.com%3E > > Following are the commands that I'm using to load files onto HBase : > > HADOOP_CLASSPATH=`${HBASE_HOME}/bin/hbase classpath` > ${HADOOP_HOME}/bin/hadoop jar ${HBASE_HOME}/hbase-0.94.6.1.jar importtsv > '-Dimporttsv.separator=;' > -Dimporttsv.columns=HBASE_ROW_KEY,PRODUCT_INFO:NAME,PRODUCT_INFO:CATEGORY,PRODUCT_INFO:GROUP,PRODUCT_INFO:COMPANY,PRODUCT_INFO:COST,PRODUCT_INFO:COLOR,PRODUCT_INFO:BLANK_COLUMN > -Dimporttsv.bulk.output=hdfs://cldx-1139-1033:9000/hbase/storefileoutput_6 > PRODUCTS hdfs://cldx-1139-1033:9000/hbase/copiedFromLocal/product_6.txt > > HADOOP_CLASSPATH=`${HBASE_HOME}/bin/hbase classpath` > ${HADOOP_HOME}/bin/hadoop jar ${HBASE_HOME}/hbase-0.94.6.1.jar > completebulkload hdfs://cldx-1139-1033:9000/hbase/storefileoutput_6 PRODUCTS > > As seen, the text files to be loaded in HBase first need to be loaded on > HDFS. Given our infrastructure constraints/limitations, I'm getting space > issues. The data in the text files is around 20GB + replication is > consuming a lot of DFS. > > Is there a way wherein a text file can be loaded directly from the local > file system onto HBase? > > Regards, > Omkar Joshi > > ________________________________ > The contents of this e-mail and any attachment(s) may contain confidential > or privileged information for the intended recipient(s). Unintended > recipients are prohibited from taking action on the basis of information in > this e-mail and using or disseminating the information, and must notify the > sender and delete it from their system. L&T Infotech will not accept > responsibility or liability for the accuracy or completeness of, or the > presence of any virus or disabling code in this e-mail" >
