Re: HDFS small files to Sequence file using Hive

*Is there a way to increase the file/block size beyond 1MB? * *Thank you!* On Mon, Sep 26, 2016 at 7:50 PM, Arun Patel wrote: > Thanks Dudu and Gopal. > > I tried HAR files and it works. > > I want to use Sequence file because I want to expose data using a table >

Re: HDFS small files to Sequence file using Hive

> Is there a way to create an external table on a directory, extract 'key' as > file name and 'value' as file content and write to a sequence file table? Do you care that it is a sequence file? The HDFS HAR format was invented for this particular problem, check if the "hadoop archive" command

RE: HDFS small files to Sequence file using Hive

...@gmail.com] Sent: Friday, September 23, 2016 7:04 PM To: user@hive.apache.org Subject: HDFS small files to Sequence file using Hive I'm trying to resolve small files issue using Hive. Is there a way to create an external table on a directory, extract 'key' as file name and 'value' as file content

HDFS small files to Sequence file using Hive

I'm trying to resolve small files issue using Hive. Is there a way to create an external table on a directory, extract 'key' as file name and 'value' as file content and write to a sequence file table? Or any other better option in Hive? Thank you Arun