Re: Writing ORC files without HDFS

[Ravi Tatapudi]

Yes. "hive" project is the right candidate, for this work.

Thanks,
 Ravi




From:   István <lecc...@gmail.com>
To:     user@orc.apache.org
Date:   01/13/2016 07:28 PM
Subject:        Re: Writing ORC files without HDFS



Ravi,

one final question: which package should I import into my project?

I found this as the best candidate:

<dependency>
<groupId>org.apache.hive</groupId>
<artifactId>hive-exec</artifactId>
<version>1.2.0</version>
</dependency

Thanks,
Istvan

On Fri, Jan 8, 2016 at 8:38 AM, Ravi Tatapudi <ravi_tatap...@in.ibm.com> 
wrote:
Hello,

You can write ORC-files on local-filesystem, by getting the local 
"FileSystem" object, as "FileSystem.getLocal(conf)". Pl. find below the 
simple-example given below & see if it works for your requirement.

=================================================
public class orcw {

        private static Configuration conf = new Configuration();
        public static Writer writer ;

public static class OrcRow
{
        int col1 ;
        String col2 ;
        String col3 ;

        OrcRow(int a, String b, String c) {
        this.col1  = a ; 
        this.col2  = b ; 
        this.col3  = c ; 
        }
}

public static void main(String[] args) throws IOException,

    InterruptedException, ClassNotFoundException {

            String path = "/tmp/orcfile1";

            try {

            conf = new Configuration();
        FileSystem fs = FileSystem.getLocal(conf);

            ObjectInspector ObjInspector = 
ObjectInspectorFactory.getReflectionObjectInspector(OrcRow.class, 
ObjectInspectorFactory.ObjectInspectorOptions.JAVA);
            writer = OrcFile.createWriter(new Path(path), 
OrcFile.writerOptions(conf).inspector(ObjInspector).stripeSize(100000).bufferSize(10000).compress(CompressionKind.ZLIB).version(OrcFile.Version.V_0_12));


        writer.addRow(new OrcRow(1,"hello","orcFile")) ;
        writer.addRow(new OrcRow(2,"hello2","orcFile2")) ;

            writer.close();
            } 
            catch (Exception e)
            {
                    e.printStackTrace();
            }
    }
}
=================================================

Thanks,
 Ravi



From:        István <lecc...@gmail.com>
To:        user@orc.apache.org
Date:        01/08/2016 01:53 AM
Subject:        Writing ORC files without HDFS




Hi all,

I am working on a project that requires me to write ORC files locally on a 
non-HDFS location. I was wondering if there is any project doing something 
similar, but I guess there is none, after spending some time on Google.

I think what needs to get done is to re-implement the ORC Writer, sort of 
similar to the following but leaving out Hadoop:

https://github.com/apache/hive/blob/master/orc/src/java/org/apache/orc/impl/WriterImpl.java


Am I on the right track implementing this? 

Let me know if you have any suggestions or links in the subject.

Thank you very much,
Istvan

-- 
the sun shines for all






-- 
the sun shines for all