Hi Ryan, In case you have not checked this might be a good starting point for you. https://orc.apache.org/docs/core-java.html#simple-example <https://orc.apache.org/docs/core-java.html#simple-example>
When I follow the code you shared, I don’t quite follow why you are creating and passing the LocalFileSystem object as you have in your code. In the link you will see a very simple example on how to instantiate a writer. The default configuration will give you a LocalFileSystem object without you having to do any special processing. Most likely the error you are seeing is coming from the point that the configuration object being set is null. Try the simple example I am sure that should clear things up for you. > On Apr 16, 2021, at 12:32 PM, Ryan Schachte <[email protected]> > wrote: > > Hi team. Desperate to understand what my issue is here, hoping someone with > better knowledge of the ORC writer and how the local hadoop fs is working. > > I'm using the LocalFileSystem for my standalone app (compacts ORC files). > Logging into the container, I see all my data written to the location I'm > reading from, but hadoop is failing internally here. > > I'm using Owen's code from orc-tools to run schema inference on a given orc > file internally in my code and this is what causes this exception. Happy to > expand on this were applicable. > > Below is my error: > Exception in thread "main" java.lang.NullPointerException: Cannot invoke > "org.apache.hadoop.conf.Configuration.getLong(String, long)" because the > return value of "org.apache.hadoop.fs.FileSystem.getConf()" is null > at org.apache.hadoop.fs.FileSystem.getDefaultBlockSize(FileSystem.java:2602) > at org.apache.hadoop.fs.FileSystem.getDefaultBlockSize(FileSystem.java:2613) > at > org.apache.hadoop.fs.RawLocalFileSystem.deprecatedGetFileStatus(RawLocalFileSystem.java:666) > at > org.apache.hadoop.fs.RawLocalFileSystem.getFileLinkStatusInternal(RawLocalFileSystem.java:989) > at > org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:658) > at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1729) > at > org.apache.hadoop.fs.RawLocalFileSystem.create(RawLocalFileSystem.java:314) > at > org.apache.hadoop.fs.RawLocalFileSystem.create(RawLocalFileSystem.java:353) > at > org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSOutputSummer.<init>(ChecksumFileSystem.java:416) > at > org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:479) > at > org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:458) > at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:1164) > at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:1144) > at org.apache.orc.impl.PhysicalFsWriter.<init>(PhysicalFsWriter.java:115) > at org.apache.orc.impl.WriterImpl.<init>(WriterImpl.java:167) > at org.apache.orc.OrcFile.createWriter(OrcFile.java:1004) > > Cheers! > Ryan
