Re: Apache ORC website revamp?

2022-07-26 Thread Dongjoon Hyun
+100 ! Thank you for your suggestion. Sounds great! I've always been feeling uncomfortable about the small width of the Apache ORC website and the absence of old version Java Doc APIs. We definitely need some modernization efforts on our website. Do you have something in your mind about a new webs

Re: [QUESTION] Decoding RLE v2 direct

2022-07-26 Thread Dongjoon Hyun
Thank you, Jorge. If you don't mind, may I ask about your usage of Apache ORC spec? I'm just wondering if you are trying to implement a new writer and reader from scratch by yourself? Dongjoon. On Mon, Jul 25, 2022 at 11:19 PM Jorge Cardoso Leitão wrote: > > Hi Dongjoon, > > Thank you for your

Re: [QUESTION] Decoding RLE v2 direct

2022-07-26 Thread Jorge Cardoso Leitão
Hi Dongjoon, Yes - I have been working with an implementation of Apache Arrow in Rust (programming language) [1], and have been adding interoperability with (storage) formats, such as JSON, Parquet and Avro. ORC is next. The exercise started by going through the spec and implementing functionalit

Re: [QUESTION] Can compression cross row boundaries?

2022-07-26 Thread Owen O'Malley
Compression in ORC not only crosses rows, but across the row groups (every 10k rows) that are the index points. Look at the ORC specification ( https://orc.apache.org/specification/ORCv1/) on Compression. Compression does not cross stripe boundaries, because that would violate the constraint that y

[jira] [Created] (ORC-1229) Move KeyProviderImpl to org.apache.orc.impl

2022-07-26 Thread William Hyun (Jira)
William Hyun created ORC-1229: - Summary: Move KeyProviderImpl to org.apache.orc.impl Key: ORC-1229 URL: https://issues.apache.org/jira/browse/ORC-1229 Project: ORC Issue Type: Task Comp

[jira] [Created] (ORC-1230) Move encryption utility functions to HadoopShimsCurrent

2022-07-26 Thread William Hyun (Jira)
William Hyun created ORC-1230: - Summary: Move encryption utility functions to HadoopShimsCurrent Key: ORC-1230 URL: https://issues.apache.org/jira/browse/ORC-1230 Project: ORC Issue Type: Task