Re: [Proposal] New decimal encoding

2018-04-11 Thread Gang Wu
Thanks Gopal for the links and they are very helpful! As Owen has suggested to create a patch to ORC specs for the proposal, this PR: https://github.com/apache/orc/pull/245 is created for discussion. If we are all on the same page and finalize the proposal, we can start coding afterwards. Any co

Re: [Proposal] New decimal encoding

2018-04-10 Thread Gopal Vijayaraghavan
Hi, I agree with your analysis about Decimals. Something similar has already gone into patch-available previously, but held back https://issues.apache.org/jira/browse/ORC-209 This is somewhat stuck behind the Vector type system evolving support for this https://issues.apache.org/jira/browse/

[Proposal] New decimal encoding

2018-04-10 Thread Wu Gang
Hi, This is Gang Wu and I have proposed this in ORC-161 but got no response therefore I put it here. Recently I have done some benchmarks between ORC and our proprietary file format. The result indicates that ORC does not have a good performance on