[GitHub] orc issue #245: ORC-161: Proposal for new decimal encodings and statistics.

2018-04-18 Thread xndai
Github user xndai commented on the issue: https://github.com/apache/orc/pull/245 +1 for Gang's proposal. ---

[GitHub] orc issue #245: ORC-161: Proposal for new decimal encodings and statistics.

2018-04-18 Thread wgtmac
Github user wgtmac commented on the issue: https://github.com/apache/orc/pull/245 As ORC v2 specs may takes a long time to finalize, develop and test with a non-trivial structural change to be production-ready. We have a very large amount of data of decimal types in production awaitin

[GitHub] orc issue #245: ORC-161: Proposal for new decimal encodings and statistics.

2018-04-12 Thread wgtmac
Github user wgtmac commented on the issue: https://github.com/apache/orc/pull/245 Will provide them after comprehensive benchmark. ---

[GitHub] orc issue #245: ORC-161: Proposal for new decimal encodings and statistics.

2018-04-12 Thread prasanthj
Github user prasanthj commented on the issue: https://github.com/apache/orc/pull/245 "we found RLEv1 + zstd may be the best combination than others in terms of both compression ration and encoding/decoding speed." do you have experimental numbers for this? ---

[GitHub] orc issue #245: ORC-161: Proposal for new decimal encodings and statistics.

2018-04-12 Thread wgtmac
Github user wgtmac commented on the issue: https://github.com/apache/orc/pull/245 After second thought, I added back DECIMAL_V1 to support RLE v1 in decimal encoding. The reason is that in our testing, we found RLEv1 + zstd may be the best combination than others in terms of both comp

[GitHub] orc issue #245: ORC-161: Proposal for new decimal encodings and statistics.

2018-04-11 Thread wgtmac
Github user wgtmac commented on the issue: https://github.com/apache/orc/pull/245 @t3rmin4t0r @omalley @majetideepak @xndai Any suggestion or concern? If we can finalize this, I can start working on it. ---