[jira] [Commented] (PARQUET-1241) [C++] Use LZ4 frame format

2019-11-02 Thread Martin Radev (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-1241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16965395#comment-16965395 ] Martin Radev commented on PARQUET-1241: --- Ok, it's good to be safe. I opened an issue: 

[jira] [Commented] (PARQUET-1241) [C++] Use LZ4 frame format

2019-11-02 Thread Antoine Pitrou (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-1241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16965366#comment-16965366 ] Antoine Pitrou commented on PARQUET-1241: - I think it would be worth asking on the LZ4 bug

[jira] [Comment Edited] (PARQUET-1241) [C++] Use LZ4 frame format

2019-11-02 Thread Martin Radev (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-1241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16965359#comment-16965359 ] Martin Radev edited comment on PARQUET-1241 at 11/2/19 1:24 PM: > Can

[jira] [Commented] (PARQUET-1241) [C++] Use LZ4 frame format

2019-11-02 Thread Martin Radev (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-1241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16965359#comment-16965359 ] Martin Radev commented on PARQUET-1241: --- > Can you spell out the reasoning in more detail? Yes.

[jira] [Commented] (PARQUET-1241) [C++] Use LZ4 frame format

2019-11-02 Thread Antoine Pitrou (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-1241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16965343#comment-16965343 ] Antoine Pitrou commented on PARQUET-1241: - > The four magic bytes cannot alias with the first

[jira] [Commented] (PARQUET-1241) [C++] Use LZ4 frame format

2019-11-02 Thread Martin Radev (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-1241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16965336#comment-16965336 ] Martin Radev commented on PARQUET-1241: --- I think this can be almost gracefully fixed without

Re: [VOTE] Add BYTE_STREAM_SPLIT encoding to Apache Parquet

2019-11-02 Thread Radev, Martin
Hello all, thanks for the vote Ryan and to Wes for the feedback. The concern with regards to adding more complex features in the Parquet spec is valid. However, the proposed encoding is very simple and I already have unpolished patches for both parquet-mr and arrow. In its design I