[jira] [Resolved] (PARQUET-1325) High-level flexible and fine-grained column level access control through encryption with pluggable key access

2018-08-20 Thread Xinli Shang (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-1325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinli Shang resolved PARQUET-1325. -- Resolution: Duplicate This Jira is replaced by Parquet-1396. > High-level flexible and

[jira] [Resolved] (PARQUET-1276) [C++] Reduce the amount of memory used for writing null decimal values

2018-08-20 Thread Antoine Pitrou (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-1276?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Antoine Pitrou resolved PARQUET-1276. - Resolution: Fixed > [C++] Reduce the amount of memory used for writing null decimal

Re: Status of column index in parquet-mr

2018-08-20 Thread Tim Armstrong
I had a similar concern to Uwe - if there are a large number of columns with variable size there does seem to be a real risk of having many tiny pages. I wonder if we could do something in-between where we allow different page sizes for different columns, but require that the row ranges for pages

[jira] [Commented] (PARQUET-1276) [C++] Reduce the amount of memory used for writing null decimal values

2018-08-20 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-1276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16586109#comment-16586109 ] ASF GitHub Bot commented on PARQUET-1276: - wesm closed pull request #493: PARQUET-1276: [C++]

Re: [DISCUSS] Solutions for improving the Arrow-Parquet C++ development morass

2018-08-20 Thread Wes McKinney
A couple more points to make re: Uwe's comments: > An important point that we should keep in (and why I was a bit concerned in > the previous times this discussion was raised) is that we have to be careful > to not pull everything that touches Arrow into the Arrow repository. An important

[jira] [Created] (PARQUET-1396) Cryptodata Interface for no-API Activation of Parquet Encryption

2018-08-20 Thread Xinli Shang (JIRA)
Xinli Shang created PARQUET-1396: Summary: Cryptodata Interface for no-API Activation of Parquet Encryption Key: PARQUET-1396 URL: https://issues.apache.org/jira/browse/PARQUET-1396 Project: Parquet

[jira] [Commented] (PARQUET-1396) Cryptodata Interface for no-API Activation of Parquet Encryption

2018-08-20 Thread Xinli Shang (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-1396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16586264#comment-16586264 ] Xinli Shang commented on PARQUET-1396: -- This is the design of Parquet-1396.  > Cryptodata

[jira] [Updated] (PARQUET-1397) Sample of usage Parquet-1396 and Parquet-1178 for column level encryption with pluggable key access

2018-08-20 Thread Xinli Shang (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-1397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinli Shang updated PARQUET-1397: - Description: This Jira provides a sample to use Parquet-1396 and Parquet-1178 column level

[jira] [Created] (PARQUET-1397) Sample of usage Parquet-1396 and Parquet-1178 for column level encryption with pluggable key access

2018-08-20 Thread Xinli Shang (JIRA)
Xinli Shang created PARQUET-1397: Summary: Sample of usage Parquet-1396 and Parquet-1178 for column level encryption with pluggable key access Key: PARQUET-1397 URL:

Re: [VOTE] Moving Apache Parquet C++ development process to a monorepo structure with Apache Arrow C++

2018-08-20 Thread Ryan Blue
+1 I think this sounds like a reasonable solution to the problem, and one that is supported by the people that will do the work. I'd appreciate some clarification on this: > The Parquet community can create scripts to "cut" Parquet C++ releases I think this should be a requirement of

Re: [VOTE] Moving Apache Parquet C++ development process to a monorepo structure with Apache Arrow C++

2018-08-20 Thread Wes McKinney
hi Ryan, On Mon, Aug 20, 2018 at 2:54 PM, Ryan Blue wrote: > +1 > > I think this sounds like a reasonable solution to the problem, and one that > is supported by the people that will do the work. > > I'd appreciate some clarification on this: > >> The Parquet community can create scripts to

[jira] [Commented] (PARQUET-1395) [C++] Tests fail due to not finding libboost_system.so

2018-08-20 Thread Antoine Pitrou (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-1395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16586027#comment-16586027 ] Antoine Pitrou commented on PARQUET-1395: - The default (\?) cmake options here are: {code} $

[jira] [Commented] (PARQUET-1394) [C++] Linking errors with boost_regex

2018-08-20 Thread Wes McKinney (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-1394?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16586053#comment-16586053 ] Wes McKinney commented on PARQUET-1394: --- This may be related to the Anaconda compiler migration

[jira] [Commented] (PARQUET-1395) [C++] Tests fail due to not finding libboost_system.so

2018-08-20 Thread Antoine Pitrou (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-1395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16585757#comment-16585757 ] Antoine Pitrou commented on PARQUET-1395: - (also seems like we're using a different linking

[jira] [Commented] (PARQUET-1395) [C++] Tests fail due to not finding libboost_system.so

2018-08-20 Thread Antoine Pitrou (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-1395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16585754#comment-16585754 ] Antoine Pitrou commented on PARQUET-1395: - Actually the problem here might be the transitive

[jira] [Commented] (PARQUET-1395) [C++] Tests fail due to not finding libboost_system.so

2018-08-20 Thread Antoine Pitrou (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-1395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16585732#comment-16585732 ] Antoine Pitrou commented on PARQUET-1395: - (note this is with gcc 7.3.0 on Ubuntu 18.04) >

[jira] [Commented] (PARQUET-1394) [C++] Linking errors with boost_regex

2018-08-20 Thread Antoine Pitrou (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-1394?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16585733#comment-16585733 ] Antoine Pitrou commented on PARQUET-1394: - Ok, so, weirdly, this seems to be solved using

[jira] [Commented] (PARQUET-1394) [C++] Linking errors with boost_regex

2018-08-20 Thread Antoine Pitrou (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-1394?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16585681#comment-16585681 ] Antoine Pitrou commented on PARQUET-1394: - Here are the only versions of boost-regex present on

[jira] [Created] (PARQUET-1394) [C++] Linking errors with boost_regex

2018-08-20 Thread Antoine Pitrou (JIRA)
Antoine Pitrou created PARQUET-1394: --- Summary: [C++] Linking errors with boost_regex Key: PARQUET-1394 URL: https://issues.apache.org/jira/browse/PARQUET-1394 Project: Parquet Issue Type:

[jira] [Created] (PARQUET-1395) [C++] Tests fail due to not finding libboost_system.so

2018-08-20 Thread Antoine Pitrou (JIRA)
Antoine Pitrou created PARQUET-1395: --- Summary: [C++] Tests fail due to not finding libboost_system.so Key: PARQUET-1395 URL: https://issues.apache.org/jira/browse/PARQUET-1395 Project: Parquet

[jira] [Commented] (PARQUET-1395) [C++] Tests fail due to not finding libboost_system.so

2018-08-20 Thread Antoine Pitrou (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-1395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16585834#comment-16585834 ] Antoine Pitrou commented on PARQUET-1395: - Ok, I don't understand anything anymore. These are

[jira] [Commented] (PARQUET-1395) [C++] Tests fail due to not finding libboost_system.so

2018-08-20 Thread Antoine Pitrou (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-1395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16585837#comment-16585837 ] Antoine Pitrou commented on PARQUET-1395: - Actually, not the same files. {code} $ sha1sum

[jira] [Commented] (PARQUET-1395) [C++] Tests fail due to not finding libboost_system.so

2018-08-20 Thread Antoine Pitrou (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-1395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16585849#comment-16585849 ] Antoine Pitrou commented on PARQUET-1395: - Ok, so things seem to work if I pass

[jira] [Commented] (PARQUET-1395) [C++] Tests fail due to not finding libboost_system.so

2018-08-20 Thread Uwe L. Korn (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-1395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16585927#comment-16585927 ] Uwe L. Korn commented on PARQUET-1395: -- We should have a look at what {{conda}} is doing

[jira] [Commented] (PARQUET-1395) [C++] Tests fail due to not finding libboost_system.so

2018-08-20 Thread Uwe L. Korn (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-1395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16585931#comment-16585931 ] Uwe L. Korn commented on PARQUET-1395: -- [~pitrou] can you post the output of ? {code} % objdump

[jira] [Commented] (PARQUET-1395) [C++] Tests fail due to not finding libboost_system.so

2018-08-20 Thread Antoine Pitrou (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-1395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16585839#comment-16585839 ] Antoine Pitrou commented on PARQUET-1395: - Also, looks like Arrow unittest are linked

[jira] [Updated] (PARQUET-1276) [C++] Reduce the amount of memory used for writing null decimal values

2018-08-20 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-1276?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated PARQUET-1276: Labels: pull-request-available (was: ) > [C++] Reduce the amount of memory used for

[jira] [Commented] (PARQUET-1276) [C++] Reduce the amount of memory used for writing null decimal values

2018-08-20 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-1276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16585920#comment-16585920 ] ASF GitHub Bot commented on PARQUET-1276: - pitrou opened a new pull request #493: PARQUET-1276:

[jira] [Commented] (PARQUET-1395) [C++] Tests fail due to not finding libboost_system.so

2018-08-20 Thread Antoine Pitrou (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-1395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16585928#comment-16585928 ] Antoine Pitrou commented on PARQUET-1395: - I upgraded to Ubuntu 18.04 this week-end. So this

[jira] [Comment Edited] (PARQUET-1395) [C++] Tests fail due to not finding libboost_system.so

2018-08-20 Thread Uwe L. Korn (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-1395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16585947#comment-16585947 ] Uwe L. Korn edited comment on PARQUET-1395 at 8/20/18 1:58 PM: --- Ok, that

[jira] [Commented] (PARQUET-1395) [C++] Tests fail due to not finding libboost_system.so

2018-08-20 Thread Uwe L. Korn (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-1395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16585947#comment-16585947 ] Uwe L. Korn commented on PARQUET-1395: -- Ok, that is definitely the root of the problem. In the

[jira] [Commented] (PARQUET-1395) [C++] Tests fail due to not finding libboost_system.so

2018-08-20 Thread Antoine Pitrou (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-1395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16586000#comment-16586000 ] Antoine Pitrou commented on PARQUET-1395: - I have cmake 3.12.0 from Anaconda. cmake 3.10 from

[jira] [Commented] (PARQUET-1395) [C++] Tests fail due to not finding libboost_system.so

2018-08-20 Thread Antoine Pitrou (JIRA)
[ https://issues.apache.org/jira/browse/PARQUET-1395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16585933#comment-16585933 ] Antoine Pitrou commented on PARQUET-1395: - {code} $ objdump -x ./build-debug/debug/memory-test