[GitHub] [parquet-mr] martin-g edited a comment on pull request #924: PARQUET-2076: Improve Travis CI build Performance

2021-11-15 Thread GitBox
martin-g edited a comment on pull request #924: URL: https://github.com/apache/parquet-mr/pull/924#issuecomment-900057808 With `-Dorg.slf4j.simpleLogger.logFile=mvn-verify.log` all the output is redirected from stdout to a file. TravisCI stops the job if there is no output for 10 minutes

[GitHub] [parquet-mr] YunLemon commented on pull request #924: PARQUET-2076: Improve Travis CI build Performance

2021-11-14 Thread GitBox
YunLemon commented on pull request #924: URL: https://github.com/apache/parquet-mr/pull/924#issuecomment-968429043 yes, [https://app.travis-ci.com/github/apache/parquet-mr/jobs/541671084](https://app.travis-ci.com/github/apache/parquet-mr/jobs/541671084), I think this build can explain wha

[GitHub] [parquet-mr] mixermt commented on pull request #754: PARQUET-1784: Column-wise configuration

2021-11-14 Thread GitBox
mixermt commented on pull request #754: URL: https://github.com/apache/parquet-mr/pull/754#issuecomment-968261545 Hi, It is not clear how the property should work with lists Example: listField: OPTIONAL F:1

[GitHub] [parquet-mr] shangxinli commented on pull request #828: PARQUET-1925: Introduce Velocity Template Engine to Parquet Generator

2021-11-12 Thread GitBox
shangxinli commented on pull request #828: URL: https://github.com/apache/parquet-mr/pull/828#issuecomment-967286140 @belugabehr As @gszadovszky asked, do you plan to work on this further? -- This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [parquet-mr] shangxinli commented on pull request #900: PARQUET-2042: Add support for unwrapping common Protobuf wrappers and…

2021-11-12 Thread GitBox
shangxinli commented on pull request #900: URL: https://github.com/apache/parquet-mr/pull/900#issuecomment-967285427 @mwong38, can you put more information in the Jira on why/what is changed? This is pretty big change and it would help people to review your code. -- This is an automated

[GitHub] [parquet-mr] shangxinli commented on pull request #907: PARQUET-2049: Deprecate Exceptions Class

2021-11-12 Thread GitBox
shangxinli commented on pull request #907: URL: https://github.com/apache/parquet-mr/pull/907#issuecomment-967283233 @belugabehr, do you want to fix the build errors? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use th

[GitHub] [parquet-mr] shangxinli commented on pull request #924: PARQUET-2076: Improve Travis CI build Performance

2021-11-12 Thread GitBox
shangxinli commented on pull request #924: URL: https://github.com/apache/parquet-mr/pull/924#issuecomment-967272796 @YunLemon, Thanks for working on this! Are we still on track to reduce the minutes to wait? -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [parquet-mr] shangxinli closed pull request #932: Parquet-2091: Fix release build error introduced by PARQUET-2043

2021-11-12 Thread GitBox
shangxinli closed pull request #932: URL: https://github.com/apache/parquet-mr/pull/932 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubs

[GitHub] [parquet-mr] shangxinli commented on pull request #935: PARQUET-2040: Uniform encryption

2021-11-12 Thread GitBox
shangxinli commented on pull request #935: URL: https://github.com/apache/parquet-mr/pull/935#issuecomment-967258392 LGTM -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. T

[GitHub] [parquet-mr] gszadovszky merged pull request #937: PARQUET-2102: Fix typo in ColumnIndexBase toString

2021-11-11 Thread GitBox
gszadovszky merged pull request #937: URL: https://github.com/apache/parquet-mr/pull/937 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsub

[GitHub] [parquet-mr] ryanrupp edited a comment on pull request #937: PARQUET-2102: Fix typo in ColumnIndexBase toString

2021-11-10 Thread GitBox
ryanrupp edited a comment on pull request #937: URL: https://github.com/apache/parquet-mr/pull/937#issuecomment-966007150 Random but had noticed this in https://github.com/trinodb/trino/issues/9890 (not related to the issue other than the toString is just used to print the stats in the exc

[GitHub] [parquet-mr] ryanrupp commented on pull request #937: PARQUET-2102: Fix typo in ColumnIndexBase toString

2021-11-10 Thread GitBox
ryanrupp commented on pull request #937: URL: https://github.com/apache/parquet-mr/pull/937#issuecomment-966007150 Random but had noticed this in https://github.com/trinodb/trino/issues/9890 -- This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [parquet-mr] ryanrupp opened a new pull request #937: PARQUET-2102: Fix typo in ColumnIndexBase toString

2021-11-10 Thread GitBox
ryanrupp opened a new pull request #937: URL: https://github.com/apache/parquet-mr/pull/937 Make sure you have checked _all_ steps below. ### Jira - [ ] My PR addresses the following [Parquet Jira](https://issues.apache.org/jira/browse/PARQUET/) issues and references them in

[GitHub] [parquet-mr] gszadovszky merged pull request #936: PARQUET-2101: Fix wrong descriptions about the default block size

2021-11-02 Thread GitBox
gszadovszky merged pull request #936: URL: https://github.com/apache/parquet-mr/pull/936 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsub

[GitHub] [parquet-mr] gszadovszky merged pull request #936: PARQUET-2101: Fix wrong descriptions about the default block size

2021-11-02 Thread GitBox
gszadovszky merged pull request #936: URL: https://github.com/apache/parquet-mr/pull/936 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsub

[GitHub] [parquet-mr] gszadovszky merged pull request #936: PARQUET-2101: Fix wrong descriptions about the default block size

2021-11-02 Thread GitBox
gszadovszky merged pull request #936: URL: https://github.com/apache/parquet-mr/pull/936 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsub

[GitHub] [parquet-mr] sekikn opened a new pull request #936: PARQUET-2101: Fix wrong descriptions about the default block size

2021-10-31 Thread GitBox
sekikn opened a new pull request #936: URL: https://github.com/apache/parquet-mr/pull/936 Make sure you have checked _all_ steps below. ### Jira - [x] My PR addresses the following [Parquet Jira](https://issues.apache.org/jira/browse/PARQUET/) issues and references them in th

[GitHub] [parquet-mr] gszadovszky commented on a change in pull request #935: PARQUET-2040: Uniform encryption

2021-10-20 Thread GitBox
gszadovszky commented on a change in pull request #935: URL: https://github.com/apache/parquet-mr/pull/935#discussion_r732870171 ## File path: parquet-hadoop/src/main/java/org/apache/parquet/crypto/AesCipher.java ## @@ -37,6 +37,12 @@ protected static final int GCM_TAG_LENGT

[GitHub] [parquet-mr] shangxinli merged pull request #928: PARQUET-2081: Encryption translation tool - Parquet-hadoop

2021-10-04 Thread GitBox
shangxinli merged pull request #928: URL: https://github.com/apache/parquet-mr/pull/928 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubs

[GitHub] [parquet-mr] shangxinli commented on pull request #928: PARQUET-2081: Encryption translation tool - Parquet-hadoop

2021-10-04 Thread GitBox
shangxinli commented on pull request #928: URL: https://github.com/apache/parquet-mr/pull/928#issuecomment-933844414 Thank @ggershinsky and @gszadovszky for spending time on it! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [parquet-mr] ggershinsky commented on pull request #928: PARQUET-2081: Encryption translation tool - Parquet-hadoop

2021-10-04 Thread GitBox
ggershinsky commented on pull request #928: URL: https://github.com/apache/parquet-mr/pull/928#issuecomment-933545241 @shangxinli looks good, I think we're down to the 2 remaining points (related to a crypto-config flag) https://github.com/apache/parquet-mr/pull/928/files#r719567036

[GitHub] [parquet-mr] ggershinsky commented on pull request #928: PARQUET-2081: Encryption translation tool - Parquet-hadoop

2021-10-04 Thread GitBox
ggershinsky commented on pull request #928: URL: https://github.com/apache/parquet-mr/pull/928#issuecomment-933506341 > So I just simply revert my change that partially renamed AAD to pageHeaderAAD. I looked at the util.java file and I see there are a lot of other places all using AAD inst

[GitHub] [parquet-mr] shangxinli edited a comment on pull request #928: PARQUET-2081: Encryption translation tool - Parquet-hadoop

2021-10-04 Thread GitBox
shangxinli edited a comment on pull request #928: URL: https://github.com/apache/parquet-mr/pull/928#issuecomment-933499949 @gszadovszky Just addressed your feedback. For the conflict, I tried to merge from upstream but it always resulted in a lot more conflict. So I just simply revert my

[GitHub] [parquet-mr] shangxinli commented on pull request #928: PARQUET-2081: Encryption translation tool - Parquet-hadoop

2021-10-04 Thread GitBox
shangxinli commented on pull request #928: URL: https://github.com/apache/parquet-mr/pull/928#issuecomment-933499949 Just addressed your feedback. For the conflict, I tried to merge from upstream but it always resulted in a lot more conflict. So I just simply revert my change that partiall

[GitHub] [parquet-mr] gszadovszky commented on a change in pull request #928: PARQUET-2081: Encryption translation tool - Parquet-hadoop

2021-10-04 Thread GitBox
gszadovszky commented on a change in pull request #928: URL: https://github.com/apache/parquet-mr/pull/928#discussion_r721165920 ## File path: parquet-column/src/main/java/org/apache/parquet/internal/column/columnindex/OffsetIndexBuilder.java ## @@ -151,22 +151,36 @@ public Of

[GitHub] [parquet-mr] rdblue commented on pull request #933: PARQUET-2094: Handle negative values in page headers

2021-10-03 Thread GitBox
rdblue commented on pull request #933: URL: https://github.com/apache/parquet-mr/pull/933#issuecomment-933027683 @gszadovszky, thanks for getting this done. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abov

[GitHub] [parquet-mr] ggershinsky commented on a change in pull request #928: PARQUET-2081: Encryption translation tool - Parquet-hadoop

2021-10-02 Thread GitBox
ggershinsky commented on a change in pull request #928: URL: https://github.com/apache/parquet-mr/pull/928#discussion_r720770145 ## File path: parquet-hadoop/src/main/java/org/apache/parquet/hadoop/util/ColumnEncryptor.java ## @@ -0,0 +1,317 @@ +/* + * Licensed to the Apache S

[GitHub] [parquet-mr] ggershinsky commented on a change in pull request #928: PARQUET-2081: Encryption translation tool - Parquet-hadoop

2021-10-02 Thread GitBox
ggershinsky commented on a change in pull request #928: URL: https://github.com/apache/parquet-mr/pull/928#discussion_r720769917 ## File path: parquet-hadoop/src/main/java/org/apache/parquet/hadoop/util/ColumnEncryptor.java ## @@ -0,0 +1,317 @@ +/* + * Licensed to the Apache S

[GitHub] [parquet-mr] shangxinli commented on a change in pull request #928: PARQUET-2081: Encryption translation tool - Parquet-hadoop

2021-10-01 Thread GitBox
shangxinli commented on a change in pull request #928: URL: https://github.com/apache/parquet-mr/pull/928#discussion_r720460937 ## File path: parquet-hadoop/src/main/java/org/apache/parquet/hadoop/util/ColumnEncryptor.java ## @@ -0,0 +1,317 @@ +/* + * Licensed to the Apache So

[GitHub] [parquet-mr] shangxinli commented on a change in pull request #928: PARQUET-2081: Encryption translation tool - Parquet-hadoop

2021-10-01 Thread GitBox
shangxinli commented on a change in pull request #928: URL: https://github.com/apache/parquet-mr/pull/928#discussion_r720460487 ## File path: parquet-hadoop/src/main/java/org/apache/parquet/hadoop/util/ColumnEncryptor.java ## @@ -0,0 +1,317 @@ +/* + * Licensed to the Apache So

[GitHub] [parquet-mr] ggershinsky commented on pull request #928: PARQUET-2081: Encryption translation tool - Parquet-hadoop

2021-10-01 Thread GitBox
ggershinsky commented on pull request #928: URL: https://github.com/apache/parquet-mr/pull/928#issuecomment-931970339 went over the code, adding a few more comments. Overall looks good; once the remaining points are handled, I'll be ok with the PR. -- This is an automated message from th

[GitHub] [parquet-mr] ggershinsky commented on a change in pull request #928: PARQUET-2081: Encryption translation tool - Parquet-hadoop

2021-10-01 Thread GitBox
ggershinsky commented on a change in pull request #928: URL: https://github.com/apache/parquet-mr/pull/928#discussion_r719992595 ## File path: parquet-hadoop/src/main/java/org/apache/parquet/hadoop/util/ColumnEncryptor.java ## @@ -0,0 +1,317 @@ +/* + * Licensed to the Apache S

[GitHub] [parquet-mr] ggershinsky commented on a change in pull request #928: PARQUET-2081: Encryption translation tool - Parquet-hadoop

2021-09-30 Thread GitBox
ggershinsky commented on a change in pull request #928: URL: https://github.com/apache/parquet-mr/pull/928#discussion_r719567391 ## File path: parquet-hadoop/src/main/java/org/apache/parquet/hadoop/ParquetFileWriter.java ## @@ -1125,7 +1154,7 @@ private static void serializeCo

[GitHub] [parquet-mr] ggershinsky commented on pull request #928: PARQUET-2081: Encryption translation tool - Parquet-hadoop

2021-09-30 Thread GitBox
ggershinsky commented on pull request #928: URL: https://github.com/apache/parquet-mr/pull/928#issuecomment-931470099 yep, a couple of quick comments. will comb again tomorrow. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[GitHub] [parquet-mr] shangxinli commented on pull request #928: PARQUET-2081: Encryption translation tool - Parquet-hadoop

2021-09-30 Thread GitBox
shangxinli commented on pull request #928: URL: https://github.com/apache/parquet-mr/pull/928#issuecomment-931428972 @ggershinsky Do you want to have another look? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[GitHub] [parquet-mr] shangxinli edited a comment on pull request #928: PARQUET-2081: Encryption translation tool - Parquet-hadoop

2021-09-30 Thread GitBox
shangxinli edited a comment on pull request #928: URL: https://github.com/apache/parquet-mr/pull/928#issuecomment-931386393 @gszadovszky The first two commits had the OffsetIndex issue. The 3rd commit added test verifyOffsetIndexes() which verified not only the offset but also the page(hea

[GitHub] [parquet-mr] shangxinli commented on pull request #928: PARQUET-2081: Encryption translation tool - Parquet-hadoop

2021-09-30 Thread GitBox
shangxinli commented on pull request #928: URL: https://github.com/apache/parquet-mr/pull/928#issuecomment-931426352 > Sorry, @shangxinli. My bad. Ignore my comment about the testing. No problem Gabor! Thank you for all the feedbacks! -- This is an automated message from the Apach

[GitHub] [parquet-mr] gszadovszky commented on pull request #928: PARQUET-2081: Encryption translation tool - Parquet-hadoop

2021-09-30 Thread GitBox
gszadovszky commented on pull request #928: URL: https://github.com/apache/parquet-mr/pull/928#issuecomment-931396500 Sorry, @shangxinli. My bad. Ignore my comment about the testing. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to G

[GitHub] [parquet-mr] shangxinli commented on a change in pull request #928: PARQUET-2081: Encryption translation tool - Parquet-hadoop

2021-09-30 Thread GitBox
shangxinli commented on a change in pull request #928: URL: https://github.com/apache/parquet-mr/pull/928#discussion_r719478762 ## File path: parquet-hadoop/src/main/java/org/apache/parquet/hadoop/util/ColumnEncryptor.java ## @@ -280,8 +288,12 @@ private void encryptPages(Tran

[GitHub] [parquet-mr] shangxinli commented on pull request #928: PARQUET-2081: Encryption translation tool - Parquet-hadoop

2021-09-30 Thread GitBox
shangxinli commented on pull request #928: URL: https://github.com/apache/parquet-mr/pull/928#issuecomment-931386393 @gszadovszky The first two commits had the OffsetIndex issue. The 3rd commit added test verifyOffsetIndexes() which verified not only the offset but also the page(header and

[GitHub] [parquet-mr] gszadovszky commented on pull request #923: [PARQUET-1968] FilterApi support In predicate

2021-09-30 Thread GitBox
gszadovszky commented on pull request #923: URL: https://github.com/apache/parquet-mr/pull/923#issuecomment-931370971 Thank you for your contribution, @huaxingao! Great work! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub an

[GitHub] [parquet-mr] huaxingao commented on pull request #923: [PARQUET-1968] FilterApi support In predicate

2021-09-30 Thread GitBox
huaxingao commented on pull request #923: URL: https://github.com/apache/parquet-mr/pull/923#issuecomment-931369233 @gszadovszky @shangxinli @viirya @dbtsai Thank you so much for all your help!! -- This is an automated message from the Apache Git Service. To respond to the message, pleas

[GitHub] [parquet-mr] gszadovszky merged pull request #933: PARQUET-2094: Handle negative values in page headers

2021-09-30 Thread GitBox
gszadovszky merged pull request #933: URL: https://github.com/apache/parquet-mr/pull/933 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsub

[GitHub] [parquet-mr] gszadovszky commented on a change in pull request #928: PARQUET-2081: Encryption translation tool - Parquet-hadoop

2021-09-30 Thread GitBox
gszadovszky commented on a change in pull request #928: URL: https://github.com/apache/parquet-mr/pull/928#discussion_r719139712 ## File path: parquet-hadoop/src/main/java/org/apache/parquet/hadoop/util/ColumnEncryptor.java ## @@ -180,21 +192,13 @@ private void processBlocks(T

[GitHub] [parquet-mr] gszadovszky merged pull request #923: [PARQUET-1968] FilterApi support In predicate

2021-09-30 Thread GitBox
gszadovszky merged pull request #923: URL: https://github.com/apache/parquet-mr/pull/923 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsub

[GitHub] [parquet-mr] shangxinli commented on a change in pull request #928: PARQUET-2081: Encryption translation tool - Parquet-hadoop

2021-09-29 Thread GitBox
shangxinli commented on a change in pull request #928: URL: https://github.com/apache/parquet-mr/pull/928#discussion_r718777078 ## File path: parquet-hadoop/src/main/java/org/apache/parquet/hadoop/util/ColumnEncryptor.java ## @@ -0,0 +1,307 @@ +/* + * Licensed to the Apache So

[GitHub] [parquet-mr] ggershinsky commented on a change in pull request #928: PARQUET-2081: Encryption translation tool - Parquet-hadoop

2021-09-29 Thread GitBox
ggershinsky commented on a change in pull request #928: URL: https://github.com/apache/parquet-mr/pull/928#discussion_r718184216 ## File path: parquet-hadoop/src/main/java/org/apache/parquet/hadoop/util/ColumnEncryptor.java ## @@ -0,0 +1,307 @@ +/* + * Licensed to the Apache S

[GitHub] [parquet-mr] shangxinli commented on a change in pull request #928: PARQUET-2081: Encryption translation tool - Parquet-hadoop

2021-09-29 Thread GitBox
shangxinli commented on a change in pull request #928: URL: https://github.com/apache/parquet-mr/pull/928#discussion_r718777078 ## File path: parquet-hadoop/src/main/java/org/apache/parquet/hadoop/util/ColumnEncryptor.java ## @@ -0,0 +1,307 @@ +/* + * Licensed to the Apache So

[GitHub] [parquet-mr] ggershinsky commented on a change in pull request #928: PARQUET-2081: Encryption translation tool - Parquet-hadoop

2021-09-28 Thread GitBox
ggershinsky commented on a change in pull request #928: URL: https://github.com/apache/parquet-mr/pull/928#discussion_r718184216 ## File path: parquet-hadoop/src/main/java/org/apache/parquet/hadoop/util/ColumnEncryptor.java ## @@ -0,0 +1,307 @@ +/* + * Licensed to the Apache S

[GitHub] [parquet-mr] shangxinli commented on pull request #928: PARQUET-2081: Encryption translation tool - Parquet-hadoop

2021-09-28 Thread GitBox
shangxinli commented on pull request #928: URL: https://github.com/apache/parquet-mr/pull/928#issuecomment-929348539 @gszadovszky Do you want to have a look at it again for offset index that Gidon mentioned? -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [parquet-mr] huaxingao commented on a change in pull request #923: [PARQUET-1968] FilterApi support In predicate

2021-09-28 Thread GitBox
huaxingao commented on a change in pull request #923: URL: https://github.com/apache/parquet-mr/pull/923#discussion_r717696646 ## File path: parquet-column/src/main/java/org/apache/parquet/column/MinMax.java ## @@ -18,25 +18,17 @@ */ package org.apache.parquet.column; -imp

[GitHub] [parquet-mr] gszadovszky commented on a change in pull request #928: PARQUET-2081: Encryption translation tool - Parquet-hadoop

2021-09-28 Thread GitBox
gszadovszky commented on a change in pull request #928: URL: https://github.com/apache/parquet-mr/pull/928#discussion_r717723359 ## File path: parquet-hadoop/src/main/java/org/apache/parquet/hadoop/util/ColumnEncryptor.java ## @@ -0,0 +1,339 @@ +/* + * Licensed to the Apache S

[GitHub] [parquet-mr] shangxinli edited a comment on pull request #928: PARQUET-2081: Encryption translation tool - Parquet-hadoop

2021-09-28 Thread GitBox
shangxinli edited a comment on pull request #928: URL: https://github.com/apache/parquet-mr/pull/928#issuecomment-929348539 @gszadovszky Do you want to have a look at it again for offsetIndex that Gidon mentioned? -- This is an automated message from the Apache Git Service. To respond t

[GitHub] [parquet-mr] gszadovszky commented on pull request #934: PARQUET-2096: Upgrade Thrift to 0.15.0

2021-09-28 Thread GitBox
gszadovszky commented on pull request #934: URL: https://github.com/apache/parquet-mr/pull/934#issuecomment-928994865 @vinooganesh, everything is fine. Only that I usually wait for additional 24 hours before merging so others have a chance to complain. -- This is an automated message fro

[GitHub] [parquet-mr] gszadovszky commented on a change in pull request #933: PARQUET-2094: Handle negative values in page headers

2021-09-28 Thread GitBox
gszadovszky commented on a change in pull request #933: URL: https://github.com/apache/parquet-mr/pull/933#discussion_r717364884 ## File path: parquet-format-structures/src/main/java/org/apache/parquet/format/MetadataValidator.java ## @@ -0,0 +1,54 @@ +/* + * Licensed to the A

[GitHub] [parquet-mr] shangxinli commented on a change in pull request #928: PARQUET-2081: Encryption translation tool - Parquet-hadoop

2021-09-28 Thread GitBox
shangxinli commented on a change in pull request #928: URL: https://github.com/apache/parquet-mr/pull/928#discussion_r717708267 ## File path: parquet-hadoop/src/main/java/org/apache/parquet/hadoop/util/ColumnEncryptor.java ## @@ -0,0 +1,307 @@ +/* + * Licensed to the Apache So

[GitHub] [parquet-mr] gszadovszky merged pull request #934: PARQUET-2096: Upgrade Thrift to 0.15.0

2021-09-28 Thread GitBox
gszadovszky merged pull request #934: URL: https://github.com/apache/parquet-mr/pull/934 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsub

[GitHub] [parquet-mr] ggershinsky commented on a change in pull request #928: PARQUET-2081: Encryption translation tool - Parquet-hadoop

2021-09-28 Thread GitBox
ggershinsky commented on a change in pull request #928: URL: https://github.com/apache/parquet-mr/pull/928#discussion_r717301743 ## File path: parquet-hadoop/src/main/java/org/apache/parquet/format/converter/ParquetMetadataConverter.java ## @@ -503,7 +503,7 @@ private void add

[GitHub] [parquet-mr] vinooganesh commented on pull request #934: PARQUET-2096: Upgrade Thrift to 0.15.0

2021-09-28 Thread GitBox
vinooganesh commented on pull request #934: URL: https://github.com/apache/parquet-mr/pull/934#issuecomment-928643532 @gszadovszky - thanks for the approval! Please let me know if there is anything else I need to do before we merge. -- This is an automated message from the Apache Git Ser

[GitHub] [parquet-mr] dongjoon-hyun commented on pull request #934: PARQUET-2096: Upgrade Thrift to 0.15.0

2021-09-28 Thread GitBox
dongjoon-hyun commented on pull request #934: URL: https://github.com/apache/parquet-mr/pull/934#issuecomment-928791691 Thank you, @vinooganesh and all. I also hit this issue. :) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [parquet-mr] gszadovszky commented on a change in pull request #923: [PARQUET-1968] FilterApi support In predicate

2021-09-28 Thread GitBox
gszadovszky commented on a change in pull request #923: URL: https://github.com/apache/parquet-mr/pull/923#discussion_r717354327 ## File path: parquet-column/src/main/java/org/apache/parquet/column/MinMax.java ## @@ -18,25 +18,17 @@ */ package org.apache.parquet.column; -i

[GitHub] [parquet-mr] rdblue commented on a change in pull request #933: PARQUET-2094: Handle negative values in page headers

2021-09-28 Thread GitBox
rdblue commented on a change in pull request #933: URL: https://github.com/apache/parquet-mr/pull/933#discussion_r716970875 ## File path: parquet-format-structures/src/main/java/org/apache/parquet/format/MetadataValidator.java ## @@ -0,0 +1,54 @@ +/* + * Licensed to the Apache

[GitHub] [parquet-mr] rdblue commented on a change in pull request #933: PARQUET-2094: Handle negative values in page headers

2021-09-28 Thread GitBox
rdblue commented on a change in pull request #933: URL: https://github.com/apache/parquet-mr/pull/933#discussion_r717768759 ## File path: parquet-format-structures/src/test/java/org/apache/parquet/format/TestUtil.java ## @@ -23,13 +23,16 @@ import static junit.framework.Asser

[GitHub] [parquet-mr] rdblue commented on a change in pull request #933: PARQUET-2094: Handle negative values in page headers

2021-09-28 Thread GitBox
rdblue commented on a change in pull request #933: URL: https://github.com/apache/parquet-mr/pull/933#discussion_r717768214 ## File path: parquet-format-structures/src/main/java/org/apache/parquet/format/InvalidParquetMetadataException.java ## @@ -0,0 +1,30 @@ +/* + * Licensed

[GitHub] [parquet-mr] gszadovszky commented on a change in pull request #928: PARQUET-2081: Encryption translation tool - Parquet-hadoop

2021-09-28 Thread GitBox
gszadovszky commented on a change in pull request #928: URL: https://github.com/apache/parquet-mr/pull/928#discussion_r717723359 ## File path: parquet-hadoop/src/main/java/org/apache/parquet/hadoop/util/ColumnEncryptor.java ## @@ -0,0 +1,339 @@ +/* + * Licensed to the Apache S

[GitHub] [parquet-mr] shangxinli edited a comment on pull request #928: PARQUET-2081: Encryption translation tool - Parquet-hadoop

2021-09-28 Thread GitBox
shangxinli edited a comment on pull request #928: URL: https://github.com/apache/parquet-mr/pull/928#issuecomment-929348539 @gszadovszky Do you want to have a look at it again for offsetIndex that Gidon mentioned? -- This is an automated message from the Apache Git Service. To respond t

[GitHub] [parquet-mr] shangxinli commented on pull request #928: PARQUET-2081: Encryption translation tool - Parquet-hadoop

2021-09-28 Thread GitBox
shangxinli commented on pull request #928: URL: https://github.com/apache/parquet-mr/pull/928#issuecomment-929348539 @gszadovszky Do you want to have a look at it again for offset index that Gidon mentioned? -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [parquet-mr] shangxinli commented on a change in pull request #928: PARQUET-2081: Encryption translation tool - Parquet-hadoop

2021-09-28 Thread GitBox
shangxinli commented on a change in pull request #928: URL: https://github.com/apache/parquet-mr/pull/928#discussion_r717708267 ## File path: parquet-hadoop/src/main/java/org/apache/parquet/hadoop/util/ColumnEncryptor.java ## @@ -0,0 +1,307 @@ +/* + * Licensed to the Apache So

[GitHub] [parquet-mr] huaxingao commented on a change in pull request #923: [PARQUET-1968] FilterApi support In predicate

2021-09-28 Thread GitBox
huaxingao commented on a change in pull request #923: URL: https://github.com/apache/parquet-mr/pull/923#discussion_r717696646 ## File path: parquet-column/src/main/java/org/apache/parquet/column/MinMax.java ## @@ -18,25 +18,17 @@ */ package org.apache.parquet.column; -imp

[GitHub] [parquet-mr] gszadovszky merged pull request #934: PARQUET-2096: Upgrade Thrift to 0.15.0

2021-09-28 Thread GitBox
gszadovszky merged pull request #934: URL: https://github.com/apache/parquet-mr/pull/934 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsub

[GitHub] [parquet-mr] gszadovszky commented on pull request #934: PARQUET-2096: Upgrade Thrift to 0.15.0

2021-09-28 Thread GitBox
gszadovszky commented on pull request #934: URL: https://github.com/apache/parquet-mr/pull/934#issuecomment-928994865 @vinooganesh, everything is fine. Only that I usually wait for additional 24 hours before merging so others have a chance to complain. -- This is an automated message fro

[GitHub] [parquet-mr] gszadovszky commented on a change in pull request #933: PARQUET-2094: Handle negative values in page headers

2021-09-28 Thread GitBox
gszadovszky commented on a change in pull request #933: URL: https://github.com/apache/parquet-mr/pull/933#discussion_r717364884 ## File path: parquet-format-structures/src/main/java/org/apache/parquet/format/MetadataValidator.java ## @@ -0,0 +1,54 @@ +/* + * Licensed to the A

[GitHub] [parquet-mr] gszadovszky commented on a change in pull request #923: [PARQUET-1968] FilterApi support In predicate

2021-09-28 Thread GitBox
gszadovszky commented on a change in pull request #923: URL: https://github.com/apache/parquet-mr/pull/923#discussion_r717354327 ## File path: parquet-column/src/main/java/org/apache/parquet/column/MinMax.java ## @@ -18,25 +18,17 @@ */ package org.apache.parquet.column; -i

[GitHub] [parquet-mr] ggershinsky commented on a change in pull request #928: PARQUET-2081: Encryption translation tool - Parquet-hadoop

2021-09-28 Thread GitBox
ggershinsky commented on a change in pull request #928: URL: https://github.com/apache/parquet-mr/pull/928#discussion_r717319745 ## File path: parquet-hadoop/src/main/java/org/apache/parquet/hadoop/util/ColumnEncryptor.java ## @@ -0,0 +1,307 @@ +/* + * Licensed to the Apache S

[GitHub] [parquet-mr] ggershinsky commented on a change in pull request #928: PARQUET-2081: Encryption translation tool - Parquet-hadoop

2021-09-28 Thread GitBox
ggershinsky commented on a change in pull request #928: URL: https://github.com/apache/parquet-mr/pull/928#discussion_r717312882 ## File path: parquet-hadoop/src/main/java/org/apache/parquet/hadoop/util/ColumnEncryptor.java ## @@ -0,0 +1,339 @@ +/* + * Licensed to the Apache S

[GitHub] [parquet-mr] ggershinsky commented on a change in pull request #928: PARQUET-2081: Encryption translation tool - Parquet-hadoop

2021-09-28 Thread GitBox
ggershinsky commented on a change in pull request #928: URL: https://github.com/apache/parquet-mr/pull/928#discussion_r717301743 ## File path: parquet-hadoop/src/main/java/org/apache/parquet/format/converter/ParquetMetadataConverter.java ## @@ -503,7 +503,7 @@ private void add

[GitHub] [parquet-mr] dongjoon-hyun commented on pull request #934: PARQUET-2096: Upgrade Thrift to 0.15.0

2021-09-27 Thread GitBox
dongjoon-hyun commented on pull request #934: URL: https://github.com/apache/parquet-mr/pull/934#issuecomment-928791691 Thank you, @vinooganesh and all. I also hit this issue. :) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [parquet-mr] vinooganesh commented on pull request #934: PARQUET-2096: Upgrade Thrift to 0.15.0

2021-09-27 Thread GitBox
vinooganesh commented on pull request #934: URL: https://github.com/apache/parquet-mr/pull/934#issuecomment-928643532 @gszadovszky - thanks for the approval! Please let me know if there is anything else I need to do before we merge. -- This is an automated message from the Apache Git Ser

[GitHub] [parquet-mr] rdblue commented on a change in pull request #933: PARQUET-2094: Handle negative values in page headers

2021-09-27 Thread GitBox
rdblue commented on a change in pull request #933: URL: https://github.com/apache/parquet-mr/pull/933#discussion_r716970875 ## File path: parquet-format-structures/src/main/java/org/apache/parquet/format/MetadataValidator.java ## @@ -0,0 +1,54 @@ +/* + * Licensed to the Apache

[GitHub] [parquet-mr] shangxinli commented on a change in pull request #928: PARQUET-2081: Encryption translation tool - Parquet-hadoop

2021-09-27 Thread GitBox
shangxinli commented on a change in pull request #928: URL: https://github.com/apache/parquet-mr/pull/928#discussion_r716877124 ## File path: pom.xml ## @@ -522,6 +522,9 @@ change to fix a integer overflow issue. TODO: remove this after Parquet

[GitHub] [parquet-mr] gszadovszky commented on a change in pull request #928: PARQUET-2081: Encryption translation tool - Parquet-hadoop

2021-09-27 Thread GitBox
gszadovszky commented on a change in pull request #928: URL: https://github.com/apache/parquet-mr/pull/928#discussion_r716526515 ## File path: pom.xml ## @@ -522,6 +522,9 @@ change to fix a integer overflow issue. TODO: remove this after Parque

[GitHub] [parquet-mr] gszadovszky commented on a change in pull request #923: [PARQUET-1968] FilterApi support In predicate

2021-09-27 Thread GitBox
gszadovszky commented on a change in pull request #923: URL: https://github.com/apache/parquet-mr/pull/923#discussion_r716474012 ## File path: parquet-column/src/main/java/org/apache/parquet/column/MinMax.java ## @@ -0,0 +1,89 @@ +/* + * Licensed to the Apache Software Foundati

[GitHub] [parquet-mr] gszadovszky commented on a change in pull request #933: PARQUET-2094: Handle negative values in page headers

2021-09-27 Thread GitBox
gszadovszky commented on a change in pull request #933: URL: https://github.com/apache/parquet-mr/pull/933#discussion_r716453533 ## File path: parquet-format-structures/src/test/java/org/apache/parquet/format/TestUtil.java ## @@ -77,6 +81,21 @@ public void testReadFileMetadata

[GitHub] [parquet-mr] gszadovszky commented on a change in pull request #933: PARQUET-2094: Handle negative values in page headers

2021-09-27 Thread GitBox
gszadovszky commented on a change in pull request #933: URL: https://github.com/apache/parquet-mr/pull/933#discussion_r716452332 ## File path: parquet-format-structures/src/main/java/org/apache/parquet/format/MetadataValidator.java ## @@ -0,0 +1,56 @@ +/* + * Licensed to the A

[GitHub] [parquet-mr] gszadovszky commented on a change in pull request #933: PARQUET-2094: Handle negative values in page headers

2021-09-27 Thread GitBox
gszadovszky commented on a change in pull request #933: URL: https://github.com/apache/parquet-mr/pull/933#discussion_r716450578 ## File path: parquet-format-structures/src/main/java/org/apache/parquet/format/MetadataValidator.java ## @@ -0,0 +1,56 @@ +/* + * Licensed to the A

[GitHub] [parquet-mr] vinooganesh commented on pull request #934: PARQUET-2096: Upgrade Thrift to 0.15.0

2021-09-26 Thread GitBox
vinooganesh commented on pull request #934: URL: https://github.com/apache/parquet-mr/pull/934#issuecomment-927473988 cc @gszadovszky @sunchao from https://github.com/apache/parquet-mr/pull/927. @sunchao - apologies for the quick upgrade after yours, I saw was building parquet from source

[GitHub] [parquet-mr] vinooganesh commented on pull request #934: PARQUET-2096: Upgrade Thrift to 0.15.0

2021-09-26 Thread GitBox
vinooganesh commented on pull request #934: URL: https://github.com/apache/parquet-mr/pull/934#issuecomment-927469278 @rdblue This is a baby commit - but could you approve running tests on it? -- This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [parquet-mr] vinooganesh opened a new pull request #934: PARQUET-2096: Upgrade Thrift to 0.15.0

2021-09-26 Thread GitBox
vinooganesh opened a new pull request #934: URL: https://github.com/apache/parquet-mr/pull/934 Make sure you have checked _all_ steps below. ### Jira - [X] My PR addresses the following [Parquet Jira](https://issues.apache.org/jira/browse/PARQUET/) issues and references them

[GitHub] [parquet-mr] huaxingao commented on a change in pull request #923: [PARQUET-1968] FilterApi support In predicate

2021-09-26 Thread GitBox
huaxingao commented on a change in pull request #923: URL: https://github.com/apache/parquet-mr/pull/923#discussion_r716278313 ## File path: parquet-hadoop/src/test/java/org/apache/parquet/filter2/recordlevel/TestRecordLevelFilters.java ## @@ -146,6 +147,33 @@ public void test

[GitHub] [parquet-mr] huaxingao commented on a change in pull request #923: [PARQUET-1968] FilterApi support In predicate

2021-09-26 Thread GitBox
huaxingao commented on a change in pull request #923: URL: https://github.com/apache/parquet-mr/pull/923#discussion_r716278171 ## File path: parquet-column/src/main/java/org/apache/parquet/internal/column/columnindex/ColumnIndexBuilder.java ## @@ -326,12 +323,27 @@ boolean isN

[GitHub] [parquet-mr] huaxingao commented on a change in pull request #923: [PARQUET-1968] FilterApi support In predicate

2021-09-26 Thread GitBox
huaxingao commented on a change in pull request #923: URL: https://github.com/apache/parquet-mr/pull/923#discussion_r716278127 ## File path: parquet-generator/src/main/java/org/apache/parquet/filter2/IncrementallyUpdatedFilterPredicateGenerator.java ## @@ -1,14 +1,14 @@ -/*

[GitHub] [parquet-mr] shangxinli commented on pull request #928: PARQUET-2081: Encryption translation tool - Parquet-hadoop

2021-09-23 Thread GitBox
shangxinli commented on pull request #928: URL: https://github.com/apache/parquet-mr/pull/928#issuecomment-926207482 @ggershinsky @gszadovszky Just pushed another commit to address more feedback. Can you have another look? -- This is an automated message from the Apache Git Service. To

[GitHub] [parquet-mr] shangxinli commented on a change in pull request #928: PARQUET-2081: Encryption translation tool - Parquet-hadoop

2021-09-23 Thread GitBox
shangxinli commented on a change in pull request #928: URL: https://github.com/apache/parquet-mr/pull/928#discussion_r715195044 ## File path: parquet-hadoop/src/main/java/org/apache/parquet/hadoop/util/ColumnEncryptor.java ## @@ -0,0 +1,351 @@ +/* + * Licensed to the Apache So

[GitHub] [parquet-mr] rdblue commented on a change in pull request #933: PARQUET-2094: Handle negative values in page headers

2021-09-23 Thread GitBox
rdblue commented on a change in pull request #933: URL: https://github.com/apache/parquet-mr/pull/933#discussion_r715155364 ## File path: parquet-format-structures/src/test/java/org/apache/parquet/format/TestUtil.java ## @@ -77,6 +81,21 @@ public void testReadFileMetadata() th

[GitHub] [parquet-mr] rdblue commented on a change in pull request #933: PARQUET-2094: Handle negative values in page headers

2021-09-23 Thread GitBox
rdblue commented on a change in pull request #933: URL: https://github.com/apache/parquet-mr/pull/933#discussion_r715154831 ## File path: parquet-format-structures/src/main/java/org/apache/parquet/format/Util.java ## @@ -130,7 +131,7 @@ public static PageHeader readPageHeader(

[GitHub] [parquet-mr] rdblue commented on a change in pull request #933: PARQUET-2094: Handle negative values in page headers

2021-09-23 Thread GitBox
rdblue commented on a change in pull request #933: URL: https://github.com/apache/parquet-mr/pull/933#discussion_r715154439 ## File path: parquet-format-structures/src/main/java/org/apache/parquet/format/MetadataValidator.java ## @@ -0,0 +1,56 @@ +/* + * Licensed to the Apache

[GitHub] [parquet-mr] rdblue commented on a change in pull request #933: PARQUET-2094: Handle negative values in page headers

2021-09-23 Thread GitBox
rdblue commented on a change in pull request #933: URL: https://github.com/apache/parquet-mr/pull/933#discussion_r715153249 ## File path: parquet-format-structures/src/main/java/org/apache/parquet/format/MetadataValidator.java ## @@ -0,0 +1,56 @@ +/* + * Licensed to the Apache

[GitHub] [parquet-mr] gszadovszky commented on a change in pull request #923: [PARQUET-1968] FilterApi support In predicate

2021-09-23 Thread GitBox
gszadovszky commented on a change in pull request #923: URL: https://github.com/apache/parquet-mr/pull/923#discussion_r714956184 ## File path: parquet-hadoop/src/main/java/org/apache/parquet/filter2/statisticslevel/StatisticsFilter.java ## @@ -186,26 +186,36 @@ private boolean

[GitHub] [parquet-mr] shangxinli commented on pull request #933: PARQUET-2094: Handle negative values in page headers

2021-09-23 Thread GitBox
shangxinli commented on pull request #933: URL: https://github.com/apache/parquet-mr/pull/933#issuecomment-925915291 LGTM -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. T

[GitHub] [parquet-mr] plygrnd commented on pull request #933: PARQUET-2094: Handle negative values in page headers

2021-09-23 Thread GitBox
plygrnd commented on pull request #933: URL: https://github.com/apache/parquet-mr/pull/933#issuecomment-925872659 Okay, LGTM then. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific c

<    8   9   10   11   12   13   14   15   16   17   >