Re: [ANNOUNCE] New Parquet PMC Member: Gang Wu

2024-05-11 Thread Gidon Gershinsky
Congrats Gang, well deserved! Cheers, Gidon On Sat, 11 May 2024 at 20:19 Xinli shang wrote: > Hi all, > > As a Parquet committer, Gang Wu has remained very active and instructive in > the community. The Parquet community invited him to be a PMC member, and he > accepted. It's my pleasure to

Re: [VOTE] Release Apache Parquet 1.14.0 RC1

2024-05-07 Thread Gidon Gershinsky
+1 (binding) - ran the tests - ran with the Iceberg encryption code Cheers, Gidon On Tue, May 7, 2024 at 4:28 AM Gang Wu wrote: > Hi, > > It has been open for more than 72 hours already. We still need 2 more > binding votes. Considering that there was a weekend during the voting > hours,

Re: [VOTE] Release Apache Parquet 1.14.0 RC0

2024-05-02 Thread Gidon Gershinsky
+1 (binding) Ran the build and tests. I'm told by the Spark community they'd like to integrate the new parquet-mr in Spark 4.0, so are interested in having the v1.14 as soon as possible. On Tue, Apr 30, 2024 at 6:26 PM Vinoo Ganesh wrote: > +1 (non-binding) > > Bumped to 1.14.0-SNAPSHOT in

Re: How the key rotation works when using Parquet Modular Encryption

2023-11-30 Thread Gidon Gershinsky
On Wed, Nov 29, 2023 at 5:40 PM Priyanshu Sharma wrote: > With Parquet Modular Encryption > 1. With each key rotation , Is it possible to avoid encryption and > decryption of existing data? > Yes > > 2. If master key rotation does not require modification of the data file > then how would the

Re: [VOTE] Release Apache Parquet Format 2.10.0 RC0

2023-11-19 Thread Gidon Gershinsky
+1 (binding). Thanks Gang. Cheers, Gidon On Fri, Nov 17, 2023 at 5:07 PM Xinli shang wrote: > +1 (binding) > > Verified the signature. Thanks Gang for leading the effort! > > On Thu, Nov 16, 2023 at 9:41 PM wish maple wrote: > > > +1 (no-binding) > > > > Thanks Gang for release! > > > >

Re: [VOTE][FORMAT] Add repetition, definition and variable length size metadata statistics

2023-11-13 Thread Gidon Gershinsky
+1 (binding) Cheers, Gidon On Tue, Nov 14, 2023 at 5:31 AM Xinli shang wrote: > Yeah, we need one more PMC to vote. If you can help, appreciate it. > > On Mon, Nov 13, 2023 at 6:23 AM Fokko Driesprong wrote: > > > +1 non-binding > > > > Great work Micah, I went through the PR and it looks

[jira] [Resolved] (PARQUET-2364) Encrypt all columns option

2023-11-08 Thread Gidon Gershinsky (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-2364?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gidon Gershinsky resolved PARQUET-2364. --- Fix Version/s: 1.14.0 Resolution: Fixed > Encrypt all columns opt

[jira] [Resolved] (PARQUET-2370) Crypto factory activation of "all column encryption" mode

2023-11-08 Thread Gidon Gershinsky (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-2370?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gidon Gershinsky resolved PARQUET-2370. --- Resolution: Fixed > Crypto factory activation of "all column encrypti

[jira] [Updated] (PARQUET-2370) Crypto factory activation of "all column encryption" mode

2023-11-08 Thread Gidon Gershinsky (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-2370?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gidon Gershinsky updated PARQUET-2370: -- Fix Version/s: 1.14.0 > Crypto factory activation of "all column encrypti

[jira] [Created] (PARQUET-2370) Crypto factory activation of "all column encryption" mode

2023-10-23 Thread Gidon Gershinsky (Jira)
Gidon Gershinsky created PARQUET-2370: - Summary: Crypto factory activation of "all column encryption" mode Key: PARQUET-2370 URL: https://issues.apache.org/jira/browse/PARQUET-2370

[jira] [Created] (PARQUET-2364) Encrypt all columns option

2023-10-16 Thread Gidon Gershinsky (Jira)
Gidon Gershinsky created PARQUET-2364: - Summary: Encrypt all columns option Key: PARQUET-2364 URL: https://issues.apache.org/jira/browse/PARQUET-2364 Project: Parquet Issue Type

[jira] [Commented] (PARQUET-2223) Parquet Data Masking for Column Encryption

2023-06-16 Thread Gidon Gershinsky (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-2223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17733538#comment-17733538 ] Gidon Gershinsky commented on PARQUET-2223: --- Yep, I also think so. I'll have a look

Re: [VOTE] Release Apache Parquet 1.13.1 RC0

2023-05-14 Thread Gidon Gershinsky
+1 ran the test suite. Cheers, Gidon On Sat, May 13, 2023 at 11:48 PM Xinli shang wrote: > +1 > > I verified the signature and ran a sanity test. > > > > On Fri, May 12, 2023 at 6:15 PM pk singh wrote: > > > Thanks Fokko, this is super-helpful and unblocks parquet 1.13 upgrade for > >

[jira] [Commented] (PARQUET-2193) Encrypting only one field in nested field prevents reading of other fields in nested field without keys

2023-05-04 Thread Gidon Gershinsky (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-2193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17719294#comment-17719294 ] Gidon Gershinsky commented on PARQUET-2193: --- [~Nageswaran] A couple of updates on this. We

[jira] [Created] (PARQUET-2297) Encrypted files should not be checked for delta encoding problem

2023-05-04 Thread Gidon Gershinsky (Jira)
Gidon Gershinsky created PARQUET-2297: - Summary: Encrypted files should not be checked for delta encoding problem Key: PARQUET-2297 URL: https://issues.apache.org/jira/browse/PARQUET-2297 Project

[jira] [Commented] (PARQUET-2193) Encrypting only one field in nested field prevents reading of other fields in nested field without keys

2023-05-02 Thread Gidon Gershinsky (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-2193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17718795#comment-17718795 ] Gidon Gershinsky commented on PARQUET-2193: --- Yep, sorry about the delay. This turned out

Re: [VOTE] Release Apache Parquet 1.13.0 RC0

2023-04-05 Thread Gidon Gershinsky
+1 Ran the tests. Thanks Gang and all contributors! Cheers, Gidon On Tue, Apr 4, 2023 at 3:54 AM Xinli shang wrote: > +1 > > Verified checksum and signature, and ran internal tests. > > Gang, thanks a lot for leading this effort! > > On Mon, Apr 3, 2023 at 12:06 AM Gábor Szádovszky wrote: >

Re: [VOTE] Release Apache Parquet 1.12.4 RC0

2023-03-28 Thread Gidon Gershinsky
+1 Verified signature and ran the tests. Thanks Gang and all contributors! Cheers, Gidon On Tue, Mar 28, 2023 at 5:19 PM Xinli shang wrote: > +1 > > Verified signature and ran internal tests. Thanks Gang for leading this > effort! > > On Mon, Mar 27, 2023 at 9:38 AM Dongjoon Hyun wrote: >

Re: Gang Wu as new Apache Parquet committer

2023-03-04 Thread Gidon Gershinsky
Congrats Gang! Cheers, Gidon On Sat, Mar 4, 2023 at 10:41 PM Micah Kornfield wrote: > Congrats! > > On Monday, February 27, 2023, Xinli shang wrote: > > > The Project Management Committee (PMC) for Apache Parquet has invited > Gang > > Wu (gangwu) to become a committer and we are pleased to

[jira] [Assigned] (PARQUET-2103) crypto exception in print toPrettyJSON

2023-01-11 Thread Gidon Gershinsky (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-2103?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gidon Gershinsky reassigned PARQUET-2103: - Assignee: Gidon Gershinsky > crypto exception in print toPrettyJ

[jira] [Updated] (PARQUET-2103) crypto exception in print toPrettyJSON

2023-01-11 Thread Gidon Gershinsky (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-2103?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gidon Gershinsky updated PARQUET-2103: -- Affects Version/s: 1.12.3 > crypto exception in print toPrettyJ

[jira] [Updated] (PARQUET-2103) crypto exception in print toPrettyJSON

2023-01-11 Thread Gidon Gershinsky (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-2103?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gidon Gershinsky updated PARQUET-2103: -- Priority: Minor (was: Major) > crypto exception in print toPrettyJ

Re: Modular encryption to support arrays and nested arrays

2022-10-31 Thread Gidon Gershinsky
Parquet columnar encryption supports these types. Currently, it requires an explicit full path for each column to be encrypted. Your sample will work with *spark.sparkContext.hadoopConfiguration.set("parquet.encryption.column.keys", "k2:rider.list.element.foo,rider.list.element.bar")* Having said

[jira] [Created] (PARQUET-2208) Add details to nested column encryption config doc and exception text

2022-10-31 Thread Gidon Gershinsky (Jira)
Gidon Gershinsky created PARQUET-2208: - Summary: Add details to nested column encryption config doc and exception text Key: PARQUET-2208 URL: https://issues.apache.org/jira/browse/PARQUET-2208

Re: Modular encryption to return null values instead of Crypto exception when bad key provided

2022-10-27 Thread Gidon Gershinsky
trying to project columns without authorization can be very costly, for two reasons: - unnecessary per-column/file calls to the (remote) KMS service, plus the cost of per-call authorization checks - red-flagging unauthorized calls and triggering "breach attempt" alerts IMO, the best way to handle

Re: Parquet modular encryption on nested fields

2022-10-26 Thread Gidon Gershinsky
There is a discussion on this at https://issues.apache.org/jira/browse/PARQUET-2193 . Basically, a workaround exists today, please check if it works for you. Currently, I'm checking options for a more permanent solution. (in the future, please send emails with text, instead of attaching it as a

[jira] [Commented] (PARQUET-2193) Encrypting only one field in nested field prevents reading of other fields in nested field without keys

2022-10-10 Thread Gidon Gershinsky (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-2193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17614917#comment-17614917 ] Gidon Gershinsky commented on PARQUET-2193: --- Welcome. >From the sound of it, this mi

[jira] [Commented] (PARQUET-2193) Encrypting only one field in nested field prevents reading of other fields in nested field without keys

2022-09-29 Thread Gidon Gershinsky (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-2193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17610868#comment-17610868 ] Gidon Gershinsky commented on PARQUET-2193: --- Hmm, looks like this method runs over all

[jira] [Commented] (PARQUET-2194) parquet.encryption.plaintext.footer parameter being true, code expects parquet.encryption.footer.key

2022-09-29 Thread Gidon Gershinsky (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-2194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17610855#comment-17610855 ] Gidon Gershinsky commented on PARQUET-2194: --- Footer key is required also in the plaintext

[jira] [Created] (PARQUET-2197) Document uniform encryption

2022-09-28 Thread Gidon Gershinsky (Jira)
Gidon Gershinsky created PARQUET-2197: - Summary: Document uniform encryption Key: PARQUET-2197 URL: https://issues.apache.org/jira/browse/PARQUET-2197 Project: Parquet Issue Type

[jira] [Commented] (PARQUET-1711) [parquet-protobuf] stack overflow when work with well known json type

2022-09-14 Thread Gidon Gershinsky (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-1711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17605098#comment-17605098 ] Gidon Gershinsky commented on PARQUET-1711: --- [~emkornfield] what do you think about these 3

[jira] [Comment Edited] (PARQUET-1711) [parquet-protobuf] stack overflow when work with well known json type

2022-09-08 Thread Gidon Gershinsky (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-1711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17602127#comment-17602127 ] Gidon Gershinsky edited comment on PARQUET-1711 at 9/9/22 5:45 AM

[jira] [Commented] (PARQUET-1711) [parquet-protobuf] stack overflow when work with well known json type

2022-09-08 Thread Gidon Gershinsky (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-1711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17602127#comment-17602127 ] Gidon Gershinsky commented on PARQUET-1711: --- Hi to all on this Jira. Looks like we have

[jira] [Resolved] (PARQUET-2040) Uniform encryption

2022-07-28 Thread Gidon Gershinsky (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-2040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gidon Gershinsky resolved PARQUET-2040. --- Resolution: Fixed > Uniform encrypt

[jira] [Resolved] (PARQUET-2136) File writer construction with encryptor

2022-07-28 Thread Gidon Gershinsky (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-2136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gidon Gershinsky resolved PARQUET-2136. --- Resolution: Fixed > File writer construction with encryp

Re: Review of Q2 Parquet report

2022-07-05 Thread Gidon Gershinsky
ently 37 committers and 27 PMC members in this project. > The Committer-to-PMC ratio is roughly 5:4. > > Community changes, past quarter: > - No new PMC members. Last addition was Gidon Gershinsky on 2021-11-23. > - No new committers. Last addition was Gidon Gershinsky on 2021-04-05. > &g

[jira] [Resolved] (PARQUET-2120) parquet-cli dictionary command fails on pages without dictionary encoding

2022-06-21 Thread Gidon Gershinsky (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-2120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gidon Gershinsky resolved PARQUET-2120. --- Resolution: Fixed > parquet-cli dictionary command fails on pages with

[jira] [Commented] (PARQUET-2120) parquet-cli dictionary command fails on pages without dictionary encoding

2022-06-21 Thread Gidon Gershinsky (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-2120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17556828#comment-17556828 ] Gidon Gershinsky commented on PARQUET-2120: --- [~shangxinli] and the Parquet community, can you

[jira] [Resolved] (PARQUET-2148) Enable uniform decryption with plaintext footer

2022-06-21 Thread Gidon Gershinsky (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-2148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gidon Gershinsky resolved PARQUET-2148. --- Resolution: Fixed > Enable uniform decryption with plaintext foo

[jira] [Resolved] (PARQUET-2144) Fix ColumnIndexBuilder for notIn predicate

2022-06-21 Thread Gidon Gershinsky (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-2144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gidon Gershinsky resolved PARQUET-2144. --- Resolution: Fixed > Fix ColumnIndexBuilder for notIn predic

[jira] [Resolved] (PARQUET-2145) Release 1.12.3

2022-06-21 Thread Gidon Gershinsky (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-2145?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gidon Gershinsky resolved PARQUET-2145. --- Resolution: Fixed > Release 1.12.3 > -- > >

[jira] [Commented] (PARQUET-2145) Release 1.12.3

2022-06-21 Thread Gidon Gershinsky (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-2145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17556825#comment-17556825 ] Gidon Gershinsky commented on PARQUET-2145: --- This version is already released, [https

[jira] [Commented] (PARQUET-2117) Add rowPosition API in parquet record readers

2022-06-13 Thread Gidon Gershinsky (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-2117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17553425#comment-17553425 ] Gidon Gershinsky commented on PARQUET-2117: --- [~sha...@uber.com] Could you add [~prakharjain09

Re: [VOTE] Release Apache Parquet 1.12.3 RC1

2022-05-22 Thread Gidon Gershinsky
+1. Downloaded, verified and tested. Cheers, Gidon On Fri, May 20, 2022 at 8:49 PM Xinli shang wrote: > Hi everyone, > > > I propose the following RC to be released as the official Apache Parquet > 1.12.3 release. > > > The commit id is f8dced182c4c1fbdec6ccb3185537b5a01e6ed6b > > * This

[jira] [Updated] (PARQUET-2101) Fix wrong descriptions about the default block size

2022-05-19 Thread Gidon Gershinsky (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-2101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gidon Gershinsky updated PARQUET-2101: -- Fix Version/s: 1.12.3 > Fix wrong descriptions about the default block s

[jira] [Updated] (PARQUET-2081) Encryption translation tool - Parquet-hadoop

2022-05-19 Thread Gidon Gershinsky (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-2081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gidon Gershinsky updated PARQUET-2081: -- Fix Version/s: 1.12.3 (was: 1.13.0) > Encryption translat

[jira] [Updated] (PARQUET-2102) Typo in ColumnIndexBase toString

2022-05-19 Thread Gidon Gershinsky (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-2102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gidon Gershinsky updated PARQUET-2102: -- Fix Version/s: 1.12.3 > Typo in ColumnIndexBase toStr

[jira] [Updated] (PARQUET-2040) Uniform encryption

2022-05-19 Thread Gidon Gershinsky (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-2040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gidon Gershinsky updated PARQUET-2040: -- Fix Version/s: 1.12.3 > Uniform encrypt

[jira] [Updated] (PARQUET-2076) Improve Travis CI build Performance

2022-05-19 Thread Gidon Gershinsky (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-2076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gidon Gershinsky updated PARQUET-2076: -- Fix Version/s: 1.12.3 > Improve Travis CI build Performa

[jira] [Updated] (PARQUET-2107) Travis failures

2022-05-19 Thread Gidon Gershinsky (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-2107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gidon Gershinsky updated PARQUET-2107: -- Fix Version/s: 1.12.3 > Travis failures > --- > >

[jira] [Updated] (PARQUET-2106) BinaryComparator should avoid doing ByteBuffer.wrap in the hot-path

2022-05-19 Thread Gidon Gershinsky (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-2106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gidon Gershinsky updated PARQUET-2106: -- Fix Version/s: 1.12.3 > BinaryComparator should avoid doing ByteBuffer.w

[jira] [Updated] (PARQUET-2105) Refactor the test code of creating the test file

2022-05-19 Thread Gidon Gershinsky (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-2105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gidon Gershinsky updated PARQUET-2105: -- Fix Version/s: 1.12.3 > Refactor the test code of creating the test f

[jira] [Updated] (PARQUET-2112) Fix typo in MessageColumnIO

2022-05-19 Thread Gidon Gershinsky (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-2112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gidon Gershinsky updated PARQUET-2112: -- Fix Version/s: 1.12.3 (was: 1.13.0) > Fix t

[jira] [Updated] (PARQUET-2128) Bump Thrift to 0.16.0

2022-05-19 Thread Gidon Gershinsky (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-2128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gidon Gershinsky updated PARQUET-2128: -- Fix Version/s: 1.12.3 > Bump Thrift to 0.1

[jira] [Updated] (PARQUET-2120) parquet-cli dictionary command fails on pages without dictionary encoding

2022-05-19 Thread Gidon Gershinsky (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-2120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gidon Gershinsky updated PARQUET-2120: -- Fix Version/s: 1.12.3 > parquet-cli dictionary command fails on pages with

[jira] [Updated] (PARQUET-2129) Add uncompressedSize to "meta" output

2022-05-19 Thread Gidon Gershinsky (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-2129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gidon Gershinsky updated PARQUET-2129: -- Fix Version/s: 1.12.3 > Add uncompressedSize to "meta

[jira] [Updated] (PARQUET-2121) Remove descriptions for the removed modules

2022-05-19 Thread Gidon Gershinsky (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-2121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gidon Gershinsky updated PARQUET-2121: -- Fix Version/s: 1.12.3 > Remove descriptions for the removed modu

[jira] [Updated] (PARQUET-2136) File writer construction with encryptor

2022-05-18 Thread Gidon Gershinsky (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-2136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gidon Gershinsky updated PARQUET-2136: -- Fix Version/s: 1.12.3 > File writer construction with encryp

[jira] [Updated] (PARQUET-2144) Fix ColumnIndexBuilder for notIn predicate

2022-05-18 Thread Gidon Gershinsky (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-2144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gidon Gershinsky updated PARQUET-2144: -- Fix Version/s: 1.12.3 > Fix ColumnIndexBuilder for notIn predic

[jira] [Updated] (PARQUET-2127) Security risk in latest parquet-jackson-1.12.2.jar

2022-05-18 Thread Gidon Gershinsky (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-2127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gidon Gershinsky updated PARQUET-2127: -- Fix Version/s: 1.12.3 > Security risk in latest parquet-jackson-1.12.2.

[jira] [Created] (PARQUET-2148) Enable uniform decryption with plaintext footer

2022-05-16 Thread Gidon Gershinsky (Jira)
Gidon Gershinsky created PARQUET-2148: - Summary: Enable uniform decryption with plaintext footer Key: PARQUET-2148 URL: https://issues.apache.org/jira/browse/PARQUET-2148 Project: Parquet

Re: Meeting notes for Parquet monthly sync - 4/27/2022

2022-05-04 Thread Gidon Gershinsky
, 2022 at 8:03 PM Xinli shang wrote: > 4/27/2022 > > Attendees (Timothy Miller, Vinoo Ganesh, Satish K, Gidon Gershinsky, Xinli > Shang, Huaxin Gao) > >1. > >Cell-Level encryption >1. > > Internal implementation and rollout > 2. >

[jira] [Created] (PARQUET-2145) Release 1.12.3

2022-05-04 Thread Gidon Gershinsky (Jira)
Gidon Gershinsky created PARQUET-2145: - Summary: Release 1.12.3 Key: PARQUET-2145 URL: https://issues.apache.org/jira/browse/PARQUET-2145 Project: Parquet Issue Type: Task

[jira] [Commented] (PARQUET-2098) Add more methods into interface of BlockCipher

2022-04-24 Thread Gidon Gershinsky (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-2098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17526997#comment-17526997 ] Gidon Gershinsky commented on PARQUET-2098: --- [~theosib-amazon] I got ~half of this (code

[jira] [Created] (PARQUET-2136) File writer construction with encryptor

2022-04-04 Thread Gidon Gershinsky (Jira)
Gidon Gershinsky created PARQUET-2136: - Summary: File writer construction with encryptor Key: PARQUET-2136 URL: https://issues.apache.org/jira/browse/PARQUET-2136 Project: Parquet Issue

Re: Parquet Column Resolution by ID

2022-02-12 Thread Gidon Gershinsky
Thanks Xinli, works well now. I've reviewed the doc. Cheers, Gidon On Fri, Feb 11, 2022 at 7:21 PM Xinli shang wrote: > Hi Gidon, > > I just shared the 'comment' permission for everybody. Let me know if you > still have issues with it. > > Xinli > > On Thu, Feb 1

Re: Parquet Column Resolution by ID

2022-02-10 Thread Gidon Gershinsky
Hi Huaxin, Can you open this document for comments? Cheers, Gidon On Fri, Feb 11, 2022 at 6:01 AM huaxin gao wrote: > Hi Parquet community, > > Xinli and I drafted a design doc to support ID based column resolution in > Parquet. Here is the link > < >

[jira] [Commented] (PARQUET-2098) Add more methods into interface of BlockCipher

2022-01-27 Thread Gidon Gershinsky (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-2098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17483575#comment-17483575 ] Gidon Gershinsky commented on PARQUET-2098: --- sure, I can take this one > Add more meth

[jira] [Commented] (PARQUET-2103) crypto exception in print toPrettyJSON

2021-11-24 Thread Gidon Gershinsky (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-2103?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17448596#comment-17448596 ] Gidon Gershinsky commented on PARQUET-2103: --- [~gszadovszky] thanks for pointing in the right

[jira] [Updated] (PARQUET-2103) crypto exception in print toPrettyJSON

2021-11-24 Thread Gidon Gershinsky (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-2103?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gidon Gershinsky updated PARQUET-2103: -- Description: In debug mode, this code  {{if (LOG.isDebugEnabled()) {}} {{  LOG.debug

[jira] [Commented] (PARQUET-2103) crypto exception in print toPrettyJSON

2021-11-22 Thread Gidon Gershinsky (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-2103?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17447372#comment-17447372 ] Gidon Gershinsky commented on PARQUET-2103: --- [~gszadovszky] [~sha...@uber.com

[jira] [Updated] (PARQUET-2103) crypto exception in print toPrettyJSON

2021-11-16 Thread Gidon Gershinsky (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-2103?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gidon Gershinsky updated PARQUET-2103: -- Description: In debug mode, this code  {{if (LOG.isDebugEnabled()) {}} {{  LOG.debug

[jira] [Created] (PARQUET-2103) crypto exception in print toPrettyJSON

2021-11-16 Thread Gidon Gershinsky (Jira)
Gidon Gershinsky created PARQUET-2103: - Summary: crypto exception in print toPrettyJSON Key: PARQUET-2103 URL: https://issues.apache.org/jira/browse/PARQUET-2103 Project: Parquet Issue

[jira] [Commented] (PARQUET-2080) Deprecate RowGroup.file_offset

2021-09-28 Thread Gidon Gershinsky (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-2080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17421323#comment-17421323 ] Gidon Gershinsky commented on PARQUET-2080: --- Oh, sorry, done. > Deprec

[jira] [Commented] (PARQUET-2080) Deprecate RowGroup.file_offset

2021-09-28 Thread Gidon Gershinsky (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-2080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17421193#comment-17421193 ] Gidon Gershinsky commented on PARQUET-2080: --- Hi [~gszadovszky] , I've prepared a short

Re: [VOTE] Release Apache Parquet 1.12.1 RC1

2021-09-15 Thread Gidon Gershinsky
A late +1 (non-binding). - ran build and test, everything was ok - ran extra tests with encryption, standalone and Spark, everything passed Thanks Xinli and all for contributing to this release! Cheers, Gidon On Wed, Sep 15, 2021 at 6:53 AM Xinli shang wrote: > The vote to release 1.12.1

[jira] [Updated] (PARQUET-2080) Deprecate RowGroup.file_offset

2021-09-14 Thread Gidon Gershinsky (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-2080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gidon Gershinsky updated PARQUET-2080: -- Description: Due to PARQUET-2078 RowGroup.file_offset is not reliable. This field

[jira] [Assigned] (PARQUET-2080) Deprecate RowGroup.file_offset

2021-09-14 Thread Gidon Gershinsky (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-2080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gidon Gershinsky reassigned PARQUET-2080: - Assignee: Gidon Gershinsky (was: Gabor Szadovszky) > Deprec

[jira] [Updated] (PARQUET-2080) Deprecate RowGroup.file_offset

2021-09-13 Thread Gidon Gershinsky (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-2080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gidon Gershinsky updated PARQUET-2080: -- Description: Due to PARQUET-2078 RowGroup.file_offset is not reliable. This field

[jira] [Commented] (PARQUET-2080) Deprecate RowGroup.file_offset

2021-09-13 Thread Gidon Gershinsky (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-2080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17414075#comment-17414075 ] Gidon Gershinsky commented on PARQUET-2080: --- [~gszadovszky] yes, I'll take it. There might

Re: [VOTE] Release Apache Parquet 1.12.1 RC0

2021-09-13 Thread Gidon Gershinsky
+1 (non-binding) - checked the sum - ran build and test, everything was ok - ran additional framework tests with the built jars, passed Cheers, Gidon On Sun, Sep 12, 2021 at 12:05 AM Xinli shang wrote: > Hi everyone, > > > I propose the following RC to be released as the official Apache

[jira] [Commented] (PARQUET-2071) Encryption translation tool

2021-08-05 Thread Gidon Gershinsky (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-2071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17393982#comment-17393982 ] Gidon Gershinsky commented on PARQUET-2071: --- A very useful tool, I'll be glad to review

[jira] [Resolved] (PARQUET-1908) CLONE - [C++] Update cpp crypto package to match signed-off specification

2021-08-03 Thread Gidon Gershinsky (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-1908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gidon Gershinsky resolved PARQUET-1908. --- Resolution: Fixed PR merged in May 2019 > CLONE - [C++] Update cpp crypto pack

Re: New Parquet PMC chair

2021-05-28 Thread Gidon Gershinsky
Congratulations Xinli, well deserved!! Cheers, Gidon On Sat, May 29, 2021 at 12:34 AM Julien Le Dem wrote: > Hello Parquet community, > The Parquet PMC discussed and decided some time ago to move to a rotating > chair. > Every year around this time the PMC will elect a new chair to represent

[jira] [Created] (PARQUET-2053) Pluggable key material store

2021-05-25 Thread Gidon Gershinsky (Jira)
Gidon Gershinsky created PARQUET-2053: - Summary: Pluggable key material store Key: PARQUET-2053 URL: https://issues.apache.org/jira/browse/PARQUET-2053 Project: Parquet Issue Type

[jira] [Updated] (PARQUET-1230) CLI tools for encrypted files

2021-05-04 Thread Gidon Gershinsky (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-1230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gidon Gershinsky updated PARQUET-1230: -- Component/s: parquet-mr Affects Version/s: 1.12.0 > CLI to

[jira] [Updated] (PARQUET-1230) CLI tools for encrypted files

2021-05-04 Thread Gidon Gershinsky (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-1230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gidon Gershinsky updated PARQUET-1230: -- Parent: (was: PARQUET-1178) Issue Type: New Feature (was: Sub-task

[jira] [Created] (PARQUET-2040) Uniform encryption

2021-04-29 Thread Gidon Gershinsky (Jira)
Gidon Gershinsky created PARQUET-2040: - Summary: Uniform encryption Key: PARQUET-2040 URL: https://issues.apache.org/jira/browse/PARQUET-2040 Project: Parquet Issue Type: Improvement

[jira] [Created] (PARQUET-2033) Make "null decryptor" exception more informative

2021-04-20 Thread Gidon Gershinsky (Jira)
Gidon Gershinsky created PARQUET-2033: - Summary: Make "null decryptor" exception more informative Key: PARQUET-2033 URL: https://issues.apache.org/jira/browse/PARQUET-2033 Projec

[jira] [Created] (PARQUET-2014) Local key wrapping with rotation

2021-04-04 Thread Gidon Gershinsky (Jira)
Gidon Gershinsky created PARQUET-2014: - Summary: Local key wrapping with rotation Key: PARQUET-2014 URL: https://issues.apache.org/jira/browse/PARQUET-2014 Project: Parquet Issue Type

[jira] [Resolved] (PARQUET-1613) Key rotation tool

2021-04-04 Thread Gidon Gershinsky (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-1613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gidon Gershinsky resolved PARQUET-1613. --- Resolution: Done handled by pr 615 > Key rotation t

[jira] [Resolved] (PARQUET-1612) Double wrapped key manager

2021-04-04 Thread Gidon Gershinsky (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-1612?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gidon Gershinsky resolved PARQUET-1612. --- Resolution: Done handled by pr 615 > Double wrapped key mana

[jira] [Resolved] (PARQUET-1178) Parquet modular encryption

2021-03-26 Thread Gidon Gershinsky (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-1178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gidon Gershinsky resolved PARQUET-1178. --- Resolution: Done Released. Thanks to all who've contributed to this new Parquet

[jira] [Updated] (PARQUET-1178) Parquet modular encryption

2021-03-26 Thread Gidon Gershinsky (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-1178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gidon Gershinsky updated PARQUET-1178: -- Fix Version/s: 1.12.0 > Parquet modular encrypt

Re: [RESULT] Release Apache Parquet 1.12.0 RC4

2021-03-25 Thread Gidon Gershinsky
Great news!!! And thanks Gabor and Xinli for handling the release process! Cheers, Gidon On Thu, Mar 25, 2021 at 7:01 PM Xinli shang wrote: > Thanks everybody for the verification and special thanks to all the > contributors to this release! This release includes awesome features and >

[jira] [Commented] (PARQUET-1997) [C++] AesEncryptor and AesDecryptor primitives are unsafe

2021-03-10 Thread Gidon Gershinsky (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-1997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17299054#comment-17299054 ] Gidon Gershinsky commented on PARQUET-1997: --- I recall we talked about that with Tham, but I

[jira] [Commented] (PARQUET-1997) [C++] AesEncryptor and AesDecryptor primitives are unsafe

2021-03-10 Thread Gidon Gershinsky (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-1997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17299041#comment-17299041 ] Gidon Gershinsky commented on PARQUET-1997: --- [~apitrou] This point is addressed by the _int

[jira] [Commented] (PARQUET-1992) Cannot build from tarball because of git submodules

2021-03-02 Thread Gidon Gershinsky (Jira)
[ https://issues.apache.org/jira/browse/PARQUET-1992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17293509#comment-17293509 ] Gidon Gershinsky commented on PARQUET-1992: --- This contribution had been added by [~mayaa

[jira] [Created] (PARQUET-1989) Deep verification of encrypted files

2021-02-28 Thread Gidon Gershinsky (Jira)
Gidon Gershinsky created PARQUET-1989: - Summary: Deep verification of encrypted files Key: PARQUET-1989 URL: https://issues.apache.org/jira/browse/PARQUET-1989 Project: Parquet Issue

Re: [RESULT] Release Apache Parquet 1.12.0 RC1

2021-01-29 Thread Gidon Gershinsky
eV1.java:29) > > > at org.apache.parquet.io > > > > > > .MessageColumnIO$MessageColumnIORecordConsumer.endMessage(MessageColumnIO.java:307) > > > at > > > > > > org.apache.spark.sql.execution.datasources.parquet.ParquetWriteSupport.cons

  1   2   3   >