shangxinli commented on code in PR #3935:
URL: https://github.com/apache/parquet-java/pull/3935#discussion_r4589794755


##########
parquet-hadoop/README.md:
##########
@@ -451,6 +451,10 @@ Note: nested column names must be specified as full 
dot-separated paths for each
 
 **Property:** `parquet.encryption.algorithm`  
 **Description:** Parquet encryption algorithm. Can be `AES_GCM_V1` or 
`AES_GCM_CTR_V1`.  
+Since 1.19.0, DataPageV2 pages of encrypted columns encrypt the repetition and 
definition levels with the values,
+as the specification requires. Earlier versions wrote the levels in plaintext. 
Version 1.19.0 reads both layouts.
+Versions 1.18 and earlier cannot read the new pages. With `AES_GCM_V1`, 
decryption fails. With `AES_GCM_CTR_V1`,
+the levels are incorrect.  

Review Comment:
   Running a parquet-java 1.18.2 reader against the Arrow CTR fixtures in this 
PR returns wrong nulls and wrong values with no error for flat optional 
columns, for example 468 of 500 rows wrong on id, even with Snappy, while GCM 
fails with a tag mismatch. So I think this note understates it. Could we add a 
writer property to keep the old layout for a release, with the new layout as 
the default, or at least spell this out in the release notes? Repro: 
https://gist.github.com/shangxinli/3816371fe541e7cbf11fc6daa554a059



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to