MaxNevermind commented on PR #1335:
URL: https://github.com/apache/parquet-java/pull/1335#issuecomment-2323010561
@wgtmac @ConeyLiu
Can you check out the tests changes? I created a single big test for the new
functionality. The documentation is still in progress.
Code clarifications.
In couple of places in tests I used anonymous nested functions and a local
nested class to try to localize/nest method's tightly related logic into a
single block. Let me know if that is too weird / too functional.
Found bug.
I found what looks like a bug in a current version of ParquetRewriter.
Probably will fill an issue.
When you try to nullify and encrypt different columns it fails. There is a
related test but it nullifies and encrypts the same column which doesn't
reproduce a bug. The bug can be reproduced by changing a single line
`maskColumns.put("DocId", MaskMode.NULLIFY);` to
`maskColumns.put("Links.Forward", MaskMode.NULLIFY);` in
testNullifyAndEncryptColumn() method. The reason of a failure as I understand
is that during the nullification we create a single column schema `MessageType
newSchema = newSchema(schema, descriptor)` and later use our main writer's
encryption with that schema but that encyptor is expecting our final target
schema, not a single column schema.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
