wgtmac commented on PR #1335:
URL: https://github.com/apache/parquet-java/pull/1335#issuecomment-2331821689
> @wgtmac @ConeyLiu Can you check out the tests changes? I created a single
big test for the new functionality. The documentation is still in progress.
It's reasonable on my side, as long as the new feature is covered.
> Some bugs. I found what looks like a bug in a current version of
ParquetRewriter. Probably will fill an issue. When you try to nullify and
encrypt different columns it fails. There is a related test but it nullifies
and encrypts the same column which doesn't reproduce a bug. The bug can be
reproduced by changing a single line `maskColumns.put("DocId",
MaskMode.NULLIFY);` to `maskColumns.put("Links.Forward", MaskMode.NULLIFY);` in
testNullifyAndEncryptColumn() method. The reason of a failure as I understand
is that during the nullification we create a single column schema `MessageType
newSchema = newSchema(schema, descriptor)` and later use our main writer's
encryption with that schema but that encyptor is expecting our final target
schema, not a single column schema.
It would be great if you have time to create a PR to fix this. Thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
