RussellSpitzer commented on a change in pull request #4376:
URL: https://github.com/apache/iceberg/pull/4376#discussion_r837630103
##########
File path:
arrow/src/main/java/org/apache/iceberg/arrow/vectorized/ArrowReader.java
##########
@@ -268,7 +268,7 @@ public void close() throws IOException {
.map(entry ->
EncryptedFiles.encryptedInput(io.newInputFile(entry.getKey()),
entry.getValue()));
// decrypt with the batch call to avoid multiple RPCs to a key server,
if possible
- Iterable<InputFile> decryptedFiles =
encryptionManager.decrypt(encrypted::iterator);
+ Iterable<InputFile> decryptedFiles =
encryptionManager.decrypt(encrypted.collect(Collectors.toList()));
Review comment:
This will fully materialize the list so I'm not sure we want to change
the approach here. Currently the only usages do single passes over the iterable
(see the forEach below) ... It is probably fine to suppress here but I also
don't think we lose to much to much by materializing right here since we
immediately put our results into a materialized map anyway.
I guess i'm leaning towards this being ok at the moment although I don't
think it materially improves the code
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
