yifeih commented on a change in pull request #107: Integrate encryption into
datasource
URL: https://github.com/apache/incubator-iceberg/pull/107#discussion_r262182454
##########
File path: spark/src/main/java/com/netflix/iceberg/spark/source/Reader.java
##########
@@ -269,16 +286,26 @@ private Schema lazyExpectedSchema() {
private final Schema tableSchema;
private final Schema expectedSchema;
private final FileIO fileIo;
+ private final Map<String, InputFile> inputFiles;
private Iterator<InternalRow> currentIterator = null;
private Closeable currentCloseable = null;
private InternalRow current = null;
- public TaskDataReader(CombinedScanTask task, Schema tableSchema, Schema
expectedSchema, FileIO fileIo) {
+ public TaskDataReader(CombinedScanTask task, Schema tableSchema, Schema
expectedSchema, FileIO fileIo,
+ EncryptionManager encryptionManager) {
this.fileIo = fileIo;
this.tasks = task.files().iterator();
this.tableSchema = tableSchema;
this.expectedSchema = expectedSchema;
+ Iterable<InputFile> decryptedFiles =
encryptionManager.decrypt(Iterables.transform(task.files(),
+ fileScanTask ->
+ EncryptedFiles.encryptedInput(
+
this.fileIo.newInputFile(fileScanTask.file().path().toString()),
+ fileScanTask.file().keyMetadata())));
Review comment:
I believe it's to make the `DataFile` API cleaner and less complicated, see
discussion thread here:
https://github.com/apache/incubator-iceberg/pull/85#discussion_r253150641
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
