hililiwei commented on a change in pull request #3991:
URL: https://github.com/apache/iceberg/pull/3991#discussion_r793274140
##########
File path:
flink/v1.14/flink/src/main/java/org/apache/iceberg/flink/IcebergTableSource.java
##########
@@ -90,12 +94,8 @@ private IcebergTableSource(TableLoader loader, TableSchema
schema, Map<String, S
@Override
public void applyProjection(int[][] projectFields) {
- this.projectedFields = new int[projectFields.length];
- for (int i = 0; i < projectFields.length; i++) {
- Preconditions.checkArgument(projectFields[i].length == 1,
- "Don't support nested projection in iceberg source now.");
- this.projectedFields[i] = projectFields[i][0];
- }
+ this.projectedFields = projectFields;
+ this.topProjectedFields = Arrays.stream(projectFields).mapToInt(array ->
array[0]).distinct().toArray();
Review comment:
Use topProjectedFields to extract data from the original file. In this
case, the nested fields in the data remain the same.
Use projectedFields to flatten the original data. In this case, nested
fields are extracted.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
