stevenzwu commented on pull request #3501: URL: https://github.com/apache/iceberg/pull/3501#issuecomment-964812003
> Overall this looks really good to me. Would there be any benefit to us if we used [`FileSourceSplit` interface](https://ci.apache.org/projects/flink/flink-docs-master/api/java/org/apache/flink/connector/file/src/FileSourceSplit.html) instead? Seems like that's a bit too general for our use case. And the SerDe is versioned anyway. @kbendick the initial implementation does have `IcebergSourceSplit` extending from `FileSourceSplit` with the purpose of leveraging the vectorized readers from Flink (suggested by Jingsong). After thinking about it more with Iceberg's row delete filter, I changed the decision to the current state (no extending from FileSourceSplit). When we are ready to support vectorized readers in Flink, we need to make sure they support delete filter properly. There is one open PR for Orc: https://github.com/apache/iceberg/pull/2566 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
