rdblue commented on a change in pull request #1509:
URL: https://github.com/apache/iceberg/pull/1509#discussion_r496268020
##########
File path: flink/src/main/java/org/apache/iceberg/flink/source/FlinkSource.java
##########
@@ -89,8 +88,18 @@ public Builder table(Table newTable) {
return this;
}
- public Builder filters(List<Expression> newFilters) {
- this.filterExpressions = newFilters;
+ public Builder hadoopConf(Configuration newConf) {
Review comment:
We try to avoid using Hadoop classes in Iceberg APIs because they are
hard to remove. Injecting a Hadoop `Configuration` is currently done in one
place: to instantiate a catalog that requires it.
The catalog creates tables and tables are associated with a `FileIO`, so the
configuration is passed down through that chain. The table's configuration
should be used for any table configuration.
MR also has a Hadoop `Configuration`, but that's required by the API so we
can't remove it from the API there. But we still prefer using the table's
`Configuration` when it is needed for components like `HadoopFileIO`.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
