pvary commented on pull request #2614: URL: https://github.com/apache/iceberg/pull/2614#issuecomment-844985765
> But what I can sure is [hconf in method param](https://github.com/apache/hive/blob/branch-2.3/ql/src/java/org/apache/hadoop/hive/ql/exec/MapOperator.java#L419) is transfer to a Map<TableName, Conf>, two table's conf (contains hive.io.file.readcolumn.names) is same here. that's why get [selectedColumns](https://github.com/apache/iceberg/blob/0.11.x/mr/src/main/java/org/apache/iceberg/mr/hive/HiveIcebergSerDe.java#L92) wrong. By reading the code there `cloneConfsForNestedColPruning(hconf);` creates a map for every table, and every conf should contain the pruning info for the specific table. See: ``` /** * For each source table, combine the nested column pruning information from all its * table scan descriptors and set it in a configuration copy. This is necessary since * the configuration property "READ_NESTED_COLUMN_PATH_CONF_STR" is set on a per-table * basis, so we can't just use a single configuration for all the tables. */ private Map<String, Configuration> cloneConfsForNestedColPruning(Configuration hconf) { ``` So you have found that the configs are not correctly set by this method and they contain the same pruning information for both tables? Thanks, Peter -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
