nssalian commented on PR #16614: URL: https://github.com/apache/iceberg/pull/16614#issuecomment-4898967381
Checked the other readers - `RowDataReader` and `BatchDataReader` are fine, so it's just these two. Both `ChangelogRowReader` and `PositionDeletesRowReader` hit the 6-arg BaseRowReader ctor, so the props default to empty and `vectored.io.enabled=false` gets dropped on changelog/.position_deletes scans. Same idea over in BaseDeleteLoader.openDeletes too - it only applies the map when format == PARQUET. Separately, +1 on not making table properties a source. Only Spark actually reads the key into the Parquet builder here - Flink's `RowDataFileScanTaskReader` builds the reader without passing table props through, so it'd just ignore `parquet.hadoop.vectored.io.enabled`. So you'd have it sitting in shared metadata with only Spark respecting it. Feels like the wrong place for a per-environment setting, and it also skips the SparkConfParser chain the rest of SparkReadConf uses. If we keep it, going through `confParser.tableProperty(...)` would at least keep it consistent. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
