szehon-ho opened a new issue, #5343: URL: https://github.com/apache/iceberg/issues/5343
#4011 allowed table properties to be set on new tables via catalog properties. It would be nice to have these propagate to runtime table properties as well for read/write time. Use case: There are a lot of table properties that should be overriden at runtime, see : https://iceberg.apache.org/docs/latest/configuration/ . For example, a lot of write-distribution-mode defaults to hash which cause unexpected shuffles (see https://github.com/apache/iceberg/issues/5224). Or users would like to override vectorization or read/write split size dynamically. But, in spark, spark.sql() cannot take any options, so users are stuck setting table properties for this. This could be one way without changing Spark, for Iceberg to be able to override table properties at runtime for cases where there is no other override available. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
