RussellSpitzer commented on code in PR #16154:
URL: https://github.com/apache/iceberg/pull/16154#discussion_r3461043977
##########
docs/docs/spark-configuration.md:
##########
@@ -199,6 +199,8 @@ val spark = SparkSession.builder()
| spark.sql.iceberg.data-planning-mode | AUTO
| Scan planning mode for data files
(`AUTO`, `LOCAL`, `DISTRIBUTED`)
|
| spark.sql.iceberg.delete-planning-mode | AUTO
| Scan planning mode for delete
files (`AUTO`, `LOCAL`, `DISTRIBUTED`)
|
| spark.sql.iceberg.advisory-partition-size | Table default
| Advisory size (bytes) used for
writing to the Table when Spark's Adaptive Query Execution is enabled. Used to
size output files |
+| spark.sql.iceberg.split-size | Table default
| Overrides `read.split.target-size`
for scan planning. Session values are honored like read options and disable
adaptive split-size adjustment |
+| spark.sql.iceberg.split-size.<table-name> | Global session
default | Table-scoped split size
override using the fully qualified table name as a suffix
|
Review Comment:
I'm not super excited about allowing for table level defaults through the
session parameters. Feels like we are working around an issue (the inability to
set config per identifier). Are we ok with just allowing the global defautl
there?
The alternative proposed here means we are essentially adding a completely
new paradigm to parsing these config entries.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
