parthchandra commented on PR #5331: URL: https://github.com/apache/datafusion-comet/pull/5331#issuecomment-5446647416
> `ocs/source/user-guide/latest/iceberg.md`. The "Current limitations" list is where people look to find out why a scan fell back, and this PR adds three new entries to it: a transform sort key, a UUID sort key, and sort-merge disabled while Iceberg still reports an ordering. Could we add those? The `### Tuning` section above already documents `dataFileConcurrencyLimit` and looks like the right home for `sortMerge.enabled` and `sortMerge.maxFilesPerPartition`, including the note that this only does anything with Iceberg's `spark.sql.iceberg.planning.preserve-data-ordering` turned on. `configs.md` is generated so that one is fine. Added the transform-sort-key and UUID-sort-key fallbacks to "Current limitations", and documented `sortMerge.enabled + sortMerge.maxFilesPerPartition` under Tuning, including the `preserve-data-ordering `pre-requisite -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
