abbywh opened a new pull request, #18389: URL: https://github.com/apache/iceberg/pull/18389
Spark treats `spark.driver.maxResultSize=0` as unlimited, but Iceberg's 256 MiB planning check treats it as too small and forces local manifest planning. Allow distributed planning for zero while keeping the local path for positive limits below the threshold. Test plan: Verify data and delete planning with zero, 128 MiB, and 256 MiB limits. The focused Spark 3.5 tests, formatting, checkstyle, and module build passed. --- **AI Disclosure** - Model: GPT-6 - Platform/Tool: Codex - Human Oversight: [unknown - human to fill in] - Prompt Summary: Fix distributed planning when Spark's driver result size limit is unlimited. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
