ajantha-bhat commented on PR #16566: URL: https://github.com/apache/iceberg/pull/16566#issuecomment-4751585646
Thanks @kevinjqliu , I agree with the maintainability concern. But the win is still worth pursuing given the GH Actions pressure and PR wall-time cost, but only if the planner is conservative and easy to reason about. I revised the PR with that in mind: - the planner now fails closed: unknown non-doc/code paths, unknown Spark/Flink version paths, missing changed-file detection, and global build/workflow changes force full CI - Spark/Flink version selection is now generic from `spark/v<version>/` and `flink/v<version>/`, checked against `knownSparkVersions` / `knownFlinkVersions` in `gradle.properties` - I removed fine-grained CVE artifact selection from this PR; CVE-relevant changes now run the full CVE matrix - added planner tests for the important routing cases, including docs-only, core/file-format canaries, Spark/Flink version paths, global changes, unknown paths, and old Spark version paths - the planner writes a CI summary so the selected plan is visible in each run The intended invariant is: stale planner logic may run more CI, but should not silently under-run coverage. Open questions: - Is there a specific remaining part of the planner that you think is still too much duplicated build knowledge? - Do you have another shape in mind that would give us this level of PR matrix reduction while keeping the logic closer to Gradle? - Would you prefer splitting this into smaller PRs? My slight preference is to keep it together because the pieces are interdependent and the total change would be roughly the same, but I’m happy to split if that makes review easier. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
