twuebi commented on PR #1392: URL: https://github.com/apache/iceberg-go/pull/1392#issuecomment-4892360461
Thanks for bringing these up, going over them one by one, though, I still don't see a case where the split path helps: - **Malformed / non-compliant manifest lists**, if the metadata is already spec-violating, preserving lineage derived from one half of a broken manifest list seems riskier than the current conservative warn-and-drop. Garbage in shouldn't become half-trusted lineage out. - **Custom remote scan planning / custom task serialization**, those are callers constructing invalid `FileScanTask`s. Not sure if the library should grow a compaction-splitting path to compensate for that. The in-repo codec does round-trip `FirstRowID` (`codec/file_scan_task.go`), so task serialization within this repo isn't affected. - **Stale plan across a v2→v3 upgrade race**, don't *all* tasks have nil `FirstRowID` in this scenario, so the group is homogeneous legacy and never enters the new split path? So the mixed case remains reachable only via broken metadata or caller bugs, and for both the existing gate + warning feels like the right response, especially given the split's cost of turning one compaction group into two output streams. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
