hedger9487 commented on PR #3816: URL: https://github.com/apache/iceberg-python/pull/3816#issuecomment-5449547583
Nice approach separating file planning from schema projection! I was following #2467 and pulled this down to test it out locally, and noticed a small detail with `_scan(branch=...)`: if a branch doesn't exist yet (or if there's a typo in the branch name), `if branch in self.table_metadata.refs` skips `use_ref` and silently falls back to returning the default scan on `main`. In `upsert()`, this causes it to match rows against `main` (for example, running `tbl.upsert(df, branch="new_branch")` on a table with 1 row on `main` reports `rows_updated=1`). Curious if we should fail fast here (e.g. `if branch is not None: return scan.use_ref(branch)`) so it raises `ValueError: Cannot scan unknown ref` consistently? Also, as a minor follow-up thought for the future: do you think it might eventually be worth exposing a projected schema override directly on `DataScan` (e.g. `scan.with_schema(...)` or `project_as(...)`), so operations don't need to manually orchestrate `plan_files()` with private helpers? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
