JingsongLi commented on code in PR #10898: URL: https://github.com/apache/paimon/pull/10898#discussion_r4435381554
########## paimon-python/pypaimon/read/fallback_read_builder.py: ########## @@ -0,0 +1,142 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. + +"""Batch read builder honouring the ``scan.fallback-branch`` table option. + +Mirrors Java's ``org.apache.paimon.table.FallbackReadFileStoreTable``: +partitions that exist in the main branch are read from main; partitions +that exist only in the fallback branch are read from the fallback branch. +All branches of a table share the same physical data files (only snapshot, +tag and schema metadata are branch scoped), and the fallback branch must +have the same row type as main, so a split planned on either branch can be +read with a single :class:`~pypaimon.read.table_read.TableRead`. +""" + +from typing import Dict, List, TYPE_CHECKING, Union + +from pypaimon.read.plan import Plan +from pypaimon.read.read_builder import ReadBuilder + + +if TYPE_CHECKING: + from pypaimon.read.split import Split + from pypaimon.read.table_read import TableRead + from pypaimon.read.table_scan import TableScan + from pypaimon.table.file_store_table import FileStoreTable + + +def _partition_key(split: "Split") -> tuple: + """Return a hashable, branch-independent key for a split's partition.""" + partition = split.partition + values = getattr(partition, "values", None) + if values is None: + return (partition,) + return tuple(values) + + +class FallbackReadBuilder(ReadBuilder): + """A :class:`ReadBuilder` that merges main and fallback-branch partitions. + + Configuration (filter, projection, read type, limit) is applied to both + the main and the fallback builders so the fallback scan reads the same + columns and rows as main. + """ + + def __init__(self, table: "FileStoreTable", fallback_branch: str): + super().__init__(table) + self._fallback_table = table.copy({"branch": fallback_branch}) Review Comment: Withdrawn finding: unsupported branch-policy fixture I am withdrawing this P1 authorization claim. The manual `AuthTableQueryResponse` hook in the reproduction bypassed the repository's policy-management validation: installing data policies/column grants on tables with branches is explicitly rejected, and branch creation rejects existing restricted policies. Consequently that fixture does not establish a supported production authorization regression. The separately reported branch-schema-history failure below is reproduced with normal filesystem catalog operations and remains valid. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
