plusplusjiajia opened a new pull request, #9949: URL: https://github.com/apache/paimon/pull/9949
### Purpose `QueryAuthSplit` carries the row filter and the column masks. It implements `Split` but is not a `DataSplit`, and `AbstractDataTableScan#plan()` wraps every split once a table has rules, so the Flink source's casts fail on any table with a row filter or a column mask. #9929 fixed this class of bug inside `paimon-core` and added `QueryAuthSplit.unwrap` as the single place to look through the wrapper; this is the same fix for the Flink source. Each site had to answer the same question: does it read the underlying split, or does it hand the split on to a read? The ones that only read it now unwrap, and the element they route, assign or key on stays the original wrapper — unwrapping something that is passed downstream would drop the rules and return unmasked, unfiltered rows instead of failing. - `ContinuousFileSplitEnumerator#assignSuggestedTask` — a wrapper matched neither the `DataSplit` nor the `ChainSplit` branch and fell through to the `IncrementalSplit` cast. - `MonitorSource#shuffleOrdered` — the key selector reads the partition and bucket. This one shuffles ahead of `ReadOperator`, so fixing the operator alone never gets reached. - `AlignedSplitAssigner` — four sites read `snapshotId()` for alignment, two of them inside a `checkArgument`. Its `instanceof PlaceholderSplit` was reading the wrapper too, which is the quieter half: a wrapped placeholder took the wrong branch rather than failing. - `AlignedContinuousFileSplitEnumerator#addSplits`, `PreAssignSplitAssigner` and `DynamicPartitionPruningAssigner` — single sites reading `snapshotId()` or `partition()`. - `ReadOperator` — unwrapping alone is not enough here, since a `ChainSplit` has no earliest file creation time to report. The event time falls back to `UNDEFINED`, which every consumer of the metric already guards for, rather than failing a read over a metric it cannot compute. - `TableScanUtils#getSnapshotId` — the quiet one. It tests for `DataSplit` and answers `Optional.empty()` otherwise, so a wrapper does not fail the cast, it reports no snapshot id at all: `ContinuousFileSplitEnumerator` falls back to `nextSnapshotId` instead of the real per-subtask id, and consumer progress and the snapshot watermark stop being reported for every split of an authorized table. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
