jianguotian opened a new pull request, #9702:
URL: https://github.com/apache/paimon/pull/9702
### Purpose
Preserve specialized `ColumnarRowIterator` implementations when an index
mapping
does not change the batch.
The read path can supply an explicit full identity mapping even when all
columns
remain in their original order. `ColumnarRowIterator.mapping` currently calls
`copy` for any non-null index mapping. The base implementation of `copy`
creates a
plain `ColumnarRowIterator`, so subclasses that inherit it lose their
additional
batch capabilities despite the mapping being a no-op.
This change:
- Returns the original iterator when no partition mapping is present and the
index mapping is exactly `[0, 1, ..., batchArity - 1]`.
- Keeps the existing copy path for projections, reordering, missing columns,
and
partition injection. A shorter identity prefix is still a projection.
- Adds regression tests for specialized-iterator preservation and the
unchanged
projection, reordering, and partition-mapping behavior.
The change is confined to `paimon-common`. It does not change public API or
add a
format-specific writer or compaction bridge.
### Tests
Validated commit: `d53ff92c8db26f16ebd1dffc88520e23a528aeb5`.
- `ColumnarRowIteratorTest`: **4/4 passed on JDK 8 and JDK 11**, with no
skipped
tests.
- Regression proof: `testIdentityMappingPreservesSpecializedIterator` fails
on
the parent production implementation because the specialized iterator is
replaced by a base iterator; the other three tests pass. All four pass with
this change.
- Non-fast Maven tests passed Checkstyle and Spotless; `paimon-common`
packaging
verification with tests skipped and `git diff --check` also passed.
```bash
# Prepare the reactor dependencies for this revision.
mvn -pl paimon-common -am -Pfast-build -DskipTests install
# Run with JAVA_HOME set to JDK 8, then JDK 11.
mvn -pl paimon-common -DwildcardSuites=none -Dtest=ColumnarRowIteratorTest
test
# Packaging and configured verification checks, separately from the tests
above.
mvn -pl paimon-common -DskipTests verify
git diff --check HEAD^ HEAD
```
Supplementary downstream validation used an **implementation-equivalent
build,
not this PR commit**: real compaction retained the specialized iterator for
all
three full-identity batches, reached batch writes without generic-vector or
row
fallback, and preserved row count and content. This is functional-path
evidence,
not a performance benchmark.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
