lxy-9602 opened a new pull request, #8332:
URL: https://github.com/apache/paimon/pull/8332
### Purpose
This is a sub-PR for MAP shared-shredding write support. The full
shared-shredding write path will be wired in later PRs.
- Add `MapSharedShreddingContext` to maintain cross-file shared-shredding
state and compute the next physical column width `K`.
- Add `MapSharedShreddingFieldDict` to assign stable field ids for MAP keys
within a shared-shredding field.
- Add `MapSharedShreddingColumnAllocator` to allocate MAP entries into
physical shared columns for each row.
- This is currently a simple temporary implementation based on row order.
- A later version will support a more sophisticated LRU-style allocation
strategy.
- Add `MapSharedShreddingRowConverter` to lazily wrap logical `InternalRow`
records and convert shared-shredding MAP fields into their physical
representation only when those fields are accessed.
Related design:
https://cwiki.apache.org/confluence/display/PAIMON/PIP-43%3A+Columnar+Storage+Optimization+for+MAP+Type+in+Paimon
### Tests
```bash
mvn -pl paimon-common -am -Pfast-build -DfailIfNoTests=false \
-Dtest=MapSharedShreddingColumnAllocatorTest,MapSharedShreddingContextTest,MapSharedShreddingFieldDictTest,MapSharedShreddingRowConverterTest
test
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
