JingsongLi opened a new pull request, #53:
URL: https://github.com/apache/paimon-mosaic/pull/53
## Summary
Add MAP type support using the same ORC-style flattened columnar storage as
ARRAY. A `MAP<K, V>` column is decomposed into three physical columns within
the same bucket: lengths (INT32), keys (type K), and values (type V). All three
independently benefit from DICT/CONST/PLAIN encoding.
Fully backward compatible — no version bump. MAP uses type byte 19. Older
readers reject at schema validation.
## Storage Format
```
MAP<INT, UTF8> with rows: {1:"a", 2:"b"}, null, {3:"c"}
Lengths (INT32): 2, null, 1 ← DICT/CONST encoding
Keys (INT32): 1, 2, 3 ← shared DICT across all maps
Values (UTF8): "a", "b", "c" ← shared DICT across all maps
```
## Changes
- **core/types.rs**: Type byte 19, validate, serialize/deserialize MAP
- **core/bucket_writer.rs**: `expand_col_types` for MAP (3 children),
`extract_map_lengths`, `flatten_map_entries`
- **core/bucket_reader.rs**: `reassemble_map_array` from physical columns
- **docs/design.html**: MAP Type Storage section, TypeDescriptor table,
Limitations update
- **docs/java-api.html, python-api.html, cpp-api.html**: MAP API examples
- **Golden file**: `v1_with_map.mosaic` binary compatibility test
## Test Plan
- [x] Rust: 3 MAP tests (basic, all-null, mixed with ARRAY) + golden file
tests
- [x] Java: 1 MAP test (`testMapType`)
- [x] Python: 2 MAP tests (basic, null values)
- [x] C++: 1 MAP test
- [x] All existing ARRAY tests pass unchanged
- [x] Golden files: v1_no_array + v1_with_array unchanged
🤖 Generated with [Claude Code](https://claude.com/claude-code)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
