wangzhigang1999 opened a new pull request, #10054:
URL: https://github.com/apache/paimon/pull/10054
### Purpose
Fixes #10053.
PyPaimon can return the wrong MAP value when projecting a key that starts
with a dot. For a MAP containing `foo=100` and `.foo=107`, selecting both keys
returns `100, 100`. This change resolves struct children by their literal names
before passing field indexes to Arrow, so the read returns `100, 107` and
preserves parent NULLs.
The change also fixes two projection cases:
- Try MAP field-name prefixes from longest to shortest until the remaining
selector parses as a string key. An invalid longer prefix no longer hides a
valid selector for a shorter field name.
- Propagate nullability through ROW ancestors when building the projected
schema. Copy the leaf type when widening its nullability so the source schema
retains its constraints.
Exact top-level field matches and the existing ROW prefix precedence remain
unchanged. The ROW path walker is extracted into a helper, and projection path
annotations now include `MapKey` steps.
### Tests
Added four regression tests, all of which fail before the fix:
- Read dot-prefixed MAP keys alone and alongside an ordinary key, including
empty and NULL maps, with Parquet and row files.
- Read the same MAP key with either quote style when a longer field-name
prefix cannot form a valid selector; retain valid longer-prefix selection and
invalid-selector skipping.
- Project a required leaf beneath a nullable ROW, then write the result to
Parquet and read it back.
- Verify ancestor and leaf nullability combinations, including unchanged
source schema constraints and fully non-null paths.
The following local test scope passed: **102 passed** (Python 3.12.6,
PyArrow 19.0.1):
```sh
cd paimon-python
python -m pytest -q \
pypaimon/tests/test_read_builder_nested_projection.py \
pypaimon/tests/test_nested_projection_e2e.py \
pypaimon/tests/map_selected_key_projection_test.py \
pypaimon/tests/test_projection_utility.py \
pypaimon/tests/test_outer_projection_record_reader.py \
pypaimon/tests/projection_predicate_index_test.py
```
Ruff formatting checks passed for the changed ranges, Ruff lint passed for
the six changed Python files, and `git diff --check` passed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
