JingsongLi opened a new pull request, #9261: URL: https://github.com/apache/paimon/pull/9261
## What changed - add a reusable resolver for multi-level `ROW` field paths and leaf field IDs - support nested index columns in Flink and Spark create/drop procedures and generic/sorted index builders - extract nested values with null propagation and preserve multi-column index order - route predicate, TopN, metadata, MERGE conflict checks, and incremental refresh through nested leaf metadata - keep exact top-level dotted field names backward compatible ## Why Global indexes previously resolved index columns only against top-level fields. Paths such as `profile.zip` could not be built or matched consistently, and parent `ROW` updates could leave nested indexes stale. ## Impact Global indexes can now target nested `ROW` fields such as `profile.zip`, including multi-level paths and nullable parent rows. `ARRAY` and `MAP` traversal remain out of scope. ## Validation - non-fast focused Common/Core tests passed for path resolution, predicate/TopN evaluation, metadata, extraction, scanners/writers, and incremental refresh - non-fast Flink tests passed, including nested full-text index creation and sorted topology coverage - non-fast Spark common test and Spark 3 compilation passed - `git diff --check` passed The nested Flink BTree E2E reaches the sorted build job but is blocked locally by a CodeGenerator `PluginLoader` NPE. The existing top-level BTree E2E reproduces the identical failure in this environment; the nested flattening path is covered by the passing sorted topology tests. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
