airborne12 opened a new pull request, #69853:
URL: https://github.com/apache/doris/pull/69853
### What problem does this PR solve?
Issue Number: None
Related PR: #69730
Problem Summary: Backport #69730 to `branch-4.2`.
A VARIANT column can declare predefined fields whose patterns overlap, for
example the glob 'a.*' and the name 'a.b'. The BE writer (variant_util.cpp)
routes a subcolumn to the first predefined field whose pattern matches its path
and builds the index of that field. `OlapTable.getInvertedIndex` looped over
all the fields without stopping and kept the last match, so a query on the same
path chose the index of a different field than the one the data was written
with, and fell back to another index or to a scan.
The loop now stops at the first match, the same order the BE uses, so a
write and a query on one path land on the same index.
#### Branch adaptations
On `branch-4.2` the predefined field's pattern type is the thrift
`TPatternType` (master has `catalog.PatternType`), and `Index` takes
`IndexDef.IndexType`. The `OlapTable.java` change applies unchanged around
those lines. The test is master's test with two adaptations: it uses
`TPatternType` and keeps the branch's own `IndexType` import, and it builds its
lists and maps with Guava's `ImmutableList.of` / `ImmutableMap.of`, like the
other tests of the file, because this branch's fe-core tests cannot use
`List.of` / `Map.of`.
The BE of `branch-4.2` also routes a subcolumn to the first matching
predefined field: `generate_sub_column_info` in `variant_util.cpp` returns on
the first match. So the FE change has the same effect on this branch. The
commit is the cherry-pick of master `7832338fba`.
### Release note
Fix the index chosen for a Variant subpath when the column declares
overlapping field patterns.
### Check List (For Author)
- Test <!-- At least one of them must be included. -->
- [ ] Regression test
- [x] Unit Test
- The test of #69730:
InvertedIndexPropertiesTest.testOverlappingFieldPatternsPickTheFirstPredefinedFieldLikeTheBe.
- [ ] Manual test (add detailed scripts or steps below)
- [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
- [ ] Previous test can cover this change.
- [ ] No code files have been changed.
- [ ] Other reason
- Behavior changed:
- [ ] No.
- [x] Yes. With overlapping field patterns, the first declared match
decides the index instead of the last.
- Does this need documentation?
- [x] No.
- [ ] Yes.
### Validation
- `InvertedIndexPropertiesTest` on this branch, locally: 28 tests pass,
including the new
`testOverlappingFieldPatternsPickTheFirstPredefinedFieldLikeTheBe`; FE
Checkstyle passes in the same build.
- With `branch-4.2`'s own `OlapTable.java` put back, the new test fails with
"expected idx_glob but was idx_exact", which is the last-match behavior this
change removes.
- The PR's CI runs the FE unit tests and the regression suites.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
