github-actions[bot] commented on code in PR #69190:
URL: https://github.com/apache/doris/pull/69190#discussion_r4301562004
##########
fe/fe-core/src/main/java/org/apache/doris/catalog/Index.java:
##########
@@ -77,10 +77,13 @@ public Index(long indexId, String indexName, List<String>
columns,
this.comment = comment;
if (indexType == IndexDef.IndexType.INVERTED) {
if (this.properties != null && !this.properties.isEmpty()) {
- if
(this.properties.containsKey(InvertedIndexUtil.INVERTED_INDEX_PARSER_KEY)
- ||
this.properties.containsKey(InvertedIndexUtil.INVERTED_INDEX_PARSER_KEY_ALIAS)
- ||
this.properties.containsKey(InvertedIndexUtil.INVERTED_INDEX_ANALYZER_NAME_KEY)
- ||
this.properties.containsKey(InvertedIndexUtil.INVERTED_INDEX_NORMALIZER_NAME_KEY))
{
+ String normalizerName = this.properties.get(
+ InvertedIndexUtil.INVERTED_INDEX_NORMALIZER_NAME_KEY);
+ if
(!InvertedIndexUtil.INVERTED_INDEX_PARSER_NONE.equalsIgnoreCase(getInvertedIndexParser())
+ || StringUtils.isNotEmpty(this.properties.get(
+
InvertedIndexUtil.INVERTED_INDEX_ANALYZER_NAME_KEY))
+ || (StringUtils.isNotEmpty(normalizerName)
+ &&
!"lowercase".equalsIgnoreCase(normalizerName))) {
Review Comment:
[P2] Apply custom-normalizer defaults before copying ADD INDEX metadata
On an upgraded catalog retaining a custom normalizer named `LowerCase` with
a splitting filter such as `word_delimiter`, `CREATE INDEX ...
PROPERTIES("normalizer"="LowerCase")` (also `ALTER TABLE ... ADD INDEX`) now
persists an index without `support_phrase`. `CreateIndexOp.validate()`
constructs this `Index` after only `resolvePolicyNames()`; later
`SchemaChangeHandler.processAddIndex()` runs `checkColumn()` and adds `"true"`
to the separate definition map, but copies only the columns back to
`alterIndex`. The policy-aware default in `normalizeInvertedIndexProperties()`
therefore never reaches the catalog copy. BE dispatches the exact custom
policy, so `foo-bar` emits multiple terms, yet the writer omits positions and
`is_normalizer_only()` permits multi-term phrase queries to request positions
that were never written. Resolve the actual binding and apply its default
before copying the catalog metadata, or synchronize the validated properties
afterward, and cover this ADD INDEX r
oute with a retained mixed-case custom policy.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
