airborne12 commented on code in PR #67537:
URL: https://github.com/apache/doris/pull/67537#discussion_r3976682959
##########
fe/fe-core/src/main/java/org/apache/doris/indexpolicy/IndexPolicyMgr.java:
##########
@@ -112,28 +110,32 @@ public void validateAnalyzerExists(String analyzerName)
throws DdlException {
if (policy.isInvalid()) {
throw new DdlException("Analyzer '" + analyzerName + "' is
invalid");
}
+ validateReferencedTokenFiltersUsableLocked(analyzerName, policy);
} finally {
readUnlock();
}
}
/**
- * Validate that {@code analyzerName} resolves to a usable analyzer graph
and report whether
- * that graph ends in a common_grams token filter. Callers need the flag
because CommonGrams
- * indexes store gram terms, which only SNII can read back.
+ * 安全网：老版本可能持久化了 BE 已不再支持的 token filter 类型（例如已删除的 common_grams）。
+ * 这类策略仍会被加载（不能让 FE 因为镜像里的一条策略起不来），但任何引用它的 analyzer
+ * 都必须在使用时被明确拒绝，而不是等到 BE 建索引/查询时才报"未知 token filter"。
*/
- public boolean validateAnalyzerUsesCommonGrams(String analyzerName) throws
DdlException {
- String normalizedName = normalizeKey(analyzerName);
- if (IndexPolicy.BUILTIN_ANALYZERS.contains(normalizedName)) {
- return false;
+ private void validateReferencedTokenFiltersUsableLocked(String
analyzerName, IndexPolicy analyzer)
Review Comment:
`common_grams` has not been deployed to production, so backward
compatibility with legacy `common_grams` configurations is not required here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
