This is an automated email from the ASF dual-hosted git repository.
jianliangqi pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
from d96e31c4d7 [opt](Nereids) not push down global limit to avoid early
gather (#21891)
add fc2b9db0ad [Feature](inverted index) add tokenize function for
inverted index (#21813)
No new revisions were added by this update.
Summary of changes:
be/src/vec/functions/function_tokenize.cpp | 141 +++++++++++++++++++++
.../{functions_geo.h => function_tokenize.h} | 70 ++++------
be/src/vec/functions/simple_function_factory.h | 2 +
.../doris/catalog/BuiltinScalarFunctions.java | 2 +
.../scalar/{SplitByChar.java => Tokenize.java} | 14 +-
.../expressions/visitor/ScalarFunctionVisitor.java | 5 +
gensrc/script/doris_builtins_functions.py | 4 +
.../data/inverted_index_p0/test_tokenize.out | 15 +++
.../suites/inverted_index_p0/test_tokenize.groovy | 73 +++++++++++
9 files changed, 277 insertions(+), 49 deletions(-)
create mode 100644 be/src/vec/functions/function_tokenize.cpp
copy be/src/vec/functions/{functions_geo.h => function_tokenize.h} (53%)
copy
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/expressions/functions/scalar/{SplitByChar.java
=> Tokenize.java} (84%)
create mode 100644 regression-test/data/inverted_index_p0/test_tokenize.out
create mode 100644
regression-test/suites/inverted_index_p0/test_tokenize.groovy
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
