Quanlong Huang has uploaded a new patch set (#2). ( http://gerrit.cloudera.org:8080/16908 )
Change subject: [WIP] IMPALA-2019(Part-1): Provide UTF-8 support in length, sub_string and reverse functions ...................................................................... [WIP] IMPALA-2019(Part-1): Provide UTF-8 support in length, sub_string and reverse functions Add query option, UTF8_MODE, for turning on/off the UTF-8 aware behavior. Add UTF-8 aware versions of length(), sub_string() and reverse(). Other functions will be added in later patches. Implementation: Introduces a new flag, is_utf8, in type descriptors. Pass it down from FE when analyzing FunctionCallExpr. Check it in string functions which support UTF-8 mode. Tests: - Expose the UTF-8 aware version of string functions as builtin functions (named by utf8_*). Add be tests for them. - Add e2e tests for the UTF8_MODE query option. - TODO: Add a table with utf-8 strings and use string functions on it Change-Id: I0aaf3544e89f8a3d531ad6afe056b3658b525b7c --- M be/src/exprs/anyval-util.cc M be/src/exprs/expr-test.cc M be/src/exprs/string-functions-ir.cc M be/src/exprs/string-functions.h M be/src/runtime/types.cc M be/src/runtime/types.h M be/src/service/query-options.cc M be/src/service/query-options.h M be/src/udf/udf-internal.h M be/src/udf/udf.cc M be/src/udf/udf.h M be/src/util/bit-util.h M common/function-registry/impala_functions.py M common/thrift/ImpalaInternalService.thrift M common/thrift/ImpalaService.thrift M common/thrift/Types.thrift M fe/src/main/java/org/apache/impala/analysis/Analyzer.java M fe/src/main/java/org/apache/impala/analysis/FunctionCallExpr.java M fe/src/main/java/org/apache/impala/catalog/ScalarType.java A testdata/workloads/functional-query/queries/QueryTest/utf8-string-functions.test A tests/query_test/test_utf8_strings.py 21 files changed, 335 insertions(+), 28 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/08/16908/2 -- To view, visit http://gerrit.cloudera.org:8080/16908 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I0aaf3544e89f8a3d531ad6afe056b3658b525b7c Gerrit-Change-Number: 16908 Gerrit-PatchSet: 2 Gerrit-Owner: Quanlong Huang <[email protected]>
