This is an automated email from the ASF dual-hosted git repository.
lihaopeng pushed a change to branch vectorized
in repository https://gitbox.apache.org/repos/asf/incubator-doris.git.
omit ba09d3d [Vectorized][Bug] Bitmap/HLL type no support cast to
varchar/char (#7737)
omit 881b15d [Vectorized][Feature] upport function conv (#7693)
omit 1ac2205 [Vectorized][Bug] Fix get wrong result when select random
column && fix get wrong has_null_tag (#7728)
omit 1759125 [Vectorized][Enhancement] use simd to speed up coalesce and
if_not_null function (#7722)
omit cc1094b [Vectorized][Enhancement] fix some bug & improve some code
(#7714)
omit ca35232 [Vectorized][Bug] fix 'negative' function ut run fail && fix
testIsBucketShuffleJoin run fail && fix some compile fail (#7688)
omit 5b0997b [Vectorized] (olap) Optimize BlockReader's performance (#7642)
omit a7ba883 [Feature][Vectorized] Support String in vec exe engine (#7670)
omit 9bc346f [Vectorized] [Function] Support do not fold constant at
vectorized (#7668)
omit f8bf00f [Vectorized] Support bloom filter predicate on vectorized
engine storage layer (#7557)
omit 4ec9f0c [vectorized] [block] Add new method get_data_type to avoid
unnecessary copy by the method get_data_type (#7600)
omit 55a6a80 [Vectorized][Feature] support
money_format/ucase/character_length (#7649)
omit 9b3897a [Vectorizd] [Function] Add string type vec support at
doris_builtins_functions[D (#7661)
omit 3b09779 [Bug] Fix function nulllable not match and largetint cast
failed (#7659)
omit 23e1518 [Function][Vec] add function coalesce (#7632)
omit 5830710 [Vectorized][Feature] fix core dump when using function
override and function alias at the same time && support substr(str,int)
override (#7640)
omit 5229418 [Bug] Fix bug of cast expr nullable and ifnull function
(#7626)
omit 4a213f2 [Refactor] Cow refactor: giveup using boost (#7567)
omit 5d711ce [Vectorized][Function] Support function and (#7618)
omit 19249d6 [Bug] Change parser string to int (#7595)
omit 8d440c6 [Bug] Fix bug of concat function and fold const expr (#7608)
omit 27d0b07 [Function] Fix error about rank/dense_rank/row_number return
always not nullable (#7561)
omit ad71c92 [Bug] Fix negative function error result and sort node eos
(#7555)
omit e0cadc4 [Vectorized Exec Engine] Support Vectorized Exec Engine In
Doris
add b2c5f25 [docs] add more faq and FE debugging method (#7422)
add 46ca012 [fix](bloom-filter) Fix error when handle empty string in
bloom filter (#7448)
add d6cc3fd [fix](materialized-view) forbidden create materialized view
with distinct (#7494)
add d457ab3 [imporvement] remove unused method from AggregateFunction
(#7496)
add a60d86c [improvement](broker) add disable cache config for broker
(#7506)
add 7b13ac5 [deps][chore] make openssl works with old glibc version
(#7541)
add 6657524 [feature](sql-block-rule) add partition_num, tablet_num,
cardinality in SqlBlockRule to block big/slow sql (#7403)
add bf4a867 [improvement](tablet-repair) add a config repair_slow_replica
(#7423)
add 5c104ec [Improvement] use "storage_cooldown_seconds" property when
storage medium is SSD (#7532)
add 9ddcf06 [improvement](load) Transaction for load job with no data for
all partitions should be considered as normal and should not be aborted (#7240)
add 738d2d2 [refactor] update parent pom version and optimize build
scripts (#7548)
add 2a2f12c [refactor & fix](exce & olap) refactor reader: rename Reader
to TabletReader (#7544)
add e1374d8 [fix](tablet-scheduler) Fix decommission backend bug (#7563)
add 5635454 [Optimize](Runtime Filter) Support merge in runtime
filter(#7546) (#7547)
add 831f4cd [improvement](website)(proc) Make web page base proc dir and
variables orderly (#7535)
add 1f88c5f [improvement](git) add vscode devcontainer config into git
ignore (#7602)
add 90aa6c8 [fix](syntax) Add STRUCT to keywords (#7606)
add 1e0e472 [fix](audit-plugin) Fix audit load plugin may stopped when
throw unexpected exceptions (#7607)
add 482bf05 [refactor](log) remove RewriteClasses unused LOG reference
(#7609)
add ad35067 [chore][docs] add deploy spark/flink connectors to maven
release repo docs (#7616)
add 3a8a85b [Optimize][Extension] optimize extension datax
doriswriter，Remove import doris via csv in Dataxwriter, only support via json
(#7568)
add 9aaa3f6 [improvement](spark-connector) Stream load http exception
handling (#7514)
add 15d54ba [fix](error-hub) use lock to protect the creation of error
hub (#7605)
add 7254bcc [refactor](spark-connector) delete useless maven dependencies
and some code variable definition issues (#7655)
add ff4284f [feature](hint)(mysql-compatibility) Support general hints in
select statement (#7664)
add 68c87de [fix](website) fix CaseList component bug (#7683)
add 83f6eef [improvement](routine-load) Make routine load work with old
kafka version (#7630)
add 4ac8b3c [fix][s3] Fix bug that can not visit aliyun oss with aws s3
sdk (#7691)
add d418887 [comminity](github) Polish PR template (#7638)
add 6864a37 [improvement](spark-connector) Throw an exception when the
data push fails and there are too many retries (#7531)
add 8685b6b [improvement](executor) Optimize lock of client cache (#7543)
add 2cf574d [docs] Improve instructions for the configuration of BE.
(#7620)
add 1b2acb6 [docs] update the document format (#7689)
add 2de7983 [docs](hive)(function) fix Hive type error and optimize alias
function example (#7694)
add a034c20 [fix](website) Add trademarks footer on official website
(#7696)
add db26495 [docs](website) Add Database ODBC version correspondence
(#7675)
add 8ac3204 [fix](show) fix ConcurrentModificationException for show proc
'/current_queries' (#7707)
add 5e1caea [fix](lateral-view) Fix some bugs about lateral view (#7721)
add ccb6c6a [docs] update seatunnel.md (#7731)
new 07abe49 [Vectorized Exec Engine] Support Vectorized Exec Engine In
Doris
new 54bd985 [Bug] Fix negative function error result and sort node eos
(#7555)
new 204a35d [Function] Fix error about rank/dense_rank/row_number return
always not nullable (#7561)
new 2d31421 [Bug] Fix bug of concat function and fold const expr (#7608)
new 3339878 [Bug] Change parser string to int (#7595)
new 326f0d7 [Vectorized][Function] Support function and (#7618)
new fb945cd [Refactor] Cow refactor: giveup using boost (#7567)
new c4623f2 [Bug] Fix bug of cast expr nullable and ifnull function
(#7626)
new bdeb6b7 [Vectorized][Feature] fix core dump when using function
override and function alias at the same time && support substr(str,int)
override (#7640)
new 3425e8a [Function][Vec] add function coalesce (#7632)
new ead467c [Bug] Fix function nulllable not match and largetint cast
failed (#7659)
new 9432587 [Vectorizd] [Function] Add string type vec support at
doris_builtins_functions[D (#7661)
new 01d9434 [Vectorized][Feature] support
money_format/ucase/character_length (#7649)
new 77e0212 [vectorized] [block] Add new method get_data_type to avoid
unnecessary copy by the method get_data_type (#7600)
new 952f0e3 [Vectorized] Support bloom filter predicate on vectorized
engine storage layer (#7557)
new a051b33 [Vectorized] [Function] Support do not fold constant at
vectorized (#7668)
new 0dd1662 [Feature][Vectorized] Support String in vec exe engine (#7670)
new 3e45025 [Vectorized] (olap) Optimize BlockReader's performance (#7642)
new 27d3898 [Vectorized][Bug] fix 'negative' function ut run fail && fix
testIsBucketShuffleJoin run fail && fix some compile fail (#7688)
new 2c38a50 [Vectorized][Enhancement] fix some bug & improve some code
(#7714)
new 28fb8c7 [Vectorized][Enhancement] use simd to speed up coalesce and
if_not_null function (#7722)
new b79496b [Vectorized][Bug] Fix get wrong result when select random
column && fix get wrong has_null_tag (#7728)
new 2af5181 [Vectorized][Feature] upport function conv (#7693)
new e9056d6 [Vectorized][Bug] Bitmap/HLL type no support cast to
varchar/char (#7737)
new fc05698 [Vectorized] Rebase code from master
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (ba09d3d)
\
N -- N -- N refs/heads/vectorized (fc05698)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
The 25 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.github/PULL_REQUEST_TEMPLATE.md | 36 +-
.github/workflows/build-extension.yml | 16 +-
.gitignore | 1 +
README.md | 3 +-
be/src/agent/task_worker_pool.cpp | 8 +-
be/src/common/config.h | 5 +
be/src/exec/hash_join_node.cpp | 3 +-
be/src/exec/json_scanner.cpp | 6 +-
be/src/exec/olap_scan_node.cpp | 4 +-
be/src/exec/olap_scanner.cpp | 190 ++++++-----
be/src/exec/olap_scanner.h | 29 +-
be/src/exec/table_function_node.cpp | 15 +-
be/src/exprs/bloomfilter_predicate.h | 11 +-
be/src/exprs/json_functions.cpp | 9 +-
be/src/exprs/json_functions.h | 5 +-
be/src/exprs/runtime_filter.cpp | 361 ++++++++++++++++++--
be/src/exprs/runtime_filter.h | 27 +-
be/src/exprs/runtime_filter_slots.h | 51 ++-
be/src/exprs/table_function/explode_bitmap.cpp | 1 +
be/src/olap/collect_iterator.cpp | 4 +-
be/src/olap/collect_iterator.h | 10 +-
be/src/olap/delete_handler.cpp | 2 +-
be/src/olap/delete_handler.h | 4 +-
be/src/olap/merger.cpp | 2 +-
be/src/olap/reader.cpp | 106 +++---
be/src/olap/reader.h | 109 +++---
be/src/olap/row_cursor.cpp | 6 +-
be/src/olap/rowset/alpha_rowset_reader.cpp | 4 +-
be/src/olap/rowset/beta_rowset_reader.cpp | 4 +-
be/src/olap/rowset/rowset_reader_context.h | 4 +-
be/src/olap/task/engine_checksum_task.cpp | 2 +-
be/src/olap/tuple_reader.cpp | 22 +-
be/src/olap/tuple_reader.h | 11 +-
be/src/runtime/client_cache.cpp | 114 ++++---
be/src/runtime/client_cache.h | 3 +
be/src/runtime/fragment_mgr.cpp | 3 +-
be/src/runtime/routine_load/data_consumer.cpp | 1 +
be/src/runtime/runtime_filter_mgr.cpp | 21 +-
be/src/runtime/runtime_filter_mgr.h | 10 +-
be/src/runtime/runtime_state.cpp | 20 +-
be/src/runtime/runtime_state.h | 1 +
.../runtime/stream_load/stream_load_executor.cpp | 7 +-
be/src/vec/exec/join/vhash_join_node.cpp | 2 +-
be/src/vec/exec/volap_scanner.cpp | 7 +-
be/src/vec/exec/volap_scanner.h | 6 +-
be/src/vec/functions/function_binary_arithmetic.h | 2 +-
be/src/vec/olap/block_reader.cpp | 2 +-
be/src/vec/olap/block_reader.h | 4 +-
be/src/vec/olap/vcollect_iterator.cpp | 6 +-
be/src/vec/olap/vcollect_iterator.h | 14 +-
be/src/vec/runtime/vdatetime_value.cpp | 2 +-
be/src/vec/runtime/vdatetime_value.h | 2 +-
be/test/exprs/json_function_test.cpp | 35 +-
be/test/exprs/runtime_filter_test.cpp | 168 ++++++++--
be/test/vec/core/block_test.cpp | 2 +-
docs/.vuepress/components/CaseList.vue | 5 +-
docs/.vuepress/public/images/close_staging.png | Bin 0 -> 195183 bytes
docs/.vuepress/public/images/release-stage.png | Bin 0 -> 224173 bytes
docs/.vuepress/public/images/staging_repo.png | Bin 0 -> 392700 bytes
docs/.vuepress/sidebar/en.js | 4 +-
docs/.vuepress/sidebar/zh-CN.js | 2 +
docs/.vuepress/theme/components/Footer.vue | 8 +-
.../en/administrator-guide/block-rule/sql-block.md | 29 +-
docs/en/administrator-guide/config/be_config.md | 12 +-
docs/en/administrator-guide/config/fe_config.md | 6 +
.../load-data/routine-load-manual.md | 2 +-
docs/en/administrator-guide/runtime-filter.md | 3 +-
docs/en/community/how-to-deploy-to-maven.md | 191 +++++++++++
docs/en/community/security.md | 38 +++
docs/en/developer-guide/debug-tool.md | 78 ++++-
docs/en/extending-doris/datax.md | 4 +-
docs/en/extending-doris/flink-doris-connector.md | 22 +-
docs/en/extending-doris/hive-of-doris.md | 4 +-
docs/en/extending-doris/odbc-of-doris.md | 94 ++++++
docs/en/extending-doris/seatunnel.md | 9 +-
docs/en/extending-doris/spark-doris-connector.md | 29 +-
docs/en/faq/faq.md | 25 +-
docs/en/installing/install-deploy.md | 6 +-
.../Data Definition/create-function.md | 4 +-
.../sql-statements/Data Manipulation/LOAD.md | 6 +
.../administrator-guide/block-rule/sql-block.md | 33 +-
docs/zh-CN/administrator-guide/config/be_config.md | 10 +-
docs/zh-CN/administrator-guide/config/fe_config.md | 10 +
docs/zh-CN/administrator-guide/export-manual.md | 2 +-
.../load-data/routine-load-manual.md | 2 +-
docs/zh-CN/administrator-guide/runtime-filter.md | 4 +-
docs/zh-CN/community/how-to-deploy-to-maven.md | 191 +++++++++++
docs/zh-CN/community/security.md | 34 ++
docs/zh-CN/developer-guide/debug-tool.md | 79 ++++-
docs/zh-CN/downloads/downloads.md | 20 ++
docs/zh-CN/extending-doris/datax.md | 4 +-
.../zh-CN/extending-doris/flink-doris-connector.md | 28 +-
docs/zh-CN/extending-doris/hive-of-doris.md | 2 +-
docs/zh-CN/extending-doris/odbc-of-doris.md | 92 +++++
docs/zh-CN/extending-doris/seatunnel.md | 39 ++-
.../zh-CN/extending-doris/spark-doris-connector.md | 28 +-
docs/zh-CN/faq/faq.md | 20 ++
docs/zh-CN/installing/install-deploy.md | 4 +-
.../Data Definition/create-function.md | 4 +-
.../sql-statements/Data Manipulation/LOAD.md | 92 ++---
env.sh | 80 ++---
extension/DataX/doriswriter/doc/doriswriter.md | 12 -
.../plugin/writer/doriswriter/DorisCodec.java | 34 +-
.../plugin/writer/doriswriter/DorisCsvCodec.java | 52 ---
.../plugin/writer/doriswriter/DorisFlushBatch.java | 108 +++---
.../plugin/writer/doriswriter/DorisJsonCodec.java | 42 +--
.../plugin/writer/doriswriter/DorisWriter.java | 117 +++----
.../writer/doriswriter/DorisWriterEmitter.java | 67 ++--
.../datax/plugin/writer/doriswriter/Key.java | 56 +---
extension/flink-doris-connector/build.sh | 38 ++-
extension/flink-doris-connector/pom.xml | 45 ++-
extension/spark-doris-connector/build.sh | 49 +--
extension/spark-doris-connector/pom.xml | 46 ++-
extension/spark-doris-connector/pom_3.0.xml | 372 ---------------------
.../org/apache/doris/spark/DorisStreamLoad.java | 18 +-
.../apache/doris/spark/backend/BackendClient.java | 2 +-
.../java/org/apache/doris/spark/cfg/Settings.java | 2 +-
.../org/apache/doris/spark/rest/RestService.java | 6 +-
.../apache/doris/spark/serialization/Routing.java | 2 +-
.../java/org/apache/doris/spark/util/IOUtils.java | 1 -
.../doris/spark/sql/DorisSourceProvider.scala | 16 +-
.../doris/spark/sql/DorisStreamLoadSink.scala | 13 +-
.../apache/doris/spark/rest/TestRestService.java | 2 +-
.../doris/spark/serialization/TestRowBatch.java | 2 +-
fe/fe-common/pom.xml | 21 +-
fe/fe-core/pom.xml | 17 +-
fe/fe-core/src/main/cup/sql_parser.cup | 76 ++++-
.../java/org/apache/doris/alter/RollupJobV2.java | 1 +
.../doris/analysis/AlterSqlBlockRuleStmt.java | 36 +-
.../doris/analysis/CreateMaterializedViewStmt.java | 25 +-
.../doris/analysis/CreateSqlBlockRuleStmt.java | 49 ++-
.../org/apache/doris/analysis/LateralViewRef.java | 6 +-
.../doris/analysis/MVColumnBitmapUnionPattern.java | 3 +
.../doris/analysis/MVColumnHLLUnionPattern.java | 3 +
.../doris/analysis/MVColumnOneChildPattern.java | 3 +
.../java/org/apache/doris/analysis/SelectList.java | 8 +-
.../doris/analysis/ShowSqlBlockRuleStmt.java | 3 +
.../org/apache/doris/blockrule/SqlBlockRule.java | 51 ++-
.../apache/doris/blockrule/SqlBlockRuleMgr.java | 55 ++-
.../apache/doris/catalog/AggregateFunction.java | 41 ++-
.../java/org/apache/doris/catalog/Catalog.java | 4 +
.../org/apache/doris/catalog/DataProperty.java | 8 +-
.../doris/catalog/MaterializedIndexMeta.java | 1 +
.../java/org/apache/doris/catalog/Replica.java | 4 +-
.../main/java/org/apache/doris/catalog/Tablet.java | 21 +-
.../org/apache/doris/clone/TabletScheduler.java | 47 ++-
.../main/java/org/apache/doris/common/Config.java | 6 +
.../java/org/apache/doris/common/ErrorCode.java | 2 +
.../org/apache/doris/common/proc/BaseProcDir.java | 6 +-
.../org/apache/doris/common/util/SqlBlockUtil.java | 99 ++++++
.../apache/doris/http/action/VariableAction.java | 9 +-
.../load/routineload/KafkaRoutineLoadJob.java | 10 -
.../doris/load/update/UpdateStmtExecutor.java | 15 -
.../org/apache/doris/planner/OlapScanNode.java | 8 +
.../org/apache/doris/planner/RuntimeFilter.java | 2 +-
.../doris/planner/RuntimeFilterGenerator.java | 16 -
.../main/java/org/apache/doris/qe/Coordinator.java | 18 +-
.../java/org/apache/doris/qe/StmtExecutor.java | 39 +--
.../doris/rewrite/RewriteAliasFunctionRule.java | 4 -
.../doris/rewrite/RewriteEncryptKeyRule.java | 3 -
.../doris/rewrite/RewriteLikePredicateRule.java | 4 -
.../doris/transaction/DatabaseTransactionMgr.java | 11 -
.../TransactionCommitFailedException.java | 2 -
fe/fe-core/src/main/jflex/sql_scanner.flex | 1 -
.../doris/analysis/AlterSqlBlockRuleStmtTest.java | 27 ++
.../analysis/CreateMaterializedViewStmtTest.java | 96 +++++-
.../doris/analysis/CreateSqlBlockRuleStmtTest.java | 54 +++
.../analysis/MVColumnBitmapUnionPatternTest.java | 22 +-
.../analysis/MVColumnHLLUnionPatternTest.java | 22 +-
.../analysis/MVColumnOneChildPatternTest.java | 17 +-
.../org/apache/doris/analysis/SelectStmtTest.java | 29 ++
.../doris/blockrule/SqlBlockRuleMgrTest.java | 146 +++++++-
.../org/apache/doris/catalog/DataPropertyTest.java | 42 +++
.../doris/clone/TabletReplicaTooSlowTest.java | 180 ++++++++++
.../doris/load/update/UpdateStmtExecutorTest.java | 10 +
.../org/apache/doris/planner/QueryPlanTest.java | 13 +-
.../java/org/apache/doris/qe/ShowExecutorTest.java | 9 +-
fe/pom.xml | 47 ++-
fe/spark-dpp/pom.xml | 14 +-
fe_plugins/auditdemo/pom.xml | 2 +-
fe_plugins/auditloader/pom.xml | 2 +-
.../doris/plugin/audit/AuditLoaderPlugin.java | 34 +-
fe_plugins/pom.xml | 26 +-
fs_brokers/apache_hdfs_broker/pom.xml | 16 +-
.../doris/broker/hdfs/FileSystemManager.java | 3 +-
gensrc/proto/internal_service.proto | 9 +
samples/connect/java/client/pom.xml | 2 +-
thirdparty/CHANGELOG.md | 5 +
thirdparty/build-thirdparty.sh | 10 +-
thirdparty/vars.sh | 10 +-
190 files changed, 3866 insertions(+), 1683 deletions(-)
create mode 100644 docs/.vuepress/public/images/close_staging.png
create mode 100644 docs/.vuepress/public/images/release-stage.png
create mode 100644 docs/.vuepress/public/images/staging_repo.png
create mode 100644 docs/en/community/how-to-deploy-to-maven.md
create mode 100644 docs/en/community/security.md
create mode 100644 docs/zh-CN/community/how-to-deploy-to-maven.md
create mode 100644 docs/zh-CN/community/security.md
delete mode 100644
extension/DataX/doriswriter/src/main/java/com/alibaba/datax/plugin/writer/doriswriter/DorisCsvCodec.java
delete mode 100644 extension/spark-doris-connector/pom_3.0.xml
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/common/util/SqlBlockUtil.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/catalog/DataPropertyTest.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/clone/TabletReplicaTooSlowTest.java
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
