This is an automated email from the ASF dual-hosted git repository.
yiguolei pushed a change to branch 2.1-tmp
in repository https://gitbox.apache.org/repos/asf/doris.git
from 1a2177adb98 [Fix](test) add sync to ensure data synchronization in
test_set_operater (#32993)
new b0b5f84e40e [feature](load) support compressed JSON format data for
broker load (#30809)
new 8a291efb0c5 [improvement](spill) avoid spill if memory is enough
(#33075)
The 2 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
be/src/exec/decompressor.cpp | 97 +++++++++++++++++++--
be/src/exec/decompressor.h | 11 ++-
.../exec/partitioned_aggregation_sink_operator.cpp | 30 ++++---
.../exec/partitioned_aggregation_sink_operator.h | 3 -
.../partitioned_aggregation_source_operator.cpp | 29 +++---
.../exec/partitioned_aggregation_source_operator.h | 2 -
be/src/pipeline/exec/spill_sort_sink_operator.cpp | 43 ++++-----
be/src/pipeline/exec/spill_sort_sink_operator.h | 3 -
.../pipeline/exec/spill_sort_source_operator.cpp | 29 ++----
be/src/pipeline/exec/spill_sort_source_operator.h | 3 -
be/src/pipeline/pipeline_x/dependency.h | 2 +
be/src/util/block_compression.cpp | 2 +-
be/src/vec/common/sort/sorter.cpp | 3 +-
be/src/vec/exec/format/csv/csv_reader.cpp | 65 +-------------
be/src/vec/exec/format/json/new_json_reader.cpp | 20 ++++-
be/src/vec/exec/format/json/new_json_reader.h | 3 +
be/src/vec/spill/spill_stream.cpp | 17 +++-
.../Load/BROKER-LOAD.md | 2 +-
.../Load/BROKER-LOAD.md | 2 +-
.../org/apache/doris/datasource/FileGroupInfo.java | 4 +-
.../load_p0/broker_load/test_compress_type.out | 2 +-
.../stream_load/basic_data_by_line.json.bz2 | Bin 0 -> 3161 bytes
.../load_p0/stream_load/basic_data_by_line.json.gz | Bin 0 -> 3519 bytes
.../stream_load/basic_data_by_line.json.lz4 | Bin 0 -> 5520 bytes
.../load_p0/broker_load/test_compress_type.groovy | 24 ++++-
25 files changed, 225 insertions(+), 171 deletions(-)
create mode 100644
regression-test/data/load_p0/stream_load/basic_data_by_line.json.bz2
create mode 100644
regression-test/data/load_p0/stream_load/basic_data_by_line.json.gz
create mode 100644
regression-test/data/load_p0/stream_load/basic_data_by_line.json.lz4
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
