This is an automated email from the ASF dual-hosted git repository.
jiacai2050 pushed a change to branch main
in repository https://gitbox.apache.org/repos/asf/incubator-horaedb.git
from 96ae4205 chore: enable merge on github (#1435)
new 22b724ca fix: ensure channel size non zero (#1345)
new 46f54f4d feat: use string for request id (#1349)
new 9f9079f5 chore: update create table integration test result (#1344)
new a498083b fix: fix create table result (#1354)
new f164cdaa Revert "fix: fix create table result" (#1355)
new b5265fad fix: fix test create table result (#1357)
new fc76a795 chore: disable frequently failed tests (#1352)
new 281111af chore: ignore flush failure when flush (#1362)
new 65130f59 feat: support metrics for number of bytes fetched from object
storage (#1363)
new 18a59b61 fix: collect metrics for `get_ranges` (#1364)
new 1d8593fc chore: disable timeout for http api (#1367)
new b5bfb2c1 fix: ignore collecting fetched bytes stats when sst file is
read only once (#1369)
new f41ad897 chore: disable block for http api (#1368)
new 785eed72 feat: avoid building dictionary for massive unique column
values (#1365)
new c5c01af6 refactor: avoid duplicate codes (#1371)
new 9619810a feat: utilize the column cardinality for deciding whether to
do dict (#1372)
new 3f5d8f45 fix: no write stall (#1388)
new da4e7eae fix: missing and verbose logs (#1398)
new 4abc7649 feat: avoid pulling unnecessary columns when querying append
mode table (#1307)
new bb4db609 chore: refactor for better readability (#1400)
new 3359a9a1 fix: skip wal encoding when data wal is disabled (#1401)
new 9974944a chore: add error log for remote server (#1407)
new b3fd4591 fix: disable percentile for distributed tables (#1406)
new b02bac2b feat: dist sql analyze (#1260)
new ce4044b8 feat: impl priority runtime for read (#1303)
new 61b123ab refactor: avoid returning metrics in non-analyze sql (#1410)
new 904e2d5e feat: block rules support query (#1420)
new 94673bd1 chore: fix error message (#1412)
new 5377dfd5 feat: try load page indexes (#1425)
new dc10253f feat: add table status check (#1418)
new 53df95ac feat: impl layered memtable to reduce duplicated encode
during scan (#1271)
new 2e900eaf chore: skip wal seq check when wal is disabled (#1430)
new 6cbf8c4a feat: update disk cache in another thread to avoid blocking
normal query process (#1431)
new 74c56412 fix: compatible for old table options (#1432)
new 56e7d6d2 Merge branch 'dev' into merge-dev-to-main
new a7fe3a40 fix: invalid license header
new f5b6b4f6 fix: upgrade the horaedb proto to latest version
new 8675544e fix: update the integration test results
new 7b7eb1bc Merge pull request #1423 from ShiKaiWi/merge-dev-to-main
The 913 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.github/workflows/ci.yml | 1 +
Cargo.lock | 41 +-
Cargo.toml | 11 +-
analytic_engine/Cargo.toml | 1 +
analytic_engine/src/compaction/scheduler.rs | 7 +-
analytic_engine/src/instance/close.rs | 45 +-
analytic_engine/src/instance/engine.rs | 8 +-
analytic_engine/src/instance/flush_compaction.rs | 314 +++++++--
analytic_engine/src/instance/mod.rs | 74 ++-
analytic_engine/src/instance/open.rs | 56 +-
analytic_engine/src/instance/read.rs | 70 +-
analytic_engine/src/instance/reorder_memtable.rs | 16 +-
analytic_engine/src/instance/serial_executor.rs | 9 +-
analytic_engine/src/instance/wal_replayer.rs | 3 +-
analytic_engine/src/instance/write.rs | 57 +-
analytic_engine/src/lib.rs | 16 +-
analytic_engine/src/manifest/details.rs | 4 +-
analytic_engine/src/memtable/columnar/factory.rs | 5 +-
analytic_engine/src/memtable/columnar/iter.rs | 46 +-
analytic_engine/src/memtable/factory.rs | 9 +-
.../src/memtable/{skiplist => layered}/factory.rs | 41 +-
analytic_engine/src/memtable/layered/iter.rs | 123 ++++
analytic_engine/src/memtable/layered/mod.rs | 732 +++++++++++++++++++++
analytic_engine/src/memtable/mod.rs | 81 ++-
analytic_engine/src/memtable/reversed_iter.rs | 10 +-
analytic_engine/src/memtable/skiplist/factory.rs | 4 +-
analytic_engine/src/memtable/skiplist/iter.rs | 32 +-
analytic_engine/src/memtable/skiplist/mod.rs | 155 +++--
.../src/{context.rs => memtable/test_util.rs} | 37 +-
analytic_engine/src/row_iter/chain.rs | 58 +-
analytic_engine/src/row_iter/dedup.rs | 40 +-
analytic_engine/src/row_iter/merge.rs | 89 ++-
analytic_engine/src/row_iter/mod.rs | 8 +-
.../src/row_iter/record_batch_stream.rs | 97 +--
analytic_engine/src/row_iter/tests.rs | 37 +-
analytic_engine/src/setup.rs | 1 +
analytic_engine/src/sst/factory.rs | 39 +-
analytic_engine/src/sst/meta_data/cache.rs | 2 +-
.../src/sst/meta_data/metadata_reader.rs | 5 +-
analytic_engine/src/sst/meta_data/mod.rs | 11 +-
analytic_engine/src/sst/metrics.rs | 33 +-
analytic_engine/src/sst/parquet/async_reader.rs | 146 ++--
analytic_engine/src/sst/parquet/encoding.rs | 54 +-
analytic_engine/src/sst/parquet/meta_data.rs | 10 +-
.../src/sst/parquet/row_group_pruner.rs | 3 +-
analytic_engine/src/sst/parquet/writer.rs | 443 ++++++++++---
analytic_engine/src/sst/reader.rs | 6 +-
analytic_engine/src/sst/writer.rs | 7 +-
analytic_engine/src/table/data.rs | 59 +-
analytic_engine/src/table/metrics.rs | 16 +-
analytic_engine/src/table/mod.rs | 2 +
analytic_engine/src/table_meta_set_impl.rs | 1 +
analytic_engine/src/table_options.rs | 66 +-
analytic_engine/src/tests/read_write_test.rs | 3 +-
analytic_engine/src/tests/table.rs | 1 +
analytic_engine/src/tests/util.rs | 5 +-
benchmarks/src/merge_memtable_bench.rs | 49 +-
benchmarks/src/merge_sst_bench.rs | 52 +-
benchmarks/src/scan_memtable_bench.rs | 12 +-
benchmarks/src/sst_bench.rs | 55 +-
benchmarks/src/sst_tools.rs | 45 +-
benchmarks/src/util.rs | 9 +-
catalog/src/schema.rs | 13 +-
catalog_impls/src/cluster_based.rs | 118 ++++
catalog_impls/src/lib.rs | 1 +
catalog_impls/src/volatile.rs | 20 +-
cluster/src/cluster_impl.rs | 21 +-
cluster/src/lib.rs | 24 +-
cluster/src/shard_set.rs | 10 +
common_types/Cargo.toml | 1 +
common_types/src/lib.rs | 1 +
common_types/src/projected_schema.rs | 341 ++++++----
common_types/src/record_batch.rs | 263 ++++----
common_types/src/request_id.rs | 40 +-
common_types/src/row/contiguous.rs | 25 +-
common_types/src/row/mod.rs | 24 +-
common_types/src/schema.rs | 13 +-
common_types/src/tests.rs | 15 +-
common_types/src/time.rs | 7 +
components/codec/src/columnar/timestamp.rs | 6 +-
components/id_allocator/src/lib.rs | 2 +-
components/logger/Cargo.toml | 1 +
components/logger/src/lib.rs | 14 +-
components/object_store/src/disk_cache.rs | 701 +++++++++++---------
components/parquet_ext/src/meta_data.rs | 4 +-
components/parquet_ext/src/reader.rs | 92 ++-
components/runtime/src/lib.rs | 4 +-
components/runtime/src/priority_runtime.rs | 101 +++
df_engine_extensions/Cargo.toml | 1 +
df_engine_extensions/src/dist_sql_query/mod.rs | 29 +-
.../src/dist_sql_query/physical_plan.rs | 64 +-
.../src/dist_sql_query/resolver.rs | 20 +-
.../src/dist_sql_query/test_util.rs | 15 +-
docs/example-cluster-1.toml | 2 +-
.../cases/common/dml/issue-1087.result | 4 +-
.../cases/common/dml/issue-341.result | 8 +-
integration_tests/cases/common/dml/issue-59.result | 2 +-
.../cases/common/explain/explain.result | 2 +-
.../cases/common/optimizer/optimizer.result | 2 +-
.../cases/env/cluster/ddl/create_tables.result | 8 +-
.../cases/env/cluster/ddl/partition_table.result | 37 +-
.../cases/env/cluster/ddl/partition_table.sql | 21 +
.../cases/env/local/ddl/query-plan.result | 72 +-
.../cases/env/local/ddl/query-plan.sql | 42 ++
interpreters/Cargo.toml | 3 +
interpreters/src/context.rs | 24 +-
interpreters/src/factory.rs | 14 +-
interpreters/src/lib.rs | 4 +-
.../src/metrics.rs | 8 +-
interpreters/src/select.rs | 108 ++-
interpreters/src/table_manipulator/meta_based.rs | 2 +-
interpreters/src/tests.rs | 25 +-
meta_client/src/types.rs | 5 +
partition_table_engine/src/scan_builder.rs | 2 +-
proxy/src/grpc/prom_query.rs | 4 +-
proxy/src/grpc/sql_query.rs | 2 +
proxy/src/http/prom.rs | 12 +-
proxy/src/http/sql.rs | 1 +
proxy/src/influxdb/mod.rs | 10 +-
proxy/src/instance.rs | 2 +
proxy/src/lib.rs | 97 +--
proxy/src/limiter.rs | 44 +-
proxy/src/metrics.rs | 6 +
proxy/src/read.rs | 75 ++-
proxy/src/write.rs | 38 +-
query_engine/Cargo.toml | 1 +
query_engine/src/config.rs | 3 +
query_engine/src/context.rs | 2 +
query_engine/src/datafusion_impl/executor.rs | 14 +-
.../src/datafusion_impl/logical_optimizer/mod.rs | 22 -
query_engine/src/datafusion_impl/mod.rs | 77 +--
query_engine/src/datafusion_impl/physical_plan.rs | 4 +-
.../src/datafusion_impl/physical_planner.rs | 45 +-
query_engine/src/datafusion_impl/task_context.rs | 24 +-
query_engine/src/executor.rs | 4 +-
query_engine/src/physical_planner.rs | 4 +-
query_frontend/Cargo.toml | 3 +
query_frontend/src/frontend.rs | 85 +--
query_frontend/src/influxql/planner.rs | 13 +-
query_frontend/src/lib.rs | 1 +
query_frontend/src/logical_optimizer/mod.rs | 52 ++
.../src}/logical_optimizer/type_conversion.rs | 5 +-
query_frontend/src/plan.rs | 236 ++++++-
query_frontend/src/planner.rs | 15 +-
query_frontend/src/promql/convert.rs | 3 +-
query_frontend/src/promql/remote.rs | 16 +-
remote_engine_client/src/client.rs | 153 +++--
remote_engine_client/src/lib.rs | 5 +-
router/src/cluster_based.rs | 5 +
.../src/grpc/remote_engine_service}/metrics.rs | 8 +-
server/src/grpc/remote_engine_service/mod.rs | 324 ++++++---
server/src/http.rs | 9 +-
server/src/server.rs | 3 +
src/horaedb/src/config.rs | 5 +-
src/horaedb/src/setup.rs | 39 +-
src/wal/src/message_queue_impl/region.rs | 2 +-
src/wal/src/rocksdb_impl/manager.rs | 2 +-
system_catalog/src/sys_catalog_table.rs | 6 +
system_catalog/src/tables.rs | 26 +-
table_engine/src/engine.rs | 4 +-
table_engine/src/partition/rule/key.rs | 8 +-
table_engine/src/partition/rule/random.rs | 7 +-
table_engine/src/provider.rs | 97 ++-
table_engine/src/remote/model.rs | 35 +-
table_engine/src/table.rs | 9 +-
tools/src/bin/sst-convert.rs | 13 +-
166 files changed, 5394 insertions(+), 2144 deletions(-)
copy analytic_engine/src/memtable/{skiplist => layered}/factory.rs (58%)
create mode 100644 analytic_engine/src/memtable/layered/iter.rs
create mode 100644 analytic_engine/src/memtable/layered/mod.rs
copy analytic_engine/src/{context.rs => memtable/test_util.rs} (57%)
create mode 100644 catalog_impls/src/cluster_based.rs
create mode 100644 components/runtime/src/priority_runtime.rs
copy {df_engine_extensions => interpreters}/src/metrics.rs (84%)
delete mode 100644 query_engine/src/datafusion_impl/logical_optimizer/mod.rs
create mode 100644 query_frontend/src/logical_optimizer/mod.rs
rename {query_engine/src/datafusion_impl =>
query_frontend/src}/logical_optimizer/type_conversion.rs (98%)
copy {df_engine_extensions/src =>
server/src/grpc/remote_engine_service}/metrics.rs (82%)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
