This is an automated email from the ASF dual-hosted git repository.
spetz pushed a change to branch feat/server-ng-perf
in repository https://gitbox.apache.org/repos/asf/iggy.git
from b691d33c4 perf(server-ng): compute batch checksum in a single produce
pass
add ddead57af fix(ci): make README CLI preflight execute its commands
(#3792)
add e2bbaea1c ci(rust): skip unnecessary setup steps (#3749)
add a684e550f feat(python): add user header and origin timestamp support
(#3613)
add 3ca8695d3 feat(connectors): add Meilisearch sink connector (#3497)
add 54d8a49dd feat(server-ng): close the partition materialisation race at
the owner (#3786)
add 198364f22 chore(deps): Bump the web group across 1 directory with 16
updates (#3807)
add 4199fed86 chore(deps): Bump fast-uri from 3.1.4 to 3.1.5 in
/foreign/node (#3812)
add e4e25648b chore(deps): Bump the csharp group with 1 update (#3810)
add b452d150d fix(node): TLS + VSR connection support (#3813)
add c51b75456 perf(server-ng): cap sealed-segment read handles with
per-partition LRU (#3806)
new 15cebf53c Merge branch 'master' into feat/server-ng-perf
The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.github/actions/rust/pre-merge/action.yml | 6 +
.../actions/utils/setup-rust-with-cache/action.yml | 14 +-
.github/workflows/coverage-baseline.yml | 1 +
Cargo.lock | 90 ++
Cargo.toml | 6 +
core/connectors/README.md | 1 +
core/connectors/sinks/README.md | 1 +
.../Cargo.toml | 14 +-
core/connectors/sinks/meilisearch_sink/README.md | 105 ++
.../config.toml | 18 +-
core/connectors/sinks/meilisearch_sink/src/lib.rs | 1412 ++++++++++++++++++
core/integration/tests/cluster/mod.rs | 1 +
.../cluster/multi_shard_partition_convergence.rs | 223 +++
.../connectors/fixtures/meilisearch/container.rs | 180 +++
.../fixtures/{quickwit => meilisearch}/mod.rs | 4 +-
.../tests/connectors/fixtures/meilisearch/sink.rs | 90 ++
core/integration/tests/connectors/fixtures/mod.rs | 2 +
.../connectors/meilisearch/meilisearch_sink.rs | 81 ++
.../tests/connectors/{delta => meilisearch}/mod.rs | 2 +-
.../connectors/{delta => meilisearch}/sink.toml | 2 +-
core/integration/tests/connectors/mod.rs | 1 +
core/metadata/src/stm/stream.rs | 16 +-
core/partitions/src/iggy_index_reader.rs | 180 ++-
core/partitions/src/iggy_partition.rs | 508 ++++++-
core/partitions/src/iggy_partitions.rs | 15 +-
core/partitions/src/log.rs | 341 ++++-
core/partitions/src/poll_plan.rs | 212 ++-
core/sdk/src/prelude.rs | 6 +-
core/server-ng/Cargo.toml | 7 +
core/server-ng/config.toml | 3 +-
core/server-ng/src/dispatch.rs | 76 +-
core/server-ng/src/partition_reconciler.rs | 1531 +++++++++++++++++++-
core/shard/src/lib.rs | 835 ++++++++++-
core/shard/src/metrics.rs | 139 +-
core/shard/src/router.rs | 40 +-
core/simulator/src/lib.rs | 17 +
examples/java/README.md | 2 +-
examples/node/README.md | 2 +-
examples/python/README.md | 23 +
examples/python/basic/producer.py | 50 +-
examples/python/getting-started/producer.py | 50 +-
examples/python/message-headers/common.py | 240 +++
.../message-headers/plain-headers/consumer.py | 77 +
.../message-headers/plain-headers/producer.py | 59 +
.../message-headers/typed-headers/consumer.py | 77 +
.../message-headers/typed-headers/producer.py | 64 +
examples/python/pyproject.toml | 5 +
examples/rust/README.md | 2 +-
examples/rust/src/tcp-tls/consumer/main.rs | 2 +-
examples/rust/src/tcp-tls/producer/main.rs | 2 +-
foreign/csharp/Directory.Packages.props | 2 +-
foreign/node/README.md | 6 +-
foreign/node/package-lock.json | 655 ++++-----
foreign/node/package.json | 2 +-
foreign/node/src/client/client.config.test.ts | 19 +-
foreign/node/src/client/client.config.ts | 3 -
foreign/node/src/client/client.socket.test.ts | 50 +-
foreign/node/src/e2e/tls.system.e2e.ts | 4 +
foreign/python/Cargo.toml | 1 +
foreign/python/Dockerfile.test | 2 +-
foreign/python/apache_iggy.pyi | 397 ++++-
foreign/python/docker-compose.test.yml | 11 +-
foreign/python/src/lib.rs | 5 +
foreign/python/src/receive_message.rs | 26 +
foreign/python/src/send_message.rs | 41 +-
foreign/python/src/user_headers.rs | 791 ++++++++++
foreign/python/tests/test_consumer_group.py | 98 ++
foreign/python/tests/test_message_operations.py | 402 ++++-
scripts/ci/python-sdk-version-sync.sh | 2 +-
scripts/ci/sync-python-interpreter-version.sh | 2 +-
scripts/ci/sync-rustc-version.sh | 2 +-
scripts/ci/third-party-licenses.sh | 2 +-
scripts/ci/uv-lock-check.sh | 2 +-
scripts/run-examples-from-readme.sh | 31 +-
scripts/utils.sh | 8 +-
web/package-lock.json | 613 ++++----
web/package.json | 30 +-
77 files changed, 9010 insertions(+), 1032 deletions(-)
copy core/connectors/sinks/{surrealdb_sink => meilisearch_sink}/Cargo.toml
(84%)
create mode 100644 core/connectors/sinks/meilisearch_sink/README.md
copy core/connectors/sinks/{elasticsearch_sink =>
meilisearch_sink}/config.toml (77%)
create mode 100644 core/connectors/sinks/meilisearch_sink/src/lib.rs
create mode 100644
core/integration/tests/cluster/multi_shard_partition_convergence.rs
create mode 100644
core/integration/tests/connectors/fixtures/meilisearch/container.rs
copy core/integration/tests/connectors/fixtures/{quickwit =>
meilisearch}/mod.rs (89%)
create mode 100644
core/integration/tests/connectors/fixtures/meilisearch/sink.rs
create mode 100644
core/integration/tests/connectors/meilisearch/meilisearch_sink.rs
copy core/integration/tests/connectors/{delta => meilisearch}/mod.rs (97%)
copy core/integration/tests/connectors/{delta => meilisearch}/sink.toml (94%)
create mode 100644 examples/python/message-headers/common.py
create mode 100644 examples/python/message-headers/plain-headers/consumer.py
create mode 100644 examples/python/message-headers/plain-headers/producer.py
create mode 100644 examples/python/message-headers/typed-headers/consumer.py
create mode 100644 examples/python/message-headers/typed-headers/producer.py
create mode 100644 foreign/python/src/user_headers.rs
