This is an automated email from the ASF dual-hosted git repository.
hubcio pushed a change to branch login-command-protocol-versin
in repository https://gitbox.apache.org/repos/asf/iggy.git
omit 5ad6730aa Merge branch 'master' into login-command-protocol-versin
omit c71e1a643 Merge branch 'master' into login-command-protocol-versin
omit d5ea5fa9a Merge branch 'master' into login-command-protocol-versin
omit 246da8b5b feat(server-ng): reject incompatible clients at login via
protocol version
add b2940dbb0 chore(deps): bump python/cpp FFI SDK deps to latest (#3552)
add 9ba4dbf12 feat(cpp): add functions related to consumer groups and
topics (#3506)
add 7c6d899b4 feat(ci,rust): declare MSRV 1.95 and enforce it in CI (#3558)
add 8a19837b7 chore: remove `once_cell` in favor of the standard library
(#3524)
add 1932014c7 feat(server-ng): port consumer group (#3527)
add bc68827ae chore(deps): bump gradle wrapper to 9.6.0 (#3563)
add 3aa225f83 chore(deps): Bump the node group across 2 directories with 2
updates (#3567)
add 0efe933f8 chore(deps): Bump rules_cc from 0.2.18 to 0.2.19 in
/foreign/cpp in the cpp group (#3565)
add 8c1fefb34 chore(deps): Bump the web group across 1 directory with 8
updates (#3566)
add 307fdb16d feat(server-ng): commit-time metadata validation with result
codes (#3553)
add 3b9ba2f93 fix(security): require read_servers permission for system
snapshot (#3579)
add 1f18a8f08 feat(connectors): add S3 sink connector (#3103)
add a9b769d3e fix(partitions): stop holding partition refs across .await
(#3557)
add f200135fc feat(server-ng): reject incompatible clients at login via
protocol version
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (5ad6730aa)
\
N -- N -- N refs/heads/login-command-protocol-versin (f200135fc)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
.github/actions/rust/pre-merge/action.yml | 24 +-
.github/config/components.yml | 16 +
.github/workflows/_build_rust_artifacts.yml | 2 +-
.github/workflows/edge-release.yml | 1 +
Cargo.lock | 365 ++++++-
Cargo.toml | 7 +-
bdd/java/Dockerfile | 2 +-
bdd/java/gradle/wrapper/gradle-wrapper.properties | 2 +-
bdd/java/gradlew | 2 +-
core/binary_protocol/Cargo.toml | 1 +
core/binary_protocol/src/codes.rs | 2 +
core/binary_protocol/src/consensus/header.rs | 12 +-
core/binary_protocol/src/consensus/mod.rs | 2 +
core/binary_protocol/src/consensus/operation.rs | 21 +-
core/binary_protocol/src/consensus/reply_result.rs | 129 +++
core/binary_protocol/src/dispatch.rs | 22 +-
core/binary_protocol/src/lib.rs | 4 +-
core/binary_protocol/src/primitives/consumer.rs | 4 +-
.../src/requests/consumer_groups/mod.rs | 2 +
...in_consumer_group.rs => sync_consumer_group.rs} | 32 +-
.../src/responses/consumer_groups/mod.rs | 2 +
.../consumer_groups/sync_consumer_group.rs | 114 +++
core/binary_protocol/src/version.rs | 4 +-
core/cli/Cargo.toml | 1 +
core/common/Cargo.toml | 1 +
core/common/src/consumer_group_client_state.rs | 219 ++++
core/common/src/error/iggy_error.rs | 6 +
core/common/src/lib.rs | 13 +
.../src/traits/binary_impls/consumer_groups.rs | 13 +
core/common/src/traits/binary_impls/messages.rs | 237 +++++
core/common/src/traits/binary_impls/system.rs | 5 +
core/common/src/traits/binary_transport.rs | 8 +
core/common/src/traits/system_client.rs | 7 +
core/connectors/README.md | 1 +
core/connectors/runtime/Cargo.toml | 1 -
core/connectors/runtime/src/source.rs | 6 +-
core/connectors/sdk/Cargo.toml | 1 -
core/connectors/sdk/src/lib.rs | 7 +-
core/connectors/sdk/src/sink.rs | 5 +-
core/connectors/sdk/src/source.rs | 5 +-
core/connectors/sinks/README.md | 2 +-
core/connectors/sinks/delta_sink/Cargo.toml | 3 +-
.../connectors/sinks/elasticsearch_sink/Cargo.toml | 3 +-
core/connectors/sinks/iceberg_sink/Cargo.toml | 3 +-
core/connectors/sinks/postgres_sink/Cargo.toml | 3 +-
core/connectors/sinks/quickwit_sink/Cargo.toml | 3 +-
.../s3_sink}/Cargo.toml | 21 +-
core/connectors/sinks/s3_sink/README.md | 164 +++
.../sinks/s3_sink/config.toml} | 49 +-
core/connectors/sinks/s3_sink/src/buffer.rs | 182 ++++
core/connectors/sinks/s3_sink/src/client.rs | 169 +++
core/connectors/sinks/s3_sink/src/formatter.rs | 393 +++++++
core/connectors/sinks/s3_sink/src/lib.rs | 498 +++++++++
core/connectors/sinks/s3_sink/src/path.rs | 245 +++++
core/connectors/sinks/s3_sink/src/sink.rs | 662 ++++++++++++
core/connectors/sinks/stdout_sink/Cargo.toml | 3 +-
core/connectors/sources/README.md | 1 -
.../sources/elasticsearch_source/Cargo.toml | 3 +-
core/connectors/sources/influxdb_source/Cargo.toml | 15 +-
.../sources/influxdb_source/dependencies.md | 1 -
core/connectors/sources/postgres_source/Cargo.toml | 3 +-
core/connectors/sources/random_source/Cargo.toml | 3 +-
core/consensus/src/impls.rs | 5 +-
core/consensus/src/metadata_helpers.rs | 2 +-
core/consensus/src/observability.rs | 4 +
core/consensus/src/plane_helpers.rs | 36 +-
core/integration/Cargo.toml | 2 +-
core/integration/src/harness/context.rs | 8 +-
core/integration/src/harness/port_reserver.rs | 180 ++--
core/integration/tests/cli/common/keyring.rs | 1 +
core/integration/tests/cli/common/mod.rs | 1 +
core/integration/tests/connectors/fixtures/mod.rs | 2 +
.../tests/connectors/fixtures/s3/fixture.rs | 308 ++++++
.../tests/connectors/fixtures/{delta => s3}/mod.rs | 2 +-
core/integration/tests/connectors/mod.rs | 1 +
.../tests/connectors/s3/mod.rs} | 1 +
core/integration/tests/connectors/s3/s3_sink.rs | 196 ++++
.../tests/connectors/{http => s3}/sink.toml | 2 +-
.../{http/sink.toml => s3/sink_rotation.toml} | 2 +-
core/integration/tests/data_integrity/mod.rs | 13 +-
core/integration/tests/mod.rs | 28 +-
core/integration/tests/sdk/protocol_version.rs | 5 +-
core/integration/tests/server/{cg.rs => cg_vsr.rs} | 39 +-
core/integration/tests/server/mod.rs | 5 +-
.../server/scenarios/authentication_scenario.rs | 12 +-
...onsumer_group_duplicate_name_create_scenario.rs | 118 +++
.../server/scenarios/log_rotation_scenario.rs | 4 +-
core/integration/tests/server/scenarios/mod.rs | 15 +-
.../tests/server/scenarios/permissions_scenario.rs | 26 +
.../scenarios/stress_produce_consume_scenario.rs | 264 +++++
core/metadata/src/impls/metadata.rs | 252 ++++-
core/metadata/src/stm/consumer_group.rs | 1081 +++++++++++++++-----
core/metadata/src/stm/mod.rs | 29 +-
core/metadata/src/stm/mux.rs | 27 +-
core/metadata/src/stm/result.rs | 476 +++++++++
core/metadata/src/stm/snapshot.rs | 12 +-
core/metadata/src/stm/stream.rs | 815 +++++++++++++--
core/metadata/src/stm/user.rs | 148 ++-
core/partitions/Cargo.toml | 1 +
core/partitions/src/iggy_partition.rs | 879 +++++++++-------
core/partitions/src/iggy_partitions.rs | 394 ++++++-
core/partitions/src/journal.rs | 331 +++---
core/partitions/src/lib.rs | 12 +-
core/partitions/src/offset_storage.rs | 4 +
core/partitions/src/poll_plan.rs | 798 +++++++++++++++
core/sdk/Cargo.toml | 1 +
.../src/client_wrappers/binary_system_client.rs | 12 +
core/sdk/src/clients/client.rs | 7 +
core/sdk/src/quic/quic_client.rs | 9 +
core/sdk/src/tcp/tcp_client.rs | 9 +
core/sdk/src/vsr.rs | 97 +-
core/sdk/src/websocket/websocket_client.rs | 9 +
core/server-ng/src/bootstrap.rs | 290 +++++-
core/server-ng/src/consumer_group.rs | 257 +++++
core/server-ng/src/dispatch.rs | 468 +++++++--
core/server-ng/src/lib.rs | 1 +
core/server-ng/src/partition_reconciler.rs | 387 ++++++-
core/server-ng/src/responses.rs | 221 +++-
core/server-ng/src/server_error.rs | 5 +
core/server-ng/src/session_manager.rs | 57 +-
.../binary/handlers/system/get_snapshot_handler.rs | 1 +
core/server/src/http/system.rs | 7 +-
core/server/src/metadata/reader.rs | 4 +
core/server_common/Cargo.toml | 1 -
core/server_common/src/consensus_message.rs | 58 +-
core/server_common/src/memory_pool.rs | 5 +-
core/shard/Cargo.toml | 1 -
core/shard/src/lib.rs | 266 ++---
core/shard/src/router.rs | 46 +-
core/simulator/src/deps.rs | 3 +-
core/simulator/src/lib.rs | 112 +-
core/simulator/src/main.rs | 4 +-
core/simulator/src/packet.rs | 14 +-
core/simulator/src/replica.rs | 4 +-
core/simulator/src/workload/auditor.rs | 32 +-
core/simulator/src/workload/mod.rs | 147 ++-
core/simulator/src/workload/ops/change_password.rs | 37 +-
.../src/workload/ops/create_consumer_group.rs | 35 +-
.../src/workload/ops/create_partitions.rs | 59 +-
.../workload/ops/create_personal_access_token.rs | 44 +-
core/simulator/src/workload/ops/create_stream.rs | 36 +-
core/simulator/src/workload/ops/create_topic.rs | 51 +-
core/simulator/src/workload/ops/create_user.rs | 46 +-
.../src/workload/ops/delete_consumer_group.rs | 47 +-
.../src/workload/ops/delete_consumer_offset.rs | 4 +-
.../src/workload/ops/delete_consumer_offset_2.rs | 4 +-
.../src/workload/ops/delete_partitions.rs | 54 +-
.../workload/ops/delete_personal_access_token.rs | 30 +-
core/simulator/src/workload/ops/delete_segments.rs | 27 +-
core/simulator/src/workload/ops/delete_stream.rs | 30 +-
core/simulator/src/workload/ops/delete_topic.rs | 39 +-
core/simulator/src/workload/ops/delete_user.rs | 30 +-
core/simulator/src/workload/ops/mod.rs | 36 +-
core/simulator/src/workload/ops/purge_stream.rs | 42 +-
core/simulator/src/workload/ops/purge_topic.rs | 44 +-
core/simulator/src/workload/ops/send_messages.rs | 4 +-
.../src/workload/ops/store_consumer_offset.rs | 13 +-
.../src/workload/ops/store_consumer_offset_2.rs | 13 +-
.../src/workload/ops/update_permissions.rs | 33 +-
core/simulator/src/workload/ops/update_stream.rs | 37 +-
core/simulator/src/workload/ops/update_topic.rs | 46 +-
core/simulator/src/workload/ops/update_user.rs | 59 +-
core/simulator/src/workload/shadow.rs | 8 +
.../java/gradle/wrapper/gradle-wrapper.properties | 2 +-
examples/java/gradlew | 2 +-
examples/node/package-lock.json | 15 +-
examples/node/package.json | 2 +-
foreign/cpp/Cargo.toml | 2 +-
foreign/cpp/MODULE.bazel | 2 +-
foreign/cpp/MODULE.bazel.lock | 8 +-
foreign/cpp/src/client.rs | 180 +++-
foreign/cpp/src/consumer_group.rs | 14 +-
foreign/cpp/src/lib.rs | 56 +-
foreign/cpp/src/topic.rs | 39 +-
foreign/cpp/tests/client/low_level_e2e.cpp | 2 -
foreign/cpp/tests/consumer_group/low_level_e2e.cpp | 282 +++++
foreign/cpp/tests/stream/low_level_e2e.cpp | 2 -
foreign/cpp/tests/topic/low_level_e2e.cpp | 839 ++++++++++++++-
foreign/java/build.gradle.kts | 4 +-
.../java/gradle/wrapper/gradle-wrapper.properties | 2 +-
foreign/java/gradlew | 2 +-
foreign/node/package-lock.json | 123 ++-
foreign/python/Cargo.toml | 8 +-
scripts/verify-crates-publish.sh | 29 +
web/package-lock.json | 556 +++++-----
web/package.json | 14 +-
186 files changed, 13571 insertions(+), 2335 deletions(-)
create mode 100644 core/binary_protocol/src/consensus/reply_result.rs
copy core/binary_protocol/src/requests/consumer_groups/{join_consumer_group.rs
=> sync_consumer_group.rs} (76%)
create mode 100644
core/binary_protocol/src/responses/consumer_groups/sync_consumer_group.rs
create mode 100644 core/common/src/consumer_group_client_state.rs
copy core/connectors/{sources/elasticsearch_source =>
sinks/s3_sink}/Cargo.toml (80%)
create mode 100644 core/connectors/sinks/s3_sink/README.md
copy core/{integration/tests/connectors/influxdb_sink.toml =>
connectors/sinks/s3_sink/config.toml} (51%)
create mode 100644 core/connectors/sinks/s3_sink/src/buffer.rs
create mode 100644 core/connectors/sinks/s3_sink/src/client.rs
create mode 100644 core/connectors/sinks/s3_sink/src/formatter.rs
create mode 100644 core/connectors/sinks/s3_sink/src/lib.rs
create mode 100644 core/connectors/sinks/s3_sink/src/path.rs
create mode 100644 core/connectors/sinks/s3_sink/src/sink.rs
create mode 100644 core/integration/tests/connectors/fixtures/s3/fixture.rs
copy core/integration/tests/connectors/fixtures/{delta => s3}/mod.rs (92%)
copy core/{partitions/src/send_messages2.rs =>
integration/tests/connectors/s3/mod.rs} (98%)
create mode 100644 core/integration/tests/connectors/s3/s3_sink.rs
copy core/integration/tests/connectors/{http => s3}/sink.toml (94%)
copy core/integration/tests/connectors/{http/sink.toml =>
s3/sink_rotation.toml} (94%)
copy core/integration/tests/server/{cg.rs => cg_vsr.rs} (73%)
create mode 100644
core/integration/tests/server/scenarios/consumer_group_duplicate_name_create_scenario.rs
create mode 100644
core/integration/tests/server/scenarios/stress_produce_consume_scenario.rs
create mode 100644 core/metadata/src/stm/result.rs
create mode 100644 core/partitions/src/poll_plan.rs
create mode 100644 core/server-ng/src/consumer_group.rs
