This is an automated email from the ASF dual-hosted git repository.
numinnex pushed a change to branch kafka_proxy_acls
in repository https://gitbox.apache.org/repos/asf/iggy.git
from df3a07d7a Merge branch 'master' into kafka_proxy_acls
add 1951e654c chore(integration): replace MinIO with Floci (#4290)
add b8bf85fd9 feat(swift): scaffold the Swift SDK package and CI lanes
(#4190)
add 8d0e28d9c feat(gateways): send Kafka records to Iggy from the Produce
handler (#4282)
add 15a49d082 feat(gateways): Implement CreateTopics and Metadata Kafka
Keys backed by the Iggy bridge (#4258)
add 3adf4fe39 docs: add how to report a bug found by an agent (#4309)
add 105ebe52f chore(deps): Bump rules_cc from 0.2.22 to 0.2.25 in
/foreign/cpp in the cpp group across 1 directory (#4310)
add e8cb28790 chore(deps): Bump the go group across 1 directory with 2
updates (#4300)
add d2f970661 fix(server): recover stalled consumer groups after restarts
(#4283)
add 11dd94c62 test(go): fail when a manual group commit on a split primary
drops membership (#4293)
new 5c8bac1ce merge master
The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.config/nextest.toml | 10 +-
.github/actions/swift/pre-merge/action.yml | 75 ++
.github/actions/utils/setup-swift/action.yml | 56 ++
.github/config/components.yml | 14 +
.github/workflows/_detect.yml | 11 +-
.github/workflows/_test.yml | 8 +
.github/workflows/pre-merge.yml | 21 +-
.gitignore | 8 +
AGENTS.md | 3 +-
CONTRIBUTING.md | 9 +
bdd/docker-compose.cluster.yml | 1 +
bdd/go/go.mod | 4 +-
bdd/go/go.sum | 8 +-
core/binary_protocol/src/consensus/command.rs | 11 +-
.../src/consensus/consumer_session.rs | 218 +++++
core/binary_protocol/src/consensus/header.rs | 6 +
core/binary_protocol/src/consensus/mod.rs | 4 +
core/binary_protocol/src/lib.rs | 22 +-
core/cli/src/args/consumer_group.rs | 6 +-
core/configs/src/common/defaults.rs | 10 +
core/configs/src/common/displays.rs | 6 +-
core/configs/src/common/server.rs | 7 +
core/configs/src/server_config/validators.rs | 98 +-
core/connectors/sinks/redshift_sink/README.md | 2 +-
.../test_consumer_group_create_command.rs | 34 +-
.../tests/cluster/client_table_restart.rs | 478 ++++++++-
.../tests/cluster/partition_primary_routing.rs | 29 +-
.../tests/connectors/fixtures/delta/fixture.rs | 158 +--
.../integration/tests/connectors/fixtures/floci.rs | 159 +++
.../tests/connectors/fixtures/iceberg/container.rs | 166 +---
core/integration/tests/connectors/fixtures/mod.rs | 1 +
.../connectors/fixtures/redshift/container.rs | 73 +-
.../tests/connectors/fixtures/redshift/mod.rs | 2 +-
.../tests/connectors/fixtures/redshift/sink.rs | 203 +---
.../tests/connectors/fixtures/s3/fixture.rs | 145 +--
core/integration/tests/connectors/s3/s3_sink.rs | 33 +-
.../stale_client_consumer_group_scenario.rs | 262 ++++-
core/metadata/src/impls/metadata.rs | 427 +++++++-
core/metadata/src/impls/recovery.rs | 53 +-
core/metadata/src/stm/consumer_group.rs | 242 ++++-
core/metadata/src/stm/mod.rs | 10 +
core/metadata/src/stm/stream.rs | 81 +-
core/server/config.toml | 19 +-
core/server/src/boot/mod.rs | 47 +-
core/server/src/boot/recovery.rs | 4 +
core/server/src/boot/threads.rs | 12 +-
core/server/src/consumer_group.rs | 5 +
core/server/src/consumer_group/lease.rs | 160 +++
core/server/src/consumer_group/liveness.rs | 1034 ++++++++++++++++++++
core/server/src/dispatch/mod.rs | 14 +-
core/server/src/dispatch/session_ops.rs | 2 +-
core/server/src/dispatch/submit.rs | 24 +-
core/server/src/dispatch/test_support.rs | 8 +
core/server/src/partition_reconciler.rs | 1 +
core/server/src/session_manager.rs | 70 +-
core/server/src/shell.rs | 4 +
core/server_common/src/consensus_message.rs | 16 +-
core/server_common/src/sharding/mod.rs | 10 +
core/shard/src/lib.rs | 75 +-
core/shard/src/poll/completion_tests.rs | 1 +
core/simulator/README.md | 1 +
core/simulator/src/client.rs | 18 +-
core/simulator/src/lib.rs | 274 +++++-
core/simulator/src/packet.rs | 7 +-
core/simulator/src/replica.rs | 12 +-
foreign/cpp/MODULE.bazel | 2 +-
foreign/cpp/MODULE.bazel.lock | 7 +-
foreign/go/client/tcp/tcp_offset_management.go | 9 +
foreign/go/tests/e2e_test.go | 49 +
foreign/swift/.swift-format | 22 +
.../consts.go => foreign/swift/Package.swift | 26 +-
foreign/swift/README.md | 87 ++
foreign/swift/Sources/Iggy/Errors/IggyError.swift | 78 ++
.../swift/Sources/Iggy/Errors/IggyErrorCode.swift | 515 ++++++++++
.../swift/Sources/Iggy/Iggy.swift | 20 +-
.../Sources/Iggy/Utilities/UInt128Value.swift | 156 +++
foreign/swift/Sources/Iggy/Wire/ByteCodec.swift | 218 +++++
foreign/swift/Tests/IggyTests/ByteCodecTests.swift | 120 +++
foreign/swift/Tests/IggyTests/ErrorCodeTests.swift | 55 ++
gateways/kafka/README.md | 120 ++-
gateways/kafka/docs/BRIDGE_MAPPING.md | 70 +-
gateways/kafka/docs/MANUAL_TESTING.md | 30 +-
gateways/kafka/docs/SCOPE.md | 122 ++-
gateways/kafka/docs/TEST_SUITE.md | 9 +-
gateways/kafka/src/bridge/config.rs | 47 +
gateways/kafka/src/bridge/error.rs | 139 ++-
gateways/kafka/src/bridge/iggy_bridge/mod.rs | 107 +-
gateways/kafka/src/bridge/iggy_bridge/offsets.rs | 37 +-
gateways/kafka/src/bridge/iggy_bridge/produce.rs | 183 +++-
gateways/kafka/src/bridge/iggy_bridge/topics.rs | 330 ++++++-
gateways/kafka/src/bridge/mod.rs | 8 +-
gateways/kafka/src/bridge/topic_map.rs | 6 +
gateways/kafka/src/protocol/api.rs | 81 +-
.../kafka/src/protocol/handlers/create_topics.rs | 711 +++++++++++++-
gateways/kafka/src/protocol/handlers/metadata.rs | 544 +++++++++-
gateways/kafka/src/protocol/handlers/produce.rs | 895 ++++++++++++++++-
gateways/kafka/src/records.rs | 976 +++++++++++++-----
gateways/kafka/tests/api_handler_tests.rs | 23 +-
.../kafka/tests/bridge_iggy_integration_tests.rs | 272 ++++-
gateways/kafka/tests/common/iggy_server.rs | 3 +-
.../kafka/tests/create_topics_real_bridge_tests.rs | 536 ++++++++++
gateways/kafka/tests/listener_robustness_tests.rs | 24 +-
gateways/kafka/tests/metadata_real_bridge_tests.rs | 408 ++++++++
gateways/kafka/tests/produce_real_bridge_tests.rs | 750 ++++++++++++++
gateways/kafka/tests/response_negative_tests.rs | 9 +-
gateways/kafka/tests/server_e2e_tests.rs | 26 +-
gateways/kafka/tests/version_firewall_tests.rs | 19 +-
licenserc.toml | 7 +
108 files changed, 11354 insertions(+), 1494 deletions(-)
create mode 100644 .github/actions/swift/pre-merge/action.yml
create mode 100644 .github/actions/utils/setup-swift/action.yml
create mode 100644 core/binary_protocol/src/consensus/consumer_session.rs
create mode 100644 core/integration/tests/connectors/fixtures/floci.rs
create mode 100644 core/server/src/consumer_group/lease.rs
create mode 100644 core/server/src/consumer_group/liveness.rs
create mode 100644 foreign/swift/.swift-format
copy examples/go/common/consts.go => foreign/swift/Package.swift (62%)
create mode 100644 foreign/swift/README.md
create mode 100644 foreign/swift/Sources/Iggy/Errors/IggyError.swift
create mode 100644 foreign/swift/Sources/Iggy/Errors/IggyErrorCode.swift
copy bdd/rust/tests/leader_redirection.rs =>
foreign/swift/Sources/Iggy/Iggy.swift (67%)
create mode 100644 foreign/swift/Sources/Iggy/Utilities/UInt128Value.swift
create mode 100644 foreign/swift/Sources/Iggy/Wire/ByteCodec.swift
create mode 100644 foreign/swift/Tests/IggyTests/ByteCodecTests.swift
create mode 100644 foreign/swift/Tests/IggyTests/ErrorCodeTests.swift
create mode 100644 gateways/kafka/tests/create_topics_real_bridge_tests.rs
create mode 100644 gateways/kafka/tests/metadata_real_bridge_tests.rs
create mode 100644 gateways/kafka/tests/produce_real_bridge_tests.rs
