This is an automated email from the ASF dual-hosted git repository.
hubcio pushed a change to branch fix/docs-audit-connector-runtime
in repository https://gitbox.apache.org/repos/asf/iggy.git
omit 186e69e67 fix(connectors): report failures and correct format
conversion
add cfd3dff6a feat(cpp): add stream, topic, and partition functions to
high level client (#4022)
add ac5c7bf3a feat(connectors): add the HTTP source webhook gateway
connector (#3798)
add 309662d33 fix(ci): run integration tests when a binary they launch
changes (#4131)
add 6138eb9fd chore(deps): Bump the node group across 2 directories with 3
updates (#4125)
add 3b9320072 chore(deps): Bump github.com/klauspost/compress from 1.19.2
to 1.20.0 in /bdd/go in the go group across 1 directory (#4147)
add 2019e145d chore(deps): Bump the web group across 1 directory with 7
updates (#4124)
add dc54783f2 fix(server): resolve non-Linux clippy errors in shard
binding (#4144)
add 2b2e61253 fix(connectors): draw the retry delay inside the cap instead
of clamping (#4129)
add a4f38467b chore(deps): Bump the python group across 3 directories with
1 update (#4110)
add 262117cd1 perf(partitions): cut persisted-mode disk traffic per
acknowledgment (#4146)
add 67e14c26b fix(connectors): close the source instance a failed start
leaves behind (#4064)
add bb5ed1702 fix(server): preserve write responses and validate CLI input
(#4150)
add 9a4fd0f75 docs(integration): say that the test harness runs prebuilt
binaries (#4160)
add 585ab6922 fix(sdk): preserve consumer progress and client error
handling (#4151)
add ecc3a3fed fix(connectors): report failures and correct format
conversion
add 83e0a96b7 fix(connectors): preserve protobuf data and redact state
errors
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (186e69e67)
\
N -- N -- N refs/heads/fix/docs-audit-connector-runtime
(83e0a96b7)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
.claude/skills/connector-source/SKILL.md | 36 +-
.claude/skills/connector-testing/SKILL.md | 51 +
.github/actions/rust/pre-merge/action.yml | 15 +
.github/config/components.yml | 2 +-
.github/workflows/_test.yml | 10 +
.github/workflows/_test_examples.yml | 10 +
AGENTS.md | 5 +-
Cargo.lock | 28 +
Cargo.toml | 2 +
README.md | 10 +-
.../features/step_definitions/messaging_steps.cpp | 31 +-
bdd/go/go.mod | 2 +-
bdd/go/go.sum | 4 +-
bdd/python/uv.lock | 38 +-
core/ai/mcp/README.md | 45 +-
core/ai/mcp/src/error.rs | 4 +
core/ai/mcp/src/log.rs | 167 +-
core/ai/mcp/src/main.rs | 118 +-
core/ai/mcp/src/service/mod.rs | 20 +-
core/ai/mcp/src/service/requests.rs | 4 +-
core/ai/mcp/src/systemd.rs | 2 +-
core/bench/Cargo.toml | 1 +
core/bench/src/benchmarks/common.rs | 4 +-
core/bench/src/utils/mod.rs | 14 +-
core/binary_protocol/src/consensus/header.rs | 43 +-
core/binary_protocol/src/consensus/operation.rs | 2 +-
.../src/requests/topics/update_topic.rs | 14 +-
.../src/responses/messages/send_messages.rs | 16 +-
core/binary_protocol/src/version.rs | 4 +-
core/cli/src/args/consumer_group.rs | 6 +-
core/cli/src/args/consumer_offset.rs | 4 +-
core/cli/src/args/message.rs | 56 +-
core/cli/src/args/mod.rs | 4 +-
core/cli/src/args/personal_access_token.rs | 4 +-
core/cli/src/args/segment.rs | 37 +-
core/cli/src/args/stream.rs | 4 +-
core/cli/src/args/system.rs | 2 +-
core/cli/src/args/topic.rs | 30 +-
.../src/commands/binary_message/send_messages.rs | 23 +-
core/common/src/consumer_group_client_state.rs | 3 +-
core/common/src/traits/binary_impls/messages.rs | 10 +-
core/common/src/traits/binary_transport.rs | 3 +-
core/common/src/traits/client.rs | 2 +-
core/common/src/types/args/mod.rs | 6 +-
.../configuration/tcp_config/tcp_client_config.rs | 2 +-
core/common/src/types/message/partitioning.rs | 12 +-
core/common/src/types/message/partitioning_kind.rs | 4 +-
core/common/src/types/options/mod.rs | 31 +-
core/configs/src/server_config/defaults.rs | 2 +
core/configs/src/server_config/displays.rs | 3 +-
core/configs/src/server_config/partition.rs | 30 +
.../connectors/http_source_github.toml | 60 +
.../connectors/http_source_partner.toml | 55 +
core/connectors/runtime/src/main.rs | 27 +-
core/connectors/runtime/src/manager/source.rs | 221 +-
core/connectors/runtime/src/metrics.rs | 2 +-
core/connectors/runtime/src/sink.rs | 13 +-
core/connectors/runtime/src/source.rs | 294 +-
core/connectors/runtime/src/state/http.rs | 52 +-
core/connectors/sdk/README.md | 6 +-
core/connectors/sdk/src/decoders/proto.rs | 49 +-
core/connectors/sdk/src/encoders/proto.rs | 25 +-
core/connectors/sdk/src/retry.rs | 91 +-
core/connectors/sdk/src/sink.rs | 7 +
core/connectors/sdk/src/source.rs | 10 +
.../connectors/sdk/src/transforms/proto_convert.rs | 26 +-
core/connectors/sdk/tests/protobuf_integration.rs | 187 +-
core/connectors/sinks/doris_sink/README.md | 4 +-
core/connectors/sources/README.md | 3 +-
.../{random_source => http_source}/Cargo.toml | 19 +-
core/connectors/sources/http_source/README.md | 382 +++
core/connectors/sources/http_source/config.toml | 68 +
core/connectors/sources/http_source/src/auth.rs | 463 +++
core/connectors/sources/http_source/src/lib.rs | 2536 ++++++++++++++++
.../sources/http_source/src/management.rs | 1535 ++++++++++
core/connectors/sources/http_source/src/metrics.rs | 497 ++++
core/connectors/sources/http_source/src/routes.rs | 666 +++++
core/connectors/sources/http_source/src/server.rs | 3128 ++++++++++++++++++++
core/connectors/sources/http_source/src/state.rs | 1437 +++++++++
core/connectors/sources/http_source/src/types.rs | 275 ++
core/consensus/src/impls.rs | 12 +-
core/cpu_allocation/src/lib.rs | 4 +-
core/integration/Cargo.toml | 1 +
core/integration/src/bench_utils.rs | 2 +-
core/integration/tests/bench.rs | 114 +
.../test_consumer_group_create_command.rs | 10 +-
.../test_consumer_group_help_command.rs | 2 +-
.../test_consumer_offset_get_command.rs | 4 +-
.../test_consumer_offset_set_command.rs | 4 +-
.../tests/cli/general/test_help_command.rs | 8 +-
.../cli/message/test_message_flush_command.rs | 30 +-
.../tests/cli/message/test_message_help_command.rs | 2 +-
.../tests/cli/message/test_message_send_command.rs | 44 +-
.../test_pat_create_command.rs | 4 +-
.../tests/cli/stream/test_stream_create_command.rs | 6 +-
.../tests/cli/system/test_me_command.rs | 2 +-
.../tests/cli/system/test_snapshot_cmd.rs | 2 +-
.../tests/cli/topic/test_topic_create_command.rs | 16 +-
.../tests/cli/topic/test_topic_update_command.rs | 20 +-
core/integration/tests/cluster/crash_durability.rs | 12 +-
.../tests/cluster/fast_primary_rejoin.rs | 9 +
.../tests/connectors/fixtures/http/mod.rs | 5 +
.../tests/connectors/fixtures/http/source.rs | 125 +
core/integration/tests/connectors/fixtures/mod.rs | 6 +-
.../tests/connectors/http/http_source.rs | 580 ++++
core/integration/tests/connectors/http/mod.rs | 3 +
.../tests/connectors/{random => http}/source.toml | 2 +-
.../http/source_config/http_github.toml} | 41 +-
.../http/source_config/http_partner.toml} | 37 +-
.../tests/connectors/random/random_source.rs | 120 +-
core/integration/tests/mcp/mod.rs | 221 +-
core/integration/tests/mod.rs | 1 +
core/integration/tests/sdk/consumer_offset.rs | 61 +
core/integration/tests/server/mod.rs | 1 +
core/integration/tests/server/telemetry.rs | 63 +
.../tests/server/topic_admission_vsr.rs | 20 +-
core/journal/src/partition_journal.rs | 701 ++++-
core/journal/src/partition_journal/segments.rs | 18 +-
core/metadata/src/stm/snapshot.rs | 30 +-
core/metadata/src/stm/stream.rs | 99 +-
core/partitions/src/iggy_index_writer.rs | 23 +-
core/partitions/src/iggy_partition.rs | 64 +-
core/partitions/src/install_backup.rs | 21 +
core/partitions/src/persistence.rs | 223 +-
core/sdk/README.md | 27 +-
core/sdk/src/clients/client.rs | 32 +-
core/sdk/src/clients/consumer.rs | 4 +-
core/sdk/src/clients/producer.rs | 21 +-
core/sdk/src/clients/producer_builder.rs | 4 +-
.../stream_builder/build/build_iggy_consumer.rs | 2 +-
.../stream_builder/config/config_iggy_consumer.rs | 35 +-
.../stream_builder/config/config_iggy_producer.rs | 5 +-
core/sdk/src/vsr.rs | 55 +-
core/sdk/src/websocket/websocket_client.rs | 37 +
core/server/config.toml | 42 +-
core/server/src/args.rs | 10 +-
core/server/src/dispatch/partition.rs | 8 +-
core/server/src/http/forward.rs | 24 +-
core/server/src/http/handlers.rs | 2 +-
core/server/src/partition_helpers.rs | 38 +-
core/server/src/responses.rs | 22 +-
core/server/src/segment_recovery.rs | 13 +-
core/server/src/server_error.rs | 7 +-
core/server/src/shard_allocator.rs | 20 +-
core/server_common/src/consensus_message.rs | 6 +-
core/server_common/src/log/logger.rs | 8 +-
core/server_common/src/memory_pool.rs | 4 +-
core/server_common/src/send_messages.rs | 4 +-
core/simulator/src/storage/tests.rs | 138 +-
examples/csharp/README.md | 31 +-
.../Utils.cs | 7 +-
.../Utils.cs | 32 +-
.../src/Iggy_SDK.Examples.Shared/ExampleHelpers.cs | 29 +-
.../Utils.cs | 7 +-
.../Utils.cs | 29 +-
.../Utils.cs | 7 +-
.../Utils.cs | 29 +-
.../Iggy_SDK.Examples.TcpTls.Consumer/Program.cs | 2 +-
examples/go/README.md | 31 +-
examples/go/getting-started/consumer/main.go | 2 +-
examples/go/getting-started/consumer/main_test.go | 89 +
examples/go/go.mod | 2 +-
examples/go/go.sum | 4 +-
examples/java/README.md | 34 +-
examples/java/build.gradle.kts | 6 +
examples/java/settings.gradle.kts | 8 +
.../apache/iggy/examples/async/AsyncConsumer.java | 14 +-
.../consumer/GettingStartedConsumer.java | 5 +-
.../consumer/MessageEnvelopeConsumer.java | 5 +-
.../consumer/MessageHeadersConsumer.java | 3 +-
.../multitenant/consumer/MultiTenantConsumer.java | 3 +-
.../multitenant/producer/MultiTenantProducer.java | 3 +-
.../iggy/examples/streambuilder/StreamBasic.java | 3 +-
.../examples/tcptls/consumer/TcpTlsConsumer.java | 11 +-
.../examples/tcptls/producer/TcpTlsProducer.java | 6 +-
.../apache/iggy/examples/ConsumerOffsetTest.java | 213 ++
.../iggy/examples/MultiTenantAccessTest.java | 141 +
examples/node/README.md | 58 +-
examples/node/package-lock.json | 10 +-
examples/node/package.json | 5 +-
examples/node/src/basic/consumer.ts | 11 +-
examples/node/src/getting-started/consumer.ts | 7 +-
examples/node/src/message-envelope/consumer.ts | 9 +-
examples/node/src/message-headers/consumer.ts | 9 +-
examples/node/src/multi-tenant/consumer.ts | 9 +-
examples/node/src/sink-data-producer/producer.ts | 2 +-
examples/node/src/tcp-tls/consumer.ts | 13 +-
examples/node/src/tcp-tls/producer.ts | 9 +-
examples/node/tests/consumer-offset.test.mjs | 106 +
examples/php/README.md | 7 +-
examples/php/src/common.php | 18 +-
examples/python/README.md | 42 +-
examples/python/pyproject.toml | 2 +-
examples/python/uv.lock | 40 +-
examples/rust/README.md | 28 +-
examples/rust/src/basic/consumer/main.rs | 3 +
examples/rust/src/basic/producer/main.rs | 3 +
.../stream-builder/stream-consumer-config/main.rs | 18 +-
.../stream-builder/stream-producer-config/main.rs | 10 +-
examples/rust/src/tcp-tls/consumer/main.rs | 4 +-
examples/rust/src/tcp-tls/producer/main.rs | 4 +-
foreign/cpp/README.md | 8 +-
foreign/cpp/include/iggy.hpp | 1810 +++++++++--
foreign/cpp/src/client.cpp | 192 +-
foreign/cpp/src/client.rs | 159 +-
foreign/cpp/src/lib.rs | 59 +-
foreign/cpp/src/type_conversion.rs | 175 +-
foreign/cpp/src/type_conversions.cpp | 118 +
foreign/cpp/tests/e2e/client.cpp | 86 +-
foreign/cpp/tests/e2e/consumer_group.cpp | 162 +-
foreign/cpp/tests/e2e/message.cpp | 156 +-
foreign/cpp/tests/e2e/partition.cpp | 438 ++-
foreign/cpp/tests/e2e/stream.cpp | 1039 +++----
foreign/cpp/tests/e2e/test_helpers.hpp | 34 +-
foreign/cpp/tests/e2e/topic.cpp | 1795 ++++++-----
foreign/cpp/tests/unit/unit_tests.cpp | 361 ++-
foreign/csharp/Iggy_SDK/Consumers/IggyConsumer.cs | 21 +-
.../Iggy_SDK/Consumers/IggyConsumerBuilder.cs | 4 +-
.../Iggy_SDK/Consumers/IggyConsumerConfig.cs | 4 +-
foreign/csharp/Iggy_SDK/Contracts/TopicOptions.cs | 10 +-
foreign/csharp/Iggy_SDK/IggyClient/IIggyTopic.cs | 4 +-
.../Implementations/HttpMessageStream.cs | 20 +-
.../Implementations/TcpMessageStream.Vsr.cs | 5 +-
.../csharp/Iggy_SDK/Publishers/IggyPublisher.cs | 41 +-
.../Iggy_SDK/Publishers/IggyPublisherBuilder.cs | 12 +-
.../Iggy_SDK/Publishers/IggyPublisherConfig.cs | 10 +-
.../Iggy_SDK/Vsr/ConsumerGroupClientState.cs | 4 +-
.../ClientTests/ExampleHelpersTests.cs | 80 +
.../ClientTests/HttpTopicOptionsTests.cs | 57 +-
.../ConsumerTests/IggyConsumerBuilderTests.cs | 38 +
.../csharp/Iggy_SDK_Tests/Iggy_SDK_Tests.csproj | 1 +
.../PublisherTests/IggyTypedPublisherTests.cs | 83 +
foreign/csharp/README.md | 291 +-
foreign/go/README.md | 28 +-
.../binary_response_deserializer.go | 4 +-
foreign/go/client/tcp/tcp_core.go | 10 +-
foreign/go/client/tcp/tcp_group_polling.go | 4 +-
foreign/go/client/tcp/tcp_messaging.go | 5 +-
foreign/go/contracts/client.go | 12 +-
foreign/go/go.mod | 2 +-
foreign/go/go.sum | 4 +-
foreign/java/README.md | 35 +-
.../apache/iggy/client/async/MessagesClient.java | 13 +-
foreign/node/README.md | 18 +-
foreign/node/package-lock.json | 142 +-
foreign/node/package.json | 2 +-
foreign/php/README.md | 44 +-
foreign/php/src/client.rs | 2 +
foreign/php/tests/IggySdkTest.php | 62 +
foreign/python/README.md | 9 +-
foreign/python/apache_iggy.pyi | 16 +-
foreign/python/src/client.rs | 8 +-
foreign/python/src/send_message.rs | 8 +-
foreign/python/uv.lock | 38 +-
web/README.md | 22 +-
web/package-lock.json | 187 +-
web/package.json | 15 +-
web/src/lib/api/ApiSchema.ts | 7 +-
web/src/lib/components/Breadcrumbs.svelte | 5 +-
.../components/Modals/DeletePartitionsModal.svelte | 2 +-
.../components/Modals/StreamSettingsModal.svelte | 2 +-
.../components/Modals/TopicSettingsModal.svelte | 7 +-
.../src/lib/utils/base64Utils.test.mjs | 27 +-
web/src/lib/utils/base64Utils.ts | 3 +-
.../topics/[topicId=i32]/+page.svelte | 2 +-
265 files changed, 21382 insertions(+), 4282 deletions(-)
create mode 100644
core/connectors/runtime/example_config/connectors/http_source_github.toml
create mode 100644
core/connectors/runtime/example_config/connectors/http_source_partner.toml
copy core/connectors/sources/{random_source => http_source}/Cargo.toml (80%)
create mode 100644 core/connectors/sources/http_source/README.md
create mode 100644 core/connectors/sources/http_source/config.toml
create mode 100644 core/connectors/sources/http_source/src/auth.rs
create mode 100644 core/connectors/sources/http_source/src/lib.rs
create mode 100644 core/connectors/sources/http_source/src/management.rs
create mode 100644 core/connectors/sources/http_source/src/metrics.rs
create mode 100644 core/connectors/sources/http_source/src/routes.rs
create mode 100644 core/connectors/sources/http_source/src/server.rs
create mode 100644 core/connectors/sources/http_source/src/state.rs
create mode 100644 core/connectors/sources/http_source/src/types.rs
create mode 100644 core/integration/tests/bench.rs
create mode 100644 core/integration/tests/connectors/fixtures/http/source.rs
create mode 100644 core/integration/tests/connectors/http/http_source.rs
copy core/integration/tests/connectors/{random => http}/source.toml (94%)
copy core/{connectors/sources/random_source/config.toml =>
integration/tests/connectors/http/source_config/http_github.toml} (53%)
copy core/{connectors/sources/random_source/config.toml =>
integration/tests/connectors/http/source_config/http_partner.toml} (59%)
create mode 100644 core/integration/tests/server/telemetry.rs
create mode 100644 examples/go/getting-started/consumer/main_test.go
create mode 100644
examples/java/src/test/java/org/apache/iggy/examples/ConsumerOffsetTest.java
create mode 100644
examples/java/src/test/java/org/apache/iggy/examples/MultiTenantAccessTest.java
create mode 100644 examples/node/tests/consumer-offset.test.mjs
create mode 100644 foreign/cpp/src/type_conversions.cpp
create mode 100644
foreign/csharp/Iggy_SDK_Tests/ClientTests/ExampleHelpersTests.cs
copy foreign/node/src/wire/command.utils.test.ts =>
web/src/lib/utils/base64Utils.test.mjs (61%)
