This is an automated email from the ASF dual-hosted git repository. bashirbekov pushed a change to branch feat/add-background-send in repository https://gitbox.apache.org/repos/asf/iggy.git
discard 2d93b889 del discard 71757fc5 del discard 09107ec0 del discard 05468f8a del discard 4aeb096c del discard 77c25b83 del discard 1263492d del discard 6fb2f982 Merge branch 'feat/add-background-send' of https://github.com/apache/iggy into feat/add-background-send discard c3069208 del discard 1685b879 del discard d51e69b5 del discard bacd54ee del discard 85334f59 old variant discard 5191209e rename send_interval to linger_time discard c1f22af6 rename batch_size to batch_length add 926fc4f7 fix(test): Fix error logging in integration sample tests (#1791) add bb97a3b3 chore(repo): fix clippy lints for Rust 1.87 (#1800) add d53a6c17 feat(server): verify the format of server_addr (#1770) add b3798361 chore(test): delete redundant code of test_server (#1801) add 5a86d24f feat: Integrate Python SDK into the monorepo (#1790) add 239a11e7 feat(bench): introduce benchmark deep linking and 'Recent' tab (#1773) add bdc84883 chore(sdk): support zero copy schema in Java SDK (#1813) add 1dffd350 feat(sdk): Support `from_connection_string()` for `QuicClient` and `HttpClient` (#1803) add 15181ee7 chore(bench): fix Dockerfile paths (#1827) add a418e823 feat: split transport from client, add heartbeat and reconnection support (#1809) add a3c090af feat(js): add node ci (#1810) add c84d3166 fix(server): fix getting first messages when segment was removed (#1828) add d1c9077c fix(python): fix property name of partitions_count in Python client (#1808) add 3dc67c33 fix(python): add missing stub_gen.rs (#1816) add ad2973f4 chore(repo): add connectors to convco commit validator (#1829) add a1dc6b22 fix(js): fix deserializePollMessages bug (#1830) add ac44ad3b feat(connectors): add connectors runtime (#1826) add b27e3cb5 chore(ci): refactor CI workflows (#1819) add 81e9bda3 fix(python): fix python sdk typing stubs (#1818) new 804eaab8 rename batch_size to batch_length new c2a7e56f rename send_interval to linger_time new 41e92446 add background send option This update added new revisions after undoing existing revisions. That is to say, some revisions that were in the old version of the branch are not in the new version. This situation occurs when a user --force pushes a change and generates a repository containing something like this: * -- * -- B -- O -- O -- O (2d93b889) \ N -- N -- N refs/heads/feat/add-background-send (41e92446) You should already have received notification emails for all of the O revisions, and so the following emails describe only the N revisions from the common base, B. Any revisions marked "omit" are not gone; other references still refer to them. Any revisions marked "discard" are gone forever. The 3 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .asf.yaml | 2 +- .convco | 2 +- .github/changed-files-config.json | 13 + .github/workflows/ci-check-common.yml | 31 +- .github/workflows/ci-check-node-sdk.yml | 86 + .github/workflows/ci-check-pr.yml | 22 +- .github/workflows/ci-check-python-sdk.yml | 154 + .../{publish_cli.yml => publish_node_sdk.yml} | 101 +- .github/workflows/sanity.yml | 150 - .github/workflows/test_pr.yml | 187 - .gitignore | 2 + Cargo.lock | 477 ++- Cargo.toml | 22 +- DEPENDENCIES.md | 33 +- README.md | 30 +- core/bench/dashboard/README.md | 17 +- core/bench/dashboard/frontend/Cargo.toml | 5 +- core/bench/dashboard/frontend/assets/style.css | 662 +++- core/bench/dashboard/frontend/src/api/mod.rs | 61 +- .../frontend/src/components/app_content.rs | 430 ++- .../frontend/src/components/chart/single_chart.rs | 12 + .../frontend/src/components/layout/main_content.rs | 80 +- .../frontend/src/components/layout/sidebar.rs | 174 +- .../bench/dashboard/frontend/src/components/mod.rs | 1 + .../logo.rs => selectors/benchmark_search_box.rs} | 31 +- .../src/components/selectors/benchmark_selector.rs | 31 +- .../src/components/selectors/hardware_selector.rs | 54 +- .../frontend/src/components/selectors/mod.rs | 2 + .../selectors/recent_benchmarks_selector.rs | 265 ++ .../src/components/selectors/view_mode_selector.rs | 71 +- core/bench/dashboard/frontend/src/main.rs | 2 +- core/bench/dashboard/frontend/src/router.rs | 4 +- .../dashboard/frontend/src/state/benchmark.rs | 295 +- core/bench/dashboard/frontend/src/state/ui.rs | 6 +- core/bench/dashboard/server/Cargo.toml | 2 +- core/bench/dashboard/server/Dockerfile | 10 +- core/bench/dashboard/server/docker-entrypoint.sh | 2 +- core/bench/dashboard/server/src/cache/query.rs | 40 +- core/bench/dashboard/server/src/handlers.rs | 29 +- core/bench/dashboard/server/src/main.rs | 7 +- core/bench/dashboard/shared/src/subtext.rs | 9 + core/bench/report/src/types/benchmark_kind.rs | 2 +- core/bench/src/plot.rs | 9 +- .../src/cli/binary_context/common.rs | 5 +- core/cli/src/args/client.rs | 1 + core/common/src/error/iggy_error.rs | 2 + core/common/src/lib.rs | 6 +- .../configuration/auth_config/connection_string.rs | 349 +- .../auth_config/connection_string_options.rs | 74 +- .../{config.rs => http_client_config.rs} | 38 +- .../http_config/http_client_config_builder.rs} | 18 +- .../http_config/http_connection_string_options.rs | 87 + .../src/types/configuration/http_config/mod.rs | 4 +- .../src/types/configuration/quick_config/mod.rs | 1 + .../quick_config/quic_client_config.rs | 27 +- .../quick_config/quic_connection_string_options.rs | 281 ++ .../src/types/configuration/tcp_config/mod.rs | 1 + .../configuration/tcp_config/tcp_client_config.rs | 6 +- .../tcp_config/tcp_client_config_builder.rs | 83 +- .../tcp_config/tcp_connection_string_options.rs | 173 + core/connectors/README.md | 91 + .../runtime}/Cargo.toml | 27 +- core/connectors/runtime/config.toml | 131 + core/connectors/runtime/src/configs.rs | 86 + .../runtime/src/error.rs} | 36 +- core/connectors/runtime/src/main.rs | 308 ++ core/connectors/runtime/src/sink.rs | 365 ++ core/connectors/runtime/src/source.rs | 318 ++ core/connectors/runtime/src/transform.rs | 50 + .../{binary_protocol => connectors/sdk}/Cargo.toml | 22 +- .../sdk/src/decoders/json.rs} | 29 +- .../sdk/src/decoders}/mod.rs | 4 +- .../sdk/src/decoders/raw.rs} | 15 +- .../sdk/src/decoders/text.rs} | 29 +- .../sdk/src/encoders/json.rs} | 33 +- .../sdk/src/encoders}/mod.rs | 4 +- .../sdk/src/encoders/raw.rs} | 20 +- .../sdk/src/encoders/text.rs} | 23 +- core/connectors/sdk/src/lib.rs | 262 ++ core/connectors/sdk/src/sink.rs | 262 ++ core/connectors/sdk/src/source.rs | 232 ++ core/connectors/sdk/src/transforms/add_fields.rs | 128 + .../sdk/src/transforms/delete_fields.rs} | 62 +- core/connectors/sdk/src/transforms/mod.rs | 71 + .../sinks/quickwit_sink}/Cargo.toml | 28 +- core/connectors/sinks/quickwit_sink/src/lib.rs | 210 ++ .../sinks/stdout_sink}/Cargo.toml | 25 +- core/connectors/sinks/stdout_sink/src/lib.rs | 89 + .../sources/test_source}/Cargo.toml | 28 +- core/connectors/sources/test_source/src/lib.rs | 168 + core/examples/Cargo.toml | 6 + core/examples/src/multi-tenant/producer/main.rs | 9 +- core/examples/src/new-sdk/producer/main.rs | 7 +- .../src/sink-data-producer/docker-compose.yml | 34 +- core/examples/src/sink-data-producer/main.rs | 148 + core/integration/Cargo.toml | 1 - core/integration/src/http_client.rs | 3 +- core/integration/src/quic_client.rs | 3 +- core/integration/src/test_server.rs | 10 +- core/integration/tests/examples/mod.rs | 3 +- .../tests/examples/test_new_publisher.rs | 127 - core/integration/tests/mod.rs | 1 - core/integration/tests/sdk/mod.rs | 0 core/integration/tests/streaming/segment.rs | 12 +- core/sdk/Cargo.toml | 2 +- core/sdk/src/client_provider.rs | 9 +- core/sdk/src/clients/client.rs | 202 +- core/sdk/src/clients/client_builder.rs | 212 +- core/sdk/src/clients/mod.rs | 2 +- core/sdk/src/clients/producer.rs | 32 +- core/sdk/src/clients/producer_builder.rs | 36 +- core/sdk/src/clients/producer_dispatcher.rs | 219 +- core/sdk/src/clients/producer_error_callback.rs | 23 +- core/sdk/src/clients/producer_sharding.rs | 155 +- core/sdk/src/http/http_client.rs | 222 +- core/sdk/src/http/mod.rs | 1 - core/sdk/src/prelude.rs | 17 +- core/sdk/src/quic/mod.rs | 1 - core/sdk/src/quic/quick_client.rs | 314 +- core/sdk/src/quic/quick_config.rs | 232 -- .../stream_builder/build/build_iggy_producer.rs | 6 +- core/sdk/src/tcp/tcp_client.rs | 285 +- core/sdk/src/tcp/tcp_connection_stream_kind.rs | 1 + core/server/Cargo.toml | 3 + core/server/src/archiver/mod.rs | 1 + core/server/src/http/mapper.rs | 5 +- core/server/src/streaming/partitions/messages.rs | 7 +- .../src/streaming/segments/indexes/indexes_mut.rs | 2 +- .../streaming/segments/types/messages_batch_mut.rs | 2 +- core/server/src/streaming/systems/streams.rs | 6 +- foreign/java/.gitignore | 6 + .../java/examples/simple-consumer/build.gradle.kts | 1 + .../main/java/org/apache/iggy/SimpleConsumer.java | 4 +- .../java/examples/simple-producer/build.gradle.kts | 1 + foreign/java/java-sdk/build.gradle.kts | 1 + .../client/blocking/tcp/BytesDeserializer.java | 45 +- .../iggy/client/blocking/tcp/BytesSerializer.java | 23 +- .../client/blocking/tcp/MessagesTcpClient.java | 31 +- .../org/apache/iggy/identifier/Identifier.java | 10 + .../main/java/org/apache/iggy/message/Message.java | 27 +- .../{PolledMessage.java => MessageHeader.java} | 20 +- .../java/org/apache/iggy/message/Partitioning.java | 5 + .../org/apache/iggy/message/PolledMessages.java | 3 +- .../iggy/client/blocking/IntegrationTest.java | 2 +- .../client/blocking/MessagesClientBaseTest.java | 12 +- foreign/node/README.md | 7 +- foreign/node/package-lock.json | 135 +- foreign/node/package.json | 8 +- foreign/node/src/client/client.socket.ts | 208 +- foreign/node/src/client/client.transport.ts | 154 + foreign/node/src/client/client.ts | 110 +- foreign/node/src/client/client.type.ts | 27 +- foreign/node/src/client/index.ts | 2 - foreign/node/src/client/tcp.client.ts | 35 - foreign/node/src/client/tls.client.ts | 37 - foreign/node/src/debug-send.ts | 102 +- foreign/node/src/debug.ts | 20 +- foreign/node/src/e2e/tcp.consumer-group.e2e.ts | 6 +- foreign/node/src/e2e/tcp.consumer-stream.e2e.ts | 10 +- foreign/node/src/e2e/tcp.send-message.e2e.ts | 22 +- foreign/node/src/stream/consumer-stream.ts | 37 +- foreign/node/src/tcp.e2e.ts | 11 +- foreign/node/src/tls.system.e2e.ts | 11 +- foreign/node/src/wire/error.code.ts | 310 +- foreign/node/src/wire/message/iggy-header.utils.ts | 4 +- foreign/node/src/wire/message/poll.utils.ts | 11 +- .../src/wire/message/send-messages.command.test.ts | 2 +- .../node/src/wire/stream/create-stream.command.ts | 2 +- .../python/.devcontainer/Dockerfile | 14 +- foreign/python/.devcontainer/devcontainer.json | 9 + .../.devcontainer}/docker-compose.yml | 26 +- foreign/python/.gitignore | 71 + foreign/python/CONTRIBUTING.md | 48 + foreign/python/Cargo.lock | 3867 ++++++++++++++++++++ foreign/python/Cargo.toml | 27 + foreign/{csharp => python}/LICENSE | 0 foreign/{go => python}/NOTICE | 2 +- foreign/python/README.md | 62 + foreign/python/iggy_py.pyi | 182 + foreign/python/pyproject.toml | 28 + foreign/python/python_examples/README.md | 20 + foreign/python/python_examples/consumer.py | 80 + .../python/python_examples/example_screenshot.png | Bin 0 -> 2373561 bytes foreign/python/python_examples/producer.py | 104 + .../mod.rs => foreign/python/src/bin/stub_gen.rs | 12 +- foreign/python/src/client.rs | 324 ++ .../tests/cli/mod.rs => foreign/python/src/lib.rs | 34 +- foreign/python/src/receive_message.rs | 133 + foreign/python/src/send_message.rs | 59 + .../system.rs => foreign/python/src/stream.rs | 47 +- .../system.rs => foreign/python/src/topic.rs | 47 +- .../test.yml => foreign/python/tests/__init__.py | 10 - foreign/python/tests/conftest.py | 115 + foreign/python/tests/test_client.py | 57 + 194 files changed, 14300 insertions(+), 2838 deletions(-) create mode 100644 .github/workflows/ci-check-node-sdk.yml create mode 100644 .github/workflows/ci-check-python-sdk.yml copy .github/workflows/{publish_cli.yml => publish_node_sdk.yml} (51%) delete mode 100644 .github/workflows/sanity.yml delete mode 100644 .github/workflows/test_pr.yml copy core/bench/dashboard/frontend/src/components/{layout/logo.rs => selectors/benchmark_search_box.rs} (52%) create mode 100644 core/bench/dashboard/frontend/src/components/selectors/recent_benchmarks_selector.rs rename core/common/src/types/configuration/http_config/{config.rs => http_client_config.rs} (57%) rename core/{sdk/src/http/http_config.rs => common/src/types/configuration/http_config/http_client_config_builder.rs} (79%) create mode 100644 core/common/src/types/configuration/http_config/http_connection_string_options.rs create mode 100644 core/common/src/types/configuration/quick_config/quic_connection_string_options.rs create mode 100644 core/common/src/types/configuration/tcp_config/tcp_connection_string_options.rs create mode 100644 core/connectors/README.md copy core/{binary_protocol => connectors/runtime}/Cargo.toml (76%) create mode 100644 core/connectors/runtime/config.toml create mode 100644 core/connectors/runtime/src/configs.rs copy core/{common/src/error/client_error.rs => connectors/runtime/src/error.rs} (52%) create mode 100644 core/connectors/runtime/src/main.rs create mode 100644 core/connectors/runtime/src/sink.rs create mode 100644 core/connectors/runtime/src/source.rs create mode 100644 core/connectors/runtime/src/transform.rs copy core/{binary_protocol => connectors/sdk}/Cargo.toml (79%) copy core/{binary_protocol/src/cli/cli_command.rs => connectors/sdk/src/decoders/json.rs} (61%) copy core/{bench/report/src/plotting => connectors/sdk/src/decoders}/mod.rs (95%) copy core/{server/src/streaming/streams/messages.rs => connectors/sdk/src/decoders/raw.rs} (74%) copy core/{binary_protocol/src/cli/cli_command.rs => connectors/sdk/src/decoders/text.rs} (62%) copy core/{server/src/binary/handlers/utils.rs => connectors/sdk/src/encoders/json.rs} (57%) copy core/{bench/report/src/plotting => connectors/sdk/src/encoders}/mod.rs (95%) copy core/{server/src/streaming/streams/partitions.rs => connectors/sdk/src/encoders/raw.rs} (69%) copy core/{server/src/streaming/streams/partitions.rs => connectors/sdk/src/encoders/text.rs} (61%) create mode 100644 core/connectors/sdk/src/lib.rs create mode 100644 core/connectors/sdk/src/sink.rs create mode 100644 core/connectors/sdk/src/source.rs create mode 100644 core/connectors/sdk/src/transforms/add_fields.rs copy core/{server/src/http/jwt/json_web_token.rs => connectors/sdk/src/transforms/delete_fields.rs} (50%) create mode 100644 core/connectors/sdk/src/transforms/mod.rs copy core/{binary_protocol => connectors/sinks/quickwit_sink}/Cargo.toml (74%) create mode 100644 core/connectors/sinks/quickwit_sink/src/lib.rs copy core/{binary_protocol => connectors/sinks/stdout_sink}/Cargo.toml (74%) create mode 100644 core/connectors/sinks/stdout_sink/src/lib.rs copy core/{binary_protocol => connectors/sources/test_source}/Cargo.toml (76%) create mode 100644 core/connectors/sources/test_source/src/lib.rs copy docker-compose.yml => core/examples/src/sink-data-producer/docker-compose.yml (60%) create mode 100644 core/examples/src/sink-data-producer/main.rs delete mode 100644 core/integration/tests/examples/test_new_publisher.rs delete mode 100644 core/integration/tests/sdk/mod.rs delete mode 100644 core/sdk/src/quic/quick_config.rs create mode 100644 foreign/java/.gitignore rename foreign/java/java-sdk/src/main/java/org/apache/iggy/message/{PolledMessage.java => MessageHeader.java} (75%) create mode 100644 foreign/node/src/client/client.transport.ts delete mode 100644 foreign/node/src/client/tcp.client.ts delete mode 100644 foreign/node/src/client/tls.client.ts copy scripts/dashboard/build_release.sh => foreign/python/.devcontainer/Dockerfile (76%) mode change 100755 => 100644 create mode 100644 foreign/python/.devcontainer/devcontainer.json copy foreign/{node => python/.devcontainer}/docker-compose.yml (78%) create mode 100644 foreign/python/.gitignore create mode 100644 foreign/python/CONTRIBUTING.md create mode 100644 foreign/python/Cargo.lock create mode 100644 foreign/python/Cargo.toml copy foreign/{csharp => python}/LICENSE (100%) copy foreign/{go => python}/NOTICE (94%) create mode 100644 foreign/python/README.md create mode 100644 foreign/python/iggy_py.pyi create mode 100644 foreign/python/pyproject.toml create mode 100644 foreign/python/python_examples/README.md create mode 100644 foreign/python/python_examples/consumer.py create mode 100644 foreign/python/python_examples/example_screenshot.png create mode 100644 foreign/python/python_examples/producer.py copy core/bench/src/analytics/time_series/processors/mod.rs => foreign/python/src/bin/stub_gen.rs (79%) create mode 100644 foreign/python/src/client.rs copy core/integration/tests/cli/mod.rs => foreign/python/src/lib.rs (56%) create mode 100644 foreign/python/src/receive_message.rs create mode 100644 foreign/python/src/send_message.rs copy core/server/src/streaming/users/permissioner_rules/system.rs => foreign/python/src/stream.rs (51%) copy core/server/src/streaming/users/permissioner_rules/system.rs => foreign/python/src/topic.rs (51%) rename .github/workflows/test.yml => foreign/python/tests/__init__.py (87%) create mode 100644 foreign/python/tests/conftest.py create mode 100644 foreign/python/tests/test_client.py
