mlevkov opened a new pull request, #3797: URL: https://github.com/apache/iggy/pull/3797
## Summary Closes #3796. The send callback runs synchronously inside the SDK's polling task (`handle_messages` in `core/connectors/sdk/src/source.rs`), on the tokio runtime shared by every connector instance loaded from the same plugin library (`static RUNTIME: OnceLock<Runtime>` in `lib.rs`). A callback that blocks pins one worker for the duration. With #3795's bounded forwarding channel, a callback legitimately blocks for backpressure while the channel is full, so enough saturated instances of one library can occupy all workers, and `iggy_source_close` for a sibling instance then waits behind them: the close blocks on the sibling's polling task getting scheduled to observe its shutdown signal. ## Change One call site: wrap the callback invocation in `tokio::task::block_in_place`, so the worker is handed off before the callback runs and the runtime keeps scheduling sibling tasks regardless of how long the callback blocks. Notes from the analysis in #3796: - The wrap has to live in the SDK, not the runtime: `block_in_place` consults the calling thread's tokio context, which is only set on the plugin runtime's own worker threads. - The SDK runtime is `Runtime::new()`, i.e. multi-threaded, which `block_in_place` requires. - No FFI or ABI change. Existing plugin binaries keep working and pick the fix up when rebuilt against the updated SDK. This composes with #3795 (bounded channel + shutdown signaling) but does not depend on it: any long-blocking callback benefits. ## Test plan - `cargo clippy -p iggy_connector_sdk --all-targets --all-features -- -D warnings` clean - `cargo test -p iggy_connector_sdk --all-features` passes - `cargo build -p iggy_connector_stdout_sink -p iggy_connector_random_source -p iggy-connectors` (macro consumers + runtime rebuild cleanly) - Behavior under saturation is exercised end to end by the integration suite in CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
