mfyuce opened a new issue, #3814:
URL: https://github.com/apache/iggy/issues/3814

   ### Bug description
   
   `quickwit_sink` predates the shared connector retry helpers 
(`sdk/src/retry.rs`
   arrived with the InfluxDB sink in #2933, roughly ten months after the sink
   itself) and was never adapted to them. Compared to the other HTTP sinks it is
   missing several guards, and each gap turns a recoverable condition into a 
hang
   or into silent data loss.
   
   **1. No request timeout.** `create_index`, `has_index` and `ingest` all 
share a
   bare `reqwest::Client::new()`, which has no timeout. Under a network 
partition,
   or against a Quickwit node that is still starting, these calls block
   indefinitely. The connector neither makes progress nor reports a failure.
   
   **2. No retries.** A transient 5xx or a connection reset during `open()` 
fails
   the connector outright, and the same during `ingest()` drops the batch. Every
   other HTTP sink routes these through retry middleware with exponential 
backoff.
   
   **3. Concurrent index creation deadlocks the connector.** `open()` calls
   `has_index()` and, if absent, `create_index()`. Two instances starting 
together,
   or one restarting next to a live one, both observe `has_index() == false` and
   both POST. The loser receives `409 Conflict`, which is treated as a fatal
   `InitError`, so that instance never opens even though the index it wanted now
   exists.
   
   **4. Non-JSON payloads are discarded.** `consume()` matches only
   `Payload::Json`; everything else hits a `warn!` and is dropped. A topic 
carrying
   raw or text messages silently loses every record, with the offset already
   committed.
   
   **5. No readiness gate.** There is no health probe before the first request, 
so
   startup races the node coming up.
   
   **6. Trailing slash in `url` produces a malformed path.** Every request 
rebuilds
   its URL from `self.config.url` with `format!` and nothing trims a trailing
   slash, so a configured `http://host:7280/` yields `//api/v1/...`.
   
   ### Affected area / component
   
   Connectors
   
   ### Deployment
   
   Compiled from source
   
   ### Reproduction
   
   For (3), start two `quickwit_sink` connectors against the same index before 
it
   exists. One opens, the other fails with `InitError` on the 409 and stays 
down.
   
   For (4), publish a message with a non-JSON payload to a topic the sink 
consumes.
   The record is dropped, the offset advances, and only a `warn!` is emitted.
   
   For (6), set `url = "http://localhost:7280/"` in `[plugin_config]` and 
observe
   the request path.
   
   ### Proposed scope
   
   #3523 addresses all six:
   
   - Build the HTTP client in `open()` with a configurable `timeout`, wrapped in
     `build_retry_client` from `iggy_connector_sdk::retry`, with configurable 
retry
     delay bounds.
   - Probe `/health/readyz` through `check_connectivity_with_retry` before 
touching
     the index.
   - Absorb `409 Conflict`, and a `400` whose body reports the index already
     exists, as success in `create_index`.
   - Wrap non-JSON payloads instead of dropping them: raw bytes are parsed as 
JSON
     when possible, otherwise carried as text, or base64 when not valid UTF-8.
   - Resolve the base URL once in `open()` with the trailing slash trimmed.
   - Add unit tests for payload handling and config defaults, alongside the
     existing integration suite in 
`core/integration/tests/connectors/quickwit/`.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to