numinnex opened a new issue, #4723: URL: https://github.com/apache/iggy/issues/4723
### Bug description When the REST catalog is unreachable or returns a 5xx, the Iceberg sink drops the batch without retrying. `write_data` (`core/connectors/sinks/iceberg_sink/src/router/mod.rs`) commits with `Transaction::commit`. That function retries only errors marked retryable, and iceberg-catalog-rest 0.10.1 marks only HTTP 409 that way (`update_table`). Connection refused, timeouts and 5xx responses therefore return at once. The sink maps them to `Error::CatalogCommitError`. The runtime logs the failure and moves on, but the consumer offset was already stored when the batch was read (see #4232), so the batch is gone. The data files written for that commit are kept on purpose, as the comment in `write_data` explains, so each failed commit also leaves orphan files behind. An external test restarted the REST catalog once and lost 4,700 rows, the worst result of any fault it injected. ### Affected area / component Connectors: `iggy_connector_iceberg_sink` ### Versions master @ 3c265dd89, iceberg-catalog-rest 0.10.1 ### Proposed fix - **Request never reached the catalog** (connect errors): retry inside the sink with a bounded backoff. This is safe. - **Outcome unknown** (5xx, timeouts): the commit may have been applied. Tag each append with a snapshot summary property through `FastAppendAction::set_snapshot_properties`, for example a commit id plus the Iggy partition and offset range. Before retrying, reload the table and check whether a snapshot with that id already exists. The same marker lets the sink skip a batch the runtime delivers again once #4232 lands, so this also feeds into #4236. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
