spetz commented on code in PR #4153:
URL: https://github.com/apache/iggy/pull/4153#discussion_r4007155226
##########
core/connectors/sinks/s3_sink/src/sink.rs:
##########
@@ -45,7 +46,27 @@ impl Sink for S3Sink {
let bucket = crate::client::create_bucket(&self.config).await?;
- crate::client::verify_bucket(&bucket).await?;
+ let prefix = self
+ .config
+ .prefix
+ .as_deref()
+ .unwrap_or_default()
+ .trim_matches('/');
+ let probe_key = if prefix.is_empty() {
Review Comment:
The startup probe moved from the bucket root into the configured prefix so
it exercises the same write scope as data uploads, but cleanup is still
best-effort `let _ = bucket.delete_object(&probe_key)`. The probe exists for
write-only IAM policies, and such a policy normally grants `s3:PutObject`
without `s3:DeleteObject`, so on exactly the deployments this targets a
zero-byte `.iggy-sink-probe` now lands inside the data prefix, survives, and is
rewritten on every open. A consumer listing the prefix sees an object the sink
did not write as data. At the bucket root it was outside the reader's path.
##########
core/connectors/sinks/elasticsearch_sink/src/lib.rs:
##########
@@ -38,6 +37,27 @@ use tracing::{error, info, warn};
sink_connector!(ElasticsearchSink);
const DEFAULT_TIMEOUT_SECONDS: u64 = 30;
+const BULK_RETRY_POLICY: RetryPolicy = RetryPolicy {
Review Comment:
`BULK_RETRY_POLICY` is a hardcoded const of three attempts, one second base,
five second cap, while every other HTTP-shaped sink takes `max_retries` and
`retry_delay` from config. With the default 30 second client timeout an
operator facing a slow cluster has no knob on a consume call that can now run
for roughly three timeouts plus backoff.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
