hubcio commented on code in PR #3953:
URL: https://github.com/apache/iggy/pull/3953#discussion_r3851722290
##########
core/sdk/src/clients/producer_dispatcher.rs:
##########
@@ -49,29 +49,17 @@ impl ProducerDispatcher {
let err_callback = config.error_callback.clone();
let (stop_tx, _) = broadcast::channel::<()>(1);
- let mut stop_rx = stop_tx.subscribe();
+ // A task that receives errors from shards when writing messages
failed.
Review Comment:
restates what the spawn does, drop it.
##########
core/sdk/src/clients/producer_dispatcher.rs:
##########
@@ -432,4 +422,52 @@ mod tests {
assert_eq!(sharding_called.load(Ordering::SeqCst), 1);
assert_eq!(error_called.load(Ordering::SeqCst), 1);
}
+
+ #[tokio::test]
+ async fn test_shutdown_reports_errors_from_final_flush() {
+ let mut mock = MockProducerCoreBackend::new();
+ // mock a failed send to be caught by the error callback
+ mock.expect_send_internal()
+ .times(1)
Review Comment:
`.times(1)` is not enforced here - the last `Arc` owner of the mock is the
shard task, so a violated expectation panics there and `shutdown()` swallows it
as "shard panicked". the `error_called` assert carries the test, just don't
rely on `times`.
##########
core/sdk/src/clients/producer_dispatcher.rs:
##########
@@ -432,4 +422,52 @@ mod tests {
assert_eq!(sharding_called.load(Ordering::SeqCst), 1);
assert_eq!(error_called.load(Ordering::SeqCst), 1);
}
+
+ #[tokio::test]
+ async fn test_shutdown_reports_errors_from_final_flush() {
+ let mut mock = MockProducerCoreBackend::new();
+ // mock a failed send to be caught by the error callback
+ mock.expect_send_internal()
+ .times(1)
+ .returning(|_, _, _, _| {
+ Box::pin(async {
+ Err(IggyError::ProducerSendFailed {
+ cause: Box::new(IggyError::Error),
+ failed: Arc::new(vec![dummy_message(10)]),
+ committed: Arc::new(Vec::new()),
+ stream_name: "1".to_string(),
+ topic_name: "1".to_string(),
+ })
+ })
+ });
+
+ let error_called = Arc::new(AtomicUsize::new(0));
+
+ let config = BackgroundConfig::builder()
+ .num_shards(1)
+ .linger_time(Duration::from_secs(60).into()) // block/ wait so
shutdown triggers the flush
+ .error_callback(Arc::new(Box::new(TestErrorCallback {
+ called: error_called.clone(),
+ })))
+ .build();
+
+ let dispatcher = ProducerDispatcher::new(Arc::new(mock), config);
+
+ dispatcher
+ .dispatch(
+ vec![dummy_message(10)],
+ dummy_identifier(),
+ dummy_identifier(),
+ None,
+ )
+ .await
+ .unwrap();
+
+ // trigger the final flush
+ dispatcher.shutdown().await;
+
+ // Passes, if the error callback is hit once after final
+ // flush from shutdown.
+ assert_eq!(error_called.load(Ordering::SeqCst), 1);
Review Comment:
the stub drops `ctx`, so this proves the callback ran but not that the
failed batch came back with it - which is the point of #3947. record
`ctx.messages.len()` in the stub and assert it is 1.
##########
core/sdk/src/clients/producer_dispatcher.rs:
##########
@@ -192,6 +180,8 @@ impl ProducerDispatcher {
}
}
+ // After shards are closed await the error callback task,
+ // that might drain queued errors from the final flush.
if let Err(e) = self._join_handle.await {
Review Comment:
`_join_handle` is awaited here, the underscore prefix says unused.
##########
core/sdk/src/clients/producer_dispatcher.rs:
##########
@@ -432,4 +422,52 @@ mod tests {
assert_eq!(sharding_called.load(Ordering::SeqCst), 1);
assert_eq!(error_called.load(Ordering::SeqCst), 1);
}
+
+ #[tokio::test]
+ async fn test_shutdown_reports_errors_from_final_flush() {
+ let mut mock = MockProducerCoreBackend::new();
+ // mock a failed send to be caught by the error callback
Review Comment:
also at 448, 466, 469. these restate the code, drop them (and the stray
"block/ ").
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
