brianbushree opened a new issue #9845:
URL: https://github.com/apache/druid/issues/9845
### Affected Version
0.17.0
### Description
We're ingesting data from Kafka and we've noticed this exception a couple
times now... Here's how it normally surfaces:
We'll see druid metrics that indicate there were 2 failed tasks. We then use
the Druid UI to grab the task IDs that failed. Around the same time we'll
notice some spike in ingest lag.
After some investigation we'll find the exception below in one of the tasks
however the other task that was listed as failed in the UI has no exceptions in
the log...
Also in both tasks, we've noticed that the report.json says the task was
COMPLETED with no error message.
```
org.apache.druid.java.util.common.ISE: Attempting to publish with empty
segment set, but total row count was not 0: [2202792].
at
org.apache.druid.segment.realtime.appenderator.BaseAppenderatorDriver.publishInBackground(BaseAppenderatorDriver.java:566)
at
org.apache.druid.segment.realtime.appenderator.StreamAppenderatorDriver.lambda$publish$1(StreamAppenderatorDriver.java:281)
at
org.apache.druid.java.util.common.concurrent.ListenableFutures$1.onSuccess(ListenableFutures.java:50)
at com.google.common.util.concurrent.Futures$4.run(Futures.java:1181)
at
com.google.common.util.concurrent.MoreExecutors$SameThreadExecutorService.execute(MoreExecutors.java:297)
at
com.google.common.util.concurrent.ExecutionList.executeListener(ExecutionList.java:156)
at
com.google.common.util.concurrent.ExecutionList.execute(ExecutionList.java:145)
at
com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:185)
at
com.google.common.util.concurrent.Futures$ChainingListenableFuture$1.run(Futures.java:872)
at
com.google.common.util.concurrent.MoreExecutors$SameThreadExecutorService.execute(MoreExecutors.java:297)
at
com.google.common.util.concurrent.Futures$ImmediateFuture.addListener(Futures.java:102)
at
com.google.common.util.concurrent.Futures$ChainingListenableFuture.run(Futures.java:868)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
```
Couple questions here:
1. What does this exception mean and what's the best way to troubleshoot
this?
2. Why does the report.json indicate that there were no errors?
3. Is it possible that a task failure from an exception like this could
cause a separate task to fail? Maybe if they were running on the same node?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
