Github user victor-wong commented on the issue:
https://github.com/apache/spark/pull/19824
@CodingCat
> One thing to note is that mute an event is a behavior change
I agree with that, so we should be careful about changing the current
behavior. I will close the PR later.
> the remaining discussion becomes âwhat does complete mean in
English?â which is not interesting to me to discuss
Maybe the remaining discussion should be how to let the user know that he
will get a StreamingListenerBatchCompleted event even if the batch failed.
What about adding some comments:
`
/** Called when processing of a batch of jobs has completed **(event if
the batch failed)**. */
def onBatchCompleted(batchCompleted: StreamingListenerBatchCompleted) { }
`
As I was using the StreamingListenerBatchCompleted to do some metadata
checkpointing stuff, which should be done only when the batch succeeded.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
