Github user cloud-fan commented on the issue:
    This is the writing code I was talking about:
        // write the data and commit this writer.
        Utils.tryWithSafeFinallyAndFailureCallbacks(block = {
          logInfo(s"Writer for partition ${context.partitionId()} is 
          val msg = dataWriter.commit()
          logInfo(s"Writer for partition ${context.partitionId()} committed.")
        })(catchBlock = {
          // If there is an error, abort this writer
          logError(s"Writer for partition ${context.partitionId()} is 
          logError(s"Writer for partition ${context.partitionId()} aborted.")
    What we can probably do is to check job cancellation periodically during 
`iter.foreach(dataWriter.write)`, e.g. do a check for every 1k writes.
    Anyway let's merge this PR first. I'm only merging to master, let's 
backport it to 2.3 if RC3 fails(very likely to happen as there are already 
several regressions show up)


To unsubscribe, e-mail:
For additional commands, e-mail:

Reply via email to