arunpandianp commented on code in PR #38814: URL: https://github.com/apache/beam/pull/38814#discussion_r3376664490
########## runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/streaming/Work.java: ########## Review Comment: > Do we want this to reflect the shared-state? It seems like if we are processing a group of keys and are processing key C but key A which was previously processed is marked as failed due to heartbeats then we'd want to stop processing key C. I think we'd need to return true here to notice that. It is shared state within a batch, if any work failed in the batch the entire batch fails. But the valid work items are retried in separate batches locally. So we need a boolean to say if the batch failed and a boolean to say if the work failed. https://github.com/arunpandianp/beam/pull/7 shows the upcoming failure handling logic, which should provide related context. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
