[
https://issues.apache.org/jira/browse/KAFKA-13676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Guozhang Wang updated KAFKA-13676:
----------------------------------
Description:
When processing in ALOS, we might as well commit progress made by other tasks
when some task encounters specific exception. If one task has an issue and we
have already successfully completed processing on at least one task it would be
good to commit those successfully processed tasks. This should prevent limit
the duplicated records downstream and also be more efficient.
Also if one task is having lots of issues the other tasks can at least make
progress. When we introduced the thread replacement mechanism this optimization
became possible.
was:
When processing in ALOS we might as well commit progress made other tasks on a
task specific exception. If one task has an issue and we have already
successfully completed processing on at least one task it would be good to
commit those successfully processed tasks. This should prevent limit the
duplicated records downstream and also be more efficient.
Also if one task is having lots of issues the other tasks can at least make
progress. When we introduced the thread replacement mechanism this optimization
became possible.
> When processing in ALOS, when one task encounters a task-specific exception
> we could still commit progress made by other tasks
> ------------------------------------------------------------------------------------------------------------------------------
>
> Key: KAFKA-13676
> URL: https://issues.apache.org/jira/browse/KAFKA-13676
> Project: Kafka
> Issue Type: Improvement
> Components: streams
> Reporter: Walker Carlson
> Assignee: Walker Carlson
> Priority: Major
>
> When processing in ALOS, we might as well commit progress made by other tasks
> when some task encounters specific exception. If one task has an issue and we
> have already successfully completed processing on at least one task it would
> be good to commit those successfully processed tasks. This should prevent
> limit the duplicated records downstream and also be more efficient.
> Also if one task is having lots of issues the other tasks can at least make
> progress. When we introduced the thread replacement mechanism this
> optimization became possible.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
