[jira] [Updated] (KAFKA-5158) Options for handling exceptions during processing
[ https://issues.apache.org/jira/browse/KAFKA-5158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-5158: - Issue Type: Task (was: Sub-task) Parent: (was: KAFKA-5156) > Options for handling exceptions during processing > - > > Key: KAFKA-5158 > URL: https://issues.apache.org/jira/browse/KAFKA-5158 > Project: Kafka > Issue Type: Task > Components: streams >Reporter: Eno Thereska > > Imagine the app-level processing of a (non-corrupted) record fails (e.g. the > user attempted to do a RPC to an external system, and this call failed). How > can you process such failed records in a scalable way? For example, imagine > you need to implement a retry policy such as "retry with exponential > backoff". Here, you have the problem that 1. you can't really pause > processing a single record because this will pause the processing of the full > stream (bottleneck!) and 2. there is no straight-forward way to "sort" failed > records based on their "next retry time". -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (KAFKA-5158) Options for handling exceptions during processing
[ https://issues.apache.org/jira/browse/KAFKA-5158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-5158: --- Fix Version/s: (was: 1.0.0) > Options for handling exceptions during processing > - > > Key: KAFKA-5158 > URL: https://issues.apache.org/jira/browse/KAFKA-5158 > Project: Kafka > Issue Type: Sub-task > Components: streams >Reporter: Eno Thereska >Assignee: Matthias J. Sax > > Imagine the app-level processing of a (non-corrupted) record fails (e.g. the > user attempted to do a RPC to an external system, and this call failed). How > can you process such failed records in a scalable way? For example, imagine > you need to implement a retry policy such as "retry with exponential > backoff". Here, you have the problem that 1. you can't really pause > processing a single record because this will pause the processing of the full > stream (bottleneck!) and 2. there is no straight-forward way to "sort" failed > records based on their "next retry time". -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (KAFKA-5158) Options for handling exceptions during processing
[ https://issues.apache.org/jira/browse/KAFKA-5158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-5158: - *Reminder to the contributor / reviewer of the PR*: please note that the code deadline for 1.0.0 is less than 2 weeks away (Oct. 4th). Please re-evaluate your JIRA and see if it still makes sense to be merged into 1.0.0 or it could be pushed out to 1.1.0, or be closed directly if the JIRA itself is not valid any more, or re-assign yourself as contributor / committer if you are no longer working on the JIRA. > Options for handling exceptions during processing > - > > Key: KAFKA-5158 > URL: https://issues.apache.org/jira/browse/KAFKA-5158 > Project: Kafka > Issue Type: Sub-task > Components: streams >Reporter: Eno Thereska >Assignee: Matthias J. Sax > Fix For: 1.0.0 > > > Imagine the app-level processing of a (non-corrupted) record fails (e.g. the > user attempted to do a RPC to an external system, and this call failed). How > can you process such failed records in a scalable way? For example, imagine > you need to implement a retry policy such as "retry with exponential > backoff". Here, you have the problem that 1. you can't really pause > processing a single record because this will pause the processing of the full > stream (bottleneck!) and 2. there is no straight-forward way to "sort" failed > records based on their "next retry time". -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (KAFKA-5158) Options for handling exceptions during processing
[ https://issues.apache.org/jira/browse/KAFKA-5158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-5158: --- Issue Type: Sub-task (was: New Feature) Parent: KAFKA-5156 > Options for handling exceptions during processing > - > > Key: KAFKA-5158 > URL: https://issues.apache.org/jira/browse/KAFKA-5158 > Project: Kafka > Issue Type: Sub-task > Components: streams >Reporter: Eno Thereska >Assignee: Matthias J. Sax > Fix For: 1.0.0 > > > Imagine the app-level processing of a (non-corrupted) record fails (e.g. the > user attempted to do a RPC to an external system, and this call failed). How > can you process such failed records in a scalable way? For example, imagine > you need to implement a retry policy such as "retry with exponential > backoff". Here, you have the problem that 1. you can't really pause > processing a single record because this will pause the processing of the full > stream (bottleneck!) and 2. there is no straight-forward way to "sort" failed > records based on their "next retry time". -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (KAFKA-5158) Options for handling exceptions during processing
[ https://issues.apache.org/jira/browse/KAFKA-5158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eno Thereska updated KAFKA-5158: Issue Type: New Feature (was: Sub-task) Parent: (was: KAFKA-5156) > Options for handling exceptions during processing > - > > Key: KAFKA-5158 > URL: https://issues.apache.org/jira/browse/KAFKA-5158 > Project: Kafka > Issue Type: New Feature > Components: streams >Reporter: Eno Thereska > Fix For: 0.11.1.0 > > > Imagine the app-level processing of a (non-corrupted) record fails (e.g. the > user attempted to do a RPC to an external system, and this call failed). How > can you process such failed records in a scalable way? For example, imagine > you need to implement a retry policy such as "retry with exponential > backoff". Here, you have the problem that 1. you can't really pause > processing a single record because this will pause the processing of the full > stream (bottleneck!) and 2. there is no straight-forward way to "sort" failed > records based on their "next retry time". -- This message was sent by Atlassian JIRA (v6.4.14#64029)
