sramazzina opened a new pull request #1244: URL: https://github.com/apache/incubator-hop/pull/1244
This PR is to let KafkaConsumer manage unexpected exception generated in the called sub pipeline by records injected from the kafka queue. This possibility is given only in case of batchSize = 1 because only in that case we have a feeling on which is the failing record and we can easily use the error path to let that record come out from the called sub pipeline. As soon as the records comes out, it is committed in kafka and we have the possibility to dump it somewhere else for further investigation. The push to the error path of the failing record gives 2 benefits: - because of the commit it will free the kafka queue from a record that is stopping other records from being processed - the pipeline is not stopped because of the unexpected unhandled exception -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
