davis-anthony commented on PR #6903: URL: https://github.com/apache/nifi/pull/6903#issuecomment-1517863606
@ChrisSamo632 Thanks for the explanation and for helping me understand the issues with memory. Yes that makes a lot of sense and after reading the code I see that the processor makes every effort to just pass the documents to the various relationships. So with that in mind I propose that we make these processors behave like the RouteOnAttribute processor where the developer can define additional relationships based on expected error types. So that the processor can simply compare the error type for each item in the collection and route to the various relationships. For example, I might define the following: conflict -> version_conflict_engine_exception schema -> mapper_parsing_exception blocked -> cluster_block_exception Then the following relationships could be routed in the flow: success, error, retry, failure, conflict, schema, blocked. The result would be that errors that match the 3 types are routed to 3 custom routes. The remaining errors and other error routes would operate as they do now. The only flaw I can think of here is that there is no definitive list of error types according to the following post, and so it takes quite a bit of sleuthing to find the possible errors and they are likely subject to change since there is no list. https://discuss.elastic.co/t/bulk-api-item-error-types/242639 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
