[ https://issues.apache.org/jira/browse/NIFI-5448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16612166#comment-16612166 ]
Mark Payne edited comment on NIFI-5448 at 9/12/18 1:59 PM: ----------------------------------------------------------- [~mike.thomsen] [~mcgilman] [~pvillard] - I am re-opening this ticket, because I have some concerns about the changes here. The idea here was to add a new property and a new relationship, and the default will be to not use that new relationship and make it auto-terminated. This way, existing flows do not become invalid and continue to work as they have been working. What I think is being overlooked here is how this affects flows that are versioned in the Flow Registry. For any flow that is stored in the Flow Registry, when upgrading to this new version, we will see that the local flow is Locally Modified because it now has a new Relationship that is auto-terminated. This is a valid "Local Modification." This is a bit annoying for users, as they have to now push a new version of the flow to the Flow Registry, but not a big deal. However, where this does become a big deal is if a user has a flow in the Flow Registry that uses UpdateAttribute and is not currently on the Latest version of the flow. For example, My Flow has 8 versions, and I am using version 6 of My Flow. For valid reasons, I want to stick with version 6 (for instance, my downstream consumer cannot handle the data that is produced by version 8). I now update my NiFi to this latest version. What will now happen is that I will have a flow that is both Locally Modified AND Not the Latest Version. If I attempt to Revert Changes, then I still have Local Modifications because those Local Modifications are due to hardcoded changes in the Processor. Now, all of this stems from the fact that our Version Conflict Management is very crude at the moment. This will certainly need to be improved in the future, so that even if we have local modifications and the versioned flow has changed, we will need to be able to resolve those conflicts. But right now, we are just not there yet. Given that, I consider this a change that does in fact break backward compatibility. I think we are going to need to roll back these changes for 1.8.0 and only introduce such changes after we have a more robust Conflict Management story. While I understand that it's not ideal, I would recommend for the specified use case, for now, using a RouteOnAttribute to check if \{{ test:matches('\d{4}-\d\{2}-\d\{2}') }} and if not, routing to an 'invalid' relationship before sending to UpdateAttribute – or alternatively using a more complex Expression using if/then/else clauses so that the toDate() method is called only on valid values. was (Author: markap14): [~mike.thomsen] [~mcgilman] [~pvillard] - I am re-opening this ticket, because I have some concerns about the changes here. The idea here was to add a new property and a new relationship, and the default will be to not use that new relationship and make it auto-terminated. This way, existing flows do not become invalid and continue to work as they have been working. What I think is being overlooked here is how this affects flows that are versioned in the Flow Registry. For any flow that is stored in the Flow Registry, when upgrading to this new version, we will see that the local flow is Locally Modified because it now has a new Relationship that is auto-terminated. This is a valid "Local Modification." This is a bit annoying for users, as they have to now push a new version of the flow to the Flow Registry, but not a big deal. However, where this does become a big deal is if a user has a flow in the Flow Registry that uses UpdateAttribute and is not currently on the Latest version of the flow. For example, My Flow has 8 versions, and I am using version 6 of My Flow. For valid reasons, I want to stick with version 6 (for instance, my downstream consumer cannot handle the data that is produced by version 8). I now update my NiFi to this latest version. What will now happen is that I will have a flow that is both Locally Modified AND Not the Latest Version. If I attempt to Revert Changes, then I still have Local Modifications because those Local Modifications are due to hardcoded changes in the Processor. Now, all of this stems from the fact that our Version Conflict Management is very crude at the moment. This will certainly need to be improved in the future, so that even if we have local modifications and the versioned flow has changed, we will need to be able to resolve those conflicts. But right now, we are just not there yet. Given that, I consider this a change that does in fact break backward compatibility. I think we are going to need to roll back these changes for 1.8.0 and only introduce such changes after we have a more robust Conflict Management story. While I understand that it's not ideal, I would recommend for the specified use case, for now, using a RouteOnAttribute to check if \{{test:matches('\d{4}\-\d\{2}\-\d\{2}')}} and if not, routing to an 'invalid' relationship before sending to UpdateAttribute – or alternatively using a more complex Expression using if/then/else clauses so that the toDate() method is called only on valid values. > Failed EL date parsing live-locks processors without a failure relationship > --------------------------------------------------------------------------- > > Key: NIFI-5448 > URL: https://issues.apache.org/jira/browse/NIFI-5448 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework > Reporter: David Koster > Assignee: Mike Thomsen > Priority: Major > Fix For: 1.8.0 > > > Processors that utilize the Expression Language need to always present a > failure relationship. > If a processor with only a success relationship, for example UpdateAttribute, > utilizes the expression language to perform type coercion to a date and > fails, the processor will be unable to dispose of the FlowFile and remain > blocked indefinitely. > Recreation flow: > GenerateFlowFile -> Update Attribute #1 -> Update Attribute #2 -> Anything > Update Attribute #1 - test = "Hello World" > Update Attribute #2 - test = ${test:toDate('yyyy-MM-dd')} > > Generates an IllegalAttributeException on UpdateAttribute. > > The behavior should match numerical type coercion and silently skip the > processing or offer failure relationships on processors supporting EL -- This message was sent by Atlassian JIRA (v7.6.3#76005)