[
https://issues.apache.org/jira/browse/NIFI-5448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16612166#comment-16612166
]
Mark Payne edited comment on NIFI-5448 at 9/12/18 1:59 PM:
-----------------------------------------------------------
[~mike.thomsen] [~mcgilman] [~pvillard] - I am re-opening this ticket, because
I have some concerns about the changes here. The idea here was to add a new
property and a new relationship, and the default will be to not use that new
relationship and make it auto-terminated. This way, existing flows do not
become invalid and continue to work as they have been working. What I think is
being overlooked here is how this affects flows that are versioned in the Flow
Registry.
For any flow that is stored in the Flow Registry, when upgrading to this new
version, we will see that the local flow is Locally Modified because it now has
a new Relationship that is auto-terminated. This is a valid "Local
Modification." This is a bit annoying for users, as they have to now push a new
version of the flow to the Flow Registry, but not a big deal.
However, where this does become a big deal is if a user has a flow in the Flow
Registry that uses UpdateAttribute and is not currently on the Latest version
of the flow. For example, My Flow has 8 versions, and I am using version 6 of
My Flow. For valid reasons, I want to stick with version 6 (for instance, my
downstream consumer cannot handle the data that is produced by version 8). I
now update my NiFi to this latest version. What will now happen is that I will
have a flow that is both Locally Modified AND Not the Latest Version. If I
attempt to Revert Changes, then I still have Local Modifications because those
Local Modifications are due to hardcoded changes in the Processor.
Now, all of this stems from the fact that our Version Conflict Management is
very crude at the moment. This will certainly need to be improved in the
future, so that even if we have local modifications and the versioned flow has
changed, we will need to be able to resolve those conflicts. But right now, we
are just not there yet. Given that, I consider this a change that does in fact
break backward compatibility. I think we are going to need to roll back these
changes for 1.8.0 and only introduce such changes after we have a more robust
Conflict Management story. While I understand that it's not ideal, I would
recommend for the specified use case, for now, using a RouteOnAttribute to
check if \{{ test:matches('\d{4}-\d\{2}-\d\{2}') }} and if not, routing to an
'invalid' relationship before sending to UpdateAttribute – or alternatively
using a more complex Expression using if/then/else clauses so that the toDate()
method is called only on valid values.
was (Author: markap14):
[~mike.thomsen] [~mcgilman] [~pvillard] - I am re-opening this ticket, because
I have some concerns about the changes here. The idea here was to add a new
property and a new relationship, and the default will be to not use that new
relationship and make it auto-terminated. This way, existing flows do not
become invalid and continue to work as they have been working. What I think is
being overlooked here is how this affects flows that are versioned in the Flow
Registry.
For any flow that is stored in the Flow Registry, when upgrading to this new
version, we will see that the local flow is Locally Modified because it now has
a new Relationship that is auto-terminated. This is a valid "Local
Modification." This is a bit annoying for users, as they have to now push a new
version of the flow to the Flow Registry, but not a big deal.
However, where this does become a big deal is if a user has a flow in the Flow
Registry that uses UpdateAttribute and is not currently on the Latest version
of the flow. For example, My Flow has 8 versions, and I am using version 6 of
My Flow. For valid reasons, I want to stick with version 6 (for instance, my
downstream consumer cannot handle the data that is produced by version 8). I
now update my NiFi to this latest version. What will now happen is that I will
have a flow that is both Locally Modified AND Not the Latest Version. If I
attempt to Revert Changes, then I still have Local Modifications because those
Local Modifications are due to hardcoded changes in the Processor.
Now, all of this stems from the fact that our Version Conflict Management is
very crude at the moment. This will certainly need to be improved in the
future, so that even if we have local modifications and the versioned flow has
changed, we will need to be able to resolve those conflicts. But right now, we
are just not there yet. Given that, I consider this a change that does in fact
break backward compatibility. I think we are going to need to roll back these
changes for 1.8.0 and only introduce such changes after we have a more robust
Conflict Management story. While I understand that it's not ideal, I would
recommend for the specified use case, for now, using a RouteOnAttribute to
check if \{{test:matches('\d{4}\-\d\{2}\-\d\{2}')}} and if not, routing to an
'invalid' relationship before sending to UpdateAttribute – or alternatively
using a more complex Expression using if/then/else clauses so that the toDate()
method is called only on valid values.
> Failed EL date parsing live-locks processors without a failure relationship
> ---------------------------------------------------------------------------
>
> Key: NIFI-5448
> URL: https://issues.apache.org/jira/browse/NIFI-5448
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
> Reporter: David Koster
> Assignee: Mike Thomsen
> Priority: Major
> Fix For: 1.8.0
>
>
> Processors that utilize the Expression Language need to always present a
> failure relationship.
> If a processor with only a success relationship, for example UpdateAttribute,
> utilizes the expression language to perform type coercion to a date and
> fails, the processor will be unable to dispose of the FlowFile and remain
> blocked indefinitely.
> Recreation flow:
> GenerateFlowFile -> Update Attribute #1 -> Update Attribute #2 -> Anything
> Update Attribute #1 - test = "Hello World"
> Update Attribute #2 - test = ${test:toDate('yyyy-MM-dd')}
>
> Generates an IllegalAttributeException on UpdateAttribute.
>
> The behavior should match numerical type coercion and silently skip the
> processing or offer failure relationships on processors supporting EL
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
