I understand the issue now, I’m not sure that a failure of ReplaceText is the
best place to catch this though. The reason I’m not sure it’s the best place
is what happens if there are multiple failures because you had multiple
expressions, just having them all routed to the same failure wouldn’t help you
make decisions on what to do with a single attribute. Perhaps a better solution
would be to use a RouteOnAttribute to check if the attributes match a certain
pattern before sending them to ReplaceText. A possible expression could be
“${actualSettlementDate:matches('[0-9]{2}/[0-9]{2}/[0-9]{4}')}” however that
would not catch things that look like dates but aren’t valid.
Thanks
Shawn Weeks
From: Juan Pablo Gardella <[email protected]>
Sent: Thursday, October 18, 2018 11:03 AM
To: [email protected]
Subject: Re: [EXT] ReplaceText cannot consume messages if Regex does not match
At search value:(?s)(^.*$)
At Replacement value:
<?xml version="1.0" encoding="UTF-8"?>
<SettleDate>${actualSettlementDate:toDate('MM/dd/yyyy'):format("yyyy-MM-dd'T'00:00:00.0000000")}</SettleDate>
The actualSettlementDate is a flowfile attribute. The problem is the
replacement value is evaluated inside the processor and the toDate method fails.
Hope it's clear now.
On Thu, 18 Oct 2018 at 12:51 Shawn Weeks
<[email protected]<mailto:[email protected]>> wrote:
I’m still trying to understand your actual issue, can your provide a screenshot
of the ReplaceText config like the attached, I need to see exactly where you’re
putting the expression. A template would also be really helpful.
Thanks
Shawn Weeks
From: Juan Pablo Gardella
<[email protected]<mailto:[email protected]>>
Sent: Thursday, October 18, 2018 10:45 AM
To: [email protected]<mailto:[email protected]>
Subject: Re: [EXT] ReplaceText cannot consume messages if Regex does not match
At ReplaceText
<https://raw.githubusercontent.com/apache/nifi/master/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ReplaceText.java>
processor we have:
[image.png]
As you can see, only if StackOverflowError is raised during the evaluation, the
flowfile is send to failure relationship. I would like to update the code to
use Exception or NifiExpressionFailedException (if it exits).
Juan
On Thu, 18 Oct 2018 at 12:33 Shawn Weeks
<[email protected]<mailto:[email protected]>> wrote:
What processor are you defining your expression in? I also may be
misunderstanding the problem because I don’t see any regular expressions
anywhere. Can you create a sample workflow showing your issue so I can take a
look at it.
Thanks
Shawn Weeks
From: Juan Pablo Gardella
<[email protected]<mailto:[email protected]>>
Sent: Thursday, October 18, 2018 10:27 AM
To: [email protected]<mailto:[email protected]>
Subject: Re: [EXT] ReplaceText cannot consume messages if Regex does not match
No, it's not a valid date. I would like if it an error happens, I would like to
throw the flowfile to failure and continue.
On Thu, 18 Oct 2018 at 12:19 Shawn Weeks
<[email protected]<mailto:[email protected]>> wrote:
Any expression language syntax has to be correct or the processor won’t run.
I’m not sure there is any way to work around that except to explicitly check
that the value you are trying to evaluate is valid. Is the attribute
“tradeDate” coming from the contents of a flow file or is it defined somewhere
else. Can you ensure it is a valid date in that format before hand?
Thanks
Shawn Weeks
From: Juan Pablo Gardella
<[email protected]<mailto:[email protected]>>
Sent: Thursday, October 18, 2018 10:13 AM
To: [email protected]<mailto:[email protected]>
Subject: Re: [EXT] ReplaceText cannot consume messages if Regex does not match
Hi, the error is not in the processor itself. It's in the expression used
against flowfile attributes. For example inside the text, I have:
<EventDate>${tradeDate:toDate('MM/dd/yyyy'):format("yyyy-MM-dd'T'00:00:00.0000000")}</EventDate>
And that is the root issue. If it's unable to convert it, the flow cannot be
consumed. How can I evaluate attributes in a non-blocker way?
Juan
On Thu, 18 Oct 2018 at 12:07 Shawn Weeks
<[email protected]<mailto:[email protected]>> wrote:
Where is your expression? That’s not the entire configuration for that
processor.
Thanks
Shawn Weeks
From: Juan Pablo Gardella
<[email protected]<mailto:[email protected]>>
Sent: Thursday, October 18, 2018 10:03 AM
To: [email protected]<mailto:[email protected]>
Subject: Re: [EXT] ReplaceText cannot consume messages if Regex does not match
Configuration:
Replacement Strategy: Always replace
EvaluationMode: Entire text
On Thu, 18 Oct 2018 at 12:01 Juan Pablo Gardella
<[email protected]<mailto:[email protected]>> wrote:
Hortonworks nifi based on 1.5.0:
Configuration:
Thanks
On Thu, 18 Oct 2018 at 11:56 Peter Wicks (pwicks)
<[email protected]<mailto:[email protected]>> wrote:
Hi Juan,
What version of NiFi are you running on?
What mode are you running ReplaceText in, all text or line by line?
Other settings that might be important? What’s your RegEx look like (if your
able to share).
--Peter
From: Juan Pablo Gardella
[mailto:[email protected]<mailto:[email protected]>]
Sent: Thursday, October 18, 2018 8:53 AM
To: [email protected]<mailto:[email protected]>
Subject: [EXT] ReplaceText cannot consume messages if Regex does not match
Hi all,
I'm seeing that ReplaceText is not able to consume messages that does not match
regex. It keeps all the messages in the input queue instead of sending them to
failure relationship. Is this the intended behavior or I have to file a ticket
in order to be fixed? In that way, the processor is not able to process bad
messages and converts in the bottleneck of a flow
Juan
