[jira] [Created] (NIFI-5144) TransformXml crashes creating CDATA (Saxon issue?)
Chris Green created NIFI-5144:
-
Summary: TransformXml crashes creating CDATA (Saxon issue?)
Key: NIFI-5144
URL: https://issues.apache.org/jira/browse/NIFI-5144
Project: Apache NiFi
Issue Type: Bug
Affects Versions: 1.6.0
Reporter: Chris Green
Hi all,
I've not submitted a Jira issue before, apologies if this is in the wrong place.
I'm attempting to transform XML containing CDATA sections, into another
structure but keeping the CDATA tags. A 3rd party XML transformer confirms the
XSLT I've got transforms the XML successfully, using xsl:output to specify
which output elements to create as CDATA.
Using the same XML and XSLT in NiFi 1.6.0 causes the following exception:
{{java.lang.IllegalArgumentException: Value of {cdata-section-elements} must be
a list of QNames in '\{uri}local' notation}}
The stack trace (below) mentions the Saxon library, which I've searched for
along with the exception and found the bug detailed
[here|https://saxonica.plan.io/issues/2795].
NiFi 1.6.0 appears to be bundled with Saxon-HE-9.6.0-5.jar, which is a version
from before the bug above was fixed (if they are related).
Full stack trace of the issue:
{{2018-05-01 09:01:18,861 ERROR [Timer-Driven Process Thread-2]
o.a.n.processors.standard.TransformXml
TransformXml[id=0ad735f3-2234-127c--38e8a135] Unable to transform
StandardFlowFileRecord[uuid=3054a4e2-b70a-4a22-8699-8dac755ebbaa,claim=StandardContentClaim
[resourceClaim=StandardResourceClaim[id=1525097440980-1,
container=stgkazoofi02_content1, section=1], offset=146962,
length=6091],offset=0,name=28527_58_20180428.xml,size=6091] due to
org.apache.nifi.processor.exception.ProcessException: IOException thrown from
TransformXml[id=0ad735f3-2234-127c--38e8a135]: java.io.IOException:
java.lang.IllegalArgumentException: Value of {cdata-section-elements} must be a
list of QNames in '\{uri}local' notation:
org.apache.nifi.processor.exception.ProcessException: IOException thrown from
TransformXml[id=0ad735f3-2234-127c--38e8a135]: java.io.IOException:
java.lang.IllegalArgumentException: Value of {cdata-section-elements} must be a
list of QNames in '\{uri}local' notation}}
{{org.apache.nifi.processor.exception.ProcessException: IOException thrown from
TransformXml[id=0ad735f3-2234-127c--38e8a135]: java.io.IOException:
java.lang.IllegalArgumentException: Value of {cdata-section-elements} must be a
list of QNames in '\{uri}local' notation}}
{{at
org.apache.nifi.controller.repository.StandardProcessSession.write(StandardProcessSession.java:2902)}}
{{at
org.apache.nifi.processors.standard.TransformXml.onTrigger(TransformXml.java:234)}}
{{at
org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)}}
{{at
org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1147)}}
{{at
org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:175)}}
{{at
org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:117)}}
{{at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)}}
{{at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)}}
{{at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)}}
{{at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)}}
{{at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)}}
{{at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)}}
{{at java.lang.Thread.run(Thread.java:748)}}
{{Caused by: java.io.IOException: java.lang.IllegalArgumentException: Value of
{cdata-section-elements} must be a list of QNames in '\{uri}local' notation}}
{{at
org.apache.nifi.processors.standard.TransformXml$2.process(TransformXml.java:261)}}
{{at
org.apache.nifi.controller.repository.StandardProcessSession.write(StandardProcessSession.java:2881)}}
{{... 12 common frames omitted}}
{{Caused by: java.lang.IllegalArgumentException: Value of
{cdata-section-elements} must be a list of QNames in '\{uri}local' notation}}
{{at net.sf.saxon.s9api.Serializer.setOutputProperty(Serializer.java:408)}}
{{at net.sf.saxon.jaxp.TransformerImpl.transform(TransformerImpl.java:138)}}
{{at
org.apache.nifi.processors.standard.TransformXml$2.process(TransformXml.java:259)}}
{{... 13 common frames omitted}}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (NIFI-5144) TransformXml crashes creating CDATA (Saxon issue?)
[
https://issues.apache.org/jira/browse/NIFI-5144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Green updated NIFI-5144:
--
Description:
Hi all,
I've not submitted a Jira issue before, apologies if this is in the wrong place.
I'm attempting to transform XML containing CDATA sections, into another
structure but keeping the CDATA tags. A 3rd party XML transformer confirms the
XSLT I've got transforms the XML successfully, using xsl:output to specify
which output elements to create as CDATA.
Using the same XML and XSLT in NiFi 1.6.0 causes the following exception:
{quote}java.lang.IllegalArgumentException: Value of {cdata-section-elements}
must be a list of QNames in '\{uri}local' notation
{quote}
The stack trace (attached) mentions the Saxon library, which I've searched for
along with the exception and found the bug detailed
[here|https://saxonica.plan.io/issues/2795].
NiFi 1.6.0 appears to be bundled with Saxon-HE-9.6.0-5.jar, which is a version
from before the bug above was fixed (if they are related).
was:
Hi all,
I've not submitted a Jira issue before, apologies if this is in the wrong place.
I'm attempting to transform XML containing CDATA sections, into another
structure but keeping the CDATA tags. A 3rd party XML transformer confirms the
XSLT I've got transforms the XML successfully, using xsl:output to specify
which output elements to create as CDATA.
Using the same XML and XSLT in NiFi 1.6.0 causes the following exception:
{{java.lang.IllegalArgumentException: Value of
{cdata-section-elements} must be a list of QNames in '\{uri}local' notation}}
The stack trace (attached) mentions the Saxon library, which I've searched for
along with the exception and found the bug detailed
[here|https://saxonica.plan.io/issues/2795].
NiFi 1.6.0 appears to be bundled with Saxon-HE-9.6.0-5.jar, which is a version
from before the bug above was fixed (if they are related).
> TransformXml crashes creating CDATA (Saxon issue?)
> --
>
> Key: NIFI-5144
> URL: https://issues.apache.org/jira/browse/NIFI-5144
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 1.6.0
>Reporter: Chris Green
>Priority: Major
> Labels: Processor, TransformXml
> Attachments: trace.txt
>
>
> Hi all,
> I've not submitted a Jira issue before, apologies if this is in the wrong
> place.
>
> I'm attempting to transform XML containing CDATA sections, into another
> structure but keeping the CDATA tags. A 3rd party XML transformer confirms
> the XSLT I've got transforms the XML successfully, using xsl:output to
> specify which output elements to create as CDATA.
> Using the same XML and XSLT in NiFi 1.6.0 causes the following exception:
>
>
> {quote}java.lang.IllegalArgumentException: Value of {cdata-section-elements}
> must be a list of QNames in '\{uri}local' notation
> {quote}
>
> The stack trace (attached) mentions the Saxon library, which I've searched
> for along with the exception and found the bug detailed
> [here|https://saxonica.plan.io/issues/2795].
> NiFi 1.6.0 appears to be bundled with Saxon-HE-9.6.0-5.jar, which is a
> version from before the bug above was fixed (if they are related).
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (NIFI-5144) TransformXml crashes creating CDATA (Saxon issue?)
[
https://issues.apache.org/jira/browse/NIFI-5144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Green updated NIFI-5144:
--
Attachment: trace.txt
Description:
Hi all,
I've not submitted a Jira issue before, apologies if this is in the wrong place.
I'm attempting to transform XML containing CDATA sections, into another
structure but keeping the CDATA tags. A 3rd party XML transformer confirms the
XSLT I've got transforms the XML successfully, using xsl:output to specify
which output elements to create as CDATA.
Using the same XML and XSLT in NiFi 1.6.0 causes the following exception:
{{java.lang.IllegalArgumentException: Value of
{cdata-section-elements} must be a list of QNames in '\{uri}local' notation}}
The stack trace (attached) mentions the Saxon library, which I've searched for
along with the exception and found the bug detailed
[here|https://saxonica.plan.io/issues/2795].
NiFi 1.6.0 appears to be bundled with Saxon-HE-9.6.0-5.jar, which is a version
from before the bug above was fixed (if they are related).
was:
Hi all,
I've not submitted a Jira issue before, apologies if this is in the wrong place.
I'm attempting to transform XML containing CDATA sections, into another
structure but keeping the CDATA tags. A 3rd party XML transformer confirms the
XSLT I've got transforms the XML successfully, using xsl:output to specify
which output elements to create as CDATA.
Using the same XML and XSLT in NiFi 1.6.0 causes the following exception:
{{java.lang.IllegalArgumentException: Value of {cdata-section-elements} must be
a list of QNames in '\{uri}local' notation}}
The stack trace (below) mentions the Saxon library, which I've searched for
along with the exception and found the bug detailed
[here|https://saxonica.plan.io/issues/2795].
NiFi 1.6.0 appears to be bundled with Saxon-HE-9.6.0-5.jar, which is a version
from before the bug above was fixed (if they are related).
Full stack trace of the issue:
{{2018-05-01 09:01:18,861 ERROR [Timer-Driven Process Thread-2]
o.a.n.processors.standard.TransformXml
TransformXml[id=0ad735f3-2234-127c--38e8a135] Unable to transform
StandardFlowFileRecord[uuid=3054a4e2-b70a-4a22-8699-8dac755ebbaa,claim=StandardContentClaim
[resourceClaim=StandardResourceClaim[id=1525097440980-1,
container=stgkazoofi02_content1, section=1], offset=146962,
length=6091],offset=0,name=28527_58_20180428.xml,size=6091] due to
org.apache.nifi.processor.exception.ProcessException: IOException thrown from
TransformXml[id=0ad735f3-2234-127c--38e8a135]: java.io.IOException:
java.lang.IllegalArgumentException: Value of {cdata-section-elements} must be a
list of QNames in '\{uri}local' notation:
org.apache.nifi.processor.exception.ProcessException: IOException thrown from
TransformXml[id=0ad735f3-2234-127c--38e8a135]: java.io.IOException:
java.lang.IllegalArgumentException: Value of {cdata-section-elements} must be a
list of QNames in '\{uri}local' notation}}
{{org.apache.nifi.processor.exception.ProcessException: IOException thrown from
TransformXml[id=0ad735f3-2234-127c--38e8a135]: java.io.IOException:
java.lang.IllegalArgumentException: Value of {cdata-section-elements} must be a
list of QNames in '\{uri}local' notation}}
{{at
org.apache.nifi.controller.repository.StandardProcessSession.write(StandardProcessSession.java:2902)}}
{{at
org.apache.nifi.processors.standard.TransformXml.onTrigger(TransformXml.java:234)}}
{{at
org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)}}
{{at
org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1147)}}
{{at
org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:175)}}
{{at
org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:117)}}
{{at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)}}
{{at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)}}
{{at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)}}
{{at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)}}
{{at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)}}
{{at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)}}
{{at java.lang.Thread.run(Thread.java:748)}}
{{Caused by: java.io.IOException: java.lang.IllegalArgumentException: Value of
{cdata-section-elements} must be a list of QNames in '\{uri}local' notation}}
{{at
org.apache.nifi.processors.standard.TransformXml$2.process(TransformXml.java:261)}}
{{at
org.apache.nifi.controller.repository.StandardProcessSession.write(StandardProcessSession.java:2881)}}
{{... 12 common frames omitted}}
{{Caused by:
[jira] [Updated] (NIFI-5144) TransformXml crashes creating CDATA (Saxon issue?)
[
https://issues.apache.org/jira/browse/NIFI-5144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Green updated NIFI-5144:
--
Description:
Hi all,
I've not submitted a Jira issue before, apologies if this is in the wrong place.
I'm attempting to transform XML containing CDATA sections, into another
structure but keeping the CDATA tags. A 3rd party XML transformer confirms the
XSLT I've got transforms the XML successfully, using xsl:output to specify
which output elements to create as CDATA.
Using the same XML and XSLT in NiFi 1.6.0 causes the following exception:
{quote}java.lang.IllegalArgumentException: Value of \{cdata-section-elements}
must be a list of QNames in '\{uri}local' notation{quote}
The stack trace (attached) mentions the Saxon library, which I've searched for
along with the exception and found the bug detailed
[here|https://saxonica.plan.io/issues/2795].
NiFi 1.6.0 appears to be bundled with Saxon-HE-9.6.0-5.jar, which is a version
from before the bug above was fixed (if they are related).
was:
Hi all,
I've not submitted a Jira issue before, apologies if this is in the wrong place.
I'm attempting to transform XML containing CDATA sections, into another
structure but keeping the CDATA tags. A 3rd party XML transformer confirms the
XSLT I've got transforms the XML successfully, using xsl:output to specify
which output elements to create as CDATA.
Using the same XML and XSLT in NiFi 1.6.0 causes the following exception:
{quote}java.lang.IllegalArgumentException: Value of {cdata-section-elements}
must be a list of QNames in '\{uri}local' notation
{quote}
The stack trace (attached) mentions the Saxon library, which I've searched for
along with the exception and found the bug detailed
[here|https://saxonica.plan.io/issues/2795].
NiFi 1.6.0 appears to be bundled with Saxon-HE-9.6.0-5.jar, which is a version
from before the bug above was fixed (if they are related).
> TransformXml crashes creating CDATA (Saxon issue?)
> --
>
> Key: NIFI-5144
> URL: https://issues.apache.org/jira/browse/NIFI-5144
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 1.6.0
>Reporter: Chris Green
>Priority: Major
> Labels: Processor, TransformXml
> Attachments: trace.txt
>
>
> Hi all,
> I've not submitted a Jira issue before, apologies if this is in the wrong
> place.
>
> I'm attempting to transform XML containing CDATA sections, into another
> structure but keeping the CDATA tags. A 3rd party XML transformer confirms
> the XSLT I've got transforms the XML successfully, using xsl:output to
> specify which output elements to create as CDATA.
> Using the same XML and XSLT in NiFi 1.6.0 causes the following exception:
>
> {quote}java.lang.IllegalArgumentException: Value of \{cdata-section-elements}
> must be a list of QNames in '\{uri}local' notation{quote}
>
> The stack trace (attached) mentions the Saxon library, which I've searched
> for along with the exception and found the bug detailed
> [here|https://saxonica.plan.io/issues/2795].
> NiFi 1.6.0 appears to be bundled with Saxon-HE-9.6.0-5.jar, which is a
> version from before the bug above was fixed (if they are related).
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (NIFI-5060) UpdateRecord substringAfter and substringAfterLast only increments by 1
Chris Green created NIFI-5060:
-
Summary: UpdateRecord substringAfter and substringAfterLast only
increments by 1
Key: NIFI-5060
URL: https://issues.apache.org/jira/browse/NIFI-5060
Project: Apache NiFi
Issue Type: Bug
Components: Extensions
Affects Versions: 1.6.0
Reporter: Chris Green
Attachments: Validate_substringafter_Behavior.xml
This is my first submitted issue, so please feel free to point me in the
correct direction if I make process mistakes.
Replication:
Drag a GenerateFlowFile onto the canvas and configure this property, and set
run schedule to some high value like 600 seconds
"Custom Text" \{"value": "01230123"}
Connect GenerateFlowFile with an UpdateAttribute set to add the attribute
"avro.schema" with a value of "{ "type": "record",
"name": "test",
"fields" : [\{"name": "value", "type": "string"}]
}"
Connect UpdateAttribute to an UpdateRecord onto the canvas, Autoterminate
success and failure. Set the Record Reader to a new JSONTreeReader. On the
JsonTreeReader configure it to use the "Use 'Schema Text' Attribute".
Create a JsonRecordSetWriter and set the Schema Text to:
"{ "type": "record",
"name": "test",
"fields" : [\{"name": "value", "type": "string"},
{"name": "example1", "type": "string"},
{"name": "example2", "type": "string"},
{"name": "example3", "type": "string"},
{"name": "example4", "type": "string"}]
}"
Add the following properties to UpdateRecord
||Heading 1||Heading 2||
|/example1|substringAfter(/value, "1") |
|/example2|substringAfter(/value, "123") |
|/example3|substringAfterLast(/value, "1")|
|/example4|substringAfterLast(/value, "123")|
Resulting record currently:
[ {
"value" : "01230123",
"example1" : "230123",
"example2" : "30123",
"example3" : "23",
"example4" : "3"
} ]
Problem:
When using the UpdateRecord processor, and the substringAfter() function after
the search phrase is found it will only increment the substring returned by 1
rather than the length of the search term.
Based off XPath and other implementations of substringAfter functions I've seen
the value returned should remove the search term rather than just the first
character of the search term.
Resulting record should be:
[ {
"value" : "01230123",
"example1" : "230123",
"example2" : "30123",
"example3" : "23",
"example4" : "3"
} ]
I'm cleaning up a fix with test code that will change the increment from 1 to
the length of the search terms.
It appears substringBefore are not impacted by the behavior as always returns
the index before the found search term which is the expected behavior
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (NIFI-5060) UpdateRecord substringAfter and substringAfterLast only increments by 1
[
https://issues.apache.org/jira/browse/NIFI-5060?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16432226#comment-16432226
]
Chris Green commented on NIFI-5060:
---
Good catch, I corrected the description and put them in codeblocks so its a
little easier to read.
> UpdateRecord substringAfter and substringAfterLast only increments by 1
> ---
>
> Key: NIFI-5060
> URL: https://issues.apache.org/jira/browse/NIFI-5060
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Affects Versions: 1.6.0
>Reporter: Chris Green
>Priority: Major
> Labels: easyfix, newbie
> Attachments: Validate_substringafter_Behavior.xml
>
>
> This is my first submitted issue, so please feel free to point me in the
> correct direction if I make process mistakes.
> Replication:
> Drag a GenerateFlowFile onto the canvas and configure this property, and set
> run schedule to some high value like 600 seconds
> "Custom Text" \{"value": "01230123"}
> Connect GenerateFlowFile with an UpdateAttribute set to add the attribute
> "avro.schema" with a value of
>
> {code:java}
> {
> "type": "record",
> "name": "test",
> "fields" : [{"name": "value", "type": "string"}]
> }
> {code}
>
>
> Connect UpdateAttribute to an UpdateRecord onto the canvas, Autoterminate
> success and failure. Set the Record Reader to a new JSONTreeReader. On the
> JsonTreeReader configure it to use the "Use 'Schema Text' Attribute".
> Create a JsonRecordSetWriter and set the Schema Text to:
>
>
> {code:java}
> {
> "type": "record",
> "name": "test",
> "fields" : [
> {"name": "value", "type": "string"},
> {"name": "example1", "type": "string"},
> {"name": "example2", "type": "string"},
> {"name": "example3", "type": "string"},
> {"name": "example4", "type": "string"}
> ]
> }
> {code}
>
> Add the following properties to UpdateRecord
>
> ||Heading 1||Heading 2||
> |/example1|substringAfter(/value, "1") |
> |/example2|substringAfter(/value, "123") |
> |/example3|substringAfterLast(/value, "1")|
> |/example4|substringAfterLast(/value, "123")|
>
> Resulting record currently:
>
> {code:java}
> [{
> "value" : "01230123",
> "example1" : "230123",
> "example2" : "30123",
> "example3" : "23",
> "example4" : "3"
> }]
> {code}
>
>
>
> Problem:
> When using the UpdateRecord processor, and the substringAfter() function
> after the search phrase is found it will only increment the substring
> returned by 1 rather than the length of the search term.
> Based off XPath and other implementations of substringAfter functions I've
> seen the value returned should remove the search term rather than just the
> first character of the search term.
>
>
> Resulting record should be:
>
> {code:java}
> [{
> "value" : "01230123",
> "example1" : "230123",
> "example2" : "0123",
> "example3" : "23",
> "example4" : ""
> }]
> {code}
>
>
> I'm cleaning up a fix with test code that will change the increment from 1 to
> the length of the search terms.
> It appears substringBefore are not impacted by the behavior as always returns
> the index before the found search term which is the expected behavior
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (NIFI-5060) UpdateRecord substringAfter and substringAfterLast only increments by 1
[
https://issues.apache.org/jira/browse/NIFI-5060?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Green updated NIFI-5060:
--
Description:
This is my first submitted issue, so please feel free to point me in the
correct direction if I make process mistakes.
Replication:
Drag a GenerateFlowFile onto the canvas and configure this property, and set
run schedule to some high value like 600 seconds
"Custom Text" \{"value": "01230123"}
Connect GenerateFlowFile with an UpdateAttribute set to add the attribute
"avro.schema" with a value of
{code:java}
{
"type": "record",
"name": "test",
"fields" : [{"name": "value", "type": "string"}]
}
{code}
Connect UpdateAttribute to an UpdateRecord onto the canvas, Autoterminate
success and failure. Set the Record Reader to a new JSONTreeReader. On the
JsonTreeReader configure it to use the "Use 'Schema Text' Attribute".
Create a JsonRecordSetWriter and set the Schema Text to:
{code:java}
{
"type": "record",
"name": "test",
"fields" : [
{"name": "value", "type": "string"},
{"name": "example1", "type": "string"},
{"name": "example2", "type": "string"},
{"name": "example3", "type": "string"},
{"name": "example4", "type": "string"}
]
}
{code}
Add the following properties to UpdateRecord
||Heading 1||Heading 2||
|/example1|substringAfter(/value, "1") |
|/example2|substringAfter(/value, "123") |
|/example3|substringAfterLast(/value, "1")|
|/example4|substringAfterLast(/value, "123")|
Resulting record currently:
{code:java}
[{
"value" : "01230123",
"example1" : "230123",
"example2" : "30123",
"example3" : "23",
"example4" : "3"
}]
{code}
Problem:
When using the UpdateRecord processor, and the substringAfter() function after
the search phrase is found it will only increment the substring returned by 1
rather than the length of the search term.
Based off XPath and other implementations of substringAfter functions I've seen
the value returned should remove the search term rather than just the first
character of the search term.
Resulting record should be:
{code:java}
[{
"value" : "01230123",
"example1" : "230123",
"example2" : "0123",
"example3" : "23",
"example4" : ""
}]
{code}
I'm cleaning up a fix with test code that will change the increment from 1 to
the length of the search terms.
It appears substringBefore are not impacted by the behavior as always returns
the index before the found search term which is the expected behavior
was:
This is my first submitted issue, so please feel free to point me in the
correct direction if I make process mistakes.
Replication:
Drag a GenerateFlowFile onto the canvas and configure this property, and set
run schedule to some high value like 600 seconds
"Custom Text" \{"value": "01230123"}
Connect GenerateFlowFile with an UpdateAttribute set to add the attribute
"avro.schema" with a value of "{ "type": "record",
"name": "test",
"fields" : [\{"name": "value", "type": "string"}]
}"
Connect UpdateAttribute to an UpdateRecord onto the canvas, Autoterminate
success and failure. Set the Record Reader to a new JSONTreeReader. On the
JsonTreeReader configure it to use the "Use 'Schema Text' Attribute".
Create a JsonRecordSetWriter and set the Schema Text to:
"{ "type": "record",
"name": "test",
"fields" : [\{"name": "value", "type": "string"},
{"name": "example1", "type": "string"},
{"name": "example2", "type": "string"},
{"name": "example3", "type": "string"},
{"name": "example4", "type": "string"}]
}"
Add the following properties to UpdateRecord
||Heading 1||Heading 2||
|/example1|substringAfter(/value, "1") |
|/example2|substringAfter(/value, "123") |
|/example3|substringAfterLast(/value, "1")|
|/example4|substringAfterLast(/value, "123")|
Resulting record currently:
[ {
"value" : "01230123",
"example1" : "230123",
"example2" : "30123",
"example3" : "23",
"example4" : "3"
} ]
Problem:
When using the UpdateRecord processor, and the substringAfter() function after
the search phrase is found it will only increment the substring returned by 1
rather than the length of the search term.
Based off XPath and other implementations of substringAfter functions I've seen
the value returned should remove the search term rather than just the first
character of the search term.
Resulting record should be:
[ {
"value" : "01230123",
"example1" : "230123",
"example2" : "30123",
"example3" : "23",
"example4" : "3"
} ]
I'm cleaning up a fix with test code that will change the increment from 1 to
the length of the search terms.
It appears substringBefore are not impacted by the behavior as always returns
the index before the found search term which is the expected behavior
> UpdateRecord substringAfter and substringAfterLast only increments by 1
>
