Guys,
It seems that NIFI ReplaceTextWithMapping Processors has a BUG with
Refreshing its Mapped file. We are using its functionality in PROD and
getting odd behaviour.
Our USAGE Scenario:
- We use NIFI primarily as a TRANSFORMATION Tool.
- Our flow involves:
1. Getting a raw csv file.
2. Split the file on per line basis:
1. So from one source flow file - we may 10000 flows
generated/splitted out.
3. For each of the splitted flow file(flowfiles for individual lines)
we perform transformation on the attributes.
4. We merge these flowfiles back and write the Output file.
As part of the transformation in Step#4, we do some mapping for one of the
field in the csv. For this we use ReplaceTextWithMapping Processor. Also
to note we update ourmapping file just before starting our flow(ie. Step #1)
Our Issue:
- We have noted for SAME key we get two DIFFERENT values in two
different flowfile.
- We noted: that one the value mapped existed in older Mapping file.
- So in essence: ReplaceTextWithMapping Processor didn't refresh its
cash uptil certain time. And thus return old value for few mapping file and
then - once in the meanwhile it has refreshed it cache - returned new
mapped value.
- So this cause the issue.
Question:
- Is this a known issue with ReplaceTextWithMapping Processor?
- If not how can I create an issue for this?
- How can I confirm this behaviour?
Thanks,
Ameer Mawia
--
http://ca.linkedin.com/in/ameermawia
Toronto, ON
