Agreed, it should absolutely be documented. I don’t have a particularly strong opinion regarding whether or not to log a warning.
Thanks -Mark > On Jul 31, 2024, at 1:00 PM, Gábor Gyimesi <gamezb...@gmail.com> wrote: > > Hi, > > MiNiFi C++ copied the behavior of the NiFi Python API to be compatible > with the NiFi python processors (although some minor differences may > apply [1]), so it works the same way in this case. > I think this behavior is understandable and can be accepted, but I > think Ferenc has a point that this at least should be documented in > the developer guide. The interface of the FlowFileTransformResult > suggests that the content can be specified for any relationship, but > it is discarded in this case without any notice. I suppose it would be > an overkill and not backwards compatible to change the API for adding > a failure specific transfer, but I think besides documenting this > behavior we could add a warning in the logs (or an exception?) if > someone specifies any content for this relationship. > > Regards, > Gábor > > [1] > https://github.com/apache/nifi-minifi-cpp/blob/main/extensions/python/PYTHON.md#using-nifi-python-processors > > On Wed, 31 Jul 2024 at 18:24, Mark Payne <marka...@hotmail.com> wrote: >> >> Hi Ferenc, >> >> I’m not overly familiar with MiNiFi C++ but this is how it works in >> traditional NiFi as well. >> This is intended. While Processors are generally free to treat relationships >> as they wish, >> the convention is always to route the original incoming FlowFile to >> ‘failure’, never a modified version of it. >> At least with respect to content. Some attributes might be added, for >> example to explain why there was a failure, >> but the content should not be changed. >> >> In the Python API we’ve generally moved toward being more prescriptive about >> some of these things. >> So while it’s possible to do so in Java, it should not be done. In Python, >> we don’t even allow it. >> >> Thanks >> -Mark >> >> >>> On Jul 31, 2024, at 10:46 AM, Ferenc Gerlits <fgerl...@apache.org> wrote: >>> >>> Hello NiFi community, >>> >>> We have implemented NiFi's FlowFileTransform API for Python processors >>> in MiNiFi C++, and we have written a few processors using it. We have >>> noticed that the "failure" relationship works like this: if the >>> transform() method returns >>> FlowFileTransformResult(relationship="failure", >>> contents=some_contents, attributes=some_attributes), then >>> - some_contents are discarded, >>> - some_attributes are merged with the attributes of the original flow file, >>> - and then the original flow file with the merged attributes is >>> routed to the failure relationship. [1] >>> >>> The fact that the contents are discarded was unexpected to me, and it >>> is not documented in the developer guide [2]. We could add this to the >>> developer guide, but I think it would make more sense if the failure >>> result worked in the same way as the success result: if contents is >>> None, then we use the original contents, but if contents is not None, >>> then it becomes the new contents of the flow file which is routed to >>> the failure relationship. >>> >>> What do you think? >>> Ferenc >>> >>> [1] >>> https://github.com/apache/nifi/blob/main/nifi-extension-bundles/nifi-py4j-bundle/nifi-py4j-bridge/src/main/java/org/apache/nifi/python/processor/FlowFileTransformProxy.java >>> [2] >>> https://github.com/apache/nifi/blob/main/nifi-docs/src/main/asciidoc/python-developer-guide.adoc >>
