I think good APIs are easy to use correctly and hard to use incorrectly.
(I stole this from C++ books.)
For this reason, I'm +1 for throwing an exception when the API user
specifies a content with a transform result headed to failure. Also +1
for making this clear in the docs.
If an exception is not possible or feasible, an error log works, too.
But we should make it as loud as feasible, so the user notices the
programming error early.
Thanks,
Marton
On 7/31/24 19:52, Mark Payne wrote:
Agreed, it should absolutely be documented. I don’t have a particularly strong
opinion regarding whether or not to log a warning.
Thanks
-Mark
On Jul 31, 2024, at 1:00 PM, Gábor Gyimesi <gamezb...@gmail.com> wrote:
Hi,
MiNiFi C++ copied the behavior of the NiFi Python API to be compatible
with the NiFi python processors (although some minor differences may
apply [1]), so it works the same way in this case.
I think this behavior is understandable and can be accepted, but I
think Ferenc has a point that this at least should be documented in
the developer guide. The interface of the FlowFileTransformResult
suggests that the content can be specified for any relationship, but
it is discarded in this case without any notice. I suppose it would be
an overkill and not backwards compatible to change the API for adding
a failure specific transfer, but I think besides documenting this
behavior we could add a warning in the logs (or an exception?) if
someone specifies any content for this relationship.
Regards,
Gábor
[1]
https://github.com/apache/nifi-minifi-cpp/blob/main/extensions/python/PYTHON.md#using-nifi-python-processors
On Wed, 31 Jul 2024 at 18:24, Mark Payne <marka...@hotmail.com> wrote:
Hi Ferenc,
I’m not overly familiar with MiNiFi C++ but this is how it works in traditional
NiFi as well.
This is intended. While Processors are generally free to treat relationships as
they wish,
the convention is always to route the original incoming FlowFile to ‘failure’,
never a modified version of it.
At least with respect to content. Some attributes might be added, for example
to explain why there was a failure,
but the content should not be changed.
In the Python API we’ve generally moved toward being more prescriptive about
some of these things.
So while it’s possible to do so in Java, it should not be done. In Python, we
don’t even allow it.
Thanks
-Mark
On Jul 31, 2024, at 10:46 AM, Ferenc Gerlits <fgerl...@apache.org> wrote:
Hello NiFi community,
We have implemented NiFi's FlowFileTransform API for Python processors
in MiNiFi C++, and we have written a few processors using it. We have
noticed that the "failure" relationship works like this: if the
transform() method returns
FlowFileTransformResult(relationship="failure",
contents=some_contents, attributes=some_attributes), then
- some_contents are discarded,
- some_attributes are merged with the attributes of the original flow file,
- and then the original flow file with the merged attributes is
routed to the failure relationship. [1]
The fact that the contents are discarded was unexpected to me, and it
is not documented in the developer guide [2]. We could add this to the
developer guide, but I think it would make more sense if the failure
result worked in the same way as the success result: if contents is
None, then we use the original contents, but if contents is not None,
then it becomes the new contents of the flow file which is routed to
the failure relationship.
What do you think?
Ferenc
[1]
https://github.com/apache/nifi/blob/main/nifi-extension-bundles/nifi-py4j-bundle/nifi-py4j-bridge/src/main/java/org/apache/nifi/python/processor/FlowFileTransformProxy.java
[2]
https://github.com/apache/nifi/blob/main/nifi-docs/src/main/asciidoc/python-developer-guide.adoc