[
https://issues.apache.org/jira/browse/NIFI-238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14276076#comment-14276076
]
Joseph Witt commented on NIFI-238:
----------------------------------
Ryan
Ok so this is good stuff to discuss.
Processor Yielding: The purpose of yielding a processor is that there is
something environmental going on like a configuration file that isn't present
yet or a connection that isn't available or something that is generally
considered to be transient and by that processor doing nothing the issue may
resolve itself.
Penalization of a flow file: If the processor gets a flow file that has some
condition of itself and the thing the processor wants to do which is considered
transient then it could penalize the flow file. This way the processor is free
to keep running on other flow files . An example of where penalizaton is
appropriate would be in PutSFTP where when attemtping to send a flow file with
the same name of some file already present on the remote end then it needs to
wait a while to try again and hopefully that conflict will resolve itself.
Relying on rollback: Rollback generally exists to protect the framework from
what are largely programming issues (for the most part) in a given processor.
Rollback is the "what we will do if some error condition occurs but isn't
handled by the processor". But in general these error conditions are something
a processor implementor should deal with (route things to a specific
relationship for the condition, penalize, yield). There are cases like
OutofMemoryErrors that could bite a processor for no fault of its own and thus
rollback is nice there too.
Routing to failure (or something appropriate for that processor): In general a
processor encounters a flow file and does whatever that processor is designed
to do. If the flowfile it gets is not fit for the preconditions of that
processor then routing to a relationship called 'failure' would be reasonable.
The key is for that processor to document which relationships it has and for
what purposes they receive data. That way someone composing a flow can come up
with cool and creative ways to put things together based on their context and
needs.
All things we need to do a great job of describing in a developer guide.
Thanks
Joe
> Add processors to write datasets using Kite
> -------------------------------------------
>
> Key: NIFI-238
> URL: https://issues.apache.org/jira/browse/NIFI-238
> Project: Apache NiFi
> Issue Type: New Feature
> Components: Extensions
> Reporter: Ryan Blue
>
> I think it would be great to have a set of processors that parse incoming
> flow files and add the data to Kite datasets.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
