[
https://issues.apache.org/jira/browse/NIFI-589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14604515#comment-14604515
]
ASF GitHub Bot commented on NIFI-589:
-------------------------------------
Github user joewitt commented on the pull request:
https://github.com/apache/incubator-nifi/pull/51#issuecomment-116190999
Should we move ahead to Flume 1.6.0 now? Appears to still compile/test/run
fine with that.
nifi/pom.xml Formatting issue with derby dependency. Tabs gone wild. Fix
dependency ref to 0.2.0
nifi/nifi-nar-bundles/nifi-flume-bundle/pom.xml Fix dependency ref to 0.2.0
Two easy check style violations to resolve:
WARNING]
src/main/java/org/apache/nifi/processors/flume/FlumeSinkProcessor.java[24:8]
(imports) UnusedImports: Unused import - org.apache.flume.Context.
[WARNING]
src/main/java/org/apache/nifi/processors/flume/util/FlowFileEvent.java[30]
(imports) AvoidStarImport: Using the '.*' form of import should be avoided -
org.apache.nifi.processors.flume.util.FlowFileEventConstants.*.
Need to establish a LICENSE and NOTICE within the nifi-flume-nar project so
that it will end up that the bar binary will have proper accounting for bundled
items and need to roll up any unique dependencies into the nifi-assembly
LICENSE and NOTICE. I reviewed all bundled dependencies and they all appear to
be legit so doing this will be tedious but not problematic. I am happy to do
this for you. There are lots of good examples of how to do this in the other
nar bundles and we’ve augmented ASF guidance for handling licensing here;
http://nifi.incubator.apache.org/licensing-guide.html
The various Flume processors within the Nar appear to have a significant
version conflict for Netty. Ranging from wanting 3.5.x to 3.8.x of Netty.
Unclear if this is important so just bringing to your attention.
The bundled dependencies include a rather old version of Jetty. I am not
clear if this will conflict with the Jetty libraries we use and a quick scan of
our docs suggests we don’t point out the limitations of Nar class loader
isolation (will make a ticket). You may have no action here just bringing it
to your attention.
Recommend renaming the processors to ‘ExecuteFlumeSource’ and
‘ExecuteFlumeSink’. The naming structure we try to follow is generally
verb-subject. I did not find that we had this documented (will make a ticket).
This processor is a perfect case for providing some ‘additional details’
documentation so that when someone looks up the usage it will show them how to
configure a fully working example.
http://nifi.incubator.apache.org/docs/nifi-docs/html/developer-guide.html#advanced-documentation
Is there any service provider construct or similar mechanism whereby we
could provide a dropdown of identified sources and sinks so the user doesn’t
have to type them by hand? Not critical just curious. Given that the other
property is pasting of a flume properties section this is already pretty
‘developer heavy’ anyway so it isn’t necessary. But if possible it could be a
nice convenience to offer.
FlumeSinkProcessor: I think because channel.setSession(…) is being called
as it is during the onTrigger method this means the processor needs to run
serially. If multiple threads are running then the channel’s session would get
overwritten potentially creating some very interesting threading issues most
likely. Is marking it is as serial only going to be sufficient? We have an
annotation for that if so which will have the framework ensure it only gets a
single thread for the processor. I think a similar issue is present on
FlumeSourceProc.
If you happen to have a template of a flow that can be used to
test/demonstrate its function that would be a great thing to have. If so
please add to the JIRA.
Looking forward to seeing this merged in. Let me know if you’d like to
address these or discuss them further.
Thanks
Joe
> Add processors that can run Apache Flume sources/sinks
> ------------------------------------------------------
>
> Key: NIFI-589
> URL: https://issues.apache.org/jira/browse/NIFI-589
> Project: Apache NiFi
> Issue Type: New Feature
> Components: Extensions
> Affects Versions: 0.1.0
> Reporter: Joey Echeverria
> Assignee: Joey Echeverria
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
