Github user joewitt commented on the pull request:
https://github.com/apache/incubator-nifi/pull/51#issuecomment-116190999
Should we move ahead to Flume 1.6.0 now? Appears to still compile/test/run
fine with that.
nifi/pom.xml Formatting issue with derby dependency. Tabs gone wild. Fix
dependency ref to 0.2.0
nifi/nifi-nar-bundles/nifi-flume-bundle/pom.xml Fix dependency ref to 0.2.0
Two easy check style violations to resolve:
WARNING]
src/main/java/org/apache/nifi/processors/flume/FlumeSinkProcessor.java[24:8]
(imports) UnusedImports: Unused import - org.apache.flume.Context.
[WARNING]
src/main/java/org/apache/nifi/processors/flume/util/FlowFileEvent.java[30]
(imports) AvoidStarImport: Using the '.*' form of import should be avoided -
org.apache.nifi.processors.flume.util.FlowFileEventConstants.*.
Need to establish a LICENSE and NOTICE within the nifi-flume-nar project so
that it will end up that the bar binary will have proper accounting for bundled
items and need to roll up any unique dependencies into the nifi-assembly
LICENSE and NOTICE. I reviewed all bundled dependencies and they all appear to
be legit so doing this will be tedious but not problematic. I am happy to do
this for you. There are lots of good examples of how to do this in the other
nar bundles and weâve augmented ASF guidance for handling licensing here;
http://nifi.incubator.apache.org/licensing-guide.html
The various Flume processors within the Nar appear to have a significant
version conflict for Netty. Ranging from wanting 3.5.x to 3.8.x of Netty.
Unclear if this is important so just bringing to your attention.
The bundled dependencies include a rather old version of Jetty. I am not
clear if this will conflict with the Jetty libraries we use and a quick scan of
our docs suggests we donât point out the limitations of Nar class loader
isolation (will make a ticket). You may have no action here just bringing it
to your attention.
Recommend renaming the processors to âExecuteFlumeSourceâ and
âExecuteFlumeSinkâ. The naming structure we try to follow is generally
verb-subject. I did not find that we had this documented (will make a ticket).
This processor is a perfect case for providing some âadditional
detailsâ documentation so that when someone looks up the usage it will show
them how to configure a fully working example.
http://nifi.incubator.apache.org/docs/nifi-docs/html/developer-guide.html#advanced-documentation
Is there any service provider construct or similar mechanism whereby we
could provide a dropdown of identified sources and sinks so the user doesnât
have to type them by hand? Not critical just curious. Given that the other
property is pasting of a flume properties section this is already pretty
âdeveloper heavyâ anyway so it isnât necessary. But if possible it could
be a nice convenience to offer.
FlumeSinkProcessor: I think because channel.setSession(â¦) is being called
as it is during the onTrigger method this means the processor needs to run
serially. If multiple threads are running then the channelâs session would
get overwritten potentially creating some very interesting threading issues
most likely. Is marking it is as serial only going to be sufficient? We have
an annotation for that if so which will have the framework ensure it only gets
a single thread for the processor. I think a similar issue is present on
FlumeSourceProc.
If you happen to have a template of a flow that can be used to
test/demonstrate its function that would be a great thing to have. If so
please add to the JIRA.
Looking forward to seeing this merged in. Let me know if youâd like to
address these or discuss them further.
Thanks
Joe
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
