[
https://issues.apache.org/jira/browse/NIFI-1842?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17941014#comment-17941014
]
David Handermann commented on NIFI-1842:
----------------------------------------
Thanks for the reply [~luigiba].
Although the approach in the pull request has some helpful scoping, it still
includes a large number of dependencies including:
- Groovy
- Netty
- Jackson
- JSON Path
- SnakeYAML
Not to mention the Camel libraries themselves.
The fact that the Processor calls the Camel Main class and waits for starting
is another red flag in terms of integration.
The proposed Processor also reads the entire FlowFile content into memory as a
ByteArrayOutputStream, which has the potential to create memory usage problems.
Several Processors require this behavior, but that is the exception, rather
than the rule.
These are just some of the concerns that appear on a general review of the
approach, beyond the remaining fundamental problems described earlier.
> implement GetWithCamel and PutWithCamel processors, as selectable-endpoint
> connectors
> -------------------------------------------------------------------------------------
>
> Key: NIFI-1842
> URL: https://issues.apache.org/jira/browse/NIFI-1842
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Extensions
> Reporter: Matthew Foley
> Priority: Major
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> While the SpringContextProcessor (NIFI-1571) provides an excellent basis for
> running Camel, it still requires the user to program Camel. In light of the
> fact that Camel provides some 80+ protocol-aware endpoints (see the list at
> https://github.com/apache/camel/tree/master/components), at least half of
> which are probably useful to NiFi and not yet available as NiFi-native
> processors, I propose the following:
> Create processors named GetWithCamel and PutWithCamel, that allow:
> * configuration-time selection from a subset of Camel-supported endpoints
> * and then providing the additional config values required by the selected
> endpoint, probably in an XML file
> These processors will of course be based on SpringContextProcessor, but
> hardwired for Camel and with added configuration glue in the UI to allow
> selecting a Camel endpoint without having to write Camel or Spring code. This
> won't be possible for all the endpoints, but will work for many useful ones.
> DESIGN ISSUES to be resolved (Thanks to [~joewitt],
> [[email protected]], [~ozhurakousky], for raising these items):
> * We want to add this feature without requiring that big blocks of Camel or
> Spring dependency code get sucked in to NiFi deployments that don't require
> them. Appropriate bundle isolation (at NiFi and possibly Camel levels), plus
> use of dynamic class loaders, are clearly part of the solution, but this
> strategy needs to be elaborated. The upcoming NiFi Extension Registry may
> also be part of the solution.
> * We want to avoid run-time, or even configuration-time, dependency
> resolution. While maven is a widely acknowledged utility for dependency
> resolution, it is not allowable in all environments. Resolving all
> dependencies into a metadata form at build time is preferable, and I believe
> achievable. A dependency resolver capable of downloading dependencies from
> maven repositories at config time, if not present in classpath, may still be
> useful as an option in some environments. The upcoming NiFi Extension
> Registry may also be part of the solution, if trusted in deployment
> environments.
> * The provenance-reporting feature of NiFi must be supported. How to
> integrate this with Camel end-points needs design work. It may need to be
> different for different end-points.
> * Effective code review will require patience, since Spring and Camel
> expertise may be less available in the community. Maintainability of the new
> processors must also be considered, given this limitation.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
