[
https://issues.apache.org/jira/browse/KAFKA-3487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15338920#comment-15338920
]
Randall Hauch commented on KAFKA-3487:
--------------------------------------
[~ewencp] wrote:
{quote}
There's lots of other stuff you can do in this area too, e.g. URLClassLaoders
that can just pull the jars from the network, or even more awesome would be a
version that works from Central/Maven repos and can just auto-resolve all
transitive dependencies in some variant of a URLClassLoader and allow all the
connectors to be loaded completely dynamically (no more restarts to install new
connectors!)
{quote}
Autoresolving classloaders work fine in many situations, but not all. Some
environments do not have access to a Maven class loader, and it's not always
repeatable/reliable. Yet not having to restart to install new connectors is
still doable even with file-based JARs.
> Support per-connector/per-task classloaders in Connect
> ------------------------------------------------------
>
> Key: KAFKA-3487
> URL: https://issues.apache.org/jira/browse/KAFKA-3487
> Project: Kafka
> Issue Type: New Feature
> Components: KafkaConnect
> Affects Versions: 0.10.0.0
> Reporter: Ewen Cheslack-Postava
> Assignee: Liquan Pei
> Priority: Critical
> Labels: needs-kip
>
> Currently we just use the default ClassLoader in Connect. However, this
> limits how we can compatibly load conflicting connector plugins. Ideally we
> would use a separate class loader per connector/task that is instantiated to
> avoid potential conflicts.
> Note that this also opens up options for other ways to provide jars to
> instantiate connectors. For example, Spark uses this to dynamically publish
> classes defined in the REPL and load them via URL:
> https://ardoris.wordpress.com/2014/03/30/how-spark-does-class-loading/ But
> much simpler examples (include URL in the connector class instead of just
> class name) are also possible and could be a nice way to more support dynamic
> sets of connectors, multiple versions of the same connector, etc.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
