[
https://issues.apache.org/jira/browse/FLUME-2921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15369983#comment-15369983
]
Mike Percy commented on FLUME-2921:
-----------------------------------
Moving some of this discussion off of reviewboard to make it easier to follow.
My concern about this update is that ES exposes Guava as a dependency and
requires Guava 18. This patch updates Guava to a version that is incompatible
with Hadoop (Hadoop is on Guava 11). Most Flume sinks inherit Hadoop
dependencies. Therefore HBase, Hive, etc inherit that as well. Guava 15 removed
some classes.
Flume's use of Guava itself doesn't concern me. It's the lack of compatibility
and undefined behavior that would result from putting different versions of
Guava in the same classpath. Flume ships all of its deps together, in the same
classpath.
In short, we can't upgrade ElasticSearch to a version that requires Guava > 11
unless a solution to this issue is found.
The most obvious fix (this is a lot of work) is to refactor Flume's plugin
architecture to support something like OSGI or Spring DM so that individual
sources and sinks get their own classloaders. This solves the dependency
problem but introduces new problems, such as debuggability.
There may be other solutions to this problem. I'm not sure.
Another approach is to delay this upgrade until ES 3.0, when Guava support is
apparently dropped. Or maybe an older version of the ES client, like 1.x,
depends on an old version of Guava.
> Support Elasticsearch 2.0+
> --------------------------
>
> Key: FLUME-2921
> URL: https://issues.apache.org/jira/browse/FLUME-2921
> Project: Flume
> Issue Type: Improvement
> Components: Sinks+Sources
> Reporter: Lior Zeno
> Assignee: Lior Zeno
> Fix For: v1.7.0
>
> Attachments: FLUME-2921-0.patch, FLUME-2921-1.patch
>
>
> Elasticsearch sink supports an ancient version of ES. We should make the sink
> work with newer versions of Elasticsearch.
> I attached a patch for that. Please note that this involves upgrading netty
> and guava.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
