[
https://issues.apache.org/jira/browse/MESOS-1203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13975764#comment-13975764
]
Bernardo Gomez Palacio commented on MESOS-1203:
-----------------------------------------------
I tried the two jar approach over the weekend and think that it
adds unecessary comllexity to the user.
Users will need to know which qualified jar to use. By default we can
qualify only the versions of the Mesos jars that use older protobufs but it
is confussing any way.
Lets say I deploy Mesos 0.17.0 and I am using HDFS 1.0.4. I will have to
explicitly change my Applications, lets say Spark, and Frameworks to
specifically use the Mesos 0.17.0 jar built for protobufs 2.4.1.
This is very confussing for users that dont even have a clue that Mesos and
HDFS use protobufs (Hadoop since v 1.0.x and above). It is also
inconvinetnt for Frameworks/Application developers. For example in Spark we
will have to resolve the Mesos qualifier according the HDFS version we are
using.
Shading the protobufs inside the jar will avoid this problem. I will soon
have this tested on Mesos 0.16.0 using HDFS 1.0.4 & 2.3.0 with Spark
1.0-master.
At first I thought the two jar aproach was good but now think that shading
is the best option.
> Shade protobuf dependency in Mesos Java library
> -----------------------------------------------
>
> Key: MESOS-1203
> URL: https://issues.apache.org/jira/browse/MESOS-1203
> Project: Mesos
> Issue Type: Improvement
> Components: build
> Reporter: Patrick Wendell
>
> Mesos's Java library uses the protobuf library which is also used by Hadoop.
> Unfortunately the protobuf library does not provide binary compatiblity
> between minor versions (for code compiled against 2.4.1 and 2.5.0 cannot run
> together in a single JVM classlaoder) .
> This makes use of Mesos via it's Java API, something that is required for
> Spark and I'm assuming other frameworks, fundamentally incompatible for
> certain Hadoop versions.
> Mesos could shade this jar using the maven shade plug-in. Take a look at the
> Parquet project for an example of shading:
> https://github.com/Parquet/parquet-format/blob/master/pom.xml#L140
> Without this fix Java users won't be able to use Mesos (< 0.17) with newer
> versions of Hadoop. Or Mesos 0.17+ with older versions of Hadoop.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
