GitHub user FavioVazquez opened a pull request:
https://github.com/apache/spark/pull/5783
[SPARK-7238] Update protobuf-java version of com.google.protobuf dependancy
This upgrade is needed when building spark for CDH5 2.5.0-cdh5.3.3 due to
incompatibilities in the protobuf version used by com.google.protobuf and the
one used in hadoop. The default version of protobuf is set to 2.4.1 in the
global properties, and this is stated in the pom.xml file:
<!-- In theory we need not directly depend on protobuf since Spark does not
directly use it. However, when building with Hadoop/YARN 2.2 Maven doesn't
correctly bump the protobuf version up from the one Mesos gives. For now we
include this variable to explicitly bump the version when building with YARN.
It would be nice to figure out why Maven can't resolve this correctly (like SBT
does). -->
So this upgrade will only be affecting the com.google.protobuf version of
java-protobuf. Tested for the Cloudera distribution 2.5.0-cdh5.3.3 using Mesos
0.22.0 in cluster mode.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/FavioVazquez/spark upgrade-protobuf-version
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/5783.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #5783
----
commit ec91ce3c405123818a4c56ef361d9cc82951677d
Author: FavioVazquez <[email protected]>
Date: 2015-04-29T17:58:09Z
- Updated protobuf-java version of com.google.protobuf dependancy to fix
blocking error when connecting to HDFS via the Hadoop Cloudera HDFS CDH5 (fix
for 2.5.0-cdh5.3.3 version)
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
