Github user mccheah commented on the pull request:
https://github.com/apache/spark/pull/3130#issuecomment-62070813
I'm pretty sure this doesn't work when Spark is built with maven. I'm going
to try with sbt, but this is what I've found so far.
I used make-distribution.sh to build this change on top of Spark 1.1
(master currently has other conflicts with our project), and built against
Hadoop 2.0.0-mr1-cdh4.6.0. Used mvn install to generate the spark-core jar,
then pointed my project at the built jar in my local maven repository.
I run my application and end up getting "unread block data" errors on Spark
jobs.
I see that the only pom.xml file that was modified on the first commit was
under assembly. How can this shading be picked up when built with maven, then?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
