[
https://issues.apache.org/jira/browse/SPARK-4123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14335637#comment-14335637
]
Brennon York commented on SPARK-4123:
-------------------------------------
Gents, need a bit of input on this one. Looks like, to get the expected output
that [~pwendell] showed, I would need to run a {{mvn clean ... install}} before
I could do any sub-project traversal ([SO
reference|https://stackoverflow.com/questions/12223754/how-to-maven-sub-projects-with-inter-dependencies]
with similar issue). I've seen this issue in
[SPARK-3355|https://github.com/apache/spark/pull/4734] as well and, in that
case, just avoided it. I've retested on my machine by running the above lines
without installation, then with installation, then clearing org/apache/spark
from my ~/.m2 directory and without again. As expected, it failed, worked, and
then failed again. So... my question is how does Jenkins store state wrt the
local repository? I'm wondering if there might not be another way to grab this
information, but if we use Maven with the above command and we need to install
spark into a local directory I can imagine build failures everywhere, esp. if
we clear / update that directory.
Thoughts?
cc [~shaneknapp] for possible Jenkins advice
> Show new dependencies added in pull requests
> --------------------------------------------
>
> Key: SPARK-4123
> URL: https://issues.apache.org/jira/browse/SPARK-4123
> Project: Spark
> Issue Type: Improvement
> Components: Project Infra
> Reporter: Patrick Wendell
> Priority: Critical
>
> We should inspect the classpath of Spark's assembly jar for every pull
> request. This only takes a few seconds in Maven and it will help weed out
> dependency changes from the master branch. Ideally we'd post any dependency
> changes in the pull request message.
> {code}
> $ mvn -Phive -Phadoop-2.4 dependency:build-classpath -pl assembly | grep -v
> INFO | tr : "\n" | awk -F/ '{print $NF}' | sort > my-classpath
> $ git checkout apache/master
> $ mvn -Phive -Phadoop-2.4 dependency:build-classpath -pl assembly | grep -v
> INFO | tr : "\n" | awk -F/ '{print $NF}' | sort > master-classpath
> $ diff my-classpath master-classpath
> < chill-java-0.3.6.jar
> < chill_2.10-0.3.6.jar
> ---
> > chill-java-0.5.0.jar
> > chill_2.10-0.5.0.jar
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
