Here's one of the types of exceptions I get (this one when running VersionsSuite from sql/hive):
[info] - 13: create client *** FAILED *** (1 second, 946 milliseconds) [info] java.lang.RuntimeException: [download failed: org.apache.httpcomponents#httpclient;4.2.5!httpclient.jar, download failed: commons-codec#commons-codec;1.4!commons-codec.jar] [info] at org.apache.spark.deploy.SparkSubmitUtils$.resolveMavenCoordinates(SparkSubmit.scala:978) This is the content of the ivy metadata file for that component: #ivy cached data file for org.apache.httpcomponents#httpclient;4.2.5 #Thu Jun 04 13:26:10 PDT 2015 artifact\:ivy\#ivy\#xml\#1855381640.is-local=true artifact\:ivy\#ivy\#xml\#1855381640.location=file\:/home/vanzin/.m2/repository/org/apache/httpcomponents/httpclient/4.2.5/httpclient-4.2.5.pom artifact\:ivy\#ivy\#xml\#1855381640.exists=true resolver=local-m2-cache artifact\:httpclient\#pom.original\#pom\#-365933676.original=artifact\:httpclient\#pom.original\#pom\#-365933676 artifact\:ivy\#ivy\#xml\#1855381640.original=artifact\:httpclient\#pom.original\#pom\#-365933676 artifact.resolver=local-m2-cache artifact\:httpclient\#pom.original\#pom\#-365933676.is-local=true artifact\:httpclient\#pom.original\#pom\#-365933676.location=file\:/home/vanzin/.m2/repository/org/apache/httpcomponents/httpclient/4.2.5/httpclient-4.2.5.pom artifact\:httpclient\#pom.original\#pom\#-365933676.exists=true If I delete that file *and* the maven copy of those artifacts, then the tests pass. But that's really annoying, since I have to use sbt and maven for different things and I really like the fact that sbt can read the maven cache directly. On Thu, Jun 4, 2015 at 10:23 AM, shane knapp <skn...@berkeley.edu> wrote: > interesting... i definitely haven't seen it happen that often in our > build system, and when it has happened, i wasn't able to determine the > cause. > > On Thu, Jun 4, 2015 at 10:16 AM, Marcelo Vanzin <van...@cloudera.com> > wrote: > >> On Thu, Jun 4, 2015 at 10:04 AM, shane knapp <skn...@berkeley.edu> wrote: >> >>> this has occasionally happened on our jenkins as well (twice since last >>> august), and deleting the cache fixes it right up. >>> >> >> Yes deleting the cache fixes things, but it's kinda annoying to have to >> do that. And yesterday when I was testing a patch that actually used the >> ivy feature, I had to do that multiple times... that slows things down a >> lot. >> >> >>> >>> On Thu, Jun 4, 2015 at 4:29 AM, Sean Owen <so...@cloudera.com> wrote: >>> >>>> I've definitely seen the "dependency path must be relative" problem, >>>> and fixed it by deleting the ivy cache, but I don't know more than >>>> this. >>>> >>>> On Thu, Jun 4, 2015 at 1:33 AM, Marcelo Vanzin <van...@cloudera.com> >>>> wrote: >>>> > Hey all, >>>> > >>>> > I've been bit by something really weird lately and I'm starting to >>>> think >>>> > it's related to the ivy support we have in Spark, and running unit >>>> tests >>>> > that use that code. >>>> > >>>> > The first thing that happens is that after running unit tests, >>>> sometimes my >>>> > sbt builds start failing with error saying something about >>>> "dependency path >>>> > must be relative" (sorry, don't have the exact error around). The >>>> dependency >>>> > path it prints is a "file:" URL. >>>> > >>>> > I have a feeling that this is because Spark uses Ivy 2.4 while sbt >>>> uses Ivy >>>> > 2.3, and those might be incompatible. So if they get mixed up, things >>>> can >>>> > break. >>>> > >>>> > The second is that sometimes unit tests fail with some weird error >>>> > downloading dependencies. When checking the ivy metadata in >>>> ~/.ivy2/cache, >>>> > the offending dependencies are pointing to my local maven repo (I have >>>> > "maven-local" as one of the entries in my ~/.sbt/repositories). >>>> > >>>> > My feeling in this case is that Spark's version of Ivy somehow doesn't >>>> > handle that case. >>>> > >>>> > So, long story short: >>>> > >>>> > - Has anyone run into either of these problems? >>>> > - Is it possible to set some env variable or something during tests >>>> to force >>>> > them to use their own directory instead of messing up and breaking my >>>> > ~/.ivy2? >>>> > >>>> > >>>> > -- >>>> > Marcelo >>>> >>>> --------------------------------------------------------------------- >>>> To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org >>>> For additional commands, e-mail: dev-h...@spark.apache.org >>>> >>>> >>> >> >> >> -- >> Marcelo >> > > -- Marcelo
