Yes, though I think the Maven Central repository is more canonical. http://repo1.maven.org/maven2/org/apache/spark/spark-core_2.10/1.5.2/
On Mon, Dec 14, 2015, 06:35 Xiaoyong Zhu <xiaoy...@microsoft.com> wrote: > Thanks! do you mean something here (for example for 1.5.1 using scala > 2.10)? > > > https://repository.apache.org/content/repositories/releases/org/apache/spark/spark-core_2.10/1.5.1/ > > > > Xiaoyong > > > > *From:* Sean Owen [mailto:so...@cloudera.com] > *Sent:* Saturday, December 12, 2015 12:45 AM > *To:* Xiaoyong Zhu <xiaoy...@microsoft.com> > *Cc:* user <user@spark.apache.org> > > > *Subject:* Re: Re: Spark assembly in Maven repo? > > > > That's exactly what the various artifacts in the Maven repo are for. The > API classes for core are in the core artifact and so on. You don't need an > assembly. > > > > On Sat, Dec 12, 2015 at 12:32 AM, Xiaoyong Zhu <xiaoy...@microsoft.com> > wrote: > > Yes, so our scenario is to treat the spark assembly as an “SDK” so users > can develop Spark applications easily without downloading them. In this > case which way do you guys think might be good? > > > > Xiaoyong > > > > *From:* fightf...@163.com [mailto:fightf...@163.com] > *Sent:* Friday, December 11, 2015 12:08 AM > *To:* Mark Hamstra <m...@clearstorydata.com> > *Cc:* Xiaoyong Zhu <xiaoy...@microsoft.com>; Jeff Zhang <zjf...@gmail.com>; > user <user@spark.apache.org>; Zhaomin Xu <z...@microsoft.com>; Joe Zhang > (SDE) <gui...@microsoft.com> > *Subject:* Re: Re: Spark assembly in Maven repo? > > > > Agree with you that assembly jar is not good to publish. However, what he > really need is to fetch > > an updatable maven jar file. > > > ------------------------------ > > fightf...@163.com > > > > *From:* Mark Hamstra <m...@clearstorydata.com> > > *Date:* 2015-12-11 15:34 > > *To:* fightf...@163.com > > *CC:* Xiaoyong Zhu <xiaoy...@microsoft.com>; Jeff Zhang <zjf...@gmail.com>; > user <user@spark.apache.org>; Zhaomin Xu <z...@microsoft.com>; Joe Zhang > (SDE) <gui...@microsoft.com> > > *Subject:* Re: RE: Spark assembly in Maven repo? > > No, publishing a spark assembly jar is not fine. See the doc attached to > https://issues.apache.org/jira/browse/SPARK-11157 > <https://na01.safelinks.protection.outlook.com/?url=https%3a%2f%2fissues.apache.org%2fjira%2fbrowse%2fSPARK-11157&data=01%7c01%7cxiaoyzhu%40064d.mgd.microsoft.com%7c999bc04245724187198808d3020238b3%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=cUBvArmmYasxoOl9wrefYO4L7JiTF5qqqZDyY%2b%2bLP3Y%3d> > and be aware that a likely goal of Spark 2.0 will be the elimination of > assemblies. > > > > On Thu, Dec 10, 2015 at 11:19 PM, fightf...@163.com <fightf...@163.com> > wrote: > > Using maven to download the assembly jar is fine. I would recommend to > deploy this > > assembly jar to your local maven repo, i.e. nexus repo, Or more likey a > snapshot repository > > > ------------------------------ > > fightf...@163.com > > > > *From:* Xiaoyong Zhu <xiaoy...@microsoft.com> > > *Date:* 2015-12-11 15:10 > > *To:* Jeff Zhang <zjf...@gmail.com> > > *CC:* user@spark.apache.org; Zhaomin Xu <z...@microsoft.com>; Joe Zhang > (SDE) <gui...@microsoft.com> > > *Subject:* RE: Spark assembly in Maven repo? > > Sorry – I didn’t make it clear. It’s actually not a “dependency” – it’s > actually that we are building a certain plugin for IntelliJ where we want > to distribute this jar. But since the jar is updated frequently we don't > want to distribute it together with our plugin but we would like to > download it via Maven. > > > > In this case what’s the recommended way? > > > > Xiaoyong > > > > *From:* Jeff Zhang [mailto:zjf...@gmail.com] > *Sent:* Thursday, December 10, 2015 11:03 PM > *To:* Xiaoyong Zhu <xiaoy...@microsoft.com> > *Cc:* user@spark.apache.org > *Subject:* Re: Spark assembly in Maven repo? > > > > I don't think make the assembly jar as dependency a good practice. You may > meet jar hell issue in that case. > > > > On Fri, Dec 11, 2015 at 2:46 PM, Xiaoyong Zhu <xiaoy...@microsoft.com> > wrote: > > Hi Experts, > > > > We have a project which has a dependency for the following jar > > > > spark-assembly-<version>-hadoop<version>.jar > > for example: > > spark-assembly-1.4.1.2.3.3.0-2983-hadoop2.7.1.2.3.3.0-2983.jar > > > > since this assembly might be updated in the future, I am not sure if there > is a Maven repo that has the above spark assembly jar? Or should we create > & upload it to Maven central? > > > > Thanks! > > > > Xiaoyong > > > > > > > > -- > > Best Regards > > Jeff Zhang > > > > >
