Re: [VOTE] Release Apache Spark 0.8.0-incubating (rc4)
Hi Marvin, These are not actually files compiled from our source code, but third-party dependencies that are not published to Maven. SBT is a Scala-based build tool where you need that JAR to run. Where exactly is the policy on including these kinds of JARs? The reason we went with them is because we saw other projects do it. For example, Kafka, which also builds using SBT, ships with sbt.jar. Matei On Dec 14, 2013, at 10:58 AM, Marvin Humphrey mar...@rectangular.com wrote: On Sat, Dec 14, 2013 at 10:37 AM, Henry Saputra henry.sapu...@gmail.com wrote: When you said However they both contain binaries, which is not good. were you talking about the spark-0.8.1-incubating-bin-* files ? There seem to be compiled files in the source archive. marvin@knut:~/spark $ tar -zxf spark-0.8.1-incubating.tgz marvin@knut:~/spark $ cd spark-0.8.1-incubating marvin@knut:~/spark/spark-0.8.1-incubating $ find . -print | grep .jar$ ./assembly/lib/net/sf/py4j/py4j/0.7/py4j-0.7.jar ./core/src/test/resources/uncommons-maths-1.2.2.jar ./repl/lib/scala-jline.jar ./sbt/sbt-launch-0.11.3-2.jar ./streaming/lib/org/apache/kafka/kafka/0.7.2-spark/kafka-0.7.2-spark.jar marvin@knut:~/spark/spark-0.8.1-incubating $ One option for addressing that issue would be to move all compiled dependencies to an accompanying -deps archive. Marvin Humphrey - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
Re: [VOTE] Release Apache Spark 0.8.0-incubating (rc4)
Hi Marvin, These are not actually files compiled from our source code, but third-party dependencies that are not published to Maven. SBT is a Scala-based build tool where you need that JAR to run. Where exactly is the policy on including these kinds of JARs? The reason we went with them is because we saw other projects do it. For example, Kafka, which also builds using SBT, ships with sbt.jar. Matei On Dec 14, 2013, at 10:58 AM, Marvin Humphrey mar...@rectangular.com wrote: On Sat, Dec 14, 2013 at 10:37 AM, Henry Saputra henry.sapu...@gmail.com wrote: When you said However they both contain binaries, which is not good. were you talking about the spark-0.8.1-incubating-bin-* files ? There seem to be compiled files in the source archive. marvin@knut:~/spark $ tar -zxf spark-0.8.1-incubating.tgz marvin@knut:~/spark $ cd spark-0.8.1-incubating marvin@knut:~/spark/spark-0.8.1-incubating $ find . -print | grep .jar$ ./assembly/lib/net/sf/py4j/py4j/0.7/py4j-0.7.jar ./core/src/test/resources/uncommons-maths-1.2.2.jar ./repl/lib/scala-jline.jar ./sbt/sbt-launch-0.11.3-2.jar ./streaming/lib/org/apache/kafka/kafka/0.7.2-spark/kafka-0.7.2-spark.jar marvin@knut:~/spark/spark-0.8.1-incubating $ One option for addressing that issue would be to move all compiled dependencies to an accompanying -deps archive. Marvin Humphrey - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
Re: [VOTE] Release Apache Spark 0.8.0-incubating (rc4)
Hi Patrick, In fact, Sonatype will let you publish non o.a.xx dependencies since they are an approved forge as well for Central sync. So you can sign up for an account there, but it's always good to get the buy in of the specific community of the upstream dependency so they are aware. These types of things though can be addressed over time and as each release improves. Cheers, Chris -Original Message- From: Patrick Wendell pwend...@gmail.com Reply-To: general@incubator.apache.org general@incubator.apache.org Date: Saturday, December 14, 2013 10:39 PM To: general@incubator.apache.org general@incubator.apache.org Subject: Re: [VOTE] Release Apache Spark 0.8.0-incubating (rc4) Thanks Marvin and Joe, Do you know which other projects/people we might reach out to for institutional knowledge of how this is done? Our specific issue is that while 99% of our dependencies are in maven, there are a small # of dependencies that don't publish anywhere via maven. One solution would be if apache let us publish these to maven central - but I'm guessing Apache limits us to publishing o.i.a.XXX artifacts. - Patrick On Sat, Dec 14, 2013 at 10:31 PM, Marvin Humphrey mar...@rectangular.com wrote: On Sat, Dec 14, 2013 at 9:24 PM, Patrick Wendell pwend...@gmail.com wrote: We are planning to do our next major release (Spark 0.9.0) while still under incubation in the next few weeks. Could I propose that we create a parallel discussion about how we might re-tool or build process with the aim towards satisfying whatever policy exists in that release? +1 That seems reasonable to me. We're winging it a bit, since discussions about under what circumstances to relax policy are ongoing (see the IP Clearance before releasing thread). However, you've taken the initiative to make a constructive proposal. We should work with you. Marvin Humphrey - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
Re: [VOTE] Release Apache Spark 0.8.0-incubating (rc4)
Hey Chris, Okay we'll focus on that distribution path for the 0.9.0 release. Thanks! On Sun, Dec 15, 2013 at 9:47 AM, Chris Mattmann mattm...@apache.org wrote: Hi Patrick, In fact, Sonatype will let you publish non o.a.xx dependencies since they are an approved forge as well for Central sync. So you can sign up for an account there, but it's always good to get the buy in of the specific community of the upstream dependency so they are aware. These types of things though can be addressed over time and as each release improves. Cheers, Chris -Original Message- From: Patrick Wendell pwend...@gmail.com Reply-To: general@incubator.apache.org general@incubator.apache.org Date: Saturday, December 14, 2013 10:39 PM To: general@incubator.apache.org general@incubator.apache.org Subject: Re: [VOTE] Release Apache Spark 0.8.0-incubating (rc4) Thanks Marvin and Joe, Do you know which other projects/people we might reach out to for institutional knowledge of how this is done? Our specific issue is that while 99% of our dependencies are in maven, there are a small # of dependencies that don't publish anywhere via maven. One solution would be if apache let us publish these to maven central - but I'm guessing Apache limits us to publishing o.i.a.XXX artifacts. - Patrick On Sat, Dec 14, 2013 at 10:31 PM, Marvin Humphrey mar...@rectangular.com wrote: On Sat, Dec 14, 2013 at 9:24 PM, Patrick Wendell pwend...@gmail.com wrote: We are planning to do our next major release (Spark 0.9.0) while still under incubation in the next few weeks. Could I propose that we create a parallel discussion about how we might re-tool or build process with the aim towards satisfying whatever policy exists in that release? +1 That seems reasonable to me. We're winging it a bit, since discussions about under what circumstances to relax policy are ongoing (see the IP Clearance before releasing thread). However, you've taken the initiative to make a constructive proposal. We should work with you. Marvin Humphrey - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
Re: [VOTE] Enable Release Checklist Experiment
+1 On 12/13/13 3:59 PM, Marvin Humphrey wrote: Greetings, As the next step in our ongoing efforts to reform the release voting process, I propose that we run an experiment allowing the PPMC members of select podlings to earn binding votes under limited circumstances by completing a release checklist. For participating podlings, the Incubator's release management guide... http://incubator.apache.org/guides/releasemanagement.html ... would be supplanted by the following documents: http://incubator.apache.org/guides/release_manifest.txt http://incubator.apache.org/guides/release.html The scope of this VOTE is limited to approving the following patch to our policy page: https://paste.apache.org/k4vJ Here is the patch content minus markup: 2013 Alternate Release Voting Process Select podlings pre-cleared by a majority vote of the IPMC MAY participate in an alternate release voting process: Should a Podling decide it wishes to perform a release, the Podling SHALL hold a vote on the Podling's dev list and create a permanently archived Release Manifest as described in the Experimental Release Guide. At least three +1 votes from PPMC members are required (see the Apache Voting Process page). If the majority of PPMC votes is positive, then the Podling SHALL send a summary of that vote to the Incubator's general list and formally request the Incubator PMC approve such a release. Formal approval requires three binding +1 votes and more positive than negative votes. Votes cast by members of the Incubator PMC are always binding. For all releases after the first, votes cast by members of the PPMC are binding if a Mentor approves the Release Manifest. Please note that the proposed change is both incremental and reversible: * It is incremental because podlings must be opted in by vote of the IPMC to participate. * It is reversible because once the experiment has run its course the policy change can be reverted with zero impact through lazy consensus. Those who may have questions about the legitimacy of allowing binding votes from non-IPMC members should see this post from Roy Fielding: http://s.apache.org/v7 Please vote: [ ] +1 Yes, apply the patch enabling the experiment. [ ] -1 No, do not apply the patch enabling the experiment. This majority VOTE will run for 7 days and will close at 13:00 PST on Friday, December 20, 2013. Votes cast by members of the Incubator PMC are binding. Here is my own +1. Marvin Humphrey - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org -- Dave Brondsema : d...@brondsema.net http://www.brondsema.net : personal http://www.splike.com : programming - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
Re: Github Permissions for Committers
jclouds uses Pull Requests to a GitHub mirror of the (primary) ASF repos as a means of collaborating on submissions with contributors [1]. We then apply the resulting patches to the ASF repos directly [2, 3]. If you'd like more information on how this works, please hop on to #jclouds on Freenode IRC! Regards ap https://wiki.apache.org/jclouds/How%20to%20Contribute [1] http://markmail.org/message/6luklhdm2lpkbccx [2] https://gist.github.com/adriancole/5592785 (lines 27-38) [3] -- Andrew Phillips Apache jclouds - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
Re: [VOTE] Release Apache Spark 0.8.0-incubating (rc4)
You might also want to have a look at how Scala itself deals with the issue of required binaries for the build: https://github.com/scala/scala/blob/master/pull-binary-libs.sh https://github.com/scala/scala/blob/master/tools/binary-repo-lib.sh I'm not necessarily a huge fan of this (builds on *nix only, for example), but it's at least another approach to throw into the mix... ap -- Andrew Phillips Apache jclouds - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
