Hi Mark, I appreciate your comment.
My thinking is that the more frequent minor and patch releases the more often end users can give them a shot and be part of the bigger release cycle for major releases. Spark's an OSS project and we all can make mistakes and my thinking is is that the more eyeballs the less the number of the mistakes. If we make very fine/minor releases often we should be able to attract more people who spend their time on testing/verification that eventually contribute to a higher quality of Spark. Pozdrawiam, Jacek Laskowski ---- https://medium.com/@jaceklaskowski/ Mastering Apache Spark 2.0 https://bit.ly/mastering-apache-spark Follow me at https://twitter.com/jaceklaskowski On Sun, Mar 19, 2017 at 10:50 PM, Mark Hamstra <m...@clearstorydata.com> wrote: > That doesn't necessarily follow, Jacek. There is a point where too frequent > releases decrease quality. That is because releases don't come for free -- > each one demands a considerable amount of time from release managers, > testers, etc. -- time that would otherwise typically be devoted to improving > (or at least adding to) the code. And that doesn't even begin to consider > the time that needs to be spent putting a new version into a larger software > distribution or that users need to put in to deploy and use a new version. > If you have an extremely lightweight deployment cycle, then small, quick > releases can make sense; but "lightweight" doesn't really describe a Spark > release. The concern for excessive overhead is a large part of the thinking > behind why we stretched out the roadmap to allow longer intervals between > scheduled releases. A similar concern does come into play for unscheduled > maintenance releases -- but I don't think that that is the forcing function > at this point: A 2.1.1 release is a good idea. > > On Sun, Mar 19, 2017 at 6:24 AM, Jacek Laskowski <ja...@japila.pl> wrote: >> >> +10000 >> >> More smaller and more frequent releases (so major releases get even more >> quality). >> >> Jacek >> >> On 13 Mar 2017 8:07 p.m., "Holden Karau" <hol...@pigscanfly.ca> wrote: >>> >>> Hi Spark Devs, >>> >>> Spark 2.1 has been out since end of December and we've got quite a few >>> fixes merged for 2.1.1. >>> >>> On the Python side one of the things I'd like to see us get out into a >>> patch release is a packaging fix (now merged) before we upload to PyPI & >>> Conda, and we also have the normal batch of fixes like toLocalIterator for >>> large DataFrames in PySpark. >>> >>> I've chatted with Felix & Shivaram who seem to think the R side is >>> looking close to in good shape for a 2.1.1 release to submit to CRAN (if >>> I've miss-spoken my apologies). The two outstanding issues that are being >>> tracked for R are SPARK-18817, SPARK-19237. >>> >>> Looking at the other components quickly it seems like structured >>> streaming could also benefit from a patch release. >>> >>> What do others think - are there any issues people are actively targeting >>> for 2.1.1? Is this too early to be considering a patch release? >>> >>> Cheers, >>> >>> Holden >>> -- >>> Cell : 425-233-8271 >>> Twitter: https://twitter.com/holdenkarau > > --------------------------------------------------------------------- To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
