Right, and I think this is where the disagreement is, and where SemVer isn't helping us much, in that the change is source compatible but not binary compatible for JDK 7. The more I think about it the more I'm fine with it, I guess it's a quirk of using the JVM.
J-D On Wed, Aug 16, 2017 at 8:25 AM, Mark Hamstra <[email protected]> wrote: > Java 7 support was deprecated in Spark 2.0.0 and documented as such in the > release notes. The removal of Java 7 support does not introduce a > source-level backwards incompatibility in the public Spark API. > > On Wed, Aug 16, 2017 at 8:20 AM, Jean-Daniel Cryans <[email protected]> > wrote: > > > Hi Mark, > > > > Thank you for your insight in Spark, we're obviously missing such > expertise > > and this has led us to make some mistakes. > > > > Perusing the documentation, I only see obvious deprecation notices in > 2.1.0 > > after SPARK-18138 was pushed. > > > > Nevertheless, I think Dan's interpretation is that the major version must > > be incremented if a backward incompatible change is made. Spark 2.1.0 > could > > run on JDK 7, 2.2.0 requires JDK 8, so what used to work doesn't anymore. > > We're far from public APIs, but I can relate to his point of view. > > > > On the bright side, I don't think this is putting Kudu in a bad spot. If > we > > upgrade kudu-spark's jar to require JDK 8 then we can also limit this > > requirement to that module so that users of other modules (like > > kudu-client) can still use it with JDK 7. This means kudu-spark users > have > > to make sure they're on JDK 8, but they'd have to do that regardless if > > they want to use Spark 2.2.0. All we need is some more lines in our pom > > files. > > > > Thanks, > > > > J-D > > > > On Wed, Aug 16, 2017 at 7:50 AM, Mark Hamstra <[email protected]> > > wrote: > > > > > And? > > > > > > Not only was the change documented, but there was more than one minor > > > release with the deprecation in place before the removal of Java 7 and > > > Scala 2.10 support in the new major release. Java 7 and Scala 2.10 have > > > never been anything but deprecated functionality in the Spark 2 API. It > > is > > > just not the Spark PMC's fault if you chose not to follow that > > deprecation > > > guidance. > > > > > > > On Aug 15, 2017, at 8:27 PM, Dan Burkert <[email protected]> > > wrote: > > > > > > > > Hi Mark, > > > > > > > > On Tue, Aug 15, 2017 at 6:49 PM, Mark Hamstra < > [email protected] > > > > > > > wrote: > > > > > > > >> You are badly mistaken > > > > > > > > > > > > My interpretation of SemVer above is based on the definition at > > > SemVer.org, > > > > which has this to say about when it's appropriate to remove > deprecated > > > > functionality: > > > > > > > >> When you deprecate part of your public API, you should do two > things: > > > (1) > > > > update your documentation to let users know about the change, (2) > > issue a > > > > new minor release with the deprecation in place. Before you > completely > > > > remove the functionality in a new major release there should be at > > least > > > > one minor release that contains the deprecation so that users can > > > smoothly > > > > transition to the new API. > > > > > > > > Also relevant, from the same source: > > > > > > > >> Major version X (X.y.z | X > 0) MUST be incremented if any backwards > > > > incompatible changes are introduced to the public API. > > > > > > > > - Dan > > > > > > > > > > > >> > > > >> On Tue, Aug 15, 2017 at 2:18 PM, Dan Burkert <[email protected] > > > > > >> wrote: > > > >> > > > >>> I'll preface my response by saying I don't think there are any hard > > and > > > >>> fast rules here, but I'd like us to try > > > >>> and continue following SemVer rules as much as possible. > > > >>> > > > >>> On Tue, Aug 15, 2017 at 2:03 PM, Grant Henke <[email protected]> > > > >> wrote: > > > >>>> > > > >>>> > > > >>>> - Should/can we drop Spark 1 support in the next minor release? > > > >>>> > > > >>> > > > >>> My interpretation is that it's permissible to stop shipping > releases > > of > > > >> an > > > >>> artifact at any point (in this case kudu-spark1_2.10), > > > >>> so I'm all for dropping Spark 1 as soon as we feel there are a > > > >> sufficiently > > > >>> low number of users. > > > >>> > > > >>> > > > >>>> - Should/can we drop Java 7 support in the next minor release? > > Does > > > >> it > > > >>>> need to be a major release? > > > >>>> > > > >>> > > > >>> My interpretation of SemVer is that we can't drop JRE 7 support > > > without a > > > >>> major version bump. That being said, > > > >>> I do think we're quickly approaching the time in which it would be > > > >>> appropriate to make this step. > > > >>> > > > >>> > > > >>>> - How should we support Spark 2.2.0 if we don't drop Java 7? > > Should > > > >> we > > > >>>> only require Java 1.8 for the Spark 2 modules? > > > >>>> > > > >>> > > > >>> Spark has put us in a difficult position here - either > > kudu-spark2_2.11 > > > >>> remains JRE 7 compatible > > > >>> and is capped at Spark 2.1, or we make an exception for > > > kudu-spark2_2.11, > > > >>> drop > > > >>> JRE 7 compatibility, and continue floating the Spark version > against > > > the > > > >>> latest 2.x release. I think given the > > > >>> velocity of the Spark project and the fact that Spark itself > doesn't > > > seem > > > >>> to have any qualms about > > > >>> breaking SemVer, we should do the latter. > > > >>> > > > >>> > > > >>>> -- > > > >>>> Grant Henke > > > >>>> Software Engineer | Cloudera > > > >>>> [email protected] | twitter.com/gchenke | > > linkedin.com/in/granthenke > > > >>>> > > > >>> > > > >> > > > > > >
