Compatibility sure. The updates to the different versions are mainly security updates. For the hadoop 3.3.3 to 3.3.4 is security related. The spark 3.2.0 to 3.3.1 is both security and a minor update.
I think both are fair updates to make, that does not require us to do a major release. We also risk that someone points out the security vulnerabilities in an upcoming release if we do not update. best regards Sebastian On 7 December 2022 at 13:43:50 +01:00, Matthias Boehm <[email protected]> wrote: > yes, backwards compatibility is important to keep in mind. In the past, we > mentioned the *minimum* Spark/Hadoop versions SystemML/SystemDS required, > while still being able to run with more recent versions. > > So let's separate two different aspects here: (1) the minimum version > compatible with our source code, and (2) the dependencies we reference for > tests and build. Although Spark and Hadoop are generally very good in terms > of backwards compatibility inside major versions, it would be good to keep > (1) and (2) in sync to avoid hidden incompatibilities. > > Are there specific reasons for this update - API changes, security warnings, > etc? > > Regards, > Matthias > > On 12/7/2022 1:06 PM, arnab phani wrote: > > > Can these upgrades harm the backward compatibility of the next SystemDS > > release? > > If so, then we either need to make a major release or delay the upgrades > > till the next major release. > > > > Regards, > > Arnab.. > > > > On Wed, Dec 7, 2022 at 1:01 PM <<[email protected]>> wrote: > > > > > > > Hi all, > > > > > > Just to let all know, > > > > > > I am considering updating the Spark, and Hadoop version of our system, > > > would this interfere with any ongoing work currently? > > > > > > Spark 3.2.0 -> 3.3.1 > > > Hadoop 3.3.3 -> 3.3.4 > > > > > > best regards > > > Sebastian > > >
