thanks, Matthias. I will kick off the process on Monday morning PST. Regards, Berthold Reinwald IBM Almaden Research Center office: (408) 927 2208; T/L: 457 2208 e-mail: [email protected]
From: Matthias Boehm <[email protected]> To: [email protected] Date: 03/16/2018 09:21 PM Subject: Re: Release Planning Thanks for the patience. Meanwhile, all known issues have been resolved and we're ready to cut an RC. Until this is done (or we have a 1.1 branch), I would recommend to limit all commits to critical fixes. Regards, Matthias On Tue, Mar 13, 2018 at 12:18 AM, Matthias Boehm <[email protected]> wrote: > just a quick update: the ANTLR issue and most other things that came up > during QA are fixed now, except for performance issues with stratified > stats for which I need some more time (SYSTEMML-2181 tracks the open > issues). So it's probably a good idea to postpone the RC1 for a few days to > avoid unnecessary release efforts. > > Regards, > Matthias > > On Sun, Mar 11, 2018 at 5:43 PM, Matthias Boehm <[email protected]> wrote: > >> well, after trying to run our perftest suite with Spark 2.3 and Spark 2.2 >> this seems to be more complicated. Although the version update from 4.5.3 >> to 4.7.1 solved the problem with Spark 2.3 (which uses 4.7.1), SystemML >> would no longer be backwards compatible with Spark 2.2 and 2.1 (which use >> 4.5.3) because ANTLR checks bidirectional mismatches.Unfortunately, >> removing ANTLR from our jar does not help because the above versions are >> binary incompatible. >> >> We need to hold off the release until we decided whether (1) we directly >> release for Spark 2.3 and drop 2.2 and 2.1, or (2) we release for Spark 2.2 >> and 2.1 with a subsequent ANTLR change and release for 2.3. >> >> Regards, >> Matthias >> >> On Thu, Mar 8, 2018 at 5:38 PM, Niketan Pansare <[email protected]> >> wrote: >> >>> +1. That will preserve current behavior on older Spark versions too. >>> >>> > On Mar 8, 2018, at 5:24 PM, Ted Yu <[email protected]> wrote: >>> > >>> > +1 on upgrading >>> > -------- Original message --------From: Matthias Boehm < >>> [email protected]> Date: 3/8/18 5:19 PM (GMT-08:00) To: >>> [email protected] Subject: Re: Release Planning >>> > related to Spark 2.3, we might want to update our ANTLR version because >>> > with Spark 2.3 every parsed DML script (i.e., multiple times with >>> imported >>> > DML files) produces the following warning: >>> > >>> > "ANTLR Tool version 4.5.3 used for code generation does not match the >>> > current runtime version 4.7" >>> > >>> > Regards, >>> > Matthias >>> > >>> >> On Thu, Mar 1, 2018 at 5:22 PM, Matthias Boehm <[email protected]> >>> wrote: >>> >> >>> >> Hi all, >>> >> >>> >> I'm sure you've seen that Spark 2.3 just got released. This lines up >>> >> beautifully with our own SystemML 1.1 release. Accordingly, I would >>> >> recommend to use Spark 2.3 for our due diligence algorithm-level Q/A. >>> How >>> >> about we shoot for an RC1 by March 12? This should give enough time >>> to run >>> >> over reasonably large data and fix all related issues. >>> >> >>> >> Regards, >>> >> Matthias >>> >> >>> >>> On Tue, Feb 6, 2018 at 12:51 PM, Matthias Boehm <[email protected]> >>> wrote: >>> >>> >>> >>> yes, absolutely. Here is a list of new features and improvements - >>> please >>> >>> feel free to extend as needed: >>> >>> >>> >>> 1) Extended Caffe2DML and Keras2DML APIs >>> >>> 2) Support for large-dense blocks >16GB in CP >>> >>> 3) New builtin functions ifelse, xor, as well as and/or/not over >>> matrices >>> >>> 4) Single-precision support for native conv2d and mm operations. >>> >>> 5) Performance features and correctness of ultra-sparse operations >>> >>> 6) Codegen: new plan cache, nary cbind >>> >>> 7) Parfor: result merge with accumulators +=, reduced initialization >>> >>> overhead >>> >>> 8) Compiler improvements: avoid unnecessary spark instructions, >>> corrected >>> >>> memory estimates >>> >>> >>> >>> Until the RC, I'd like to support all deep learning builtin >>> functions in >>> >>> codegen, add a couple of pending parfor improvements, and >>> potentially do a >>> >>> first cut of the compiler/runtime integration for parameter servers. >>> >>> >>> >>> Regards, >>> >>> Matthias >>> >>> >>> >>> >>> >>> >>> >>> On Tue, Feb 6, 2018 at 12:36 PM, Niketan Pansare <[email protected] >>> > >>> >>> wrote: >>> >>> >>> >>>> +1. >>> >>>> >>> >>>> We should consider including single precision native BLAS in the >>> release >>> >>>> notes as well. If possible, we should add JNI wrappers for PowerPC, >>> Windows >>> >>>> and Mac too in this release. >>> >>>> >>> >>>>> On Feb 6, 2018, at 12:27 PM, Berthold Reinwald < >>> [email protected]> >>> >>>> wrote: >>> >>>>> >>> >>>>> sure. >>> >>>>> >>> >>>>> Makes sense. Codegen and Keras2DML made good progress, and many >>> other >>> >>>>> fixes/improvements. >>> >>>>> >>> >>>>> What else do we want time/track/highlight for it? >>> >>>>> >>> >>>>> Regards, >>> >>>>> Berthold Reinwald >>> >>>>> IBM Almaden Research Center >>> >>>>> office: (408) 927 2208; T/L: 457 2208 >>> >>>>> e-mail: [email protected] >>> >>>>> >>> >>>>> >>> >>>>> >>> >>>>> From: Matthias Boehm <[email protected]> >>> >>>>> To: [email protected] >>> >>>>> Date: 02/05/2018 11:05 PM >>> >>>>> Subject: Release Planning >>> >>>>> >>> >>>>> >>> >>>>> >>> >>>>> Hi all, >>> >>>>> >>> >>>>> since our 1.0 release in Dec, we already got a number enhancements >>> and >>> >>>> new >>> >>>>> features in, so I think it would be good to discuss the timeline >>> for >>> >>>> our >>> >>>>> next SystemML 1.1 release. How about, we target mid March for a >>> first >>> >>>> RC? >>> >>>>> Also, Berthold would you be willing to serve again as the release >>> >>>> manager? >>> >>>>> >>> >>>>> Regards, >>> >>>>> Matthias >>> >>>>> >>> >>>>> >>> >>>>> >>> >>>>> >>> >>>> >>> >>>> >>> >>> >>> >> >>> >>> >> >
