To simplify release candidate validation, I would like to propose that the distribution profile only builds the following 7 (out of the current included 10) artifacts:
systemml-0.11.0-incubating-SNAPSHOT-javadoc.jar systemml-0.11.0-incubating-SNAPSHOT-sources.jar systemml-0.11.0-incubating-SNAPSHOT-src.tar.gz systemml-0.11.0-incubating-SNAPSHOT-src.zip systemml-0.11.0-incubating-SNAPSHOT-standalone.tar.gz (rename w/o "-standalone") systemml-0.11.0-incubating-SNAPSHOT-standalone.zip (rename w/o "-standalone") systemml-0.11.0-incubating-SNAPSHOT.jar The following could still be built using maven profiles but would not be in the distribution profile: systemml-0.11.0-incubating-SNAPSHOT-standalone.jar systemml-0.11.0-incubating-SNAPSHOT.tar.gz (also rename) systemml-0.11.0-incubating-SNAPSHOT.zip (also rename) This would decrease the number of our artifacts by 30% which means that we can validate the release faster, and the release candidate will also be more likely to pass external validation/voting. Deron On Thu, Aug 18, 2016 at 12:05 AM, Berthold Reinwald <reinw...@us.ibm.com> wrote: > This makes sense. Couple of comments. > > - wrt SystemML on Spark 1.x, SystemML 0.11 target date should be the > latest in Oct. Ideally it should be earlier - maybe September - depending > on community demand as it will contain bug fixes for features introduced > in 0.10. The master branch will stay on Spark 1.x til then. New features > can be integrated but as they are still partial, they will be marked as > experimental. > > - wrt SystemML on Spark 2.0, as the timeline is fairly short, instead of > bi-monthly I'd suggest more frequent sync with the branch on Spark 2.0 > > - Comes early October, we should switch the master to Spark 2.0, and > immediately create a release for 2.0. > > - wrt Roadmap 0.11 items, except for Frame support and MLContext and may > be DL lib, all the other features should be marked experimental. > > Regards, > Berthold Reinwald > IBM Almaden Research Center > office: (408) 927 2208; T/L: 457 2208 > e-mail: reinw...@us.ibm.com > > > > From: dusenberr...@gmail.com > To: dev@systemml.incubator.apache.org > Date: 08/17/2016 04:52 PM > Subject: Re: [DISCUSS] SystemML with Spark 2.0 support and roadmap > > > > +1 > > This sounds like a good plan to allow us to continue supporting the Spark > 1.x line in the short term, with a plan for moving to Spark 2.x support > soon. > > -Mike > > -- > > Mike Dusenberry > GitHub: github.com/dusenberrymw > LinkedIn: linkedin.com/in/mikedusenberry > > Sent from my iPhone. > > > > On Aug 17, 2016, at 2:59 PM, Deron Eriksson <deroneriks...@gmail.com> > wrote: > > > > +1 > > Continuing to support Spark 1.4/1.6 for now while setting a cutover date > > for 2.0 sounds like a great idea. This allows for the creation of a > really > > solid release for 1.x, which greatly benefits SystemML users using Spark > > 1.x. It also gives these users a general date that they can use to plan > > migration to Spark 2.0 when that becomes the SystemML standard so that > they > > can benefit from the latest improvements to the project. > > > > Deron > > > > > >> On Wed, Aug 17, 2016 at 2:32 PM, Acs S <ac...@yahoo.com.invalid> wrote: > >> > >> Seems, mail is not retaining format. I am attaching same text through > PDF > >> file. > >> If there is any other better option please let me know. > >> > >> > >> -Arvind > >> > >> > >> > >> > >> > >> ----- Forwarded Message ----- > >> *From:* Acs S <ac...@yahoo.com.INVALID> > >> *To:* Dev <dev@systemml.incubator.apache.org> > >> *Sent:* Wednesday, August 17, 2016 2:18 PM > >> *Subject:* [DISCUSS] SystemML with Spark 2.0 support and roadmap > >> > >> > >> Spark 2.0 has released, we need to support SystemML on Spark 2.0 to be > >> uptodate with latest version of Spark. This brings us a challenge to > >> support our consumers until they move to Spark 2.0.Based on some > >> brainstorming, I can propose following options to keep SystemML being > >> supported on latest Spark version quickly. > >> > >> Supporting SystemML on Spark 1.x We can continue to support SystemML on > >> Spark 1.x code base for short period of time by adding fixes and > features > >> on main branch. We will release SystemML with support to Spark 1.x > next > >> version (0.11) around beginning of Oct 2016 (Lets target for Oct 1st > 2016) > >> Supporting SystemML on Spark 2.0 (Preview code) For exploiters of Spark > >> 2.0, we can make SystemML on Spark 2.0 immediately based on branch > created > >> on top of latest master branch code. Glen has some prototype code to > >> transform SystemML code to be compatible with Spark 2.0, he can merge > his > >> code with new branch targeted to support SystemML on Spark 2.0 This > would > >> be "Preview" version code, and we can update it on frequent basis (on > >> bi-monthy basis). Supporting SystemML on Spark 2.0 We will have full > >> support of SystemML on Spark 2.0 before end of year 2016. We will > formalize > >> release date by end of Sept 2016. At the same time we will discuss if > we > >> can move support of SystemML on Spark 1.x to maintenance mode (Only > >> required bug fixes will be merged from main branch) or we need to > support > >> both SystemML on Spark 2.0 and Spark 1.x for some additional time. > >> SystemML Roadmap 0.11 (on Spark 1.x) (Targeted to Oct 1st 2016) - Deep > >> Learning (Library of Network layers?) - Frame - New MLContext API - > >> Python DSL integration (Preview) - Compressed Linear Algebra (Preview) > - > >> Hydra R integration - New Algorithms (?) > >> 0.12 (Spark 2.0) (Targeted to 4Q 2016) - GPU support (Local > >> mode/Distributed mode?) > >> - New Algorithms (?) > >> Please feel free to comment on support and roadmap points. > >> > >> > >> -Arvind > >> > >> > > > > >
