+1 This sounds like a good plan to allow us to continue supporting the Spark 1.x line in the short term, with a plan for moving to Spark 2.x support soon.
-Mike -- Mike Dusenberry GitHub: github.com/dusenberrymw LinkedIn: linkedin.com/in/mikedusenberry Sent from my iPhone. > On Aug 17, 2016, at 2:59 PM, Deron Eriksson <deroneriks...@gmail.com> wrote: > > +1 > Continuing to support Spark 1.4/1.6 for now while setting a cutover date > for 2.0 sounds like a great idea. This allows for the creation of a really > solid release for 1.x, which greatly benefits SystemML users using Spark > 1.x. It also gives these users a general date that they can use to plan > migration to Spark 2.0 when that becomes the SystemML standard so that they > can benefit from the latest improvements to the project. > > Deron > > >> On Wed, Aug 17, 2016 at 2:32 PM, Acs S <ac...@yahoo.com.invalid> wrote: >> >> Seems, mail is not retaining format. I am attaching same text through PDF >> file. >> If there is any other better option please let me know. >> >> >> -Arvind >> >> >> >> >> >> ----- Forwarded Message ----- >> *From:* Acs S <ac...@yahoo.com.INVALID> >> *To:* Dev <dev@systemml.incubator.apache.org> >> *Sent:* Wednesday, August 17, 2016 2:18 PM >> *Subject:* [DISCUSS] SystemML with Spark 2.0 support and roadmap >> >> >> Spark 2.0 has released, we need to support SystemML on Spark 2.0 to be >> uptodate with latest version of Spark. This brings us a challenge to >> support our consumers until they move to Spark 2.0.Based on some >> brainstorming, I can propose following options to keep SystemML being >> supported on latest Spark version quickly. >> >> Supporting SystemML on Spark 1.x We can continue to support SystemML on >> Spark 1.x code base for short period of time by adding fixes and features >> on main branch. We will release SystemML with support to Spark 1.x next >> version (0.11) around beginning of Oct 2016 (Lets target for Oct 1st 2016) >> Supporting SystemML on Spark 2.0 (Preview code) For exploiters of Spark >> 2.0, we can make SystemML on Spark 2.0 immediately based on branch created >> on top of latest master branch code. Glen has some prototype code to >> transform SystemML code to be compatible with Spark 2.0, he can merge his >> code with new branch targeted to support SystemML on Spark 2.0 This would >> be "Preview" version code, and we can update it on frequent basis (on >> bi-monthy basis). Supporting SystemML on Spark 2.0 We will have full >> support of SystemML on Spark 2.0 before end of year 2016. We will formalize >> release date by end of Sept 2016. At the same time we will discuss if we >> can move support of SystemML on Spark 1.x to maintenance mode (Only >> required bug fixes will be merged from main branch) or we need to support >> both SystemML on Spark 2.0 and Spark 1.x for some additional time. >> SystemML Roadmap 0.11 (on Spark 1.x) (Targeted to Oct 1st 2016) - Deep >> Learning (Library of Network layers?) - Frame - New MLContext API - >> Python DSL integration (Preview) - Compressed Linear Algebra (Preview) - >> Hydra R integration - New Algorithms (?) >> 0.12 (Spark 2.0) (Targeted to 4Q 2016) - GPU support (Local >> mode/Distributed mode?) >> - New Algorithms (?) >> Please feel free to comment on support and roadmap points. >> >> >> -Arvind >> >>
