Re: Sub projects in Language and run time for parameter servers [SYSTEMML-2083]

2018-03-16 Thread Chamath Abeysinghe
Hi Matthias, After going through JIRA sub projects and references you provide I thought of drafting proposal focusing the Distributed spark backend project because it seems challenging and exciting area to explore :-). I have sketched a rough

Re: Sub projects in Language and run time for parameter servers [SYSTEMML-2083]

2018-03-09 Thread Matthias Boehm
Hi Chamath, ad 1: Yes, this is absolutely correct. However, it is important to realize that within the workers, we want to run dml functions, and for these we'll reuse our existing compiler, runtime, operations, and data structures. ad 2: Yes, this is also correct. Indeed we can use an existing