Hi Janardhan,
The python interface currently does not run with spark, partially because we are focusing on federated execution. The future intention is to not put the spark controls inside the python API, since it is the java execution that should handle spark. I think this is an upgrade since the handling of spark from both the python API and internally in systemds can be confusing. best regards Sebastian ________________________________ From: Janardhan <[email protected]> Sent: Wednesday, August 12, 2020 2:22:52 PM To: [email protected] Subject: Re: [QUESTION] About MLContext. Thanks. Hi Sebastian, What is the equivalent of the following snippet to SystemDS Python api? >>> from systemml import MLContext>>> ml = MLContext(spark) Welcome to Apache SystemML!Version 1.0.0-SNAPSHOT I looked at the `l2svm` but do we have a generic object to which we could pass spark. Thank you, Janardhan On Mon, Jul 20, 2020 at 1:47 PM Janardhan <[email protected]> wrote: > Hi all, > > We have a bit of a backlog in MLContext test coverage. This one > is open for discussion. > > The preliminary work[1] on reusing existing R scripts for codegen, > for MLContext testing. > > Deciding on at least till release date helps our team to add algorithms > one by one, to the extent of the codegen covered algorithms. > > [1] https://github.com/apache/systemds/pull/997 [https://avatars3.githubusercontent.com/u/47359?s=400&v=4]<https://github.com/apache/systemds/pull/997> [SYSTEMDS-1863] Full MLContext test for LinearReg by j143 · Pull Request #997 · apache/systemds · GitHub<https://github.com/apache/systemds/pull/997> github.com Takes advantage of existing R algorithm scripts used for codegen testing. This would improve the testing by allowing us to provide all the necessary inputs into the script. > > Thank you, > Janardhan > > On Fri, Jul 17, 2020 at 10:07 PM Janardhan <[email protected]> > wrote: > >> Hi, SYSTEMDS-2572 the MLContext API is not loading into spark-shell. Of >> course, everything else works fine! >> >> Thank you, >> Janardhan >> >> On Fri, 17 Jul, 2020, 18:19 Matthias Boehm, <[email protected]> wrote: >> >>> yes, these nn scripts are packed into our SystemDS jar and can be used >>> (via imports) in MLContext scripts as well. If there are issues, please >>> feel free to file a bug report. >>> >>> Regards, >>> Matthias >>> >>> On 7/17/2020 11:12 AM, Janardhan wrote: >>> > Hi, Is `nn` library being provided with MLContext? >>> > >>> > Thank you, >>> > Janardhan >>> > >>> >>
