but just to be clear there is still room for adding a new python
MLContext too (with similar APIs to our current).
Regards,
Matthias
On 8/12/2020 2:43 PM, Baunsgaard, Sebastian wrote:
Hi Janardhan,
The python interface currently does not run with spark, partially because we
are focusing on federated execution.
The future intention is to not put the spark controls inside the python API,
since it is the java execution that should handle spark.
I think this is an upgrade since the handling of spark from both the python API
and internally in systemds can be confusing.
best regards
Sebastian
________________________________
From: Janardhan <[email protected]>
Sent: Wednesday, August 12, 2020 2:22:52 PM
To: [email protected]
Subject: Re: [QUESTION] About MLContext. Thanks.
Hi Sebastian,
What is the equivalent of the following snippet to SystemDS Python api?
from systemml import MLContext>>> ml = MLContext(spark)
Welcome to Apache SystemML!Version 1.0.0-SNAPSHOT
I looked at the `l2svm` but do we have a generic object to which we could
pass spark.
Thank you,
Janardhan
On Mon, Jul 20, 2020 at 1:47 PM Janardhan <[email protected]>
wrote:
Hi all,
We have a bit of a backlog in MLContext test coverage. This one
is open for discussion.
The preliminary work[1] on reusing existing R scripts for codegen,
for MLContext testing.
Deciding on at least till release date helps our team to add algorithms
one by one, to the extent of the codegen covered algorithms.
[1] https://github.com/apache/systemds/pull/997
[https://avatars3.githubusercontent.com/u/47359?s=400&v=4]<https://github.com/apache/systemds/pull/997>
[SYSTEMDS-1863] Full MLContext test for LinearReg by j143 · Pull Request #997 ·
apache/systemds · GitHub<https://github.com/apache/systemds/pull/997>
github.com
Takes advantage of existing R algorithm scripts used for codegen testing. This
would improve the testing by allowing us to provide all the necessary inputs
into the script.
Thank you,
Janardhan
On Fri, Jul 17, 2020 at 10:07 PM Janardhan <[email protected]>
wrote:
Hi, SYSTEMDS-2572 the MLContext API is not loading into spark-shell. Of
course, everything else works fine!
Thank you,
Janardhan
On Fri, 17 Jul, 2020, 18:19 Matthias Boehm, <[email protected]> wrote:
yes, these nn scripts are packed into our SystemDS jar and can be used
(via imports) in MLContext scripts as well. If there are issues, please
feel free to file a bug report.
Regards,
Matthias
On 7/17/2020 11:12 AM, Janardhan wrote:
Hi, Is `nn` library being provided with MLContext?
Thank you,
Janardhan
