Re: Spark Core

2017-07-13 Thread Matthias Boehm
test") > > > # SystemML Specifications: > > > from pyspark.sql import SQLContext > import systemml as sml > sqlCtx = SQLContext(sc) > ml = sml.MLContext(sc) > > > > Thank you! > > Arijit > > > From: Matthias Boehm

Re: Spark Core

2017-07-13 Thread arijit chakraborty
= sml.MLContext(sc) Thank you! Arijit ____________ From: Matthias Boehm Sent: Thursday, July 13, 2017 1:37:29 AM To: dev@systemml.apache.org Subject: Re: Spark Core Well, we explicitly cleanup all intermediates that are no longer used. You can use -explain to output the

Re: Spark Core

2017-07-12 Thread Matthias Boehm
Well, we explicitly cleanup all intermediates that are no longer used. You can use -explain to output the runtime plan, which includes rmvar (remove variable), cpvar (copy variable) and mvvar (move variable) instructions that internally cleanup intermediates. This cleanup removes data from memory,