Re: [VOTE] Apache SystemML 0.11.0-incubating (RC5)

+1 On Wed, Nov 2, 2016 at 2:29 PM, Glenn Weidner wrote: > I see it's been removed as part of https://issues.apache.org/ > jira/browse/SYSTEMML-926. I will validate the remaining artifacts on > Windows. > > --Glenn > > > [image: Inactive hide details for Glenn Weidner---11/02/2016 02:22:49 > PM--

Re: [DISCUSS] Apache SystemML Release 1.0.0

+1 Regards, Sourav > On ২৫ আগস্ট, ২০১৬, at ৫:৪১ অপরাহ্ণ, Niketan Pansare > wrote: > > +1 for calling next release as 1.0. > > Thanks, > > Niketan Pansare > IBM Almaden Research Center > E-mail: npansar At us.ibm.com > http://researcher.watson.ibm.com/researcher/view.php?person=us-npansar >

Re: Calling System ML from sparkR

, Acs S wrote: > Its in very early stage. Connection between HydraR and SystemML has been > enabled, with pending functionality (dev/testing).It will take some time to > get into mature state. We will keep you posted through Apache SystemML site. > -Arvind > > From: Sour

Re: Calling System ML from sparkR

n Wed, Aug 17, 2016 at 12:09 PM, Deron Eriksson wrote: > Hi Sourav, > > Great question. Work is currently being performed by Alok Singh (see > https://issues.apache.org/jira/browse/SYSTEMML-860) regarding this topic. > > Deron > > > On Mon, Aug 15, 2016 at 9:31 AM, So

Calling System ML from sparkR

Hi, Is there any work going on to call System ML dml scripts form SparkR using R syntax ? I understand it was possible using BigR (available in IBM Big Insights). Wondering whether something similar can be achieved from Spark R. Regards, Sourav

Re: Using RDMA for Deep Learning

sion (for > example: on fault-tolerance, permissions, homogenous clusters, etc) :) > > Thanks, > > Niketan Pansare > IBM Almaden Research Center > E-mail: npansar At us.ibm.com > http://researcher.watson.ibm.com/researcher/view.php?person=us-npansar > > [image: Inactive hi

Using RDMA for Deep Learning

Hi, Was checking CaffeOnSpark for Deep Learning which has just got open sourced by Yahoo - http://yahoohadoop.tumblr.com/post/139916563586/caffeonspark-open-sourced-for-distributed-deep?soc_src=mail&soc_trk=ma . Wondering whether SystemML can also leverage the RDMA based Model synchronization app

Re: SystemML Notebook docker image

I tried that too. But didn't work. Regards, Sourav On Mon, Feb 15, 2016 at 12:27 PM, Deron Eriksson wrote: > Hi Sourav, > > Actually, looking at the error message again, perhaps the issue was that > you need to use your github username for authentication (sourav-mazumder)

Re: SystemML Notebook docker image

uciano Resende wrote: > Sourav, > > To make things simple, if you could create a jira and attach the notebook, > I will handle the rest. > > On Mon, Feb 15, 2016 at 11:54 AM, Sourav Mazumder < > sourav.mazumde...@gmail.com> wrote: > > > Hi Deron, > > > >

Re: SystemML Notebook docker image

gmail.com@github.com remote: Permission to apache/incubator-systemml.git denied to sourav-mazumder. Alternatively if someone can create a folder for my notebook ("2BCHR4T1Q") under incubator-systemml <https://github.com/sourav-mazumder/incubator-systemml>/ samples <https://github

Re: SystemML Notebook docker image

eb 6, 2016 at 11:55 AM, Luciano Resende wrote: > On Fri, Feb 5, 2016 at 5:26 PM, Sourav Mazumder < > sourav.mazumde...@gmail.com > > wrote: > > > Hi Deron, > > > > I can surely share that. Can I upload it somewhere in the SystemML's > site ? > > >

Re: SystemML Notebook docker image

t is a path that is going to be most useful for data > scientists to do exploratory work with SystemML? Is there anything that you > would like to see improved with regards to the MLContext API? > > Deron > > > On Thu, Feb 4, 2016 at 4:01 PM, Sourav Mazumder < > sourav.

Re: SystemML Notebook docker image

Hi, I have a complete end to end Modeling and Prediction using Zepplein and also visualization of the prediction using R plots. I can share the same too if that is useful. Regards, Sourav On Thu, Feb 4, 2016 at 3:20 PM, Luciano Resende wrote: > I started experimenting with some nice ways to e

Re: Is there any equivalent of kron function in DML ?

n large data for X, however, we might want to put this on the roadmap > for 2016. Could you share the characteristics of the workloads you have in > mind? > > > Regards, > Matthias > > Sourav Mazumder ---12/14/2015 11:56:52 PM---Hi, Is there any

Is there any equivalent of kron function in DML ?

Hi, Is there any equivalent to kron function (of Matlab) in DML ? Can I use the matrix multiplication %*% for the same ? Regards, Sourav

Re: Neural Net

Thanks Shirish for the input. I'll try out the same and let you know if I face any problem. Regards, Sourav On Thu, Dec 10, 2015 at 4:35 PM, Sourav Mazumder < sourav.mazumde...@gmail.com> wrote: > Hi, > > i could not see mention of Neural Net algorithm in the Algorithm Refe

Neural Net

Hi, i could not see mention of Neural Net algorithm in the Algorithm Reference in System ML documentation. Is it supported ? Or can a DML script for neural net be developed using the DML language ? Regards, Sourav

Re: Using GLM-predict

added "ID" to the > DataFrame, but did not inform SystemML that ID was inserted. To do that, > please replace "ml.registerInput("X", predDfIn)" to "ml.registerInput("X", > predDfIn, true)". > > Thanks, > > Niketan Pansare > I

Re: Using GLM-predict

d by zipWithIndex (which is also used by > RDDConverterUtilsExt.*addIDToDataFrame*) > For binary block to DataFrame conversion, see > https://github.com/apache/incubator-systemml/blob/master/src/main/java/org/apache/sysml/runtime/instructions/spark/utils/RDDConverterUtilsExt.java#L364 >

Re: Using GLM-predict

be > flagged as cancer. Else output the label with highest probability. Using > this suggestion, the data scientist modifies the DML as follows: > zeroOneMat = ppred(prob[cancerColID], threshold, ">") > prediction = zeroOneMat*cancerColID + (1-zeroOneMat)*rowIndexMax(prob) >

Re: Using GLM-predict

> IBM Almaden Research Center > E-mail: npansar At us.ibm.com > http://researcher.watson.ibm.com/researcher/view.php?person=us-npansar > > [image: Inactive hide details for Sourav Mazumder ---12/09/2015 11:15:19 > AM---Hi Niketan, The code you provided works fine. The

Re: Using GLM-predict

-- > > > 3. Say I get back prediction matrix as an output (from predictions = > rowIndexMax(means);). Now can I read add that as a column to my original > data frame (the one from which I created the feature vector for the > original model) ? My concern

Re: Using GLM-predict

ion, recall, etc.). You can find the best > threshold for the given data set by finding a value that gives the desired > classifier performance (for example, a threshold that gives roughly equal > precision and recall). Such an optimization is obviously done during the > training phase usin

Re: Using GLM-predict

ssifier performance (for example, a threshold that gives roughly equal > precision and recall). Such an optimization is obviously done during the > training phase using a held out test set. > > If you wish, you can also modify the DML script to perform this entire > process. > &

Re: DML script parsing error in Spark

> Niketan Pansare > IBM Almaden Research Center > E-mail: npansar At us.ibm.com > http://researcher.watson.ibm.com/researcher/view.php?person=us-npansar > > [image: Inactive hide details for Sourav Mazumder ---12/08/2015 11:17:43 > AM---Hi Shirish, Passing 'B' as " &qu

Using GLM-predict

Hi, I have used GLM.dml to create a model using some sample data. It returns to me the matrix of Beta, B. Now I want to use this matrix of Beta on a new set of data points and generate predicted value of the dependent variable/observation. When I checked GLM-predict, I could see that one can pas

Re: DML script parsing error in Spark

Got a solution to this problem. I was not passing the right value of the Response matrix. With the proper value of response matrix this problem does not occur. Regards, Sourav On Tue, Dec 8, 2015 at 11:12 AM, Sourav Mazumder < sourav.mazumde...@gmail.com> wrote: > Hi Shirish, > >

Re: DML script parsing error in Spark

on the way > you invoke the script. In a typical scenario, the data comes from HDFS > files. Additionally, in the context of MLContext, it may also come from > RDDs/DataFrames via *registerInput()*. In the context of JMLC, the data can > come in as an in-memory data structure (e.g.,

DML script parsing error in Spark

Hi, I'm facing issue while parsing any DML script. What I'm trying something like - val ml = new MLContext(sc)ml.reset() ml.registerInput("X", Xfc, 3569, 4) ml.registerInput("Y", yDc, 1, 4) ml.registerOutput("beta_out") val cmdLineParams = Map("X" -> " ", "Y" -> " ") val outputs = ml.execute("

Re: Using GLM with Spark

def ($dfam, 1); > > > > The above DML code essentially says $X and $Y are required parameters (a > > design decision that GLM script writer made), whereas $fmt and $dfam are > > optional as they are assigned default values when not explicitly > provided. > > Both these construct

Re: Using GLM with Spark

M script writer made), whereas $fmt and $dfam are > optional as they are assigned default values when not explicitly provided. > Both these constructs are important tools in the arsenal of DML script > writer. By not guarding a dollar parameter with ifdef, the DML script > writer ensures

Using GLM with Spark

Hi, Trying to use GLM with Spark. I go through the documentation of the same in http://apache.github.io/incubator-systemml/algorithms-regression.html#generalized-linear-models I see that inputs like X and Y have to supplied using a file and the file has to be there in HDFS. Is this understanding

Re: API documentation for SystemML

ay to generate Javadocs for the entire Sys]Deron > > Eriksson ---12/07/2015 02:50:30 PM---Hi Sourav, One way to generate > > Javadocs for the entire SystemML project is "mvn > > > > From: Deron Eriksson > > To: dev@systemml.incubator.apache.org > > Date: 12

API documentation for SystemML

Hi, Is there any Scala/Java API documentation available for classes like MatrixCharacteristics, RDDConverterUtils ? What I need to understand is what all such helper utilities available and the deatils of their signature/APIs. Regards, Sourav