Re: 2.1.0-rc2 cut; committers please set fix version for branch-2.1 to 2.1.1 instead

I went ahead and re-marked all the existing 2.1.1 fix version JIRAs (that had gone into branch-2.1 since RC1 but before RC2) for Spark ML to 2.1.0 On Thu, 8 Dec 2016 at 09:20 Reynold Xin wrote: > Thanks. >

2.1.0-rc2 cut; committers please set fix version for branch-2.1 to 2.1.1 instead

Thanks.

Re: Reduce memory usage of UnsafeInMemorySorter

Unfortunately, I don't have a repro, and I'm only seeing this at scale. But I was able to get around the issue by fiddling with the distribution of my data before asking GraphFrames to process it. (I think that's where the error was being thrown from.) On Wed, Dec 7, 2016 at 7:32 AM Kazuaki

Re: modifications to ALS.scala

I am new to development with spark, how do I do that? Can I write up a custom implementation under package org.apache.spark.ml.recommendation, and specify "spark-mllib" along with others as a library dependency? -- View this message in context:

Re: modifications to ALS.scala

What about putting a custom als implementation into sparks name space? harini schrieb am Do. 8. Dez. 2016 um 00:01: > Hi all, I am trying to implement ALS with a slightly modified objective > function, which will require minor changes to fit -> train -> > computeFactors

modifications to ALS.scala

Hi all,I am trying to implement ALS with a slightly modified objective function, which will require minor changes to fit -> train -> computeFactors within ALS.scala - Is there a way

Publishing of the Spectral LDA model on Spark Packages

Hello, We just published the Spectral LDA model on Spark Packages. It’s an alternative approach to the LDA modelling based on tensor decompositions. We first build the 2nd, 3rd-moment tensors from empirical word counts, then orthogonalise them and perform decomposition on the 3rd-moment

Re: Can I add a new method to RDD class?

Thank you so much, Mark and everyone else responded for putting up with my ignorance. > On Dec 7, 2016, at 2:50 PM, Mark Hamstra [via Apache Spark Developers List] > wrote: > > The easiest way is probably with: > > mvn versions:set

Re: Can I add a new method to RDD class?

The easiest way is probably with: mvn versions:set -DnewVersion=your_new_version On Wed, Dec 7, 2016 at 11:31 AM, Teng Long wrote: > Hi Holden, > > Can you please tell me how to edit version numbers efficiently? the > correct way? I'm really struggling with this and

Re: Reduce memory usage of UnsafeInMemorySorter

I do not have a repro, too. But, when I took a quick browse at the file 'UnsafeInMemorySort.java', I am afraid about the similar cast issue like https://issues.apache.org/jira/browse/SPARK-18458 at the following line.

Re: Seeing bytecode of each task ececuted.

If not the bytecode the code in each task executed will suffice. On Tuesday, December 6, 2016 10:25 PM, Mr rty ff wrote: Hi If there are some way to see the bytecode in each task that is executed by spark. Thanks