Re: [ANNOUNCE] Announcing Spark 2.0.1

Yeah I see the apache maven repos have the 2.0.1 artifacts at https://repository.apache.org/content/repositories/releases/org/apache/spark/spark-core_2.11/ -- Not sure why they haven't synced to maven central yet Shivaram On Wed, Oct 5, 2016 at 8:37 PM, Luciano Resende

Re: [discuss] separate API annotation into two components: InterfaceAudience & InterfaceStability

I think this is fairly important to do so I went ahead and created a PR for the first mini step: https://github.com/apache/spark/pull/15374 On Wed, Aug 24, 2016 at 9:48 AM, Reynold Xin wrote: > Looks like I'm general people like it. Next step is for somebody to take > the

Re: [ANNOUNCE] Announcing Spark 2.0.1

It usually don't take that long to be synced, I still don't see any 2.0.1 related artifacts on maven central http://search.maven.org/#search%7Cga%7C1%7Cg%3A%22org.apache.spark%22%20AND%20v%3A%222.0.1%22 On Tue, Oct 4, 2016 at 1:23 PM, Reynold Xin wrote: > They have been

Re: StructuredStreaming Custom Sinks (motivated by Structured Streaming Machine Learning)

Thanks for the thoughtful comments, Michael and Shivaram. From what I’ve seen in this thread and on JIRA, it looks like the current plan with regard to application-facing APIs for sinks is roughly: 1. Rewrite incremental query compilation for Structured Streaming. 2. Redesign Structured

Re: welcoming Xiao Li as a committer

Congrats, Xiao! Sincerely, DB Tsai -- Web: https://www.dbtsai.com PGP Key ID: 0x9DCC1DBD7FC7BBB2 On Wed, Oct 5, 2016 at 2:36 PM, Fred Reiss wrote: > Congratulations, Xiao! > > Fred > > > On Tuesday, October 4, 2016,

Re: welcoming Xiao Li as a committer

Congratulations, Xiao! Fred On Tuesday, October 4, 2016, Joseph Bradley wrote: > Congrats! > > On Tue, Oct 4, 2016 at 4:09 PM, Kousuke Saruta > wrote: > >> Congratulations Xiao! >> >>

PySpark UDF Performance Exploration w/Jython (Early/rough 2~3X improvement*) [SPARK-15369]

Hi Python Spark Developers & Users, As Datasets/DataFrames are becoming the core building block of Spark, and as someone who cares about Python Spark performance, I've been looking more at PySpark UDF performance. I've got an early WIP/request for comments pull request open

Re: [ANNOUNCE] Announcing Spark 2.0.1

https://github.com/apache/spark/releases/tag/v2.0.1 ? On Wed, Oct 5, 2016 at 8:06 PM Michael Gummelt wrote: > There seems to be no 2.0.1 tag? > > https://github.com/apache/spark/tags > > On Tue, Oct 4, 2016 at 1:23 PM, Reynold Xin wrote: > > They

Re: [ANNOUNCE] Announcing Spark 2.0.1

There is now. Thanks for the email. On Wed, Oct 5, 2016 at 12:06 PM, Michael Gummelt wrote: > There seems to be no 2.0.1 tag? > > https://github.com/apache/spark/tags > > On Tue, Oct 4, 2016 at 1:23 PM, Reynold Xin wrote: > >> They have been

Re: [ANNOUNCE] Announcing Spark 2.0.1

There seems to be no 2.0.1 tag? https://github.com/apache/spark/tags On Tue, Oct 4, 2016 at 1:23 PM, Reynold Xin wrote: > They have been published yesterday, but can take a while to propagate. > > > On Tue, Oct 4, 2016 at 12:58 PM, Prajwal Tuladhar >

Re: java.util.NoSuchElementException when serializing Map with default value

Hi Jakob, I had multiple versions of Spark installed in my machine. The code now works without issues in spark-shell and the IDE. I have verified this with Spark 1.6 and 2.0. Cheers, Kabeer. On Mon, 3 Oct, 2016 at 7:30 PM, Jakob Odersky wrote: Hi Kabeer, which version

Re: [ML]Random Forest Error : Size exceeds Integer.MAX_VALUE

Could you please file a bug report JIRA and also include more info about what you ran? * Random forest Param settings * dataset dimensionality, partitions, etc. Thanks! On Tue, Oct 4, 2016 at 10:44 PM, Samkit Shah wrote: > Hello folks, > I am running Random Forest from ml

Re: welcoming Xiao Li as a committer

Congrats! On Tue, Oct 4, 2016 at 4:09 PM, Kousuke Saruta wrote: > Congratulations Xiao! > > - Kousuke > On 2016/10/05 7:44, Bryan Cutler wrote: > > Congrats Xiao! > > On Tue, Oct 4, 2016 at 11:14 AM, Holden Karau > wrote: > >> Congratulations :D