Re: [spark-streaming] Is this LocalInputDStream useful to someone?

2014-03-27 Thread Pascal Voitot Dev
Hi guys, Nobody saw/answered my question so I resend it ;) Thanks Pascal On Thu, Mar 20, 2014 at 9:17 AM, Pascal Voitot Dev pascal.voitot@gmail.com wrote: Hi guys, In my recent blog post (http://mandubian.com/2014/03/08/zpark-ml-nio-1/), I needed to have an InputDStream helper looking

Re: Spark 0.9.1 release

2014-03-27 Thread Tathagata Das
I have cut another release candidate, RC3, with two important bug fixes. See the following JIRAs for more details. 1. Bug with intercepts in MLLib's GLM: https://spark-project.atlassian.net/browse/SPARK-1327 2. Bug in PySpark's RDD.top() ordering:

[DISCUSS] Shepherding PRs

2014-03-27 Thread Andy Konwinski
I thought this email exchange from the Mesos dev list was worth sharing. The Mesos project is trying out a process wherein they assign shepherds (who are committers) to significant issues. I'm not proposing that this necessarily makes sense for us, but I thought it might be worth discussing.

Re: [DISCUSS] Shepherding PRs

2014-03-27 Thread Evan Chan
+1. I really like this idea. I know having a shepherd would have been really helpful for a couple of changes. On Thu, Mar 27, 2014 at 8:18 AM, Andy Konwinski andykonwin...@gmail.comwrote: I thought this email exchange from the Mesos dev list was worth sharing. The Mesos project is trying

Re: Any suggestion about JIRA 1006 MLlib ALS gets stack overflow with too many iterations?

2014-03-27 Thread Debasish Das
Hi Matei, I am hitting similar problems with 10 ALS iterations...I am running with 24 gb executor memory on 10 nodes for 20M x 3 M matrix with rank =50 The first iteration of flatMaps run fine which means that the memory requirements are good per iteration... If I do check-pointing on RDD, most

Re: Scala 2.10.4

2014-03-27 Thread Matei Zaharia
Sounds good. Feel free to send a PR even though it’s a small change (it leads to better Git history and such). Matei On Mar 27, 2014, at 4:15 PM, Mark Hamstra m...@clearstorydata.com wrote: FYI, Spark master does build cleanly and the tests do run successfully with Scala version set to

Re: Mailbomb from amplabs jenkins ?

2014-03-27 Thread Nan Zhu
yes, it sends for every PR you were involved I think Patrick is doing something on Jenkins, he just stopped some testing jobs manually Best, -- Nan Zhu On Thursday, March 27, 2014 at 11:07 PM, Mridul Muralidharan wrote: Got some 100 odd mails from jenkins (?) with Can one of the admins