Recreating JIRA SPARK-8142

2015-06-09 Thread Devl Devel
Hi All We are having some trouble with: sparkConf.set(spark.driver.userClassPathFirst,true); sparkConf.set(spark.executor.userClassPathFirst,true); and would appreciate some independent verification. The issue comes down to this: Spark 1.3.1 hadoop 2.6 is deployed on the cluster. In my

Re: GraphX: New graph operator

2015-06-09 Thread jinkui.sjk
i have a pull request about this issue. https://github.com/apache/spark/pull/6685 https://github.com/apache/spark/pull/6685 the union operation of two graph is useful in practice. And it’s necessary to provide operation on the Graph level. On 3 Jun, 2015, at 2:58 pm, Reynold Xin

Re: [VOTE] Release Apache Spark 1.4.0 (RC4)

2015-06-09 Thread Sean McNamara
+1 tested /w OS X + deployed one of our streaming apps onto a staging yarn cluster. Sean On Jun 2, 2015, at 9:54 PM, Patrick Wendell pwend...@gmail.com wrote: Please vote on releasing the following candidate as Apache Spark version 1.4.0! The tag to be voted on is v1.4.0-rc3 (commit

Re: Spark on Mesos vs Yarn

2015-06-09 Thread Timothy Chen
Hi Nik, Bharath is mostly referring to Spark commiters in this thread. Tim On Tue, Jun 9, 2015 at 9:51 PM, Niklas Nielsen nik...@mesosphere.io wrote: Hi Bharath (and rest of Spark dev list!), Just a small shout out: I am a Apache Mesos Committer and would love to help out with anything you

Re: Stages with non-arithmetic numbering Timing metrics in event logs

2015-06-09 Thread Mike Hynes
Hi Imran, Thank you for your email. In examing the condition (t2 - t1) (t_ser + t_deser + t_exec), I have found it to be true, although I have not included the t_{wait_for_read} in this, since it is---so far as I can tell---been either zero or negligible compared to the task time. Thanks, Mike

Re: Stages with non-arithmetic numbering Timing metrics in event logs

2015-06-09 Thread Mike Hynes
Ahhh---forgive my typo: what I mean is, (t2 - t1) = (t_ser + t_deser + t_exec) is satisfied, empirically. On 6/10/15, Mike Hynes 91m...@gmail.com wrote: Hi Imran, Thank you for your email. In examing the condition (t2 - t1) (t_ser + t_deser + t_exec), I have found it to be true, although I

About akka used in spark

2015-06-09 Thread wangtao (A)
Hi guys, I see group id of akka used in spark is org.spark-project.akka. What is its difference with the typesafe one? What is its version? And where can we get the source code? Regards.

Re: Spark on Mesos vs Yarn

2015-06-09 Thread Bharath Ravi Kumar
All, Despite the common origin of spark mesos, the stability and adoption of mesos, and the age of the spark-mesos binding, I find the mesos support less mature, with fundamental shortcomings (like framework auth https://issues.apache.org/jira/browse/SPARK-6284) remaining unresolved. If there's

Re: Spark on Mesos vs Yarn

2015-06-09 Thread Niklas Nielsen
Hi Bharath (and rest of Spark dev list!), Just a small shout out: I am a Apache Mesos Committer and would love to help out with anything you need to get this going. Cheers, Nik On 9 June 2015 at 21:10, Bharath Ravi Kumar reachb...@gmail.com wrote: All, Despite the common origin of spark