Re: [VOTE] Release Apache Spark 2.0.1 (RC2)

There is no "mesos" profile in 2.0.1. On Sat, Sep 24, 2016 at 2:19 PM, Jacek Laskowski wrote: > Hi, > > I keep asking myself why are you guys not including -Pmesos in your > builds? Is this on purpose or have you overlooked it? > > Pozdrawiam, > Jacek Laskowski > >

Re: Documentation for package ‘SparkR’ version mismatch

Thanks for reporting. I sent out an email for rc3 fixing the issue. We have also automated the version number update for documentation pages so this won't happen again in the future. On Fri, Sep 23, 2016 at 12:25 AM, Jagadeesan As wrote: > Hi Reyonld, > > While checking the

[VOTE] Release Apache Spark 2.0.1 (RC3)

Please vote on releasing the following candidate as Apache Spark version 2.0.1. The vote is open until Tue, Sep 27, 2016 at 15:30 PDT and passes if a majority of at least 3+1 PMC votes are cast. [ ] +1 Release this package as Apache Spark 2.0.1 [ ] -1 Do not release this package because ... The

Re: [VOTE] Release Apache Spark 2.0.1 (RC2)

Hi all, The R API documentation version error was reported in a separate thread. I've built a release candidate (RC3) and will send out a new vote email in a bit. On Thu, Sep 22, 2016 at 11:01 PM, Reynold Xin wrote: > Please vote on releasing the following candidate as

Re: [VOTE] Release Apache Spark 2.0.1 (RC2)

The binary artifact that's published here is built with -Pmesos. The 'real' artifact from a process standpoint is the source release, however. That's why we do (should) test the source release foremost. I suppose individuals are invited to test with a configuration that is of interest to them,

Re: [VOTE] Release Apache Spark 2.0.1 (RC2)

Hi, I keep asking myself why are you guys not including -Pmesos in your builds? Is this on purpose or have you overlooked it? Pozdrawiam, Jacek Laskowski https://medium.com/@jaceklaskowski/ Mastering Apache Spark 2.0 http://bit.ly/mastering-apache-spark Follow me at

Re: [VOTE] Release Apache Spark 2.0.1 (RC2)

+1 (non binding) I compiled and tested on the following two systems. - CentOS 7.2 / Oracle JDK 1.8.0_77 / R 3.3.1 with -Pyarn -Phadoop-2.7 -Pkinesis-asl -Phive -Phive-thriftserver -Dsparkr - CentOS 7.2 / Open JDK 1.8.0_102 with -Pyarn -Phadoop-2.7 -Pkinesis-asl -Phive -Phive-thriftserver Bests,

Re: Why Expression.deterministic method and Nondeterministic trait?

Hi Reynold, That's my point indeed. If non-determinism is a trait/property of a class (objects really at runtime) it's by definition part of a trait and only Nondeterministic expressions are...well...non-deterministic. I'm yet to review the code regarding the trait, but I suspect that there are

Re: Spark job fails as soon as it starts. Driver requested a total number of 168510 executor

We have too many (large) files. We have about 30k partitions with about 4 years worth data and we need to process entire history in a one time monolithic job. I would like to know how spark decides the number of executors requested. I've seen testcases where the max executors count is Integer's

Re: What's the use of RangePartitioner.hashCode

thank you! -- View this message in context: http://apache-spark-developers-list.1001551.n3.nabble.com/What-s-the-use-of-RangePartitioner-hashCode-tp18953p19037.html Sent from the Apache Spark Developers List mailing list archive at Nabble.com.

Re: @scala.annotation.varargs or @_root_.scala.annotation.varargs?

On Sat, Sep 24, 2016 at 5:27 AM, Hyukjin Kwon wrote: > Then, are we going to submit a PR and fix this maybe? https://issues.apache.org/jira/browse/SPARK-17656 Thanks Hyukjin! Unless someone beats me to it, I'm going to have a PR over the weekend. Jacek