Re: Unable to run applications on spark in standalone cluster mode

2015-10-25 Thread Rohith P
No.. the ./sbin/start-master.sh --ip option did not work... It is still the same error -- View this message in context: http://apache-spark-developers-list.1001551.n3.nabble.com/Unable-to-run-applications-on-spark-in-standalone-cluster-mode-tp14683p14779.html Sent from the Apache Spark Develop

Re: [VOTE] Release Apache Spark 1.5.2 (RC1)

2015-10-25 Thread Josh Rosen
Hi Mark, The shuffle memory leaks that I identified in SPARK-11239 have been around for multiple releases and it's not clear whether they have caused performance problems in real workloads, so I would say that it's fine to move the release forward without including my patch. If we have to cut anot

Duplicate (?) code paths to handle Executor failures

2015-10-25 Thread Kay Ousterhout
Hi all, I noticed that when the JVM for an executor fails, in Standalone mode, we have two duplicate code paths that handle the failure, one via Akka, and the second via the Worker/ExecutorRunner: via Akka: (1) CoarseGrainedSchedulerBackend is notified that the remote Akka endpoint is disconnecte

Re: [VOTE] Release Apache Spark 1.5.2 (RC1)

2015-10-25 Thread Mark Hamstra
Should 1.5.2 wait for Josh's fix of SPARK-11293? On Sun, Oct 25, 2015 at 2:25 PM, Sean Owen wrote: > The signatures and licenses are fine. I continue to get failures in > these tests though, with "-Pyarn -Phadoop-2.6 -Phive > -Phive-thriftserver" on Ubuntu 15 / Java 7. > > - Unpersisting HttpBro

Re: [VOTE] Release Apache Spark 1.5.2 (RC1)

2015-10-25 Thread Sean Owen
The signatures and licenses are fine. I continue to get failures in these tests though, with "-Pyarn -Phadoop-2.6 -Phive -Phive-thriftserver" on Ubuntu 15 / Java 7. - Unpersisting HttpBroadcast on executors and driver in distributed mode *** FAILED *** java.util.concurrent.TimeoutException: Can'

Re: [VOTE] Release Apache Spark 1.5.2 (RC1)

2015-10-25 Thread Mark Hamstra
You're correct, Sean: That build change isn't in branch-1.5, so the two-phase build is still needed there. On Sun, Oct 25, 2015 at 9:30 AM, Sean Owen wrote: > I believe you still need to "clean package" and then "test" > separately. Or did the change to make that unnecessary go in to 1.5? > > FW

spark-sql / apache-drill / jboss-tiied

2015-10-25 Thread Pranay Tonpay
Hi, In terms of federated query, has anyone done any evaluation between spark-sql and drill and jboss-tiied. I have a very urgent requirement for creating a virtualized layer (sitting atop several databases) and am evaluating these 3 as an option.. Any help would be appreciated. I know Spark-SQL ha

Re: [VOTE] Release Apache Spark 1.5.2 (RC1)

2015-10-25 Thread Sean Owen
I believe you still need to "clean package" and then "test" separately. Or did the change to make that unnecessary go in to 1.5? FWIW I do not see this (my results coming soon) On Sun, Oct 25, 2015 at 4:28 PM, Ted Yu wrote: > When I ran the following command: > ~/apache-maven-3.3.3/bin/mvn -Phiv

Re: Adding support for truncate operator

2015-10-25 Thread Shagun Sodhani
My bad. I did not specify that I meant truncate operator on a column similar to how other maths operators work. On Sun, Oct 25, 2015 at 9:36 PM, Ted Yu wrote: > Have you seen the following ? > [SPARK-3907][SQL] Add truncate table support > > Cheers > > On Sun, Oct 25, 2015 at 9:01 AM, Shagun Sod

Re: [VOTE] Release Apache Spark 1.5.2 (RC1)

2015-10-25 Thread Ted Yu
When I ran the following command: ~/apache-maven-3.3.3/bin/mvn -Phive -Phive-thriftserver -Pyarn -Phadoop-2.4 -Dhadoop.version=2.6.0 package I got: testChildProcLauncher(org.apache.spark.launcher.SparkLauncherSuite) Time elapsed: 0.031 sec <<< FAILURE! java.lang.AssertionError: expected:<0> but

Re: Adding support for truncate operator

2015-10-25 Thread Ted Yu
Have you seen the following ? [SPARK-3907][SQL] Add truncate table support Cheers On Sun, Oct 25, 2015 at 9:01 AM, Shagun Sodhani wrote: > Hi! I noticed that SparkSQL does not support truncate operator as of now. > Can we add it? I am willing to send over a PR for it >

Adding support for truncate operator

2015-10-25 Thread Shagun Sodhani
Hi! I noticed that SparkSQL does not support truncate operator as of now. Can we add it? I am willing to send over a PR for it

[VOTE] Release Apache Spark 1.5.2 (RC1)

2015-10-25 Thread Reynold Xin
Please vote on releasing the following candidate as Apache Spark version 1.5.2. The vote is open until Wed Oct 28, 2015 at 08:00 UTC and passes if a majority of at least 3 +1 PMC votes are cast. [ ] +1 Release this package as Apache Spark 1.5.2 [ ] -1 Do not release this package because ... The