git commit: The default version of yarn is equal to the hadoop version

2014-05-03 Thread pwendell
Repository: spark Updated Branches: refs/heads/master 92b2902ca -> fb0543224 The default version of yarn is equal to the hadoop version This is a part of [PR 590](https://github.com/apache/spark/pull/590) Author: witgo Closes #626 from witgo/yarn_version and squashes the following commits:

git commit: The default version of yarn is equal to the hadoop version

2014-05-03 Thread pwendell
Repository: spark Updated Branches: refs/heads/branch-1.0 e88a63625 -> acbf307c2 The default version of yarn is equal to the hadoop version This is a part of [PR 590](https://github.com/apache/spark/pull/590) Author: witgo Closes #626 from witgo/yarn_version and squashes the following commi

svn commit: r1592308 - in /spark: community.md site/community.html

2014-05-03 Thread matei
Author: matei Date: Sun May 4 04:30:08 2014 New Revision: 1592308 URL: http://svn.apache.org/r1592308 Log: new lists Modified: spark/community.md spark/site/community.html Modified: spark/community.md URL: http://svn.apache.org/viewvc/spark/community.md?rev=1592308&r1=1592307&r2=159230

svn commit: r1592307 - in /spark: community.md site/community.html

2014-05-03 Thread matei
Author: matei Date: Sun May 4 04:28:46 2014 New Revision: 1592307 URL: http://svn.apache.org/r1592307 Log: mailing list link Modified: spark/community.md spark/site/community.html Modified: spark/community.md URL: http://svn.apache.org/viewvc/spark/community.md?rev=1592307&r1=1592306&r

git commit: [SQL] Better logging when applying rules.

2014-05-03 Thread rxin
Repository: spark Updated Branches: refs/heads/branch-1.0 8406ac48d -> e24d5cd48 [SQL] Better logging when applying rules. Author: Michael Armbrust Closes #616 from marmbrus/ruleLogging and squashes the following commits: 39c09fe [Michael Armbrust] Fix off by one error. 5af3537 [Michael Arm

git commit: [SQL] Better logging when applying rules.

2014-05-03 Thread rxin
Repository: spark Updated Branches: refs/heads/master 4669a84ab -> b29571470 [SQL] Better logging when applying rules. Author: Michael Armbrust Closes #616 from marmbrus/ruleLogging and squashes the following commits: 39c09fe [Michael Armbrust] Fix off by one error. 5af3537 [Michael Armbrus

git commit: EC2 configurable workers

2014-05-03 Thread pwendell
Repository: spark Updated Branches: refs/heads/branch-1.0 36e687d90 -> 8406ac48d EC2 configurable workers Added option to configure number of worker instances and to set SPARK_MASTER_OPTS Depends on: https://github.com/mesos/spark-ec2/pull/46 Author: Allan Douglas R. de Oliveira Closes #6

git commit: EC2 configurable workers

2014-05-03 Thread pwendell
Repository: spark Updated Branches: refs/heads/master 34719ba32 -> 4669a84ab EC2 configurable workers Added option to configure number of worker instances and to set SPARK_MASTER_OPTS Depends on: https://github.com/mesos/spark-ec2/pull/46 Author: Allan Douglas R. de Oliveira Closes #612 f

svn commit: r1592285 - in /spark: downloads.md site/downloads.html

2014-05-03 Thread matei
Author: matei Date: Sat May 3 22:00:28 2014 New Revision: 1592285 URL: http://svn.apache.org/r1592285 Log: simplify downloads a bit Modified: spark/downloads.md spark/site/downloads.html Modified: spark/downloads.md URL: http://svn.apache.org/viewvc/spark/downloads.md?rev=1592285&r1=15

svn commit: r1592284 - in /spark: examples.md index.md site/examples.html site/index.html

2014-05-03 Thread matei
Author: matei Date: Sat May 3 21:55:05 2014 New Revision: 1592284 URL: http://svn.apache.org/r1592284 Log: show Python examples first Modified: spark/examples.md spark/index.md spark/site/examples.html spark/site/index.html Modified: spark/examples.md URL: http://svn.apache.org

git commit: SPARK-1689 AppClient should indicate app is dead() when removed

2014-05-03 Thread pwendell
Repository: spark Updated Branches: refs/heads/branch-1.0 bc3bfeaff -> 36e687d90 SPARK-1689 AppClient should indicate app is dead() when removed Previously, we indicated disconnected(), which keeps the application in a limbo state where it has no executors but thinks it will get them soon. T

git commit: SPARK-1689 AppClient should indicate app is dead() when removed

2014-05-03 Thread pwendell
Repository: spark Updated Branches: refs/heads/master ce72c72ae -> 34719ba32 SPARK-1689 AppClient should indicate app is dead() when removed Previously, we indicated disconnected(), which keeps the application in a limbo state where it has no executors but thinks it will get them soon. This

git commit: [Bugfix] Tachyon file cleanup logical error

2014-05-03 Thread pwendell
Repository: spark Updated Branches: refs/heads/branch-1.0 08c4d1112 -> bc3bfeaff [Bugfix] Tachyon file cleanup logical error Should lookup `shutdownDeleteTachyonPaths` instead of `shutdownDeletePaths`. Together with a minor style clean up: `find {...}.isDefined` to `exists {...}`. Author: Ch

git commit: [Bugfix] Tachyon file cleanup logical error

2014-05-03 Thread pwendell
Repository: spark Updated Branches: refs/heads/master 11d549417 -> ce72c72ae [Bugfix] Tachyon file cleanup logical error Should lookup `shutdownDeleteTachyonPaths` instead of `shutdownDeletePaths`. Together with a minor style clean up: `find {...}.isDefined` to `exists {...}`. Author: Cheng

git commit: SPARK-1663. Corrections for several compile errors in streaming code examples, and updates to follow API changes

2014-05-03 Thread pwendell
Repository: spark Updated Branches: refs/heads/branch-1.0 0441515f2 -> 08c4d1112 SPARK-1663. Corrections for several compile errors in streaming code examples, and updates to follow API changes I gave the Streaming code examples, both Scala and Java, a test run today. I turned up a number of

git commit: SPARK-1663. Corrections for several compile errors in streaming code examples, and updates to follow API changes

2014-05-03 Thread pwendell
Repository: spark Updated Branches: refs/heads/master 3d0a02dff -> 11d549417 SPARK-1663. Corrections for several compile errors in streaming code examples, and updates to follow API changes I gave the Streaming code examples, both Scala and Java, a test run today. I turned up a number of sma

git commit: [WIP] SPARK-1676: Cache Hadoop UGIs by default to prevent FileSystem leak

2014-05-03 Thread adav
Repository: spark Updated Branches: refs/heads/branch-0.9 54c3b7e3b -> 45561cd9f [WIP] SPARK-1676: Cache Hadoop UGIs by default to prevent FileSystem leak Move the doAs in Executor higher up so that we only have 1 ugi and aren't leaking filesystems. Fix spark on yarn to work when the cluster

git commit: [WIP] SPARK-1676: Cache Hadoop UGIs by default to prevent FileSystem leak

2014-05-03 Thread adav
Repository: spark Updated Branches: refs/heads/master 9347565f4 -> 3d0a02dff [WIP] SPARK-1676: Cache Hadoop UGIs by default to prevent FileSystem leak Move the doAs in Executor higher up so that we only have 1 ugi and aren't leaking filesystems. Fix spark on yarn to work when the cluster is r

git commit: [WIP] SPARK-1676: Cache Hadoop UGIs by default to prevent FileSystem leak

2014-05-03 Thread adav
Repository: spark Updated Branches: refs/heads/branch-1.0 34f22bcc4 -> 0441515f2 [WIP] SPARK-1676: Cache Hadoop UGIs by default to prevent FileSystem leak Move the doAs in Executor higher up so that we only have 1 ugi and aren't leaking filesystems. Fix spark on yarn to work when the cluster

git commit: Update SchemaRDD.scala

2014-05-03 Thread pwendell
Repository: spark Updated Branches: refs/heads/branch-1.0 d2cbd3d76 -> 34f22bcc4 Update SchemaRDD.scala Modify spelling errors Author: ArcherShao Closes #619 from ArcherShao/patch-1 and squashes the following commits: 2957195 [ArcherShao] Update SchemaRDD.scala (cherry picked from commit 9

git commit: Update SchemaRDD.scala

2014-05-03 Thread pwendell
Repository: spark Updated Branches: refs/heads/master 0a1442176 -> 9347565f4 Update SchemaRDD.scala Modify spelling errors Author: ArcherShao Closes #619 from ArcherShao/patch-1 and squashes the following commits: 2957195 [ArcherShao] Update SchemaRDD.scala Project: http://git-wip-us.apa

git commit: SPARK-1700: Close socket file descriptors on task completion

2014-05-03 Thread adav
Repository: spark Updated Branches: refs/heads/branch-1.0 a314342da -> d2cbd3d76 SPARK-1700: Close socket file descriptors on task completion This will ensure that sockets do not build up over the course of a job, and that cancellation successfully cleans up sockets. Tested in standalone mod