svn commit: r1624421 - in /spark: releases/_posts/2014-09-11-spark-release-1-1-0.md site/releases/spark-release-1-1-0.html

2014-09-11 Thread pwendell
Author: pwendell Date: Thu Sep 11 23:47:07 2014 New Revision: 1624421 URL: http://svn.apache.org/r1624421 Log: Adding missing files from previos commit Added: spark/releases/_posts/2014-09-11-spark-release-1-1-0.md spark/site/releases/spark-release-1-1-0.html Added: spark/releases

svn commit: r1624423 - in /spark: js/downloads.js site/js/downloads.js

2014-09-11 Thread pwendell
Author: pwendell Date: Thu Sep 11 23:53:16 2014 New Revision: 1624423 URL: http://svn.apache.org/r1624423 Log: Adding proper link tracking Modified: spark/js/downloads.js spark/site/js/downloads.js Modified: spark/js/downloads.js URL: http://svn.apache.org/viewvc/spark/js/downloads.js

svn commit: r1624424 - in /spark: news/_posts/2014-09-11-spark-1-1-0-released.md site/news/index.html site/news/spark-1-1-0-released.html

2014-09-11 Thread pwendell
Author: pwendell Date: Fri Sep 12 00:01:09 2014 New Revision: 1624424 URL: http://svn.apache.org/r1624424 Log: Minor typo's in the release post Modified: spark/news/_posts/2014-09-11-spark-1-1-0-released.md spark/site/news/index.html spark/site/news/spark-1-1-0-released

svn commit: r1624427 - in /spark: news/_posts/2014-09-11-spark-1-1-0-released.md releases/_posts/2014-09-11-spark-release-1-1-0.md site/news/spark-1-1-0-released.html site/releases/spark-release-1-1-0

2014-09-11 Thread pwendell
Author: pwendell Date: Fri Sep 12 00:08:19 2014 New Revision: 1624427 URL: http://svn.apache.org/r1624427 Log: More 1.1 typo fixes Modified: spark/news/_posts/2014-09-11-spark-1-1-0-released.md spark/releases/_posts/2014-09-11-spark-release-1-1-0.md spark/site/news/spark-1-1-0

svn commit: r1624428 - in /spark: news/_posts/2014-09-11-spark-1-1-0-released.md releases/_posts/2014-09-11-spark-release-1-1-0.md site/news/index.html site/news/spark-1-1-0-released.html site/release

2014-09-11 Thread pwendell
Author: pwendell Date: Fri Sep 12 00:10:00 2014 New Revision: 1624428 URL: http://svn.apache.org/r1624428 Log: More 1.1 typo fixes Modified: spark/news/_posts/2014-09-11-spark-1-1-0-released.md spark/releases/_posts/2014-09-11-spark-release-1-1-0.md spark/site/news/index.html

svn commit: r1624453 - in /spark: downloads.md releases/_posts/2014-09-11-spark-release-1-1-0.md site/downloads.html site/releases/spark-release-1-1-0.html

2014-09-11 Thread pwendell
Author: pwendell Date: Fri Sep 12 05:57:56 2014 New Revision: 1624453 URL: http://svn.apache.org/r1624453 Log: More changes from typo's etc for the 1.1 content. Modified: spark/downloads.md spark/releases/_posts/2014-09-11-spark-release-1-1-0.md spark/site/downloads.html

svn commit: r1624578 - in /spark: releases/_posts/2014-09-11-spark-release-1-1-0.md site/releases/spark-release-1-1-0.html

2014-09-12 Thread pwendell
Author: pwendell Date: Fri Sep 12 16:03:14 2014 New Revision: 1624578 URL: http://svn.apache.org/r1624578 Log: Adding missing release credit Modified: spark/releases/_posts/2014-09-11-spark-release-1-1-0.md spark/site/releases/spark-release-1-1-0.html Modified: spark/releases/_posts

svn commit: r1624579 - in /spark: releases/_posts/2014-09-11-spark-release-1-1-0.md site/releases/spark-release-1-1-0.html

2014-09-12 Thread pwendell
Author: pwendell Date: Fri Sep 12 16:04:01 2014 New Revision: 1624579 URL: http://svn.apache.org/r1624579 Log: Adding missing release credit Modified: spark/releases/_posts/2014-09-11-spark-release-1-1-0.md spark/site/releases/spark-release-1-1-0.html Modified: spark/releases/_posts

svn commit: r1624639 - in /spark: downloads.md js/downloads.js site/downloads.html site/js/downloads.js

2014-09-12 Thread pwendell
Author: pwendell Date: Fri Sep 12 20:36:11 2014 New Revision: 1624639 URL: http://svn.apache.org/r1624639 Log: Adding pretty titles for download links. Modified: spark/downloads.md spark/js/downloads.js spark/site/downloads.html spark/site/js/downloads.js Modified: spark

git commit: MAINTENANCE: Automated closing of pull requests.

2014-09-12 Thread pwendell
Repository: spark Updated Branches: refs/heads/master 8194fc662 -> eae81b0bf MAINTENANCE: Automated closing of pull requests. This commit exists to close the following pull requests on Github: Closes #930 (close requested by 'andrewor14') Closes #867 (close requested by 'marmbrus') Closes #18

git commit: [SPARK-3217] Add Guava to classpath when SPARK_PREPEND_CLASSES is set.

2014-09-12 Thread pwendell
Repository: spark Updated Branches: refs/heads/master 1d767967e -> af2583826 [SPARK-3217] Add Guava to classpath when SPARK_PREPEND_CLASSES is set. When that option is used, the compiled classes from the build directory are prepended to the classpath. Now that we avoid packaging Guava, that me

svn commit: r6504 - /release/spark/spark-0.9.1/

2014-09-14 Thread pwendell
Author: pwendell Date: Sun Sep 14 22:37:02 2014 New Revision: 6504 Log: Removing 0.9.1 release. Removed: release/spark/spark-0.9.1/ - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands

svn commit: r6506 - /release/spark/spark-1.0.1/

2014-09-14 Thread pwendell
Author: pwendell Date: Sun Sep 14 22:37:38 2014 New Revision: 6506 Log: Remvoing Spark 1.0.1 release. Removed: release/spark/spark-1.0.1/ - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional

svn commit: r6505 - /release/spark/spark-1.0.0/

2014-09-14 Thread pwendell
Author: pwendell Date: Sun Sep 14 22:37:23 2014 New Revision: 6505 Log: Removing Spark 1.0.0 release. Removed: release/spark/spark-1.0.0/ - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional

svn commit: r6507 - /release/spark/spark-0.8.1-incubating/

2014-09-14 Thread pwendell
Author: pwendell Date: Sun Sep 14 22:38:17 2014 New Revision: 6507 Log: Removing spark 0.8.1-incubating release. Removed: release/spark/spark-0.8.1-incubating/ - To unsubscribe, e-mail: commits-unsubscr

svn commit: r6508 - /release/spark/spark-0.9.2/

2014-09-14 Thread pwendell
Author: pwendell Date: Sun Sep 14 22:41:11 2014 New Revision: 6508 Log: Removing Spark 0.9.2 release. Removed: release/spark/spark-0.9.2/ - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional

svn commit: r1624923 - in /spark: js/downloads.js site/js/downloads.js

2014-09-14 Thread pwendell
Author: pwendell Date: Sun Sep 14 22:57:54 2014 New Revision: 1624923 URL: http://svn.apache.org/r1624923 Log: Using archive link for older releases Modified: spark/js/downloads.js spark/site/js/downloads.js Modified: spark/js/downloads.js URL: http://svn.apache.org/viewvc/spark/js

git commit: SPARK-3039: Allow spark to be built using avro-mapred for hadoop2

2014-09-14 Thread pwendell
Repository: spark Updated Branches: refs/heads/branch-1.1 70f93d5a9 -> 78887f94a SPARK-3039: Allow spark to be built using avro-mapred for hadoop2 SPARK-3039: Adds the maven property "avro.mapred.classifier" to build spark-assembly with avro-mapred with support for the new Hadoop API. Sets th

git commit: SPARK-3039: Allow spark to be built using avro-mapred for hadoop2

2014-09-14 Thread pwendell
Repository: spark Updated Branches: refs/heads/master 4e3fbe8cd -> c243b21a8 SPARK-3039: Allow spark to be built using avro-mapred for hadoop2 SPARK-3039: Adds the maven property "avro.mapred.classifier" to build spark-assembly with avro-mapred with support for the new Hadoop API. Sets this

git commit: [SPARK-3452] Maven build should skip publishing artifacts people shouldn...

2014-09-14 Thread pwendell
Repository: spark Updated Branches: refs/heads/master c243b21a8 -> f493f7982 [SPARK-3452] Maven build should skip publishing artifacts people shouldn... ...'t depend on Publish local in maven term is `install` and publish otherwise is `deploy` So disabled both for following projects. Autho

git commit: [SPARK-3425] do not set MaxPermSize for OpenJDK 1.8

2014-09-15 Thread pwendell
R_REMOTE_NAME = os.environ.get("PR_REMOTE_NAME", "apache-github") # Remote name which points to Apache git PUSH_REMOTE_NAME = os.environ.get("PUSH_REMOTE_NAME", "apache") # ASF JIRA username -JIRA_USERNAME = os.environ.get("JIRA_USERNAME&qu

git commit: [SPARK-3518] Remove wasted statement in JsonProtocol

2014-09-15 Thread pwendell
Repository: spark Updated Branches: refs/heads/master fe2b1d6a2 -> e59fac1f9 [SPARK-3518] Remove wasted statement in JsonProtocol Author: Kousuke Saruta Closes #2380 from sarutak/SPARK-3518 and squashes the following commits: 8a1464e [Kousuke Saruta] Replaced a variable with simple field re

git commit: [SPARK-3518] Remove wasted statement in JsonProtocol

2014-09-15 Thread pwendell
Repository: spark Updated Branches: refs/heads/branch-1.1 78887f94a -> 99a6c5e5e [SPARK-3518] Remove wasted statement in JsonProtocol Author: Kousuke Saruta Closes #2380 from sarutak/SPARK-3518 and squashes the following commits: 8a1464e [Kousuke Saruta] Replaced a variable with simple fiel

git commit: [SPARK-3040] pick up a more proper local ip address for Utils.findLocalIpAddress method

2014-09-15 Thread pwendell
Repository: spark Updated Branches: refs/heads/master ecf0c0293 -> febafefa5 [SPARK-3040] pick up a more proper local ip address for Utils.findLocalIpAddress method Short version: NetworkInterface.getNetworkInterfaces returns ifs in reverse order compared to ifconfig output. It may pick up i

git commit: SPARK-3069 [DOCS] Build instructions in README are outdated

2014-09-16 Thread pwendell
Repository: spark Updated Branches: refs/heads/master febafefa5 -> 61e21fe7f SPARK-3069 [DOCS] Build instructions in README are outdated Here's my crack at Bertrand's suggestion. The Github `README.md` contains build info that's outdated. It should just point to the current online docs, and

git commit: [SPARK-2182] Scalastyle rule blocking non ascii characters.

2014-09-16 Thread pwendell
Repository: spark Updated Branches: refs/heads/master 61e21fe7f -> 7b8008f5a [SPARK-2182] Scalastyle rule blocking non ascii characters. ...erators. Author: Prashant Sharma Closes #2358 from ScrapCodes/scalastyle-unicode and squashes the following commits: 12a20f2 [Prashant Sharma] [SPARK

git commit: [SPARK-787] Add S3 configuration parameters to the EC2 deploy scripts

2014-09-16 Thread pwendell
Repository: spark Updated Branches: refs/heads/master ec1adecbb -> b20171267 [SPARK-787] Add S3 configuration parameters to the EC2 deploy scripts When deploying to AWS, there is additional configuration that is required to read S3 files. EMR creates it automatically, there is no reason that

git commit: Add a Community Projects page

2014-09-16 Thread pwendell
Repository: spark Updated Branches: refs/heads/master b20171267 -> a6e1712f1 Add a Community Projects page This adds a new page to the docs listing community projects -- those created outside of Apache Spark that are of interest to the community of Spark users. Anybody can add to it just b

git commit: [Minor]ignore all config files in conf

2014-09-16 Thread pwendell
Repository: spark Updated Branches: refs/heads/master 0a7091e68 -> 008a5ed48 [Minor]ignore all config files in conf Some config files in ```conf``` should ignore, such as conf/fairscheduler.xml conf/hive-log4j.properties conf/metrics.properties ... So ignore all ```sh``

git commit: [Docs] minor grammar fix

2014-09-17 Thread pwendell
Repository: spark Updated Branches: refs/heads/master 7d1a37239 -> 8fbd5f4a9 [Docs] minor grammar fix Author: Nicholas Chammas Closes #2430 from nchammas/patch-2 and squashes the following commits: d476bfb [Nicholas Chammas] [Docs] minor grammar fix Project: http://git-wip-us.apache.org/r

svn commit: r1625829 - in /spark: _config.yml news/_posts/2012-12-21-spark-and-shark-in-the-news.md news/_posts/2013-06-21-spark-featured-in-wired.md

2014-09-17 Thread pwendell
Author: pwendell Date: Wed Sep 17 21:17:41 2014 New Revision: 1625829 URL: http://svn.apache.org/r1625829 Log: Changes to provide portability across Jekyll environments Modified: spark/_config.yml spark/news/_posts/2012-12-21-spark-and-shark-in-the-news.md spark/news/_posts/2013-06

svn commit: r1625830 - in /spark: releases/_posts/2014-09-11-spark-release-1-1-0.md site/releases/spark-release-1-1-0.html

2014-09-17 Thread pwendell
Author: pwendell Date: Wed Sep 17 21:21:22 2014 New Revision: 1625830 URL: http://svn.apache.org/r1625830 Log: Typos and omissions in 1.1.0 release notes Modified: spark/releases/_posts/2014-09-11-spark-release-1-1-0.md spark/site/releases/spark-release-1-1-0.html Modified: spark

svn commit: r1625867 - in /spark/site/docs/1.1.0: ./ css/

2014-09-17 Thread pwendell
Author: pwendell Date: Thu Sep 18 00:53:30 2014 New Revision: 1625867 URL: http://svn.apache.org/r1625867 Log: Fixing version in 1.1 documentation Modified: spark/site/docs/1.1.0/bagel-programming-guide.html spark/site/docs/1.1.0/css/bootstrap.min.css spark/site/docs/1.1.0/graphx

git commit: [SPARK-3565]Fix configuration item not consistent with document

2014-09-17 Thread pwendell
Repository: spark Updated Branches: refs/heads/branch-1.1 3f1f9744b -> 32fe9 [SPARK-3565]Fix configuration item not consistent with document https://issues.apache.org/jira/browse/SPARK-3565 "spark.ports.maxRetries" should be "spark.port.maxRetries". Make the configuration keys in documen

git commit: [SPARK-3565]Fix configuration item not consistent with document

2014-09-17 Thread pwendell
Repository: spark Updated Branches: refs/heads/master 1147973f1 -> 3f169bfe3 [SPARK-3565]Fix configuration item not consistent with document https://issues.apache.org/jira/browse/SPARK-3565 "spark.ports.maxRetries" should be "spark.port.maxRetries". Make the configuration keys in document an

git commit: [SPARK-3534] Add hive-thriftserver to SQL tests

2014-09-17 Thread pwendell
Repository: spark Updated Branches: refs/heads/master 3f169bfe3 -> 5547fa1ee [SPARK-3534] Add hive-thriftserver to SQL tests Addresses the problem pointed out in [this comment](https://github.com/apache/spark/pull/2441#issuecomment-55990116). Author: Nicholas Chammas Closes #2442 from ncha

git commit: [Minor] rat exclude dependency-reduced-pom.xml

2014-09-17 Thread pwendell
Repository: spark Updated Branches: refs/heads/master 5547fa1ee -> 6772afec2 [Minor] rat exclude dependency-reduced-pom.xml Author: GuoQiang Li Closes #2326 from witgo/rat-excludes and squashes the following commits: 860904e [GuoQiang Li] rat exclude dependency-reduced-pom.xml Project: ht

git commit: [SPARK-3547]Using a special exit code instead of 1 to represent ClassNotFoundExcepti...

2014-09-18 Thread pwendell
Repository: spark Updated Branches: refs/heads/master 6772afec2 -> 3447d1009 [SPARK-3547]Using a special exit code instead of 1 to represent ClassNotFoundExcepti... ...on As improvement of https://github.com/apache/spark/pull/1944, we should use more special exit code to represent ClassNotF

git commit: SPARK-3579 Jekyll doc generation is different across environments.

2014-09-18 Thread pwendell
hen building the upstream docs. 2. We add a configuration for a property that for some reason varies across packages of Jekyll/Kramdown even with the same version. Author: Patrick Wendell Closes #2443 from pwendell/jekyll and squashes the following commits: 54ee2ab [Patrick Wendell] SPARK-3

git commit: [SPARK-3566] [BUILD] .gitignore and .rat-excludes should consider Windows cmd file and Emacs' backup files

2014-09-18 Thread pwendell
Repository: spark Updated Branches: refs/heads/master 3ad4176cf -> 6cab838b9 [SPARK-3566] [BUILD] .gitignore and .rat-excludes should consider Windows cmd file and Emacs' backup files Author: Kousuke Saruta Closes #2426 from sarutak/emacs-metafiles-ignore and squashes the following commits

git commit: [SPARK-3589][Minor]remove redundant code

2014-09-18 Thread pwendell
Repository: spark Updated Branches: refs/heads/master 6cab838b9 -> 471e6a3a4 [SPARK-3589][Minor]remove redundant code https://issues.apache.org/jira/browse/SPARK-3589 "export CLASSPATH" in spark-class is redundant since same variable is exported before. We could reuse defined value "isYarnCl

git commit: [SPARK-3589][Minor]remove redundant code

2014-09-18 Thread pwendell
Repository: spark Updated Branches: refs/heads/branch-1.1 32fe9 -> 2b286926b [SPARK-3589][Minor]remove redundant code https://issues.apache.org/jira/browse/SPARK-3589 "export CLASSPATH" in spark-class is redundant since same variable is exported before. We could reuse defined value "isYa

git commit: MAINTENANCE: Automated closing of pull requests.

2014-09-19 Thread pwendell
Repository: spark Updated Branches: refs/heads/master 3bbbdd818 -> a48956f58 MAINTENANCE: Automated closing of pull requests. This commit exists to close the following pull requests on Github: Closes #726 (close requested by 'pwendell') Closes #151 (close requested by 'pw

git commit: SPARK-3574. Shuffle finish time always reported as -1

2014-09-20 Thread pwendell
Repository: spark Updated Branches: refs/heads/master 5f8833c67 -> 7c8ad1c08 SPARK-3574. Shuffle finish time always reported as -1 The included test waits 100 ms after job completion for task completion events to come in so it can verify they have reasonable finish times. Does anyone know a

git commit: [SPARK-3599]Avoid loading properties file frequently

2014-09-20 Thread pwendell
Repository: spark Updated Branches: refs/heads/master 293ce8514 -> 8e875d2af [SPARK-3599]Avoid loading properties file frequently https://issues.apache.org/jira/browse/SPARK-3599 Author: WangTao Author: WangTaoTheTonic Closes #2454 from WangTaoTheTonic/avoidLoadingFrequently and squashes t

git commit: MAINTENANCE: Automated closing of pull requests.

2014-09-20 Thread pwendell
Repository: spark Updated Branches: refs/heads/master 8e875d2af -> d112a6c79 MAINTENANCE: Automated closing of pull requests. This commit exists to close the following pull requests on Github: Closes #1328 (close requested by 'pwendell') Closes #2314 (close requested by '

git commit: [SPARK-3595] Respect configured OutputCommitters when calling saveAsHadoopFile

2014-09-21 Thread pwendell
ate based on comments from pwendell a11d9f3 [Ian Hummel] Fix formatting 4359664 [Ian Hummel] Add an example showing usage 8b6be94 [Ian Hummel] Add ability to specify OutputCommitter, espcially useful when writing to an S3 bucket from an EMR cluster Project: http://git-wip-us.apache.org/repos/asf/sp

git commit: [SPARK-3595] Respect configured OutputCommitters when calling saveAsHadoopFile

2014-09-21 Thread pwendell
ate based on comments from pwendell a11d9f3 [Ian Hummel] Fix formatting 4359664 [Ian Hummel] Add an example showing usage 8b6be94 [Ian Hummel] Add ability to specify OutputCommitter, espcially useful when writing to an S3 bucket from an EMR cluster Project: http://git-wip-us.apache.org/repos/

git commit: Revert "[SPARK-3595] Respect configured OutputCommitters when calling saveAsHadoopFile"

2014-09-21 Thread pwendell
Repository: spark Updated Branches: refs/heads/branch-1.1 7a766577a -> f5bf7dedb Revert "[SPARK-3595] Respect configured OutputCommitters when calling saveAsHadoopFile" This reverts commit 7a766577a466377bf504fa2d8c3ca454844a6ea6. [NOTE: After some thought I decided not to merge this into 1.

git commit: [Minor]ignore .idea_modules

2014-09-21 Thread pwendell
Repository: spark Updated Branches: refs/heads/master a0454efe2 -> fd0b32c52 [Minor]ignore .idea_modules ignore .idea_modules , ```sbt/sbt gen-idea``` generate this dir. Author: wangfei Closes #2476 from scwf/patch-4 and squashes the following commits: e6ab88a [wangfei] ignore .idea_modul

git commit: [SPARK-3647] Add more exceptions to Guava relocation.

2014-09-23 Thread pwendell
Repository: spark Updated Branches: refs/heads/master a08153f8a -> 8dfe79ffb [SPARK-3647] Add more exceptions to Guava relocation. Guava's Optional refers to some package private classes / methods, and when those are relocated the code stops working, throwing exceptions. So add the affected cl

git commit: SPARK-3612. Executor shouldn't quit if heartbeat message fails to reach ...

2014-09-23 Thread pwendell
Repository: spark Updated Branches: refs/heads/branch-1.1 aab0a1dda -> ffd97be32 SPARK-3612. Executor shouldn't quit if heartbeat message fails to reach ... ...the driver Author: Sandy Ryza Closes #2487 from sryza/sandy-spark-3612 and squashes the following commits: 2b7353d [Sandy Ryza] SP

git commit: SPARK-3612. Executor shouldn't quit if heartbeat message fails to reach ...

2014-09-23 Thread pwendell
Repository: spark Updated Branches: refs/heads/master 8dfe79ffb -> d79238d03 SPARK-3612. Executor shouldn't quit if heartbeat message fails to reach ... ...the driver Author: Sandy Ryza Closes #2487 from sryza/sandy-spark-3612 and squashes the following commits: 2b7353d [Sandy Ryza] SPARK-

git commit: [Build] Diff from branch point

2014-09-24 Thread pwendell
Repository: spark Updated Branches: refs/heads/master 729952a5e -> c42912606 [Build] Diff from branch point Sometimes Jenkins posts [spurious reports of new classes being added](https://github.com/apache/spark/pull/2339#issuecomment-56570170). I believe this stems from diffing the patch agai

git commit: [SPARK-3659] Set EC2 version to 1.1.0 and update version map

2014-09-24 Thread pwendell
Repository: spark Updated Branches: refs/heads/master c42912606 -> 50f863365 [SPARK-3659] Set EC2 version to 1.1.0 and update version map This brings the master branch in sync with branch-1.1 Author: Shivaram Venkataraman Closes #2510 from shivaram/spark-ec2-version and squashes the followi

git commit: [SPARK-3615][Streaming]Fix Kafka unit test hard coded Zookeeper port issue

2014-09-24 Thread pwendell
Repository: spark Updated Branches: refs/heads/master bb96012b7 -> 74fb2ecf7 [SPARK-3615][Streaming]Fix Kafka unit test hard coded Zookeeper port issue Details can be seen in [SPARK-3615](https://issues.apache.org/jira/browse/SPARK-3615). Author: jerryshao Closes #2483 from jerryshao/SPARK

git commit: [SPARK-546] Add full outer join to RDD and DStream.

2014-09-24 Thread pwendell
Repository: spark Updated Branches: refs/heads/master 74fb2ecf7 -> 8ca4ecb6a [SPARK-546] Add full outer join to RDD and DStream. leftOuterJoin and rightOuterJoin are already implemented. This patch adds fullOuterJoin. Author: Aaron Staple Closes #1395 from staple/SPARK-546 and squashes th

git commit: [SPARK-2778] [yarn] Add yarn integration tests.

2014-09-24 Thread pwendell
Repository: spark Updated Branches: refs/heads/master 8ca4ecb6a -> b8487713d [SPARK-2778] [yarn] Add yarn integration tests. This patch adds a couple of, currently, very simple integration tests to make sure both client and cluster modes are working. The tests don't do much yet other than run

git commit: [SPARK-3584] sbin/slaves doesn't work when we use password authentication for SSH

2014-09-25 Thread pwendell
Repository: spark Updated Branches: refs/heads/master ff637c938 -> 0dc868e78 [SPARK-3584] sbin/slaves doesn't work when we use password authentication for SSH Author: Kousuke Saruta Closes #2444 from sarutak/slaves-scripts-modification and squashes the following commits: eff7394 [Kousuke

git commit: [SPARK-3686][STREAMING] Wait for sink to commit the channel before check...

2014-09-25 Thread pwendell
Repository: spark Updated Branches: refs/heads/master 86bce7649 -> b235e0136 [SPARK-3686][STREAMING] Wait for sink to commit the channel before check... ...ing for the channel size. Author: Hari Shreedharan Closes #2531 from harishreedharan/sparksinksuite-fix and squashes the following com

git commit: [SPARK-3695]shuffle fetch fail output

2014-09-26 Thread pwendell
Repository: spark Updated Branches: refs/heads/master ec9df6a76 -> 30461c6ac [SPARK-3695]shuffle fetch fail output should output detailed host and port in error message Author: Daoyuan Wang Closes #2539 from adrian-wang/fetchfail and squashes the following commits: 6c1b1e0 [Daoyuan Wang] s

git commit: Revert "[SPARK-3007][SQL]Add Dynamic Partition support to Spark Sql hive"

2014-09-30 Thread pwendell
Repository: spark Updated Branches: refs/heads/master b167a8c7e -> b64fcbd2d Revert "[SPARK-3007][SQL]Add Dynamic Partition support to Spark Sql hive" This reverts commit 0bbe7faeffa17577ae8a33dfcd8c4c783db5c909. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-w

git commit: HOTFIX: Ignore flaky tests in YARN

2014-09-30 Thread pwendell
Repository: spark Updated Branches: refs/heads/master b64fcbd2d -> 157e7d0f6 HOTFIX: Ignore flaky tests in YARN Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/157e7d0f Tree: http://git-wip-us.apache.org/repos/asf/spark/tr

git commit: [SPARK-3757] mvn clean doesn't delete some files

2014-10-01 Thread pwendell
Repository: spark Updated Branches: refs/heads/master 3888ee2f3 -> 0bfd3afb0 [SPARK-3757] mvn clean doesn't delete some files Added directory to be deleted into maven-clean-plugin in pom.xml. Author: Masayoshi TSUZUKI Closes #2613 from tsudukim/feature/SPARK-3757 and squashes the following

git commit: [SPARK-3756] [Core]check exception is caused by an address-port collision properly

2014-10-01 Thread pwendell
Repository: spark Updated Branches: refs/heads/master 6390aae4e -> 2fedb5ddd [SPARK-3756] [Core]check exception is caused by an address-port collision properly Jetty server use MultiException to handle exceptions when start server refer https://github.com/eclipse/jetty.project/blob/jetty-8.1

git commit: [SPARK-3756] [Core]check exception is caused by an address-port collision properly

2014-10-01 Thread pwendell
Repository: spark Updated Branches: refs/heads/branch-1.1 13f33cfff -> b4f690d36 [SPARK-3756] [Core]check exception is caused by an address-port collision properly Jetty server use MultiException to handle exceptions when start server refer https://github.com/eclipse/jetty.project/blob/jetty

git commit: Revert "[SPARK-3755][Core] Do not bind port 1 - 1024 to server in spark"

2014-10-01 Thread pwendell
Repository: spark Updated Branches: refs/heads/master a31f4ff22 -> 4e79970d3 Revert "[SPARK-3755][Core] Do not bind port 1 - 1024 to server in spark" This reverts commit 6390aae4eacbabfb1c53fb828b824f6a6518beff. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wi

git commit: MAINTENANCE: Automated closing of pull requests.

2014-10-01 Thread pwendell
Repository: spark Updated Branches: refs/heads/master 29c351320 -> f341e1c8a MAINTENANCE: Automated closing of pull requests. This commit exists to close the following pull requests on Github: Closes #1375 (close requested by 'pwendell') Closes #476 (close requested by 'men

git commit: SPARK-1767: Prefer HDFS-cached replicas when scheduling data-local tasks

2014-10-02 Thread pwendell
Repository: spark Updated Branches: refs/heads/master bbdf1de84 -> 6e27cb630 SPARK-1767: Prefer HDFS-cached replicas when scheduling data-local tasks This change reorders the replicas returned by HadoopRDD#getPreferredLocations so that replicas cached by HDFS are at the start of the list. Thi

git commit: [SPARK-3007][SQL] Adds dynamic partitioning support

2014-10-03 Thread pwendell
Repository: spark Updated Branches: refs/heads/master fbe8e9856 -> bec0d0eaa [SPARK-3007][SQL] Adds dynamic partitioning support PR #2226 was reverted because it broke Jenkins builds for unknown reason. This debugging PR aims to fix the Jenkins build. This PR also fixes two bugs: 1. Compres

svn commit: r7964 - /release/spark/spark-1.1.0/

2015-02-08 Thread pwendell
Author: pwendell Date: Mon Feb 9 05:50:16 2015 New Revision: 7964 Log: Removing Spark 1.1.0 release. Removed: release/spark/spark-1.1.0/ - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional

svn commit: r7965 - /dev/spark/spark-1.2.1-rc3/

2015-02-08 Thread pwendell
Author: pwendell Date: Mon Feb 9 06:29:04 2015 New Revision: 7965 Log: Adding Spark 1.2.1 RC3 Added: dev/spark/spark-1.2.1-rc3/ dev/spark/spark-1.2.1-rc3/spark-1.2.1-bin-cdh4.tgz (with props) dev/spark/spark-1.2.1-rc3/spark-1.2.1-bin-cdh4.tgz.asc (with props) dev/spark/spark

svn commit: r7966 - /dev/spark/spark-1.2.1-rc3/ /release/spark/spark-1.2.1/

2015-02-08 Thread pwendell
Author: pwendell Date: Mon Feb 9 06:34:02 2015 New Revision: 7966 Log: Spark release 1.2.1 Added: release/spark/spark-1.2.1/ - copied from r7965, dev/spark/spark-1.2.1-rc3/ Removed: dev/spark/spark-1.2.1-rc3

svn commit: r1658576 - in /spark: ./ js/ site/ site/docs/ site/docs/1.2.1/ site/docs/1.2.1/api/ site/docs/1.2.1/api/java/ site/docs/1.2.1/api/java/org/ site/docs/1.2.1/api/java/org/apache/ site/docs/1

2015-02-09 Thread pwendell
Author: pwendell Date: Mon Feb 9 21:16:41 2015 New Revision: 1658576 URL: http://svn.apache.org/r1658576 Log: Adding Spark 1.2.1 release [This commit notification would consist of 763 parts, which exceeds the limit of 50 ones, so it was shortened to the summary

svn commit: r1658579 - in /spark: releases/_posts/2015-02-09-spark-release-1-2-1.md site/releases/spark-release-1-2-1.html

2015-02-09 Thread pwendell
Author: pwendell Date: Mon Feb 9 21:27:38 2015 New Revision: 1658579 URL: http://svn.apache.org/r1658579 Log: Adding release source doc Added: spark/releases/_posts/2015-02-09-spark-release-1-2-1.md Modified: spark/site/releases/spark-release-1-2-1.html Added: spark/releases/_posts

svn commit: r1658778 - in /spark: _layouts/ site/ site/graphx/ site/mllib/ site/news/ site/releases/ site/screencasts/ site/sql/ site/streaming/

2015-02-10 Thread pwendell
Author: pwendell Date: Tue Feb 10 18:51:07 2015 New Revision: 1658778 URL: http://svn.apache.org/r1658778 Log: Fixing latest doc link to point to 1.2.1 Modified: spark/_layouts/global.html spark/site/community.html spark/site/documentation.html spark/site/downloads.html spark

spark git commit: SPARK-5613: Catch the ApplicationNotFoundException exception to avoid thread from getting killed on yarn restart.

2015-02-10 Thread pwendell
Repository: spark Updated Branches: refs/heads/branch-1.3 dbfce30fc -> c294216cd SPARK-5613: Catch the ApplicationNotFoundException exception to avoid thread from getting killed on yarn restart. [SPARK-5613] Added a catch block to catch the ApplicationNotFoundException. Without this catch b

spark git commit: [SPARK-5493] [core] Add option to impersonate user.

2015-02-10 Thread pwendell
Repository: spark Updated Branches: refs/heads/branch-1.3 445dbc721 -> 8e75b0ed0 [SPARK-5493] [core] Add option to impersonate user. Hadoop has a feature that allows users to impersonate other users when submitting applications or talking to HDFS, for example. These impersonated users are refe

spark git commit: [SPARK-5493] [core] Add option to impersonate user.

2015-02-10 Thread pwendell
Repository: spark Updated Branches: refs/heads/master e28b6bdbb -> ed167e70c [SPARK-5493] [core] Add option to impersonate user. Hadoop has a feature that allows users to impersonate other users when submitting applications or talking to HDFS, for example. These impersonated users are referred

spark git commit: Updating versions for Spark 1.3

2015-02-10 Thread pwendell
Repository: spark Updated Branches: refs/heads/branch-1.3 ba3aa8fcb -> 2f52489d2 Updating versions for Spark 1.3 Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/2f52489d Tree: http://git-wip-us.apache.org/repos/asf/spark/t

Git Push Summary

2015-02-10 Thread pwendell
Repository: spark Updated Tags: refs/tags/v1.3.0-snapshot1 [created] c2e400103 - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org

[2/2] spark git commit: Preparing Spark release v1.3.0-snapshot1

2015-02-10 Thread pwendell
Preparing Spark release v1.3.0-snapshot1 Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/c2e40010 Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/c2e40010 Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/c2e4001

[1/2] spark git commit: Preparing development version 1.3.1-SNAPSHOT

2015-02-10 Thread pwendell
Repository: spark Updated Branches: refs/heads/branch-1.3 2f52489d2 -> db80d0fe2 Preparing development version 1.3.1-SNAPSHOT Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/db80d0fe Tree: http://git-wip-us.apache.org/repo

spark git commit: HOTFIX: Java 6 compilation error in Spark SQL

2015-02-10 Thread pwendell
Repository: spark Updated Branches: refs/heads/master f86a89a2e -> 7e2f8821e HOTFIX: Java 6 compilation error in Spark SQL Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/7e2f8821 Tree: http://git-wip-us.apache.org/repos/a

[2/3] spark git commit: Revert "Preparing Spark release v1.3.0-snapshot1"

2015-02-10 Thread pwendell
Revert "Preparing Spark release v1.3.0-snapshot1" This reverts commit c2e4001030cfb881ff33d448fc0aeaf4f05dad0f. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/536dae90 Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/

Git Push Summary

2015-02-10 Thread pwendell
Repository: spark Updated Tags: refs/tags/v1.3.0-snapshot1 [deleted] c2e400103 - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org

[1/3] spark git commit: Revert "Preparing development version 1.3.1-SNAPSHOT"

2015-02-10 Thread pwendell
Repository: spark Updated Branches: refs/heads/branch-1.3 db80d0fe2 -> 15180bcf8 Revert "Preparing development version 1.3.1-SNAPSHOT" This reverts commit db80d0fe21daa3202ff217cbefb999ce77c5aa9e. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org

[3/3] spark git commit: HOTFIX: Java 6 compilation error in Spark SQL

2015-02-10 Thread pwendell
HOTFIX: Java 6 compilation error in Spark SQL Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/15180bcf Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/15180bcf Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/15

Git Push Summary

2015-02-10 Thread pwendell
Repository: spark Updated Tags: refs/tags/v1.3.0-snapshot1 [created] 53068f56f - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org

[1/2] spark git commit: Preparing development version 1.3.1-SNAPSHOT

2015-02-10 Thread pwendell
Repository: spark Updated Branches: refs/heads/branch-1.3 15180bcf8 -> ba12b793f Preparing development version 1.3.1-SNAPSHOT Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/ba12b793 Tree: http://git-wip-us.apache.org/repo

[2/2] spark git commit: Preparing Spark release v1.3.0-snapshot1

2015-02-10 Thread pwendell
Preparing Spark release v1.3.0-snapshot1 Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/53068f56 Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/53068f56 Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/53068f5

spark git commit: HOTFIX: Adding Junit to Hive tests for Maven build

2015-02-10 Thread pwendell
Repository: spark Updated Branches: refs/heads/master 7e2f8821e -> c2131c0cd HOTFIX: Adding Junit to Hive tests for Maven build Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/c2131c0c Tree: http://git-wip-us.apache.org/re

spark git commit: HOTFIX: Adding Junit to Hive tests for Maven build

2015-02-10 Thread pwendell
Repository: spark Updated Branches: refs/heads/branch-1.3 ba12b793f -> 0386fc4d6 HOTFIX: Adding Junit to Hive tests for Maven build Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/0386fc4d Tree: http://git-wip-us.apache.or

Git Push Summary

2015-02-10 Thread pwendell
Repository: spark Updated Tags: refs/tags/v1.3.0-snapshot1 [deleted] 53068f56f - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org

[1/2] spark git commit: Revert "Preparing development version 1.3.1-SNAPSHOT"

2015-02-10 Thread pwendell
Repository: spark Updated Branches: refs/heads/branch-1.3 0386fc4d6 -> 6a91d5993 Revert "Preparing development version 1.3.1-SNAPSHOT" This reverts commit ba12b793f1f4f432e71439e2a7ebacce74d9c472. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org

[2/2] spark git commit: Revert "Preparing Spark release v1.3.0-snapshot1"

2015-02-10 Thread pwendell
Revert "Preparing Spark release v1.3.0-snapshot1" This reverts commit 53068f56f40bf03b7fc52e5980fb7e205903fc8b. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/6a91d599 Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/

Git Push Summary

2015-02-10 Thread pwendell
Repository: spark Updated Tags: refs/tags/v1.3.0-snapshot1 [created] d97bfc6f2 - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org

[1/2] spark git commit: Preparing Spark release v1.3.0-snapshot1

2015-02-10 Thread pwendell
Repository: spark Updated Branches: refs/heads/branch-1.3 6a91d5993 -> e57c81b8c Preparing Spark release v1.3.0-snapshot1 Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/d97bfc6f Tree: http://git-wip-us.apache.org/repos/as

[2/2] spark git commit: Preparing development version 1.3.1-SNAPSHOT

2015-02-10 Thread pwendell
Preparing development version 1.3.1-SNAPSHOT Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/e57c81b8 Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/e57c81b8 Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/e57

spark git commit: [SPARK-5227] [SPARK-5679] Disable FileSystem cache in WholeTextFileRecordReaderSuite

2015-02-13 Thread pwendell
Repository: spark Updated Branches: refs/heads/branch-1.3 fccd38d2e -> 152147f5f [SPARK-5227] [SPARK-5679] Disable FileSystem cache in WholeTextFileRecordReaderSuite This patch fixes two difficult-to-reproduce Jenkins test failures in InputOutputMetricsSuite (SPARK-5227 and SPARK-5679). The

<    2   3   4   5   6   7   8   9   10   11   >