spark git commit: SPARK-4457. Document how to build for Hadoop versions greater than 2.4

2014-11-24 Thread tgraves
Repository: spark Updated Branches: refs/heads/master 9b2a3c612 - 29372b631 SPARK-4457. Document how to build for Hadoop versions greater than 2.4 Author: Sandy Ryza sa...@cloudera.com Closes #3322 from sryza/sandy-spark-4457 and squashes the following commits: 5e72b77 [Sandy Ryza] Feedback

spark git commit: [SPARK-4479][SQL] Avoids unnecessary defensive copies when sort based shuffle is on

2014-11-24 Thread marmbrus
Repository: spark Updated Branches: refs/heads/branch-1.2 1a12ca339 - ee1bc892a [SPARK-4479][SQL] Avoids unnecessary defensive copies when sort based shuffle is on This PR is a workaround for SPARK-4479. Two changes are introduced: when merge sort is bypassed in `ExternalSorter`, 1. also

spark git commit: [SQL] Fix comment in HiveShim

2014-11-24 Thread marmbrus
Repository: spark Updated Branches: refs/heads/master a6d7b61f9 - d5834f073 [SQL] Fix comment in HiveShim This file is for Hive 0.13.1 I think. Author: Daniel Darabos darabos.dan...@gmail.com Closes #3432 from darabos/patch-2 and squashes the following commits: 4fd22ed [Daniel Darabos] Fix

spark git commit: [SQL] Fix comment in HiveShim

2014-11-24 Thread marmbrus
Repository: spark Updated Branches: refs/heads/branch-1.2 ee1bc892a - 1e3d22b9f [SQL] Fix comment in HiveShim This file is for Hive 0.13.1 I think. Author: Daniel Darabos darabos.dan...@gmail.com Closes #3432 from darabos/patch-2 and squashes the following commits: 4fd22ed [Daniel Darabos]

spark git commit: [SQL] Fix path in HiveFromSpark

2014-11-24 Thread marmbrus
Repository: spark Updated Branches: refs/heads/master d5834f073 - b38411930 [SQL] Fix path in HiveFromSpark It require us to run ```HiveFromSpark``` in specified dir because ```HiveFromSpark``` use relative path, this leads to ```run-example```

spark git commit: [SQL] Fix path in HiveFromSpark

2014-11-24 Thread marmbrus
Repository: spark Updated Branches: refs/heads/branch-1.2 1e3d22b9f - 0e7fa7f63 [SQL] Fix path in HiveFromSpark It require us to run ```HiveFromSpark``` in specified dir because ```HiveFromSpark``` use relative path, this leads to ```run-example```

spark git commit: [SPARK-4487][SQL] Fix attribute reference resolution error when using ORDER BY.

2014-11-24 Thread marmbrus
Repository: spark Updated Branches: refs/heads/branch-1.2 0e7fa7f63 - 97b7eb4d9 [SPARK-4487][SQL] Fix attribute reference resolution error when using ORDER BY. When we use ORDER BY clause, at first, attributes referenced by projection are resolved (1). And then, attributes referenced at

spark git commit: [SPARK-4487][SQL] Fix attribute reference resolution error when using ORDER BY.

2014-11-24 Thread marmbrus
Repository: spark Updated Branches: refs/heads/master b38411930 - dd1c9cb36 [SPARK-4487][SQL] Fix attribute reference resolution error when using ORDER BY. When we use ORDER BY clause, at first, attributes referenced by projection are resolved (1). And then, attributes referenced at ORDER BY

[2/2] spark git commit: [SPARK-4145] Web UI job pages

2014-11-24 Thread pwendell
[SPARK-4145] Web UI job pages This PR adds two new pages to the Spark Web UI: - A jobs overview page, which shows details on running / completed / failed jobs. - A job details page, which displays information on an individual job's stages. The jobs overview page is now the default UI homepage;

[2/2] spark git commit: [SPARK-4145] Web UI job pages

2014-11-24 Thread pwendell
[SPARK-4145] Web UI job pages This PR adds two new pages to the Spark Web UI: - A jobs overview page, which shows details on running / completed / failed jobs. - A job details page, which displays information on an individual job's stages. The jobs overview page is now the default UI homepage;

spark git commit: [SPARK-4518][SPARK-4519][Streaming] Refactored file stream to prevent files from being processed multiple times

2014-11-24 Thread tdas
Repository: spark Updated Branches: refs/heads/branch-1.2 2d35cc085 - 6fa3e415d [SPARK-4518][SPARK-4519][Streaming] Refactored file stream to prevent files from being processed multiple times Because of a corner case, a file already selected for batch t can get considered again for batch

svn commit: r7249 - /dev/spark/spark-1.1.1-rc2/

2014-11-24 Thread andrewor14
Author: andrewor14 Date: Mon Nov 24 22:16:16 2014 New Revision: 7249 Log: Add spark-1.1.1-rc2 Added: dev/spark/spark-1.1.1-rc2/ dev/spark/spark-1.1.1-rc2/spark-1.1.1-bin-cdh4.tgz (with props) dev/spark/spark-1.1.1-rc2/spark-1.1.1-bin-cdh4.tgz.asc

svn commit: r7250 - /dev/spark/spark-1.1.1-rc2/ /release/spark/spark-1.1.1/

2014-11-24 Thread andrewor14
Author: andrewor14 Date: Mon Nov 24 22:25:56 2014 New Revision: 7250 Log: Rename spark-1.1.1-rc2 to spark-1.1.1 Added: release/spark/spark-1.1.1/ - copied from r7249, dev/spark/spark-1.1.1-rc2/ Removed: dev/spark/spark-1.1.1-rc2/

Git Push Summary

2014-11-24 Thread andrewor14
Repository: spark Updated Tags: refs/tags/v1.1.1-rc1 [deleted] defa2f1a5 - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org

spark git commit: Update versions to 1.1.2-SNAPSHOT

2014-11-24 Thread andrewor14
Repository: spark Updated Branches: refs/heads/branch-1.1 1df1c1d9b - 63717375d Update versions to 1.1.2-SNAPSHOT Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/63717375 Tree:

svn commit: r1641509 - in /spark/site/docs: ./ 1.1.1/ 1.1.1/api/ 1.1.1/api/java/ 1.1.1/api/java/org/ 1.1.1/api/java/org/apache/ 1.1.1/api/java/org/apache/spark/ 1.1.1/api/java/org/apache/spark/annotat

2014-11-24 Thread andrewor14
Author: andrewor14 Date: Mon Nov 24 23:38:52 2014 New Revision: 1641509 URL: http://svn.apache.org/r1641509 Log: Add docs for Spark 1.1.1 [This commit notification would consist of 484 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.]

svn commit: r7251 - /release/spark/spark-1.0.2/

2014-11-24 Thread andrewor14
Author: andrewor14 Date: Mon Nov 24 23:43:26 2014 New Revision: 7251 Log: Remove Spark 1.0.2 Removed: release/spark/spark-1.0.2/ - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands,

spark git commit: [SPARK-4562] [MLlib] speedup vector

2014-11-24 Thread meng
Repository: spark Updated Branches: refs/heads/master cb0e9b098 - b660de7a9 [SPARK-4562] [MLlib] speedup vector This PR change the underline array of DenseVector to numpy.ndarray to avoid the conversion, because most of the users will using numpy.array. It also improve the serialization of

spark git commit: get raw vectors for further processing in Word2Vec

2014-11-24 Thread meng
Repository: spark Updated Branches: refs/heads/branch-1.2 9ea67fc1d - 2acbd2884 get raw vectors for further processing in Word2Vec e.g. clustering Author: tkaessmann tobias.kaessm...@s24.com Closes #3309 from tkaessmann/branch-1.2 and squashes the following commits: e3a3142 [tkaessmann]

spark git commit: [SPARK-4578] fix asDict() with nested Row()

2014-11-24 Thread pwendell
Repository: spark Updated Branches: refs/heads/branch-1.2 2acbd2884 - 8371bc208 [SPARK-4578] fix asDict() with nested Row() The Row object is created on the fly once the field is accessed, so we should access them by getattr() in asDict(0 Author: Davies Liu dav...@databricks.com Closes

spark git commit: [SPARK-4578] fix asDict() with nested Row()

2014-11-24 Thread pwendell
Repository: spark Updated Branches: refs/heads/master b660de7a9 - 050616b40 [SPARK-4578] fix asDict() with nested Row() The Row object is created on the fly once the field is accessed, so we should access them by getattr() in asDict(0 Author: Davies Liu dav...@databricks.com Closes #3434

spark git commit: [SPARK-4548] []SPARK-4517] improve performance of python broadcast

2014-11-24 Thread joshrosen
Repository: spark Updated Branches: refs/heads/branch-1.2 8371bc208 - 841f247a5 [SPARK-4548] []SPARK-4517] improve performance of python broadcast Re-implement the Python broadcast using file: 1) serialize the python object using cPickle, write into disks. 2) Create a wrapper in JVM (for the

spark git commit: [SPARK-4548] []SPARK-4517] improve performance of python broadcast

2014-11-24 Thread joshrosen
Repository: spark Updated Branches: refs/heads/master 050616b40 - 6cf507685 [SPARK-4548] []SPARK-4517] improve performance of python broadcast Re-implement the Python broadcast using file: 1) serialize the python object using cPickle, write into disks. 2) Create a wrapper in JVM (for the

spark git commit: [SPARK-4266] [Web-UI] Reduce stage page load time.

2014-11-24 Thread kayousterhout
Repository: spark Updated Branches: refs/heads/master 6cf507685 - d24d5bf06 [SPARK-4266] [Web-UI] Reduce stage page load time. The commit changes the java script used to show/hide additional metrics in order to reduce page load time. SPARK-4016 significantly increased page load time for the

spark git commit: [SPARK-4266] [Web-UI] Reduce stage page load time.

2014-11-24 Thread kayousterhout
Repository: spark Updated Branches: refs/heads/branch-1.2 841f247a5 - 47d4fceff [SPARK-4266] [Web-UI] Reduce stage page load time. The commit changes the java script used to show/hide additional metrics in order to reduce page load time. SPARK-4016 significantly increased page load time for

spark git commit: [SPARK-4525] Mesos should decline unused offers

2014-11-24 Thread pwendell
Repository: spark Updated Branches: refs/heads/master d24d5bf06 - b043c2742 [SPARK-4525] Mesos should decline unused offers Functionally, this is just a small change on top of #3393 (by jongyoul). The issue being addressed is discussed in the comments there. I have not yet added a test for

[2/2] spark git commit: [SPARK-4525] Mesos should decline unused offers

2014-11-24 Thread pwendell
[SPARK-4525] Mesos should decline unused offers Functionally, this is just a small change on top of #3393 (by jongyoul). The issue being addressed is discussed in the comments there. I have not yet added a test for the bug there. I will add one shortly. I've also done some minor

[1/2] spark git commit: Revert [SPARK-4525] Mesos should decline unused offers

2014-11-24 Thread pwendell
Repository: spark Updated Branches: refs/heads/master b043c2742 - f0afb623d Revert [SPARK-4525] Mesos should decline unused offers This reverts commit b043c27424d05e3200e7ba99a1a65656b57fa2f0. I accidentally committed this using my own authorship credential. However, I should have given

[2/2] spark git commit: [SPARK-4525] Mesos should decline unused offers

2014-11-24 Thread pwendell
[SPARK-4525] Mesos should decline unused offers Functionally, this is just a small change on top of #3393 (by jongyoul). The issue being addressed is discussed in the comments there. I have not yet added a test for the bug there. I will add one shortly. I've also done some minor

[1/2] spark git commit: Revert [SPARK-4525] Mesos should decline unused offers

2014-11-24 Thread pwendell
Repository: spark Updated Branches: refs/heads/branch-1.2 4b4797309 - 10e433919 Revert [SPARK-4525] Mesos should decline unused offers This reverts commit 4b4797309457b9301710b6e98550817337005eca. I accidentally committed this using my own authorship credential. However, I should have given

spark git commit: [SQL] Compute timeTaken correctly

2014-11-24 Thread rxin
Repository: spark Updated Branches: refs/heads/branch-1.2 10e433919 - 259cb26fc [SQL] Compute timeTaken correctly ```timeTaken``` should not count the time of printing result. Author: w00228970 wangf...@huawei.com Closes #3423 from scwf/time-taken-bug and squashes the following commits:

spark git commit: [DOC][Build] Wrong cmd for build spark with apache hadoop 2.4.X and hive 12

2014-11-24 Thread pwendell
Repository: spark Updated Branches: refs/heads/master 723be60e2 - 0fe54cff1 [DOC][Build] Wrong cmd for build spark with apache hadoop 2.4.X and hive 12 Author: wangfei wangf...@huawei.com Closes #3335 from scwf/patch-10 and squashes the following commits: d343113 [wangfei] add '-Phive'