svn commit: r1712484 - in /spark: _layouts/ images/ site/ site/graphx/ site/images/ site/mllib/ site/news/ site/releases/ site/screencasts/ site/sql/ site/streaming/

2015-11-03 Thread pwendell
Author: pwendell Date: Wed Nov 4 07:17:45 2015 New Revision: 1712484 URL: http://svn.apache.org/viewvc?rev=1712484&view=rev Log: Adding trademark to logo Added: spark/images/spark-logo-trademark.png (with props) spark/images/spark-logo-trademark.xcf (with props) spark/site/images

spark git commit: [SPARK-11455][SQL] fix case sensitivity of partition by

2015-11-03 Thread yhuai
Repository: spark Updated Branches: refs/heads/master e352de0db -> 2692bdb7d [SPARK-11455][SQL] fix case sensitivity of partition by depend on `caseSensitive` to do column name equality check, instead of just `==` Author: Wenchen Fan Closes #9410 from cloud-fan/partition. Project: http://

spark git commit: [SPARK-11329] [SQL] Cleanup from spark-11329 fix.

2015-11-03 Thread yhuai
Repository: spark Updated Branches: refs/heads/master d648a4ad5 -> e352de0db [SPARK-11329] [SQL] Cleanup from spark-11329 fix. Author: Nong Closes #9442 from nongli/spark-11483. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/

spark git commit: [DOC] Missing link to R DataFrame API doc

2015-11-03 Thread shivaram
Repository: spark Updated Branches: refs/heads/master 5051262d4 -> d648a4ad5 [DOC] Missing link to R DataFrame API doc Author: lewuathe Author: Lewuathe Closes #9394 from Lewuathe/missing-link-to-R-dataframe. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wi

spark git commit: [SPARK-11489][SQL] Only include common first order statistics in GroupedData

2015-11-03 Thread rxin
Repository: spark Updated Branches: refs/heads/master 53e9cee3e -> 5051262d4 [SPARK-11489][SQL] Only include common first order statistics in GroupedData We added a bunch of higher order statistics such as skewness and kurtosis to GroupedData. I don't think they are common enough to justify b

spark git commit: [SPARK-11466][CORE] Avoid mockito in multi-threaded FsHistoryProviderSuite test.

2015-11-03 Thread vanzin
Repository: spark Updated Branches: refs/heads/master 680b4e7bc -> 53e9cee3e [SPARK-11466][CORE] Avoid mockito in multi-threaded FsHistoryProviderSuite test. The test functionality should be the same, but without using mockito; logs don't really say anything useful but I suspect it may be the

spark git commit: Fix typo in WebUI

2015-11-03 Thread rxin
Repository: spark Updated Branches: refs/heads/branch-1.5 072afc6f4 -> c022c0aa4 Fix typo in WebUI Author: Jacek Laskowski Closes #9444 from jaceklaskowski/TImely-fix. (cherry picked from commit 680b4e7bca935dc1569f35fa319bdfb01a12f7e0) Signed-off-by: Reynold Xin Project: http://git-wip-

spark git commit: Fix typo in WebUI

2015-11-03 Thread rxin
Repository: spark Updated Branches: refs/heads/master f6fcb4874 -> 680b4e7bc Fix typo in WebUI Author: Jacek Laskowski Closes #9444 from jaceklaskowski/TImely-fix. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/680b4e7

spark git commit: [SPARK-11477] [SQL] support create Dataset from RDD

2015-11-03 Thread marmbrus
Repository: spark Updated Branches: refs/heads/master 1d04dc95c -> f6fcb4874 [SPARK-11477] [SQL] support create Dataset from RDD Author: Wenchen Fan Closes #9434 from cloud-fan/rdd2ds and squashes the following commits: 0892d72 [Wenchen Fan] support create Dataset from RDD Project: http:/

spark git commit: [SPARK-11424] Guard against double-close() of RecordReaders (branch-1.3 backport)

2015-11-03 Thread joshrosen
Repository: spark Updated Branches: refs/heads/branch-1.3 0ce148533 -> b90e5cba2 [SPARK-11424] Guard against double-close() of RecordReaders (branch-1.3 backport) This is a branch-1.3 backport of #9382, a fix for SPARK-11424. Author: Josh Rosen Closes #9423 from JoshRosen/hadoop-decompress

spark git commit: [SPARK-11467][SQL] add Python API for stddev/variance

2015-11-03 Thread rxin
Repository: spark Updated Branches: refs/heads/master a9676cc71 -> 1d04dc95c [SPARK-11467][SQL] add Python API for stddev/variance Add Python API for stddev/stddev_pop/stddev_samp/variance/var_pop/var_samp/skewness/kurtosis Author: Davies Liu Closes #9424 from davies/py_var. Project: htt

spark git commit: [SPARK-11407][SPARKR] Add doc for running from RStudio

2015-11-03 Thread shivaram
Repository: spark Updated Branches: refs/heads/master ebf8b0b48 -> a9676cc71 [SPARK-11407][SPARKR] Add doc for running from RStudio ![image](https://cloud.githubusercontent.com/assets/8969467/10871746/612ba44a-80a4-11e5-99a0-40b9931dee52.png) (This is without css, but you get the idea) shivara

spark git commit: [SPARK-10978][SQL] Allow data sources to eliminate filters

2015-11-03 Thread yhuai
Repository: spark Updated Branches: refs/heads/master b2e4b314d -> ebf8b0b48 [SPARK-10978][SQL] Allow data sources to eliminate filters This PR adds a new method `unhandledFilters` to `BaseRelation`. Data sources which implement this method properly may avoid the overhead of defensive filter

[1/2] spark git commit: Preparing Spark release v1.5.2-rc2

2015-11-03 Thread pwendell
Repository: spark Updated Branches: refs/heads/branch-1.5 979566690 -> 072afc6f4 Preparing Spark release v1.5.2-rc2 Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/49c30c1f Tree: http://git-wip-us.apache.org/repos/asf/spar

[2/2] spark git commit: Preparing development version 1.5.3-SNAPSHOT

2015-11-03 Thread pwendell
Preparing development version 1.5.3-SNAPSHOT Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/072afc6f Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/072afc6f Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/072

Git Push Summary

2015-11-03 Thread pwendell
Repository: spark Updated Tags: refs/tags/v1.5.2-rc2 [created] 49c30c1f6 - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org

spark git commit: [SPARK-9790][YARN] Expose in WebUI if NodeManager is the reason why executors were killed.

2015-11-03 Thread vanzin
Repository: spark Updated Branches: refs/heads/master f54ff19b1 -> b2e4b314d [SPARK-9790][YARN] Expose in WebUI if NodeManager is the reason why executors were killed. Author: Mark Grover Closes #8093 from markgrover/nm2. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit:

spark git commit: Update branch-1.5 for 1.5.2 release.

2015-11-03 Thread rxin
Repository: spark Updated Branches: refs/heads/branch-1.5 5604ce9c1 -> 979566690 Update branch-1.5 for 1.5.2 release. Author: Reynold Xin Closes #9435 from rxin/patch1.5.2. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commi

spark git commit: [SPARK-11349][ML] Support transform string label for RFormula

2015-11-03 Thread meng
Repository: spark Updated Branches: refs/heads/master 3434572b1 -> f54ff19b1 [SPARK-11349][ML] Support transform string label for RFormula Currently ```RFormula``` can only handle label with ```NumericType``` or ```BinaryType``` (cast it to ```DoubleType``` as the label of Linear Regression

spark git commit: [MINOR][ML] Fix naming conventions of AFTSurvivalRegression coefficients

2015-11-03 Thread meng
Repository: spark Updated Branches: refs/heads/master d6f10aa7e -> 3434572b1 [MINOR][ML] Fix naming conventions of AFTSurvivalRegression coefficients Rename ```regressionCoefficients``` back to ```coefficients```, and name ```weights``` to ```parameters```. See discussion [here](https://gith

spark git commit: [SPARK-9836][ML] Provide R-like summary statistics for OLS via normal equation solver

2015-11-03 Thread meng
Repository: spark Updated Branches: refs/heads/master d6035d97c -> d6f10aa7e [SPARK-9836][ML] Provide R-like summary statistics for OLS via normal equation solver https://issues.apache.org/jira/browse/SPARK-9836 Author: Yanbo Liang Closes #9413 from yanboliang/spark-9836. Project: http:/

spark git commit: [SPARK-10304] [SQL] Partition discovery should throw an exception if the dir structure is invalid

2015-11-03 Thread davies
Repository: spark Updated Branches: refs/heads/master 57446eb69 -> d6035d97c [SPARK-10304] [SQL] Partition discovery should throw an exception if the dir structure is invalid JIRA: https://issues.apache.org/jira/browse/SPARK-10304 This patch detects if the structure of partition directories

spark git commit: [SPARK-11256] Mark all Stage/ResultStage/ShuffleMapStage internal state as private.

2015-11-03 Thread rxin
Repository: spark Updated Branches: refs/heads/master d188a6776 -> 57446eb69 [SPARK-11256] Mark all Stage/ResultStage/ShuffleMapStage internal state as private. Author: Reynold Xin Closes #9219 from rxin/stage-cleanup1. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: ht

spark git commit: [SPARK-10533][SQL] handle scientific notation in sqlParser

2015-11-03 Thread lian
Repository: spark Updated Branches: refs/heads/master 233e534ac -> d188a6776 [SPARK-10533][SQL] handle scientific notation in sqlParser https://issues.apache.org/jira/browse/SPARK-10533 val df = sqlContext.createDataFrame(Seq(("a",1.0),("b",2.0),("c",3.0))) df.filter("_2 < 2.0e1").show Scien

spark git commit: [SPARK-11344] Made ApplicationDescription and DriverDescription case classes

2015-11-03 Thread srowen
Repository: spark Updated Branches: refs/heads/master b86f2cab6 -> 233e534ac [SPARK-11344] Made ApplicationDescription and DriverDescription case classes DriverDescription refactored to case class because it included no mutable fields. ApplicationDescription had one mutable field, which was

spark git commit: [SPARK-11404] [SQL] Support for groupBy using column expressions

2015-11-03 Thread marmbrus
Repository: spark Updated Branches: refs/heads/master 425ff03f5 -> b86f2cab6 [SPARK-11404] [SQL] Support for groupBy using column expressions This PR adds a new method `groupBy(cols: Column*)` to `Dataset` that allows users to group using column expressions instead of a lambda function. Sinc

spark git commit: [SPARK-11436] [SQL] rebind right encoder when join 2 datasets

2015-11-03 Thread marmbrus
Repository: spark Updated Branches: refs/heads/master 67e23b39a -> 425ff03f5 [SPARK-11436] [SQL] rebind right encoder when join 2 datasets When we join 2 datasets, we will combine 2 encoders into a tupled one, and use it as the encoder for the jioned dataset. Assume both of the 2 encoders are

spark git commit: [SPARK-11188] [SQL] Elide stacktraces in bin/spark-sql for AnalysisExceptions

2015-11-03 Thread marmbrus
Repository: spark Updated Branches: refs/heads/branch-1.5 b85bf8f49 -> 5604ce9c1 [SPARK-11188] [SQL] Elide stacktraces in bin/spark-sql for AnalysisExceptions Only print the error message to the console for Analysis Exceptions in sql-shell Author: Dilip Biswal Closes #9374 from dilipbiswal/

spark git commit: [SPARK-10429] [SQL] make mutableProjection atomic

2015-11-03 Thread marmbrus
Repository: spark Updated Branches: refs/heads/master d728d5c98 -> 67e23b39a [SPARK-10429] [SQL] make mutableProjection atomic Right now, SQL's mutable projection updates every value of the mutable project after it evaluates the corresponding expression. This makes the behavior of MutablePro

spark git commit: [SPARK-9858][SPARK-9859][SPARK-9861][SQL] Add an ExchangeCoordinator to estimate the number of post-shuffle partitions for aggregates and joins

2015-11-03 Thread yhuai
Repository: spark Updated Branches: refs/heads/master c34c27fe9 -> d728d5c98 [SPARK-9858][SPARK-9859][SPARK-9861][SQL] Add an ExchangeCoordinator to estimate the number of post-shuffle partitions for aggregates and joins https://issues.apache.org/jira/browse/SPARK-9858 https://issues.apache.o