spark git commit: [SPARK-1406] Mllib pmml model export

2015-04-30 Thread meng
Repository: spark Updated Branches: refs/heads/master 445951449 - 254e05097 [SPARK-1406] Mllib pmml model export See PDF attached to the JIRA issue 1406. The contribution is my original work and I license the work to the project under the project's open source license. Author: Vincenzo

spark git commit: [HOTFIX] Disabling flaky test (fix in progress as part of SPARK-7224)

2015-04-30 Thread pwendell
Repository: spark Updated Branches: refs/heads/branch-1.3 ae461e73f - 3bce87ebd [HOTFIX] Disabling flaky test (fix in progress as part of SPARK-7224) Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/3bce87eb Tree:

spark git commit: [SPARK-7224] added mock repository generator for --packages tests

2015-04-30 Thread pwendell
Repository: spark Updated Branches: refs/heads/master 47bf406d6 - 7dacc08ab [SPARK-7224] added mock repository generator for --packages tests This patch contains an `IvyTestUtils` file, which dynamically generates jars and pom files to test the `--packages` feature without having to rely on

spark git commit: [SPARK-5342] [YARN] Allow long running Spark apps to run on secure YARN/HDFS

2015-04-30 Thread tgraves
Repository: spark Updated Branches: refs/heads/master 7dacc08ab - 6c65da6bb [SPARK-5342] [YARN] Allow long running Spark apps to run on secure YARN/HDFS Current Spark apps running on Secure YARN/HDFS would not be able to write data to HDFS after 7 days, since delegation tokens cannot be

spark git commit: HOTFIX: Disable buggy dependency checker

2015-04-30 Thread pwendell
Repository: spark Updated Branches: refs/heads/master 36a7a6807 - a9fc50552 HOTFIX: Disable buggy dependency checker Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/a9fc5055 Tree:

spark git commit: [SPARK-7282] [STREAMING] Fix the race conditions in StreamingListenerSuite

2015-04-30 Thread tdas
Repository: spark Updated Branches: refs/heads/master beeafcfd6 - 69a739c7f [SPARK-7282] [STREAMING] Fix the race conditions in StreamingListenerSuite Fixed the following flaky test ```Scala [info] StreamingListenerSuite: [info] - batch info reporting (782 milliseconds) [info] - receiver info

spark git commit: [SPARK-7248] implemented random number generators for DataFrames

2015-04-30 Thread rxin
Repository: spark Updated Branches: refs/heads/master 69a739c7f - b5347a466 [SPARK-7248] implemented random number generators for DataFrames Adds the functions `rand` (Uniform Dist) and `randn` (Normal Dist.) as expressions to DataFrames. cc mengxr rxin Author: Burak Yavuz

spark git commit: [SPARK-6479] [BLOCK MANAGER] Create off-heap block storage API

2015-04-30 Thread pwendell
Repository: spark Updated Branches: refs/heads/master b5347a466 - 36a7a6807 [SPARK-6479] [BLOCK MANAGER] Create off-heap block storage API This is the classes for creating off-heap block storage API. It also includes the migration for Tachyon. The diff seems to be big, but it mainly just

spark git commit: [SPARK-6913][SQL] Fixed java.sql.SQLException: No suitable driver found

2015-04-30 Thread marmbrus
Repository: spark Updated Branches: refs/heads/master a0d8a61ab - e991255e7 [SPARK-6913][SQL] Fixed java.sql.SQLException: No suitable driver found Fixed `java.sql.SQLException: No suitable driver found` when loading DataFrame into Spark SQL if the driver is supplied with `--jars` argument.

spark git commit: [SPARK-7093] [SQL] Using newPredicate in NestedLoopJoin to enable code generation

2015-04-30 Thread marmbrus
Repository: spark Updated Branches: refs/heads/master ee0441393 - 079733817 [SPARK-7093] [SQL] Using newPredicate in NestedLoopJoin to enable code generation Using newPredicate in NestedLoopJoin instead of InterpretedPredicate to make it can make use of code generation Author: scwf

spark git commit: [SPARK-5213] [SQL] Pluggable SQL Parser Support

2015-04-30 Thread marmbrus
Repository: spark Updated Branches: refs/heads/master e991255e7 - 3ba5aaab8 [SPARK-5213] [SQL] Pluggable SQL Parser Support This PR aims to make the SQL Parser Pluggable, and user can register it's own parser via Spark SQL CLI. ``` # add the jar into the classpath

spark git commit: Revert [SPARK-5213] [SQL] Pluggable SQL Parser Support

2015-04-30 Thread pwendell
Repository: spark Updated Branches: refs/heads/master 473552fa5 - beeafcfd6 Revert [SPARK-5213] [SQL] Pluggable SQL Parser Support This reverts commit 3ba5aaab8266822545ac82b9e733fd25cc215a77. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit:

spark git commit: [SPARK-7280][SQL] Add drop column/s on a data frame

2015-04-30 Thread rxin
Repository: spark Updated Branches: refs/heads/master 149b3ee2d - ee0441393 [SPARK-7280][SQL] Add drop column/s on a data frame Takes a column name/s and returns a new DataFrame that drops a column/s. Author: rakeshchalasani vnit.rak...@gmail.com Closes #5818 from rakeshchalasani/SPARK-7280

spark git commit: [SPARK-7109] [SQL] Push down left side filter for left semi join

2015-04-30 Thread marmbrus
Repository: spark Updated Branches: refs/heads/master 079733817 - a0d8a61ab [SPARK-7109] [SQL] Push down left side filter for left semi join Now in spark sql optimizer we only push down right side filter for left semi join, actually we can push down left side filter because left semi join is

spark git commit: [SPARK-7123] [SQL] support table.star in sqlcontext

2015-04-30 Thread marmbrus
Repository: spark Updated Branches: refs/heads/master 3ba5aaab8 - 473552fa5 [SPARK-7123] [SQL] support table.star in sqlcontext Run following sql get error `SELECT r.* FROM testData l join testData2 r on (l.key = r.a)` Author: scwf wangf...@huawei.com Closes #5690 from scwf/tablestar and

spark git commit: Revert [SPARK-5342] [YARN] Allow long running Spark apps to run on secure YARN/HDFS

2015-04-30 Thread pwendell
Repository: spark Updated Branches: refs/heads/master adbdb19a7 - e0628f2fa Revert [SPARK-5342] [YARN] Allow long running Spark apps to run on secure YARN/HDFS This reverts commit 6c65da6bb7d1213e6a4a9f7fd1597d029d87d07c. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit:

spark git commit: [SPARK-7196][SQL] Support precision and scale of decimal type for JDBC

2015-04-30 Thread rxin
Repository: spark Updated Branches: refs/heads/branch-1.3 ec196ab1c - f64b99421 [SPARK-7196][SQL] Support precision and scale of decimal type for JDBC JIRA: https://issues.apache.org/jira/browse/SPARK-7196 Author: Liang-Chi Hsieh vii...@gmail.com Closes #5777 from viirya/jdbc_precision and

spark git commit: [SPARK-7196][SQL] Support precision and scale of decimal type for JDBC

2015-04-30 Thread rxin
Repository: spark Updated Branches: refs/heads/master e0628f2fa - 6702324b6 [SPARK-7196][SQL] Support precision and scale of decimal type for JDBC JIRA: https://issues.apache.org/jira/browse/SPARK-7196 Author: Liang-Chi Hsieh vii...@gmail.com Closes #5777 from viirya/jdbc_precision and

spark git commit: [SPARK-7267][SQL]Push down Project when it's child is Limit

2015-04-30 Thread rxin
Repository: spark Updated Branches: refs/heads/master 07a86205f - 77cc25fb7 [SPARK-7267][SQL]Push down Project when it's child is Limit SQL ``` select key from (select key,value from t1 limit 100) t2 limit 10 ``` Optimized Logical Plan before modifying ``` == Optimized Logical Plan == Limit

spark git commit: [Build] Enable MiMa checks for SQL

2015-04-30 Thread rxin
Repository: spark Updated Branches: refs/heads/master 77cc25fb7 - fa01bec48 [Build] Enable MiMa checks for SQL Now that 1.3 has been released, we should enable MiMa checks for the `sql` subproject. Author: Josh Rosen joshro...@databricks.com Closes #5727 from

spark git commit: [SPARK-7242][SQL][MLLIB] Frequent items for DataFrames

2015-04-30 Thread rxin
Repository: spark Updated Branches: refs/heads/master 1c3e402e6 - 149b3ee2d [SPARK-7242][SQL][MLLIB] Frequent items for DataFrames Finding frequent items with possibly false positives, using the algorithm described in `http://www.cs.umd.edu/~samir/498/karp.pdf`. public API under: ```

spark git commit: [SPARK-7207] [ML] [BUILD] Added ml.recommendation, ml.regression to SparkBuild

2015-04-30 Thread meng
Repository: spark Updated Branches: refs/heads/master 6c65da6bb - adbdb19a7 [SPARK-7207] [ML] [BUILD] Added ml.recommendation, ml.regression to SparkBuild Added ml.recommendation, ml.regression to SparkBuild CC: mengxr Author: Joseph K. Bradley jos...@databricks.com Closes #5758 from

spark git commit: [SPARK-7288] Suppress compiler warnings due to use of sun.misc.Unsafe; add facade in front of Unsafe; remove use of Unsafe.setMemory

2015-04-30 Thread rxin
Repository: spark Updated Branches: refs/heads/master 6702324b6 - 07a86205f [SPARK-7288] Suppress compiler warnings due to use of sun.misc.Unsafe; add facade in front of Unsafe; remove use of Unsafe.setMemory This patch suppresses compiler warnings due to our use of `sun.misc.Unsafe`

spark git commit: [SPARK-7279] Removed diffSum which is theoretical zero in LinearRegression and coding formating

2015-04-30 Thread meng
Repository: spark Updated Branches: refs/heads/master fa01bec48 - 1c3e402e6 [SPARK-7279] Removed diffSum which is theoretical zero in LinearRegression and coding formating Author: DB Tsai d...@netflix.com Closes #5809 from dbtsai/format and squashes the following commits: 6904eed [DB Tsai]

spark git commit: [SPARK-5529] [CORE] Add expireDeadHosts in HeartbeatReceiver

2015-04-30 Thread srowen
Repository: spark Updated Branches: refs/heads/branch-1.3 3bce87ebd - ec196ab1c [SPARK-5529] [CORE] Add expireDeadHosts in HeartbeatReceiver If a blockManager has not send heartBeat more than 120s, BlockManagerMasterActor will remove it. But coarseGrainedSchedulerBackend can only remove