spark git commit: [SPARK-17113] [SHUFFLE] Job failure due to Executor OOM in offheap mode

2016-08-19 Thread davies
Repository: spark Updated Branches: refs/heads/master 071eaaf9d -> cf0cce903 [SPARK-17113] [SHUFFLE] Job failure due to Executor OOM in offheap mode ## What changes were proposed in this pull request? This PR fixes executor OOM in offheap mode due to bug in Cooperative Memory Management for

spark git commit: [SPARK-16686][SQL] Remove PushProjectThroughSample since it is handled by ColumnPruning

2016-08-19 Thread rxin
Repository: spark Updated Branches: refs/heads/branch-2.0 d0707c6ba -> 3276ccfac [SPARK-16686][SQL] Remove PushProjectThroughSample since it is handled by ColumnPruning We push down `Project` through `Sample` in `Optimizer` by the rule `PushProjectThroughSample`. However, if the projected

spark git commit: [SPARK-17113] [SHUFFLE] Job failure due to Executor OOM in offheap mode

2016-08-19 Thread davies
Repository: spark Updated Branches: refs/heads/branch-2.0 3276ccfac -> ae89c8e17 [SPARK-17113] [SHUFFLE] Job failure due to Executor OOM in offheap mode ## What changes were proposed in this pull request? This PR fixes executor OOM in offheap mode due to bug in Cooperative Memory Management

spark git commit: [SPARK-16961][CORE] Fixed off-by-one error that biased randomizeInPlace

2016-08-19 Thread srowen
Repository: spark Updated Branches: refs/heads/master 287bea130 -> 5377fc623 [SPARK-16961][CORE] Fixed off-by-one error that biased randomizeInPlace JIRA issue link: https://issues.apache.org/jira/browse/SPARK-16961 Changed one line of Utils.randomizeInPlace to allow elements to stay in

spark git commit: [SPARK-16961][CORE] Fixed off-by-one error that biased randomizeInPlace

2016-08-19 Thread srowen
Repository: spark Updated Branches: refs/heads/branch-2.0 05b180faa -> d55d1f454 [SPARK-16961][CORE] Fixed off-by-one error that biased randomizeInPlace JIRA issue link: https://issues.apache.org/jira/browse/SPARK-16961 Changed one line of Utils.randomizeInPlace to allow elements to stay in

spark git commit: [SPARK-16994][SQL] Whitelist operators for predicate pushdown

2016-08-19 Thread wenchen
Repository: spark Updated Branches: refs/heads/master 072acf5e1 -> 67e59d464 [SPARK-16994][SQL] Whitelist operators for predicate pushdown ## What changes were proposed in this pull request? This patch changes predicate pushdown optimization rule (PushDownPredicate) from using a blacklist to

spark git commit: [SPARK-16994][SQL] Whitelist operators for predicate pushdown

2016-08-19 Thread wenchen
Repository: spark Updated Branches: refs/heads/branch-2.0 d55d1f454 -> e0c60f185 [SPARK-16994][SQL] Whitelist operators for predicate pushdown ## What changes were proposed in this pull request? This patch changes predicate pushdown optimization rule (PushDownPredicate) from using a

spark git commit: [SPARK-16965][MLLIB][PYSPARK] Fix bound checking for SparseVector.

2016-08-19 Thread srowen
Repository: spark Updated Branches: refs/heads/master 864be9359 -> 072acf5e1 [SPARK-16965][MLLIB][PYSPARK] Fix bound checking for SparseVector. ## What changes were proposed in this pull request? 1. In scala, add negative low bound checking and put all the low/upper bound checking in one

spark git commit: [SPARK-17141][ML] MinMaxScaler should remain NaN value.

2016-08-19 Thread yliang
Repository: spark Updated Branches: refs/heads/master 5377fc623 -> 864be9359 [SPARK-17141][ML] MinMaxScaler should remain NaN value. ## What changes were proposed in this pull request? In the existing code, ```MinMaxScaler``` handle ```NaN``` value indeterminately. * If a column has identity

spark git commit: [SPARK-16673][WEB UI] New Executor Page removed conditional for Logs and Thread Dump columns

2016-08-19 Thread tgraves
Repository: spark Updated Branches: refs/heads/master 67e59d464 -> e98eb2146 [SPARK-16673][WEB UI] New Executor Page removed conditional for Logs and Thread Dump columns ## What changes were proposed in this pull request? When #13670 switched `ExecutorsPage` to use JQuery DataTables it

spark git commit: [SPARK-11227][CORE] UnknownHostException can be thrown when NameNode HA is enabled.

2016-08-19 Thread tgraves
Repository: spark Updated Branches: refs/heads/master e98eb2146 -> 071eaaf9d [SPARK-11227][CORE] UnknownHostException can be thrown when NameNode HA is enabled. ## What changes were proposed in this pull request? If the following conditions are satisfied, executors don't load properties in

spark git commit: [SPARK-11227][CORE] UnknownHostException can be thrown when NameNode HA is enabled.

2016-08-19 Thread tgraves
Repository: spark Updated Branches: refs/heads/branch-2.0 e0c60f185 -> d0707c6ba [SPARK-11227][CORE] UnknownHostException can be thrown when NameNode HA is enabled. ## What changes were proposed in this pull request? If the following conditions are satisfied, executors don't load properties

spark git commit: [SPARK-16443][SPARKR] Alternating Least Squares (ALS) wrapper

2016-08-19 Thread meng
Repository: spark Updated Branches: refs/heads/master cf0cce903 -> acac7a508 [SPARK-16443][SPARKR] Alternating Least Squares (ALS) wrapper ## What changes were proposed in this pull request? Add Alternating Least Squares wrapper in SparkR. Unit tests have been updated. ## How was this patch

spark git commit: [SPARK-17158][SQL] Change error message for out of range numeric literals

2016-08-19 Thread rxin
Repository: spark Updated Branches: refs/heads/branch-2.0 efe832200 -> 379b12729 [SPARK-17158][SQL] Change error message for out of range numeric literals ## What changes were proposed in this pull request? Modifies error message for numeric literals to Numeric literal does not fit in range

spark git commit: [SPARK-17158][SQL] Change error message for out of range numeric literals

2016-08-19 Thread rxin
Repository: spark Updated Branches: refs/heads/master a117afa7c -> ba1737c21 [SPARK-17158][SQL] Change error message for out of range numeric literals ## What changes were proposed in this pull request? Modifies error message for numeric literals to Numeric literal does not fit in range

spark git commit: [SPARK-17149][SQL] array.sql for testing array related functions

2016-08-19 Thread rxin
Repository: spark Updated Branches: refs/heads/master acac7a508 -> a117afa7c [SPARK-17149][SQL] array.sql for testing array related functions ## What changes were proposed in this pull request? This patch creates array.sql in SQLQueryTestSuite for testing array related functions, including:

spark git commit: [SPARK-17150][SQL] Support SQL generation for inline tables

2016-08-19 Thread wenchen
Repository: spark Updated Branches: refs/heads/branch-2.0 379b12729 -> f7458c71d [SPARK-17150][SQL] Support SQL generation for inline tables ## What changes were proposed in this pull request? This patch adds support for SQL generation for inline tables. With this, it would be possible to

spark git commit: [SPARK-17150][SQL] Support SQL generation for inline tables

2016-08-19 Thread wenchen
Repository: spark Updated Branches: refs/heads/master ba1737c21 -> 45d40d9f6 [SPARK-17150][SQL] Support SQL generation for inline tables ## What changes were proposed in this pull request? This patch adds support for SQL generation for inline tables. With this, it would be possible to create