spark git commit: [SPARK-13659] Refactor BlockStore put*() APIs to remove returnValues

2016-03-07 Thread andrewor14
Repository: spark Updated Branches: refs/heads/master 017cdf2be -> e52e597db [SPARK-13659] Refactor BlockStore put*() APIs to remove returnValues In preparation for larger refactoring, this patch removes the confusing `returnValues` option from the BlockStore put() APIs: returning the value

spark git commit: [SPARK-13711][CORE] Don't call SparkUncaughtExceptionHandler in AppClient as it's in driver

2016-03-07 Thread zsxwing
Repository: spark Updated Branches: refs/heads/branch-1.6 695c8a257 -> bace137f2 [SPARK-13711][CORE] Don't call SparkUncaughtExceptionHandler in AppClient as it's in driver ## What changes were proposed in this pull request? AppClient runs in the driver side. It should not call

spark git commit: [SPARK-13711][CORE] Don't call SparkUncaughtExceptionHandler in AppClient as it's in driver

2016-03-07 Thread zsxwing
Repository: spark Updated Branches: refs/heads/master 25bba58d1 -> 017cdf2be [SPARK-13711][CORE] Don't call SparkUncaughtExceptionHandler in AppClient as it's in driver ## What changes were proposed in this pull request? AppClient runs in the driver side. It should not call

spark git commit: [SPARK-13404] [SQL] Create variables for input row when it's actually used

2016-03-07 Thread davies
Repository: spark Updated Branches: refs/heads/master da7bfac48 -> 25bba58d1 [SPARK-13404] [SQL] Create variables for input row when it's actually used ## What changes were proposed in this pull request? This PR change the way how we generate the code for the output variables passing from a

spark git commit: [SPARK-13689][SQL] Move helper things in CatalystQl to new utils object

2016-03-07 Thread andrewor14
Repository: spark Updated Branches: refs/heads/master 46f25c241 -> da7bfac48 [SPARK-13689][SQL] Move helper things in CatalystQl to new utils object ## What changes were proposed in this pull request? When we add more DDL parsing logic in the future, SparkQl will become very big. To keep it

spark git commit: [SPARK-13648] Add Hive Cli to classes for isolated classloader

2016-03-07 Thread marmbrus
Repository: spark Updated Branches: refs/heads/branch-1.6 cf4e62ec2 -> 695c8a257 [SPARK-13648] Add Hive Cli to classes for isolated classloader ## What changes were proposed in this pull request? Adding the hive-cli classes to the classloader ## How was this patch tested? The hive

spark git commit: [SPARK-13648] Add Hive Cli to classes for isolated classloader

2016-03-07 Thread marmbrus
Repository: spark Updated Branches: refs/heads/master e720dda42 -> 46f25c241 [SPARK-13648] Add Hive Cli to classes for isolated classloader ## What changes were proposed in this pull request? Adding the hive-cli classes to the classloader ## How was this patch tested? The hive

[1/4] spark git commit: [SPARK-13665][SQL] Separate the concerns of HadoopFsRelation

2016-03-07 Thread rxin
Repository: spark Updated Branches: refs/heads/master 0eea12a3d -> e720dda42 http://git-wip-us.apache.org/repos/asf/spark/blob/e720dda4/sql/hive/src/test/scala/org/apache/spark/sql/sources/SimpleTextRelation.scala -- diff

[2/4] spark git commit: [SPARK-13665][SQL] Separate the concerns of HadoopFsRelation

2016-03-07 Thread rxin
http://git-wip-us.apache.org/repos/asf/spark/blob/e720dda4/sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/json/JsonSuite.scala -- diff --git

spark git commit: [SPARK-13596][BUILD] Move misc top-level build files into appropriate subdirs

2016-03-07 Thread rxin
Repository: spark Updated Branches: refs/heads/master 8577260ab -> 0eea12a3d [SPARK-13596][BUILD] Move misc top-level build files into appropriate subdirs ## What changes were proposed in this pull request? Move many top-level files in dev/ or other appropriate directory. In particular, put

spark git commit: [SPARK-13442][SQL] Make type inference recognize boolean types

2016-03-07 Thread rxin
Repository: spark Updated Branches: refs/heads/master e1fb85799 -> 8577260ab [SPARK-13442][SQL] Make type inference recognize boolean types ## What changes were proposed in this pull request? https://issues.apache.org/jira/browse/SPARK-13442 This PR adds the support for inferring

[2/2] spark git commit: [SPARK-529][CORE][YARN] Add type-safe config keys to SparkConf.

2016-03-07 Thread vanzin
[SPARK-529][CORE][YARN] Add type-safe config keys to SparkConf. This is, in a way, the basics to enable SPARK-529 (which was closed as won't fix but I think is still valuable). In fact, Spark SQL created something for that, and this change basically factors out that code and inserts it into

[1/2] spark git commit: [SPARK-529][CORE][YARN] Add type-safe config keys to SparkConf.

2016-03-07 Thread vanzin
Repository: spark Updated Branches: refs/heads/master e9e67b39a -> e1fb85799 http://git-wip-us.apache.org/repos/asf/spark/blob/e1fb8579/yarn/src/main/scala/org/apache/spark/scheduler/cluster/SchedulerExtensionService.scala --

spark git commit: [SPARK-13655] Improve isolation between tests in KinesisBackedBlockRDDSuite

2016-03-07 Thread zsxwing
Repository: spark Updated Branches: refs/heads/master b6071a700 -> e9e67b39a [SPARK-13655] Improve isolation between tests in KinesisBackedBlockRDDSuite This patch modifies `KinesisBackedBlockRDDTests` to increase the isolation between tests in order to fix a bug which causes the tests to

spark git commit: [MINOR][DOC] improve the doc for "spark.memory.offHeap.size"

2016-03-07 Thread zsxwing
Repository: spark Updated Branches: refs/heads/branch-1.6 2434f16cc -> cf4e62ec2 [MINOR][DOC] improve the doc for "spark.memory.offHeap.size" The description of "spark.memory.offHeap.size" in the current document does not clearly state that memory is counted with bytes This PR contains

spark git commit: [SPARK-13722][SQL] No Push Down for Non-deterministics Predicates through Generate

2016-03-07 Thread marmbrus
Repository: spark Updated Branches: refs/heads/master a3ec50a4b -> b6071a700 [SPARK-13722][SQL] No Push Down for Non-deterministics Predicates through Generate What changes were proposed in this pull request? Non-deterministic predicates should not be pushed through Generate. How

spark git commit: [MINOR][DOC] improve the doc for "spark.memory.offHeap.size"

2016-03-07 Thread zsxwing
Repository: spark Updated Branches: refs/heads/master e72914f37 -> a3ec50a4b [MINOR][DOC] improve the doc for "spark.memory.offHeap.size" The description of "spark.memory.offHeap.size" in the current document does not clearly state that memory is counted with bytes This PR contains a

spark git commit: [SPARK-12243][BUILD][PYTHON] PySpark tests are slow in Jenkins.

2016-03-07 Thread joshrosen
Repository: spark Updated Branches: refs/heads/master ef7700317 -> e72914f37 [SPARK-12243][BUILD][PYTHON] PySpark tests are slow in Jenkins. ## What changes were proposed in this pull request? In the Jenkins pull request builder, PySpark tests take around [962 seconds

spark git commit: [SPARK-13495][SQL] Add Null Filters in the query plan for Filters/Joins based on their data constraints

2016-03-07 Thread yhuai
Repository: spark Updated Branches: refs/heads/master 489641117 -> ef7700317 [SPARK-13495][SQL] Add Null Filters in the query plan for Filters/Joins based on their data constraints ## What changes were proposed in this pull request? This PR adds an optimizer rule to eliminate reading

spark git commit: [SPARK-13694][SQL] QueryPlan.expressions should always include all expressions

2016-03-07 Thread marmbrus
Repository: spark Updated Branches: refs/heads/master d7eac9d79 -> 489641117 [SPARK-13694][SQL] QueryPlan.expressions should always include all expressions ## What changes were proposed in this pull request? It's weird that expressions don't always have all the expressions in it. This PR

spark git commit: [SPARK-13651] Generator outputs are not resolved correctly resulting in run time error

2016-03-07 Thread davies
Repository: spark Updated Branches: refs/heads/master 03f57a6c2 -> d7eac9d79 [SPARK-13651] Generator outputs are not resolved correctly resulting in run time error ## What changes were proposed in this pull request? ``` Seq(("id1", "value1")).toDF("key", "value").registerTempTable("src")

spark git commit: [SPARK-13599][BUILD] remove transitive groovy dependencies from spark-hive and spark-hiveserver (branch 1.6)

2016-03-07 Thread srowen
Repository: spark Updated Branches: refs/heads/branch-1.6 18ef2f25e -> 2434f16cc [SPARK-13599][BUILD] remove transitive groovy dependencies from spark-hive and spark-hiveserver (branch 1.6) ## What changes were proposed in this pull request? This is just the patch of #11449 cherry picked to

spark git commit: Fixing the type of the sentiment happiness value

2016-03-07 Thread srowen
Repository: spark Updated Branches: refs/heads/master 4b13896eb -> 03f57a6c2 Fixing the type of the sentiment happiness value ## What changes were proposed in this pull request? Added the conversion to int for the 'happiness value' read from the file. Otherwise, later on line 75 the

spark git commit: [SPARK-13705][DOCS] UpdateStateByKey Operation documentation incorrectly refers to StatefulNetworkWordCount

2016-03-07 Thread srowen
Repository: spark Updated Branches: refs/heads/branch-1.6 704a54c55 -> 18ef2f25e [SPARK-13705][DOCS] UpdateStateByKey Operation documentation incorrectly refers to StatefulNetworkWordCount ## What changes were proposed in this pull request? The reference to StatefulNetworkWordCount.scala

spark git commit: [SPARK-13705][DOCS] UpdateStateByKey Operation documentation incorrectly refers to StatefulNetworkWordCount

2016-03-07 Thread srowen
Repository: spark Updated Branches: refs/heads/master bc7a3ec29 -> 4b13896eb [SPARK-13705][DOCS] UpdateStateByKey Operation documentation incorrectly refers to StatefulNetworkWordCount ## What changes were proposed in this pull request? The reference to StatefulNetworkWordCount.scala from

spark git commit: [SPARK-13685][SQL] Rename catalog.Catalog to ExternalCatalog

2016-03-07 Thread rxin
Repository: spark Updated Branches: refs/heads/master ee913e6e2 -> bc7a3ec29 [SPARK-13685][SQL] Rename catalog.Catalog to ExternalCatalog ## What changes were proposed in this pull request? Today we have `analysis.Catalog` and `catalog.Catalog`. In the future the former will call the