spark git commit: [SPARK-16331][SQL] Reduce code generation time

Repository: spark Updated Branches: refs/heads/master aa6564f37 -> 14cf61e90 [SPARK-16331][SQL] Reduce code generation time ## What changes were proposed in this pull request? During the code generation, a `LocalRelation` often has a huge `Vector` object as `data`. In the simple example below

spark git commit: [SPARK-14608][ML] transformSchema needs better documentation

Repository: spark Updated Branches: refs/heads/branch-2.0 80a7bff89 -> cc3c44b11 [SPARK-14608][ML] transformSchema needs better documentation ## What changes were proposed in this pull request? jira: https://issues.apache.org/jira/browse/SPARK-14608 PipelineStage.transformSchema currently has

spark git commit: [SPARK-14608][ML] transformSchema needs better documentation

Repository: spark Updated Branches: refs/heads/master 38f4d6f44 -> aa6564f37 [SPARK-14608][ML] transformSchema needs better documentation ## What changes were proposed in this pull request? jira: https://issues.apache.org/jira/browse/SPARK-14608 PipelineStage.transformSchema currently has mini

spark git commit: [SPARK-15820][PYSPARK][SQL] Add Catalog.refreshTable into python API

Repository: spark Updated Branches: refs/heads/branch-2.0 94d61de9c -> 80a7bff89 [SPARK-15820][PYSPARK][SQL] Add Catalog.refreshTable into python API ## What changes were proposed in this pull request? Add Catalog.refreshTable API into python interface for Spark-SQL. ## How was this patch te

spark git commit: [SPARK-15954][SQL] Disable loading test tables in Python tests

Repository: spark Updated Branches: refs/heads/master 4a981dc87 -> 38f4d6f44 [SPARK-15954][SQL] Disable loading test tables in Python tests ## What changes were proposed in this pull request? This patch introduces a flag to disable loading test tables in TestHiveSparkSession and disables that

spark git commit: [SPARK-15954][SQL] Disable loading test tables in Python tests

Repository: spark Updated Branches: refs/heads/branch-2.0 79c96c999 -> 94d61de9c [SPARK-15954][SQL] Disable loading test tables in Python tests ## What changes were proposed in this pull request? This patch introduces a flag to disable loading test tables in TestHiveSparkSession and disables

spark git commit: [SPARK-15643][DOC][ML] Add breaking changes to ML migration guide

Repository: spark Updated Branches: refs/heads/master dab105161 -> 4a981dc87 [SPARK-15643][DOC][ML] Add breaking changes to ML migration guide This PR adds the breaking changes from [SPARK-14810](https://issues.apache.org/jira/browse/SPARK-14810) to the migration guide. ## How was this patc

spark git commit: [SPARK-15643][DOC][ML] Add breaking changes to ML migration guide

Repository: spark Updated Branches: refs/heads/branch-2.0 d3027c45f -> 79c96c999 [SPARK-15643][DOC][ML] Add breaking changes to ML migration guide This PR adds the breaking changes from [SPARK-14810](https://issues.apache.org/jira/browse/SPARK-14810) to the migration guide. ## How was this

spark git commit: [SPARK-16328][ML][MLLIB][PYSPARK] Add 'asML' and 'fromML' conversion methods to PySpark linalg

Repository: spark Updated Branches: refs/heads/branch-2.0 17c7522c8 -> d3027c45f [SPARK-16328][ML][MLLIB][PYSPARK] Add 'asML' and 'fromML' conversion methods to PySpark linalg The move to `ml.linalg` created `asML`/`fromML` utility methods in Scala/Java for converting between representations

spark git commit: [SPARK-16328][ML][MLLIB][PYSPARK] Add 'asML' and 'fromML' conversion methods to PySpark linalg

Repository: spark Updated Branches: refs/heads/master 85f2303ec -> dab105161 [SPARK-16328][ML][MLLIB][PYSPARK] Add 'asML' and 'fromML' conversion methods to PySpark linalg The move to `ml.linalg` created `asML`/`fromML` utility methods in Scala/Java for converting between representations. Th

spark git commit: [SPARK-16276][SQL] Implement elt SQL function

Repository: spark Updated Branches: refs/heads/master 3d75a5b2a -> 85f2303ec [SPARK-16276][SQL] Implement elt SQL function ## What changes were proposed in this pull request? This patch implements the elt function, as it is implemented in Hive. ## How was this patch tested? Added expression u

spark git commit: [SPARK-16313][SQL] Spark should not silently drop exceptions in file listing

Repository: spark Updated Branches: refs/heads/branch-2.0 4dc7d377f -> 17c7522c8 [SPARK-16313][SQL] Spark should not silently drop exceptions in file listing ## What changes were proposed in this pull request? Spark silently drops exceptions during file listing. This is a very bad behavior be

spark git commit: [SPARK-16313][SQL] Spark should not silently drop exceptions in file listing

Repository: spark Updated Branches: refs/heads/master fb41670c9 -> 3d75a5b2a [SPARK-16313][SQL] Spark should not silently drop exceptions in file listing ## What changes were proposed in this pull request? Spark silently drops exceptions during file listing. This is a very bad behavior becaus

spark git commit: [SPARK-16336][SQL] Suggest doing table refresh upon FileNotFoundException

Repository: spark Updated Branches: refs/heads/branch-2.0 03008e049 -> 4dc7d377f [SPARK-16336][SQL] Suggest doing table refresh upon FileNotFoundException ## What changes were proposed in this pull request? This patch appends a message to suggest users running refresh table or reloading data

spark git commit: [SPARK-16336][SQL] Suggest doing table refresh upon FileNotFoundException

Repository: spark Updated Branches: refs/heads/master 5d00a7bc1 -> fb41670c9 [SPARK-16336][SQL] Suggest doing table refresh upon FileNotFoundException ## What changes were proposed in this pull request? This patch appends a message to suggest users running refresh table or reloading data fram

spark git commit: [SPARK-16256][DOCS] Fix window operation diagram

Repository: spark Updated Branches: refs/heads/branch-2.0 f17ffef38 -> 03008e049 [SPARK-16256][DOCS] Fix window operation diagram Author: Tathagata Das Closes #14001 from tdas/SPARK-16256-2. (cherry picked from commit 5d00a7bc19ddeb1b5247733b55095a03ee7b1a30) Signed-off-by: Tathagata Das

spark git commit: [SPARK-16256][DOCS] Fix window operation diagram

Repository: spark Updated Branches: refs/heads/master c62263340 -> 5d00a7bc1 [SPARK-16256][DOCS] Fix window operation diagram Author: Tathagata Das Closes #14001 from tdas/SPARK-16256-2. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/a

spark git commit: [SPARK-16212][STREAMING][KAFKA] code cleanup from review feedback

Repository: spark Updated Branches: refs/heads/master 46395db80 -> c62263340 [SPARK-16212][STREAMING][KAFKA] code cleanup from review feedback ## What changes were proposed in this pull request? code cleanup in kafka-0-8 to match suggested changes for kafka-0-10 branch ## How was this patch t

spark git commit: [SPARK-16289][SQL] Implement posexplode table generating function

Repository: spark Updated Branches: refs/heads/master fdf9f94f8 -> 46395db80 [SPARK-16289][SQL] Implement posexplode table generating function ## What changes were proposed in this pull request? This PR implements `posexplode` table generating function. Currently, master branch raises the fo

spark git commit: [SPARK-15865][CORE] Blacklist should not result in job hanging with less than 4 executors

Repository: spark Updated Branches: refs/heads/master 07f46afc7 -> fdf9f94f8 [SPARK-15865][CORE] Blacklist should not result in job hanging with less than 4 executors ## What changes were proposed in this pull request? Before this change, when you turn on blacklisting with `spark.scheduler.

spark git commit: [SPARK-13850] Force the sorter to Spill when number of elements in th…

Repository: spark Updated Branches: refs/heads/branch-2.0 98056a1f8 -> f17ffef38 [SPARK-13850] Force the sorter to Spill when number of elements in th… Force the sorter to Spill when number of elements in the pointer array reach a certain size. This is to workaround the issue of timSort fai

spark git commit: [SPARK-13850] Force the sorter to Spill when number of elements in th…

Repository: spark Updated Branches: refs/heads/master 5344bade8 -> 07f46afc7 [SPARK-13850] Force the sorter to Spill when number of elements in th… ## What changes were proposed in this pull request? Force the sorter to Spill when number of elements in the pointer array reach a certain siz

spark git commit: [SPARK-15820][PYSPARK][SQL] Add Catalog.refreshTable into python API

Repository: spark Updated Branches: refs/heads/master 5320adc86 -> 5344bade8 [SPARK-15820][PYSPARK][SQL] Add Catalog.refreshTable into python API ## What changes were proposed in this pull request? Add Catalog.refreshTable API into python interface for Spark-SQL. ## How was this patch tested

spark git commit: [BUILD] Fix version in poms related to kafka-0-10

Repository: spark Updated Branches: refs/heads/branch-2.0 56207fc3b -> 98056a1f8 [BUILD] Fix version in poms related to kafka-0-10 self explanatory Author: Tathagata Das Closes #13994 from tdas/SPARK-12177-1. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wi

spark git commit: [SPARK-16071][SQL] Checks size limit when doubling the array size in BufferHolder

Repository: spark Updated Branches: refs/heads/branch-2.0 6a4f4c1d7 -> 56207fc3b [SPARK-16071][SQL] Checks size limit when doubling the array size in BufferHolder ## What changes were proposed in this pull request? This PR Checks the size limit when doubling the array size in BufferHolder to

spark git commit: [SPARK-16071][SQL] Checks size limit when doubling the array size in BufferHolder

Repository: spark Updated Branches: refs/heads/master de8ab313e -> 5320adc86 [SPARK-16071][SQL] Checks size limit when doubling the array size in BufferHolder ## What changes were proposed in this pull request? This PR Checks the size limit when doubling the array size in BufferHolder to av

spark git commit: [SPARK-12177][TEST] Removed test to avoid compilation issue in scala 2.10

Repository: spark Updated Branches: refs/heads/branch-2.0 1d274455c -> 6a4f4c1d7 [SPARK-12177][TEST] Removed test to avoid compilation issue in scala 2.10 ## What changes were proposed in this pull request? The commented lines failed scala 2.10 build. This is because of change in behavior of

spark git commit: [SPARK-12177][TEST] Removed test to avoid compilation issue in scala 2.10

Repository: spark Updated Branches: refs/heads/master b30a2dc7c -> de8ab313e [SPARK-12177][TEST] Removed test to avoid compilation issue in scala 2.10 ## What changes were proposed in this pull request? The commented lines failed scala 2.10 build. This is because of change in behavior of cas

spark git commit: [SPARK-16241][ML] model loading backward compatibility for ml NaiveBayes

Repository: spark Updated Branches: refs/heads/master 2c3d96134 -> b30a2dc7c [SPARK-16241][ML] model loading backward compatibility for ml NaiveBayes ## What changes were proposed in this pull request? model loading backward compatibility for ml NaiveBayes ## How was this patch tested? exis

spark git commit: [SPARK-16241][ML] model loading backward compatibility for ml NaiveBayes

Repository: spark Updated Branches: refs/heads/branch-2.0 c8a7c2305 -> 1d274455c [SPARK-16241][ML] model loading backward compatibility for ml NaiveBayes ## What changes were proposed in this pull request? model loading backward compatibility for ml NaiveBayes ## How was this patch tested?