spark git commit: [HOTFIX] DDLSuite was broken by 93e9714

2016-05-31 Thread andrewor14
Repository: spark Updated Branches: refs/heads/master ac38bdc75 -> 1dd925644 [HOTFIX] DDLSuite was broken by 93e9714 Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/1dd92564 Tree: http://git-wip-us.apache.org/repos/asf/spa

spark git commit: [HOTFIX] DDLSuite was broken by 93e9714

2016-05-31 Thread andrewor14
Repository: spark Updated Branches: refs/heads/branch-2.0 d34c0fc10 -> 0ade44cc4 [HOTFIX] DDLSuite was broken by 93e9714 Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/0ade44cc Tree: http://git-wip-us.apache.org/repos/asf

spark git commit: [SPARK-15601][CORE] CircularBuffer's toString() to print only the contents written if buffer isn't full

2016-05-31 Thread srowen
Repository: spark Updated Branches: refs/heads/branch-1.6 ea84b3373 -> 714f4d78a [SPARK-15601][CORE] CircularBuffer's toString() to print only the contents written if buffer isn't full 1. The class allocated 4x space than needed as it was using `Int` to store the `Byte` values 2. If Circula

spark git commit: [SPARK-15601][CORE] CircularBuffer's toString() to print only the contents written if buffer isn't full

2016-05-31 Thread srowen
Repository: spark Updated Branches: refs/heads/branch-2.0 db96f398b -> d34c0fc10 [SPARK-15601][CORE] CircularBuffer's toString() to print only the contents written if buffer isn't full ## What changes were proposed in this pull request? 1. The class allocated 4x space than needed as it was u

spark git commit: [SPARK-15601][CORE] CircularBuffer's toString() to print only the contents written if buffer isn't full

2016-05-31 Thread srowen
Repository: spark Updated Branches: refs/heads/master 04f925ede -> ac38bdc75 [SPARK-15601][CORE] CircularBuffer's toString() to print only the contents written if buffer isn't full ## What changes were proposed in this pull request? 1. The class allocated 4x space than needed as it was using

spark git commit: [SPARK-15236][SQL][SPARK SHELL] Add spark-defaults property to switch to use InMemoryCatalog

2016-05-31 Thread andrewor14
Repository: spark Updated Branches: refs/heads/branch-2.0 459fd34a0 -> db96f398b [SPARK-15236][SQL][SPARK SHELL] Add spark-defaults property to switch to use InMemoryCatalog ## What changes were proposed in this pull request? This PR change REPL/Main to check this property `spark.sql.catalog

spark git commit: [SPARK-15236][SQL][SPARK SHELL] Add spark-defaults property to switch to use InMemoryCatalog

2016-05-31 Thread andrewor14
Repository: spark Updated Branches: refs/heads/master 85d6b0db9 -> 04f925ede [SPARK-15236][SQL][SPARK SHELL] Add spark-defaults property to switch to use InMemoryCatalog ## What changes were proposed in this pull request? This PR change REPL/Main to check this property `spark.sql.catalogImpl

spark git commit: [SPARK-15618][SQL][MLLIB] Use SparkSession.builder.sparkContext if applicable.

2016-05-31 Thread andrewor14
Repository: spark Updated Branches: refs/heads/branch-2.0 ac4cb1718 -> 459fd34a0 [SPARK-15618][SQL][MLLIB] Use SparkSession.builder.sparkContext if applicable. This PR changes function `SparkSession.builder.sparkContext(..)` from **private[sql]** into **private[spark]**, and uses it if applic

spark git commit: [SPARK-15618][SQL][MLLIB] Use SparkSession.builder.sparkContext if applicable.

2016-05-31 Thread andrewor14
Repository: spark Updated Branches: refs/heads/master 93e97147e -> 85d6b0db9 [SPARK-15618][SQL][MLLIB] Use SparkSession.builder.sparkContext if applicable. ## What changes were proposed in this pull request? This PR changes function `SparkSession.builder.sparkContext(..)` from **private[sql]

spark git commit: [MINOR] Slightly better error message when attempting to query hive tables w/in-mem catalog

2016-05-31 Thread andrewor14
Repository: spark Updated Branches: refs/heads/branch-2.0 4b19c9776 -> ac4cb1718 [MINOR] Slightly better error message when attempting to query hive tables w/in-mem catalog andrewor14 Author: Eric Liang Closes #13427 from ericl/better-error-msg. (cherry picked from commit 93e97147eb499dde

spark git commit: [MINOR] Slightly better error message when attempting to query hive tables w/in-mem catalog

2016-05-31 Thread andrewor14
Repository: spark Updated Branches: refs/heads/master 196a0d827 -> 93e97147e [MINOR] Slightly better error message when attempting to query hive tables w/in-mem catalog andrewor14 Author: Eric Liang Closes #13427 from ericl/better-error-msg. Project: http://git-wip-us.apache.org/repos/as

spark git commit: [MINOR][SQL][DOCS] Fix docs of Dataset.scala and SQLImplicits.scala.

2016-05-31 Thread andrewor14
Repository: spark Updated Branches: refs/heads/branch-2.0 b8de4ad7d -> 4b19c9776 [MINOR][SQL][DOCS] Fix docs of Dataset.scala and SQLImplicits.scala. This PR fixes a sample code, a description, and indentations in docs. Manual. Author: Dongjoon Hyun Closes #13420 from dongjoon-hyun/minor_f

spark git commit: [MINOR][SQL][DOCS] Fix docs of Dataset.scala and SQLImplicits.scala.

2016-05-31 Thread andrewor14
Repository: spark Updated Branches: refs/heads/master dad5a6881 -> 196a0d827 [MINOR][SQL][DOCS] Fix docs of Dataset.scala and SQLImplicits.scala. This PR fixes a sample code, a description, and indentations in docs. Manual. Author: Dongjoon Hyun Closes #13420 from dongjoon-hyun/minor_fix_d

spark git commit: [SPARK-12988][SQL] Can't drop top level columns that contain dots

2016-05-31 Thread yhuai
Repository: spark Updated Branches: refs/heads/branch-2.0 7f240eaee -> b8de4ad7d [SPARK-12988][SQL] Can't drop top level columns that contain dots ## What changes were proposed in this pull request? Fixes "Can't drop top level columns that contain dots". This work is based on dilipbiswal's h

spark git commit: [SPARK-15670][JAVA API][SPARK CORE] label_accumulator_deprecate_in_java_spark_context

2016-05-31 Thread andrewor14
Repository: spark Updated Branches: refs/heads/branch-2.0 00fca2a05 -> 7f240eaee [SPARK-15670][JAVA API][SPARK CORE] label_accumulator_deprecate_in_java_spark_context ## What changes were proposed in this pull request? Add deprecate annotation for acumulator V1 interface in JavaSparkContext

spark git commit: [SPARK-15670][JAVA API][SPARK CORE] label_accumulator_deprecate_in_java_spark_context

2016-05-31 Thread andrewor14
Repository: spark Updated Branches: refs/heads/master 06514d689 -> dad5a6881 [SPARK-15670][JAVA API][SPARK CORE] label_accumulator_deprecate_in_java_spark_context ## What changes were proposed in this pull request? Add deprecate annotation for acumulator V1 interface in JavaSparkContext clas

spark git commit: [SPARK-12988][SQL] Can't drop top level columns that contain dots

2016-05-31 Thread yhuai
Repository: spark Updated Branches: refs/heads/master 0f2471346 -> 06514d689 [SPARK-12988][SQL] Can't drop top level columns that contain dots ## What changes were proposed in this pull request? Fixes "Can't drop top level columns that contain dots". This work is based on dilipbiswal's https

spark git commit: [CORE][DOC][MINOR] typos + links

2016-05-31 Thread andrewor14
Repository: spark Updated Branches: refs/heads/branch-2.0 f0e8738c1 -> 00fca2a05 [CORE][DOC][MINOR] typos + links ## What changes were proposed in this pull request? A very tiny change to javadoc (which I don't mind if gets merged with a bigger change). I've just found it annoying and couldn

spark git commit: [CORE][DOC][MINOR] typos + links

2016-05-31 Thread andrewor14
Repository: spark Updated Branches: refs/heads/master 8ca01a6fe -> 0f2471346 [CORE][DOC][MINOR] typos + links ## What changes were proposed in this pull request? A very tiny change to javadoc (which I don't mind if gets merged with a bigger change). I've just found it annoying and couldn't r

spark git commit: [SPARK-15680][SQL] Disable comments in generated code in order to avoid perf. issues

2016-05-31 Thread rxin
Repository: spark Updated Branches: refs/heads/master 223f1d58c -> 8ca01a6fe [SPARK-15680][SQL] Disable comments in generated code in order to avoid perf. issues ## What changes were proposed in this pull request? In benchmarks involving tables with very wide and complex schemas (thousands o

spark git commit: [SPARK-15680][SQL] Disable comments in generated code in order to avoid perf. issues

2016-05-31 Thread rxin
Repository: spark Updated Branches: refs/heads/branch-2.0 978f54e76 -> f0e8738c1 [SPARK-15680][SQL] Disable comments in generated code in order to avoid perf. issues ## What changes were proposed in this pull request? In benchmarks involving tables with very wide and complex schemas (thousan

spark git commit: [SPARK-15662][SQL] Add since annotation for classes in sql.catalog

2016-05-31 Thread andrewor14
Repository: spark Updated Branches: refs/heads/branch-2.0 1f4c4def9 -> 978f54e76 [SPARK-15662][SQL] Add since annotation for classes in sql.catalog ## What changes were proposed in this pull request? This patch does a few things: 1. Adds since version annotation to methods and classes in sql.

spark git commit: [SPARK-15662][SQL] Add since annotation for classes in sql.catalog

2016-05-31 Thread andrewor14
Repository: spark Updated Branches: refs/heads/master 695470429 -> 223f1d58c [SPARK-15662][SQL] Add since annotation for classes in sql.catalog ## What changes were proposed in this pull request? This patch does a few things: 1. Adds since version annotation to methods and classes in sql.cata

spark git commit: [CORE][MINOR][DOC] Removing incorrect scaladoc

2016-05-31 Thread srowen
Repository: spark Updated Branches: refs/heads/branch-2.0 ef8d16ec4 -> 1f4c4def9 [CORE][MINOR][DOC] Removing incorrect scaladoc ## What changes were proposed in this pull request? I don't think the method will ever throw an exception so removing a false comment. Sorry srowen and rxin again -

spark git commit: [CORE][MINOR][DOC] Removing incorrect scaladoc

2016-05-31 Thread srowen
Repository: spark Updated Branches: refs/heads/master 57adb77e6 -> 695470429 [CORE][MINOR][DOC] Removing incorrect scaladoc ## What changes were proposed in this pull request? I don't think the method will ever throw an exception so removing a false comment. Sorry srowen and rxin again -- I

spark git commit: [SPARK-15451][BUILD] Use jdk7's rt.jar when available.

2016-05-31 Thread vanzin
Repository: spark Updated Branches: refs/heads/master 90b11439b -> 57adb77e6 [SPARK-15451][BUILD] Use jdk7's rt.jar when available. This helps with preventing jdk8-specific calls being checked in, because PR builders are running the compiler with the wrong settings. If the JAVA_7_HOME env var

spark git commit: [SPARK-15451][BUILD] Use jdk7's rt.jar when available.

2016-05-31 Thread vanzin
Repository: spark Updated Branches: refs/heads/branch-2.0 df4f87106 -> ef8d16ec4 [SPARK-15451][BUILD] Use jdk7's rt.jar when available. This helps with preventing jdk8-specific calls being checked in, because PR builders are running the compiler with the wrong settings. If the JAVA_7_HOME env

spark git commit: [SPARK-15517][SQL][STREAMING] Add support for complete output mode in Structure Streaming

2016-05-31 Thread marmbrus
Repository: spark Updated Branches: refs/heads/branch-2.0 8657942ce -> df4f87106 [SPARK-15517][SQL][STREAMING] Add support for complete output mode in Structure Streaming ## What changes were proposed in this pull request? Currently structured streaming only supports append output mode. This

spark git commit: [SPARK-15517][SQL][STREAMING] Add support for complete output mode in Structure Streaming

2016-05-31 Thread marmbrus
Repository: spark Updated Branches: refs/heads/master dfe2cbeb4 -> 90b11439b [SPARK-15517][SQL][STREAMING] Add support for complete output mode in Structure Streaming ## What changes were proposed in this pull request? Currently structured streaming only supports append output mode. This PR

spark git commit: [SPARK-15557] [SQL] cast the string into DoubleType when it's used together with decimal

2016-05-31 Thread davies
Repository: spark Updated Branches: refs/heads/branch-2.0 acf56c5db -> 8657942ce [SPARK-15557] [SQL] cast the string into DoubleType when it's used together with decimal In this case, the result type of the expression becomes DECIMAL(38, 36) as we promote the individual string literals to DE

spark git commit: [SPARK-15557] [SQL] cast the string into DoubleType when it's used together with decimal

2016-05-31 Thread davies
Repository: spark Updated Branches: refs/heads/master 2df6ca848 -> dfe2cbeb4 [SPARK-15557] [SQL] cast the string into DoubleType when it's used together with decimal In this case, the result type of the expression becomes DECIMAL(38, 36) as we promote the individual string literals to DECIMA

spark git commit: [SPARK-15327] [SQL] fix split expression in whole stage codegen

2016-05-31 Thread davies
Repository: spark Updated Branches: refs/heads/branch-2.0 7feb79085 -> acf56c5db [SPARK-15327] [SQL] fix split expression in whole stage codegen ## What changes were proposed in this pull request? Right now, we will split the code for expressions into multiple functions when it exceed 64k, w

spark git commit: [SPARK-15327] [SQL] fix split expression in whole stage codegen

2016-05-31 Thread davies
Repository: spark Updated Branches: refs/heads/master 594484cd8 -> 2df6ca848 [SPARK-15327] [SQL] fix split expression in whole stage codegen ## What changes were proposed in this pull request? Right now, we will split the code for expressions into multiple functions when it exceed 64k, which

spark git commit: Revert "[SPARK-11753][SQL][TEST-HADOOP2.2] Make allowNonNumericNumbers option work

2016-05-31 Thread zsxwing
Repository: spark Updated Branches: refs/heads/master c6de5832b -> 9a74de18a Revert "[SPARK-11753][SQL][TEST-HADOOP2.2] Make allowNonNumericNumbers option work ## What changes were proposed in this pull request? This reverts commit c24b6b679c3efa053f7de19be73eb36dc70d9930. Sent a PR to run

spark git commit: [MINOR][DOC][ML] ml.clustering scala & python api doc sync

2016-05-31 Thread mlnick
Repository: spark Updated Branches: refs/heads/branch-2.0 20a07e443 -> 7feb79085 [MINOR][DOC][ML] ml.clustering scala & python api doc sync ## What changes were proposed in this pull request? Since we done Scala API audit for ml.clustering at #13148, we should also fix and update the correspo

spark git commit: [MINOR][DOC][ML] ml.clustering scala & python api doc sync

2016-05-31 Thread mlnick
Repository: spark Updated Branches: refs/heads/master 9a74de18a -> 594484cd8 [MINOR][DOC][ML] ml.clustering scala & python api doc sync ## What changes were proposed in this pull request? Since we done Scala API audit for ml.clustering at #13148, we should also fix and update the correspondin

spark git commit: Revert "[SPARK-11753][SQL][TEST-HADOOP2.2] Make allowNonNumericNumbers option work

2016-05-31 Thread zsxwing
Repository: spark Updated Branches: refs/heads/branch-2.0 45472f8e0 -> 20a07e443 Revert "[SPARK-11753][SQL][TEST-HADOOP2.2] Make allowNonNumericNumbers option work ## What changes were proposed in this pull request? This reverts commit c24b6b679c3efa053f7de19be73eb36dc70d9930. Sent a PR to r

spark git commit: [SPARK-15622][SQL] Wrap the parent classloader of Janino's classloader in the ParentClassLoader.

2016-05-31 Thread yhuai
Repository: spark Updated Branches: refs/heads/branch-2.0 29b94fdb3 -> 45472f8e0 [SPARK-15622][SQL] Wrap the parent classloader of Janino's classloader in the ParentClassLoader. ## What changes were proposed in this pull request? At https://github.com/aunkrig/janino/blob/janino_2.7.8/janino/

spark git commit: [SPARK-15622][SQL] Wrap the parent classloader of Janino's classloader in the ParentClassLoader.

2016-05-31 Thread yhuai
Repository: spark Updated Branches: refs/heads/master 2bfed1a0c -> c6de5832b [SPARK-15622][SQL] Wrap the parent classloader of Janino's classloader in the ParentClassLoader. ## What changes were proposed in this pull request? At https://github.com/aunkrig/janino/blob/janino_2.7.8/janino/src/

spark git commit: [SPARK-15658][SQL] UDT serializer should declare its data type as udt instead of udt.sqlType

2016-05-31 Thread yhuai
Repository: spark Updated Branches: refs/heads/branch-2.0 6347ff512 -> 29b94fdb3 [SPARK-15658][SQL] UDT serializer should declare its data type as udt instead of udt.sqlType ## What changes were proposed in this pull request? When we build serializer for UDT object, we should declare its dat

spark git commit: [SPARK-15658][SQL] UDT serializer should declare its data type as udt instead of udt.sqlType

2016-05-31 Thread yhuai
Repository: spark Updated Branches: refs/heads/master d67c82e4b -> 2bfed1a0c [SPARK-15658][SQL] UDT serializer should declare its data type as udt instead of udt.sqlType ## What changes were proposed in this pull request? When we build serializer for UDT object, we should declare its data ty

spark git commit: [SPARK-15647][SQL] Fix Boundary Cases in OptimizeCodegen Rule

2016-05-31 Thread wenchen
Repository: spark Updated Branches: refs/heads/branch-2.0 e11046457 -> 6347ff512 [SPARK-15647][SQL] Fix Boundary Cases in OptimizeCodegen Rule What changes were proposed in this pull request? The following condition in the Optimizer rule `OptimizeCodegen` is not right. ```Scala branches.

spark git commit: [SPARK-15647][SQL] Fix Boundary Cases in OptimizeCodegen Rule

2016-05-31 Thread wenchen
Repository: spark Updated Branches: refs/heads/master 2bfc4f152 -> d67c82e4b [SPARK-15647][SQL] Fix Boundary Cases in OptimizeCodegen Rule What changes were proposed in this pull request? The following condition in the Optimizer rule `OptimizeCodegen` is not right. ```Scala branches.size

spark git commit: [SPARK-15649][SQL] Avoid to serialize MetastoreRelation in HiveTableScanExec

2016-05-31 Thread rxin
Repository: spark Updated Branches: refs/heads/branch-2.0 2e3ead20c -> e11046457 [SPARK-15649][SQL] Avoid to serialize MetastoreRelation in HiveTableScanExec ## What changes were proposed in this pull request? in HiveTableScanExec, schema is lazy and is related with relation.attributeMap. So

spark git commit: [SPARK-15649][SQL] Avoid to serialize MetastoreRelation in HiveTableScanExec

2016-05-31 Thread rxin
Repository: spark Updated Branches: refs/heads/master 95db8a44f -> 2bfc4f152 [SPARK-15649][SQL] Avoid to serialize MetastoreRelation in HiveTableScanExec ## What changes were proposed in this pull request? in HiveTableScanExec, schema is lazy and is related with relation.attributeMap. So it n

spark git commit: [SPARK-15528][SQL] Fix race condition in NumberConverter

2016-05-31 Thread srowen
Repository: spark Updated Branches: refs/heads/branch-1.6 c53c83ce8 -> ea84b3373 [SPARK-15528][SQL] Fix race condition in NumberConverter ## What changes were proposed in this pull request? A local variable in NumberConverter is wrongly shared between threads. This pr fixes the race condition.

spark git commit: [SPARK-15528][SQL] Fix race condition in NumberConverter

2016-05-31 Thread srowen
Repository: spark Updated Branches: refs/heads/branch-2.0 3d00c7015 -> 2e3ead20c [SPARK-15528][SQL] Fix race condition in NumberConverter ## What changes were proposed in this pull request? A local variable in NumberConverter is wrongly shared between threads. This pr fixes the race condition.

spark git commit: [SPARK-15528][SQL] Fix race condition in NumberConverter

2016-05-31 Thread srowen
Repository: spark Updated Branches: refs/heads/master 6878f3e2e -> 95db8a44f [SPARK-15528][SQL] Fix race condition in NumberConverter ## What changes were proposed in this pull request? A local variable in NumberConverter is wrongly shared between threads. This pr fixes the race condition. ##

spark git commit: [SPARK-15641] HistoryServer to not show invalid date for incomplete application

2016-05-31 Thread srowen
Repository: spark Updated Branches: refs/heads/branch-2.0 2b35ce818 -> 3d00c7015 [SPARK-15641] HistoryServer to not show invalid date for incomplete application ## What changes were proposed in this pull request? For incomplete applications in HistoryServer, the complete column will show "-"

spark git commit: [SPARK-15641] HistoryServer to not show invalid date for incomplete application

2016-05-31 Thread srowen
Repository: spark Updated Branches: refs/heads/master 675921040 -> 6878f3e2e [SPARK-15641] HistoryServer to not show invalid date for incomplete application ## What changes were proposed in this pull request? For incomplete applications in HistoryServer, the complete column will show "-" inst