spark git commit: [SPARK][EXAMPLE] Added missing semicolon in quick-start-guide example

Repository: spark Updated Branches: refs/heads/branch-2.0 1b1c849bf -> 5ecd3c23a [SPARK][EXAMPLE] Added missing semicolon in quick-start-guide example ## What changes were proposed in this pull request? Added missing semicolon in quick-start-guide java example code which wasn't compiling

spark git commit: [SPARK-18640] Add synchronization to TaskScheduler.runningTasksByExecutors

Repository: spark Updated Branches: refs/heads/branch-2.0 8b33aa089 -> 1b1c849bf [SPARK-18640] Add synchronization to TaskScheduler.runningTasksByExecutors ## What changes were proposed in this pull request? The method `TaskSchedulerImpl.runningTasksByExecutors()` accesses the mutable

spark git commit: [SPARK-18640] Add synchronization to TaskScheduler.runningTasksByExecutors

Repository: spark Updated Branches: refs/heads/branch-2.1 eae85da38 -> 7c0e2962d [SPARK-18640] Add synchronization to TaskScheduler.runningTasksByExecutors ## What changes were proposed in this pull request? The method `TaskSchedulerImpl.runningTasksByExecutors()` accesses the mutable

spark git commit: [SPARK-17680][SQL][TEST] Added test cases for InMemoryRelation

Repository: spark Updated Branches: refs/heads/branch-2.1 81e3f9711 -> b386943b2 [SPARK-17680][SQL][TEST] Added test cases for InMemoryRelation ## What changes were proposed in this pull request? This pull request adds test cases for the following cases: - keep all data types with null or

spark git commit: [SPARK-17680][SQL][TEST] Added test cases for InMemoryRelation

Repository: spark Updated Branches: refs/heads/master 0f5f52a3d -> ad67993b7 [SPARK-17680][SQL][TEST] Added test cases for InMemoryRelation ## What changes were proposed in this pull request? This pull request adds test cases for the following cases: - keep all data types with null or

spark git commit: [SPARK-18050][SQL] do not create default database if it already exists

Repository: spark Updated Branches: refs/heads/master 70ad07a9d -> f129ebcd3 [SPARK-18050][SQL] do not create default database if it already exists ## What changes were proposed in this pull request? When we try to create the default database, we ask hive to do nothing if it already exists.

spark git commit: [SPARK-18050][SQL] do not create default database if it already exists

Repository: spark Updated Branches: refs/heads/branch-2.1 599dac159 -> 835f03f34 [SPARK-18050][SQL] do not create default database if it already exists ## What changes were proposed in this pull request? When we try to create the default database, we ask hive to do nothing if it already

spark git commit: [SPARK-18507][SQL] HiveExternalCatalog.listPartitions should only call getTable once

Repository: spark Updated Branches: refs/heads/branch-2.1 0e624e990 -> fa360134d [SPARK-18507][SQL] HiveExternalCatalog.listPartitions should only call getTable once ## What changes were proposed in this pull request? HiveExternalCatalog.listPartitions should only call `getTable` once,

spark git commit: [SPARK-18507][SQL] HiveExternalCatalog.listPartitions should only call getTable once

Repository: spark Updated Branches: refs/heads/master 45ea46b7b -> 702cd403f [SPARK-18507][SQL] HiveExternalCatalog.listPartitions should only call getTable once ## What changes were proposed in this pull request? HiveExternalCatalog.listPartitions should only call `getTable` once, instead

spark git commit: [SPARK-18361][PYSPARK] Expose RDD localCheckpoint in PySpark

Repository: spark Updated Branches: refs/heads/branch-2.1 b0a73c9be -> 406f33987 [SPARK-18361][PYSPARK] Expose RDD localCheckpoint in PySpark ## What changes were proposed in this pull request? Expose RDD's localCheckpoint() and associated functions in PySpark. ## How was this patch tested?

spark git commit: [SPARK-18517][SQL] DROP TABLE IF EXISTS should not warn for non-existing tables

Repository: spark Updated Branches: refs/heads/branch-2.1 251a99276 -> b0a73c9be [SPARK-18517][SQL] DROP TABLE IF EXISTS should not warn for non-existing tables ## What changes were proposed in this pull request? Currently, `DROP TABLE IF EXISTS` shows warning for non-existing tables.

spark git commit: [SPARK-18517][SQL] DROP TABLE IF EXISTS should not warn for non-existing tables

Repository: spark Updated Branches: refs/heads/master 70176871a -> ddd02f50b [SPARK-18517][SQL] DROP TABLE IF EXISTS should not warn for non-existing tables ## What changes were proposed in this pull request? Currently, `DROP TABLE IF EXISTS` shows warning for non-existing tables. However,

spark git commit: [SPARK-18361][PYSPARK] Expose RDD localCheckpoint in PySpark

Repository: spark Updated Branches: refs/heads/master 07beb5d21 -> 70176871a [SPARK-18361][PYSPARK] Expose RDD localCheckpoint in PySpark ## What changes were proposed in this pull request? Expose RDD's localCheckpoint() and associated functions in PySpark. ## How was this patch tested? I

spark git commit: [SPARK-17686][CORE] Support printing out scala and java version with spark-submit --version command

Repository: spark Updated Branches: refs/heads/master db8784fea -> 7bf8a4049 [SPARK-17686][CORE] Support printing out scala and java version with spark-submit --version command ## What changes were proposed in this pull request? In our universal gateway service we need to specify different

spark git commit: [SPARK-17899][SQL] add a debug mode to keep raw table properties in HiveExternalCatalog

Repository: spark Updated Branches: refs/heads/master 6f2fa6c54 -> db8784fea [SPARK-17899][SQL] add a debug mode to keep raw table properties in HiveExternalCatalog ## What changes were proposed in this pull request? Currently `HiveExternalCatalog` will filter out the Spark SQL internal

spark git commit: [SPARK-11272][WEB UI] Add support for downloading event logs from HistoryServer UI

Repository: spark Updated Branches: refs/heads/master 7222a25a1 -> 6f2fa6c54 [SPARK-11272][WEB UI] Add support for downloading event logs from HistoryServer UI ## What changes were proposed in this pull request? This is a reworked PR based on feedback in #9238 after it was closed and not

spark git commit: [SPARK-16827] Stop reporting spill metrics as shuffle metrics

Repository: spark Updated Branches: refs/heads/master 2b01d3c70 -> e56614cba [SPARK-16827] Stop reporting spill metrics as shuffle metrics ## What changes were proposed in this pull request? Fix a bug where spill metrics were being reported as shuffle metrics. Eventually these spill metrics

spark git commit: [SPARK-17715][SCHEDULER] Make task launch logs DEBUG

Repository: spark Updated Branches: refs/heads/master cb87b3ced -> 027dea8f2 [SPARK-17715][SCHEDULER] Make task launch logs DEBUG ## What changes were proposed in this pull request? Ramp down the task launch logs from INFO to DEBUG. Task launches can happen orders of magnitude more than

spark git commit: [SPARK-17672] Spark 2.0 history server web Ui takes too long for a single application

Repository: spark Updated Branches: refs/heads/branch-2.0 f7839e47c -> 7c9450b00 [SPARK-17672] Spark 2.0 history server web Ui takes too long for a single application Added a new API getApplicationInfo(appId: String) in class ApplicationHistoryProvider and class SparkUI to get app info. In

spark git commit: [SPARK-17672] Spark 2.0 history server web Ui takes too long for a single application

Repository: spark Updated Branches: refs/heads/master 7f779e743 -> cb87b3ced [SPARK-17672] Spark 2.0 history server web Ui takes too long for a single application Added a new API getApplicationInfo(appId: String) in class ApplicationHistoryProvider and class SparkUI to get app info. In this

spark git commit: [SPARK-17648][CORE] TaskScheduler really needs offers to be an IndexedSeq

Repository: spark Updated Branches: refs/heads/master 958200497 -> 7f779e743 [SPARK-17648][CORE] TaskScheduler really needs offers to be an IndexedSeq ## What changes were proposed in this pull request? The Seq[WorkerOffer] is accessed by index, so it really should be an IndexedSeq,

spark git commit: [Docs] Update spark-standalone.md to fix link

Repository: spark Updated Branches: refs/heads/branch-2.0 8a58f2e8e -> f4594900d [Docs] Update spark-standalone.md to fix link Corrected a link to the configuration.html page, it was pointing to a page that does not exist (configurations.html). Documentation change, verified in preview.

spark git commit: [Docs] Update spark-standalone.md to fix link

Repository: spark Updated Branches: refs/heads/master 7c7586aef -> 00be16df6 [Docs] Update spark-standalone.md to fix link Corrected a link to the configuration.html page, it was pointing to a page that does not exist (configurations.html). Documentation change, verified in preview.

spark git commit: [SPARK-17512][CORE] Avoid formatting to python path for yarn and mesos cluster mode

Repository: spark Updated Branches: refs/heads/branch-2.0 cd0bd89d7 -> 59e6ab11a [SPARK-17512][CORE] Avoid formatting to python path for yarn and mesos cluster mode ## What changes were proposed in this pull request? Yarn and mesos cluster mode support remote python path (HDFS/S3 scheme) by

spark git commit: [SPARK-17512][CORE] Avoid formatting to python path for yarn and mesos cluster mode

Repository: spark Updated Branches: refs/heads/master 9fcf1c51d -> 8c3ee2bc4 [SPARK-17512][CORE] Avoid formatting to python path for yarn and mesos cluster mode ## What changes were proposed in this pull request? Yarn and mesos cluster mode support remote python path (HDFS/S3 scheme) by

spark git commit: [SPARK-17623][CORE] Clarify type of TaskEndReason with a failed task.

Repository: spark Updated Branches: refs/heads/master 2cd1bfa4f -> 9fcf1c51d [SPARK-17623][CORE] Clarify type of TaskEndReason with a failed task. ## What changes were proposed in this pull request? In TaskResultGetter, enqueueFailedTask currently deserializes the result as a TaskEndReason.

spark git commit: [SPARK-17438][WEBUI] Show Application.executorLimit in the application page

Repository: spark Updated Branches: refs/heads/branch-2.0 f56035ba6 -> d6191a067 [SPARK-17438][WEBUI] Show Application.executorLimit in the application page ## What changes were proposed in this pull request? This PR adds `Application.executorLimit` to the applicatino page ## How was this

spark git commit: [SPARK-17438][WEBUI] Show Application.executorLimit in the application page

Repository: spark Updated Branches: refs/heads/master cdea1d134 -> 80d665592 [SPARK-17438][WEBUI] Show Application.executorLimit in the application page ## What changes were proposed in this pull request? This PR adds `Application.executorLimit` to the applicatino page ## How was this patch

spark git commit: [SPARK-15966][DOC] Add closing tag to fix rendering issue for Spark monitoring

Repository: spark Updated Branches: refs/heads/master 9040d83bc -> f1bf0d2f3 [SPARK-15966][DOC] Add closing tag to fix rendering issue for Spark monitoring ## What changes were proposed in this pull request? Adds the missing closing tag for spark.ui.view.acls.groups ## How was this patch

spark git commit: [SPARK-15749][SQL] make the error message more meaningful

Repository: spark Updated Branches: refs/heads/branch-2.0 27e274c3e -> 2280ad8a3 [SPARK-15749][SQL] make the error message more meaningful ## What changes were proposed in this pull request? For table test1 (C1 varchar (10), C2 varchar (10)), when I insert a row using ```

spark git commit: [SPARK-15868][WEB UI] Executors table in Executors tab should sort Executor IDs in numerical order

Repository: spark Updated Branches: refs/heads/branch-2.0 fb0fab63c -> 27e274c3e [SPARK-15868][WEB UI] Executors table in Executors tab should sort Executor IDs in numerical order ## What changes were proposed in this pull request? Currently the Executors table sorts by id using a string

spark git commit: [SPARK-15868][WEB UI] Executors table in Executors tab should sort Executor IDs in numerical order

Repository: spark Updated Branches: refs/heads/master 2d27eb1e7 -> e849285df [SPARK-15868][WEB UI] Executors table in Executors tab should sort Executor IDs in numerical order ## What changes were proposed in this pull request? Currently the Executors table sorts by id using a string sort

spark git commit: [MINOR][DOCS][SQL] Fix some comments about types(TypeCoercion, Partition) and exceptions.

Repository: spark Updated Branches: refs/heads/branch-2.0 7d8cddfb4 -> fb0fab63c [MINOR][DOCS][SQL] Fix some comments about types(TypeCoercion,Partition) and exceptions. ## What changes were proposed in this pull request? This PR contains a few changes on code comments. - `HiveTypeCoercion`

spark git commit: [MINOR][DOCS][SQL] Fix some comments about types(TypeCoercion, Partition) and exceptions.

Repository: spark Updated Branches: refs/heads/master 796429d71 -> 2d27eb1e7 [MINOR][DOCS][SQL] Fix some comments about types(TypeCoercion,Partition) and exceptions. ## What changes were proposed in this pull request? This PR contains a few changes on code comments. - `HiveTypeCoercion` is

spark git commit: [SPARK-15998][SQL] Verification of SQLConf HIVE_METASTORE_PARTITION_PRUNING

Repository: spark Updated Branches: refs/heads/branch-2.0 1230516d9 -> 7d8cddfb4 [SPARK-15998][SQL] Verification of SQLConf HIVE_METASTORE_PARTITION_PRUNING What changes were proposed in this pull request? `HIVE_METASTORE_PARTITION_PRUNING` is a public `SQLConf`. When `true`, some

spark git commit: [SPARK-15998][SQL] Verification of SQLConf HIVE_METASTORE_PARTITION_PRUNING

Repository: spark Updated Branches: refs/heads/master 7a89f2adb -> 796429d71 [SPARK-15998][SQL] Verification of SQLConf HIVE_METASTORE_PARTITION_PRUNING What changes were proposed in this pull request? `HIVE_METASTORE_PARTITION_PRUNING` is a public `SQLConf`. When `true`, some

spark git commit: [SQL] Minor HashAggregateExec string output fixes

Repository: spark Updated Branches: refs/heads/branch-2.0 938988757 -> 1230516d9 [SQL] Minor HashAggregateExec string output fixes ## What changes were proposed in this pull request? This PR fixes some minor `.toString` format issues for `HashAggregateExec`. Before: ```

spark git commit: [SQL] Minor HashAggregateExec string output fixes

Repository: spark Updated Branches: refs/heads/master acef843f6 -> 7a89f2adb [SQL] Minor HashAggregateExec string output fixes ## What changes were proposed in this pull request? This PR fixes some minor `.toString` format issues for `HashAggregateExec`. Before: ```

spark git commit: [SPARK-15975] Fix improper Popen retcode code handling in dev/run-tests

Repository: spark Updated Branches: refs/heads/branch-1.6 cffc0800b -> 0a8ada506 [SPARK-15975] Fix improper Popen retcode code handling in dev/run-tests In the `dev/run-tests.py` script we check a `Popen.retcode` for success using `retcode > 0`, but this is subtlety wrong because Popen's

spark git commit: [SPARK-15975] Fix improper Popen retcode code handling in dev/run-tests

Repository: spark Updated Branches: refs/heads/branch-2.0 d9dd46edd -> 938988757 [SPARK-15975] Fix improper Popen retcode code handling in dev/run-tests In the `dev/run-tests.py` script we check a `Popen.retcode` for success using `retcode > 0`, but this is subtlety wrong because Popen's

spark git commit: [SPARK-15975] Fix improper Popen retcode code handling in dev/run-tests

Repository: spark Updated Branches: refs/heads/branch-1.5 6043fa8df -> 1891e04a6 [SPARK-15975] Fix improper Popen retcode code handling in dev/run-tests In the `dev/run-tests.py` script we check a `Popen.retcode` for success using `retcode > 0`, but this is subtlety wrong because Popen's

spark git commit: [SPARK-15975] Fix improper Popen retcode code handling in dev/run-tests

Repository: spark Updated Branches: refs/heads/master bbad4cb48 -> acef843f6 [SPARK-15975] Fix improper Popen retcode code handling in dev/run-tests In the `dev/run-tests.py` script we check a `Popen.retcode` for success using `retcode > 0`, but this is subtlety wrong because Popen's return

spark git commit: [SPARK-15978][SQL] improve 'show tables' command related codes

Repository: spark Updated Branches: refs/heads/branch-2.0 095ddb4c9 -> d9dd46edd [SPARK-15978][SQL] improve 'show tables' command related codes ## What changes were proposed in this pull request? I've found some minor issues in "show tables" command: 1. In the `SessionCatalog.scala`,

spark git commit: [SPARK-15978][SQL] improve 'show tables' command related codes

Repository: spark Updated Branches: refs/heads/master 457126e42 -> bbad4cb48 [SPARK-15978][SQL] improve 'show tables' command related codes ## What changes were proposed in this pull request? I've found some minor issues in "show tables" command: 1. In the `SessionCatalog.scala`,

spark git commit: [SPARK-15736][CORE][BRANCH-1.6] Gracefully handle loss of DiskStore files

Repository: spark Updated Branches: refs/heads/branch-1.6 0a13e4c07 -> 4259a2858 [SPARK-15736][CORE][BRANCH-1.6] Gracefully handle loss of DiskStore files If an RDD partition is cached on disk and the DiskStore file is lost, then reads of that cached partition will fail and the missing

spark git commit: [SPARK-15715][SQL] Fix alter partition with storage information in Hive

ION (width=3) SET SERDE 'com.sparkbricks.serde.ColumnarSerDe' WITH SERDEPROPERTIES ('compress'='true') ``` ## How was this patch tested? `HiveExternalCatalogSuite` Author: Andrew Or <and...@databricks.com> Closes #13453 from andrewor14/alter-partition-storage. Project: http:

spark git commit: [SPARK-15715][SQL] Fix alter partition with storage information in Hive

ION (width=3) SET SERDE 'com.sparkbricks.serde.ColumnarSerDe' WITH SERDEPROPERTIES ('compress'='true') ``` ## How was this patch tested? `HiveExternalCatalogSuite` Author: Andrew Or <and...@databricks.com> Closes #13453 from andrewor14/alter-partition-storage. (cherry picked fr

spark git commit: [SPARK-15740][MLLIB] ignore big model load / save in Word2VecSuite

Repository: spark Updated Branches: refs/heads/branch-2.0 1bce96db5 -> 0c721eedc [SPARK-15740][MLLIB] ignore big model load / save in Word2VecSuite ## What changes were proposed in this pull request? andrewor14 noticed some OOM errors caused by "test big model load / save" in

spark git commit: [SPARK-15740][MLLIB] ignore big model load / save in Word2VecSuite

Repository: spark Updated Branches: refs/heads/master f34aadc54 -> e23370ec6 [SPARK-15740][MLLIB] ignore big model load / save in Word2VecSuite ## What changes were proposed in this pull request? andrewor14 noticed some OOM errors caused by "test big model load / save" in Word2

spark git commit: [SPARK-15718][SQL] better error message for writing bucketed data

Repository: spark Updated Branches: refs/heads/master 229f90225 -> f34aadc54 [SPARK-15718][SQL] better error message for writing bucketed data ## What changes were proposed in this pull request? Currently we don't support bucketing for `save` and `insertInto`. For `save`, we just write the

spark git commit: [SPARK-15718][SQL] better error message for writing bucketed data

Repository: spark Updated Branches: refs/heads/branch-2.0 1551a72cb -> 1bce96db5 [SPARK-15718][SQL] better error message for writing bucketed data ## What changes were proposed in this pull request? Currently we don't support bucketing for `save` and `insertInto`. For `save`, we just write

spark git commit: [SPARK-15736][CORE] Gracefully handle loss of DiskStore files

Repository: spark Updated Branches: refs/heads/master 5855e0057 -> 229f90225 [SPARK-15736][CORE] Gracefully handle loss of DiskStore files If an RDD partition is cached on disk and the DiskStore file is lost, then reads of that cached partition will fail and the missing partition is supposed

spark git commit: [SPARK-15736][CORE] Gracefully handle loss of DiskStore files

Repository: spark Updated Branches: refs/heads/branch-2.0 0802ff9f6 -> 1551a72cb [SPARK-15736][CORE] Gracefully handle loss of DiskStore files If an RDD partition is cached on disk and the DiskStore file is lost, then reads of that cached partition will fail and the missing partition is

spark git commit: [SPARK-15711][SQL] Ban CREATE TEMPORARY TABLE USING AS SELECT

Repository: spark Updated Branches: refs/heads/branch-2.0 32b025e94 -> a55454eb6 [SPARK-15711][SQL] Ban CREATE TEMPORARY TABLE USING AS SELECT ## What changes were proposed in this pull request? This PR bans syntax like `CREATE TEMPORARY TABLE USING AS SELECT` `CREATE TEMPORARY TABLE ...

spark git commit: [SPARK-15711][SQL] Ban CREATE TEMPORARY TABLE USING AS SELECT

Repository: spark Updated Branches: refs/heads/master 9aff6f3b1 -> d109a1bee [SPARK-15711][SQL] Ban CREATE TEMPORARY TABLE USING AS SELECT ## What changes were proposed in this pull request? This PR bans syntax like `CREATE TEMPORARY TABLE USING AS SELECT` `CREATE TEMPORARY TABLE ... USING

spark git commit: [SPARK-15646][SQL] When spark.sql.hive.convertCTAS is true, the conversion rule needs to respect TEXTFILE/SEQUENCEFILE format and the user-defined location

Repository: spark Updated Branches: refs/heads/branch-2.0 35195f6ce -> 5a835b99f [SPARK-15646][SQL] When spark.sql.hive.convertCTAS is true, the conversion rule needs to respect TEXTFILE/SEQUENCEFILE format and the user-defined location ## What changes were proposed in this pull request?

spark git commit: [SPARK-15646][SQL] When spark.sql.hive.convertCTAS is true, the conversion rule needs to respect TEXTFILE/SEQUENCEFILE format and the user-defined location

Repository: spark Updated Branches: refs/heads/master c8fb776d4 -> 6dddb70c3 [SPARK-15646][SQL] When spark.sql.hive.convertCTAS is true, the conversion rule needs to respect TEXTFILE/SEQUENCEFILE format and the user-defined location ## What changes were proposed in this pull request? When

spark git commit: [HOTFIX] DDLSuite was broken by 93e9714

Repository: spark Updated Branches: refs/heads/master ac38bdc75 -> 1dd925644 [HOTFIX] DDLSuite was broken by 93e9714 Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/1dd92564 Tree:

spark git commit: [HOTFIX] DDLSuite was broken by 93e9714

Repository: spark Updated Branches: refs/heads/branch-2.0 d34c0fc10 -> 0ade44cc4 [HOTFIX] DDLSuite was broken by 93e9714 Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/0ade44cc Tree:

spark git commit: [SPARK-15236][SQL][SPARK SHELL] Add spark-defaults property to switch to use InMemoryCatalog

Repository: spark Updated Branches: refs/heads/branch-2.0 459fd34a0 -> db96f398b [SPARK-15236][SQL][SPARK SHELL] Add spark-defaults property to switch to use InMemoryCatalog ## What changes were proposed in this pull request? This PR change REPL/Main to check this property

spark git commit: [SPARK-15236][SQL][SPARK SHELL] Add spark-defaults property to switch to use InMemoryCatalog

Repository: spark Updated Branches: refs/heads/master 85d6b0db9 -> 04f925ede [SPARK-15236][SQL][SPARK SHELL] Add spark-defaults property to switch to use InMemoryCatalog ## What changes were proposed in this pull request? This PR change REPL/Main to check this property

spark git commit: [SPARK-15618][SQL][MLLIB] Use SparkSession.builder.sparkContext if applicable.

Repository: spark Updated Branches: refs/heads/master 93e97147e -> 85d6b0db9 [SPARK-15618][SQL][MLLIB] Use SparkSession.builder.sparkContext if applicable. ## What changes were proposed in this pull request? This PR changes function `SparkSession.builder.sparkContext(..)` from

spark git commit: [MINOR] Slightly better error message when attempting to query hive tables w/in-mem catalog

Repository: spark Updated Branches: refs/heads/branch-2.0 4b19c9776 -> ac4cb1718 [MINOR] Slightly better error message when attempting to query hive tables w/in-mem catalog andrewor14 Author: Eric Liang <e...@databricks.com> Closes #13427 from ericl/better-error-msg. (cherry pi

spark git commit: [MINOR] Slightly better error message when attempting to query hive tables w/in-mem catalog

Repository: spark Updated Branches: refs/heads/master 196a0d827 -> 93e97147e [MINOR] Slightly better error message when attempting to query hive tables w/in-mem catalog andrewor14 Author: Eric Liang <e...@databricks.com> Closes #13427 from ericl/better-error-msg. Project: http:

spark git commit: [MINOR][SQL][DOCS] Fix docs of Dataset.scala and SQLImplicits.scala.

Repository: spark Updated Branches: refs/heads/master dad5a6881 -> 196a0d827 [MINOR][SQL][DOCS] Fix docs of Dataset.scala and SQLImplicits.scala. This PR fixes a sample code, a description, and indentations in docs. Manual. Author: Dongjoon Hyun Closes #13420 from

spark git commit: [SPARK-15670][JAVA API][SPARK CORE] label_accumulator_deprecate_in_java_spark_context

Repository: spark Updated Branches: refs/heads/branch-2.0 00fca2a05 -> 7f240eaee [SPARK-15670][JAVA API][SPARK CORE] label_accumulator_deprecate_in_java_spark_context ## What changes were proposed in this pull request? Add deprecate annotation for acumulator V1 interface in JavaSparkContext

spark git commit: [SPARK-15670][JAVA API][SPARK CORE] label_accumulator_deprecate_in_java_spark_context

Repository: spark Updated Branches: refs/heads/master 06514d689 -> dad5a6881 [SPARK-15670][JAVA API][SPARK CORE] label_accumulator_deprecate_in_java_spark_context ## What changes were proposed in this pull request? Add deprecate annotation for acumulator V1 interface in JavaSparkContext

spark git commit: [CORE][DOC][MINOR] typos + links

Repository: spark Updated Branches: refs/heads/branch-2.0 f0e8738c1 -> 00fca2a05 [CORE][DOC][MINOR] typos + links ## What changes were proposed in this pull request? A very tiny change to javadoc (which I don't mind if gets merged with a bigger change). I've just found it annoying and

spark git commit: [CORE][DOC][MINOR] typos + links

Repository: spark Updated Branches: refs/heads/master 8ca01a6fe -> 0f2471346 [CORE][DOC][MINOR] typos + links ## What changes were proposed in this pull request? A very tiny change to javadoc (which I don't mind if gets merged with a bigger change). I've just found it annoying and couldn't

spark git commit: [SPARK-15662][SQL] Add since annotation for classes in sql.catalog

Repository: spark Updated Branches: refs/heads/branch-2.0 1f4c4def9 -> 978f54e76 [SPARK-15662][SQL] Add since annotation for classes in sql.catalog ## What changes were proposed in this pull request? This patch does a few things: 1. Adds since version annotation to methods and classes in

spark git commit: [SPARK-15662][SQL] Add since annotation for classes in sql.catalog

Repository: spark Updated Branches: refs/heads/master 695470429 -> 223f1d58c [SPARK-15662][SQL] Add since annotation for classes in sql.catalog ## What changes were proposed in this pull request? This patch does a few things: 1. Adds since version annotation to methods and classes in

spark git commit: [HOTFIX] Scala 2.10 compile GaussianMixtureModel

Repository: spark Updated Branches: refs/heads/branch-2.0 17f43cc87 -> 5ea58898c [HOTFIX] Scala 2.10 compile GaussianMixtureModel Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/5ea58898 Tree:

spark git commit: [HOTFIX] Scala 2.10 compile GaussianMixtureModel

Repository: spark Updated Branches: refs/heads/master 1b98fa2e4 -> b376a4eab [HOTFIX] Scala 2.10 compile GaussianMixtureModel Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/b376a4ea Tree:

spark git commit: [SPARK-15569] Reduce frequency of updateBytesWritten function in Disk…

Repository: spark Updated Branches: refs/heads/master 5bdbedf22 -> ce756daa4 [SPARK-15569] Reduce frequency of updateBytesWritten function in Disk… ## What changes were proposed in this pull request? Profiling a Spark job spilling large amount of intermediate data we found that

spark git commit: [SPARK-15569] Reduce frequency of updateBytesWritten function in Disk…

Repository: spark Updated Branches: refs/heads/branch-2.0 89fdb6972 -> 30e87b55b [SPARK-15569] Reduce frequency of updateBytesWritten function in Disk… ## What changes were proposed in this pull request? Profiling a Spark job spilling large amount of intermediate data we found that

spark git commit: [MINOR][DOCS] Typo fixes in Dataset scaladoc

xin andrewor14 ## How was this patch tested? Compiled locally Author: Xinh Huynh <xinh_hu...@yahoo.com> Closes #13330 from xinhhuynh/fix-dataset-typos. (cherry picked from commit 5bdbedf2201efa6c34392aa9eff709761f027e1d) Signed-off-by: Andrew Or <and...@databricks.com> Project:

spark git commit: [MINOR][DOCS] Typo fixes in Dataset scaladoc

xin andrewor14 ## How was this patch tested? Compiled locally Author: Xinh Huynh <xinh_hu...@yahoo.com> Closes #13330 from xinhhuynh/fix-dataset-typos. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/5bdbedf2 Tree: http:

spark git commit: [SPARK-15597][SQL] Add SparkSession.emptyDataset

Repository: spark Updated Branches: refs/heads/branch-2.0 e69639f43 -> f52a95248 [SPARK-15597][SQL] Add SparkSession.emptyDataset ## What changes were proposed in this pull request? This patch adds a new function emptyDataset to SparkSession, for creating an empty dataset. ## How was this

spark git commit: [SPARK-15597][SQL] Add SparkSession.emptyDataset

Repository: spark Updated Branches: refs/heads/master 635fb30f8 -> a52e68133 [SPARK-15597][SQL] Add SparkSession.emptyDataset ## What changes were proposed in this pull request? This patch adds a new function emptyDataset to SparkSession, for creating an empty dataset. ## How was this patch

spark git commit: [SPARK-15599][SQL][DOCS] API docs for `createDataset` functions in SparkSession

Repository: spark Updated Branches: refs/heads/branch-2.0 a14c88acc -> e69639f43 [SPARK-15599][SQL][DOCS] API docs for `createDataset` functions in SparkSession ## What changes were proposed in this pull request? Adds API docs and usage examples for the 3 `createDataset` calls in

spark git commit: [SPARK-15599][SQL][DOCS] API docs for `createDataset` functions in SparkSession

Repository: spark Updated Branches: refs/heads/master 4538443e2 -> 635fb30f8 [SPARK-15599][SQL][DOCS] API docs for `createDataset` functions in SparkSession ## What changes were proposed in this pull request? Adds API docs and usage examples for the 3 `createDataset` calls in `SparkSession`

spark git commit: [SPARK-15584][SQL] Abstract duplicate code: `spark.sql.sources.` properties

Repository: spark Updated Branches: refs/heads/master d24e25157 -> 4538443e2 [SPARK-15584][SQL] Abstract duplicate code: `spark.sql.sources.` properties ## What changes were proposed in this pull request? This PR replaces `spark.sql.sources.` strings with `CreateDataSourceTableUtils.*`

spark git commit: [SPARK-15584][SQL] Abstract duplicate code: `spark.sql.sources.` properties

Repository: spark Updated Branches: refs/heads/branch-2.0 a355edeef -> a14c88acc [SPARK-15584][SQL] Abstract duplicate code: `spark.sql.sources.` properties ## What changes were proposed in this pull request? This PR replaces `spark.sql.sources.` strings with `CreateDataSourceTableUtils.*`

spark git commit: [SPARK-15603][MLLIB] Replace SQLContext with SparkSession in ML/MLLib

Repository: spark Updated Branches: refs/heads/branch-2.0 2cb84dd23 -> a355edeef [SPARK-15603][MLLIB] Replace SQLContext with SparkSession in ML/MLLib ## What changes were proposed in this pull request? This PR replaces all deprecated `SQLContext` occurrences with `SparkSession` in

spark git commit: [SPARK-15603][MLLIB] Replace SQLContext with SparkSession in ML/MLLib

Repository: spark Updated Branches: refs/heads/master c17272902 -> d24e25157 [SPARK-15603][MLLIB] Replace SQLContext with SparkSession in ML/MLLib ## What changes were proposed in this pull request? This PR replaces all deprecated `SQLContext` occurrences with `SparkSession` in `ML/MLLib`

spark git commit: [SPARK-15538][SPARK-15539][SQL] Truncate table fixes round 2

out `PARTITION`) (2) Disallow truncating external tables or views ## How was this patch tested? `DDLSuite` Author: Andrew Or <and...@databricks.com> Closes #13315 from andrewor14/truncate-table. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/

spark git commit: [SPARK-15538][SPARK-15539][SQL] Truncate table fixes round 2

ION`) (2) Disallow truncating external tables or views ## How was this patch tested? `DDLSuite` Author: Andrew Or <and...@databricks.com> Closes #13315 from andrewor14/truncate-table. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/

spark git commit: [SPARK-15552][SQL] Remove unnecessary private[sql] methods in SparkSession

Repository: spark Updated Branches: refs/heads/master 594a1bf20 -> 0f61d6efb [SPARK-15552][SQL] Remove unnecessary private[sql] methods in SparkSession ## What changes were proposed in this pull request? SparkSession has a list of unnecessary private[sql] methods. These methods cause some

spark git commit: [SPARK-15520][SQL] Also set sparkContext confs when using SparkSession builder in pyspark

Repository: spark Updated Branches: refs/heads/branch-2.0 69b3e9cee -> eb0c49799 [SPARK-15520][SQL] Also set sparkContext confs when using SparkSession builder in pyspark ## What changes were proposed in this pull request? Also sets confs in the underlying sc when using

spark git commit: [SPARK-15520][SQL] Also set sparkContext confs when using SparkSession builder in pyspark

Repository: spark Updated Branches: refs/heads/master 2b1ac6cea -> 594a1bf20 [SPARK-15520][SQL] Also set sparkContext confs when using SparkSession builder in pyspark ## What changes were proposed in this pull request? Also sets confs in the underlying sc when using

spark git commit: [SPARK-15539][SQL] DROP TABLE throw exception if table doesn't exist

hor: Andrew Or <and...@databricks.com> Closes #13307 from andrewor14/drop-table. (cherry picked from commit 2b1ac6cea882246ef0e655bb2c134ef1656a5068) Signed-off-by: Andrew Or <and...@databricks.com> Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.a

spark git commit: [SPARK-15539][SQL] DROP TABLE throw exception if table doesn't exist

rew Or <and...@databricks.com> Closes #13307 from andrewor14/drop-table. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/2b1ac6ce Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/2b1ac6ce Diff: http://git-wip-us.a

spark git commit: [SPARK-15534][SPARK-15535][SQL] Truncate table fixes

ons. - Remove `TRUNCATE TABLE ... COLUMN`, which was never supported by either Spark or Hive. ## How was this patch tested? Jenkins. Author: Andrew Or <and...@databricks.com> Closes #13302 from andrewor14/truncate-table. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: h

spark git commit: [SPARK-15534][SPARK-15535][SQL] Truncate table fixes

row exceptions. - Remove `TRUNCATE TABLE ... COLUMN`, which was never supported by either Spark or Hive. ## How was this patch tested? Jenkins. Author: Andrew Or <and...@databricks.com> Closes #13302 from andrewor14/truncate-table. (cherry picked from commit ee682fe293b47988056b540ee46ca4986130998

spark git commit: [MINOR][MLLIB][STREAMING][SQL] Fix typos

Repository: spark Updated Branches: refs/heads/master d6d3e5071 -> 02c8072ee [MINOR][MLLIB][STREAMING][SQL] Fix typos fixed typos for source code for components [mllib] [streaming] and [SQL] None and obvious. Author: lfzCarlosC Closes #13298 from lfzCarlosC/master.

spark git commit: [MINOR][MLLIB][STREAMING][SQL] Fix typos

Repository: spark Updated Branches: refs/heads/branch-2.0 4009ddafd -> 6fc367e50 [MINOR][MLLIB][STREAMING][SQL] Fix typos fixed typos for source code for components [mllib] [streaming] and [SQL] None and obvious. Author: lfzCarlosC Closes #13298 from

spark git commit: [MINOR][CORE] Fix a HadoopRDD log message and remove unused imports in rdd files.

Repository: spark Updated Branches: refs/heads/branch-2.0 c75ec5eaa -> 4009ddafd [MINOR][CORE] Fix a HadoopRDD log message and remove unused imports in rdd files. ## What changes were proposed in this pull request? This PR fixes the following typos in log message and comments of

spark git commit: [MINOR][CORE] Fix a HadoopRDD log message and remove unused imports in rdd files.

Repository: spark Updated Branches: refs/heads/master 8239fdcb9 -> d6d3e5071 [MINOR][CORE] Fix a HadoopRDD log message and remove unused imports in rdd files. ## What changes were proposed in this pull request? This PR fixes the following typos in log message and comments of

spark git commit: [SPARK-15520][SQL] SparkSession builder in python should also allow overriding confs of existing sessions

low setting confs correctly. This was a leftover TODO from https://github.com/apache/spark/pull/13200. ## How was this patch tested? Python doc tests. cc andrewor14 Author: Eric Liang <e...@databricks.com> Closes #13289 from ericl/spark-15520. Project: http://git-wip-us.apache.org/repos/

spark git commit: [SPARK-15520][SQL] SparkSession builder in python should also allow overriding confs of existing sessions

low setting confs correctly. This was a leftover TODO from https://github.com/apache/spark/pull/13200. ## How was this patch tested? Python doc tests. cc andrewor14 Author: Eric Liang <e...@databricks.com> Closes #13289 from ericl/spark-15520. (cherry picked fr

  1   2   3   4   5   6   7   8   9   10   >