spark git commit: [SPARK-3365][SQL]Wrong schema generated for List type

Repository: spark Updated Branches: refs/heads/master 2aea892eb -> 1c8633f3f [SPARK-3365][SQL]Wrong schema generated for List type This PR fix the issue SPARK-3365. The reason is Spark generated wrong schema for the type `List` in `ScalaReflection.scala` for example: the generated schema for

spark git commit: [SQL] Fix docs of SQLContext.tables

Repository: spark Updated Branches: refs/heads/branch-1.3 b9f332ab6 -> a8f560c4e [SQL] Fix docs of SQLContext.tables Author: Yin Huai Closes #4579 from yhuai/tablesDoc and squashes the following commits: 7f8964c [Yin Huai] Fix doc. (cherry picked from commit 2aea892ebd4d6c802defeef35ef7ebf

spark git commit: [SPARK-3365][SQL]Wrong schema generated for List type

Repository: spark Updated Branches: refs/heads/branch-1.3 edbac178d -> b9f332ab6 [SPARK-3365][SQL]Wrong schema generated for List type This PR fix the issue SPARK-3365. The reason is Spark generated wrong schema for the type `List` in `ScalaReflection.scala` for example: the generated schema

spark git commit: [SQL] Fix docs of SQLContext.tables

Repository: spark Updated Branches: refs/heads/master 1d0596a16 -> 2aea892eb [SQL] Fix docs of SQLContext.tables Author: Yin Huai Closes #4579 from yhuai/tablesDoc and squashes the following commits: 7f8964c [Yin Huai] Fix doc. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Co

spark git commit: [SPARK-3299][SQL]Public API in SQLContext to list tables

Repository: spark Updated Branches: refs/heads/branch-1.3 925fd84a1 -> edbac178d [SPARK-3299][SQL]Public API in SQLContext to list tables https://issues.apache.org/jira/browse/SPARK-3299 Author: Yin Huai Closes #4547 from yhuai/tables and squashes the following commits: 6c8f92e [Yin Huai]

spark git commit: [SPARK-3299][SQL]Public API in SQLContext to list tables

Repository: spark Updated Branches: refs/heads/master c025a4688 -> 1d0596a16 [SPARK-3299][SQL]Public API in SQLContext to list tables https://issues.apache.org/jira/browse/SPARK-3299 Author: Yin Huai Closes #4547 from yhuai/tables and squashes the following commits: 6c8f92e [Yin Huai] Add

spark git commit: Revert "[SPARK-5762] Fix shuffle write time for sort-based shuffle"

Repository: spark Updated Branches: refs/heads/branch-1.2 d24971a62 -> 0ba065f0a Revert "[SPARK-5762] Fix shuffle write time for sort-based shuffle" This reverts commit 9c5454d06e56917521a15697c36f76a33a94dd1e. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip

spark git commit: [SQL] Move SaveMode to SQL package.

Repository: spark Updated Branches: refs/heads/master ada993e95 -> c025a4688 [SQL] Move SaveMode to SQL package. Author: Yin Huai Closes #4542 from yhuai/moveSaveMode and squashes the following commits: 65a4425 [Yin Huai] Move SaveMode to sql package. Project: http://git-wip-us.apache.org

spark git commit: [SQL] Move SaveMode to SQL package.

Repository: spark Updated Branches: refs/heads/branch-1.3 5c9db4e75 -> 925fd84a1 [SQL] Move SaveMode to SQL package. Author: Yin Huai Closes #4542 from yhuai/moveSaveMode and squashes the following commits: 65a4425 [Yin Huai] Move SaveMode to sql package. (cherry picked from commit c025a46

spark git commit: [SPARK-5335] Fix deletion of security groups within a VPC

Repository: spark Updated Branches: refs/heads/branch-1.2 c7bac577a -> d24971a62 [SPARK-5335] Fix deletion of security groups within a VPC Please see https://issues.apache.org/jira/browse/SPARK-5335. The fix itself is in e58a8b01a8bedcbfbbc6d04b1c1489255865cf87 commit. Two earlier commits ar

spark git commit: [SPARK-5335] Fix deletion of security groups within a VPC

Repository: spark Updated Branches: refs/heads/branch-1.3 f7103b343 -> 5c9db4e75 [SPARK-5335] Fix deletion of security groups within a VPC Please see https://issues.apache.org/jira/browse/SPARK-5335. The fix itself is in e58a8b01a8bedcbfbbc6d04b1c1489255865cf87 commit. Two earlier commits ar

spark git commit: [SPARK-5335] Fix deletion of security groups within a VPC

Repository: spark Updated Branches: refs/heads/master d5fc51491 -> ada993e95 [SPARK-5335] Fix deletion of security groups within a VPC Please see https://issues.apache.org/jira/browse/SPARK-5335. The fix itself is in e58a8b01a8bedcbfbbc6d04b1c1489255865cf87 commit. Two earlier commits are fi

spark git commit: [SPARK-5755] [SQL] remove unnecessary Add

Repository: spark Updated Branches: refs/heads/branch-1.3 c7eb9ee2c -> f7103b343 [SPARK-5755] [SQL] remove unnecessary Add explain extended select +key from src; before: == Parsed Logical Plan == 'Project [(0 + 'key) AS _c0#8] 'UnresolvedRelation [src], None == Analyzed Logical Plan == P

spark git commit: [SPARK-5755] [SQL] remove unnecessary Add

Repository: spark Updated Branches: refs/heads/master ee04a8b19 -> d5fc51491 [SPARK-5755] [SQL] remove unnecessary Add explain extended select +key from src; before: == Parsed Logical Plan == 'Project [(0 + 'key) AS _c0#8] 'UnresolvedRelation [src], None == Analyzed Logical Plan == Proje

spark git commit: [SPARK-5573][SQL] Add explode to dataframes

Repository: spark Updated Branches: refs/heads/master c352ffbdb -> ee04a8b19 [SPARK-5573][SQL] Add explode to dataframes Author: Michael Armbrust Closes #4546 from marmbrus/explode and squashes the following commits: eefd33a [Michael Armbrust] whitespace a8d496c [Michael Armbrust] Merge rem

spark git commit: [SPARK-5573][SQL] Add explode to dataframes

Repository: spark Updated Branches: refs/heads/branch-1.3 b0c79daf4 -> c7eb9ee2c [SPARK-5573][SQL] Add explode to dataframes Author: Michael Armbrust Closes #4546 from marmbrus/explode and squashes the following commits: eefd33a [Michael Armbrust] whitespace a8d496c [Michael Armbrust] Merge

spark git commit: [SPARK-5758][SQL] Use LongType as the default type for integers in JSON schema inference.

Repository: spark Updated Branches: refs/heads/branch-1.3 bf0d15c52 -> b0c79daf4 [SPARK-5758][SQL] Use LongType as the default type for integers in JSON schema inference. Author: Yin Huai Closes #4544 from yhuai/jsonUseLongTypeByDefault and squashes the following commits: 6e2ffc2 [Yin Hua

spark git commit: [SPARK-5758][SQL] Use LongType as the default type for integers in JSON schema inference.

Repository: spark Updated Branches: refs/heads/master 0bf031582 -> c352ffbdb [SPARK-5758][SQL] Use LongType as the default type for integers in JSON schema inference. Author: Yin Huai Closes #4544 from yhuai/jsonUseLongTypeByDefault and squashes the following commits: 6e2ffc2 [Yin Huai] U

spark git commit: [SPARK-5780] [PySpark] Mute the logging during unit tests

Repository: spark Updated Branches: refs/heads/branch-1.2 9c5454d06 -> c7bac577a [SPARK-5780] [PySpark] Mute the logging during unit tests There a bunch of logging coming from driver and worker, it's noisy and scaring, and a lots of exception in it, people are confusing about the tests are fa

spark git commit: [SPARK-5780] [PySpark] Mute the logging during unit tests

Repository: spark Updated Branches: refs/heads/master 26c816e73 -> 0bf031582 [SPARK-5780] [PySpark] Mute the logging during unit tests There a bunch of logging coming from driver and worker, it's noisy and scaring, and a lots of exception in it, people are confusing about the tests are failin

spark git commit: [SPARK-5780] [PySpark] Mute the logging during unit tests

Repository: spark Updated Branches: refs/heads/branch-1.3 11a0d5b6d -> bf0d15c52 [SPARK-5780] [PySpark] Mute the logging during unit tests There a bunch of logging coming from driver and worker, it's noisy and scaring, and a lots of exception in it, people are confusing about the tests are fa

spark git commit: SPARK-5747: Fix wordsplitting bugs in make-distribution.sh

Repository: spark Updated Branches: refs/heads/branch-1.3 02d5b32bb -> 11a0d5b6d SPARK-5747: Fix wordsplitting bugs in make-distribution.sh The `$MVN` command variable may have spaces, so when referring to it, must wrap in quotes. Author: David Y. Ross Closes #4540 from dyross/dyr-fix-make

spark git commit: SPARK-5747: Fix wordsplitting bugs in make-distribution.sh

Repository: spark Updated Branches: refs/heads/master 947b8bd82 -> 26c816e73 SPARK-5747: Fix wordsplitting bugs in make-distribution.sh The `$MVN` command variable may have spaces, so when referring to it, must wrap in quotes. Author: David Y. Ross Closes #4540 from dyross/dyr-fix-make-dis

spark git commit: [SPARK-5759][Yarn]ExecutorRunnable should catch YarnException while NMClient start contain...

Repository: spark Updated Branches: refs/heads/master 1d5663e92 -> 947b8bd82 [SPARK-5759][Yarn]ExecutorRunnable should catch YarnException while NMClient start contain... some time since some reasons, it lead to some exception while NMClient start some containers.example:we do not config spa

spark git commit: [SPARK-5759][Yarn]ExecutorRunnable should catch YarnException while NMClient start contain...

Repository: spark Updated Branches: refs/heads/branch-1.3 11d108030 -> 02d5b32bb [SPARK-5759][Yarn]ExecutorRunnable should catch YarnException while NMClient start contain... some time since some reasons, it lead to some exception while NMClient start some containers.example:we do not config

spark git commit: [SPARK-5760][SPARK-5761] Fix standalone rest protocol corner cases + revamp tests

Repository: spark Updated Branches: refs/heads/master 47c73d410 -> 1d5663e92 [SPARK-5760][SPARK-5761] Fix standalone rest protocol corner cases + revamp tests The changes are summarized in the commit message. Test or test-related code accounts for 90% of the lines changed. Author: Andrew Or

spark git commit: [SPARK-5760][SPARK-5761] Fix standalone rest protocol corner cases + revamp tests

Repository: spark Updated Branches: refs/heads/branch-1.3 0040fc509 -> 11d108030 [SPARK-5760][SPARK-5761] Fix standalone rest protocol corner cases + revamp tests The changes are summarized in the commit message. Test or test-related code accounts for 90% of the lines changed. Author: Andre

spark git commit: [SPARK-5762] Fix shuffle write time for sort-based shuffle

Repository: spark Updated Branches: refs/heads/branch-1.2 b78a686eb -> 9c5454d06 [SPARK-5762] Fix shuffle write time for sort-based shuffle mateiz was excluding the time to write this final file from the shuffle write time intentional? Author: Kay Ousterhout Closes #4559 from kayousterhout

spark git commit: [SPARK-5762] Fix shuffle write time for sort-based shuffle

Repository: spark Updated Branches: refs/heads/master 629d0143e -> 47c73d410 [SPARK-5762] Fix shuffle write time for sort-based shuffle mateiz was excluding the time to write this final file from the shuffle write time intentional? Author: Kay Ousterhout Closes #4559 from kayousterhout/SPA

spark git commit: [SPARK-5762] Fix shuffle write time for sort-based shuffle

Repository: spark Updated Branches: refs/heads/branch-1.3 9a1de4b20 -> 0040fc509 [SPARK-5762] Fix shuffle write time for sort-based shuffle mateiz was excluding the time to write this final file from the shuffle write time intentional? Author: Kay Ousterhout Closes #4559 from kayousterhout

spark git commit: [SPARK-5765][Examples]Fixed word split problem in run-example and compute-classpath

Repository: spark Updated Branches: refs/heads/master 9c8076502 -> 629d0143e [SPARK-5765][Examples]Fixed word split problem in run-example and compute-classpath Author: Venkata Ramana G Author: Venkata Ramana Gollamudi Closes #4561 from gvramana/word_split and squashes the following commi

spark git commit: [SPARK-5765][Examples]Fixed word split problem in run-example and compute-classpath

Repository: spark Updated Branches: refs/heads/branch-1.2 64254eeec -> b78a686eb [SPARK-5765][Examples]Fixed word split problem in run-example and compute-classpath Author: Venkata Ramana G Author: Venkata Ramana Gollamudi Closes #4561 from gvramana/word_split and squashes the following c

spark git commit: [SPARK-5765][Examples]Fixed word split problem in run-example and compute-classpath

Repository: spark Updated Branches: refs/heads/branch-1.3 74f34bb8b -> 9a1de4b20 [SPARK-5765][Examples]Fixed word split problem in run-example and compute-classpath Author: Venkata Ramana G Author: Venkata Ramana Gollamudi Closes #4561 from gvramana/word_split and squashes the following c

spark git commit: [EC2] Update default Spark version to 1.2.1

Repository: spark Updated Branches: refs/heads/master 893d6fd70 -> 9c8076502 [EC2] Update default Spark version to 1.2.1 Author: Katsunori Kanda Closes #4566 from potix2/ec2-update-version-1-2-1 and squashes the following commits: 77e7840 [Katsunori Kanda] [EC2] Update default Spark versio

spark git commit: [SPARK-5645] Added local read bytes/time to task metrics

Repository: spark Updated Branches: refs/heads/branch-1.3 e3a975d45 -> 74f34bb8b [SPARK-5645] Added local read bytes/time to task metrics ksakellis I stumbled on your JIRA for this yesterday; I know it's assigned to you but I'd already done this for my own uses a while ago so thought I could

spark git commit: [SPARK-5645] Added local read bytes/time to task metrics

Repository: spark Updated Branches: refs/heads/master aa4ca8b87 -> 893d6fd70 [SPARK-5645] Added local read bytes/time to task metrics ksakellis I stumbled on your JIRA for this yesterday; I know it's assigned to you but I'd already done this for my own uses a while ago so thought I could hel

spark git commit: [SQL] Improve error messages

Repository: spark Updated Branches: refs/heads/branch-1.3 cbd659e5f -> e3a975d45 [SQL] Improve error messages Author: Michael Armbrust Author: wangfei Closes #4558 from marmbrus/errorMessages and squashes the following commits: 5e5ab50 [Michael Armbrust] Merge pull request #15 from scwf/er

spark git commit: [SQL] Improve error messages

Repository: spark Updated Branches: refs/heads/master 6a1be026c -> aa4ca8b87 [SQL] Improve error messages Author: Michael Armbrust Author: wangfei Closes #4558 from marmbrus/errorMessages and squashes the following commits: 5e5ab50 [Michael Armbrust] Merge pull request #15 from scwf/errorM

spark git commit: [SQL][DOCS] Update sql documentation

Repository: spark Updated Branches: refs/heads/master bc57789bb -> 6a1be026c [SQL][DOCS] Update sql documentation Updated examples using the new api and added DataFrame concept Author: Antonio Navarro Perez Closes #4560 from ajnavarro/ajnavarro-doc-sql-update and squashes the following com

spark git commit: [SQL][DOCS] Update sql documentation

Repository: spark Updated Branches: refs/heads/branch-1.3 e26c14990 -> cbd659e5f [SQL][DOCS] Update sql documentation Updated examples using the new api and added DataFrame concept Author: Antonio Navarro Perez Closes #4560 from ajnavarro/ajnavarro-doc-sql-update and squashes the following

spark git commit: SPARK-5776 JIRA version not of form x.y.z breaks merge_spark_pr.py

Repository: spark Updated Branches: refs/heads/master 99bd50066 -> bc57789bb SPARK-5776 JIRA version not of form x.y.z breaks merge_spark_pr.py Consider only x.y.z verisons from JIRA. CC JoshRosen who will probably know this script well. Alternative is to call the version "2.0.0" after all in

spark git commit: [SPARK-5655] Don't chmod700 application files if running in YARN

Repository: spark Updated Branches: refs/heads/branch-1.2 53de2378e -> 64254eeec [SPARK-5655] Don't chmod700 application files if running in YARN [Was previously PR4507] As per SPARK-5655, recently committed code chmod 700s all application files created on the local fs by a spark executor. T

spark git commit: [SPARK-5757][MLLIB] replace SQL JSON usage in model import/export by json4s

Repository: spark Updated Branches: refs/heads/master 466b1f671 -> 99bd50066 [SPARK-5757][MLLIB] replace SQL JSON usage in model import/export by json4s This PR detaches MLlib model import/export code from SQL's JSON support, and hence unblocks #4544 . yhuai Author: Xiangrui Meng Closes #4

spark git commit: [SPARK-5757][MLLIB] replace SQL JSON usage in model import/export by json4s

Repository: spark Updated Branches: refs/heads/branch-1.3 e23c8f5c8 -> e26c14990 [SPARK-5757][MLLIB] replace SQL JSON usage in model import/export by json4s This PR detaches MLlib model import/export code from SQL's JSON support, and hence unblocks #4544 . yhuai Author: Xiangrui Meng Close

spark git commit: [SPARK-5655] Don't chmod700 application files if running in YARN

Repository: spark Updated Branches: refs/heads/branch-1.3 3c1b9bf65 -> e23c8f5c8 [SPARK-5655] Don't chmod700 application files if running in YARN [Was previously PR4507] As per SPARK-5655, recently committed code chmod 700s all application files created on the local fs by a spark executor. T

spark git commit: [SPARK-5655] Don't chmod700 application files if running in YARN

Repository: spark Updated Branches: refs/heads/master 9a6efbccf -> 466b1f671 [SPARK-5655] Don't chmod700 application files if running in YARN [Was previously PR4507] As per SPARK-5655, recently committed code chmod 700s all application files created on the local fs by a spark executor. This

spark git commit: ignore cache paths for RAT tests

Repository: spark Updated Branches: refs/heads/master 9a3ea49f7 -> 9a6efbccf ignore cache paths for RAT tests RAT fails on cache paths. add to .rat-excludes Author: Oren Mazor Closes #4569 from orenmazor/apache_master and squashes the following commits: d0c9e7e [Oren Mazor] ignore cache pa

spark git commit: SPARK-5727 [BUILD] Remove Debian packaging

Repository: spark Updated Branches: refs/heads/master a38e23c30 -> 9a3ea49f7 SPARK-5727 [BUILD] Remove Debian packaging (for master / 1.4 only) Author: Sean Owen Closes #4526 from srowen/SPARK-5727.2 and squashes the following commits: 83ba49c [Sean Owen] Remove Debian packaging Project: