spark git commit: [SPARK-6538][SQL] Add missing nullable Metastore fields when merging a Parquet schema

2015-03-27 Thread lian
Repository: spark Updated Branches: refs/heads/branch-1.3 70068588c -> 5e04f4518 [SPARK-6538][SQL] Add missing nullable Metastore fields when merging a Parquet schema Opening to replace #5188. When Spark SQL infers a schema for a DataFrame, it will take the union of all field types present

spark git commit: [SPARK-6538][SQL] Add missing nullable Metastore fields when merging a Parquet schema

2015-03-27 Thread lian
Repository: spark Updated Branches: refs/heads/master 3af733430 -> 5909f0973 [SPARK-6538][SQL] Add missing nullable Metastore fields when merging a Parquet schema Opening to replace #5188. When Spark SQL infers a schema for a DataFrame, it will take the union of all field types present in t

spark git commit: [SPARK-6564][SQL] SQLContext.emptyDataFrame should contain 0 row, not 1 row

2015-03-27 Thread rxin
Repository: spark Updated Branches: refs/heads/branch-1.3 fefd49f4b -> 70068588c [SPARK-6564][SQL] SQLContext.emptyDataFrame should contain 0 row, not 1 row Author: Reynold Xin Closes #5226 from rxin/empty-df and squashes the following commits: 1306d88 [Reynold Xin] Proper fix. e135bb9 [Rey

spark git commit: [SPARK-6564][SQL] SQLContext.emptyDataFrame should contain 0 row, not 1 row

2015-03-27 Thread rxin
Repository: spark Updated Branches: refs/heads/master d5497ab13 -> 3af733430 [SPARK-6564][SQL] SQLContext.emptyDataFrame should contain 0 row, not 1 row Author: Reynold Xin Closes #5226 from rxin/empty-df and squashes the following commits: 1306d88 [Reynold Xin] Proper fix. e135bb9 [Reynold

spark git commit: [SPARK-6526][ML] Add Normalizer transformer in ML package

2015-03-27 Thread jkbradley
Repository: spark Updated Branches: refs/heads/master 887e1b72d -> d5497ab13 [SPARK-6526][ML] Add Normalizer transformer in ML package See [SPARK-6526](https://issues.apache.org/jira/browse/SPARK-6526). mengxr Should we add test suite for this transformer? There is no test suite for all feat

spark git commit: [SPARK-6544][build] Increment Avro version from 1.7.6 to 1.7.7

2015-03-27 Thread pwendell
Repository: spark Updated Branches: refs/heads/branch-1.3 b902a9552 -> fefd49f4b [SPARK-6544][build] Increment Avro version from 1.7.6 to 1.7.7 Fixes bug causing Kryo serialization to fail with Avro files in between stages. https://issues.apache.org/jira/browse/AVRO-1476?focusedCommentId=1399

spark git commit: [SPARK-6574] [PySpark] fix sql example

2015-03-27 Thread marmbrus
Repository: spark Updated Branches: refs/heads/branch-1.3 bc7518956 -> b902a9552 [SPARK-6574] [PySpark] fix sql example Fix the import in sql example. Author: Davies Liu Closes #5230 from davies/fix_sql_example and squashes the following commits: 7ecc5f4 [Davies Liu] fix sql example (cher

spark git commit: [SPARK-6574] [PySpark] fix sql example

2015-03-27 Thread marmbrus
Repository: spark Updated Branches: refs/heads/master 5d9c37c23 -> 887e1b72d [SPARK-6574] [PySpark] fix sql example Fix the import in sql example. Author: Davies Liu Closes #5230 from davies/fix_sql_example and squashes the following commits: 7ecc5f4 [Davies Liu] fix sql example Project:

spark git commit: [SPARK-6550][SQL] Use analyzed plan in DataFrame

2015-03-27 Thread marmbrus
Repository: spark Updated Branches: refs/heads/branch-1.3 b318858b9 -> bc7518956 [SPARK-6550][SQL] Use analyzed plan in DataFrame This is based on bug and test case proposed by viirya. See #5203 for a excellent description of the problem. TLDR; The problem occurs because the function `group

spark git commit: [SPARK-6550][SQL] Use analyzed plan in DataFrame

2015-03-27 Thread marmbrus
Repository: spark Updated Branches: refs/heads/master aa2b99174 -> 5d9c37c23 [SPARK-6550][SQL] Use analyzed plan in DataFrame This is based on bug and test case proposed by viirya. See #5203 for a excellent description of the problem. TLDR; The problem occurs because the function `groupBy(S

spark git commit: [SPARK-6544][build] Increment Avro version from 1.7.6 to 1.7.7

2015-03-27 Thread srowen
Repository: spark Updated Branches: refs/heads/master da546b7ba -> aa2b99174 [SPARK-6544][build] Increment Avro version from 1.7.6 to 1.7.7 Fixes bug causing Kryo serialization to fail with Avro files in between stages. https://issues.apache.org/jira/browse/AVRO-1476?focusedCommentId=13999249

spark git commit: [SPARK-6556][Core] Fix wrong parsing logic of executorTimeoutMs and checkTimeoutIntervalMs in HeartbeatReceiver

2015-03-27 Thread srowen
Repository: spark Updated Branches: refs/heads/master f43a61031 -> da546b7ba [SPARK-6556][Core] Fix wrong parsing logic of executorTimeoutMs and checkTimeoutIntervalMs in HeartbeatReceiver The current reading logic of `executorTimeoutMs` is: ```Scala private val executorTimeoutMs = sc.conf.ge

spark git commit: [SPARK-6341][mllib] Upgrade breeze from 0.11.1 to 0.11.2

2015-03-27 Thread meng
Repository: spark Updated Branches: refs/heads/branch-1.3 54d92b542 -> b318858b9 [SPARK-6341][mllib] Upgrade breeze from 0.11.1 to 0.11.2 There are any bugs of breeze's SparseVector at 0.11.1. You know, Spark 1.3 depends on breeze 0.11.1. So I think we should upgrade it to 0.11.2. https://iss

spark git commit: [SPARK-6341][mllib] Upgrade breeze from 0.11.1 to 0.11.2

2015-03-27 Thread meng
Repository: spark Updated Branches: refs/heads/master 49d2ec63e -> f43a61031 [SPARK-6341][mllib] Upgrade breeze from 0.11.1 to 0.11.2 There are any bugs of breeze's SparseVector at 0.11.1. You know, Spark 1.3 depends on breeze 0.11.1. So I think we should upgrade it to 0.11.2. https://issues.