[GitHub] spark pull request #19884: [WIP][SPARK-22324][SQL][PYTHON] Upgrade Arrow to ...

2017-12-11 Thread BryanCutler
Github user BryanCutler commented on a diff in the pull request: https://github.com/apache/spark/pull/19884#discussion_r156242013 --- Diff: dev/deps/spark-deps-hadoop-2.6 --- @@ -144,7 +144,9 @@ metrics-json-3.1.5.jar metrics-jvm-3.1.5.jar minlog-1.3.0.jar

[GitHub] spark pull request #19884: [WIP][SPARK-22324][SQL][PYTHON] Upgrade Arrow to ...

2017-12-11 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/19884#discussion_r156241339 --- Diff: dev/deps/spark-deps-hadoop-2.6 --- @@ -144,7 +144,9 @@ metrics-json-3.1.5.jar metrics-jvm-3.1.5.jar minlog-1.3.0.jar

[GitHub] spark pull request #19884: [WIP][SPARK-22324][SQL][PYTHON] Upgrade Arrow to ...

2017-12-11 Thread BryanCutler
Github user BryanCutler commented on a diff in the pull request: https://github.com/apache/spark/pull/19884#discussion_r156240909 --- Diff: dev/deps/spark-deps-hadoop-2.6 --- @@ -144,7 +144,9 @@ metrics-json-3.1.5.jar metrics-jvm-3.1.5.jar minlog-1.3.0.jar

[GitHub] spark pull request #19884: [WIP][SPARK-22324][SQL][PYTHON] Upgrade Arrow to ...

2017-12-11 Thread BryanCutler
Github user BryanCutler commented on a diff in the pull request: https://github.com/apache/spark/pull/19884#discussion_r156237523 --- Diff: pom.xml --- @@ -185,7 +185,7 @@ 2.8 1.8 1.0.0 -0.4.0 +0.8.0-SNAPSHOT --- End diff --

[GitHub] spark pull request #19884: [WIP][SPARK-22324][SQL][PYTHON] Upgrade Arrow to ...

2017-12-11 Thread wesm
Github user wesm commented on a diff in the pull request: https://github.com/apache/spark/pull/19884#discussion_r156097961 --- Diff: python/pyspark/serializers.py --- @@ -223,27 +223,13 @@ def _create_batch(series, timezone): series = [series] series = ((s,

[GitHub] spark pull request #19884: [WIP][SPARK-22324][SQL][PYTHON] Upgrade Arrow to ...

2017-12-10 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/19884#discussion_r155989817 --- Diff: pom.xml --- @@ -185,7 +185,7 @@ 2.8 1.8 1.0.0 -0.4.0 +0.8.0-SNAPSHOT --- End diff -- Please

[GitHub] spark pull request #19884: [WIP][SPARK-22324][SQL][PYTHON] Upgrade Arrow to ...

2017-12-10 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/19884#discussion_r155983224 --- Diff: python/pyspark/serializers.py --- @@ -223,27 +223,13 @@ def _create_batch(series, timezone): series = [series] series = ((s,

[GitHub] spark pull request #19884: [WIP][SPARK-22324][SQL][PYTHON] Upgrade Arrow to ...

2017-12-08 Thread wesm
Github user wesm commented on a diff in the pull request: https://github.com/apache/spark/pull/19884#discussion_r155854009 --- Diff: pom.xml --- @@ -185,7 +185,7 @@ 2.8 1.8 1.0.0 -0.4.0 +0.8.0-SNAPSHOT --- End diff -- Should

[GitHub] spark pull request #19884: [WIP][SPARK-22324][SQL][PYTHON] Upgrade Arrow to ...

2017-12-07 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/19884#discussion_r155708588 --- Diff: pom.xml --- @@ -185,7 +185,7 @@ 2.8 1.8 1.0.0 -0.4.0 +0.8.0-SNAPSHOT --- End diff -- Can we

[GitHub] spark pull request #19884: [WIP][SPARK-22324][SQL][PYTHON] Upgrade Arrow to ...

2017-12-07 Thread BryanCutler
Github user BryanCutler commented on a diff in the pull request: https://github.com/apache/spark/pull/19884#discussion_r155668850 --- Diff: python/pyspark/sql/types.py --- @@ -1658,13 +1657,13 @@ def from_arrow_type(at): spark_type = FloatType() elif at ==

[GitHub] spark pull request #19884: [WIP][SPARK-22324][SQL][PYTHON] Upgrade Arrow to ...

2017-12-07 Thread wesm
Github user wesm commented on a diff in the pull request: https://github.com/apache/spark/pull/19884#discussion_r155647982 --- Diff: python/pyspark/sql/types.py --- @@ -1658,13 +1657,13 @@ def from_arrow_type(at): spark_type = FloatType() elif at ==

[GitHub] spark pull request #19884: [WIP][SPARK-22324][SQL][PYTHON] Upgrade Arrow to ...

2017-12-07 Thread wesm
Github user wesm commented on a diff in the pull request: https://github.com/apache/spark/pull/19884#discussion_r155647741 --- Diff: python/pyspark/sql/types.py --- @@ -1658,13 +1657,13 @@ def from_arrow_type(at): spark_type = FloatType() elif at ==

[GitHub] spark pull request #19884: [WIP][SPARK-22324][SQL][PYTHON] Upgrade Arrow to ...

2017-12-07 Thread BryanCutler
Github user BryanCutler commented on a diff in the pull request: https://github.com/apache/spark/pull/19884#discussion_r155626249 --- Diff: python/pyspark/sql/types.py --- @@ -1658,13 +1657,13 @@ def from_arrow_type(at): spark_type = FloatType() elif at ==

[GitHub] spark pull request #19884: [WIP][SPARK-22324][SQL][PYTHON] Upgrade Arrow to ...

2017-12-04 Thread BryanCutler
Github user BryanCutler commented on a diff in the pull request: https://github.com/apache/spark/pull/19884#discussion_r154811610 --- Diff: pom.xml --- @@ -185,7 +185,7 @@ 2.8 1.8 1.0.0 -0.4.0 +0.8.0-SNAPSHOT --- End diff --

[GitHub] spark pull request #19884: [WIP][SPARK-22324][SQL][PYTHON] Upgrade Arrow to ...

2017-12-04 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/19884#discussion_r154810506 --- Diff: pom.xml --- @@ -185,7 +185,7 @@ 2.8 1.8 1.0.0 -0.4.0 +0.8.0-SNAPSHOT --- End diff --

[GitHub] spark pull request #19884: [WIP][SPARK-22324][SQL][PYTHON] Upgrade Arrow to ...

2017-12-04 Thread BryanCutler
GitHub user BryanCutler opened a pull request: https://github.com/apache/spark/pull/19884 [WIP][SPARK-22324][SQL][PYTHON] Upgrade Arrow to 0.8.0 ## What changes were proposed in this pull request? Upgrade Spark to Arrow 0.8.0 for Java and Python ## How was this