[GitHub] spark pull request #18049: [SPARK-20830][PYSPARK][SQL] Add posexplode and po...

2017-06-21 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/18049#discussion_r123376328 --- Diff: python/pyspark/sql/tests.py --- @@ -272,6 +276,11 @@ def test_explode(self): self.assertEqual(result[0][0

[GitHub] spark issue #18332: [SPARK-21125][PYTHON] Extend setJobDescription to PySpar...

2017-06-21 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/18332 Thanks! merging to master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark pull request #18049: [SPARK-20830][PYSPARK][SQL] Add posexplode and po...

2017-06-21 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/18049#discussion_r123341338 --- Diff: python/pyspark/sql/tests.py --- @@ -272,6 +276,11 @@ def test_explode(self): self.assertEqual(result[0][0

[GitHub] spark issue #14830: [SPARK-16992][PYSPARK][DOCS] import sort and autopep8 on...

2017-06-26 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/14830 Hi, are you still working on this? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #14963: [SPARK-16992][PYSPARK] Virtualenv for Pylint and pep8 in...

2017-06-26 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/14963 Hi, are you still working on this? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #17096: [SPARK-15243][ML][SQL][PYTHON] Add missing support for u...

2017-06-26 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/17096 @HyukjinKwon @holdenk Hi, are you still working on this? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark issue #14567: [SPARK-16992][PYSPARK] Python Pep8 formatting and import...

2017-06-26 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/14567 Hi, are you still working on this? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #18332: [SPARK-21125][PYTHON] Extend setJobDescription to PySpar...

2017-06-19 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/18332 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] spark issue #15927: [SPARK-18500][SQL] Make GenericStrategy be able to prune...

2017-06-19 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/15927 I'd close this for now. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark pull request #15927: [SPARK-18500][SQL] Make GenericStrategy be able t...

2017-06-19 Thread ueshin
Github user ueshin closed the pull request at: https://github.com/apache/spark/pull/15927 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] spark pull request #18332: [SPARK-21125][PYTHON] Extend setJobDescription to...

2017-06-19 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/18332#discussion_r122804678 --- Diff: python/pyspark/context.py --- @@ -942,6 +942,12 @@ def getLocalProperty(self, key): """ return self._jsc.g

[GitHub] spark pull request #18332: [SPARK-21125][PYTHON] Extend setJobDescription to...

2017-06-19 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/18332#discussion_r122804655 --- Diff: core/src/main/scala/org/apache/spark/api/java/JavaSparkContext.scala --- @@ -757,6 +757,9 @@ class JavaSparkContext(val sc: SparkContext

[GitHub] spark issue #15113: [SPARK-17508][PYSPARK][ML] PySpark treat Param values No...

2017-06-20 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/15113 @BryanCutler Hi, are you still working on this? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark issue #13794: [SPARK-15574][ML][PySpark] Python meta-algorithms in Sca...

2017-06-20 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/13794 @yinxusen Hi, are you still working on this? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark issue #16537: [SPARK-19165][PYTHON][SQL] UserDefinedFunction.__call__ ...

2017-06-20 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/16537 @zero323 Hi, are you still working on this? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18378: [SPARK-21163][SQL] DataFrame.toPandas should respect the...

2017-06-21 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/18378 LGTM, pending Jenkins. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #18054: [SPARK-20763][SQL][Backport-2.1] The function of `month`...

2017-05-22 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/18054 Thanks, merging to 2.1. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #18054: [SPARK-20763][SQL][Backport-2.1] The function of `month`...

2017-05-22 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/18054 Could you close the PR manually? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #17698: [SPARK-20403][SQL]Modify the instructions of some functi...

2017-05-22 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/17698 @10110346 Hi, you can use the command @gatorsmile mentioned above to generate the result file. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub

[GitHub] spark pull request #18094: [Spark-20775][SQL] Added scala support from_json

2017-05-24 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/18094#discussion_r118421269 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/functions.scala --- @@ -3072,6 +3072,22 @@ object functions { * @since 2.1.0

[GitHub] spark pull request #18094: [Spark-20775][SQL] Added scala support from_json

2017-05-24 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/18094#discussion_r118421254 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/functions.scala --- @@ -3060,7 +3060,7 @@ object functions { from_json(e, schema, Map.empty

[GitHub] spark issue #18016: [SPARK-20786][SQL]Improve ceil and floor handle the valu...

2017-05-25 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/18016 @heary-cao Hi, I found `Ceil` and `Floor` modified by this pr produce different results between interpret and codegen. We also need to modify `doGenCode()` to handle `LongType` correctly. Can

[GitHub] spark pull request #18057: [SPARK-20786][SQL][Backport-2.2]Improve ceil and ...

2017-05-25 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/18057#discussion_r118437741 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/mathExpressions.scala --- @@ -232,9 +232,10 @@ case class Ceil(child

[GitHub] spark issue #18054: [SPARK-20763][SQL][Backport-2.1] The function of `month`...

2017-05-21 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/18054 Jenkins, test this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #18054: [SPARK-20763][SQL][Backport-2.1] The function of `month`...

2017-05-22 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/18054 Jenkins, retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #18053: [SPARK-20763][SQL][Backport-2.1] The function of `month`...

2017-05-21 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/18053 @10110346 Hi, I guess you created a topic branch from `master`, but you need to create the topic branch from `branch-2.1` and then submit a pr to the `branch-2.1`. --- If your project is set up

[GitHub] spark issue #18054: [SPARK-20763][SQL][Backport-2.1] The function of `month`...

2017-05-22 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/18054 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] spark pull request #18136: [SPARK-20910][SQL] Add build-int SQL function - U...

2017-05-28 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/18136#discussion_r118855831 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/misc.scala --- @@ -104,3 +107,25 @@ case class CurrentDatabase() extends

[GitHub] spark pull request #18136: [SPARK-20910][SQL] Add build-int SQL function - U...

2017-05-28 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/18136#discussion_r118855760 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/misc.scala --- @@ -104,3 +107,25 @@ case class CurrentDatabase() extends

[GitHub] spark pull request #18134: [SPARK-20909][SQL] Add build-int SQL function - D...

2017-05-28 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/18134#discussion_r118858221 --- Diff: sql/core/src/test/resources/sql-tests/inputs/datetime.sql --- @@ -6,3 +6,5 @@ select current_date = current_date(), current_timestamp

[GitHub] spark pull request #18075: [SPARK-18016][SQL][CATALYST] Code Generation: Con...

2017-05-29 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/18075#discussion_r118866096 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala --- @@ -233,10 +222,128 @@ class CodegenContext

[GitHub] spark pull request #18075: [SPARK-18016][SQL][CATALYST] Code Generation: Con...

2017-05-29 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/18075#discussion_r118872149 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala --- @@ -233,10 +222,128 @@ class CodegenContext

[GitHub] spark pull request #18075: [SPARK-18016][SQL][CATALYST] Code Generation: Con...

2017-05-29 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/18075#discussion_r118865349 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala --- @@ -233,10 +222,128 @@ class CodegenContext

[GitHub] spark pull request #18075: [SPARK-18016][SQL][CATALYST] Code Generation: Con...

2017-05-29 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/18075#discussion_r118868339 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/generators.scala --- @@ -190,15 +190,15 @@ case class Stack(children: Seq

[GitHub] spark pull request #18075: [SPARK-18016][SQL][CATALYST] Code Generation: Con...

2017-05-29 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/18075#discussion_r118866774 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala --- @@ -233,10 +222,128 @@ class CodegenContext

[GitHub] spark issue #18075: [SPARK-18016][SQL][CATALYST] Code Generation: Constant P...

2017-05-30 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/18075 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] spark issue #18134: [SPARK-20909][SQL] Add build-int SQL function - DAYOFWEE...

2017-05-30 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/18134 Thanks! Merging to master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #18155: [SPARK-20876][SQL][Backport-2.2]If the input parameter i...

2017-05-30 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/18155 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] spark issue #18147: [SPARK-19236][SQL][FOLLOW-UP] Added createOrReplaceGloba...

2017-05-30 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/18147 LGTM except for @cloud-fan's comment. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18155: [SPARK-20876][SQL][Backport-2.2]If the input parameter i...

2017-05-31 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/18155 Jenkins, retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #18155: [SPARK-20876][SQL][Backport-2.2]If the input parameter i...

2017-05-31 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/18155 LGTM, cc @gatorsmile --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] spark pull request #17617: [SPARK-20244][Core] Handle incorrect bytesRead me...

2017-05-31 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/17617#discussion_r119301228 --- Diff: core/src/main/scala/org/apache/spark/deploy/SparkHadoopUtil.scala --- @@ -21,8 +21,10 @@ import java.io.IOException import

[GitHub] spark pull request #18183: [SPARK-20961][SQL] generalize the dictionary in C...

2017-06-02 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/18183#discussion_r119796465 --- Diff: sql/core/src/main/java/org/apache/spark/sql/execution/vectorized/ParquetDictionary.java --- @@ -0,0 +1,51 @@ +/* + * Licensed

[GitHub] spark pull request #18164: [SPARK-19732][SQL][PYSPARK] fillna bools

2017-06-02 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/18164#discussion_r119804978 --- Diff: python/pyspark/sql/tests.py --- @@ -1697,40 +1697,56 @@ def test_fillna(self): schema = StructType([ StructField

[GitHub] spark issue #18164: [SPARK-19732][SQL][PYSPARK] fillna bools

2017-06-02 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/18164 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] spark issue #18164: [SPARK-19732][SQL][PYSPARK] fillna bools

2017-06-02 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/18164 @HyukjinKwon Yes, I think it's okay to add this. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request #18164: [SPARK-19732][SQL][PYSPARK] fillna bools

2017-06-02 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/18164#discussion_r119805698 --- Diff: python/pyspark/sql/tests.py --- @@ -1697,40 +1697,56 @@ def test_fillna(self): schema = StructType([ StructField

[GitHub] spark pull request #18164: [SPARK-19732][SQL][PYSPARK] fillna bools

2017-06-02 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/18164#discussion_r119791032 --- Diff: python/pyspark/sql/tests.py --- @@ -1697,40 +1697,56 @@ def test_fillna(self): schema = StructType([ StructField

[GitHub] spark pull request #18164: [SPARK-19732][SQL][PYSPARK] fillna bools

2017-06-02 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/18164#discussion_r119800727 --- Diff: python/pyspark/sql/tests.py --- @@ -1697,40 +1697,56 @@ def test_fillna(self): schema = StructType([ StructField

[GitHub] spark pull request #18136: [SPARK-20910][SQL] Add build-int SQL function - U...

2017-05-31 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/18136#discussion_r119512921 --- Diff: sql/core/src/test/resources/sql-tests/inputs/string-functions.sql --- @@ -14,4 +14,4 @@ select replace('abc', 'b', '123'); select replace

[GitHub] spark issue #18075: [SPARK-18016][SQL][CATALYST] Code Generation: Constant P...

2017-05-31 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/18075 LGTM, cc @cloud-fan. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] spark pull request #18136: [SPARK-20910][SQL] Add build-int SQL function - U...

2017-05-31 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/18136#discussion_r119515971 --- Diff: sql/core/src/test/resources/sql-tests/inputs/string-functions.sql --- @@ -14,4 +14,4 @@ select replace('abc', 'b', '123'); select replace

[GitHub] spark issue #18164: [SPARK-19732][SQL][PYSPARK] Add fill functions for nulls...

2017-06-02 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/18164 Thanks! Merging to master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #18164: [SPARK-19732][SQL][PYSPARK] Add fill functions for nulls...

2017-06-02 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/18164 LGTM. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] spark issue #18136: [SPARK-20910][SQL] Add build-in SQL function - UUID

2017-06-01 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/18136 Thanks! Merging to master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #18136: [SPARK-20910][SQL] Add build-int SQL function - UUID

2017-05-31 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/18136 LGTM, pending tests. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] spark pull request #18139: [SPARK-20787][PYTHON] PySpark can't handle dateti...

2017-06-01 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/18139#discussion_r119580044 --- Diff: python/pyspark/sql/types.py --- @@ -187,8 +187,12 @@ def needConversion(self): def toInternal(self, dt): if dt

[GitHub] spark pull request #18139: [SPARK-20787][PYTHON] PySpark can't handle dateti...

2017-06-01 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/18139#discussion_r119570547 --- Diff: python/pyspark/sql/types.py --- @@ -187,8 +187,12 @@ def needConversion(self): def toInternal(self, dt): if dt

[GitHub] spark issue #17267: [SPARK-19926][PYSPARK] Make pyspark exception more user-...

2017-06-13 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/17267 +1 for adding a test. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] spark issue #17267: [SPARK-19926][PYSPARK] Make pyspark exception more user-...

2017-06-13 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/17267 Correct me if I'm wrong, but I got the following message after this patch in Python 3.6: ```python Traceback (most recent call last): File "", line 1, in Fil

[GitHub] spark pull request #17227: [SPARK-19507][PySpark][SQL] Show field name in _v...

2017-06-15 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/17227#discussion_r122283340 --- Diff: python/pyspark/sql/tests.py --- @@ -2367,6 +2380,151 @@ def range_frame_match(): importlib.reload(window) + +class

[GitHub] spark pull request #17227: [SPARK-19507][PySpark][SQL] Show field name in _v...

2017-06-15 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/17227#discussion_r122282508 --- Diff: python/pyspark/sql/tests.py --- @@ -2367,6 +2380,151 @@ def range_frame_match(): importlib.reload(window) + +class

[GitHub] spark pull request #17227: [SPARK-19507][PySpark][SQL] Show field name in _v...

2017-06-15 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/17227#discussion_r122273265 --- Diff: python/pyspark/sql/tests.py --- @@ -2367,6 +2380,151 @@ def range_frame_match(): importlib.reload(window) + +class

[GitHub] spark pull request #17227: [SPARK-19507][PySpark][SQL] Show field name in _v...

2017-06-15 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/17227#discussion_r122287524 --- Diff: python/pyspark/sql/tests.py --- @@ -2367,6 +2380,151 @@ def range_frame_match(): importlib.reload(window) + +class

[GitHub] spark pull request #17227: [SPARK-19507][PySpark][SQL] Show field name in _v...

2017-06-15 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/17227#discussion_r122282700 --- Diff: python/pyspark/sql/tests.py --- @@ -2367,6 +2380,151 @@ def range_frame_match(): importlib.reload(window) + +class

[GitHub] spark pull request #18260: [SPARK-21046][SQL] simplify the array offset and ...

2017-06-11 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/18260#discussion_r121285736 --- Diff: sql/core/src/main/java/org/apache/spark/sql/execution/vectorized/ColumnVector.java --- @@ -518,19 +519,13 @@ private void

[GitHub] spark pull request #18260: [SPARK-21046][SQL] simplify the array offset and ...

2017-06-11 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/18260#discussion_r121298160 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/vectorized/ColumnarBatchSuite.scala --- @@ -659,18 +659,18 @@ class ColumnarBatchSuite

[GitHub] spark pull request #18260: [SPARK-21046][SQL] simplify the array offset and ...

2017-06-11 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/18260#discussion_r121295792 --- Diff: sql/core/src/main/java/org/apache/spark/sql/execution/vectorized/OffHeapColumnVector.java --- @@ -438,10 +401,9 @@ public void loadBytes

[GitHub] spark pull request #18260: [SPARK-21046][SQL] simplify the array offset and ...

2017-06-11 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/18260#discussion_r121296069 --- Diff: sql/core/src/main/java/org/apache/spark/sql/execution/vectorized/OnHeapColumnVector.java --- @@ -366,55 +364,23 @@ public double getDouble(int

[GitHub] spark issue #18260: [SPARK-21046][SQL] simplify the array offset and length ...

2017-06-11 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/18260 LGTM except for one comment, pending Jenkins. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request #18260: [SPARK-21046][SQL] simplify the array offset and ...

2017-06-11 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/18260#discussion_r121296265 --- Diff: sql/core/src/main/java/org/apache/spark/sql/execution/vectorized/OnHeapColumnVector.java --- @@ -366,55 +364,23 @@ public double getDouble(int

[GitHub] spark pull request #18260: [SPARK-21046][SQL] simplify the array offset and ...

2017-06-11 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/18260#discussion_r121295705 --- Diff: sql/core/src/main/java/org/apache/spark/sql/execution/vectorized/OffHeapColumnVector.java --- @@ -34,6 +34,8 @@ // The data stored

[GitHub] spark pull request #18260: [SPARK-21046][SQL] simplify the array offset and ...

2017-06-11 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/18260#discussion_r121295725 --- Diff: sql/core/src/main/java/org/apache/spark/sql/execution/vectorized/OnHeapColumnVector.java --- @@ -42,6 +42,8 @@ // Array for each type

[GitHub] spark pull request #18260: [SPARK-21046][SQL] simplify the array offset and ...

2017-06-11 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/18260#discussion_r121296826 --- Diff: sql/core/src/main/java/org/apache/spark/sql/execution/vectorized/OffHeapColumnVector.java --- @@ -438,10 +401,9 @@ public void loadBytes

[GitHub] spark pull request #18260: [SPARK-21046][SQL] simplify the array offset and ...

2017-06-11 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/18260#discussion_r121297255 --- Diff: sql/core/src/main/java/org/apache/spark/sql/execution/vectorized/OffHeapColumnVector.java --- @@ -438,10 +401,9 @@ public void loadBytes

[GitHub] spark issue #18294: [SPARK-21089] [SQL] Fix DESC EXTENDED/FORMATTED to Show ...

2017-06-14 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/18294 LGTM. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] spark pull request #18324: [SPARK-21045][PYSPARK]Fixed executor blocked beca...

2017-06-18 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/18324#discussion_r122620265 --- Diff: python/pyspark/worker.py --- @@ -177,8 +180,11 @@ def process(): process() except Exception: try

[GitHub] spark issue #14198: [SPARK-16542][SQL][PYSPARK] Fix bugs about types that re...

2017-06-13 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/14198 @holdenk Are you still working on this? If so, could you rebase or merge master to fix conflicts please? --- If your project is set up for it, you can reply to this email and have your reply appear

[GitHub] spark issue #14198: [SPARK-16542][SQL][PYSPARK] Fix bugs about types that re...

2017-06-13 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/14198 @zasdfgbnm Are you still working on this? If so, could you rebase or merge master to fix conflicts please? --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark issue #18252: [SPARK-17914][SQL] Fix parsing of timestamp strings with...

2017-06-09 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/18252 Looks like [DateTimeUtils.scala#L411-L414](https://github.com/aokolnychyi/spark/blob/2f232a7bda28fb42759ee35923044f886a1ff19e/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util

[GitHub] spark issue #16781: [SPARK-12297][SQL] Hive compatibility for Parquet Timest...

2017-05-07 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/16781 LGTM, pending Jenkins. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #16781: [SPARK-12297][SQL] Hive compatibility for Parquet Timest...

2017-05-07 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/16781 Thanks! Merging to master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #16781: [SPARK-12297][SQL] Hive compatibility for Parquet Timest...

2017-05-07 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/16781 Jenkins, retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #17678: [SPARK-20381][SQL] Add SQL metrics of numOutputRows for ...

2017-05-04 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/17678 Jenkins, retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark pull request #17678: [SPARK-20381][SQL] Add SQL metrics of numOutputRo...

2017-05-03 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/17678#discussion_r114700907 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/aggregate/ObjectAggregationIterator.scala --- @@ -83,6 +85,7 @@ class

[GitHub] spark issue #18139: [SPARK-20787][PYTHON] PySpark can't handle datetimes bef...

2017-06-01 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/18139 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] spark pull request #18444: [SPARK-16542][SQL][PYSPARK] Fix bugs about types ...

2017-06-28 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/18444#discussion_r124465362 --- Diff: python/pyspark/sql/types.py --- @@ -958,12 +968,17 @@ def _infer_type(obj): return MapType(_infer_type(key), _infer_type(value

[GitHub] spark pull request #18444: [SPARK-16542][SQL][PYSPARK] Fix bugs about types ...

2017-06-28 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/18444#discussion_r124464895 --- Diff: python/pyspark/sql/tests.py --- @@ -2259,6 +2261,60 @@ def test_BinaryType_serialization(self): df = self.spark.createDataFrame(data

[GitHub] spark issue #18444: [SPARK-16542][SQL][PYSPARK] Fix bugs about types that re...

2017-06-28 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/18444 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] spark issue #19284: [SPARK-22067][SQL] ArrowWriter should use position when ...

2017-09-19 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/19284 LGTM. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #19284: [SPARK-22067][SQL] ArrowWriter should use position when ...

2017-09-19 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/19284 Thanks! merging to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark pull request #18754: [WIP][SPARK-21552][SQL] Add DecimalType support t...

2017-09-19 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/18754#discussion_r139612110 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/arrow/ArrowWriter.scala --- @@ -224,6 +226,25 @@ private[arrow] class DoubleWriter(val

[GitHub] spark pull request #19147: [WIP][SPARK-21190][SQL][PYTHON] Vectorized UDFs i...

2017-09-22 Thread ueshin
Github user ueshin closed the pull request at: https://github.com/apache/spark/pull/19147 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19147: [WIP][SPARK-21190][SQL][PYTHON] Vectorized UDFs in Pytho...

2017-09-22 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/19147 I'd close this in favor of #18659. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e

[GitHub] spark pull request #19325: [SPARK-22106][PYSPARK][SQL] Disable 0-parameter p...

2017-09-22 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/19325#discussion_r140626292 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/python/ArrowEvalPythonExec.scala --- @@ -51,10 +51,12 @@ case class ArrowEvalPythonExec

[GitHub] spark issue #19333: [SPARK-22109][SQL][BRANCH-2.2] Resolves type conflicts b...

2017-09-23 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/19333 LGTM, pending Jenkins. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark issue #19333: [SPARK-22109][SQL][BRANCH-2.2] Resolves type conflicts b...

2017-09-23 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/19333 @HyukjinKwon Could you close this PR? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands

[GitHub] spark issue #19331: [SPARK-22109][SQL] Resolves type conflicts between strin...

2017-09-23 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/19331 @HyukjinKwon Could you send a pr to backport this to branch-2.2 please? --- - To unsubscribe, e-mail: reviews-unsubscr

[GitHub] spark issue #19333: [SPARK-22109][SQL][BRANCH-2.2] Resolves type conflicts b...

2017-09-23 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/19333 Thanks! merging to branch-2.2. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark issue #19331: [SPARK-22109][SQL] Resolves type conflicts between strin...

2017-09-23 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/19331 LGTM. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

<    4   5   6   7   8   9   10   11   12   13   >