[GitHub] spark pull request #21103: [SPARK-23915][SQL] Add array_except function

2018-07-26 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/21103#discussion_r205663032 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -3805,3 +3799,230 @@ object ArrayUnion

[GitHub] spark pull request #21103: [SPARK-23915][SQL] Add array_except function

2018-07-26 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/21103#discussion_r205495042 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -3805,3 +3799,330 @@ object ArrayUnion

[GitHub] spark pull request #21103: [SPARK-23915][SQL] Add array_except function

2018-07-25 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/21103#discussion_r205310335 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -3805,3 +3799,330 @@ object ArrayUnion

[GitHub] spark issue #21867: [SPARK-24307][CORE] Add conf to revert to old code.

2018-07-25 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/21867 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #21867: [SPARK-24307][CORE] Add conf to revert to old code.

2018-07-25 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/21867 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark pull request #21867: [SPARK-24307][CORE] Add conf to revert to old cod...

2018-07-24 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/21867#discussion_r204972249 --- Diff: core/src/main/scala/org/apache/spark/storage/BlockManager.scala --- @@ -731,7 +731,14 @@ private[spark] class BlockManager

[GitHub] spark issue #21772: [SPARK-24809] [SQL] Serializing LongToUnsafeRowMap in ex...

2018-07-24 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/21772 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #21848: [SPARK-24890] [SQL] Short circuiting the `if` condition ...

2018-07-24 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/21848 @dbtsai I have a question. How does the current code check the following condition? > Stateful expression must have a side eff

[GitHub] spark issue #21860: [SPARK-24901][SQL]Merge the codegen of RegularHashMap an...

2018-07-24 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/21860 Could you please let us know which existing test verifies the newly added code generation path? --- - To unsubscribe, e-mail

[GitHub] spark pull request #21857: [SPARK-21274][SQL] Implement EXCEPT ALL clause.

2018-07-24 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/21857#discussion_r204750587 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/generators.scala --- @@ -222,6 +222,32 @@ case class Stack(children: Seq

[GitHub] spark pull request #21857: [SPARK-21274][SQL] Implement EXCEPT ALL clause.

2018-07-24 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/21857#discussion_r204743359 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/generators.scala --- @@ -222,6 +222,32 @@ case class Stack(children: Seq

[GitHub] spark issue #21855: [SPARK-22499][FOLLOWUP][SQL] Reduce input string express...

2018-07-24 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/21855 LGTM. With Spark 2.2, I have just confirmed the followings: ``` N=1500: passed N=1800: exception N=2000: exception

[GitHub] spark issue #21855: [SPARK-22499][FOLLOWUP][SQL] Reduce input string express...

2018-07-24 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/21855 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark pull request #21103: [SPARK-23915][SQL] Add array_except function

2018-07-23 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/21103#discussion_r204613831 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -3805,3 +3799,331 @@ object ArrayUnion

[GitHub] spark issue #21811: [SPARK-24801][CORE] Avoid memory waste by empty byte[] a...

2018-07-23 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/21811 @countmdm Sorry for overlooking the JIRA description. I got the situation. While the memory pool could be, it is too complex. LGTM

[GitHub] spark issue #21811: [SPARK-24801][CORE] Avoid memory waste by empty byte[] a...

2018-07-23 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/21811 @countmdm I see. We may be interested in ratio regarding `byte[] / all allocated memory` before and after. Not interested in other objects (e.g. an object including customer's name

[GitHub] spark issue #21852: [SPARK-24893] [SQL] Remove the entire CaseWhen if all th...

2018-07-23 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/21852 This PR also has [the similar issue](https://github.com/apache/spark/pull/21848#issuecomment-407166299) if a condition has a side effect

[GitHub] spark issue #21811: [SPARK-24801][CORE] Avoid memory waste by empty byte[] a...

2018-07-23 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/21811 I see. SGTM. Would it be possible to attach heap profiling (allocated size for each type) before and after this PR to record the difference

[GitHub] spark issue #21102: [SPARK-23913][SQL] Add array_intersect function

2018-07-23 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/21102 I want to hear opinion of others about the order of a result. cc @ueshin --- - To unsubscribe, e-mail: reviews-unsubscr

[GitHub] spark pull request #21802: [SPARK-23928][SQL] Add shuffle collection functio...

2018-07-22 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/21802#discussion_r204270437 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -1184,6 +1186,137 @@ case class ArraySort

[GitHub] spark pull request #21830: [SPARK-24878][SQL] Fix reverse function for array...

2018-07-22 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/21830#discussion_r204250861 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -1244,46 +1244,50 @@ case class Reverse

[GitHub] spark issue #21811: [SPARK-24801][CORE] Avoid memory waste by empty byte[] a...

2018-07-22 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/21811 Does it make sense to release `byteChannel` at `deallocate()`? ``` if (byteChannel != null) { byteChannel = null

[GitHub] spark pull request #21802: [SPARK-23928][SQL] Add shuffle collection functio...

2018-07-22 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/21802#discussion_r204249027 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/CollectionExpressionsSuite.scala --- @@ -1419,4 +1421,71 @@ class

[GitHub] spark issue #19222: [SPARK-10399][SPARK-23879][CORE][SQL] Introduce multiple...

2018-07-22 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/19222 ping @rednaxelafx --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark pull request #21758: [SPARK-24795][CORE] Implement barrier execution m...

2018-07-22 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/21758#discussion_r204236923 --- Diff: core/src/test/scala/org/apache/spark/rdd/RDDBarrierSuite.scala --- @@ -0,0 +1,43 @@ +/* + * Licensed to the Apache Software Foundation (ASF

[GitHub] spark pull request #21758: [SPARK-24795][CORE] Implement barrier execution m...

2018-07-22 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/21758#discussion_r204236759 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala --- @@ -359,20 +366,55 @@ private[spark] class TaskSchedulerImpl

[GitHub] spark pull request #21758: [SPARK-24795][CORE] Implement barrier execution m...

2018-07-22 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/21758#discussion_r204236692 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala --- @@ -359,20 +366,55 @@ private[spark] class TaskSchedulerImpl

[GitHub] spark pull request #21758: [SPARK-24795][CORE] Implement barrier execution m...

2018-07-22 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/21758#discussion_r204236526 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala --- @@ -359,20 +366,55 @@ private[spark] class TaskSchedulerImpl

[GitHub] spark pull request #21758: [SPARK-24795][CORE] Implement barrier execution m...

2018-07-22 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/21758#discussion_r204235875 --- Diff: core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala --- @@ -1426,6 +1506,18 @@ class DAGScheduler

[GitHub] spark pull request #21758: [SPARK-24795][CORE] Implement barrier execution m...

2018-07-22 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/21758#discussion_r204235813 --- Diff: core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala --- @@ -1411,6 +1420,77 @@ class DAGScheduler

[GitHub] spark pull request #21758: [SPARK-24795][CORE] Implement barrier execution m...

2018-07-22 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/21758#discussion_r204232986 --- Diff: core/src/main/scala/org/apache/spark/MapOutputTracker.scala --- @@ -434,6 +434,18 @@ private[spark] class MapOutputTrackerMaster

[GitHub] spark issue #21822: [SPARK-24865] Remove AnalysisBarrier - WIP

2018-07-22 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/21822 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #21102: [SPARK-23913][SQL] Add array_intersect function

2018-07-21 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/21102 cc @ueshin --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #19449: [SPARK-22219][SQL] Refactor code to get a value for "spa...

2018-07-21 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/19449 cc @gatorsmile --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #21103: [SPARK-23915][SQL] Add array_except function

2018-07-21 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/21103 cc @ueshin --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #21102: [SPARK-23913][SQL] Add array_intersect function

2018-07-21 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/21102 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark pull request #21103: [SPARK-23915][SQL] Add array_except function

2018-07-21 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/21103#discussion_r204218477 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -3805,3 +3799,332 @@ object ArrayUnion

[GitHub] spark issue #21785: [SPARK-24529][BUILD][test-maven][FOLLOW-UP] Set spotbugs...

2018-07-21 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/21785 @HyukjinKwon @wangyum thank you. I took a vacation. If there is no error in amp build infra, I think that this is a better way

[GitHub] spark issue #21785: [SPARK-24529][BUILD][test-maven][FOLLOW-UP] Set spotbugs...

2018-07-21 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/21785 @HyukjinKwon @wangyum thank you. If there is no error in amp build infra, I think that this is a better way

[GitHub] spark pull request #21103: [SPARK-23915][SQL] Add array_except function

2018-07-21 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/21103#discussion_r204214620 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -3805,3 +3799,332 @@ object ArrayUnion

[GitHub] spark issue #21103: [SPARK-23915][SQL] Add array_except function

2018-07-20 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/21103 cc @ueshin --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark pull request #21103: [SPARK-23915][SQL] Add array_except function

2018-07-20 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/21103#discussion_r203960424 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -3805,3 +3799,332 @@ object ArrayUnion

[GitHub] spark issue #21103: [SPARK-23915][SQL] Add array_except function

2018-07-19 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/21103 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark pull request #21102: [SPARK-23913][SQL] Add array_intersect function

2018-07-18 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/21102#discussion_r203319710 --- Diff: core/src/main/scala/org/apache/spark/util/collection/OpenHashSet.scala --- @@ -163,7 +187,7 @@ class OpenHashSet[@specialized(Long, Int) T

[GitHub] spark pull request #21102: [SPARK-23913][SQL] Add array_intersect function

2018-07-18 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/21102#discussion_r203318288 --- Diff: core/src/main/scala/org/apache/spark/util/collection/OpenHashSet.scala --- @@ -85,9 +85,13 @@ class OpenHashSet[@specialized(Long, Int) T: ClassTag

[GitHub] spark pull request #21102: [SPARK-23913][SQL] Add array_intersect function

2018-07-18 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/21102#discussion_r203315417 --- Diff: core/src/main/scala/org/apache/spark/util/collection/OpenHashSet.scala --- @@ -114,6 +118,21 @@ class OpenHashSet[@specialized(Long, Int) T

[GitHub] spark issue #21785: [SPARK-24529][BUILD][test-maven][FOLLOW-UP] Set spotbugs...

2018-07-17 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/21785 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #21785: [SPARK-24529][BUILD][test-maven][FOLLOW-UP] Set spotbugs...

2018-07-17 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/21785 I got the same error in my Mac. I understand that the motivation of forking is [this solution](https://stackoverflow.com/questions/22609357/jenkins-fatal-error-timeout-killed-the-sub-process

[GitHub] spark issue #21785: [SPARK-24529][BUILD][test-maven][FOLLOW-UP] Set spotbugs...

2018-07-17 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/21785 @wangyum my environment is ``` $ java -version openjdk version "1.8.0_171" OpenJDK Runtime Environment (build 1.8.0_171-8u171-b11-0ubuntu0.16.04.1-b11) OpenJDK 64-Bit

[GitHub] spark issue #21794: [SPARK-24834][CORE] use java comparison for float and do...

2018-07-17 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/21794 It would be good to add test cases for them since it is not covered now. --- - To unsubscribe, e-mail: reviews-unsubscr

[GitHub] spark issue #21794: [SPARK-24834][CORE] use java comparison for float and do...

2018-07-17 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/21794 I think this is required since SparkSQL does not distinguish 0.0 from -0.0. Am I correct? cc @gatorsmile @maropu

[GitHub] spark pull request #21352: [SPARK-24305][SQL][FOLLOWUP] Avoid serialization ...

2018-07-17 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/21352#discussion_r203013625 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -737,21 +733,22 @@ case class MapConcat

[GitHub] spark issue #21785: [SPARK-24529][BUILD][test-maven][FOLLOW-UP] Set spotbugs...

2018-07-17 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/21785 In my environment, `make-distribution.sh` works well without fork. ``` ... [INFO] [INFO] --- maven-source-plugin:3.0.1:jar-no-fork (create-source-jar) @ spark-assembly_2.11

[GitHub] spark issue #21791: [SPARK-24832][SQL] Improve inputMetrics's bytesRead upda...

2018-07-17 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/21791 Is it better to add a test to `InputOutputMetricsSuite`? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

[GitHub] spark issue #20636: [SPARK-23415][SQL][TEST] Make behavior of BufferHolderSp...

2018-07-17 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/20636 cc @cloud-fan --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #21102: [SPARK-23913][SQL] Add array_intersect function

2018-07-17 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/21102 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #19449: [SPARK-22219][SQL] Refactor code to get a value for "spa...

2018-07-17 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/19449 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #20636: [SPARK-23415][SQL][TEST] Make behavior of BufferHolderSp...

2018-07-17 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/20636 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #21785: [SPARK-24529][BUILD][test-maven][FOLLOW-UP] Set spotbugs...

2018-07-17 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/21785 Now, I am checking `make-distribution.sh` in my environment. If my memory is correct, the error in `spark-sql_2.11` or `spark-catalyst_2.11` was a motivation to stop forking of spotbugs

[GitHub] spark issue #19449: [SPARK-22219][SQL] Refactor code to get a value for "spa...

2018-07-17 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/19449 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #21102: [SPARK-23913][SQL] Add array_intersect function

2018-07-17 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/21102 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #21537: [SPARK-24505][SQL] Convert strings in codegen to blocks:...

2018-07-16 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/21537 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #20636: [SPARK-23415][SQL][TEST] Make behavior of BufferHolderSp...

2018-07-16 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/20636 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark pull request #19449: [SPARK-22219][SQL] Refactor code to get a value f...

2018-07-16 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/19449#discussion_r202869932 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -751,6 +751,12 @@ object SQLConf { .booleanConf

[GitHub] spark issue #21102: [SPARK-23913][SQL] Add array_intersect function

2018-07-16 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/21102 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark pull request #21102: [SPARK-23913][SQL] Add array_intersect function

2018-07-16 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/21102#discussion_r202782954 --- Diff: core/src/test/scala/org/apache/spark/util/collection/OpenHashSetSuite.scala --- @@ -73,6 +73,46 @@ class OpenHashSetSuite extends SparkFunSuite

[GitHub] spark issue #20636: [SPARK-23415][SQL][TEST] Make behavior of BufferHolderSp...

2018-07-16 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/20636 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #19449: [SPARK-22219][SQL] Refactor code to get a value for "spa...

2018-07-16 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/19449 Sorry for leaving this for a while. I will update it using `StaticSQLConf` soon. --- - To unsubscribe, e-mail: reviews-unsubscr

[GitHub] spark issue #21102: [SPARK-23913][SQL] Add array_intersect function

2018-07-15 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/21102 cc @ueshin --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark pull request #21777: [WIP][SPARK-24498][SQL] Add JDK compiler for runt...

2018-07-15 Thread kiszk
GitHub user kiszk opened a pull request: https://github.com/apache/spark/pull/21777 [WIP][SPARK-24498][SQL] Add JDK compiler for runtime codegen ## What changes were proposed in this pull request? This PR allow a user to select Javac bytecode compiler to compile

[GitHub] spark issue #21103: [SPARK-23915][SQL] Add array_except function

2018-07-14 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/21103 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #21102: [SPARK-23913][SQL] Add array_intersect function

2018-07-14 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/21102 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark pull request #21537: [SPARK-24505][SQL] Convert strings in codegen to ...

2018-07-13 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/21537#discussion_r202271473 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala --- @@ -579,6 +579,18 @@ class CodegenContext

[GitHub] spark pull request #21537: [SPARK-24505][SQL] Convert strings in codegen to ...

2018-07-13 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/21537#discussion_r202269577 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/Cast.scala --- @@ -720,31 +719,36 @@ case class Cast(child: Expression

[GitHub] spark pull request #21061: [SPARK-23914][SQL] Add array_union function

2018-07-11 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/21061#discussion_r201904049 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -3261,3 +3261,322 @@ case class

[GitHub] spark pull request #21061: [SPARK-23914][SQL] Add array_union function

2018-07-11 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/21061#discussion_r201692534 --- Diff: sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/UnsafeArrayData.java --- @@ -463,14 +463,27 @@ private static UnsafeArrayData

[GitHub] spark pull request #21061: [SPARK-23914][SQL] Add array_union function

2018-07-11 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/21061#discussion_r201688229 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -3261,3 +3261,322 @@ case class

[GitHub] spark issue #21061: [SPARK-23914][SQL] Add array_union function

2018-07-09 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/21061 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #21061: [SPARK-23914][SQL] Add array_union function

2018-07-09 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/21061 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #21706: [SPARK-24702] Fix Unable to cast to calendar interval in...

2018-07-09 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/21706 @gatorsmile @ueshin could you kick test for this PR? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

[GitHub] spark pull request #21061: [SPARK-23914][SQL] Add array_union function

2018-07-09 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/21061#discussion_r201109190 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -3261,3 +3261,323 @@ case class

[GitHub] spark issue #21542: [SPARK-24529][Build][test-maven] Add spotbugs into maven...

2018-07-09 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/21542 Does it make sense to compare these execution times [thisPR](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92746/) and [master](https://amplab.cs.berkeley.edu/jenkins/job/spark

[GitHub] spark issue #21542: [SPARK-24529][Build][test-maven] Add spotbugs into maven...

2018-07-09 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/21542 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark pull request #21542: [SPARK-24529][Build][test-maven] Add spotbugs int...

2018-07-09 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/21542#discussion_r200926335 --- Diff: pom.xml --- @@ -2606,6 +2606,28 @@ + +com.github.spotbugs +spotbugs

[GitHub] spark pull request #21061: [SPARK-23914][SQL] Add array_union function

2018-07-09 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/21061#discussion_r200925450 --- Diff: python/pyspark/sql/functions.py --- @@ -2013,6 +2013,25 @@ def array_distinct(col): return Column(sc._jvm.functions.array_distinct

[GitHub] spark pull request #21542: [SPARK-24529][Build][test-maven] Add spotbugs int...

2018-07-09 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/21542#discussion_r200922589 --- Diff: pom.xml --- @@ -2606,6 +2606,28 @@ + +com.github.spotbugs +spotbugs

[GitHub] spark pull request #21061: [SPARK-23914][SQL] Add array_union function

2018-07-08 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/21061#discussion_r200887096 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -3261,3 +3261,323 @@ case class

[GitHub] spark pull request #21061: [SPARK-23914][SQL] Add array_union function

2018-07-08 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/21061#discussion_r200885976 --- Diff: sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/UnsafeArrayData.java --- @@ -450,7 +450,7 @@ public UnsafeArrayData copy

[GitHub] spark pull request #21061: [SPARK-23914][SQL] Add array_union function

2018-07-08 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/21061#discussion_r200883268 --- Diff: sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/UnsafeArrayData.java --- @@ -450,7 +450,7 @@ public UnsafeArrayData copy

[GitHub] spark issue #21542: [SPARK-24529][Build][test-maven] Add spotbugs into maven...

2018-07-08 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/21542 ping @cloud-fan @viirya @HyukjinKwon --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands

[GitHub] spark issue #21061: [SPARK-23914][SQL] Add array_union function

2018-07-08 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/21061 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #21726: Branch 2.3

2018-07-06 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/21726 Could you please close this? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark pull request #21061: [SPARK-23914][SQL] Add array_union function

2018-07-06 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/21061#discussion_r200572386 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -3261,3 +3261,339 @@ case class

[GitHub] spark issue #19222: [SPARK-10399][SPARK-23879][CORE][SQL] Introduce multiple...

2018-07-05 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/19222 ping @rednaxelafx --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark pull request #21618: [SPARK-20408][SQL] Get the glob path in parallel ...

2018-07-05 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/21618#discussion_r200465965 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -656,6 +656,25 @@ object SQLConf { .intConf

[GitHub] spark pull request #21618: [SPARK-20408][SQL] Get the glob path in parallel ...

2018-07-05 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/21618#discussion_r200465855 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSource.scala --- @@ -724,4 +726,35 @@ object DataSource extends Logging

[GitHub] spark pull request #21618: [SPARK-20408][SQL] Get the glob path in parallel ...

2018-07-05 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/21618#discussion_r200462611 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -656,6 +656,25 @@ object SQLConf { .intConf

[GitHub] spark pull request #21706: [SPARK-24702] Fix Unable to cast to calendar inte...

2018-07-05 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/21706#discussion_r200457047 --- Diff: sql/core/src/test/resources/sql-tests/inputs/cast.sql --- @@ -42,4 +42,23 @@ SELECT CAST('9223372036854775808' AS long); DESC FUNCTION

[GitHub] spark pull request #21542: [SPARK-24529][Build][test-maven] Add spotbugs int...

2018-07-04 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/21542#discussion_r200193562 --- Diff: resource-managers/kubernetes/core/pom.xml --- @@ -47,6 +47,12 @@ test + --- End diff -- This change

[GitHub] spark issue #21542: [SPARK-24529][Build][test-maven] Add spotbugs into maven...

2018-07-04 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/21542 cc @cloud-fan @viirya @HyukjinKwon --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e

<    2   3   4   5   6   7   8   9   10   11   >