[GitHub] spark pull request #22198: [SPARK-25121][SQL] Supports multi-part table name...

2018-08-26 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/22198#discussion_r212843948 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/DataFrameJoinSuite.scala --- @@ -191,6 +195,48 @@ class DataFrameJoinSuite extends QueryTest

[GitHub] spark issue #22204: [SPARK-25196][SQL] Analyze column statistics in cached q...

2018-08-26 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/22204 ping --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #22198: [SPARK-25121][SQL] Supports multi-part table names for b...

2018-08-25 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/22198 Thanks, @dongjoon-hyun! I'll check and merge that. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

[GitHub] spark issue #22198: [SPARK-25121][SQL] Supports multi-part table names for b...

2018-08-25 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/22198 aha, I see. IMO we need to apply the hint in the case, too. I'll fix. --- - To unsubscribe, e-mail: reviews-unsubscr

[GitHub] spark issue #21931: [SPARK-24978][SQL]Add spark.sql.fast.hash.aggregate.row....

2018-08-25 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21931 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark pull request #22232: [SPARK-25237][SQL]remove updateBytesReadWithFileS...

2018-08-25 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/22232#discussion_r212796191 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/FileScanRDD.scala --- @@ -208,7 +199,6 @@ class FileScanRDD

[GitHub] spark issue #22232: [SPARK-25237][SQL]remove updateBytesReadWithFileSize bec...

2018-08-25 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/22232 I'm not sure we can test the case though, for example, how about the sequence below? ``` import org.apache.spark.TaskContext spark.range(10).selectExpr("id AS c0", &quo

[GitHub] spark issue #22232: [SPARK-25237][SQL]remove updateBytesReadWithFileSize bec...

2018-08-25 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/22232 btw, can you clean up the title and the description..? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

[GitHub] spark issue #22232: [SPARK-25237][SQL]remove updateBytesReadWithFileSize bec...

2018-08-25 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/22232 Is it difficult to add tests for checking the metric in the case `select * from t limit 1`? --- - To unsubscribe, e-mail

[GitHub] spark pull request #22232: [SPARK-25237][SQL]remove updateBytesReadWithFileS...

2018-08-25 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/22232#discussion_r212793049 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/FileScanRDD.scala --- @@ -208,7 +199,6 @@ class FileScanRDD

[GitHub] spark issue #22198: [SPARK-25121][SQL] Supports multi-part table names for b...

2018-08-24 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/22198 @dilipbiswal @gatorsmile ping --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark issue #22204: [SPARK-25196][SQL] Analyze column statistics in cached q...

2018-08-24 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/22204 @gatorsmile ping --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark pull request #22141: [SPARK-25154][SQL] Support NOT IN sub-queries ins...

2018-08-24 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/22141#discussion_r212785260 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/subquery.scala --- @@ -137,13 +137,21 @@ object RewritePredicateSubquery

[GitHub] spark pull request #22218: [SPARK-25228][CORE]Add executor CPU time metric.

2018-08-24 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/22218#discussion_r212784518 --- Diff: core/src/main/scala/org/apache/spark/executor/ExecutorSource.scala --- @@ -73,6 +75,13 @@ class ExecutorSource(threadPool: ThreadPoolExecutor

[GitHub] spark pull request #22227: [SPARK-25202] [Core] Implements split with limit ...

2018-08-24 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/7#discussion_r212783216 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/regexpExpressions.scala --- @@ -232,30 +232,41 @@ case class RLike(left

[GitHub] spark pull request #22227: [SPARK-25202] [Core] Implements split with limit ...

2018-08-24 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/7#discussion_r212783068 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/functions.scala --- @@ -2554,7 +2554,27 @@ object functions { * @since 1.5.0

[GitHub] spark pull request #22227: [SPARK-25202] [Core] Implements split with limit ...

2018-08-24 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/7#discussion_r212782784 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/regexpExpressions.scala --- @@ -232,30 +232,41 @@ case class RLike(left

[GitHub] spark pull request #22227: [SPARK-25202] [Core] Implements split with limit ...

2018-08-24 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/7#discussion_r212782576 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/functions.scala --- @@ -2554,7 +2554,27 @@ object functions { * @since 1.5.0

[GitHub] spark pull request #22227: [SPARK-25202] [Core] Implements split with limit ...

2018-08-24 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/7#discussion_r212781563 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/functions.scala --- @@ -2554,7 +2554,27 @@ object functions { * @since 1.5.0

[GitHub] spark issue #22227: [SPARK-25202] [Core] Implements split with limit sql fun...

2018-08-24 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/7 Can you add tests in `StringFunctionsSuite`, too? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

[GitHub] spark issue #22227: [SPARK-25202] [Core] Implements split with limit sql fun...

2018-08-24 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/7 @gatorsmile @ueshin can you trigger this test? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22227: [SPARK-25202] [Core] Implements split with limit sql fun...

2018-08-24 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/7 not `[CORE]` but `[SQL]` in the title. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands

[GitHub] spark pull request #22204: [SPARK-25196][SQL] Analyze column statistics in c...

2018-08-23 Thread maropu
GitHub user maropu opened a pull request: https://github.com/apache/spark/pull/22204 [SPARK-25196][SQL] Analyze column statistics in cached query ## What changes were proposed in this pull request? This pr proposed a new API to analyze column statistics in cached query

[GitHub] spark pull request #22198: [SPARK-25121][SQL] Supports multi-part table name...

2018-08-23 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/22198#discussion_r212249623 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/DataFrameJoinSuite.scala --- @@ -191,6 +195,39 @@ class DataFrameJoinSuite extends QueryTest

[GitHub] spark pull request #22198: [SPARK-25121][SQL] Supports multi-part table name...

2018-08-23 Thread maropu
GitHub user maropu opened a pull request: https://github.com/apache/spark/pull/22198 [SPARK-25121][SQL] Supports multi-part table names for broadcast hint resolution ## What changes were proposed in this pull request? This pr fixed code to respect a database name for broadcast

[GitHub] spark issue #22153: [SPARK-23034][SQL] Show RDD/relation names in RDD/In-Mem...

2018-08-22 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/22153 my bad, this pr doesn't affect cache tables in webui. I'll drop these. Actually, this affects hive tables and rdds only; ``` >> Hive table case sql("CREATE TABLE t(c1 int)

[GitHub] spark issue #20345: [SPARK-23172][SQL] Expand the ReorderJoin rule to handle...

2018-08-22 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/20345 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #20345: [SPARK-23172][SQL] Expand the ReorderJoin rule to handle...

2018-08-22 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/20345 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #21770: [SPARK-24806][SQL] Brush up generated code so that JDK c...

2018-08-22 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21770 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #20345: [SPARK-23172][SQL] Expand the ReorderJoin rule to handle...

2018-08-22 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/20345 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #22154: [SPARK-23711][SPARK-25140][SQL] Catch correct exceptions...

2018-08-21 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/22154 (it it jus a note) btw, currently, if expr codegen fails, the many error messages could happen in both a driver side and executor sides. I feel this is a little noisy for users. I think it'd

[GitHub] spark pull request #22141: [SPARK-25154][SQL] Support NOT IN sub-queries ins...

2018-08-21 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/22141#discussion_r211801302 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/subquery.scala --- @@ -137,13 +137,21 @@ object RewritePredicateSubquery

[GitHub] spark issue #20345: [SPARK-23172][SQL] Expand the ReorderJoin rule to handle...

2018-08-21 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/20345 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark pull request #22154: [SPARK-23711][SPARK-25140][SQL] Catch correct exc...

2018-08-21 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/22154#discussion_r211792988 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/CodeGeneratorWithInterpretedFallback.scala --- @@ -63,7 +49,10 @@ abstract

[GitHub] spark pull request #22154: [SPARK-23711][SPARK-25140][SQL] Catch correct exc...

2018-08-21 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/22154#discussion_r211787035 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/CodeGeneratorWithInterpretedFallback.scala --- @@ -63,7 +49,10 @@ abstract

[GitHub] spark issue #22154: [SPARK-23711][SPARK-25140][SQL] Catch correct exceptions...

2018-08-21 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/22154 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #22153: [SPARK-23034][SQL] Show RDD/relation names in RDD/In-Mem...

2018-08-21 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/22153 @gatorsmile @cloud-fan --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark issue #22163: [SPARK-25166][CORE]Reduce the number of write operations...

2018-08-21 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/22163 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark pull request #22168: [SPARK-24985][SQL][WIP] Fix OOM in Full Outer Joi...

2018-08-21 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/22168#discussion_r211577003 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/joins/SortMergeJoinExec.scala --- @@ -1099,7 +,7 @@ private class

[GitHub] spark issue #19691: [SPARK-14922][SPARK-17732][SQL]ALTER TABLE DROP PARTITIO...

2018-08-21 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/19691 @DazhuangSu Can you resolve the conflict? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22154: [SPARK-23711][SPARK-25140][SQL] Catch correct exceptions...

2018-08-21 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/22154 @rednaxelafx Thanks for your checks ;) addressed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

[GitHub] spark pull request #22154: [SPARK-23711][SPARK-25140][SQL] Catch correct exc...

2018-08-21 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/22154#discussion_r211565872 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/CodeGeneratorWithInterpretedFallbackSuite.scala --- @@ -40,4 +55,13

[GitHub] spark pull request #22154: [SPARK-23711][SPARK-25140][SQL] Catch correct exc...

2018-08-21 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/22154#discussion_r211565120 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/CodeGeneratorWithInterpretedFallbackSuite.scala --- @@ -40,4 +55,13

[GitHub] spark pull request #17400: [SPARK-19981][SQL] Respect aliases in output part...

2018-08-21 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/17400#discussion_r211561522 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/exchange/EnsureRequirements.scala --- @@ -321,6 +321,58 @@ case class EnsureRequirements

[GitHub] spark issue #22153: [SPARK-23034][SQL] Show RDD/relation names in RDD/In-Mem...

2018-08-21 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/22153 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark pull request #22154: [SPARK-23711][SPARK-25140][SQL] Catch correct exc...

2018-08-20 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/22154#discussion_r211451705 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/Projection.scala --- @@ -180,7 +180,10 @@ object UnsafeProjection

[GitHub] spark issue #20345: [SPARK-23172][SQL] Expand the ReorderJoin rule to handle...

2018-08-20 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/20345 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark pull request #22154: [SPARK-23711][SPARK-25140][SQL] Catch correct exc...

2018-08-20 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/22154#discussion_r211435227 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/CodeGeneratorWithInterpretedFallback.scala --- @@ -63,7 +49,10 @@ abstract

[GitHub] spark issue #22153: [SPARK-23034][SQL] Show RDD/relation names in RDD/In-Mem...

2018-08-20 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/22153 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #22154: [SPARK-23711][SPARK-25140][SQL] Catch correct exceptions...

2018-08-20 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/22154 cc: @gatorsmile @cloud-fan @viirya @rednaxelafx --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22154: [SPARK-23711][SPARK-25140][SQL] Catch correct exceptions...

2018-08-20 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/22154 IIUC `CacheLoader` throws `ExecutionException ` when `CodeGenerator.doCompile` throws `InternalCompilerException` or `CompileException`; https://github.com/apache/spark/blob/master/sql

[GitHub] spark pull request #22154: [SPARK-23711][SPARK-25140][SQL] Catch correct exc...

2018-08-20 Thread maropu
GitHub user maropu opened a pull request: https://github.com/apache/spark/pull/22154 [SPARK-23711][SPARK-25140][SQL] Catch correct exceptions when expr codegen fails ## What changes were proposed in this pull request? This pr is to fix bugs when expr codegen fails; we need

[GitHub] spark pull request #22153: [SPARK-23034][SQL] Show RDD/relation names in RDD...

2018-08-20 Thread maropu
GitHub user maropu opened a pull request: https://github.com/apache/spark/pull/22153 [SPARK-23034][SQL] Show RDD/relation names in RDD/In-Memory/Hive table scan nodes ## What changes were proposed in this pull request? This pr proposed to show RDD/relation names in RDD

[GitHub] spark issue #20226: [SPARK-23034][SQL] Override `nodeName` for all *ScanExec...

2018-08-17 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/20226 sure, will do, too. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #17400: [SPARK-19981][SQL] Update output partitioning info. when...

2018-08-15 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/17400 ok, thanks. I'll resume this work after the freeze. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

[GitHub] spark issue #17400: [SPARK-19981][SQL] Update output partitioning info. when...

2018-08-14 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/17400 If possible, could you describe that problem in your case to encourage this work? --- - To unsubscribe, e-mail: reviews-unsubscr

[GitHub] spark issue #17400: [SPARK-19981][SQL] Update output partitioning info. when...

2018-08-14 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/17400 I think that's because the priority is not much high. This issue causes any problem in your query? --- - To unsubscribe, e-mail

[GitHub] spark pull request #22008: [SPARK-24928][SQL] Optimize cross join according ...

2018-08-08 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/22008#discussion_r208802495 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -158,8 +158,9 @@ abstract class Optimizer

[GitHub] spark pull request #22036: [SPARK-25028][SQL] Avoid NPE when analyzing parti...

2018-08-08 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/22036#discussion_r208795446 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/StatisticsCollectionSuite.scala --- @@ -204,6 +204,24 @@ class StatisticsCollectionSuite extends

[GitHub] spark issue #21860: [SPARK-24901][SQL]Merge the codegen of RegularHashMap an...

2018-08-08 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21860 btw, we still need the impl. of the vectorized hash map (the comment says this is for test and benchmark only) in future releases? @hvanhovell @cloud-fan

[GitHub] spark pull request #21860: [SPARK-24901][SQL]Merge the codegen of RegularHas...

2018-08-08 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21860#discussion_r208788502 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/WholeStageCodegenSuite.scala --- @@ -232,6 +232,25 @@ class WholeStageCodegenSuite extends

[GitHub] spark issue #21868: [SPARK-24906][SQL] Adaptively enlarge split / partition ...

2018-08-08 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21868 Is this a parquet-specific issue? e.g., how about ORC? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

[GitHub] spark issue #22018: [SPARK-25038][SQL] Accelerate Spark Plan generation when...

2018-08-08 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/22018 Can you narrow down the title and description? I thinks the current one is obscure.. --- - To unsubscribe, e-mail: reviews

[GitHub] spark pull request #22018: [SPARK-25038][SQL] Accelerate Spark Plan generati...

2018-08-08 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/22018#discussion_r208783652 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/InMemoryFileIndex.scala --- @@ -297,7 +297,7 @@ object InMemoryFileIndex

[GitHub] spark pull request #21860: [SPARK-24901][SQL]Merge the codegen of RegularHas...

2018-08-06 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21860#discussion_r207809333 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/WholeStageCodegenSuite.scala --- @@ -232,6 +232,23 @@ class WholeStageCodegenSuite extends

[GitHub] spark issue #21931: [SPARK-24978][SQL]Add spark.sql.fast.hash.aggregate.row....

2018-08-06 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21931 cc: @cloud-fan @hvanhovell --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark pull request #21931: [SPARK-24978][SQL]Add spark.sql.fast.hash.aggrega...

2018-08-06 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21931#discussion_r207802603 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -1437,6 +1437,15 @@ object SQLConf { .intConf

[GitHub] spark pull request #21102: [SPARK-23913][SQL] Add array_intersect function

2018-08-05 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21102#discussion_r207758427 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/DataFrameFunctionsSuite.scala --- @@ -1647,6 +1647,60 @@ class DataFrameFunctionsSuite extends

[GitHub] spark issue #21608: [SPARK-24626] [SQL] Improve location size calculation in...

2018-08-05 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21608 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #22002: [FOLLOW-UP][SPARK-23772][SQL] Provide an option to ignor...

2018-08-05 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/22002 LGTM cc: @HyukjinKwon --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark pull request #22002: [FOLLOW-UP][SPARK-23772][SQL] Provide an option t...

2018-08-05 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/22002#discussion_r207754359 --- Diff: python/pyspark/sql/readwriter.py --- @@ -267,7 +267,7 @@ def json(self, path, schema=None, primitivesAsString=None, prefersDecimal=None

[GitHub] spark issue #20345: [SPARK-23172][SQL] Expand the ReorderJoin rule to handle...

2018-08-03 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/20345 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #21668: [SPARK-24690][SQL] Add a new config to control plan stat...

2018-08-02 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21668 @cloud-fan ping --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #20345: [SPARK-23172][SQL] Expand the ReorderJoin rule to handle...

2018-08-02 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/20345 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #21964: [SPARK-24788][SQL] RelationalGroupedDataset.toString wit...

2018-08-02 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21964 @gatorsmile ping --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #21933: [SPARK-24917][CORE] make chunk size configurable

2018-08-02 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21933 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #21964: [SPARK-24788][SQL] RelationalGroupedDataset.toString wit...

2018-08-02 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21964 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark pull request #21754: [SPARK-24705][SQL] ExchangeCoordinator broken whe...

2018-08-02 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21754#discussion_r207207259 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/exchange/ExchangeCoordinator.scala --- @@ -83,16 +83,17 @@ import

[GitHub] spark issue #21969: [SPARK-24945][SQL] Switching to uniVocity 2.7.3

2018-08-02 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21969 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark pull request #21754: [SPARK-24705][SQL] ExchangeCoordinator broken whe...

2018-08-02 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21754#discussion_r207195382 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/exchange/ExchangeCoordinator.scala --- @@ -83,16 +83,17 @@ import

[GitHub] spark pull request #21754: [SPARK-24705][SQL] ExchangeCoordinator broken whe...

2018-08-02 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21754#discussion_r207192928 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/exchange/ExchangeCoordinator.scala --- @@ -117,10 +118,6 @@ class ExchangeCoordinator

[GitHub] spark issue #21964: [SPARK-24788][SQL] RelationalGroupedDataset.toString wit...

2018-08-02 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21964 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #21931: [SPARK-24978][SQL]Add spark.sql.fast.hash.aggregate.row....

2018-08-02 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21931 What does the benchmark result suggest? The result should be `1048576` by default? --- - To unsubscribe, e-mail: reviews

[GitHub] spark issue #21754: [SPARK-24705][SQL] ExchangeCoordinator broken when dupli...

2018-08-02 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21754 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark pull request #21754: [SPARK-24705][SQL] ExchangeCoordinator broken whe...

2018-08-02 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21754#discussion_r207178639 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/exchange/ExchangeCoordinator.scala --- @@ -83,16 +83,17 @@ import

[GitHub] spark pull request #21944: [SPARK-24988][SQL]Add a castBySchema method which...

2018-08-02 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21944#discussion_r207160479 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala --- @@ -1367,6 +1367,22 @@ class Dataset[T] private[sql

[GitHub] spark pull request #21957: [SPARK-24994][SQL] When the data type of the fiel...

2018-08-02 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21957#discussion_r207155086 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSourceStrategy.scala --- @@ -436,8 +436,9 @@ object DataSourceStrategy

[GitHub] spark issue #21754: [SPARK-24705][SQL] ExchangeCoordinator broken when dupli...

2018-08-02 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21754 How about the fix based on [the suggestion](https://github.com/apache/spark/pull/21754/commits/f961760f64ceabd582bd78fd2b383f1405988816

[GitHub] spark issue #21754: [SPARK-24705][SQL] ExchangeCoordinator broken when dupli...

2018-08-02 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21754 yea, I think that is another approach to fix this issue. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

[GitHub] spark issue #21964: [SPARK-24788][SQL] RelationalGroupedDataset.toString wit...

2018-08-02 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21964 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #21941: [SPARK-24966][SQL] Implement precedence rules for set op...

2018-08-02 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21941 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #21941: [SPARK-24966][SQL] Implement precedence rules for set op...

2018-08-02 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21941 no idea, but `HiveClientSuites` seems flaky: https://issues.apache.org/jira/browse/SPARK-23622 (the error message is different though

[GitHub] spark pull request #21754: [SPARK-24705][SQL] ExchangeCoordinator broken whe...

2018-08-02 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21754#discussion_r207116871 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/ExchangeCoordinatorSuite.scala --- @@ -278,6 +278,25 @@ class ExchangeCoordinatorSuite

[GitHub] spark issue #21754: [SPARK-24705][SQL] ExchangeCoordinator broken when dupli...

2018-08-02 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21754 Oh, my bad. I just wanted to say; `EnsureRequirements ` sets `2` in ExchangeCoordinator, then the number changes from `2` to `1

[GitHub] spark issue #21892: [SPARK-24945][SQL] Switching to uniVocity 2.7.2

2018-08-02 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21892 Also, can you update the description? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands

[GitHub] spark issue #21963: [SPARK-21274][FOLLOW-UP][SQL] Enable support of MINUS AL...

2018-08-02 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21963 Probably, IMO we need a new jira for this. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21964: [SPARK-24788][SQL] RelationalGroupedDataset.toString wit...

2018-08-02 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21964 @gatorsmile `KeyValueGroupedDataset` has the same issue? It seems there is no chance for `KeyValueGroupedDataset` to have unresolved exprs. https://github.com/apache/spark/pull/21752

[GitHub] spark pull request #21964: [SPARK-24788][SQL] RelationalGroupedDataset.toStr...

2018-08-02 Thread maropu
GitHub user maropu opened a pull request: https://github.com/apache/spark/pull/21964 [SPARK-24788][SQL] RelationalGroupedDataset.toString with unresolved exprs should not fail ## What changes were proposed in this pull request? In the current master, `toString` throws

[GitHub] spark pull request #21754: [SPARK-24705][SQL] ExchangeCoordinator broken whe...

2018-08-01 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21754#discussion_r207106352 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/exchange/Exchange.scala --- @@ -52,6 +52,14 @@ case class ReusedExchangeExec(override val

[GitHub] spark pull request #21754: [SPARK-24705][SQL] Cannot reuse an exchange opera...

2018-08-01 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21754#discussion_r207105078 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/exchange/Exchange.scala --- @@ -52,6 +52,14 @@ case class ReusedExchangeExec(override val

<    1   2   3   4   5   6   7   8   9   10   >