[GitHub] spark issue #19848: [SPARK-22162] Executors and the driver should use consis...

2017-12-14 Thread squito
Github user squito commented on the issue: https://github.com/apache/spark/pull/19848 i dunno what the requirements are -- I was hoping you would know the hadoop committer semantics better than me! I suppose a uuid is really the only get something globally unique, as you could even

[GitHub] spark issue #19971: [SPARK-22774][SQL][Test] Add compilation check into TPCD...

2017-12-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19971 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19971: [SPARK-22774][SQL][Test] Add compilation check into TPCD...

2017-12-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19971 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84913/ Test PASSed. ---

[GitHub] spark issue #19971: [SPARK-22774][SQL][Test] Add compilation check into TPCD...

2017-12-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19971 **[Test build #84913 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84913/testReport)** for PR 19971 at commit

[GitHub] spark pull request #19681: [SPARK-20652][sql] Store SQL UI data in the new a...

2017-12-14 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19681#discussion_r156987695 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/ui/SQLAppStatusStore.scala --- @@ -0,0 +1,179 @@ +/* + * Licensed to the Apache

[GitHub] spark issue #19792: [SPARK-22566][PYTHON] Better error message for `_merge_t...

2017-12-14 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/19792 Will take another look soon tomorrow. Sorry that is getting delayed again and again but I just realised this code path is a little bit tricky .. ---

[GitHub] spark issue #19975: [SPARK-22781][SS] Support creating streaming dataset wit...

2017-12-14 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/19975 Hi, @tdas and @zsxwing . Could you review this PR? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

[GitHub] spark issue #19792: [SPARK-22566][PYTHON] Better error message for `_merge_t...

2017-12-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19792 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19792: [SPARK-22566][PYTHON] Better error message for `_merge_t...

2017-12-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19792 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84918/ Test PASSed. ---

[GitHub] spark issue #19792: [SPARK-22566][PYTHON] Better error message for `_merge_t...

2017-12-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19792 **[Test build #84918 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84918/testReport)** for PR 19792 at commit

[GitHub] spark issue #19980: [SPARK-22785][SQL] remove ColumnVector.anyNullsSet

2017-12-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19980 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84912/ Test PASSed. ---

[GitHub] spark issue #19980: [SPARK-22785][SQL] remove ColumnVector.anyNullsSet

2017-12-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19980 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19980: [SPARK-22785][SQL] remove ColumnVector.anyNullsSet

2017-12-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19980 **[Test build #84912 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84912/testReport)** for PR 19980 at commit

[GitHub] spark issue #19976: [SPARK-22660][BUILD] Use position() and limit() to fix a...

2017-12-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19976 **[Test build #4011 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4011/testReport)** for PR 19976 at commit

[GitHub] spark issue #19976: [SPARK-22660][BUILD] Use position() and limit() to fix a...

2017-12-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19976 **[Test build #4010 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4010/testReport)** for PR 19976 at commit

[GitHub] spark issue #19977: [SPARK-22771][SQL] Concatenate binary inputs into a bina...

2017-12-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19977 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84910/ Test PASSed. ---

[GitHub] spark issue #19977: [SPARK-22771][SQL] Concatenate binary inputs into a bina...

2017-12-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19977 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19977: [SPARK-22771][SQL] Concatenate binary inputs into a bina...

2017-12-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19977 **[Test build #84910 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84910/testReport)** for PR 19977 at commit

[GitHub] spark issue #19811: [SPARK-18016][SQL] Code Generation: Constant Pool Limit ...

2017-12-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19811 **[Test build #84919 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84919/testReport)** for PR 19811 at commit

[GitHub] spark issue #19971: [SPARK-22774][SQL][Test] Add compilation check into TPCD...

2017-12-14 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/19971 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19980: [SPARK-22785][SQL] remove ColumnVector.anyNullsSet

2017-12-14 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/19980 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19792: [SPARK-22566][PYTHON] Better error message for `_merge_t...

2017-12-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19792 **[Test build #84918 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84918/testReport)** for PR 19792 at commit

[GitHub] spark issue #19971: [SPARK-22774][SQL][Test] Add compilation check into TPCD...

2017-12-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19971 **[Test build #84917 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84917/testReport)** for PR 19971 at commit

[GitHub] spark pull request #19792: [SPARK-22566][PYTHON] Better error message for `_...

2017-12-14 Thread gberger
Github user gberger commented on a diff in the pull request: https://github.com/apache/spark/pull/19792#discussion_r156971910 --- Diff: python/pyspark/sql/types.py --- @@ -1083,7 +1083,11 @@ def _infer_schema(row): elif hasattr(row, "_fields"): # namedtuple

[GitHub] spark pull request #19971: [SPARK-22774][SQL][Test] Add compilation check in...

2017-12-14 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/19971#discussion_r156970863 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/TPCDSQuerySuite.scala --- @@ -348,13 +351,41 @@ class TPCDSQuerySuite extends QueryTest with

[GitHub] spark pull request #19861: [SPARK-22387][SQL] Propagate session configs to d...

2017-12-14 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19861#discussion_r156970195 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/sources/v2/DataSourceV2UtilsSuite.scala --- @@ -0,0 +1,43 @@ +/* + * Licensed to the

[GitHub] spark issue #19979: [SPARK-22644][ML][TEST][FOLLOW-UP] ML regression testsui...

2017-12-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19979 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84911/ Test PASSed. ---

[GitHub] spark issue #19979: [SPARK-22644][ML][TEST][FOLLOW-UP] ML regression testsui...

2017-12-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19979 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19979: [SPARK-22644][ML][TEST][FOLLOW-UP] ML regression testsui...

2017-12-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19979 **[Test build #84911 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84911/testReport)** for PR 19979 at commit

[GitHub] spark pull request #19861: [SPARK-22387][SQL] Propagate session configs to d...

2017-12-14 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19861#discussion_r156969856 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/DataSourceV2Utils.scala --- @@ -0,0 +1,54 @@ +/* + * Licensed to

[GitHub] spark issue #19811: [SPARK-18016][SQL] Code Generation: Constant Pool Limit ...

2017-12-14 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/19811 Another use case is for frequently-executed variable (e.g. row variable) for performance. We want to keep it as non-array variable. ---

[GitHub] spark pull request #19861: [SPARK-22387][SQL] Propagate session configs to d...

2017-12-14 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19861#discussion_r156969297 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/DataFrameWriter.scala --- @@ -238,7 +239,16 @@ final class DataFrameWriter[T] private[sql](ds:

[GitHub] spark pull request #19811: [SPARK-18016][SQL] Code Generation: Constant Pool...

2017-12-14 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/19811#discussion_r156968572 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala --- @@ -138,21 +138,50 @@ class CodegenContext {

[GitHub] spark pull request #19971: [SPARK-22774][SQL][Test] Add compilation check in...

2017-12-14 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19971#discussion_r156968031 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/TPCDSQuerySuite.scala --- @@ -348,13 +351,41 @@ class TPCDSQuerySuite extends QueryTest with

[GitHub] spark pull request #19811: [SPARK-18016][SQL] Code Generation: Constant Pool...

2017-12-14 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/19811#discussion_r156967829 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/DataSourceScanExec.scala --- @@ -353,8 +353,8 @@ case class FileSourceScanExec( }

[GitHub] spark pull request #19971: [SPARK-22774][SQL][Test] Add compilation check in...

2017-12-14 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19971#discussion_r156967793 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/TPCDSQuerySuite.scala --- @@ -348,13 +351,41 @@ class TPCDSQuerySuite extends QueryTest with

[GitHub] spark pull request #19811: [SPARK-18016][SQL] Code Generation: Constant Pool...

2017-12-14 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/19811#discussion_r156967588 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/DataSourceScanExec.scala --- @@ -110,8 +110,8 @@ case class RowDataSourceScanExec(

[GitHub] spark pull request #19811: [SPARK-18016][SQL] Code Generation: Constant Pool...

2017-12-14 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/19811#discussion_r156967475 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/ColumnarBatchScan.scala --- @@ -68,30 +68,26 @@ private[sql] trait ColumnarBatchScan

[GitHub] spark issue #19811: [SPARK-18016][SQL] Code Generation: Constant Pool Limit ...

2017-12-14 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/19811 My major concern is how to reason about when we need to force inline. I can understand the use case of keeping the variable name unchanged(e.g. `MapObjects`), but what about other cases? ---

[GitHub] spark pull request #19811: [SPARK-18016][SQL] Code Generation: Constant Pool...

2017-12-14 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/19811#discussion_r156966971 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/objects/objects.scala --- @@ -1205,14 +1203,14 @@ case class

[GitHub] spark pull request #19811: [SPARK-18016][SQL] Code Generation: Constant Pool...

2017-12-14 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19811#discussion_r156967021 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala --- @@ -138,21 +138,50 @@ class

[GitHub] spark pull request #19811: [SPARK-18016][SQL] Code Generation: Constant Pool...

2017-12-14 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/19811#discussion_r156966787 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/objects/objects.scala --- @@ -1159,14 +1158,14 @@ case class

[GitHub] spark issue #19751: [SPARK-20653][core] Add cleaning of old elements from th...

2017-12-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19751 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19751: [SPARK-20653][core] Add cleaning of old elements from th...

2017-12-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19751 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84909/ Test PASSed. ---

[GitHub] spark pull request #19811: [SPARK-18016][SQL] Code Generation: Constant Pool...

2017-12-14 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19811#discussion_r156966742 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala --- @@ -163,11 +192,49 @@ class

[GitHub] spark issue #19751: [SPARK-20653][core] Add cleaning of old elements from th...

2017-12-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19751 **[Test build #84909 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84909/testReport)** for PR 19751 at commit

[GitHub] spark issue #18692: [SPARK-21417][SQL] Infer join conditions using propagate...

2017-12-14 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/18692 you are right, then I don't know if there is any valid use case for inferring join condition from literals... --- - To

[GitHub] spark pull request #19811: [SPARK-18016][SQL] Code Generation: Constant Pool...

2017-12-14 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/19811#discussion_r156964736 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/datetimeExpressions.scala --- @@ -484,18 +484,17 @@ case class

[GitHub] spark issue #19941: [SPARK-22753][SQL] Get rid of dataSource.writeAndRead

2017-12-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19941 **[Test build #84916 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84916/testReport)** for PR 19941 at commit

[GitHub] spark pull request #19941: [SPARK-22753][SQL] Get rid of dataSource.writeAnd...

2017-12-14 Thread xuanyuanking
Github user xuanyuanking commented on a diff in the pull request: https://github.com/apache/spark/pull/19941#discussion_r156960449 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSource.scala --- @@ -490,20 +489,19 @@ case class DataSource(

[GitHub] spark pull request #19811: [SPARK-18016][SQL] Code Generation: Constant Pool...

2017-12-14 Thread mgaido91
Github user mgaido91 commented on a diff in the pull request: https://github.com/apache/spark/pull/19811#discussion_r156959928 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala --- @@ -138,21 +138,50 @@ class

[GitHub] spark pull request #19941: [SPARK-22753][SQL] Get rid of dataSource.writeAnd...

2017-12-14 Thread xuanyuanking
Github user xuanyuanking commented on a diff in the pull request: https://github.com/apache/spark/pull/19941#discussion_r156958793 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/InsertIntoDataSourceDirCommand.scala --- @@ -67,8 +67,9 @@ case class

[GitHub] spark issue #19893: [SPARK-16139][TEST] Add logging functionality for leaked...

2017-12-14 Thread gaborgsomogyi
Github user gaborgsomogyi commented on the issue: https://github.com/apache/spark/pull/19893 The last suspicious big group of threads (at least for me) is broadcast-exchange.* but as I've seen this is not false positive because the threadpool never stopped. In

[GitHub] spark pull request #19811: [SPARK-18016][SQL] Code Generation: Constant Pool...

2017-12-14 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/19811#discussion_r156954383 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala --- @@ -138,21 +138,50 @@ class CodegenContext {

[GitHub] spark pull request #19811: [SPARK-18016][SQL] Code Generation: Constant Pool...

2017-12-14 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/19811#discussion_r156953610 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/codegen/GeneratedProjectionSuite.scala --- @@ -219,4 +219,31 @@ class

[GitHub] spark pull request #19811: [SPARK-18016][SQL] Code Generation: Constant Pool...

2017-12-14 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/19811#discussion_r156953675 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala --- @@ -163,11 +192,49 @@ class CodegenContext {

[GitHub] spark issue #19980: [SPARK-22785][SQL] remove ColumnVector.anyNullsSet

2017-12-14 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/19980 LGTM pending Jenkins. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19893: [SPARK-16139][TEST] Add logging functionality for leaked...

2017-12-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19893 **[Test build #84915 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84915/testReport)** for PR 19893 at commit

[GitHub] spark issue #19893: [SPARK-16139][TEST] Add logging functionality for leaked...

2017-12-14 Thread gaborgsomogyi
Github user gaborgsomogyi commented on the issue: https://github.com/apache/spark/pull/19893 Seems like the new feature caught some false positives in SQL: ``` = THREAD AUDIT POST ACTION CALLED WITHOUT PRE ACTION IN SUITE o.a.s.sql.sources.DataSourceAnalysisSuite

[GitHub] spark issue #19893: [SPARK-16139][TEST] Add logging functionality for leaked...

2017-12-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19893 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84907/ Test PASSed. ---

[GitHub] spark issue #19893: [SPARK-16139][TEST] Add logging functionality for leaked...

2017-12-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19893 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19893: [SPARK-16139][TEST] Add logging functionality for leaked...

2017-12-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19893 **[Test build #84907 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84907/testReport)** for PR 19893 at commit

[GitHub] spark issue #19861: [SPARK-22387][SQL] Propagate session configs to data sou...

2017-12-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19861 **[Test build #84914 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84914/testReport)** for PR 19861 at commit

[GitHub] spark issue #19971: [SPARK-22774][SQL][Test] Add compilation check into TPCD...

2017-12-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19971 **[Test build #84913 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84913/testReport)** for PR 19971 at commit

[GitHub] spark pull request #19971: [SPARK-22774][SQL][Test] Add compilation check in...

2017-12-14 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/19971#discussion_r156945302 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/TPCDSQuerySuite.scala --- @@ -29,7 +32,7 @@ import org.apache.spark.util.Utils * This test

[GitHub] spark pull request #19811: [SPARK-18016][SQL] Code Generation: Constant Pool...

2017-12-14 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/19811#discussion_r156943716 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/ColumnarBatchScan.scala --- @@ -68,30 +68,26 @@ private[sql] trait ColumnarBatchScan

[GitHub] spark pull request #19792: [SPARK-22566][PYTHON] Better error message for `_...

2017-12-14 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/19792#discussion_r156941615 --- Diff: python/pyspark/sql/types.py --- @@ -1083,7 +1083,11 @@ def _infer_schema(row): elif hasattr(row, "_fields"): # namedtuple

[GitHub] spark issue #18692: [SPARK-21417][SQL] Infer join conditions using propagate...

2017-12-14 Thread aokolnychyi
Github user aokolnychyi commented on the issue: https://github.com/apache/spark/pull/18692 I am not sure we can infer ``a == b`` if ``a in (0, 2, 3, 4)`` and ``b in (0, 2, 3, 4)``. table 'a' ``` a1 a2 1 2 3 3 4 5 ``` table 'b' ```

[GitHub] spark pull request #19811: [SPARK-18016][SQL] Code Generation: Constant Pool...

2017-12-14 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/19811#discussion_r156941474 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala --- @@ -163,11 +192,49 @@ class CodegenContext {

[GitHub] spark pull request #19811: [SPARK-18016][SQL] Code Generation: Constant Pool...

2017-12-14 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/19811#discussion_r156940683 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/codegen/GeneratedProjectionSuite.scala --- @@ -219,4 +219,31 @@ class

[GitHub] spark issue #19980: [SPARK-22785][SQL] remove ColumnVector.anyNullsSet

2017-12-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19980 **[Test build #84912 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84912/testReport)** for PR 19980 at commit

[GitHub] spark pull request #19980: [SPARK-22785][SQL] remove ColumnVector.anyNullsSe...

2017-12-14 Thread cloud-fan
GitHub user cloud-fan opened a pull request: https://github.com/apache/spark/pull/19980 [SPARK-22785][SQL] remove ColumnVector.anyNullsSet ## What changes were proposed in this pull request? `ColumnVector.anyNullsSet` is not called anywhere except tests, and we can easily

[GitHub] spark issue #19980: [SPARK-22785][SQL] remove ColumnVector.anyNullsSet

2017-12-14 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/19980 cc @ueshin @kiszk @gatorsmile --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark pull request #19811: [SPARK-18016][SQL] Code Generation: Constant Pool...

2017-12-14 Thread mgaido91
Github user mgaido91 commented on a diff in the pull request: https://github.com/apache/spark/pull/19811#discussion_r156935514 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala --- @@ -138,21 +138,50 @@ class

[GitHub] spark pull request #19974: [SPARK-22779][sql] Resolve default values for fal...

2017-12-14 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19974#discussion_r156935029 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/internal/SQLConfSuite.scala --- @@ -280,4 +280,34 @@ class SQLConfSuite extends QueryTest with

[GitHub] spark pull request #19811: [SPARK-18016][SQL] Code Generation: Constant Pool...

2017-12-14 Thread mgaido91
Github user mgaido91 commented on a diff in the pull request: https://github.com/apache/spark/pull/19811#discussion_r156932945 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/limit.scala --- @@ -71,17 +71,15 @@ trait BaseLimitExec extends UnaryExecNode with

[GitHub] spark pull request #19811: [SPARK-18016][SQL] Code Generation: Constant Pool...

2017-12-14 Thread mgaido91
Github user mgaido91 commented on a diff in the pull request: https://github.com/apache/spark/pull/19811#discussion_r156932923 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/limit.scala --- @@ -71,17 +71,15 @@ trait BaseLimitExec extends UnaryExecNode with

[GitHub] spark pull request #19811: [SPARK-18016][SQL] Code Generation: Constant Pool...

2017-12-14 Thread mgaido91
Github user mgaido91 commented on a diff in the pull request: https://github.com/apache/spark/pull/19811#discussion_r156932789 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala --- @@ -163,11 +192,49 @@ class

[GitHub] spark pull request #19811: [SPARK-18016][SQL] Code Generation: Constant Pool...

2017-12-14 Thread mgaido91
Github user mgaido91 commented on a diff in the pull request: https://github.com/apache/spark/pull/19811#discussion_r156932214 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/ColumnarBatchScan.scala --- @@ -68,30 +68,26 @@ private[sql] trait ColumnarBatchScan

[GitHub] spark pull request #19811: [SPARK-18016][SQL] Code Generation: Constant Pool...

2017-12-14 Thread mgaido91
Github user mgaido91 commented on a diff in the pull request: https://github.com/apache/spark/pull/19811#discussion_r156932172 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/ColumnarBatchScan.scala --- @@ -68,30 +68,26 @@ private[sql] trait ColumnarBatchScan

[GitHub] spark issue #19976: [SPARK-22660][BUILD] Use position() and limit() to fix a...

2017-12-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19976 **[Test build #4010 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4010/testReport)** for PR 19976 at commit

[GitHub] spark issue #19978: [SPARK-22784][CORE] Configure reading buffer size in Spa...

2017-12-14 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/19978 That seems quite large, but I'm only speculating. If it defaulted to something that's just not tiny, would that help? 1M events? ---

[GitHub] spark pull request #19811: [SPARK-18016][SQL] Code Generation: Constant Pool...

2017-12-14 Thread mgaido91
Github user mgaido91 commented on a diff in the pull request: https://github.com/apache/spark/pull/19811#discussion_r156930498 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala --- @@ -189,15 +255,37 @@ class

[GitHub] spark pull request #19811: [SPARK-18016][SQL] Code Generation: Constant Pool...

2017-12-14 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19811#discussion_r156930403 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala --- @@ -163,11 +192,49 @@ class

[GitHub] spark issue #19971: [SPARK-22774][SQL][Test] Add compilation check into TPCD...

2017-12-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19971 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19971: [SPARK-22774][SQL][Test] Add compilation check into TPCD...

2017-12-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19971 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84906/ Test PASSed. ---

[GitHub] spark issue #19971: [SPARK-22774][SQL][Test] Add compilation check into TPCD...

2017-12-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19971 **[Test build #84906 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84906/testReport)** for PR 19971 at commit

[GitHub] spark issue #19979: [SPARK-22644][ML][TEST][FOLLOW-UP] ML regression testsui...

2017-12-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19979 **[Test build #84911 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84911/testReport)** for PR 19979 at commit

[GitHub] spark pull request #19979: [SPARK-22644][ML][TEST][FOLLOW-UP] ML regression ...

2017-12-14 Thread WeichenXu123
GitHub user WeichenXu123 opened a pull request: https://github.com/apache/spark/pull/19979 [SPARK-22644][ML][TEST][FOLLOW-UP] ML regression testsuite add StructuredStreaming test ## What changes were proposed in this pull request? ML regression testsuite add

[GitHub] spark issue #19978: [SPARK-22784][CORE] Configure reading buffer size in Spa...

2017-12-14 Thread MikhailErofeev
Github user MikhailErofeev commented on the issue: https://github.com/apache/spark/pull/19978 I don't mind to just set it to a higher value. Moreover, the current default value (2048) is small in any case. For my log files, 30M buffer was the best value (a bigger one did not

[GitHub] spark pull request #19811: [SPARK-18016][SQL] Code Generation: Constant Pool...

2017-12-14 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19811#discussion_r156928231 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/DataSourceScanExec.scala --- @@ -110,8 +110,8 @@ case class RowDataSourceScanExec(

[GitHub] spark pull request #19811: [SPARK-18016][SQL] Code Generation: Constant Pool...

2017-12-14 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19811#discussion_r156928270 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/DataSourceScanExec.scala --- @@ -353,8 +353,8 @@ case class FileSourceScanExec(

[GitHub] spark pull request #19811: [SPARK-18016][SQL] Code Generation: Constant Pool...

2017-12-14 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19811#discussion_r156928205 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/ColumnarBatchScan.scala --- @@ -68,30 +68,26 @@ private[sql] trait ColumnarBatchScan

[GitHub] spark pull request #19811: [SPARK-18016][SQL] Code Generation: Constant Pool...

2017-12-14 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19811#discussion_r156926848 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/codegen/GeneratedProjectionSuite.scala --- @@ -219,4 +219,31 @@ class

[GitHub] spark issue #19977: [SPARK-22771][SQL] Concatenate binary inputs into a bina...

2017-12-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19977 **[Test build #84910 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84910/testReport)** for PR 19977 at commit

[GitHub] spark pull request #19811: [SPARK-18016][SQL] Code Generation: Constant Pool...

2017-12-14 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19811#discussion_r156926555 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/codegen/GeneratedProjectionSuite.scala --- @@ -219,4 +219,31 @@ class

[GitHub] spark pull request #19811: [SPARK-18016][SQL] Code Generation: Constant Pool...

2017-12-14 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19811#discussion_r156926377 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/codegen/GeneratedProjectionSuite.scala --- @@ -219,4 +219,31 @@ class

[GitHub] spark pull request #19811: [SPARK-18016][SQL] Code Generation: Constant Pool...

2017-12-14 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19811#discussion_r156926146 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/CodeGenerationSuite.scala --- @@ -401,4 +401,26 @@ class

[GitHub] spark issue #14151: [SPARK-16496][SQL] Add wholetext as option for reading t...

2017-12-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14151 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84905/ Test PASSed. ---

[GitHub] spark issue #14151: [SPARK-16496][SQL] Add wholetext as option for reading t...

2017-12-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14151 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

<    1   2   3   4   5   6   7   8   >