[GitHub] spark pull request #21100: [SPARK-24012][SQL] Union of map and other compati...

2018-04-23 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/21100#discussion_r183601024 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/SQLQuerySuite.scala --- @@ -896,6 +896,25 @@ class SQLQuerySuite extends QueryTest with

[GitHub] spark pull request #21100: [SPARK-24012][SQL] Union of map and other compati...

2018-04-23 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/21100#discussion_r183601150 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/SQLQuerySuite.scala --- @@ -896,6 +896,25 @@ class SQLQuerySuite extends QueryTest with

[GitHub] spark issue #20937: [SPARK-23094][SPARK-23723][SPARK-23724][SQL] Support cus...

2018-04-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20937 **[Test build #89751 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89751/testReport)** for PR 20937 at commit

[GitHub] spark issue #20937: [SPARK-23094][SPARK-23723][SPARK-23724][SQL] Support cus...

2018-04-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20937 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89751/ Test PASSed. ---

[GitHub] spark issue #21136: [SPARK-24061][SS]Add TypedFilter support for continuous ...

2018-04-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21136 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #21137: [SPARK-23589][SQL][FOLLOW-UP] Reuse InternalRow i...

2018-04-23 Thread maropu
GitHub user maropu opened a pull request: https://github.com/apache/spark/pull/21137 [SPARK-23589][SQL][FOLLOW-UP] Reuse InternalRow in ExternalMapToCatalyst eval ## What changes were proposed in this pull request? This pr is a follow-up of #20980 and fixes code to reuse

[GitHub] spark issue #21137: [SPARK-23589][SQL][FOLLOW-UP] Reuse InternalRow in Exter...

2018-04-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21137 **[Test build #89760 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89760/testReport)** for PR 21137 at commit

[GitHub] spark pull request #21126: [SPARK-24050][SS] Calculate input / processing ra...

2018-04-23 Thread jose-torres
Github user jose-torres commented on a diff in the pull request: https://github.com/apache/spark/pull/21126#discussion_r183605577 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/streaming/StreamingQuerySuite.scala --- @@ -492,6 +492,77 @@ class StreamingQuerySuite extends

[GitHub] spark issue #21137: [SPARK-23589][SQL][FOLLOW-UP] Reuse InternalRow in Exter...

2018-04-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21137 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/2608/

[GitHub] spark issue #21135: [SPARK-24060][TEST] StreamingSymmetricHashJoinHelperSuit...

2018-04-23 Thread jose-torres
Github user jose-torres commented on the issue: https://github.com/apache/spark/pull/21135 LGTM, I think it's broadly correct for query nodes to assume the session has been initialized --- - To unsubscribe, e-mail:

[GitHub] spark issue #21137: [SPARK-23589][SQL][FOLLOW-UP] Reuse InternalRow in Exter...

2018-04-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21137 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #21123: [SPARK-24045][SQL]Create base class for file data...

2018-04-23 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/21123#discussion_r183589226 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/sources/v2/FileDataSourceV2Suite.scala --- @@ -0,0 +1,179 @@ +/* + * Licensed to the

[GitHub] spark pull request #21123: [SPARK-24045][SQL]Create base class for file data...

2018-04-23 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/21123#discussion_r183589146 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/internal/BaseSessionStateBuilder.scala --- @@ -158,6 +158,7 @@ abstract class

[GitHub] spark issue #21113: [MINOR][DOCS] Fix comments of SQLExecution#withExecution...

2018-04-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21113 **[Test build #89753 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89753/testReport)** for PR 21113 at commit

[GitHub] spark issue #21116: [SPARK-24038][SS] Refactor continuous writing to its own...

2018-04-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21116 **[Test build #89747 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89747/testReport)** for PR 21116 at commit

[GitHub] spark issue #21116: [SPARK-24038][SS] Refactor continuous writing to its own...

2018-04-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21116 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89747/ Test PASSed. ---

[GitHub] spark issue #21116: [SPARK-24038][SS] Refactor continuous writing to its own...

2018-04-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21116 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21127: [SPARK-24052][CORE][UI] Add spark version information on...

2018-04-23 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/21127 Version info is already available, in the code and in the UI. "Compiled by" info has never struck me as useful. The rest is from the env. I don't think this adds anything. ---

[GitHub] spark issue #21127: [SPARK-24052][CORE][UI] Add spark version information on...

2018-04-23 Thread caneGuy
Github user caneGuy commented on the issue: https://github.com/apache/spark/pull/21127 How about the other information? As mentioned,the build info @vanzin --- - To unsubscribe, e-mail:

[GitHub] spark issue #20923: [SPARK-23807][BUILD] Add Hadoop 3.1 profile with relevan...

2018-04-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20923 **[Test build #89748 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89748/testReport)** for PR 20923 at commit

[GitHub] spark issue #21078: [SPARK-23990][ML] Instruments logging improvements - ML ...

2018-04-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21078 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21078: [SPARK-23990][ML] Instruments logging improvements - ML ...

2018-04-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21078 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89754/ Test PASSed. ---

[GitHub] spark pull request #21073: [SPARK-23936][SQL] Implement map_concat

2018-04-23 Thread bersprockets
Github user bersprockets commented on a diff in the pull request: https://github.com/apache/spark/pull/21073#discussion_r183600236 --- Diff: python/pyspark/sql/functions.py --- @@ -2186,6 +2186,29 @@ def map_values(col): return

[GitHub] spark issue #21078: [SPARK-23990][ML] Instruments logging improvements - ML ...

2018-04-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21078 **[Test build #89754 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89754/testReport)** for PR 21078 at commit

[GitHub] spark issue #21136: [SPARK-24061][SS]Add TypedFilter support for continuous ...

2018-04-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21136 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #21136: [SPARK-24061][SS]Add TypedFilter support for cont...

2018-04-23 Thread xuanyuanking
Github user xuanyuanking commented on a diff in the pull request: https://github.com/apache/spark/pull/21136#discussion_r183604217 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/analysis/UnsupportedOperationsSuite.scala --- @@ -771,7 +778,16 @@ class

[GitHub] spark pull request #21100: [SPARK-24012][SQL] Union of map and other compati...

2018-04-23 Thread liutang123
Github user liutang123 commented on a diff in the pull request: https://github.com/apache/spark/pull/21100#discussion_r183608838 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/SQLQuerySuite.scala --- @@ -896,6 +896,25 @@ class SQLQuerySuite extends QueryTest with

[GitHub] spark pull request #21123: [SPARK-24045][SQL]Create base class for file data...

2018-04-23 Thread gengliangwang
Github user gengliangwang commented on a diff in the pull request: https://github.com/apache/spark/pull/21123#discussion_r183611332 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/sources/v2/FileDataSourceV2Suite.scala --- @@ -0,0 +1,179 @@ +/* + * Licensed to the

[GitHub] spark pull request #21123: [SPARK-24045][SQL]Create base class for file data...

2018-04-23 Thread gengliangwang
Github user gengliangwang commented on a diff in the pull request: https://github.com/apache/spark/pull/21123#discussion_r183611071 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/sources/v2/FileDataSourceV2Suite.scala --- @@ -0,0 +1,179 @@ +/* + * Licensed to the

[GitHub] spark issue #21113: [MINOR][DOCS] Fix comments of SQLExecution#withExecution...

2018-04-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21113 **[Test build #4155 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4155/testReport)** for PR 21113 at commit

[GitHub] spark issue #20923: [SPARK-23807][BUILD] Add Hadoop 3.1 profile with relevan...

2018-04-23 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/20923 +1 for @jerryshao 's comment. Some of Hive UTs will fail with Hadoop 3 profile. --- - To unsubscribe, e-mail:

[GitHub] spark pull request #21123: [SPARK-24045][SQL]Create base class for file data...

2018-04-23 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/21123#discussion_r183589841 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/sources/v2/FileDataSourceV2Suite.scala --- @@ -0,0 +1,179 @@ +/* + * Licensed to the

[GitHub] spark pull request #21123: [SPARK-24045][SQL]Create base class for file data...

2018-04-23 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/21123#discussion_r183589976 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/sources/v2/FileDataSourceV2Suite.scala --- @@ -0,0 +1,179 @@ +/* + * Licensed to the

[GitHub] spark issue #21129: [SPARK-7132][ML] Add fit with validation set to spark.ml...

2018-04-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21129 **[Test build #89756 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89756/testReport)** for PR 21129 at commit

[GitHub] spark issue #21129: [SPARK-7132][ML] Add fit with validation set to spark.ml...

2018-04-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21129 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21129: [SPARK-7132][ML] Add fit with validation set to spark.ml...

2018-04-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21129 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/2607/

[GitHub] spark issue #21100: [SPARK-24012][SQL] Union of map and other compatible col...

2018-04-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21100 **[Test build #89757 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89757/testReport)** for PR 21100 at commit

[GitHub] spark issue #20940: [SPARK-23429][CORE] Add executor memory metrics to heart...

2018-04-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20940 **[Test build #89758 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89758/testReport)** for PR 20940 at commit

[GitHub] spark pull request #21135: [SPARK-24060][TEST] StreamingSymmetricHashJoinHel...

2018-04-23 Thread pwoody
GitHub user pwoody opened a pull request: https://github.com/apache/spark/pull/21135 [SPARK-24060][TEST] StreamingSymmetricHashJoinHelperSuite should initialize after SparkSession creation ## What changes were proposed in this pull request? We should ensure that the

[GitHub] spark pull request #21073: [SPARK-23936][SQL] Implement map_concat

2018-04-23 Thread bersprockets
Github user bersprockets commented on a diff in the pull request: https://github.com/apache/spark/pull/21073#discussion_r18365 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/CollectionExpressionsSuite.scala --- @@ -56,6 +58,26 @@ class

[GitHub] spark issue #20937: [SPARK-23094][SPARK-23723][SPARK-23724][SQL] Support cus...

2018-04-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20937 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #21136: [SPARK-24061][SS]Add TypedFilter support for cont...

2018-04-23 Thread yanlin-Lynn
GitHub user yanlin-Lynn opened a pull request: https://github.com/apache/spark/pull/21136 [SPARK-24061][SS]Add TypedFilter support for continuous processing ## What changes were proposed in this pull request? Add TypedFilter support for continuous processing application.

[GitHub] spark issue #21136: [SPARK-24061][SS]Add TypedFilter support for continuous ...

2018-04-23 Thread yanlin-Lynn
Github user yanlin-Lynn commented on the issue: https://github.com/apache/spark/pull/21136 @xuanyuanking , please help to review for this path. Thank you! --- - To unsubscribe, e-mail:

[GitHub] spark issue #20998: [SPARK-23888][CORE] correct the comment of hasAttemptOnH...

2018-04-23 Thread Ngone51
Github user Ngone51 commented on the issue: https://github.com/apache/spark/pull/20998 Agree and thank you @squito . And thanks for all of you. @felixcheung @mridulm @jiangxb1987 @srowen --- - To

[GitHub] spark issue #20146: [SPARK-11215][ML] Add multiple columns support to String...

2018-04-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20146 **[Test build #89761 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89761/testReport)** for PR 20146 at commit

[GitHub] spark issue #19887: [SPARK-21168] KafkaRDD should always set kafka clientId.

2018-04-23 Thread koeninger
Github user koeninger commented on the issue: https://github.com/apache/spark/pull/19887 Seems ok to me, long as it passes retest --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19887: [SPARK-21168] KafkaRDD should always set kafka clientId.

2018-04-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19887 **[Test build #89733 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89733/testReport)** for PR 19887 at commit

[GitHub] spark issue #21113: [MINOR][DOCS] Fix comments of SQLExecution#withExecution...

2018-04-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21113 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21113: [MINOR][DOCS] Fix comments of SQLExecution#withExecution...

2018-04-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21113 **[Test build #89730 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89730/testReport)** for PR 21113 at commit

[GitHub] spark issue #21113: [MINOR][DOCS] Fix comments of SQLExecution#withExecution...

2018-04-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21113 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89730/ Test FAILed. ---

[GitHub] spark issue #21123: [SPARK-24045][SQL]Create base class for file data source...

2018-04-23 Thread gengliangwang
Github user gengliangwang commented on the issue: https://github.com/apache/spark/pull/21123 retest this please. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20998: [SPARK-23888][CORE] correct the comment of hasAttemptOnH...

2018-04-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20998 **[Test build #89728 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89728/testReport)** for PR 20998 at commit

[GitHub] spark issue #20998: [SPARK-23888][CORE] correct the comment of hasAttemptOnH...

2018-04-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20998 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89728/ Test PASSed. ---

[GitHub] spark issue #21123: [SPARK-24045][SQL]Create base class for file data source...

2018-04-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21123 **[Test build #89734 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89734/testReport)** for PR 21123 at commit

[GitHub] spark issue #20998: [SPARK-23888][CORE] correct the comment of hasAttemptOnH...

2018-04-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20998 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21123: [SPARK-24045][SQL]Create base class for file data source...

2018-04-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21123 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/2596/

[GitHub] spark issue #21123: [SPARK-24045][SQL]Create base class for file data source...

2018-04-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21123 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19887: [SPARK-21168] KafkaRDD should always set kafka clientId.

2018-04-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19887 **[Test build #89733 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89733/testReport)** for PR 19887 at commit

[GitHub] spark issue #19887: [SPARK-21168] KafkaRDD should always set kafka clientId.

2018-04-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19887 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89733/ Test PASSed. ---

[GitHub] spark issue #19887: [SPARK-21168] KafkaRDD should always set kafka clientId.

2018-04-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19887 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19887: [SPARK-21168] KafkaRDD should always set kafka clientId.

2018-04-23 Thread koeninger
Github user koeninger commented on the issue: https://github.com/apache/spark/pull/19887 Merging to master, thanks @liu-zhaokun --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21128: [SPARK-24053][CORE] Support add subdirectory named as us...

2018-04-23 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/21128 Moreover, you can set `spark.yarn.stagingDir` to whatever you want, and may even reference env variables or system properties. e.g. ``` spark.yarn.stagingDir=/tmp/${system:user.name}

<    1   2   3   4   5   6