[GitHub] spark issue #22173: [SPARK-24335] Spark external shuffle server improvement ...

2018-08-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22173 **[Test build #95499 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95499/testReport)** for PR 22173 at commit

[GitHub] spark issue #22138: [SPARK-25151][SS] Apply Apache Commons Pool to KafkaData...

2018-08-30 Thread HeartSaVioR
Github user HeartSaVioR commented on the issue: https://github.com/apache/spark/pull/22138 retest this, please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #22138: [SPARK-25151][SS] Apply Apache Commons Pool to KafkaData...

2018-08-30 Thread HeartSaVioR
Github user HeartSaVioR commented on the issue: https://github.com/apache/spark/pull/22138 Two tests failed and not relevant to this patch. * org.apache.spark.scheduler.DAGSchedulerSuite.Barrier task failures from the same stage attempt don't trigger multiple stage retries

[GitHub] spark pull request #21721: [SPARK-24748][SS] Support for reporting custom me...

2018-08-30 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/21721#discussion_r214231328 --- Diff: sql/core/src/main/java/org/apache/spark/sql/sources/v2/CustomMetrics.java --- @@ -0,0 +1,33 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request #22296: [SPARK-24748][SS][FOLLOWUP] Switch custom metrics...

2018-08-30 Thread HyukjinKwon
GitHub user HyukjinKwon opened a pull request: https://github.com/apache/spark/pull/22296 [SPARK-24748][SS][FOLLOWUP] Switch custom metrics to Unstable APIs ## What changes were proposed in this pull request? This PR proposes to switch the api stability from `Evolving` to

[GitHub] spark pull request #22213: [SPARK-25221][DEPLOY] Consistent trailing whitesp...

2018-08-30 Thread jerryshao
Github user jerryshao commented on a diff in the pull request: https://github.com/apache/spark/pull/22213#discussion_r214231103 --- Diff: core/src/test/scala/org/apache/spark/deploy/SparkSubmitSuite.scala --- @@ -1144,6 +1144,46 @@ class SparkSubmitSuite

[GitHub] spark issue #21721: [SPARK-24748][SS] Support for reporting custom metrics v...

2018-08-30 Thread arunmahadevan
Github user arunmahadevan commented on the issue: https://github.com/apache/spark/pull/21721 @HyukjinKwon yes we can mark it unstable. Like I mentioned multiple times in previous comments the traits added here like CustomMetrics, SupportsCustomReaderMetrics etc have nothing specific

[GitHub] spark issue #22138: [SPARK-25151][SS] Apply Apache Commons Pool to KafkaData...

2018-08-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22138 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22138: [SPARK-25151][SS] Apply Apache Commons Pool to KafkaData...

2018-08-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22138 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95502/ Test FAILed. ---

[GitHub] spark issue #22138: [SPARK-25151][SS] Apply Apache Commons Pool to KafkaData...

2018-08-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22138 **[Test build #95502 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95502/testReport)** for PR 22138 at commit

[GitHub] spark issue #22264: [SPARK-25256][SQL][TEST] Plan mismatch errors in Hive te...

2018-08-30 Thread sadhen
Github user sadhen commented on the issue: https://github.com/apache/spark/pull/22264 scala/bug#11123 had been added into https://github.com/scala/bug/milestone/93 . I will spare some time working on it. ---

[GitHub] spark issue #22183: [SPARK-25132][SQL][BACKPORT-2.3] Case-insensitive field ...

2018-08-30 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/22183 BTW, @gatorsmile and @cloud-fan, do you know who did this ^ and why? --- - To unsubscribe, e-mail:

[GitHub] spark issue #22283: [SPARK-25283][CORE] Fix for a deadlock in UnionRDD

2018-08-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22283 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95495/ Test PASSed. ---

[GitHub] spark issue #22283: [SPARK-25283][CORE] Fix for a deadlock in UnionRDD

2018-08-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22283 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22283: [SPARK-25283][CORE] Fix for a deadlock in UnionRDD

2018-08-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22283 **[Test build #95495 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95495/testReport)** for PR 22283 at commit

[GitHub] spark pull request #22240: [SPARK-25248] [CORE] Audit barrier Scala APIs for...

2018-08-30 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22240#discussion_r214229966 --- Diff: core/src/main/scala/org/apache/spark/BarrierTaskContext.scala --- @@ -145,20 +143,77 @@ class BarrierTaskContext( /** * ::

[GitHub] spark issue #22213: [SPARK-25221][DEPLOY] Consistent trailing whitespace tre...

2018-08-30 Thread gerashegalov
Github user gerashegalov commented on the issue: https://github.com/apache/spark/pull/22213 thanks for the comment @steveloughran. I'll add more tests for now and see how the discussion goes from there. as for transition to UTF it means to be fully correct Spark needs to

[GitHub] spark issue #22264: [SPARK-25256][SQL][TEST] Plan mismatch errors in Hive te...

2018-08-30 Thread sadhen
Github user sadhen commented on the issue: https://github.com/apache/spark/pull/22264 ``` Welcome to __ / __/__ ___ _/ /__ _\ \/ _ \/ _ `/ __/ '_/ /___/ .__/\_,_/_/ /_/\_\ version 2.4.0-SNAPSHOT /_/

[GitHub] spark issue #22273: [SPARK-25272][PYTHON][TEST] Add test to better indicate ...

2018-08-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22273 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95510/ Test FAILed. ---

[GitHub] spark issue #22273: [SPARK-25272][PYTHON][TEST] Add test to better indicate ...

2018-08-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22273 **[Test build #95510 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95510/testReport)** for PR 22273 at commit

[GitHub] spark issue #22273: [SPARK-25272][PYTHON][TEST] Add test to better indicate ...

2018-08-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22273 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22226: [SPARK-25252][SQL] Support arrays of any types by to_jso...

2018-08-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/6 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95496/ Test PASSed. ---

[GitHub] spark issue #22295: [SPARK-25255][PYTHON]Add getActiveSession to SparkSessio...

2018-08-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22295 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95509/ Test PASSed. ---

[GitHub] spark issue #22295: [SPARK-25255][PYTHON]Add getActiveSession to SparkSessio...

2018-08-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22295 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22226: [SPARK-25252][SQL] Support arrays of any types by to_jso...

2018-08-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/6 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22295: [SPARK-25255][PYTHON]Add getActiveSession to SparkSessio...

2018-08-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22295 **[Test build #95509 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95509/testReport)** for PR 22295 at commit

[GitHub] spark issue #22226: [SPARK-25252][SQL] Support arrays of any types by to_jso...

2018-08-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/6 **[Test build #95496 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95496/testReport)** for PR 6 at commit

[GitHub] spark issue #22048: [SPARK-25108][SQL] Fix the show method to display the wi...

2018-08-30 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/22048 +1 for adding `scalastyle:off nonascii` in that part. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark issue #22048: [SPARK-25108][SQL] Fix the show method to display the wi...

2018-08-30 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/22048 style errors, we cannot include non-ascii characters in files; ``` Scalastyle checks failed at following occurrences: [error]

[GitHub] spark issue #22048: [SPARK-25108][SQL] Fix the show method to display the wi...

2018-08-30 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/22048 Have a look at the test results, which show the style problem it flags: https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4305/console ``` [error]

[GitHub] spark issue #22292: [SPARK-25286][CORE] Removing the dangerous parmap

2018-08-30 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22292 For future, let us keep a connection with #22233 that is the original one --- - To unsubscribe, e-mail:

[GitHub] spark issue #22266: [SPARK-25270] lint-python: Add flake8 to find syntax err...

2018-08-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22266 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95501/ Test FAILed. ---

[GitHub] spark issue #22227: [SPARK-25202] [SQL] Implements split with limit sql func...

2018-08-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/7 **[Test build #95511 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95511/testReport)** for PR 7 at commit

[GitHub] spark issue #22266: [SPARK-25270] lint-python: Add flake8 to find syntax err...

2018-08-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22266 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22266: [SPARK-25270] lint-python: Add flake8 to find syntax err...

2018-08-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22266 **[Test build #95501 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95501/testReport)** for PR 22266 at commit

[GitHub] spark issue #21721: [SPARK-24748][SS] Support for reporting custom metrics v...

2018-08-30 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21721 Can we remove and replace this to another one when we are clear on how to deal with continuous mode? I merged this after having sufficient talks with @HeartSaVioR. The JIRA

[GitHub] spark pull request #22284: [SPARK-25278][SQL] Avoid duplicated Exec nodes wh...

2018-08-30 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/22284#discussion_r214227657 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/metric/SQLMetricsSuite.scala --- @@ -497,6 +497,17 @@ class SQLMetricsSuite extends

[GitHub] spark issue #22292: [SPARK-25286][CORE] Removing the dangerous parmap

2018-08-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22292 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95492/ Test PASSed. ---

[GitHub] spark issue #22292: [SPARK-25286][CORE] Removing the dangerous parmap

2018-08-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22292 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22292: [SPARK-25286][CORE] Removing the dangerous parmap

2018-08-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22292 **[Test build #95492 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95492/testReport)** for PR 22292 at commit

[GitHub] spark pull request #21721: [SPARK-24748][SS] Support for reporting custom me...

2018-08-30 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/21721#discussion_r214227481 --- Diff: sql/core/src/main/java/org/apache/spark/sql/sources/v2/CustomMetrics.java --- @@ -0,0 +1,33 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request #22284: [SPARK-25278][SQL] Avoid duplicated Exec nodes wh...

2018-08-30 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/22284#discussion_r214227423 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/planning/QueryPlanner.scala --- @@ -81,7 +81,7 @@ abstract class

[GitHub] spark pull request #22284: [SPARK-25278][SQL] Avoid duplicated Exec nodes wh...

2018-08-30 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/22284#discussion_r214227196 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/metric/SQLMetricsSuite.scala --- @@ -497,6 +497,17 @@ class SQLMetricsSuite extends

[GitHub] spark issue #22197: [SPARK-25207][SQL] Case-insensitve field resolution for ...

2018-08-30 Thread yucai
Github user yucai commented on the issue: https://github.com/apache/spark/pull/22197 @cloud-fan, tests have passed. And I will use a followup PR to make it cleaner. --- - To unsubscribe, e-mail:

[GitHub] spark pull request #22284: [SPARK-25278][SQL] Avoid duplicated Exec nodes wh...

2018-08-30 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/22284#discussion_r214227012 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/planning/QueryPlanner.scala --- @@ -81,7 +81,7 @@ abstract class

[GitHub] spark issue #22273: [SPARK-25272][PYTHON][TEST] Add test to better indicate ...

2018-08-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22273 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22273: [SPARK-25272][PYTHON][TEST] Add test to better indicate ...

2018-08-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22273 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #22273: [SPARK-25272][PYTHON][TEST] Add test to better indicate ...

2018-08-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22273 **[Test build #95510 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95510/testReport)** for PR 22273 at commit

[GitHub] spark issue #22273: [SPARK-25272][PYTHON][TEST] Add test to better indicate ...

2018-08-30 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/22273 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #22273: [SPARK-25272][PYTHON][TEST] Add test to better indicate ...

2018-08-30 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/22273 > Yeah but for people who don't know these tests or how the _have_xxx flags work, having an additional positive confirmation that the Arrow tests are run is nice. We already are doing this for

[GitHub] spark issue #22010: [SPARK-21436][CORE] Take advantage of known partitioner ...

2018-08-30 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/22010 I am sorry guys. I rushed to take a look. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #22273: [SPARK-25272][PYTHON][TEST] Add test to better in...

2018-08-30 Thread BryanCutler
Github user BryanCutler commented on a diff in the pull request: https://github.com/apache/spark/pull/22273#discussion_r214225809 --- Diff: python/pyspark/sql/tests.py --- @@ -4090,6 +4090,25 @@ def __init__(self, **kwargs): _make_type_verifier(data_type,

[GitHub] spark issue #22048: [SPARK-25108][SQL] Fix the show method to display the wi...

2018-08-30 Thread xuejianbest
Github user xuejianbest commented on the issue: https://github.com/apache/spark/pull/22048 Please help me @srowen I don't know where I need to make changes. --- - To unsubscribe, e-mail:

[GitHub] spark issue #22281: [SPARK-25280][SQL] Add support for USING syntax for Data...

2018-08-30 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/22281 @rdblue, Thanks, I will take a look and see if we can keep the current approach for the write side too. --- - To

[GitHub] spark issue #22048: [SPARK-25108][SQL] Fix the show method to display the wi...

2018-08-30 Thread xuejianbest
Github user xuejianbest commented on the issue: https://github.com/apache/spark/pull/22048 Why did it fail? What else should I do? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark issue #22295: [SPARK-25255][PYTHON]Add getActiveSession to SparkSessio...

2018-08-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22295 **[Test build #95509 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95509/testReport)** for PR 22295 at commit

[GitHub] spark issue #22295: [SPARK-25255][PYTHON]Add getActiveSession to SparkSessio...

2018-08-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22295 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22295: [SPARK-25255][PYTHON]Add getActiveSession to SparkSessio...

2018-08-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22295 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #22281: [SPARK-25280][SQL] Add support for USING syntax for Data...

2018-08-30 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/22281 > I think one possible design is to not support USING in data source v2, and always ask data sources to provide a catalog. I thought we almost all agreed with supporting USING syntax.

[GitHub] spark issue #22232: [SPARK-25237][SQL]remove updateBytesReadWithFileSize bec...

2018-08-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22232 **[Test build #95508 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95508/testReport)** for PR 22232 at commit

[GitHub] spark issue #22232: [SPARK-25237][SQL]remove updateBytesReadWithFileSize bec...

2018-08-30 Thread wzhfy
Github user wzhfy commented on the issue: https://github.com/apache/spark/pull/22232 ok to test --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #22295: [SPARK-25255][PYTHON]Add getActiveSession to SparkSessio...

2018-08-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22295 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22295: [SPARK-25255][PYTHON]Add getActiveSession to SparkSessio...

2018-08-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22295 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95507/ Test FAILed. ---

[GitHub] spark issue #22295: [SPARK-25255][PYTHON]Add getActiveSession to SparkSessio...

2018-08-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22295 **[Test build #95507 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95507/testReport)** for PR 22295 at commit

[GitHub] spark issue #22295: [SPARK-25255][PYTHON]Add getActiveSession to SparkSessio...

2018-08-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22295 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #22295: [SPARK-25255][PYTHON]Add getActiveSession to SparkSessio...

2018-08-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22295 **[Test build #95507 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95507/testReport)** for PR 22295 at commit

[GitHub] spark issue #22295: [SPARK-25255][PYTHON]Add getActiveSession to SparkSessio...

2018-08-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22295 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #22295: [SPARK-25255][PYTHON]Add getActiveSession to Spar...

2018-08-30 Thread huaxingao
GitHub user huaxingao opened a pull request: https://github.com/apache/spark/pull/22295 [SPARK-25255][PYTHON]Add getActiveSession to SparkSession in PySpark ## What changes were proposed in this pull request? add getActiveSession in session.py ## How was this

[GitHub] spark issue #22291: [SPARK-25007][R]Add array_intersect/array_except/array_u...

2018-08-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22291 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22291: [SPARK-25007][R]Add array_intersect/array_except/array_u...

2018-08-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22291 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95491/ Test PASSed. ---

[GitHub] spark issue #22291: [SPARK-25007][R]Add array_intersect/array_except/array_u...

2018-08-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22291 **[Test build #95491 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95491/testReport)** for PR 22291 at commit

[GitHub] spark pull request #22237: [SPARK-25243][SQL] Use FailureSafeParser in from_...

2018-08-30 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/22237#discussion_r214218549 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/JsonFunctionsSuite.scala --- @@ -469,4 +470,26 @@ class JsonFunctionsSuite extends QueryTest with

[GitHub] spark issue #22273: [SPARK-25272][PYTHON][TEST] Add test to better indicate ...

2018-08-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22273 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22273: [SPARK-25272][PYTHON][TEST] Add test to better indicate ...

2018-08-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22273 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95506/ Test FAILed. ---

[GitHub] spark issue #22273: [SPARK-25272][PYTHON][TEST] Add test to better indicate ...

2018-08-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22273 **[Test build #95506 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95506/testReport)** for PR 22273 at commit

[GitHub] spark issue #22293: [SPARK-25288][Tests]Fix flaky Kafka transaction tests

2018-08-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22293 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95505/ Test PASSed. ---

[GitHub] spark issue #22293: [SPARK-25288][Tests]Fix flaky Kafka transaction tests

2018-08-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22293 **[Test build #4324 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4324/testReport)** for PR 22293 at commit

[GitHub] spark issue #22293: [SPARK-25288][Tests]Fix flaky Kafka transaction tests

2018-08-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22293 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22293: [SPARK-25288][Tests]Fix flaky Kafka transaction tests

2018-08-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22293 **[Test build #4320 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4320/testReport)** for PR 22293 at commit

[GitHub] spark issue #22293: [SPARK-25288][Tests]Fix flaky Kafka transaction tests

2018-08-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22293 **[Test build #95505 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95505/testReport)** for PR 22293 at commit

[GitHub] spark issue #21721: [SPARK-24748][SS] Support for reporting custom metrics v...

2018-08-30 Thread arunmahadevan
Github user arunmahadevan commented on the issue: https://github.com/apache/spark/pull/21721 There are many unknowns to be figured out for continuous mode. Though the way to capture the metrics would be different for continuous execution, the interface of whats reported is not

[GitHub] spark issue #22293: [SPARK-25288][Tests]Fix flaky Kafka transaction tests

2018-08-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22293 **[Test build #4327 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4327/testReport)** for PR 22293 at commit

[GitHub] spark issue #22293: [SPARK-25288][Tests]Fix flaky Kafka transaction tests

2018-08-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22293 **[Test build #4325 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4325/testReport)** for PR 22293 at commit

[GitHub] spark issue #22293: [SPARK-25288][Tests]Fix flaky Kafka transaction tests

2018-08-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22293 **[Test build #4328 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4328/testReport)** for PR 22293 at commit

[GitHub] spark issue #22293: [SPARK-25288][Tests]Fix flaky Kafka transaction tests

2018-08-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22293 **[Test build #4322 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4322/testReport)** for PR 22293 at commit

[GitHub] spark issue #22293: [SPARK-25288][Tests]Fix flaky Kafka transaction tests

2018-08-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22293 **[Test build #4323 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4323/testReport)** for PR 22293 at commit

[GitHub] spark issue #22293: [SPARK-25288][Tests]Fix flaky Kafka transaction tests

2018-08-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22293 **[Test build #4321 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4321/testReport)** for PR 22293 at commit

[GitHub] spark issue #22293: [SPARK-25288][Tests]Fix flaky Kafka transaction tests

2018-08-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22293 **[Test build #4326 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4326/testReport)** for PR 22293 at commit

[GitHub] spark pull request #22264: [SPARK-25256][SQL][TEST] Plan mismatch errors in ...

2018-08-30 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/22264#discussion_r214215448 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/QueryTest.scala --- @@ -290,6 +290,16 @@ object QueryTest { Row.fromSeq(row.toSeq.map {

[GitHub] spark issue #22227: [SPARK-25202] [SQL] Implements split with limit sql func...

2018-08-30 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/7 You need to handle that in both codegen(doGenCode) and interpreter(nullSafeEval) path. Also, can you add tests to check if they have the same behaviour in the limit=0/limit=-1 cases? You'd be better

[GitHub] spark issue #22273: [SPARK-25272][PYTHON][TEST] Add test to better indicate ...

2018-08-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22273 **[Test build #95506 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95506/testReport)** for PR 22273 at commit

[GitHub] spark issue #21721: [SPARK-24748][SS] Support for reporting custom metrics v...

2018-08-30 Thread HeartSaVioR
Github user HeartSaVioR commented on the issue: https://github.com/apache/spark/pull/21721 Unfortunate thing is that continuous mode allows different epochs between partitions, hence query progress just doesn't fit so hard to address SPARK-23887 for now. My 2 cents is continuous mode

[GitHub] spark issue #22273: [SPARK-25272][PYTHON][TEST] Add test to better indicate ...

2018-08-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22273 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22273: [SPARK-25272][PYTHON][TEST] Add test to better indicate ...

2018-08-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22273 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #21721: [SPARK-24748][SS] Support for reporting custom metrics v...

2018-08-30 Thread HeartSaVioR
Github user HeartSaVioR commented on the issue: https://github.com/apache/spark/pull/21721 @zsxwing @gatorsmile @cloud-fan As you all know, IMHO, the thing is that continuous mode doesn't support StreamingQueryProgress for now. We allowed these incomplete state because

[GitHub] spark issue #21721: [SPARK-24748][SS] Support for reporting custom metrics v...

2018-08-30 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/21721 @arunmahadevan yeah, it's better to figure out the solution for continuous mode as well. As you mentioned, the current SQL metrics are not updated unless the task completes, so we may need to add

[GitHub] spark issue #21721: [SPARK-24748][SS] Support for reporting custom metrics v...

2018-08-30 Thread arunmahadevan
Github user arunmahadevan commented on the issue: https://github.com/apache/spark/pull/21721 The CustomMetrics are traits which can be mixed in if necessary. (see https://github.com/apache/spark/pull/21721#issuecomment-403878383) and does not affect any other API as such. When query

[GitHub] spark issue #22293: [SPARK-25288][Tests]Fix flaky Kafka transaction tests

2018-08-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22293 **[Test build #4328 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4328/testReport)** for PR 22293 at commit

[GitHub] spark issue #22293: [SPARK-25288][Tests]Fix flaky Kafka transaction tests

2018-08-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22293 **[Test build #4327 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4327/testReport)** for PR 22293 at commit

[GitHub] spark issue #22293: [SPARK-25288][Tests]Fix flaky Kafka transaction tests

2018-08-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22293 **[Test build #4326 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4326/testReport)** for PR 22293 at commit

<    1   2   3   4   5   6   7   >