[GitHub] spark issue #20645: SPARK-23472: Add defaultJavaOptions for drivers and exec...

2018-02-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20645 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/975/

[GitHub] spark issue #20645: SPARK-23472: Add defaultJavaOptions for drivers and exec...

2018-02-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20645 **[Test build #87569 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87569/testReport)** for PR 20645 at commit

[GitHub] spark pull request #20572: [SPARK-17147][STREAMING][KAFKA] Allow non-consecu...

2018-02-20 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/20572#discussion_r169491061 --- Diff: external/kafka-0-10/src/test/scala/org/apache/spark/streaming/kafka010/KafkaRDDSuite.scala --- @@ -22,12 +22,17 @@ import java.{ util => ju }

[GitHub] spark pull request #20572: [SPARK-17147][STREAMING][KAFKA] Allow non-consecu...

2018-02-20 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/20572#discussion_r169490896 --- Diff: external/kafka-0-10/src/main/scala/org/apache/spark/streaming/kafka010/KafkaRDD.scala --- @@ -178,51 +196,128 @@ private[spark] class KafkaRDD[K,

[GitHub] spark pull request #20572: [SPARK-17147][STREAMING][KAFKA] Allow non-consecu...

2018-02-20 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/20572#discussion_r169491286 --- Diff: external/kafka-0-10/src/test/scala/org/apache/spark/streaming/kafka010/mocks/MockScheduler.scala --- @@ -0,0 +1,108 @@ +/* + * Licensed

[GitHub] spark pull request #20572: [SPARK-17147][STREAMING][KAFKA] Allow non-consecu...

2018-02-20 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/20572#discussion_r169489088 --- Diff: external/kafka-0-10/src/main/scala/org/apache/spark/streaming/kafka010/CachedKafkaConsumer.scala --- @@ -83,13 +81,50 @@ class

[GitHub] spark pull request #20572: [SPARK-17147][STREAMING][KAFKA] Allow non-consecu...

2018-02-20 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/20572#discussion_r169489462 --- Diff: external/kafka-0-10/src/main/scala/org/apache/spark/streaming/kafka010/CachedKafkaConsumer.scala --- @@ -53,7 +51,7 @@ class

[GitHub] spark pull request #20572: [SPARK-17147][STREAMING][KAFKA] Allow non-consecu...

2018-02-20 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/20572#discussion_r169490350 --- Diff: external/kafka-0-10/src/main/scala/org/apache/spark/streaming/kafka010/KafkaRDD.scala --- @@ -178,51 +196,128 @@ private[spark] class KafkaRDD[K,

[GitHub] spark pull request #20572: [SPARK-17147][STREAMING][KAFKA] Allow non-consecu...

2018-02-20 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/20572#discussion_r169489226 --- Diff: external/kafka-0-10/src/main/scala/org/apache/spark/streaming/kafka010/CachedKafkaConsumer.scala --- @@ -83,13 +81,50 @@ class

[GitHub] spark pull request #20572: [SPARK-17147][STREAMING][KAFKA] Allow non-consecu...

2018-02-20 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/20572#discussion_r169490949 --- Diff: external/kafka-0-10/src/main/scala/org/apache/spark/streaming/kafka010/KafkaRDD.scala --- @@ -178,51 +196,128 @@ private[spark] class KafkaRDD[K,

[GitHub] spark pull request #20572: [SPARK-17147][STREAMING][KAFKA] Allow non-consecu...

2018-02-20 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/20572#discussion_r169490790 --- Diff: external/kafka-0-10/src/main/scala/org/apache/spark/streaming/kafka010/KafkaRDD.scala --- @@ -87,47 +89,63 @@ private[spark] class KafkaRDD[K,

[GitHub] spark pull request #20572: [SPARK-17147][STREAMING][KAFKA] Allow non-consecu...

2018-02-20 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/20572#discussion_r169491574 --- Diff: external/kafka-0-10/src/test/scala/org/apache/spark/streaming/kafka010/mocks/MockScheduler.scala --- @@ -0,0 +1,108 @@ +/* + * Licensed

[GitHub] spark pull request #20572: [SPARK-17147][STREAMING][KAFKA] Allow non-consecu...

2018-02-20 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/20572#discussion_r169491706 --- Diff: external/kafka-0-10/src/test/scala/org/apache/spark/streaming/kafka010/mocks/MockTime.scala --- @@ -0,0 +1,51 @@ +/* + * Licensed to the

[GitHub] spark pull request #20646: [SPARK-23408][SS] Synchronize successive AddDataM...

2018-02-20 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/20646#discussion_r169521344 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/streaming/StreamTest.scala --- @@ -429,7 +429,25 @@ trait StreamTest extends QueryTest with

[GitHub] spark issue #20644: [SPARK-23470][ui] Use first attempt of last stage to def...

2018-02-20 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/20644 Verified that this patch fixes the issue. LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20604: [SPARK-23365][CORE] Do not adjust num executors when kil...

2018-02-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20604 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87567/ Test FAILed. ---

[GitHub] spark issue #20644: [SPARK-23470][ui] Use first attempt of last stage to def...

2018-02-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20644 **[Test build #87568 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87568/testReport)** for PR 20644 at commit

[GitHub] spark issue #20644: [SPARK-23470][ui] Use first attempt of last stage to def...

2018-02-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20644 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20646: [SPARK-23408][SS] Synchronize successive AddDataMemory a...

2018-02-20 Thread jose-torres
Github user jose-torres commented on the issue: https://github.com/apache/spark/pull/20646 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20646: [SPARK-23408][SS] Synchronize successive AddDataMemory a...

2018-02-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20646 **[Test build #87572 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87572/testReport)** for PR 20646 at commit

[GitHub] spark issue #20646: [SPARK-23408][SS] Synchronize successive AddDataMemory a...

2018-02-20 Thread jose-torres
Github user jose-torres commented on the issue: https://github.com/apache/spark/pull/20646 > java.lang.RuntimeException: [unresolved dependency: com.sun.jersey#jersey-core;1.14: configuration not found in com.sun.jersey#jersey-core;1.14: 'master(compile)'. Missing configuration:

[GitHub] spark pull request #20631: [SPARK-23454][SS][DOCS] Added trigger information...

2018-02-20 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/20631 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20613: [SPARK-23368][SQL] Avoid unnecessary Exchange or Sort af...

2018-02-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20613 **[Test build #87565 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87565/testReport)** for PR 20613 at commit

[GitHub] spark issue #20613: [SPARK-23368][SQL] Avoid unnecessary Exchange or Sort af...

2018-02-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20613 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20631: [SPARK-23454][SS][DOCS] Added trigger information to the...

2018-02-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20631 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20631: [SPARK-23454][SS][DOCS] Added trigger information to the...

2018-02-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20631 **[Test build #87570 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87570/testReport)** for PR 20631 at commit

[GitHub] spark issue #20631: [SPARK-23454][SS][DOCS] Added trigger information to the...

2018-02-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20631 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87570/ Test PASSed. ---

[GitHub] spark issue #20604: [SPARK-23365][CORE] Do not adjust num executors when kil...

2018-02-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20604 **[Test build #87567 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87567/testReport)** for PR 20604 at commit

[GitHub] spark issue #20604: [SPARK-23365][CORE] Do not adjust num executors when kil...

2018-02-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20604 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20646: [SPARK-23408][SS] Synchronize successive AddDataMemory a...

2018-02-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20646 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87571/ Test FAILed. ---

[GitHub] spark issue #20646: [SPARK-23408][SS] Synchronize successive AddDataMemory a...

2018-02-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20646 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20646: [SPARK-23408][SS] Synchronize successive AddDataMemory a...

2018-02-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20646 **[Test build #87571 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87571/testReport)** for PR 20646 at commit

[GitHub] spark pull request #20643: [SPARK-23468][core] Stringify auth secret before ...

2018-02-20 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/20643 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #20057: [SPARK-22880][SQL] Add cascadeTruncate option to ...

2018-02-20 Thread klinvill
Github user klinvill commented on a diff in the pull request: https://github.com/apache/spark/pull/20057#discussion_r169526767 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/jdbc/TeradataDialect.scala --- @@ -31,4 +31,19 @@ private case object TeradataDialect extends

[GitHub] spark issue #20613: [SPARK-23368][SQL] Avoid unnecessary Exchange or Sort af...

2018-02-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20613 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/977/

[GitHub] spark issue #20613: [SPARK-23368][SQL] Avoid unnecessary Exchange or Sort af...

2018-02-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20613 **[Test build #87574 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87574/testReport)** for PR 20613 at commit

[GitHub] spark issue #20613: [SPARK-23368][SQL] Avoid unnecessary Exchange or Sort af...

2018-02-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20613 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20616: [SPARK-23434][SQL] Spark should not warn `metadata direc...

2018-02-20 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/20616 The warning error messages in kerberized environments are the one in PR/JIRA description. For example, Apache Spark 2.2.1 binary on kerberized cluster shows the following. ```

[GitHub] spark issue #20613: [SPARK-23368][SQL] Avoid unnecessary Exchange or Sort af...

2018-02-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20613 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87565/ Test FAILed. ---

[GitHub] spark issue #20646: [SPARK-23408][SS] Synchronize successive AddDataMemory a...

2018-02-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20646 **[Test build #87571 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87571/testReport)** for PR 20646 at commit

[GitHub] spark issue #20646: [SPARK-23408][SS] Synchronize successive AddDataMemory a...

2018-02-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20646 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #20616: [SPARK-23434][SQL] Spark should not warn `metadat...

2018-02-20 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/20616 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20645: SPARK-23472: Add defaultJavaOptions for drivers and exec...

2018-02-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20645 **[Test build #87569 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87569/testReport)** for PR 20645 at commit

[GitHub] spark issue #20645: SPARK-23472: Add defaultJavaOptions for drivers and exec...

2018-02-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20645 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20646: [SPARK-23408][SS] Synchronize successive AddDataMemory a...

2018-02-20 Thread tdas
Github user tdas commented on the issue: https://github.com/apache/spark/pull/20646 Actually, I am having second thoughts about this. This is fundamentally changing how the tests work, especially for stress tests. The stress tests actually test these corner cases (by randomly adding

[GitHub] spark issue #20612: [SPARK-23424][SQL]Add codegenStageId in comment

2018-02-20 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/20612 thanks, merging to master! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #17619: [SPARK-19755][Mesos] Blacklist is always active for Meso...

2018-02-20 Thread squito
Github user squito commented on the issue: https://github.com/apache/spark/pull/17619 for anyone watching this: @IgorBerman submitted an updated version of this here https://github.com/apache/spark/pull/20640 which I plan to merge unless there are any objections. ---

[GitHub] spark pull request #20645: SPARK-23472: Add defaultJavaOptions for drivers a...

2018-02-20 Thread rdblue
GitHub user rdblue opened a pull request: https://github.com/apache/spark/pull/20645 SPARK-23472: Add defaultJavaOptions for drivers and executors. ## What changes were proposed in this pull request? This adds two new config properties: spark.driver.defaultJavaOptions and

[GitHub] spark issue #20631: [SPARK-23454][SS][DOCS] Added trigger information to the...

2018-02-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20631 **[Test build #87570 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87570/testReport)** for PR 20631 at commit

[GitHub] spark issue #20616: [SPARK-23434][SQL] Spark should not warn `metadata direc...

2018-02-20 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/20616 @dongjoon-hyun could you also post the error happening in kerberized environments? --- - To unsubscribe, e-mail:

[GitHub] spark issue #20632: [SPARK-3159] added subtree pruning in the translation fr...

2018-02-20 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/20632 Run `./dev/scalastyle` to replicate what Jenkins will do. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark issue #20631: [SPARK-23454][SS][DOCS] Added trigger information to the...

2018-02-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20631 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20631: [SPARK-23454][SS][DOCS] Added trigger information to the...

2018-02-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20631 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/976/

[GitHub] spark issue #20632: [SPARK-3159] added subtree pruning in the translation fr...

2018-02-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20632 **[Test build #4103 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4103/testReport)** for PR 20632 at commit

[GitHub] spark issue #20631: [SPARK-23454][SS][DOCS] Added trigger information to the...

2018-02-20 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/20631 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20616: [SPARK-23434][SQL] Spark should not warn `metadata direc...

2018-02-20 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/20616 Here, it is. It's `AccessControlException`, @zsxwing . ``` 18/02/20 23:46:53 WARN streaming.FileStreamSink: Error while looking for metadata directory.

[GitHub] spark pull request #20640: [SPARK-19755][Mesos] Blacklist is always active f...

2018-02-20 Thread kayousterhout
Github user kayousterhout commented on a diff in the pull request: https://github.com/apache/spark/pull/20640#discussion_r169497847 --- Diff: resource-managers/mesos/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosCoarseGrainedSchedulerBackend.scala --- @@ -648,15

[GitHub] spark issue #20616: [SPARK-23434][SQL] Spark should not warn `metadata direc...

2018-02-20 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/20616 LGTM. Merging to master. Thanks! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #20643: [SPARK-23468][core] Stringify auth secret before storing...

2018-02-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20643 **[Test build #87566 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87566/testReport)** for PR 20643 at commit

[GitHub] spark issue #20646: [SPARK-23408][SS] Synchronize successive AddDataMemory a...

2018-02-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20646 **[Test build #87572 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87572/testReport)** for PR 20646 at commit

[GitHub] spark issue #20646: [SPARK-23408][SS] Synchronize successive AddDataMemory a...

2018-02-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20646 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87572/ Test FAILed. ---

[GitHub] spark issue #20646: [SPARK-23408][SS] Synchronize successive AddDataMemory a...

2018-02-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20646 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20613: [SPARK-23368][SQL] Avoid unnecessary Exchange or Sort af...

2018-02-20 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/20613 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20618: [SPARK-23329][SQL] Fix documentation of trigonometric fu...

2018-02-20 Thread misutoth
Github user misutoth commented on the issue: https://github.com/apache/spark/pull/20618 Sure, lets do that, no problem. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #20644: [SPARK-23470][ui] Use first attempt of last stage to def...

2018-02-20 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/20644 Thanks for fixing this so fast. I'm testing it now. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark issue #20644: [SPARK-23470][ui] Use first attempt of last stage to def...

2018-02-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20644 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87568/ Test PASSed. ---

[GitHub] spark issue #20646: [SPARK-23408][SS] Synchronize successive AddDataMemory a...

2018-02-20 Thread jose-torres
Github user jose-torres commented on the issue: https://github.com/apache/spark/pull/20646 (https://issues.apache.org/jira/browse/SPARK-23369 was already filed for previous flake) --- - To unsubscribe, e-mail:

[GitHub] spark pull request #20644: [SPARK-23470][ui] Use first attempt of last stage...

2018-02-20 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/20644 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #14936: [SPARK-7877][MESOS] Allow configuration of framework tim...

2018-02-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14936 **[Test build #87573 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87573/testReport)** for PR 14936 at commit

[GitHub] spark issue #20645: SPARK-23472: Add defaultJavaOptions for drivers and exec...

2018-02-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20645 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87569/ Test PASSed. ---

[GitHub] spark pull request #20057: [SPARK-22880][SQL] Add cascadeTruncate option to ...

2018-02-20 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/20057#discussion_r169530627 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/jdbc/TeradataDialect.scala --- @@ -31,4 +31,19 @@ private case object TeradataDialect extends

[GitHub] spark issue #20616: [SPARK-23434][SQL] Spark should not warn `metadata direc...

2018-02-20 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/20616 With more manual tests, I observed that the original situation happens on only kerberized environments. I updated PR/JIRA description. ---

[GitHub] spark pull request #20646: [SPARK-23408][SS] Synchronize successive AddDataM...

2018-02-20 Thread jose-torres
GitHub user jose-torres opened a pull request: https://github.com/apache/spark/pull/20646 [SPARK-23408][SS] Synchronize successive AddDataMemory actions in StreamTest. ## What changes were proposed in this pull request? The stream-stream join tests add data to multiple

[GitHub] spark issue #20646: [SPARK-23408][SS] Synchronize successive AddDataMemory a...

2018-02-20 Thread jose-torres
Github user jose-torres commented on the issue: https://github.com/apache/spark/pull/20646 @tdas --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #20419: [SPARK-23032][SQL][FOLLOW-UP]Add codegenStageId i...

2018-02-20 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/20419 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #20612: [SPARK-23424][SQL]Add codegenStageId in comment

2018-02-20 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/20612 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20645: SPARK-23472: Add defaultJavaOptions for drivers and exec...

2018-02-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20645 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20616: [SPARK-23434][SQL] Spark should not warn `metadata direc...

2018-02-20 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/20616 @dongjoon-hyun I meant the stack trace thrown from `fs.exists`. --- - To unsubscribe, e-mail:

[GitHub] spark issue #20635: [SPARK-23053][CORE][BRANCH-2.1] taskBinarySerialization ...

2018-02-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20635 **[Test build #4102 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4102/testReport)** for PR 20635 at commit

[GitHub] spark issue #20632: [SPARK-3159] added subtree pruning in the translation fr...

2018-02-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20632 **[Test build #4103 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4103/testReport)** for PR 20632 at commit

[GitHub] spark pull request #20646: [SPARK-23408][SS] Synchronize successive AddDataM...

2018-02-20 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/20646#discussion_r169522041 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/streaming/StreamTest.scala --- @@ -429,7 +429,25 @@ trait StreamTest extends QueryTest with

[GitHub] spark issue #20572: [SPARK-17147][STREAMING][KAFKA] Allow non-consecutive of...

2018-02-20 Thread justinrmiller
Github user justinrmiller commented on the issue: https://github.com/apache/spark/pull/20572 Not to pile on, but I was encountering non-consecutive offsets on non-compacted topics once or twice a day at our volume. This patch fixes this (haven't had a problem with it since) and I'm

[GitHub] spark issue #20618: [SPARK-23329][SQL] Fix documentation of trigonometric fu...

2018-02-20 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20618 I just thought about this again and yup I am fine. @misutoth shall we just fix R and Python ones too here? I think we could just target to fix the current functions in other language's API.

[GitHub] spark pull request #20640: [SPARK-19755][Mesos] Blacklist is always active f...

2018-02-20 Thread kayousterhout
Github user kayousterhout commented on a diff in the pull request: https://github.com/apache/spark/pull/20640#discussion_r169500415 --- Diff: resource-managers/mesos/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosCoarseGrainedSchedulerBackend.scala --- @@ -571,7

[GitHub] spark issue #20616: [SPARK-23434][SQL] Spark should not warn `metadata direc...

2018-02-20 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/20616 Thank you, @zsxwing and @cloud-fan . --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20643: [SPARK-23468][core] Stringify auth secret before storing...

2018-02-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20643 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87566/ Test PASSed. ---

[GitHub] spark issue #20643: [SPARK-23468][core] Stringify auth secret before storing...

2018-02-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20643 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20622: [SPARK-23441][SS] Remove queryExecutionThread.interrupt(...

2018-02-20 Thread tdas
Github user tdas commented on the issue: https://github.com/apache/spark/pull/20622 @jose-torres I had a long offline chat with @zsxwing, kudos to him for catching a corner case in the current solution. The following sequence of events may occur. - In the query thread, the

[GitHub] spark issue #20644: [SPARK-23470][ui] Use first attempt of last stage to def...

2018-02-20 Thread sameeragarwal
Github user sameeragarwal commented on the issue: https://github.com/apache/spark/pull/20644 merging this to master/2.3. Thanks! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20635: [SPARK-23053][CORE][BRANCH-2.1] taskBinarySerialization ...

2018-02-20 Thread squito
Github user squito commented on the issue: https://github.com/apache/spark/pull/20635 thanks @ivoson , merged! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #14936: [SPARK-7877][MESOS] Allow configuration of framework tim...

2018-02-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14936 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87573/ Test FAILed. ---

[GitHub] spark issue #14936: [SPARK-7877][MESOS] Allow configuration of framework tim...

2018-02-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14936 Build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #20572: [SPARK-17147][STREAMING][KAFKA] Allow non-consecutive of...

2018-02-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20572 **[Test build #87575 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87575/testReport)** for PR 20572 at commit

[GitHub] spark pull request #20572: [SPARK-17147][STREAMING][KAFKA] Allow non-consecu...

2018-02-20 Thread koeninger
Github user koeninger commented on a diff in the pull request: https://github.com/apache/spark/pull/20572#discussion_r169538019 --- Diff: external/kafka-0-10/src/main/scala/org/apache/spark/streaming/kafka010/CachedKafkaConsumer.scala --- @@ -53,7 +51,7 @@ class

[GitHub] spark issue #20572: [SPARK-17147][STREAMING][KAFKA] Allow non-consecutive of...

2018-02-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20572 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87575/ Test PASSed. ---

[GitHub] spark pull request #20647: [SPARK-23303][SQL] improve the explain result for...

2018-02-20 Thread cloud-fan
GitHub user cloud-fan opened a pull request: https://github.com/apache/spark/pull/20647 [SPARK-23303][SQL] improve the explain result for data source v2 relations ## What changes were proposed in this pull request? The current explain result for data source v2 relation is

[GitHub] spark issue #20647: [SPARK-23303][SQL] improve the explain result for data s...

2018-02-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20647 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20647: [SPARK-23303][SQL] improve the explain result for data s...

2018-02-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20647 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/980/

[GitHub] spark issue #20647: [SPARK-23303][SQL] improve the explain result for data s...

2018-02-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20647 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #20572: [SPARK-17147][STREAMING][KAFKA] Allow non-consecu...

2018-02-20 Thread koeninger
Github user koeninger commented on a diff in the pull request: https://github.com/apache/spark/pull/20572#discussion_r169537541 --- Diff: external/kafka-0-10/src/main/scala/org/apache/spark/streaming/kafka010/CachedKafkaConsumer.scala --- @@ -83,13 +81,50 @@ class

  1   2   3   4   >