[GitHub] spark pull request #22288: [SPARK-22148][SPARK-15815][Scheduler] Acquire new...

2018-10-26 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/22288#discussion_r228609147 --- Diff: core/src/test/scala/org/apache/spark/scheduler/TaskSchedulerImplSuite.scala --- @@ -503,6 +507,181 @@ class TaskSchedulerImplSuite extends

[GitHub] spark pull request #22288: [SPARK-22148][SPARK-15815][Scheduler] Acquire new...

2018-10-26 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/22288#discussion_r228605235 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala --- @@ -415,9 +420,54 @@ private[spark] class TaskSchedulerImpl(

[GitHub] spark pull request #22288: [SPARK-22148][SPARK-15815][Scheduler] Acquire new...

2018-10-26 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/22288#discussion_r228609353 --- Diff: core/src/test/scala/org/apache/spark/scheduler/TaskSchedulerImplSuite.scala --- @@ -503,6 +507,181 @@ class TaskSchedulerImplSuite extends

[GitHub] spark pull request #22288: [SPARK-22148][SPARK-15815][Scheduler] Acquire new...

2018-10-26 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/22288#discussion_r228606711 --- Diff: core/src/test/scala/org/apache/spark/scheduler/TaskSchedulerImplSuite.scala --- @@ -503,6 +507,181 @@ class TaskSchedulerImplSuite extends

[GitHub] spark pull request #22288: [SPARK-22148][SPARK-15815][Scheduler] Acquire new...

2018-10-26 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/22288#discussion_r228604552 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala --- @@ -415,9 +420,54 @@ private[spark] class TaskSchedulerImpl(

[GitHub] spark pull request #22288: [SPARK-22148][SPARK-15815][Scheduler] Acquire new...

2018-10-26 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/22288#discussion_r228604322 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala --- @@ -415,9 +420,54 @@ private[spark] class TaskSchedulerImpl(

[GitHub] spark issue #22830: [SPARK-25838][ML] Remove formatVersion from Saveable

2018-10-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22830 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22854: [SPARK-25854][BUILD] fix `build/mvn` not to fail during ...

2018-10-26 Thread shaneknapp
Github user shaneknapp commented on the issue: https://github.com/apache/spark/pull/22854 let's not merge until this build passes: https://amplab.cs.berkeley.edu/jenkins/job/sknapp-testing-spark-branch-2.4-test-maven-hadoop-2.7/12/ ---

[GitHub] spark issue #22830: [SPARK-25838][ML] Remove formatVersion from Saveable

2018-10-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22830 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98088/ Test PASSed. ---

[GitHub] spark issue #22809: [SPARK-19851][SQL] Add support for EVERY and ANY (SOME) ...

2018-10-26 Thread dilipbiswal
Github user dilipbiswal commented on the issue: https://github.com/apache/spark/pull/22809 @cloud-fan @mgaido91 I have incorporated the comments. Could we please check if things look okay now ? --- - To

[GitHub] spark issue #22830: [SPARK-25838][ML] Remove formatVersion from Saveable

2018-10-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22830 **[Test build #98088 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98088/testReport)** for PR 22830 at commit

[GitHub] spark issue #22820: [SPARK-25828][K8S] Bumping Kubernetes-Client version to ...

2018-10-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22820 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-make-spark-distribution-unified/4545/ ---

[GitHub] spark pull request #22853: [SPARK-25845][SQL] Fix MatchError for calendar in...

2018-10-26 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/22853#discussion_r228608016 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/DataFrameWindowFramesSuite.scala --- @@ -267,6 +267,25 @@ class DataFrameWindowFramesSuite extends

[GitHub] spark pull request #22854: [SPARK-25854] fix mvn to not always exit 1

2018-10-26 Thread shaneknapp
Github user shaneknapp commented on a diff in the pull request: https://github.com/apache/spark/pull/22854#discussion_r228607477 --- Diff: build/mvn --- @@ -163,8 +163,14 @@ export MAVEN_OPTS=${MAVEN_OPTS:-"$_COMPILE_JVM_OPTS"} echo "Using \`mvn\` from path: $MVN_BIN"

[GitHub] spark pull request #22854: [SPARK-25854] fix mvn to not always exit 1

2018-10-26 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/22854#discussion_r228606581 --- Diff: build/mvn --- @@ -163,8 +163,14 @@ export MAVEN_OPTS=${MAVEN_OPTS:-"$_COMPILE_JVM_OPTS"} echo "Using \`mvn\` from path: $MVN_BIN"

[GitHub] spark issue #22029: [SPARK-24395][SQL] IN operator should return NULL when c...

2018-10-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22029 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98089/ Test PASSed. ---

[GitHub] spark pull request #22624: [SPARK-23781][CORE] Merge token renewer functiona...

2018-10-26 Thread ifilonenko
Github user ifilonenko commented on a diff in the pull request: https://github.com/apache/spark/pull/22624#discussion_r228605725 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/security/KubernetesHadoopDelegationTokenManager.scala --- @@

[GitHub] spark issue #22029: [SPARK-24395][SQL] IN operator should return NULL when c...

2018-10-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22029 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22029: [SPARK-24395][SQL] IN operator should return NULL when c...

2018-10-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22029 **[Test build #98089 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98089/testReport)** for PR 22029 at commit

[GitHub] spark pull request #22823: [SPARK-25676][SQL][TEST] Improve BenchmarkWideTab...

2018-10-26 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/22823#discussion_r228605621 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/benchmark/BenchmarkWideTable.scala --- @@ -1,52 +0,0 @@ -/* - * Licensed to

[GitHub] spark issue #22820: [SPARK-25828][K8S] Bumping Kubernetes-Client version to ...

2018-10-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22820 **[Test build #98102 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98102/testReport)** for PR 22820 at commit

[GitHub] spark pull request #22754: [SPARK-25776][CORE]The disk write buffer size mus...

2018-10-26 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22754#discussion_r228601036 --- Diff: core/src/main/java/org/apache/spark/util/collection/unsafe/sort/UnsafeSorterSpillWriter.java --- @@ -62,6 +62,8 @@ public UnsafeSorterSpillWriter(

[GitHub] spark issue #22815: [SPARK-25821][SQL] Remove SQLContext methods deprecated ...

2018-10-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22815 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #22847: [SPARK-25850][SQL] Make the split threshold for t...

2018-10-26 Thread gengliangwang
Github user gengliangwang commented on a diff in the pull request: https://github.com/apache/spark/pull/22847#discussion_r228600757 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -812,6 +812,17 @@ object SQLConf { .intConf

[GitHub] spark issue #22815: [SPARK-25821][SQL] Remove SQLContext methods deprecated ...

2018-10-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22815 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #22854: [SPARK-25854] fix mvn to not always exit 1

2018-10-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22854 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #22854: [SPARK-25854] fix mvn to not always exit 1

2018-10-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22854 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19045: [WIP][SPARK-20628][CORE][K8S] Keep track of nodes (/ spo...

2018-10-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19045 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #22854: [SPARK-25854] fix mvn to not always exit 1

2018-10-26 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/22854 Probably unnecessary to change the timeout, but LGTM. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark issue #19045: [WIP][SPARK-20628][CORE][K8S] Keep track of nodes (/ spo...

2018-10-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19045 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19045: [WIP][SPARK-20628][CORE][K8S] Keep track of nodes (/ spo...

2018-10-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19045 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-make-spark-distribution-unified/4542/ ---

[GitHub] spark issue #22824: [SPARK-25834] [Structured Streaming]Update Mode should n...

2018-10-26 Thread koeninger
Github user koeninger commented on the issue: https://github.com/apache/spark/pull/22824 @arunmahadevan I think this is clear, even if it is redundant. --- - To unsubscribe, e-mail:

[GitHub] spark issue #22854: [SPARK-25854] fix mvn to not always exit 1

2018-10-26 Thread shaneknapp
Github user shaneknapp commented on the issue: https://github.com/apache/spark/pull/22854 `mvn clean package` script logic worked! ``` [INFO] [INFO] BUILD SUCCESS [INFO]

[GitHub] spark issue #22852: [SPARK-25023] Clarify Spark security documentation

2018-10-26 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/22852 I'm fine with this, although I wonder if having a stronger wording about Spark standalone just not being secure would be better. Even if you enable auth, everybody needs to know the same auth

[GitHub] spark pull request #22847: [SPARK-25850][SQL] Make the split threshold for t...

2018-10-26 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22847#discussion_r228598058 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -812,6 +812,17 @@ object SQLConf { .intConf

[GitHub] spark issue #22854: [SPARK-25854] fix mvn to not always exit 1

2018-10-26 Thread shaneknapp
Github user shaneknapp commented on the issue: https://github.com/apache/spark/pull/22854 here's the test build run that's actually testing the changes to mvn: https://amplab.cs.berkeley.edu/jenkins/job/sknapp-testing-spark-branch-2.4-test-maven-hadoop-2.7/12/ ---

[GitHub] spark issue #19045: [WIP][SPARK-20628][CORE][K8S] Keep track of nodes (/ spo...

2018-10-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19045 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-make-spark-distribution-unified/4542/ ---

[GitHub] spark issue #22824: [SPARK-25834] [Structured Streaming]Update Mode should n...

2018-10-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22824 **[Test build #98101 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98101/testReport)** for PR 22824 at commit

[GitHub] spark issue #22824: [SPARK-25834] [Structured Streaming]Update Mode should n...

2018-10-26 Thread koeninger
Github user koeninger commented on the issue: https://github.com/apache/spark/pull/22824 jenkins, ok to test --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #22815: [SPARK-25821][SQL] Remove SQLContext methods deprecated ...

2018-10-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22815 **[Test build #98100 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98100/testReport)** for PR 22815 at commit

[GitHub] spark issue #22295: [SPARK-25255][PYTHON]Add getActiveSession to SparkSessio...

2018-10-26 Thread huaxingao
Github user huaxingao commented on the issue: https://github.com/apache/spark/pull/22295 Thank you very much for your help! ! @holdenk @HyukjinKwon --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

[GitHub] spark pull request #22295: [SPARK-25255][PYTHON]Add getActiveSession to Spar...

2018-10-26 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/22295 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #22854: [SPARK-25854] fix mvn to not always exit 1

2018-10-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22854 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request #22815: [SPARK-25821][SQL] Remove SQLContext methods depr...

2018-10-26 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/22815#discussion_r228594291 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/SQLContext.scala --- @@ -54,6 +54,7 @@ import org.apache.spark.sql.util.ExecutionListenerManager *

[GitHub] spark issue #22854: [SPARK-25854] fix mvn to not always exit 1

2018-10-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22854 **[Test build #98099 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98099/testReport)** for PR 22854 at commit

[GitHub] spark issue #22854: [SPARK-25854] fix mvn to not always exit 1

2018-10-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22854 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21588: [SPARK-24590][BUILD] Make Jenkins tests passed with hado...

2018-10-26 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/21588 Does this upgrade Hive for execution or also for metastore? Spark supports virtually all Hive metastore versions out there, and a lot of deployments do run different versions of Spark against the same

[GitHub] spark issue #22295: [SPARK-25255][PYTHON]Add getActiveSession to SparkSessio...

2018-10-26 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/22295 Merged to master for 3.0. Thanks for fixing this @huaxingao :) --- - To unsubscribe, e-mail:

[GitHub] spark issue #22854: [SPARK-25854] fix mvn to not always exit 1

2018-10-26 Thread shaneknapp
Github user shaneknapp commented on the issue: https://github.com/apache/spark/pull/22854 test this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #22854: [SPARK-25854] fix mvn to not always exit 1

2018-10-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22854 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98094/ Test FAILed. ---

[GitHub] spark issue #22854: [SPARK-25854] fix mvn to not always exit 1

2018-10-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22854 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19045: [WIP][SPARK-20628][CORE][K8S] Keep track of nodes (/ spo...

2018-10-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19045 **[Test build #98098 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98098/testReport)** for PR 19045 at commit

[GitHub] spark issue #20503: [SPARK-23299][SQL][PYSPARK] Fix __repr__ behaviour for R...

2018-10-26 Thread ashashwat
Github user ashashwat commented on the issue: https://github.com/apache/spark/pull/20503 @holdenk I am on it. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #22854: [SPARK-25854] fix mvn to not always exit 1

2018-10-26 Thread shaneknapp
Github user shaneknapp commented on a diff in the pull request: https://github.com/apache/spark/pull/22854#discussion_r228591941 --- Diff: build/mvn --- @@ -163,8 +163,19 @@ export MAVEN_OPTS=${MAVEN_OPTS:-"$_COMPILE_JVM_OPTS"} echo "Using \`mvn\` from path: $MVN_BIN"

[GitHub] spark issue #15670: [SPARK-18161] [Python] Allow pickle to serialize >4 GB o...

2018-10-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15670 **[Test build #98097 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98097/consoleFull)** for PR 15670 at commit

[GitHub] spark issue #22425: [SPARK-23367][Build] Include python document style check...

2018-10-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22425 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22425: [SPARK-23367][Build] Include python document style check...

2018-10-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22425 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #18339: [SPARK-21094][PYTHON] Add popen_kwargs to launch_gateway

2018-10-26 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/18339 Since @HyukjinKwon's concerns for this PR have been addressed if @parente can update this to master would be lovely to get this in for 3+ since I'm working on some multi-language pipeline stuff

[GitHub] spark issue #15670: [SPARK-18161] [Python] Allow pickle to serialize >4 GB o...

2018-10-26 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/15670 Jenkins ok to test. @inpefess if you can update this PR to master now is a great time to get this in since the next release after 2.4 is going to be 3 so it's easier to change formats and stuff.

[GitHub] spark issue #20503: [SPARK-23299][SQL][PYSPARK] Fix __repr__ behaviour for R...

2018-10-26 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/20503 Jenkins ok to test. Gentle ping again to @ashashwat - are you still interested in this PR? --- - To unsubscribe, e-mail:

[GitHub] spark issue #22425: [SPARK-23367][Build] Include python document style check...

2018-10-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22425 **[Test build #98096 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98096/testReport)** for PR 22425 at commit

[GitHub] spark issue #22425: [SPARK-23367][Build] Include python document style check...

2018-10-26 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/22425 I think the scaladoc error is unrelated, jenkins retest this please. --- - To unsubscribe, e-mail:

[GitHub] spark issue #15496: [SPARK-17950] [Python] Match SparseVector behavior with ...

2018-10-26 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/15496 While you update to master i might include in the docstring that the similar funcitonality in densevector is done with manual delegation in `_delegate`. ---

[GitHub] spark issue #15496: [SPARK-17950] [Python] Match SparseVector behavior with ...

2018-10-26 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/15496 Jenkins Ok to test --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #15496: [SPARK-17950] [Python] Match SparseVector behavior with ...

2018-10-26 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/15496 Sorry for letting this slide for so long. This looks really close, I think now that we don't have append I don't have the concerns with the copy any more. Can you update this to master and we can

[GitHub] spark pull request #22854: [SPARK-25854] fix mvn to not always exit 1

2018-10-26 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/22854#discussion_r228588061 --- Diff: build/mvn --- @@ -163,8 +163,19 @@ export MAVEN_OPTS=${MAVEN_OPTS:-"$_COMPILE_JVM_OPTS"} echo "Using \`mvn\` from path: $MVN_BIN"

[GitHub] spark issue #22775: [SPARK-24709][SQL][FOLLOW-UP] Make schema_of_json's inpu...

2018-10-26 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/22775 Actually this is not that hard. The conflict comes from the fact that in 2.4 `schema_of_json` doesn't take `option` parameter. I've fixed the conflict and pushed to 2.4. You can take a

[GitHub] spark pull request #22854: [SPARK-25854] fix mvn to not always exit 1

2018-10-26 Thread shaneknapp
Github user shaneknapp commented on a diff in the pull request: https://github.com/apache/spark/pull/22854#discussion_r228584294 --- Diff: build/mvn --- @@ -163,8 +163,19 @@ export MAVEN_OPTS=${MAVEN_OPTS:-"$_COMPILE_JVM_OPTS"} echo "Using \`mvn\` from path: $MVN_BIN"

[GitHub] spark issue #21654: [SPARK-24671][PySpark] DataFrame length using a dunder/m...

2018-10-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21654 **[Test build #98095 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98095/testReport)** for PR 21654 at commit

[GitHub] spark issue #22854: [SPARK-25854] fix mvn to not always exit 1

2018-10-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22854 **[Test build #98094 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98094/testReport)** for PR 22854 at commit

[GitHub] spark pull request #22854: [SPARK-25854] fix mvn to not always exit 1

2018-10-26 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/22854#discussion_r228583684 --- Diff: build/mvn --- @@ -163,8 +163,19 @@ export MAVEN_OPTS=${MAVEN_OPTS:-"$_COMPILE_JVM_OPTS"} echo "Using \`mvn\` from path: $MVN_BIN" 1>&2

[GitHub] spark issue #22854: [SPARK-25854] fix mvn to not always exit 1

2018-10-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22854 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #21654: [SPARK-24671][PySpark] DataFrame length using a dunder/m...

2018-10-26 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/21654 hey @kokes this is out of sync with master, can you merge in the latest master? I'm going to follow up on the dev@ list for the plan which @HyukjinKwon wants to see (please feel free to join in

[GitHub] spark issue #22854: [SPARK-25854] fix mvn to not always exit 1

2018-10-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22854 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21654: [SPARK-24671][PySpark] DataFrame length using a dunder/m...

2018-10-26 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/21654 Jenkins, ok to test. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #22598: [SPARK-25501][SS] Add kafka delegation token supp...

2018-10-26 Thread arunmahadevan
Github user arunmahadevan commented on a diff in the pull request: https://github.com/apache/spark/pull/22598#discussion_r228583252 --- Diff: core/src/main/scala/org/apache/spark/deploy/security/KafkaDelegationTokenProvider.scala --- @@ -0,0 +1,65 @@ +/* + * Licensed to

[GitHub] spark pull request #22854: [SPARK-25854] fix mvn to not always exit 1

2018-10-26 Thread shaneknapp
GitHub user shaneknapp opened a pull request: https://github.com/apache/spark/pull/22854 [SPARK-25854] fix mvn to not always exit 1 ## What changes were proposed in this pull request? the final line in the mvn helper script in build/ attempts to shut down the zinc server.

[GitHub] spark issue #22853: [SPARK-25845][SQL] Fix MatchError for calendar interval ...

2018-10-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22853 **[Test build #98093 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98093/testReport)** for PR 22853 at commit

[GitHub] spark issue #22853: [SPARK-25845][SQL] Fix MatchError for calendar interval ...

2018-10-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22853 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22853: [SPARK-25845][SQL] Fix MatchError for calendar interval ...

2018-10-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22853 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #22838: [SPARK-25835][K8s] Create kubernetes-tests profile and u...

2018-10-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22838 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98085/ Test PASSed. ---

[GitHub] spark issue #22838: [SPARK-25835][K8s] Create kubernetes-tests profile and u...

2018-10-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22838 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #21348: [SPARK-22739][Catalyst] Additional Expression Sup...

2018-10-26 Thread bdrillard
Github user bdrillard commented on a diff in the pull request: https://github.com/apache/spark/pull/21348#discussion_r228577979 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/objects/objects.scala --- @@ -1799,3 +1805,65 @@ case class

[GitHub] spark issue #22838: [SPARK-25835][K8s] Create kubernetes-tests profile and u...

2018-10-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22838 **[Test build #98085 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98085/testReport)** for PR 22838 at commit

[GitHub] spark pull request #22853: [SPARK-25845][SQL] Fix MatchError for calendar in...

2018-10-26 Thread jiangxb1987
GitHub user jiangxb1987 opened a pull request: https://github.com/apache/spark/pull/22853 [SPARK-25845][SQL] Fix MatchError for calendar interval type in range frame left boundary ## What changes were proposed in this pull request? WindowSpecDefinition checks start < last,

[GitHub] spark issue #22850: [MINOR][DOC] Fix comment error of HiveUtils

2018-10-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22850 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22850: [MINOR][DOC] Fix comment error of HiveUtils

2018-10-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22850 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98087/ Test PASSed. ---

[GitHub] spark pull request #22827: [SPARK-25832][SQL][BRANCH-2.4] Revert newly added...

2018-10-26 Thread gatorsmile
Github user gatorsmile closed the pull request at: https://github.com/apache/spark/pull/22827 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #22850: [MINOR][DOC] Fix comment error of HiveUtils

2018-10-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22850 **[Test build #98087 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98087/testReport)** for PR 22850 at commit

[GitHub] spark pull request #22598: [SPARK-25501][SS] Add kafka delegation token supp...

2018-10-26 Thread soenkeliebau
Github user soenkeliebau commented on a diff in the pull request: https://github.com/apache/spark/pull/22598#discussion_r228576259 --- Diff: core/src/main/scala/org/apache/spark/deploy/security/KafkaDelegationTokenProvider.scala --- @@ -0,0 +1,65 @@ +/* + * Licensed to

[GitHub] spark issue #21348: [SPARK-22739][Catalyst] Additional Expression Support fo...

2018-10-26 Thread bdrillard
Github user bdrillard commented on the issue: https://github.com/apache/spark/pull/21348 Hi @xuanyuanking, if you'd like to take on the work to fold my prior work in databricks/spark-avro#217 into Spark, that sounds good to me. Please do include me in pull-requests on this topic. Our

[GitHub] spark issue #22802: [SPARK-25806][SQL]The instance of FileSplit is redundant...

2018-10-26 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/22802 OK, that's pretty trivial, but I agree. Any other instances of this type of thing? --- - To unsubscribe, e-mail:

[GitHub] spark issue #21348: [SPARK-22739][Catalyst] Additional Expression Support fo...

2018-10-26 Thread xuanyuanking
Github user xuanyuanking commented on the issue: https://github.com/apache/spark/pull/21348 The jira SPARK-25789 guide me here, thanks for @bdrillard your great job, we also meet the requirement on supporting dataset of avro during Structure Streaming. I'm adapting your code in

[GitHub] spark pull request #21348: [SPARK-22739][Catalyst] Additional Expression Sup...

2018-10-26 Thread xuanyuanking
Github user xuanyuanking commented on a diff in the pull request: https://github.com/apache/spark/pull/21348#discussion_r228563440 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/objects/objects.scala --- @@ -1799,3 +1805,65 @@ case class

[GitHub] spark issue #22843: [SPARK-16693][SPARKR] Remove methods deprecated

2018-10-26 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/22843 I personally am OK with removing anything deprecated before 2.4. Even things deprecated in 2.4 are technically fair game. If any of these were deprecated pretty recently, might be debatable, but I

[GitHub] spark issue #22764: [SPARK-25765][ML] Add training cost to BisectingKMeans s...

2018-10-26 Thread WeichenXu123
Github user WeichenXu123 commented on the issue: https://github.com/apache/spark/pull/22764 I think it can target for 3.0. since 2.4 will be released soon and this PR looks a little complex and need take some time to check carefully. ---

[GitHub] spark issue #22848: [SPARK-25851][SQL][MINOR] Fix deprecated API warning in ...

2018-10-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22848 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22848: [SPARK-25851][SQL][MINOR] Fix deprecated API warning in ...

2018-10-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22848 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #22848: [SPARK-25851][SQL][MINOR] Fix deprecated API warning in ...

2018-10-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22848 **[Test build #98092 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98092/testReport)** for PR 22848 at commit

[GitHub] spark pull request #22848: [SPARK-25851][SQL][MINOR] Fix deprecated API warn...

2018-10-26 Thread gengliangwang
Github user gengliangwang commented on a diff in the pull request: https://github.com/apache/spark/pull/22848#discussion_r228559067 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/ui/SQLListener.scala --- @@ -89,12 +89,12 @@ private class LongLongTupleConverter

<    1   2   3   4   5   >