[GitHub] spark pull request #22504: [SPARK-25118][Submit] Persist Driver Logs in Clie...

2018-10-26 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/22504#discussion_r228669149 --- Diff: core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala --- @@ -274,11 +275,20 @@ private[history] class

[GitHub] spark issue #22852: [SPARK-25023] Clarify Spark security documentation

2018-10-26 Thread tgravescs
Github user tgravescs commented on the issue: https://github.com/apache/spark/pull/22852 I would rather see someone more familiar with K8s that uses it document it. --- - To unsubscribe, e-mail:

[GitHub] spark pull request #22854: [SPARK-25854][BUILD] fix `build/mvn` not to fail ...

2018-10-26 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/22854 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #22820: [SPARK-25828][K8S] Bumping Kubernetes-Client version to ...

2018-10-26 Thread ifilonenko
Github user ifilonenko commented on the issue: https://github.com/apache/spark/pull/22820 @felixcheung and @erikerlandson for merge --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22854: [SPARK-25854][BUILD] fix `build/mvn` not to fail during ...

2018-10-26 Thread shaneknapp
Github user shaneknapp commented on the issue: https://github.com/apache/spark/pull/22854 yeah... wasn't worried about the PRB failing. thanks for the merge/backport! --- - To unsubscribe, e-mail:

[GitHub] spark issue #22820: [SPARK-25828][K8S] Bumping Kubernetes-Client version to ...

2018-10-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22820 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22820: [SPARK-25828][K8S] Bumping Kubernetes-Client version to ...

2018-10-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22820 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98102/ Test PASSed. ---

[GitHub] spark issue #22852: [SPARK-25023] Clarify Spark security documentation

2018-10-26 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/22852 I don't feel strongly about it; go ahead. If someone lands on this page, do they pretty easily come away with the impression they need to set spark.authenticate and network security if they

[GitHub] spark issue #22854: [SPARK-25854][BUILD] fix `build/mvn` not to fail during ...

2018-10-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22854 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22854: [SPARK-25854][BUILD] fix `build/mvn` not to fail during ...

2018-10-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22854 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98104/ Test FAILed. ---

[GitHub] spark issue #22820: [SPARK-25828][K8S] Bumping Kubernetes-Client version to ...

2018-10-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22820 **[Test build #98102 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98102/testReport)** for PR 22820 at commit

[GitHub] spark issue #22854: [SPARK-25854][BUILD] fix `build/mvn` not to fail during ...

2018-10-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22854 **[Test build #98104 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98104/testReport)** for PR 22854 at commit

[GitHub] spark issue #22852: [SPARK-25023] Clarify Spark security documentation

2018-10-26 Thread tgravescs
Github user tgravescs commented on the issue: https://github.com/apache/spark/pull/22852 so I had filed a jira to update mesos docs more detail about security things (https://issues.apache.org/jira/browse/SPARK-25024) which I need to follow up on, but I didn't file one for k8s. It

[GitHub] spark pull request #22598: [SPARK-25501][SS] Add kafka delegation token supp...

2018-10-26 Thread arunmahadevan
Github user arunmahadevan commented on a diff in the pull request: https://github.com/apache/spark/pull/22598#discussion_r228671263 --- Diff: core/src/main/scala/org/apache/spark/deploy/security/HadoopDelegationTokenManager.scala --- @@ -66,7 +66,8 @@ private[spark] class

[GitHub] spark pull request #22288: [SPARK-22148][SPARK-15815][Scheduler] Acquire new...

2018-10-26 Thread tgravescs
Github user tgravescs commented on a diff in the pull request: https://github.com/apache/spark/pull/22288#discussion_r228668756 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala --- @@ -415,9 +420,54 @@ private[spark] class TaskSchedulerImpl(

[GitHub] spark issue #22288: [SPARK-22148][SPARK-15815][Scheduler] Acquire new execut...

2018-10-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22288 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #22288: [SPARK-22148][SPARK-15815][Scheduler] Acquire new execut...

2018-10-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22288 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22288: [SPARK-22148][SPARK-15815][Scheduler] Acquire new execut...

2018-10-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22288 **[Test build #98107 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98107/testReport)** for PR 22288 at commit

[GitHub] spark pull request #22598: [SPARK-25501][SS] Add kafka delegation token supp...

2018-10-26 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/22598#discussion_r228667186 --- Diff: core/src/main/scala/org/apache/spark/deploy/security/KafkaDelegationTokenProvider.scala --- @@ -0,0 +1,65 @@ +/* + * Licensed to the

[GitHub] spark issue #22854: [SPARK-25854][BUILD] fix `build/mvn` not to fail during ...

2018-10-26 Thread shaneknapp
Github user shaneknapp commented on the issue: https://github.com/apache/spark/pull/22854 build is green, and everything looks to be behaving normally! https://amplab.cs.berkeley.edu/jenkins/job/sknapp-testing-spark-branch-2.4-test-maven-hadoop-2.7/12/console ready

[GitHub] spark pull request #22809: [SPARK-19851][SQL] Add support for EVERY and ANY ...

2018-10-26 Thread dilipbiswal
Github user dilipbiswal commented on a diff in the pull request: https://github.com/apache/spark/pull/22809#discussion_r228666789 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/UnevaluableAggs.scala --- @@ -0,0 +1,62 @@ +/* + *

[GitHub] spark pull request #22598: [SPARK-25501][SS] Add kafka delegation token supp...

2018-10-26 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/22598#discussion_r228666433 --- Diff: core/src/main/scala/org/apache/spark/deploy/security/KafkaDelegationTokenProvider.scala --- @@ -0,0 +1,65 @@ +/* + * Licensed to the

[GitHub] spark pull request #22598: [SPARK-25501][SS] Add kafka delegation token supp...

2018-10-26 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/22598#discussion_r228665965 --- Diff: core/src/main/scala/org/apache/spark/deploy/security/HadoopDelegationTokenManager.scala --- @@ -66,7 +66,8 @@ private[spark] class

[GitHub] spark issue #19045: [WIP][SPARK-20628][CORE][K8S] Keep track of nodes (/ spo...

2018-10-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19045 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98098/ Test PASSed. ---

[GitHub] spark issue #19045: [WIP][SPARK-20628][CORE][K8S] Keep track of nodes (/ spo...

2018-10-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19045 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19045: [WIP][SPARK-20628][CORE][K8S] Keep track of nodes (/ spo...

2018-10-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19045 **[Test build #98098 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98098/testReport)** for PR 19045 at commit

[GitHub] spark issue #22852: [SPARK-25023] Clarify Spark security documentation

2018-10-26 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/22852 I looked at the existing docs after I wrote my comment and saw that paragraph. I think it's fine, maybe just in need of some update now that we have k8s, although I'm not sure yet how

[GitHub] spark issue #22425: [SPARK-23367][Build] Include python document style check...

2018-10-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22425 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98096/ Test PASSed. ---

[GitHub] spark issue #22425: [SPARK-23367][Build] Include python document style check...

2018-10-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22425 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22856: [SPARK-25856][SQL][MINOR] Remove AverageLike and CountLi...

2018-10-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22856 **[Test build #98106 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98106/testReport)** for PR 22856 at commit

[GitHub] spark issue #22856: [SPARK-25856][SQL][MINOR] Remove AverageLike and CountLi...

2018-10-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22856 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22852: [SPARK-25023] Clarify Spark security documentation

2018-10-26 Thread tgravescs
Github user tgravescs commented on the issue: https://github.com/apache/spark/pull/22852 we can add stronger wording for standalone if you want, I know the text was recently updated (I believe by you) to have the below: > For other resource managers,

[GitHub] spark issue #22856: [SPARK-25856][SQL][MINOR] Remove AverageLike and CountLi...

2018-10-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22856 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #22425: [SPARK-23367][Build] Include python document style check...

2018-10-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22425 **[Test build #98096 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98096/testReport)** for PR 22425 at commit

[GitHub] spark pull request #22856: [SPARK-25856][SQL][MINOR] Remove AverageLike and ...

2018-10-26 Thread dilipbiswal
GitHub user dilipbiswal opened a pull request: https://github.com/apache/spark/pull/22856 [SPARK-25856][SQL][MINOR] Remove AverageLike and CountLike classes ## What changes were proposed in this pull request? These two classes were added for regr_ expression support

[GitHub] spark issue #18339: [SPARK-21094][PYTHON] Add popen_kwargs to launch_gateway

2018-10-26 Thread parente
Github user parente commented on the issue: https://github.com/apache/spark/pull/18339 @holdenk Took a note to look at this weekend. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21654: [SPARK-24671][PySpark] DataFrame length using a dunder/m...

2018-10-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21654 Build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #21654: [SPARK-24671][PySpark] DataFrame length using a dunder/m...

2018-10-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21654 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98095/ Test FAILed. ---

[GitHub] spark issue #21654: [SPARK-24671][PySpark] DataFrame length using a dunder/m...

2018-10-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21654 **[Test build #98095 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98095/testReport)** for PR 21654 at commit

[GitHub] spark issue #22815: [SPARK-25821][SQL] Remove SQLContext methods deprecated ...

2018-10-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22815 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22815: [SPARK-25821][SQL] Remove SQLContext methods deprecated ...

2018-10-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22815 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98100/ Test PASSed. ---

[GitHub] spark issue #22815: [SPARK-25821][SQL] Remove SQLContext methods deprecated ...

2018-10-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22815 **[Test build #98100 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98100/testReport)** for PR 22815 at commit

[GitHub] spark issue #21710: [SPARK-24207][R]add R API for PrefixSpan

2018-10-26 Thread huaxingao
Github user huaxingao commented on the issue: https://github.com/apache/spark/pull/21710 @felixcheung I am terribly sorry that I missed your comment for the ml doc and example for 2.4. Is it still time to merge in 2.4? I saw one of my PR got merged in 2.4 last night. I can submit a

[GitHub] spark issue #22624: [SPARK-23781][CORE] Merge token renewer functionality in...

2018-10-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22624 **[Test build #98105 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98105/testReport)** for PR 22624 at commit

[GitHub] spark issue #22624: [SPARK-23781][CORE] Merge token renewer functionality in...

2018-10-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22624 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22624: [SPARK-23781][CORE] Merge token renewer functionality in...

2018-10-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22624 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #22624: [SPARK-23781][CORE] Merge token renewer functionality in...

2018-10-26 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/22624 I tested the last update with yarn cluster and client, both seem happy. --- - To unsubscribe, e-mail:

[GitHub] spark issue #22824: [SPARK-25834] [Structured Streaming]Update Mode should n...

2018-10-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22824 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98101/ Test FAILed. ---

[GitHub] spark issue #22824: [SPARK-25834] [Structured Streaming]Update Mode should n...

2018-10-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22824 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22824: [SPARK-25834] [Structured Streaming]Update Mode should n...

2018-10-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22824 **[Test build #98101 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98101/testReport)** for PR 22824 at commit

[GitHub] spark issue #22624: [SPARK-23781][CORE] Merge token renewer functionality in...

2018-10-26 Thread squito
Github user squito commented on the issue: https://github.com/apache/spark/pull/22624 >> Is there any place which documents this overall flow? > > Not yet. I plan to write a README.md explaining this, separately, since there's a lot of confusion not just about the flow,

[GitHub] spark pull request #22809: [SPARK-19851][SQL] Add support for EVERY and ANY ...

2018-10-26 Thread dilipbiswal
Github user dilipbiswal commented on a diff in the pull request: https://github.com/apache/spark/pull/22809#discussion_r228644504 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/UnevaluableAggs.scala --- @@ -0,0 +1,62 @@ +/* + *

[GitHub] spark pull request #22624: [SPARK-23781][CORE] Merge token renewer functiona...

2018-10-26 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/22624#discussion_r228644038 --- Diff: core/src/main/scala/org/apache/spark/deploy/security/HadoopDelegationTokenManager.scala --- @@ -17,76 +17,175 @@ package

[GitHub] spark pull request #22288: [SPARK-22148][SPARK-15815][Scheduler] Acquire new...

2018-10-26 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/22288#discussion_r228642238 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala --- @@ -415,9 +420,54 @@ private[spark] class TaskSchedulerImpl(

[GitHub] spark issue #22853: [SPARK-25845][SQL] Fix MatchError for calendar interval ...

2018-10-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22853 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98093/ Test PASSed. ---

[GitHub] spark issue #22853: [SPARK-25845][SQL] Fix MatchError for calendar interval ...

2018-10-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22853 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #22288: [SPARK-22148][SPARK-15815][Scheduler] Acquire new...

2018-10-26 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/22288#discussion_r228642015 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala --- @@ -415,9 +420,54 @@ private[spark] class TaskSchedulerImpl(

[GitHub] spark issue #22853: [SPARK-25845][SQL] Fix MatchError for calendar interval ...

2018-10-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22853 **[Test build #98093 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98093/testReport)** for PR 22853 at commit

[GitHub] spark pull request #22809: [SPARK-19851][SQL] Add support for EVERY and ANY ...

2018-10-26 Thread mgaido91
Github user mgaido91 commented on a diff in the pull request: https://github.com/apache/spark/pull/22809#discussion_r228640680 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/UnevaluableAggs.scala --- @@ -0,0 +1,62 @@ +/* + *

[GitHub] spark pull request #22809: [SPARK-19851][SQL] Add support for EVERY and ANY ...

2018-10-26 Thread dilipbiswal
Github user dilipbiswal commented on a diff in the pull request: https://github.com/apache/spark/pull/22809#discussion_r228638310 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/UnevaluableAggs.scala --- @@ -0,0 +1,62 @@ +/* + *

[GitHub] spark issue #22855: [SPARK-25839] [Core] Implement use of KryoPool in KryoSe...

2018-10-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22855 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22855: [SPARK-25839] [Core] Implement use of KryoPool in KryoSe...

2018-10-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22855 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22855: [SPARK-25839] [Core] Implement use of KryoPool in KryoSe...

2018-10-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22855 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #22288: [SPARK-22148][SPARK-15815][Scheduler] Acquire new...

2018-10-26 Thread dhruve
Github user dhruve commented on a diff in the pull request: https://github.com/apache/spark/pull/22288#discussion_r228637254 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala --- @@ -415,9 +420,54 @@ private[spark] class TaskSchedulerImpl(

[GitHub] spark pull request #22288: [SPARK-22148][SPARK-15815][Scheduler] Acquire new...

2018-10-26 Thread dhruve
Github user dhruve commented on a diff in the pull request: https://github.com/apache/spark/pull/22288#discussion_r228636999 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala --- @@ -415,9 +420,54 @@ private[spark] class TaskSchedulerImpl(

[GitHub] spark pull request #22809: [SPARK-19851][SQL] Add support for EVERY and ANY ...

2018-10-26 Thread mgaido91
Github user mgaido91 commented on a diff in the pull request: https://github.com/apache/spark/pull/22809#discussion_r228637047 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/Expression.scala --- @@ -282,6 +283,31 @@ trait RuntimeReplaceable

[GitHub] spark pull request #22855: [SPARK-25839] [Core] Implement use of KryoPool in...

2018-10-26 Thread patrickbrownsync
GitHub user patrickbrownsync opened a pull request: https://github.com/apache/spark/pull/22855 [SPARK-25839] [Core] Implement use of KryoPool in KryoSerializer ## What changes were proposed in this pull request? * Implement (optional) use of KryoPool in KryoSerializer, an

[GitHub] spark pull request #22288: [SPARK-22148][SPARK-15815][Scheduler] Acquire new...

2018-10-26 Thread dhruve
Github user dhruve commented on a diff in the pull request: https://github.com/apache/spark/pull/22288#discussion_r228636880 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala --- @@ -415,9 +420,54 @@ private[spark] class TaskSchedulerImpl(

[GitHub] spark issue #15670: [SPARK-18161] [Python] Allow pickle to serialize >4 GB o...

2018-10-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15670 Build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #15670: [SPARK-18161] [Python] Allow pickle to serialize >4 GB o...

2018-10-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15670 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98097/ Test FAILed. ---

[GitHub] spark issue #15670: [SPARK-18161] [Python] Allow pickle to serialize >4 GB o...

2018-10-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15670 **[Test build #98097 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98097/consoleFull)** for PR 15670 at commit

[GitHub] spark pull request #22809: [SPARK-19851][SQL] Add support for EVERY and ANY ...

2018-10-26 Thread dilipbiswal
Github user dilipbiswal commented on a diff in the pull request: https://github.com/apache/spark/pull/22809#discussion_r228636110 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/Expression.scala --- @@ -282,6 +283,31 @@ trait RuntimeReplaceable

[GitHub] spark pull request #22809: [SPARK-19851][SQL] Add support for EVERY and ANY ...

2018-10-26 Thread mgaido91
Github user mgaido91 commented on a diff in the pull request: https://github.com/apache/spark/pull/22809#discussion_r228635413 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/UnevaluableAggs.scala --- @@ -0,0 +1,62 @@ +/* + *

[GitHub] spark pull request #22809: [SPARK-19851][SQL] Add support for EVERY and ANY ...

2018-10-26 Thread mgaido91
Github user mgaido91 commented on a diff in the pull request: https://github.com/apache/spark/pull/22809#discussion_r228635201 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/Expression.scala --- @@ -282,6 +283,31 @@ trait RuntimeReplaceable

[GitHub] spark pull request #22809: [SPARK-19851][SQL] Add support for EVERY and ANY ...

2018-10-26 Thread dilipbiswal
Github user dilipbiswal commented on a diff in the pull request: https://github.com/apache/spark/pull/22809#discussion_r228634827 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/Expression.scala --- @@ -282,6 +283,31 @@ trait RuntimeReplaceable

[GitHub] spark pull request #22809: [SPARK-19851][SQL] Add support for EVERY and ANY ...

2018-10-26 Thread dilipbiswal
Github user dilipbiswal commented on a diff in the pull request: https://github.com/apache/spark/pull/22809#discussion_r228634159 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/UnevaluableAggs.scala --- @@ -0,0 +1,62 @@ +/* + *

[GitHub] spark pull request #22809: [SPARK-19851][SQL] Add support for EVERY and ANY ...

2018-10-26 Thread mgaido91
Github user mgaido91 commented on a diff in the pull request: https://github.com/apache/spark/pull/22809#discussion_r228633284 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/UnevaluableAggs.scala --- @@ -0,0 +1,62 @@ +/* + *

[GitHub] spark issue #22624: [SPARK-23781][CORE] Merge token renewer functionality in...

2018-10-26 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/22624 > Is there any place which documents this overall flow? Not yet. I plan to write a README.md explaining this, separately, since there's a lot of confusion not just about the flow, but

[GitHub] spark pull request #22809: [SPARK-19851][SQL] Add support for EVERY and ANY ...

2018-10-26 Thread mgaido91
Github user mgaido91 commented on a diff in the pull request: https://github.com/apache/spark/pull/22809#discussion_r228631452 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/Expression.scala --- @@ -282,6 +283,31 @@ trait RuntimeReplaceable

[GitHub] spark pull request #22624: [SPARK-23781][CORE] Merge token renewer functiona...

2018-10-26 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/22624#discussion_r228630414 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/security/KubernetesHadoopDelegationTokenManager.scala --- @@ -18,45

[GitHub] spark pull request #22624: [SPARK-23781][CORE] Merge token renewer functiona...

2018-10-26 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/22624#discussion_r228630089 --- Diff: core/src/main/scala/org/apache/spark/deploy/security/HadoopDelegationTokenManager.scala --- @@ -110,32 +209,105 @@ private[spark] class

[GitHub] spark pull request #22624: [SPARK-23781][CORE] Merge token renewer functiona...

2018-10-26 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/22624#discussion_r228628515 --- Diff: core/src/main/scala/org/apache/spark/deploy/security/HadoopDelegationTokenManager.scala --- @@ -17,76 +17,175 @@ package

[GitHub] spark issue #22817: [SPARK-25816][SQL] Fix attribute resolution in nested ex...

2018-10-26 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/22817 Will take a look at this tonight. Thanks for reporting this! --- - To unsubscribe, e-mail:

[GitHub] spark issue #22848: [SPARK-25851][SQL][MINOR] Fix deprecated API warning in ...

2018-10-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22848 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22848: [SPARK-25851][SQL][MINOR] Fix deprecated API warning in ...

2018-10-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22848 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98092/ Test PASSed. ---

[GitHub] spark issue #22824: [SPARK-25834] [Structured Streaming]Update Mode should n...

2018-10-26 Thread arunmahadevan
Github user arunmahadevan commented on the issue: https://github.com/apache/spark/pull/22824 I mean its easy to miss if a new "case" is added and "update" mode is not supported. Even now how about LeftSemi, LeftAnti, FullOuter etc? ---

[GitHub] spark issue #22848: [SPARK-25851][SQL][MINOR] Fix deprecated API warning in ...

2018-10-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22848 **[Test build #98092 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98092/testReport)** for PR 22848 at commit

[GitHub] spark issue #22854: [SPARK-25854][BUILD] fix `build/mvn` not to fail during ...

2018-10-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22854 **[Test build #98104 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98104/testReport)** for PR 22854 at commit

[GitHub] spark issue #22854: [SPARK-25854][BUILD] fix `build/mvn` not to fail during ...

2018-10-26 Thread shaneknapp
Github user shaneknapp commented on the issue: https://github.com/apache/spark/pull/22854 > Merged build finished. Test FAILed. that was me killing the 1st PRB build --- - To unsubscribe, e-mail:

[GitHub] spark issue #22854: [SPARK-25854][BUILD] fix `build/mvn` not to fail during ...

2018-10-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22854 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #22854: [SPARK-25854][BUILD] fix `build/mvn` not to fail during ...

2018-10-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22854 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98099/ Test FAILed. ---

[GitHub] spark issue #22854: [SPARK-25854][BUILD] fix `build/mvn` not to fail during ...

2018-10-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22854 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22854: [SPARK-25854][BUILD] fix `build/mvn` not to fail during ...

2018-10-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22854 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22854: [SPARK-25854][BUILD] fix `build/mvn` not to fail during ...

2018-10-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22854 **[Test build #98103 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98103/testReport)** for PR 22854 at commit

[GitHub] spark pull request #22854: [SPARK-25854][BUILD] fix `build/mvn` not to fail ...

2018-10-26 Thread shaneknapp
Github user shaneknapp commented on a diff in the pull request: https://github.com/apache/spark/pull/22854#discussion_r228616782 --- Diff: build/mvn --- @@ -163,8 +163,14 @@ export MAVEN_OPTS=${MAVEN_OPTS:-"$_COMPILE_JVM_OPTS"} echo "Using \`mvn\` from path: $MVN_BIN"

[GitHub] spark pull request #22843: [SPARK-16693][SPARKR] Remove methods deprecated

2018-10-26 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/22843#discussion_r228616400 --- Diff: R/pkg/tests/fulltests/test_sparkSQL.R --- @@ -3477,39 +3447,6 @@ test_that("Window functions on a DataFrame", { expect_equal(result,

[GitHub] spark issue #22820: [SPARK-25828][K8S] Bumping Kubernetes-Client version to ...

2018-10-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22820 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #22820: [SPARK-25828][K8S] Bumping Kubernetes-Client version to ...

2018-10-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22820 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-make-spark-distribution-unified/4545/ ---

[GitHub] spark issue #22820: [SPARK-25828][K8S] Bumping Kubernetes-Client version to ...

2018-10-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22820 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #22288: [SPARK-22148][SPARK-15815][Scheduler] Acquire new...

2018-10-26 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/22288#discussion_r228608360 --- Diff: core/src/test/scala/org/apache/spark/scheduler/TaskSchedulerImplSuite.scala --- @@ -503,6 +507,145 @@ class TaskSchedulerImplSuite extends

<    1   2   3   4   5   >