[jira] [Assigned] (SPARK-26742) Bump Kubernetes Client Version to 4.1.2

[ https://issues.apache.org/jira/browse/SPARK-26742?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-26742: -- Assignee: Jiaxin Shan > Bump Kubernetes Client Version to 4.

Re: [discuss] 2.4.1-rcX release, k8s client PRs, build system infrastructure update

Sounds good. On Wed, Mar 13, 2019 at 12:17 PM shane knapp wrote: > > ok awesome. let's shoot for 3pm PST. > > On Wed, Mar 13, 2019 at 11:59 AM Marcelo Vanzin wrote: >> >> On Wed, Mar 13, 2019 at 11:53 AM shane knapp wrote: >> > On Wed, Mar 13, 2019 a

Re: [discuss] 2.4.1-rcX release, k8s client PRs, build system infrastructure update

On Wed, Mar 13, 2019 at 11:53 AM shane knapp wrote: > On Wed, Mar 13, 2019 at 11:49 AM Marcelo Vanzin wrote: >> >> Do the upgraded minikube/k8s versions break the current master client >> version too? >> > yes. Ah, so that part kinda sucks. Let's do this

Re: [discuss] 2.4.1-rcX release, k8s client PRs, build system infrastructure update

Do the upgraded minikube/k8s versions break the current master client version too? I'm not super concerned about 2.4 integration tests being broken for a little bit. It's very uncommon for new PRs to be open against branch-2.4 that would affect k8s. But I really don't want master to break. So if

[jira] [Resolved] (SPARK-26103) OutOfMemory error with large query plans

[ https://issues.apache.org/jira/browse/SPARK-26103?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-26103. Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 23169 [https

[jira] [Assigned] (SPARK-26103) OutOfMemory error with large query plans

[ https://issues.apache.org/jira/browse/SPARK-26103?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-26103: -- Assignee: Dave DeCaprio > OutOfMemory error with large query pl

[jira] [Assigned] (LIVY-568) Upgrade jackson version from 2.9.5 to 2.9.8

[ https://issues.apache.org/jira/browse/LIVY-568?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned LIVY-568: --- Assignee: Gyorgy Gal > Upgrade jackson version from 2.9.5 to 2.

[jira] [Assigned] (SPARK-26927) Race condition may cause dynamic allocation not working

[ https://issues.apache.org/jira/browse/SPARK-26927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-26927: -- Assignee: liupengcheng > Race condition may cause dynamic allocation not work

[jira] [Resolved] (SPARK-26927) Race condition may cause dynamic allocation not working

[ https://issues.apache.org/jira/browse/SPARK-26927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-26927. Resolution: Fixed Fix Version/s: 2.3.4 2.4.2

Re: Livy-0.6 release?

e is, shall we enable it by default or leave it disabled? > > Thanks > Saisai > > Jeff Zhang 于2019年3月12日周二 上午10:54写道： >> >> Thanks Marcelo, I can help to test it in zeppelin side which use livy as one >> interpreter. >> >> Marcelo Vanzin 于2019年3月12日周二

[jira] [Resolved] (LIVY-568) Upgrade jackson version from 2.9.5 to 2.9.8

[ https://issues.apache.org/jira/browse/LIVY-568?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved LIVY-568. - Resolution: Fixed Fix Version/s: 0.6.0 Issue resolved by pull request 153 [https

Re: Livy-0.6 release?

; Got it. Thanks Marcelo. >> >> I see LIVY-551 is now part of the master. Hope to see Livy 0.6 perhaps soon. >> >> >> Thank you! >> Ruslan Dautkhanov >> >> >> On Tue, Feb 5, 2019 at 12:38 PM Marcelo Vanzin wrote: >>> >>> I thi

[jira] [Resolved] (SPARK-27073) Fix a race condition when handling of IdleStateEvent

[ https://issues.apache.org/jira/browse/SPARK-27073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-27073. Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 23989 [https

[jira] [Resolved] (SPARK-27004) Code for https uri authentication in Spark Submit needs to be removed

[ https://issues.apache.org/jira/browse/SPARK-27004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-27004. Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 24033 [https

[jira] [Assigned] (SPARK-27004) Code for https uri authentication in Spark Submit needs to be removed

[ https://issues.apache.org/jira/browse/SPARK-27004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-27004: -- Assignee: Marcelo Vanzin > Code for https uri authentication in Spark Submit ne

[jira] [Resolved] (SPARK-27129) Add JSON event serialization methods in JSONUtils for blacklisted events

[ https://issues.apache.org/jira/browse/SPARK-27129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-27129. Resolution: Invalid Unless you can show why this (or in this case the lack of this) is a

[jira] [Assigned] (SPARK-27061) Expose Driver UI port on driver service to access UI using service

[ https://issues.apache.org/jira/browse/SPARK-27061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-27061: -- Assignee: Chandu Kavar > Expose Driver UI port on driver service to access UI us

[jira] [Resolved] (SPARK-27061) Expose Driver UI port on driver service to access UI using service

[ https://issues.apache.org/jira/browse/SPARK-27061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-27061. Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 23990 [https

[jira] [Commented] (SPARK-26998) spark.ssl.keyStorePassword in plaintext on 'ps -ef' output of executor processes in Standalone mode

[ https://issues.apache.org/jira/browse/SPARK-26998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16789766#comment-16789766 ] Marcelo Vanzin commented on SPARK-26998: There are 3 ways to solve this:

[jira] [Resolved] (SPARK-27044) Maven dependency resolution does not support classifiers

[ https://issues.apache.org/jira/browse/SPARK-27044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-27044. Resolution: Duplicate > Maven dependency resolution does not support classifi

Re: [VOTE] Release Apache Spark 2.4.1 (RC6)

ing this issue. > > Should we create a new rc7? > > DB Tsai | Siri Open Source Technologies [not a contribution] |  Apple, > Inc > > > On Mar 8, 2019, at 10:54 AM, Marcelo Vanzin > > wrote: > > > > I personally find it a little weird to not have the

Re: [VOTE] Release Apache Spark 2.4.1 (RC6)

I personally find it a little weird to not have the commit in branch-2.4. Not that this would happen, but if the v2.4.1-rc6 tag is overwritten (e.g. accidentally) then you lose the reference to that commit, and then the exact commit from which the rc was generated is lost. On Fri, Mar 8, 2019 at

[jira] [Updated] (SPARK-26604) Register channel for stream request

[ https://issues.apache.org/jira/browse/SPARK-26604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin updated SPARK-26604: --- Fix Version/s: 2.4.2 2.3.4 > Register channel for stream requ

[jira] [Created] (SPARK-27094) Thread interrupt being swallowed while launching executors in YarnAllocator

Marcelo Vanzin created SPARK-27094: -- Summary: Thread interrupt being swallowed while launching executors in YarnAllocator Key: SPARK-27094 URL: https://issues.apache.org/jira/browse/SPARK-27094

[jira] [Assigned] (SPARK-27075) Sorting table column in SQL WEBUI page throws 'IllegalArgumentException'

[ https://issues.apache.org/jira/browse/SPARK-27075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-27075: -- Assignee: Shahid K I > Sorting table column in SQL WEBUI page thr

[jira] [Resolved] (SPARK-27075) Sorting table column in SQL WEBUI page throws 'IllegalArgumentException'

[ https://issues.apache.org/jira/browse/SPARK-27075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-27075. Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 23994 [https

[jira] [Resolved] (SPARK-27022) Kafka delegation token support

[ https://issues.apache.org/jira/browse/SPARK-27022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-27022. Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 23929 [https

[jira] [Assigned] (SPARK-27022) Kafka delegation token support

[ https://issues.apache.org/jira/browse/SPARK-27022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-27022: -- Assignee: Gabor Somogyi > Kafka delegation token supp

[jira] [Resolved] (SPARK-27019) Spark UI's SQL tab shows inconsistent values

[ https://issues.apache.org/jira/browse/SPARK-27019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-27019. Resolution: Fixed Fix Version/s: 2.4.2 3.0.0 Issue resolved by

[jira] [Assigned] (SPARK-27019) Spark UI's SQL tab shows inconsistent values

[ https://issues.apache.org/jira/browse/SPARK-27019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-27019: -- Assignee: Shahid K I > Spark UI's SQL tab shows inconsisten

[jira] [Commented] (SPARK-27072) Changing the parameter value of completedJob.sort to X prints stacktrace in sparkWebUI

[ https://issues.apache.org/jira/browse/SPARK-27072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16785889#comment-16785889 ] Marcelo Vanzin commented on SPARK-27072: I'd expect an error 401 here

[jira] [Resolved] (SPARK-27015) spark-submit does not properly escape arguments sent to Mesos dispatcher

[ https://issues.apache.org/jira/browse/SPARK-27015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-27015. Resolution: Fixed Assignee: Martin Loncaric Fix Version/s: (was: 2.5.0

[jira] [Resolved] (SPARK-27021) Leaking Netty event loop group for shuffle chunk fetch requests

[ https://issues.apache.org/jira/browse/SPARK-27021?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-27021. Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 23930 [https

[jira] [Assigned] (SPARK-27021) Leaking Netty event loop group for shuffle chunk fetch requests

[ https://issues.apache.org/jira/browse/SPARK-27021?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-27021: -- Assignee: Attila Zsolt Piros > Leaking Netty event loop group for shuffle ch

[jira] [Resolved] (SPARK-26928) Add driver CPU Time to the metrics system

[ https://issues.apache.org/jira/browse/SPARK-26928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-26928. Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 23838 [https

[jira] [Assigned] (SPARK-26928) Add driver CPU Time to the metrics system

[ https://issues.apache.org/jira/browse/SPARK-26928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-26928: -- Assignee: Luca Canali > Add driver CPU Time to the metrics sys

[jira] [Resolved] (SPARK-27012) Storage tab shows rdd details even after executor ended

[ https://issues.apache.org/jira/browse/SPARK-27012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-27012. Resolution: Fixed Assignee: Ajith S Fix Version/s: 3.0.0 > Storage

[jira] [Resolved] (SPARK-27059) spark-submit on kubernetes cluster does not recognise k8s --master property

[ https://issues.apache.org/jira/browse/SPARK-27059?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-27059. Resolution: Invalid > spark-submit on kubernetes cluster does not recognise k8s --mas

[jira] [Commented] (SPARK-27059) spark-submit on kubernetes cluster does not recognise k8s --master property

[ https://issues.apache.org/jira/browse/SPARK-27059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16784762#comment-16784762 ] Marcelo Vanzin commented on SPARK-27059: Sounds like a problem with your sy

[jira] [Commented] (SPARK-27059) spark-submit on kubernetes cluster does not recognise k8s --master property

[ https://issues.apache.org/jira/browse/SPARK-27059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16784674#comment-16784674 ] Marcelo Vanzin commented on SPARK-27059: You're most probably using

[jira] [Assigned] (SPARK-24120) Show `Jobs` page when `jobId` is missing

[ https://issues.apache.org/jira/browse/SPARK-24120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-24120: -- Assignee: Marcelo Vanzin > Show `Jobs` page when `jobId` is miss

[jira] [Assigned] (SPARK-24120) Show `Jobs` page when `jobId` is missing

[ https://issues.apache.org/jira/browse/SPARK-24120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-24120: -- Assignee: (was: Marcelo Vanzin) > Show `Jobs` page when `jobId` is miss

[jira] [Assigned] (SPARK-25564) Add output bytes metrics for each Executor

[ https://issues.apache.org/jira/browse/SPARK-25564?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-25564: -- Assignee: Marcelo Vanzin > Add output bytes metrics for each Execu

[jira] [Assigned] (SPARK-25564) Add output bytes metrics for each Executor

[ https://issues.apache.org/jira/browse/SPARK-25564?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-25564: -- Assignee: Marcelo Vanzin > Add output bytes metrics for each Execu

[jira] [Assigned] (SPARK-25564) Add output bytes metrics for each Executor

[ https://issues.apache.org/jira/browse/SPARK-25564?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-25564: -- Assignee: (was: Marcelo Vanzin) > Add output bytes metrics for each Execu

[jira] [Assigned] (SPARK-25564) Add output bytes metrics for each Executor

[ https://issues.apache.org/jira/browse/SPARK-25564?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-25564: -- Assignee: (was: Marcelo Vanzin) > Add output bytes metrics for each Execu

[jira] [Assigned] (SPARK-26792) Apply custom log URL to Spark UI

[ https://issues.apache.org/jira/browse/SPARK-26792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-26792: -- Assignee: Jungtaek Lim > Apply custom log URL to Spark

[jira] [Resolved] (SPARK-26792) Apply custom log URL to Spark UI

[ https://issues.apache.org/jira/browse/SPARK-26792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-26792. Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 23790 [https

[jira] [Assigned] (SPARK-25681) Delegation Tokens fetched twice upon HadoopFSDelegationTokenProvider creation

[ https://issues.apache.org/jira/browse/SPARK-25681?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-25681: -- Assignee: (was: Marcelo Vanzin) > Delegation Tokens fetched twice u

[jira] [Resolved] (SPARK-26995) Running Spark in Docker image with Alpine Linux 3.9.0 throws errors when using snappy

[ https://issues.apache.org/jira/browse/SPARK-26995?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-26995. Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 23898 [https

[jira] [Assigned] (SPARK-25681) Delegation Tokens fetched twice upon HadoopFSDelegationTokenProvider creation

[ https://issues.apache.org/jira/browse/SPARK-25681?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-25681: -- Assignee: Marcelo Vanzin > Delegation Tokens fetched twice u

[jira] [Assigned] (SPARK-25750) Integration Testing for Kerberos Support for Spark on Kubernetes

[ https://issues.apache.org/jira/browse/SPARK-25750?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-25750: -- Assignee: Marcelo Vanzin > Integration Testing for Kerberos Support for Spark

[jira] [Assigned] (SPARK-25750) Integration Testing for Kerberos Support for Spark on Kubernetes

[ https://issues.apache.org/jira/browse/SPARK-25750?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-25750: -- Assignee: (was: Marcelo Vanzin) > Integration Testing for Kerberos Support

[jira] [Assigned] (SPARK-24135) [K8s] Executors that fail to start up because of init-container errors are not retried and limit the executor pool size

[ https://issues.apache.org/jira/browse/SPARK-24135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-24135: -- Assignee: Marcelo Vanzin > [K8s] Executors that fail to start up because of i

[jira] [Assigned] (SPARK-24135) [K8s] Executors that fail to start up because of init-container errors are not retried and limit the executor pool size

[ https://issues.apache.org/jira/browse/SPARK-24135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-24135: -- Assignee: (was: Marcelo Vanzin) > [K8s] Executors that fail to start up beca

[jira] [Assigned] (SPARK-26995) Running Spark in Docker image with Alpine Linux 3.9.0 throws errors when using snappy

[ https://issues.apache.org/jira/browse/SPARK-26995?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-26995: -- Assignee: Luca Canali > Running Spark in Docker image with Alpine Linux 3.9.0 thr

Re: [VOTE] Redirect github notifications to issues@

FYI, it should all be done now, so if you want to get notifications, either sign up to issues@, or watch the repo on github. On Fri, Feb 22, 2019 at 9:49 AM Marcelo Vanzin wrote: > > Jira link: https://issues.apache.org/jira/browse/INFRA-17892 > > On Fri, Feb 22, 2019 at 9:44 AM Ma

[jira] [Resolved] (SPARK-26997) k8s integration tests failing after client upgraded to 4.1.2

[ https://issues.apache.org/jira/browse/SPARK-26997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-26997. Resolution: Fixed Target Version/s: 3.0.0 I reverted the client upgrade and re

[jira] [Resolved] (SPARK-26774) Document threading concerns in TaskSchedulerImpl

[ https://issues.apache.org/jira/browse/SPARK-26774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-26774. Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 23874 [https

[jira] [Assigned] (SPARK-26774) Document threading concerns in TaskSchedulerImpl

[ https://issues.apache.org/jira/browse/SPARK-26774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-26774: -- Assignee: Imran Rashid > Document threading concerns in TaskSchedulerI

[jira] [Assigned] (SPARK-27002) Latest kafka delegation token not always picked up

[ https://issues.apache.org/jira/browse/SPARK-27002?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-27002: -- Assignee: Gabor Somogyi > Latest kafka delegation token not always picked

[jira] [Resolved] (SPARK-27002) Latest kafka delegation token not always picked up

[ https://issues.apache.org/jira/browse/SPARK-27002?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-27002. Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 23906 [https

[jira] [Assigned] (SPARK-24063) Control maximum epoch backlog

[ https://issues.apache.org/jira/browse/SPARK-24063?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-24063: -- Assignee: Gabor Somogyi > Control maximum epoch back

[jira] [Resolved] (SPARK-24063) Control maximum epoch backlog

[ https://issues.apache.org/jira/browse/SPARK-24063?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-24063. Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 23156 [https

[jira] [Assigned] (SPARK-24736) --py-files not functional for non local URLs. It appears to pass non-local URL's into PYTHONPATH directly.

[ https://issues.apache.org/jira/browse/SPARK-24736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-24736: -- Assignee: Marcelo Vanzin > --py-files not functional for non local URLs. It appe

[jira] [Resolved] (SPARK-24736) --py-files not functional for non local URLs. It appears to pass non-local URL's into PYTHONPATH directly.

[ https://issues.apache.org/jira/browse/SPARK-24736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-24736. Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 23793 [https

[jira] [Resolved] (SPARK-22860) Spark workers log ssl passwords passed to the executors

[ https://issues.apache.org/jira/browse/SPARK-22860?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-22860. Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 23820 [https

[jira] [Assigned] (SPARK-22860) Spark workers log ssl passwords passed to the executors

[ https://issues.apache.org/jira/browse/SPARK-22860?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-22860: -- Assignee: Jungtaek Lim > Spark workers log ssl passwords passed to the execut

[jira] [Reopened] (SPARK-25590) kubernetes-model-2.0.0.jar masks default Spark logging config

[ https://issues.apache.org/jira/browse/SPARK-25590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reopened SPARK-25590: Assignee: (was: Jiaxin Shan) Reopened since the patch was reverted. > kuberne

[jira] [Reopened] (SPARK-26742) Bump Kubernetes Client Version to 4.1.1

[ https://issues.apache.org/jira/browse/SPARK-26742?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reopened SPARK-26742: Assignee: (was: Jiaxin Shan) Reopened since we had to revert the patch. > B

[jira] [Commented] (SPARK-26997) k8s integration tests failing after client upgraded to 4.1.2

[ https://issues.apache.org/jira/browse/SPARK-26997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16778376#comment-16778376 ] Marcelo Vanzin commented on SPARK-26997: It's linked in the descripti

[jira] [Commented] (SPARK-26997) k8s integration tests failing after client upgraded to 4.1.2

[ https://issues.apache.org/jira/browse/SPARK-26997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16778240#comment-16778240 ] Marcelo Vanzin commented on SPARK-26997: I'm using virtualbo

[jira] [Created] (SPARK-26997) k8s integration tests failing after client upgraded to 4.1.2

Marcelo Vanzin created SPARK-26997: -- Summary: k8s integration tests failing after client upgraded to 4.1.2 Key: SPARK-26997 URL: https://issues.apache.org/jira/browse/SPARK-26997 Project: Spark

[jira] [Resolved] (SPARK-26674) Consolidate CompositeByteBuf when reading large frame

[ https://issues.apache.org/jira/browse/SPARK-26674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-26674. Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 23602 [https

[jira] [Assigned] (SPARK-26674) Consolidate CompositeByteBuf when reading large frame

[ https://issues.apache.org/jira/browse/SPARK-26674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-26674: -- Assignee: liupengcheng > Consolidate CompositeByteBuf when reading large fr

[jira] [Created] (SPARK-26989) Flaky test:DAGSchedulerSuite.Barrier task failures from the same stage attempt don't trigger multiple stage retries

Marcelo Vanzin created SPARK-26989: -- Summary: Flaky test:DAGSchedulerSuite.Barrier task failures from the same stage attempt don't trigger multiple stage retries Key: SPARK-26989 URL: https://issues.apach

[jira] [Resolved] (SPARK-25035) Replicating disk-stored blocks should avoid memory mapping

[ https://issues.apache.org/jira/browse/SPARK-25035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-25035. Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 23688 [https

[jira] [Assigned] (SPARK-25035) Replicating disk-stored blocks should avoid memory mapping

[ https://issues.apache.org/jira/browse/SPARK-25035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-25035: -- Assignee: Attila Zsolt Piros > Replicating disk-stored blocks should avoid mem

[jira] [Assigned] (SPARK-26895) When running spark 2.3 as a proxy user (--proxy-user), SparkSubmit fails to resolve globs owned by target user

[ https://issues.apache.org/jira/browse/SPARK-26895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-26895: -- Assignee: Alessandro Bellina > When running spark 2.3 as a proxy user (--proxy-u

[jira] [Resolved] (SPARK-26895) When running spark 2.3 as a proxy user (--proxy-user), SparkSubmit fails to resolve globs owned by target user

[ https://issues.apache.org/jira/browse/SPARK-26895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-26895. Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 23806 [https

Re: [VOTE] Redirect github notifications to issues@

Jira link: https://issues.apache.org/jira/browse/INFRA-17892 On Fri, Feb 22, 2019 at 9:44 AM Marcelo Vanzin wrote: > > Thanks all, vote passes with 11 +1s, no -1s. > > I'll file the INFRA ticket and link it back here soon. > > On Tue, Feb 19, 2019 at 1:35 PM Marcelo Van

Re: [VOTE] Redirect github notifications to issues@

Thanks all, vote passes with 11 +1s, no -1s. I'll file the INFRA ticket and link it back here soon. On Tue, Feb 19, 2019 at 1:35 PM Marcelo Vanzin wrote: > > I'm opening a vote based on recent discussions about the extra noise > generated by github updates going to d

[jira] [Resolved] (SPARK-26954) Do not attemp when user code throws exception

[ https://issues.apache.org/jira/browse/SPARK-26954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-26954. Resolution: Not A Problem See discussion in the PR for why this is not a problem (or

Re: [VOTE] Release Apache Spark 2.4.1 (RC2)

Just wanted to point out that https://issues.apache.org/jira/browse/SPARK-26859 is not in this RC, and is marked as a correctness bug. (The fix is in the 2.4 branch, just not in rc2.) On Wed, Feb 20, 2019 at 12:07 PM DB Tsai wrote: > > Please vote on releasing the following candidate as Apache Sp

[jira] [Resolved] (SPARK-26877) Support user-level app staging directory in yarn mode when spark.yarn.stagingDir specified

[ https://issues.apache.org/jira/browse/SPARK-26877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-26877. Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 23786 [https

[jira] [Assigned] (SPARK-26877) Support user-level app staging directory in yarn mode when spark.yarn.stagingDir specified

[ https://issues.apache.org/jira/browse/SPARK-26877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-26877: -- Assignee: liupengcheng > Support user-level app staging directory in yarn mode w

Re: Github notification and Jira Issues (was: Re: [VOTE] Redirect github notifications to issues@)

On Wed, Feb 20, 2019 at 11:13 AM Pascal Schumacher wrote: > Am 20.02.2019 um 19:39 schrieb Marcelo Vanzin: > > Rob: > >> I almost think that we should have Pull Requests generate jiras. > > I've seen this set up in a couple of projects and jira becomes > > unr

Re: [VOTE] Redirect github notifications to issues@

On Wed, Feb 20, 2019 at 5:41 AM Gary Gregory wrote: > Is this a LAZY VOTE? Sorry, but not familiar with the semantics of when to call a lazy vs. non-lazy vote. Given the current number of votes, does it matter? Rob: > I almost think that we should have Pull Requests generate jiras. I've seen th

[jira] [Resolved] (CRYPTO-138) failed to run on openssl 1.1.0g

[ https://issues.apache.org/jira/browse/CRYPTO-138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved CRYPTO-138. --- Resolution: Fixed Fix Version/s: 1.1.0 Should be fixed now: https://github.com/apache

[jira] [Resolved] (SPARK-26933) spark-submit does not make zip files provided with --py-files visible to pyspark

[ https://issues.apache.org/jira/browse/SPARK-26933?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-26933. Resolution: Not A Problem Did something wrong when testing this in the morning. This is

[jira] [Resolved] (SPARK-24894) Invalid DNS name due to hostname truncation

[ https://issues.apache.org/jira/browse/SPARK-24894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-24894. Resolution: Fixed Assignee: Marcelo Vanzin Fix Version/s: 3.0.0 > Inva

[jira] [Resolved] (SPARK-26882) lint-scala script does not check all components

[ https://issues.apache.org/jira/browse/SPARK-26882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-26882. Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 23792 [https

[jira] [Assigned] (SPARK-26891) Flaky test:YarnSchedulerBackendSuite."RequestExecutors reflects node blacklist and is serializable"

[ https://issues.apache.org/jira/browse/SPARK-26891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-26891: -- Assignee: Attila Zsolt Piros > Flaky test:YarnSchedulerBackendSu

[jira] [Resolved] (SPARK-26891) Flaky test:YarnSchedulerBackendSuite."RequestExecutors reflects node blacklist and is serializable"

[ https://issues.apache.org/jira/browse/SPARK-26891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-26891. Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 23801 [https

[VOTE] Redirect github notifications to issues@

I'm opening a vote based on recent discussions about the extra noise generated by github updates going to dev@. So please vote: - +1 to redirect github updates of all commons repos to the issues@ list - -1 to keep things as is If the vote passes, I'll take care of opening an infra ticket referenc

[jira] [Created] (SPARK-26934) python dependencies with "local:" URIs are not visible to executors

Marcelo Vanzin created SPARK-26934: -- Summary: python dependencies with "local:" URIs are not visible to executors Key: SPARK-26934 URL: https://issues.apache.org/jira/browse/SPARK-26934

[jira] [Created] (SPARK-26933) spark-submit does not make zip files provided with --py-files visible to pyspark

Marcelo Vanzin created SPARK-26933: -- Summary: spark-submit does not make zip files provided with --py-files visible to pyspark Key: SPARK-26933 URL: https://issues.apache.org/jira/browse/SPARK-26933

[jira] [Commented] (SPARK-24736) --py-files not functional for non local URLs. It appears to pass non-local URL's into PYTHONPATH directly.

[ https://issues.apache.org/jira/browse/SPARK-24736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16772337#comment-16772337 ] Marcelo Vanzin commented on SPARK-24736: I'm going to fork this iss

[jira] [Updated] (SPARK-26873) FileFormatWriter creates inconsistent MR job IDs

[ https://issues.apache.org/jira/browse/SPARK-26873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin updated SPARK-26873: --- Fix Version/s: 2.3.4 > FileFormatWriter creates inconsistent MR job

Re: merge script stopped working; Python 2/3 input() issue?

BTW the main script has this that the website script does not: if sys.version < '3': input = raw_input # noqa On Fri, Feb 15, 2019 at 3:55 PM Sean Owen wrote: > > I'm seriously confused on this one. The spark-website merge script > just stopped working for me. It fails on the call to input()

<    1   2   3   4   5   6   7   8   9   10   >