[jira] [Commented] (SPARK-10915) Add support for UDAFs in Python

2017-06-27 Thread Erik Erlandson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16065787#comment-16065787 ] Erik Erlandson commented on SPARK-10915: This would be great for exposing {{TDige

[jira] [Created] (SPARK-21277) Spark is invoking an incorrect serializer after UDAF completion

2017-07-01 Thread Erik Erlandson (JIRA)
Erik Erlandson created SPARK-21277: -- Summary: Spark is invoking an incorrect serializer after UDAF completion Key: SPARK-21277 URL: https://issues.apache.org/jira/browse/SPARK-21277 Project: Spark

[jira] [Commented] (SPARK-21277) Spark is invoking an incorrect serializer after UDAF completion

2017-07-05 Thread Erik Erlandson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21277?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16075818#comment-16075818 ] Erik Erlandson commented on SPARK-21277: It would be ideal to document the requir

[jira] [Created] (SPARK-27296) User Defined Aggregating Functions (UDAFs) have a major efficiency problem

2019-03-27 Thread Erik Erlandson (JIRA)
Erik Erlandson created SPARK-27296: -- Summary: User Defined Aggregating Functions (UDAFs) have a major efficiency problem Key: SPARK-27296 URL: https://issues.apache.org/jira/browse/SPARK-27296 Projec

[jira] [Commented] (SPARK-27296) User Defined Aggregating Functions (UDAFs) have a major efficiency problem

2019-03-28 Thread Erik Erlandson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16804176#comment-16804176 ] Erik Erlandson commented on SPARK-27296: My initial proposal would be to alter t

[jira] [Created] (SPARK-18278) Support native submission of spark jobs to a kubernetes cluster

2016-11-04 Thread Erik Erlandson (JIRA)
Erik Erlandson created SPARK-18278: -- Summary: Support native submission of spark jobs to a kubernetes cluster Key: SPARK-18278 URL: https://issues.apache.org/jira/browse/SPARK-18278 Project: Spark

[jira] [Commented] (SPARK-18278) Support native submission of spark jobs to a kubernetes cluster

2016-11-04 Thread Erik Erlandson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15637591#comment-15637591 ] Erik Erlandson commented on SPARK-18278: Current prototype: https://github.com/fo

[jira] [Updated] (SPARK-18278) Support native submission of spark jobs to a kubernetes cluster

2016-11-04 Thread Erik Erlandson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Erlandson updated SPARK-18278: --- External issue URL: https://github.com/kubernetes/kubernetes/issues/34377 External issue

[jira] [Commented] (SPARK-18278) Support native submission of spark jobs to a kubernetes cluster

2016-11-07 Thread Erik Erlandson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15644841#comment-15644841 ] Erik Erlandson commented on SPARK-18278: I agree with [~willbenton] that Kube is

[jira] [Commented] (SPARK-18278) Support native submission of spark jobs to a kubernetes cluster

2016-11-07 Thread Erik Erlandson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15644919#comment-15644919 ] Erik Erlandson commented on SPARK-18278: Another comment on external plug-ins for

[jira] [Commented] (SPARK-24432) Add support for dynamic resource allocation

2021-03-18 Thread Erik Erlandson (Jira)
[ https://issues.apache.org/jira/browse/SPARK-24432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17304397#comment-17304397 ] Erik Erlandson commented on SPARK-24432: [~dongjoon] should this be closed, now

[jira] [Commented] (SPARK-24793) Make spark-submit more useful with k8s

2018-07-12 Thread Erik Erlandson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16541923#comment-16541923 ] Erik Erlandson commented on SPARK-24793: I am concerned that this is outside the

[jira] [Commented] (SPARK-24793) Make spark-submit more useful with k8s

2018-07-12 Thread Erik Erlandson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16541930#comment-16541930 ] Erik Erlandson commented on SPARK-24793: Another possible angle (not mutually ex

[jira] [Commented] (SPARK-24793) Make spark-submit more useful with k8s

2018-07-12 Thread Erik Erlandson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16542102#comment-16542102 ] Erik Erlandson commented on SPARK-24793: Also a good point that {{--kill}} and {

[jira] [Comment Edited] (SPARK-24793) Make spark-submit more useful with k8s

2018-07-12 Thread Erik Erlandson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16542102#comment-16542102 ] Erik Erlandson edited comment on SPARK-24793 at 7/12/18 7:01 PM: -

[jira] [Commented] (SPARK-24615) Accelerator-aware task scheduling for Spark

2018-07-31 Thread Erik Erlandson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16564558#comment-16564558 ] Erik Erlandson commented on SPARK-24615: Am I understanding correctly that this

[jira] [Commented] (SPARK-24580) List scenarios to be handled by barrier execution mode properly

2018-08-01 Thread Erik Erlandson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16566154#comment-16566154 ] Erik Erlandson commented on SPARK-24580: This is blocking SPARK-24582 which is m

[jira] [Commented] (SPARK-24817) Implement BarrierTaskContext.barrier()

2018-08-01 Thread Erik Erlandson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16566159#comment-16566159 ] Erik Erlandson commented on SPARK-24817: I'm curious about what the {{barrier}}

[jira] [Commented] (SPARK-24817) Implement BarrierTaskContext.barrier()

2018-08-02 Thread Erik Erlandson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16567301#comment-16567301 ] Erik Erlandson commented on SPARK-24817: Thanks [~jiangxb] - I'd expect that des

[jira] [Commented] (SPARK-24817) Implement BarrierTaskContext.barrier()

2018-08-02 Thread Erik Erlandson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16567358#comment-16567358 ] Erik Erlandson commented on SPARK-24817: I have been looking at the use cases fo

[jira] [Created] (SPARK-25128) multiple simultaneous job submissions against k8s backend cause driver pods to hang

2018-08-15 Thread Erik Erlandson (JIRA)
Erik Erlandson created SPARK-25128: -- Summary: multiple simultaneous job submissions against k8s backend cause driver pods to hang Key: SPARK-25128 URL: https://issues.apache.org/jira/browse/SPARK-25128

[jira] [Commented] (SPARK-25128) multiple simultaneous job submissions against k8s backend cause driver pods to hang

2018-08-15 Thread Erik Erlandson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16581537#comment-16581537 ] Erik Erlandson commented on SPARK-25128: [~mcheah], [~liyinan926], wdyt? > mult

[jira] [Commented] (SPARK-7768) Make user-defined type (UDT) API public

2018-08-22 Thread Erik Erlandson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16588934#comment-16588934 ] Erik Erlandson commented on SPARK-7768: --- We use `UserDefinedType`, for example here

[jira] [Commented] (SPARK-21097) Dynamic allocation will preserve cached data

2018-08-27 Thread Erik Erlandson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16594271#comment-16594271 ] Erik Erlandson commented on SPARK-21097: I'm wondering if this is going to be su

[jira] [Created] (SPARK-25275) require memberhip in wheel to run 'su' (in dockerfiles)

2018-08-29 Thread Erik Erlandson (JIRA)
Erik Erlandson created SPARK-25275: -- Summary: require memberhip in wheel to run 'su' (in dockerfiles) Key: SPARK-25275 URL: https://issues.apache.org/jira/browse/SPARK-25275 Project: Spark I

[jira] [Commented] (SPARK-25275) require memberhip in wheel to run 'su' (in dockerfiles)

2018-08-30 Thread Erik Erlandson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16597935#comment-16597935 ] Erik Erlandson commented on SPARK-25275: {{merge_spark_pr.py}} failed to close t

[jira] [Resolved] (SPARK-25275) require memberhip in wheel to run 'su' (in dockerfiles)

2018-08-30 Thread Erik Erlandson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25275?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Erlandson resolved SPARK-25275. Resolution: Fixed Fix Version/s: 2.4.0 > require memberhip in wheel to run 'su' (in

[jira] [Created] (SPARK-25287) Check for JIRA_USERNAME and JIRA_PASSWORD up front in merge_spark_pr.py

2018-08-30 Thread Erik Erlandson (JIRA)
Erik Erlandson created SPARK-25287: -- Summary: Check for JIRA_USERNAME and JIRA_PASSWORD up front in merge_spark_pr.py Key: SPARK-25287 URL: https://issues.apache.org/jira/browse/SPARK-25287 Project:

[jira] [Assigned] (SPARK-25287) Check for JIRA_USERNAME and JIRA_PASSWORD up front in merge_spark_pr.py

2018-08-30 Thread Erik Erlandson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Erlandson reassigned SPARK-25287: -- Assignee: Erik Erlandson > Check for JIRA_USERNAME and JIRA_PASSWORD up front in merg

[jira] [Resolved] (SPARK-25287) Check for JIRA_USERNAME and JIRA_PASSWORD up front in merge_spark_pr.py

2018-08-30 Thread Erik Erlandson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Erlandson resolved SPARK-25287. Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 22294 [https:

[jira] [Commented] (SPARK-24434) Support user-specified driver and executor pod templates

2018-08-31 Thread Erik Erlandson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16599092#comment-16599092 ] Erik Erlandson commented on SPARK-24434: There are a few related, but separate,

[jira] [Commented] (SPARK-24434) Support user-specified driver and executor pod templates

2018-08-31 Thread Erik Erlandson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16599283#comment-16599283 ] Erik Erlandson commented on SPARK-24434: Stavros, yes, I knew you were working o

[jira] [Commented] (SPARK-24434) Support user-specified driver and executor pod templates

2018-08-31 Thread Erik Erlandson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16599299#comment-16599299 ] Erik Erlandson commented on SPARK-24434: To amplify a little from my points abov

[jira] [Commented] (SPARK-24135) [K8s] Executors that fail to start up because of init-container errors are not retried and limit the executor pool size

2018-05-01 Thread Erik Erlandson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16459954#comment-16459954 ] Erik Erlandson commented on SPARK-24135: I think it makes sense to detect these f

[jira] [Commented] (SPARK-24135) [K8s] Executors that fail to start up because of init-container errors are not retried and limit the executor pool size

2018-05-02 Thread Erik Erlandson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16461154#comment-16461154 ] Erik Erlandson commented on SPARK-24135: IIRC the dynamic allocation heuristic wa

[jira] [Commented] (SPARK-24135) [K8s] Executors that fail to start up because of init-container errors are not retried and limit the executor pool size

2018-05-02 Thread Erik Erlandson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16461324#comment-16461324 ] Erik Erlandson commented on SPARK-24135: > In the case of the executor failing to

[jira] [Commented] (SPARK-24248) [K8S] Use the Kubernetes cluster as the backing store for the state of pods

2018-05-16 Thread Erik Erlandson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24248?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16477812#comment-16477812 ] Erik Erlandson commented on SPARK-24248: Is the design above using re-sync as the

[jira] [Created] (SPARK-24435) Support user-supplied YAML that can be merged with k8s pod descriptions

2018-05-30 Thread Erik Erlandson (JIRA)
Erik Erlandson created SPARK-24435: -- Summary: Support user-supplied YAML that can be merged with k8s pod descriptions Key: SPARK-24435 URL: https://issues.apache.org/jira/browse/SPARK-24435 Project:

[jira] [Resolved] (SPARK-24435) Support user-supplied YAML that can be merged with k8s pod descriptions

2018-05-30 Thread Erik Erlandson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24435?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Erlandson resolved SPARK-24435. Resolution: Duplicate > Support user-supplied YAML that can be merged with k8s pod descrip

[jira] [Commented] (SPARK-24434) Support user-specified driver and executor pod templates

2018-05-30 Thread Erik Erlandson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16495615#comment-16495615 ] Erik Erlandson commented on SPARK-24434: Is the template-based solution being ex

[jira] [Commented] (SPARK-24091) Internally used ConfigMap prevents use of user-specified ConfigMaps carrying Spark configs files

2018-05-30 Thread Erik Erlandson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16495619#comment-16495619 ] Erik Erlandson commented on SPARK-24091: If we support user-supplied yaml, that

[jira] [Commented] (SPARK-24434) Support user-specified driver and executor pod templates

2018-05-30 Thread Erik Erlandson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16495642#comment-16495642 ] Erik Erlandson commented on SPARK-24434: [~skonto] given the number of ideas tha

[jira] [Commented] (SPARK-24434) Support user-specified driver and executor pod templates

2018-05-31 Thread Erik Erlandson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16496991#comment-16496991 ] Erik Erlandson commented on SPARK-24434: [~foxish] is there a technical (or ux)

[jira] [Commented] (SPARK-24434) Support user-specified driver and executor pod templates

2018-06-01 Thread Erik Erlandson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16498337#comment-16498337 ] Erik Erlandson commented on SPARK-24434: My current take on UX around this featu

[jira] [Comment Edited] (SPARK-24434) Support user-specified driver and executor pod templates

2018-06-01 Thread Erik Erlandson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16498337#comment-16498337 ] Erik Erlandson edited comment on SPARK-24434 at 6/1/18 5:53 PM: --

[jira] [Commented] (SPARK-24534) Add a way to bypass entrypoint.sh script if no spark cmd is passed

2018-06-13 Thread Erik Erlandson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16511804#comment-16511804 ] Erik Erlandson commented on SPARK-24534: I think this has potential use for cust

[jira] [Resolved] (SPARK-24534) Add a way to bypass entrypoint.sh script if no spark cmd is passed

2018-06-19 Thread Erik Erlandson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Erlandson resolved SPARK-24534. Resolution: Fixed Fix Version/s: 2.4.0 > Add a way to bypass entrypoint.sh script i

[jira] [Commented] (SPARK-26973) Kubernetes version support strategy on test nodes / backend

2019-02-22 Thread Erik Erlandson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16775435#comment-16775435 ] Erik Erlandson commented on SPARK-26973: A couple other points: * Currently, k8

[jira] [Commented] (SPARK-25128) multiple simultaneous job submissions against k8s backend cause driver pods to hang

2018-09-06 Thread Erik Erlandson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16605916#comment-16605916 ] Erik Erlandson commented on SPARK-25128: Retargeting to next release sounds good

[jira] [Updated] (SPARK-25128) multiple simultaneous job submissions against k8s backend cause driver pods to hang

2018-09-06 Thread Erik Erlandson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Erlandson updated SPARK-25128: --- Target Version/s: 3.0.0 (was: 2.4.0, 2.3.3) Priority: Minor (was: Major) > mul

[jira] [Updated] (SPARK-25782) Add PCA Aggregator to support grouping

2018-10-19 Thread Erik Erlandson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Erlandson updated SPARK-25782: --- Target Version/s: 3.0.0 Component/s: ML Issue Type: New Feature (was: Imp

[jira] [Commented] (SPARK-25782) Add PCA Aggregator to support grouping

2018-10-19 Thread Erik Erlandson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16657394#comment-16657394 ] Erik Erlandson commented on SPARK-25782: Thanks [~mttsndrs]! I agree it makes s

[jira] [Commented] (SPARK-25782) Add PCA Aggregator to support grouping

2018-10-19 Thread Erik Erlandson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16657398#comment-16657398 ] Erik Erlandson commented on SPARK-25782: An ML Estimator also arguably would be

[jira] [Commented] (SPARK-25828) Bumping Version of kubernetes.client to latest version

2018-10-24 Thread Erik Erlandson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16662978#comment-16662978 ] Erik Erlandson commented on SPARK-25828: cc [~skonto] > Bumping Version of kube

[jira] [Resolved] (SPARK-25828) Bumping Version of kubernetes.client to latest version

2018-10-26 Thread Erik Erlandson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Erlandson resolved SPARK-25828. Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 22820 [https:

[jira] [Assigned] (SPARK-25828) Bumping Version of kubernetes.client to latest version

2018-10-26 Thread Erik Erlandson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Erlandson reassigned SPARK-25828: -- Assignee: Ilan Filonenko > Bumping Version of kubernetes.client to latest version > -

[jira] [Commented] (SPARK-23680) entrypoint.sh does not accept arbitrary UIDs, returning as an error

2018-03-14 Thread Erik Erlandson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16398701#comment-16398701 ] Erik Erlandson commented on SPARK-23680: [~rmartine] thanks for catching this! It

[jira] [Updated] (SPARK-23680) entrypoint.sh does not accept arbitrary UIDs, returning as an error

2018-03-14 Thread Erik Erlandson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Erlandson updated SPARK-23680: --- Flags: Important Labels: easyfix (was: ) > entrypoint.sh does not accept arbitrary UIDs

[jira] [Resolved] (SPARK-23680) entrypoint.sh does not accept arbitrary UIDs, returning as an error

2018-03-16 Thread Erik Erlandson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Erlandson resolved SPARK-23680. Resolution: Fixed Target Version/s: 2.3.1, 2.4.0 merged to master > entrypoint.s

[jira] [Commented] (SPARK-23680) entrypoint.sh does not accept arbitrary UIDs, returning as an error

2018-03-16 Thread Erik Erlandson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16402266#comment-16402266 ] Erik Erlandson commented on SPARK-23680: commit workflow indicates to set the Ass

[jira] [Updated] (SPARK-23891) Debian based Dockerfile

2018-04-12 Thread Erik Erlandson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Erlandson updated SPARK-23891: --- Priority: Minor (was: Major) Issue Type: New Feature (was: Bug) > Debian based Docker

[jira] [Commented] (SPARK-23891) Debian based Dockerfile

2018-04-12 Thread Erik Erlandson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16436185#comment-16436185 ] Erik Erlandson commented on SPARK-23891: The question of what OS base to use for

[jira] [Commented] (SPARK-23891) Debian based Dockerfile

2018-04-12 Thread Erik Erlandson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16436196#comment-16436196 ] Erik Erlandson commented on SPARK-23891: I do think that these reports are very u

[jira] [Commented] (SPARK-23891) Debian based Dockerfile

2018-04-14 Thread Erik Erlandson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16438410#comment-16438410 ] Erik Erlandson commented on SPARK-23891: [~SercanKaraoglu] thanks for the informa

[jira] [Commented] (SPARK-22647) Docker files for image creation

2017-12-13 Thread Erik Erlandson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16289988#comment-16289988 ] Erik Erlandson commented on SPARK-22647: I'd like to propose migrating our images

[jira] [Commented] (SPARK-23137) spark.kubernetes.executor.podNamePrefix is ignored

2018-01-17 Thread Erik Erlandson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16329721#comment-16329721 ] Erik Erlandson commented on SPARK-23137: +1, a more general "app prefix" seems mo

[jira] [Created] (SPARK-23324) Announce new Kubernetes back-end for 2.3 release notes

2018-02-02 Thread Erik Erlandson (JIRA)
Erik Erlandson created SPARK-23324: -- Summary: Announce new Kubernetes back-end for 2.3 release notes Key: SPARK-23324 URL: https://issues.apache.org/jira/browse/SPARK-23324 Project: Spark Is

[jira] [Commented] (SPARK-23324) Announce new Kubernetes back-end for 2.3 release notes

2018-02-02 Thread Erik Erlandson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16351020#comment-16351020 ] Erik Erlandson commented on SPARK-23324: cc [~sameer], [~foxish] > Announce new

[jira] [Updated] (SPARK-24434) Support user-specified driver and executor pod templates

2018-11-26 Thread Erik Erlandson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24434?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Erlandson updated SPARK-24434: --- Fix Version/s: 3.0.0 > Support user-specified driver and executor pod templates > --

[jira] [Commented] (SPARK-3250) More Efficient Sampling

2014-09-11 Thread Erik Erlandson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14130809#comment-14130809 ] Erik Erlandson commented on SPARK-3250: --- I developed prototype iterator classes for

[jira] [Commented] (SPARK-3250) More Efficient Sampling

2014-09-18 Thread Erik Erlandson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14139965#comment-14139965 ] Erik Erlandson commented on SPARK-3250: --- PR: https://github.com/apache/spark/pull/24

[jira] [Closed] (SPARK-2315) drop, dropRight and dropWhile which take RDD input and return RDD

2015-06-13 Thread Erik Erlandson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Erlandson closed SPARK-2315. - Resolution: Won't Fix There is now an implementation on the silex project: http://silex.freevariab

[jira] [Commented] (SPARK-1493) Apache RAT excludes don't work with file path (instead of file name)

2014-06-09 Thread Erik Erlandson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14025895#comment-14025895 ] Erik Erlandson commented on SPARK-1493: --- RAT itself appears to preclude exclusion us

[jira] [Comment Edited] (SPARK-1493) Apache RAT excludes don't work with file path (instead of file name)

2014-06-09 Thread Erik Erlandson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14025895#comment-14025895 ] Erik Erlandson edited comment on SPARK-1493 at 6/9/14 11:13 PM:

[jira] [Commented] (SPARK-1493) Apache RAT excludes don't work with file path (instead of file name)

2014-06-20 Thread Erik Erlandson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14038921#comment-14038921 ] Erik Erlandson commented on SPARK-1493: --- I submitted a proposal patch for RAT-161, w

[jira] [Commented] (SPARK-1406) PMML model evaluation support via MLib

2014-06-20 Thread Erik Erlandson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14039256#comment-14039256 ] Erik Erlandson commented on SPARK-1406: --- How about a PMML pickler/unpickler, written

[jira] [Commented] (SPARK-1856) Standardize MLlib interfaces

2014-06-20 Thread Erik Erlandson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14039495#comment-14039495 ] Erik Erlandson commented on SPARK-1856: --- How does this jira relate (if at all) to th

[jira] [Commented] (SPARK-2262) Extreme Learning Machines (ELM) for MLLib

2014-06-24 Thread Erik Erlandson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14042731#comment-14042731 ] Erik Erlandson commented on SPARK-2262: --- I'd like to have this assigned to me > Ext

[jira] [Created] (SPARK-2262) Extreme Learning Machines (ELM) for MLLib

2014-06-24 Thread Erik Erlandson (JIRA)
Erik Erlandson created SPARK-2262: - Summary: Extreme Learning Machines (ELM) for MLLib Key: SPARK-2262 URL: https://issues.apache.org/jira/browse/SPARK-2262 Project: Spark Issue Type: New Fea

[jira] [Created] (SPARK-2315) drop, dropRight and dropWhile which take RDD input and return RDD

2014-06-27 Thread Erik Erlandson (JIRA)
Erik Erlandson created SPARK-2315: - Summary: drop, dropRight and dropWhile which take RDD input and return RDD Key: SPARK-2315 URL: https://issues.apache.org/jira/browse/SPARK-2315 Project: Spark

[jira] [Commented] (SPARK-2315) drop, dropRight and dropWhile which take RDD input and return RDD

2014-06-27 Thread Erik Erlandson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2315?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14046725#comment-14046725 ] Erik Erlandson commented on SPARK-2315: --- PR: https://github.com/apache/spark/pull/1

[jira] [Commented] (SPARK-2352) [MLLIB] Add Artificial Neural Network (ANN) to Spark

2014-07-06 Thread Erik Erlandson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14053119#comment-14053119 ] Erik Erlandson commented on SPARK-2352: --- Related: SPARK-2262 > [MLLIB] Add Artific

[jira] [Commented] (SPARK-1486) Support multi-model training in MLlib

2014-07-15 Thread Erik Erlandson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14062339#comment-14062339 ] Erik Erlandson commented on SPARK-1486: --- Does the dev on this issue effectively subs

[jira] [Commented] (SPARK-2488) Model SerDe in MLlib

2014-07-15 Thread Erik Erlandson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14062486#comment-14062486 ] Erik Erlandson commented on SPARK-2488: --- Related, possible duplicate: SPARK-1406

[jira] [Commented] (SPARK-2315) drop, dropRight and dropWhile which take RDD input and return RDD

2014-07-30 Thread Erik Erlandson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2315?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14079300#comment-14079300 ] Erik Erlandson commented on SPARK-2315: --- Updated the PR with a proper lazy-transform

[jira] [Commented] (SPARK-1021) sortByKey() launches a cluster job when it shouldn't

2014-07-30 Thread Erik Erlandson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14080250#comment-14080250 ] Erik Erlandson commented on SPARK-1021: --- I deferred the compute of the partition bou

[jira] [Created] (SPARK-2911) provide rdd.parent[T](j) to obtain jth parent of rdd

2014-08-07 Thread Erik Erlandson (JIRA)
Erik Erlandson created SPARK-2911: - Summary: provide rdd.parent[T](j) to obtain jth parent of rdd Key: SPARK-2911 URL: https://issues.apache.org/jira/browse/SPARK-2911 Project: Spark Issue Ty

[jira] [Commented] (SPARK-2911) provide rdd.parent[T](j) to obtain jth parent of rdd

2014-08-08 Thread Erik Erlandson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14090734#comment-14090734 ] Erik Erlandson commented on SPARK-2911: --- OK, shall I do it as part of this jira or f

[jira] [Created] (SPARK-2991) RDD transforms for scan and scanLeft

2014-08-12 Thread Erik Erlandson (JIRA)
Erik Erlandson created SPARK-2991: - Summary: RDD transforms for scan and scanLeft Key: SPARK-2991 URL: https://issues.apache.org/jira/browse/SPARK-2991 Project: Spark Issue Type: New Feature

[jira] [Created] (SPARK-2992) The transforms formerly known as non-lazy

2014-08-12 Thread Erik Erlandson (JIRA)
Erik Erlandson created SPARK-2992: - Summary: The transforms formerly known as non-lazy Key: SPARK-2992 URL: https://issues.apache.org/jira/browse/SPARK-2992 Project: Spark Issue Type: Umbrell

[jira] [Updated] (SPARK-2991) RDD transforms for scan and scanLeft

2014-08-12 Thread Erik Erlandson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Erlandson updated SPARK-2991: -- Issue Type: Sub-task (was: New Feature) Parent: SPARK-2992 > RDD transforms for scan a

[jira] [Updated] (SPARK-1021) sortByKey() launches a cluster job when it shouldn't

2014-08-12 Thread Erik Erlandson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1021?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Erlandson updated SPARK-1021: -- Issue Type: Sub-task (was: Bug) Parent: SPARK-2992 > sortByKey() launches a cluster jo

[jira] [Updated] (SPARK-2315) drop, dropRight and dropWhile which take RDD input and return RDD

2014-08-12 Thread Erik Erlandson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Erlandson updated SPARK-2315: -- Issue Type: Sub-task (was: New Feature) Parent: SPARK-2992 > drop, dropRight and dropW

[jira] [Created] (SPARK-3104) Jenkins failing to test some PRs when asked to

2014-08-18 Thread Erik Erlandson (JIRA)
Erik Erlandson created SPARK-3104: - Summary: Jenkins failing to test some PRs when asked to Key: SPARK-3104 URL: https://issues.apache.org/jira/browse/SPARK-3104 Project: Spark Issue Type: Bu

[jira] [Commented] (SPARK-2360) CSV import to SchemaRDDs

2014-08-22 Thread Erik Erlandson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14107233#comment-14107233 ] Erik Erlandson commented on SPARK-2360: --- It appears that this is not a pure lazy tra

[jira] [Commented] (SPARK-3250) More Efficient Sampling

2014-08-28 Thread Erik Erlandson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14114789#comment-14114789 ] Erik Erlandson commented on SPARK-3250: --- I did some experiments with sampling that m

[jira] [Commented] (SPARK-2991) RDD transforms for scan and scanLeft

2015-08-26 Thread Erik Erlandson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14715417#comment-14715417 ] Erik Erlandson commented on SPARK-2991: --- This RFE now has a [pull-req|https://github

[jira] [Commented] (SPARK-30520) Eliminate deprecation warnings for UserDefinedAggregateFunction

2020-06-15 Thread Erik Erlandson (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17136153#comment-17136153 ] Erik Erlandson commented on SPARK-30520: Starting in spark 3.0, any custom aggre

[jira] [Created] (SPARK-32159) New udaf(Aggregator) has an integration bug with UnresolvedMapObjects serialization

2020-07-02 Thread Erik Erlandson (Jira)
Erik Erlandson created SPARK-32159: -- Summary: New udaf(Aggregator) has an integration bug with UnresolvedMapObjects serialization Key: SPARK-32159 URL: https://issues.apache.org/jira/browse/SPARK-32159

[jira] [Updated] (SPARK-32159) New udaf(Aggregator) has an integration bug with UnresolvedMapObjects serialization

2020-07-02 Thread Erik Erlandson (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Erlandson updated SPARK-32159: --- Description: The new user defined aggregator feature (SPARK-27296) based on calling 'functi

  1   2   >