[jira] [Updated] (SPARK-1359) SGD implementation is not efficient

2018-03-26 Thread Michael Gummelt (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1359?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Gummelt updated SPARK-1359: --- Remaining Estimate: (was: 168h) Original Estimate: (was: 168h) > SGD

[jira] [Updated] (SPARK-1359) SGD implementation is not efficient

2018-03-26 Thread Michael Gummelt (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1359?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Gummelt updated SPARK-1359: --- Remaining Estimate: 168h Original Estimate: 168h > SGD implementation is not efficient >

[jira] [Updated] (SPARK-1359) SGD implementation is not efficient

2018-03-26 Thread Michael Gummelt (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1359?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Gummelt updated SPARK-1359: --- Remaining Estimate: (was: 504h) Original Estimate: (was: 504h) > SGD

[jira] [Updated] (SPARK-1359) SGD implementation is not efficient

2018-03-26 Thread Michael Gummelt (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1359?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Gummelt updated SPARK-1359: --- Remaining Estimate: 504h Original Estimate: 504h > SGD implementation is not efficient >

[jira] [Commented] (SPARK-20812) Add Mesos Secrets support to the spark dispatcher

2017-06-14 Thread Michael Gummelt (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16049380#comment-16049380 ] Michael Gummelt commented on SPARK-20812: - The dispatcher won't know about the secrets. It will

[jira] [Updated] (SPARK-20812) Add Mesos Secrets support to the spark dispatcher

2017-06-13 Thread Michael Gummelt (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Gummelt updated SPARK-20812: Description: Mesos 1.4 will support secrets. In order to support sending keytabs through

[jira] [Updated] (SPARK-20434) Move Hadoop delegation token code from yarn to core

2017-06-12 Thread Michael Gummelt (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20434?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Gummelt updated SPARK-20434: Summary: Move Hadoop delegation token code from yarn to core (was: Move Kerberos

[jira] [Created] (SPARK-21000) Add labels support to the Spark Dispatcher

2017-06-06 Thread Michael Gummelt (JIRA)
Michael Gummelt created SPARK-21000: --- Summary: Add labels support to the Spark Dispatcher Key: SPARK-21000 URL: https://issues.apache.org/jira/browse/SPARK-21000 Project: Spark Issue Type:

[jira] [Commented] (SPARK-4899) Support Mesos features: roles and checkpoints

2017-05-24 Thread Michael Gummelt (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16023383#comment-16023383 ] Michael Gummelt commented on SPARK-4899: Thanks Kamal. I responded to the thread, which I'll copy

[jira] [Commented] (SPARK-4899) Support Mesos features: roles and checkpoints

2017-05-23 Thread Michael Gummelt (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16021766#comment-16021766 ] Michael Gummelt commented on SPARK-4899: [~drcrallen] Can you link me to the conversation you had

[jira] [Commented] (SPARK-4899) Support Mesos features: roles and checkpoints

2017-05-23 Thread Michael Gummelt (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16021750#comment-16021750 ] Michael Gummelt commented on SPARK-4899: These are two separate features, which need two separate

[jira] [Updated] (SPARK-20812) Add Mesos Secrets support to the spark dispatcher

2017-05-19 Thread Michael Gummelt (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Gummelt updated SPARK-20812: Description: Mesos 1.3 supports secrets. In order to support sending keytabs through the

[jira] [Created] (SPARK-20812) Add Mesos Secrets support to the spark dispatcher

2017-05-19 Thread Michael Gummelt (JIRA)
Michael Gummelt created SPARK-20812: --- Summary: Add Mesos Secrets support to the spark dispatcher Key: SPARK-20812 URL: https://issues.apache.org/jira/browse/SPARK-20812 Project: Spark

[jira] [Commented] (SPARK-16627) --jars doesn't work in Mesos mode

2017-05-18 Thread Michael Gummelt (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16016253#comment-16016253 ] Michael Gummelt commented on SPARK-16627: - I'm not completely sure, but I believe that the

[jira] [Commented] (SPARK-12559) Cluster mode doesn't work with --packages

2017-05-18 Thread Michael Gummelt (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16016243#comment-16016243 ] Michael Gummelt commented on SPARK-12559: - I changed the title from "Standalone cluster mode" to

[jira] [Updated] (SPARK-12559) Cluster mode doesn't work with --packages

2017-05-18 Thread Michael Gummelt (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Gummelt updated SPARK-12559: Summary: Cluster mode doesn't work with --packages (was: Standalone cluster mode doesn't

[jira] [Commented] (SPARK-20447) spark mesos scheduler suppress call

2017-04-25 Thread Michael Gummelt (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15983314#comment-15983314 ] Michael Gummelt commented on SPARK-20447: - The scheduler doesn't support suppression, no, but it

[jira] [Created] (SPARK-20434) Move Kerberos delegation token code from yarn to core

2017-04-21 Thread Michael Gummelt (JIRA)
Michael Gummelt created SPARK-20434: --- Summary: Move Kerberos delegation token code from yarn to core Key: SPARK-20434 URL: https://issues.apache.org/jira/browse/SPARK-20434 Project: Spark

[jira] [Commented] (SPARK-20328) HadoopRDDs create a MapReduce JobConf, but are not MapReduce jobs

2017-04-14 Thread Michael Gummelt (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15969379#comment-15969379 ] Michael Gummelt commented on SPARK-20328: - Ah, yes, of course. Thanks. > HadoopRDDs create a

[jira] [Commented] (SPARK-16742) Kerberos support for Spark on Mesos

2017-04-14 Thread Michael Gummelt (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15969341#comment-15969341 ] Michael Gummelt commented on SPARK-16742: - [~jerryshao] No, but you can look at our solution

[jira] [Comment Edited] (SPARK-16742) Kerberos support for Spark on Mesos

2017-04-14 Thread Michael Gummelt (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15969341#comment-15969341 ] Michael Gummelt edited comment on SPARK-16742 at 4/14/17 6:01 PM: --

[jira] [Commented] (SPARK-20328) HadoopRDDs create a MapReduce JobConf, but are not MapReduce jobs

2017-04-13 Thread Michael Gummelt (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15968469#comment-15968469 ] Michael Gummelt commented on SPARK-20328: - bq. I have no idea what that means. I'm pretty sure a

[jira] [Commented] (SPARK-20328) HadoopRDDs create a MapReduce JobConf, but are not MapReduce jobs

2017-04-13 Thread Michael Gummelt (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15968432#comment-15968432 ] Michael Gummelt commented on SPARK-20328: - bq. It depends. e.g. on YARN, when you submit in

[jira] [Comment Edited] (SPARK-20328) HadoopRDDs create a MapReduce JobConf, but are not MapReduce jobs

2017-04-13 Thread Michael Gummelt (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15968416#comment-15968416 ] Michael Gummelt edited comment on SPARK-20328 at 4/14/17 12:02 AM: --- bq.

[jira] [Comment Edited] (SPARK-20328) HadoopRDDs create a MapReduce JobConf, but are not MapReduce jobs

2017-04-13 Thread Michael Gummelt (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15968411#comment-15968411 ] Michael Gummelt edited comment on SPARK-20328 at 4/13/17 11:59 PM: --- bq.

[jira] [Comment Edited] (SPARK-20328) HadoopRDDs create a MapReduce JobConf, but are not MapReduce jobs

2017-04-13 Thread Michael Gummelt (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15968416#comment-15968416 ] Michael Gummelt edited comment on SPARK-20328 at 4/13/17 11:59 PM: --- bq.

[jira] [Commented] (SPARK-20328) HadoopRDDs create a MapReduce JobConf, but are not MapReduce jobs

2017-04-13 Thread Michael Gummelt (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15968416#comment-15968416 ] Michael Gummelt commented on SPARK-20328: - > It shouldn't need to do it not for the reasons you

[jira] [Commented] (SPARK-20328) HadoopRDDs create a MapReduce JobConf, but are not MapReduce jobs

2017-04-13 Thread Michael Gummelt (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15968411#comment-15968411 ] Michael Gummelt commented on SPARK-20328: - > The Mesos backend (I mean the code in Spark, not the

[jira] [Comment Edited] (SPARK-20328) HadoopRDDs create a MapReduce JobConf, but are not MapReduce jobs

2017-04-13 Thread Michael Gummelt (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15968388#comment-15968388 ] Michael Gummelt edited comment on SPARK-20328 at 4/13/17 11:27 PM: --- Hey

[jira] [Comment Edited] (SPARK-20328) HadoopRDDs create a MapReduce JobConf, but are not MapReduce jobs

2017-04-13 Thread Michael Gummelt (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15968388#comment-15968388 ] Michael Gummelt edited comment on SPARK-20328 at 4/13/17 11:27 PM: --- Hey

[jira] [Commented] (SPARK-20328) HadoopRDDs create a MapReduce JobConf, but are not MapReduce jobs

2017-04-13 Thread Michael Gummelt (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15968388#comment-15968388 ] Michael Gummelt commented on SPARK-20328: - Hey [~vanzin], thanks for the response. Everything

[jira] [Updated] (SPARK-20328) HadoopRDDs create a MapReduce JobConf, but are not MapReduce jobs

2017-04-13 Thread Michael Gummelt (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Gummelt updated SPARK-20328: Description: In order to obtain {{InputSplit}} information, {{HadoopRDD}} creates a

[jira] [Updated] (SPARK-20328) HadoopRDDs create a MapReduce JobConf, but are not MapReduce jobs

2017-04-13 Thread Michael Gummelt (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Gummelt updated SPARK-20328: Description: In order to obtain {{InputSplit}} information, {{HadoopRDD}} creates a

[jira] [Commented] (SPARK-20328) HadoopRDDs create a MapReduce JobConf, but are not MapReduce jobs

2017-04-13 Thread Michael Gummelt (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15968232#comment-15968232 ] Michael Gummelt commented on SPARK-20328: - cc [~colorant] [~hfeng] [~vanzin] > HadoopRDDs create

[jira] [Created] (SPARK-20328) HadoopRDDs create a MapReduce JobConf, but are not MapReduce jobs

2017-04-13 Thread Michael Gummelt (JIRA)
Michael Gummelt created SPARK-20328: --- Summary: HadoopRDDs create a MapReduce JobConf, but are not MapReduce jobs Key: SPARK-20328 URL: https://issues.apache.org/jira/browse/SPARK-20328 Project:

[jira] [Commented] (SPARK-16742) Kerberos support for Spark on Mesos

2017-04-10 Thread Michael Gummelt (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15963601#comment-15963601 ] Michael Gummelt commented on SPARK-16742: - bq. So, assuming that Mesos is configured properly,

[jira] [Commented] (SPARK-16742) Kerberos support for Spark on Mesos

2017-04-10 Thread Michael Gummelt (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15963583#comment-15963583 ] Michael Gummelt commented on SPARK-16742: - bq. That sounds problematic. The way YARN works is

[jira] [Commented] (SPARK-16742) Kerberos support for Spark on Mesos

2017-04-10 Thread Michael Gummelt (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15963469#comment-15963469 ] Michael Gummelt commented on SPARK-16742: - [~jerryshao] Great! The current RPC used in Mesos is

[jira] [Comment Edited] (SPARK-16742) Kerberos support for Spark on Mesos

2017-04-10 Thread Michael Gummelt (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15963446#comment-15963446 ] Michael Gummelt edited comment on SPARK-16742 at 4/10/17 8:35 PM: --

[jira] [Comment Edited] (SPARK-16742) Kerberos support for Spark on Mesos

2017-04-10 Thread Michael Gummelt (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15963446#comment-15963446 ] Michael Gummelt edited comment on SPARK-16742 at 4/10/17 8:20 PM: --

[jira] [Commented] (SPARK-16742) Kerberos support for Spark on Mesos

2017-04-10 Thread Michael Gummelt (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15963446#comment-15963446 ] Michael Gummelt commented on SPARK-16742: - bq. The most basic feature needed for any

[jira] [Comment Edited] (SPARK-16742) Kerberos support for Spark on Mesos

2017-04-10 Thread Michael Gummelt (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15963446#comment-15963446 ] Michael Gummelt edited comment on SPARK-16742 at 4/10/17 8:18 PM: --

[jira] [Commented] (SPARK-16742) Kerberos support for Spark on Mesos

2017-04-09 Thread Michael Gummelt (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15962450#comment-15962450 ] Michael Gummelt commented on SPARK-16742: - Also, note that the above Mesos implementation is not

[jira] [Comment Edited] (SPARK-16742) Kerberos support for Spark on Mesos

2017-04-09 Thread Michael Gummelt (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15962440#comment-15962440 ] Michael Gummelt edited comment on SPARK-16742 at 4/10/17 5:28 AM: -- Hi

[jira] [Commented] (SPARK-16742) Kerberos support for Spark on Mesos

2017-04-09 Thread Michael Gummelt (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15962440#comment-15962440 ] Michael Gummelt commented on SPARK-16742: - Hi [~vanzin], [~ganger85] and Strat.io are pulling

[jira] [Commented] (SPARK-20054) [Mesos] Detectability for resource starvation

2017-03-21 Thread Michael Gummelt (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15935605#comment-15935605 ] Michael Gummelt commented on SPARK-20054: - Sounds like this could be solved just by having some

[jira] [Commented] (SPARK-19373) Mesos implementation of spark.scheduler.minRegisteredResourcesRatio looks at acquired cores rather than registerd cores

2017-03-01 Thread Michael Gummelt (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15891332#comment-15891332 ] Michael Gummelt commented on SPARK-19373: - [~skonto] Either decline or hoard. > Mesos

[jira] [Updated] (SPARK-19373) Mesos implementation of spark.scheduler.minRegisteredResourcesRatio looks at acquired cores rather than registerd cores

2017-02-28 Thread Michael Gummelt (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19373?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Gummelt updated SPARK-19373: Affects Version/s: 1.6.3 2.0.2 > Mesos implementation of

[jira] [Commented] (SPARK-19373) Mesos implementation of spark.scheduler.minRegisteredResourcesRatio looks at acquired cores rather than registerd cores

2017-02-28 Thread Michael Gummelt (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15888692#comment-15888692 ] Michael Gummelt commented on SPARK-19373: - This change makes it so that the user can instruct the

[jira] [Updated] (SPARK-19702) Add Suppress/Revive support to the Mesos Spark Dispatcher

2017-02-27 Thread Michael Gummelt (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Gummelt updated SPARK-19702: Description: Due to the problem described here:

[jira] [Updated] (SPARK-19702) Increasse refuse_seconds timeout in the Mesos Spark Dispatcher

2017-02-27 Thread Michael Gummelt (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Gummelt updated SPARK-19702: Summary: Increasse refuse_seconds timeout in the Mesos Spark Dispatcher (was: Add

[jira] [Created] (SPARK-19702) Add Suppress/Revive support to the Mesos Spark Dispatcher

2017-02-22 Thread Michael Gummelt (JIRA)
Michael Gummelt created SPARK-19702: --- Summary: Add Suppress/Revive support to the Mesos Spark Dispatcher Key: SPARK-19702 URL: https://issues.apache.org/jira/browse/SPARK-19702 Project: Spark

[jira] [Created] (SPARK-19703) Add Suppress/Revive support to the Mesos Spark Driver

2017-02-22 Thread Michael Gummelt (JIRA)
Michael Gummelt created SPARK-19703: --- Summary: Add Suppress/Revive support to the Mesos Spark Driver Key: SPARK-19703 URL: https://issues.apache.org/jira/browse/SPARK-19703 Project: Spark

[jira] [Commented] (SPARK-19479) Spark Mesos artifact split causes spark-core dependency to not pull in mesos impl

2017-02-06 Thread Michael Gummelt (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15855091#comment-15855091 ] Michael Gummelt commented on SPARK-19479: - Yea, sorry for the inconvenience, but I announced this

[jira] (SPARK-16742) Kerberos support for Spark on Mesos

2017-01-30 Thread Michael Gummelt (JIRA)
Title: Message Title Michael Gummelt commented on SPARK-16742

[jira] (SPARK-16742) Kerberos support for Spark on Mesos

2017-01-30 Thread Michael Gummelt (JIRA)
Title: Message Title Michael Gummelt commented on SPARK-16742

[jira] (SPARK-16784) Configurable log4j settings

2017-01-29 Thread Michael Gummelt (JIRA)
Title: Message Title Michael Gummelt updated an issue

[jira] (SPARK-16784) Configurable log4j settings

2017-01-29 Thread Michael Gummelt (JIRA)
Title: Message Title Michael Gummelt commented on SPARK-16784

[jira] (SPARK-16784) Configurable log4j settings

2017-01-29 Thread Michael Gummelt (JIRA)
Title: Message Title Michael Gummelt edited a comment on SPARK-16784

[jira] [Created] (SPARK-19373) Mesos implementation of spark.scheduler.minRegisteredResourcesRatio looks at acquired cores rather than registerd cores

2017-01-26 Thread Michael Gummelt (JIRA)
Michael Gummelt created SPARK-19373: --- Summary: Mesos implementation of spark.scheduler.minRegisteredResourcesRatio looks at acquired cores rather than registerd cores Key: SPARK-19373 URL:

[jira] [Updated] (SPARK-10643) Support remote application download in client mode spark submit

2016-11-18 Thread Michael Gummelt (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Gummelt updated SPARK-10643: Summary: Support remote application download in client mode spark submit (was: Support

[jira] [Commented] (SPARK-10643) Support HDFS application download in client mode spark submit

2016-11-18 Thread Michael Gummelt (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15677467#comment-15677467 ] Michael Gummelt commented on SPARK-10643: - It's not just HDFS. HTTP urls fail as well: {code}

[jira] [Commented] (SPARK-18232) Support Mesos CNI

2016-11-15 Thread Michael Gummelt (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15667791#comment-15667791 ] Michael Gummelt commented on SPARK-18232: - [~rxin] Fix Version should be 2.1.0, right? > Support

[jira] [Created] (SPARK-18232) Support Mesos CNI

2016-11-02 Thread Michael Gummelt (JIRA)
Michael Gummelt created SPARK-18232: --- Summary: Support Mesos CNI Key: SPARK-18232 URL: https://issues.apache.org/jira/browse/SPARK-18232 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-16522) [MESOS] Spark application throws exception on exit

2016-10-31 Thread Michael Gummelt (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15622992#comment-15622992 ] Michael Gummelt commented on SPARK-16522: - This JIRA was for a bug in Mesos. If you're getting

[jira] [Updated] (SPARK-17454) Use Mesos disk resources

2016-09-26 Thread Michael Gummelt (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Gummelt updated SPARK-17454: Summary: Use Mesos disk resources (was: Add option to specify Mesos resource offer

[jira] [Commented] (SPARK-17454) Add option to specify Mesos resource offer constraints

2016-09-23 Thread Michael Gummelt (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17454?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15517007#comment-15517007 ] Michael Gummelt commented on SPARK-17454: - So you're trying to only launch executors on nodes

[jira] [Updated] (SPARK-16742) Kerberos support for Spark on Mesos

2016-09-12 Thread Michael Gummelt (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16742?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Gummelt updated SPARK-16742: Description: We at Mesosphere have written Kerberos support for Spark on Mesos. We'll be

[jira] [Updated] (SPARK-16742) Kerberos support for Spark on Mesos

2016-09-12 Thread Michael Gummelt (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16742?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Gummelt updated SPARK-16742: Description: We at Mesosphere have written Kerberos support for Spark on Mesos. We'll be

[jira] [Updated] (SPARK-16742) Kerberos support for Spark on Mesos

2016-09-12 Thread Michael Gummelt (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16742?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Gummelt updated SPARK-16742: Description: We at Mesosphere have written Kerberos support for Spark on Mesos. We'll be

[jira] [Commented] (SPARK-17454) Add option to specify Mesos resource offer constraints

2016-09-11 Thread Michael Gummelt (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17454?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15482682#comment-15482682 ] Michael Gummelt commented on SPARK-17454: - As of Spark 2.0, Mesos mode supports

[jira] [Created] (SPARK-17419) Mesos virtual network support

2016-09-06 Thread Michael Gummelt (JIRA)
Michael Gummelt created SPARK-17419: --- Summary: Mesos virtual network support Key: SPARK-17419 URL: https://issues.apache.org/jira/browse/SPARK-17419 Project: Spark Issue Type: Task

[jira] [Updated] (SPARK-17067) Revocable resource support

2016-09-01 Thread Michael Gummelt (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Gummelt updated SPARK-17067: Description: Blocked by https://issues.apache.org/jira/browse/MESOS-4392 > Revocable

[jira] [Closed] (SPARK-11183) enable support for mesos 0.24+

2016-09-01 Thread Michael Gummelt (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Gummelt closed SPARK-11183. --- Resolution: Done > enable support for mesos 0.24+ > -- > >

[jira] [Closed] (SPARK-6679) java.lang.ClassNotFoundException on Mesos fine grained mode and input replication

2016-09-01 Thread Michael Gummelt (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Gummelt closed SPARK-6679. -- Resolution: Won't Fix fine-grained is deprecated > java.lang.ClassNotFoundException on Mesos

[jira] [Closed] (SPARK-5197) Support external shuffle service in fine-grained mode on mesos cluster

2016-09-01 Thread Michael Gummelt (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Gummelt closed SPARK-5197. -- Resolution: Won't Fix fine-grained is deprecated > Support external shuffle service in

[jira] [Created] (SPARK-17320) Spark Mesos module not building on PRs

2016-08-30 Thread Michael Gummelt (JIRA)
Michael Gummelt created SPARK-17320: --- Summary: Spark Mesos module not building on PRs Key: SPARK-17320 URL: https://issues.apache.org/jira/browse/SPARK-17320 Project: Spark Issue Type:

[jira] [Closed] (SPARK-16627) --jars doesn't work in Mesos mode

2016-08-25 Thread Michael Gummelt (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16627?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Gummelt closed SPARK-16627. --- Resolution: Won't Fix > --jars doesn't work in Mesos mode >

[jira] [Commented] (SPARK-17240) SparkConf is Serializable but contains a non-serializable field

2016-08-25 Thread Michael Gummelt (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15437436#comment-15437436 ] Michael Gummelt commented on SPARK-17240: - cc [~vanzin] > SparkConf is Serializable but contains

[jira] [Created] (SPARK-17240) SparkConf is Serializable but contains a non-serializable field

2016-08-25 Thread Michael Gummelt (JIRA)
Michael Gummelt created SPARK-17240: --- Summary: SparkConf is Serializable but contains a non-serializable field Key: SPARK-17240 URL: https://issues.apache.org/jira/browse/SPARK-17240 Project: Spark

[jira] [Commented] (SPARK-10401) spark-submit --unsupervise

2016-08-19 Thread Michael Gummelt (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15428572#comment-15428572 ] Michael Gummelt commented on SPARK-10401: - This should probably be a separate JIRA, but I'm just

[jira] [Commented] (SPARK-17067) Revocable resource support

2016-08-15 Thread Michael Gummelt (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15421698#comment-15421698 ] Michael Gummelt commented on SPARK-17067: - Add revocable resource support:

[jira] [Created] (SPARK-17067) Revocable resource support

2016-08-15 Thread Michael Gummelt (JIRA)
Michael Gummelt created SPARK-17067: --- Summary: Revocable resource support Key: SPARK-17067 URL: https://issues.apache.org/jira/browse/SPARK-17067 Project: Spark Issue Type: Improvement

[jira] [Comment Edited] (SPARK-16784) Configurable log4j settings

2016-08-11 Thread Michael Gummelt (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16784?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15417857#comment-15417857 ] Michael Gummelt edited comment on SPARK-16784 at 8/11/16 8:11 PM: --

[jira] [Commented] (SPARK-16784) Configurable log4j settings

2016-08-11 Thread Michael Gummelt (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16784?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15417856#comment-15417856 ] Michael Gummelt commented on SPARK-16784: - `log4j.debug=true` only results in log4j printing its

[jira] [Reopened] (SPARK-16784) Configurable log4j settings

2016-08-11 Thread Michael Gummelt (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Gummelt reopened SPARK-16784: - `log4j.debug=true` only results in log4j printing its debugging messages. It doesn't turn

[jira] [Comment Edited] (SPARK-16784) Configurable log4j settings

2016-08-11 Thread Michael Gummelt (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16784?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15417857#comment-15417857 ] Michael Gummelt edited comment on SPARK-16784 at 8/11/16 8:10 PM: --

[jira] [Issue Comment Deleted] (SPARK-16784) Configurable log4j settings

2016-08-11 Thread Michael Gummelt (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Gummelt updated SPARK-16784: Comment: was deleted (was: `log4j.debug=true` only results in log4j printing its debugging

[jira] [Created] (SPARK-17002) Document that spark.ssl.protocol. is required for SSL

2016-08-10 Thread Michael Gummelt (JIRA)
Michael Gummelt created SPARK-17002: --- Summary: Document that spark.ssl.protocol. is required for SSL Key: SPARK-17002 URL: https://issues.apache.org/jira/browse/SPARK-17002 Project: Spark

[jira] [Reopened] (SPARK-16522) [MESOS] Spark application throws exception on exit

2016-08-09 Thread Michael Gummelt (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Gummelt reopened SPARK-16522: - Reopening so we can track this until it's merged into the 2.0 branch. Also changed the fix

[jira] [Updated] (SPARK-16522) [MESOS] Spark application throws exception on exit

2016-08-09 Thread Michael Gummelt (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Gummelt updated SPARK-16522: Fix Version/s: (was: 2.1.0) 2.0.1 > [MESOS] Spark application

[jira] [Commented] (SPARK-16967) Collect Mesos support code into a module/profile

2016-08-09 Thread Michael Gummelt (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15414004#comment-15414004 ] Michael Gummelt commented on SPARK-16967: - Will do > Collect Mesos support code into a

[jira] [Commented] (SPARK-12909) Spark on Mesos accessing Secured HDFS w/Kerberos

2016-08-09 Thread Michael Gummelt (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15413999#comment-15413999 ] Michael Gummelt commented on SPARK-12909: - I agree. I just spoke with Reynold about this. I'll

[jira] [Commented] (SPARK-12909) Spark on Mesos accessing Secured HDFS w/Kerberos

2016-08-08 Thread Michael Gummelt (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15412624#comment-15412624 ] Michael Gummelt commented on SPARK-12909: - DC/OS Spark has this functionality, and we'll be

[jira] [Commented] (SPARK-11638) Run Spark on Mesos with bridge networking

2016-08-08 Thread Michael Gummelt (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15412461#comment-15412461 ] Michael Gummelt commented on SPARK-11638: - [~radekg] > The only advantage we had was using the

[jira] [Commented] (SPARK-16944) [MESOS] Improve data locality when launching new executors when dynamic allocation is enabled

2016-08-07 Thread Michael Gummelt (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15411315#comment-15411315 ] Michael Gummelt commented on SPARK-16944: - Yea, we typically call it "delay scheduling". It was

[jira] [Commented] (SPARK-16944) [MESOS] Improve data locality when launching new executors when dynamic allocation is enabled

2016-08-07 Thread Michael Gummelt (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15411309#comment-15411309 ] Michael Gummelt commented on SPARK-16944: - Since Mesos is offer based, it's up to the Spark

[jira] [Commented] (SPARK-16944) [MESOS] Improve data locality when launching new executors when dynamic allocation is enabled

2016-08-07 Thread Michael Gummelt (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15411307#comment-15411307 ] Michael Gummelt commented on SPARK-16944: - I think we can improve both with and without dynamic

[jira] [Commented] (SPARK-11638) Run Spark on Mesos with bridge networking

2016-08-07 Thread Michael Gummelt (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15411277#comment-15411277 ] Michael Gummelt commented on SPARK-11638: - This JIRA is complex and a lot of it is out of date.

[jira] [Created] (SPARK-16927) Mesos Cluster Dispatcher default properties

2016-08-05 Thread Michael Gummelt (JIRA)
Michael Gummelt created SPARK-16927: --- Summary: Mesos Cluster Dispatcher default properties Key: SPARK-16927 URL: https://issues.apache.org/jira/browse/SPARK-16927 Project: Spark Issue

  1   2   >