GitHub user pmackles opened a pull request:
https://github.com/apache/spark/pull/19515
[SPARK-22287][MESOS] SPARK_DAEMON_MEMORY not honored by MesosClusterDâ¦
â¦ispatcher
## What changes were proposed in this pull request?
Allow JVM max heap size to be controlled
GitHub user pmackles opened a pull request:
https://github.com/apache/spark/pull/19358
[SPARK-22135] metrics in spark-dispatcher not being registered properly
## What changes were proposed in this pull request?
Fix a trivial bug with how metrics are registered in the mesos
Github user pmackles commented on the issue:
https://github.com/apache/spark/pull/19358
thanks for the quick review @srowen and @jerryshao - both requested changes
committed
---
-
To unsubscribe, e-mail: reviews
Github user pmackles commented on the issue:
https://github.com/apache/spark/pull/19515
Hi @ArtRand - Based on my testing and interpretation of the code,
```SPARK_DRIVER_MEMORY``` has no affect on MesosClusterDispatcher. Heap size
always winds up being set to ```-Xmx1G
Github user pmackles commented on the issue:
https://github.com/apache/spark/pull/19543
@susanxhuynh, @ttashi-rms, @ArtRand - thanks for the feedback. I was able
to make all of the suggested changes
Github user pmackles commented on the issue:
https://github.com/apache/spark/pull/19543
@felixcheung - any chance of getting this merged into the upcoming 2.2.1
release? I cleaned up the merge conflict
Github user pmackles commented on the issue:
https://github.com/apache/spark/pull/19515
@felixcheung - any chance of getting this tiny change merged and included
in the upcoming 2.2.1 release?
---
-
To unsubscribe
Github user pmackles commented on the issue:
https://github.com/apache/spark/pull/19543
@ttashi-rms v2.2 and up
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user pmackles commented on the issue:
https://github.com/apache/spark/pull/19543
@ArtRand - thanks for the feedback. I made the suggested changes to the
docs and added some more tests - including a test w/ multiple constraints
Github user pmackles commented on the issue:
https://github.com/apache/spark/pull/19543
My apologies. Reread the docs and ran style checks locally this time.
Fingers crossed
---
-
To unsubscribe, e-mail: reviews
Github user pmackles commented on the issue:
https://github.com/apache/spark/pull/19543
@felixcheung - fixed scala-style issues and also updated the docs to
include the new property
---
-
To unsubscribe, e-mail
GitHub user pmackles opened a pull request:
https://github.com/apache/spark/pull/19543
[SPARK-19606][MESOS] Support constraints in spark-dispatcher
## What changes were proposed in this pull request?
A discussed in SPARK-19606, the addition of a new config property named
Github user pmackles commented on the issue:
https://github.com/apache/spark/pull/19515
@ArtRand - WDYT? I was going to switch it to ```SPARK_DISPATCHER_MEMORY```
but then I noticed that the other env vars for MesosClusterDispatcher or also
prefixed with ```SPARK_DAEMON_*``` so I
Github user pmackles commented on the issue:
https://github.com/apache/spark/pull/19515
@ArtRand - yeah default would remain the same which is 1g
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
GitHub user pmackles opened a pull request:
https://github.com/apache/spark/pull/20335
[SPARK-23088][CORE] History server not showing incomplete/running
applications
## What changes were proposed in this pull request?
History server not showing incomplete/running
GitHub user pmackles opened a pull request:
https://github.com/apache/spark/pull/21027
[SPARK-23943][MESOS][DEPLOY] Improve observability of
MesosRestServer/MesosClusterDiâ¦
See https://issues.apache.org/jira/browse/SPARK-23943 for details on
proposed changes
Tested
GitHub user pmackles opened a pull request:
https://github.com/apache/spark/pull/21006
[SPARK-22256][MESOS] - Introduce spark.mesos.driver.memoryOverhead
When running spark driver in a container such as when using the Mesos
dispatcher service, we need to apply the same rules
GitHub user pmackles opened a pull request:
https://github.com/apache/spark/pull/21075
[SPARK-23988][MESOS] Improve handling of appResource in mesos dispatcher
when using Docker
Improve/fix handling of appResource for mesos dispatcher when using docker
Tested with new unit
18 matches
Mail list logo
