Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/10761
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is ena
Github user HyukjinKwon commented on the pull request:
https://github.com/apache/spark/pull/10761#issuecomment-213164527
@kevincox Could you please close this for now? You can easily reopen this
once you start to work.
---
If your project is set up for it, you can reply to this email
Github user HyukjinKwon commented on the pull request:
https://github.com/apache/spark/pull/10761#issuecomment-213151310
@kevincox right.
- Usually, I believe such changes need a JIRA which manages the issues in
Spark. Usually there sould be a JIRA to discuss a proper way to fix an
Github user srowen commented on the pull request:
https://github.com/apache/spark/pull/10761#issuecomment-213102473
This should be closed in the meantime
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project doe
Github user kevincox commented on the pull request:
https://github.com/apache/spark/pull/10761#issuecomment-213100491
@HyukjinKwon The maintainers didn't seem interested and I was busy. I might
pursue this further this summer.
---
If your project is set up for it, you can reply to th
Github user HyukjinKwon commented on the pull request:
https://github.com/apache/spark/pull/10761#issuecomment-212692762
ping @kevincox
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this f
Github user andrewor14 commented on the pull request:
https://github.com/apache/spark/pull/10761#issuecomment-202642465
Let's close this PR until we have a proper design discussion on a JIRA.
---
If your project is set up for it, you can reply to this email and have your
reply appear
Github user andrewor14 commented on the pull request:
https://github.com/apache/spark/pull/10761#issuecomment-178224421
@kevincox please follow
https://cwiki.apache.org/confluence/display/SPARK/Contributing+to+Spark. This
task needs a JIRA and you need to link it in the title. For big
Github user kevincox commented on the pull request:
https://github.com/apache/spark/pull/10761#issuecomment-172707827
It is similar but it has a number of advantages including that it can be
cleaner (you can wait until after you finish a task and prepare it for
shuffling) and you get
Github user jerryshao commented on the pull request:
https://github.com/apache/spark/pull/10761#issuecomment-172700451
Hi @kevincox , IIUC looks like your description of dynamic allocation is
quite similar to some kinds of preemption mechanism in the cluster manager.
>This al
Github user kevincox commented on the pull request:
https://github.com/apache/spark/pull/10761#issuecomment-171851889
Fair enough. I tried to keep the tests as similar as possible so that it is
quite clear that the functionality hasn't changed. I have also done some
testing on my own
Github user rxin commented on the pull request:
https://github.com/apache/spark/pull/10761#issuecomment-171851470
OK that makes sense. Thanks for explaining. The biggest problem with
refactoring like this is you'd need to find somebody to review it, and it is
pretty hard to review ...
Github user kevincox commented on the pull request:
https://github.com/apache/spark/pull/10761#issuecomment-171850023
I'm implementing a system where Spark can reduce the number of executors in
low-resource situations. This allows jobs to utilize an entire cluster when it
is unneeded
Github user rxin commented on the pull request:
https://github.com/apache/spark/pull/10761#issuecomment-171844919
What future features are you thinking about? Refactoring like this for the
sake of refactoring is pretty dangerous ...
---
If your project is set up for it, you can reply
Github user kevincox commented on the pull request:
https://github.com/apache/spark/pull/10761#issuecomment-171835801
Note that this refactoring was performed to make way for future features in
the allocation manager.
---
If your project is set up for it, you can reply to this email
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/10761#issuecomment-171835570
Can one of the admins verify this patch?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your p
GitHub user kevincox opened a pull request:
https://github.com/apache/spark/pull/10761
Refactor ExecutorAllocationManager.
This changes ExecuorAllocationManager from a tree of if
statements run on an interval to a clean event-driven state machine.
You can merge this pull request
17 matches
Mail list logo
