[jira] [Commented] (FLINK-6379) Implement FLIP-6 Mesos Resource Manager
[ https://issues.apache.org/jira/browse/FLINK-6379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16067846#comment-16067846 ] ASF GitHub Bot commented on FLINK-6379: --- Github user asfgit closed the pull request at: https://github.com/apache/flink/pull/3942 > Implement FLIP-6 Mesos Resource Manager > --- > > Key: FLINK-6379 > URL: https://issues.apache.org/jira/browse/FLINK-6379 > Project: Flink > Issue Type: Sub-task > Components: Mesos >Reporter: Eron Wright >Assignee: Eron Wright > > Given the new ResourceManager of FLIP-6, implement a new > MesosResourceManager. > The minimal effort would be to implement a new resource manager while > continuing to use the various local actors (launch coordinator, task monitor, > etc.) which implement the various FSMs associated with Mesos scheduling. > The Fenzo library would continue to solve the packing problem of matching > resource offers to slot requests. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (FLINK-6379) Implement FLIP-6 Mesos Resource Manager
[ https://issues.apache.org/jira/browse/FLINK-6379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16066083#comment-16066083 ] ASF GitHub Bot commented on FLINK-6379: --- Github user tillrohrmann commented on the issue: https://github.com/apache/flink/pull/3942 Really good work @EronWright. I've completed the review and rebased it on the latest master. While doing the review I applied one fix for a race condition in the tests, another fix to address the newly introduced checkstyle rules and one small change which removes the `MesosResourceManager's` internal RPC calls from the public interface, because they should only be needed by the RM's internal components. All small things. I really liked how you wrote the tests :-) Once Travis gives green light, I will merge your PR. As a follow up, I noticed two things: 1. Proper shutdown of internal components 2. Getting rid (at some point) of Akka dependency > Implement FLIP-6 Mesos Resource Manager > --- > > Key: FLINK-6379 > URL: https://issues.apache.org/jira/browse/FLINK-6379 > Project: Flink > Issue Type: Sub-task > Components: Mesos >Reporter: Eron Wright >Assignee: Eron Wright > > Given the new ResourceManager of FLIP-6, implement a new > MesosResourceManager. > The minimal effort would be to implement a new resource manager while > continuing to use the various local actors (launch coordinator, task monitor, > etc.) which implement the various FSMs associated with Mesos scheduling. > The Fenzo library would continue to solve the packing problem of matching > resource offers to slot requests. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (FLINK-6379) Implement FLIP-6 Mesos Resource Manager
[ https://issues.apache.org/jira/browse/FLINK-6379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16061510#comment-16061510 ] ASF GitHub Bot commented on FLINK-6379: --- Github user tillrohrmann commented on the issue: https://github.com/apache/flink/pull/3942 Sorry for the long delay Eron. I've started reviewing the code last week. I'm not completely done yet, but so far things look good. As a follow up we could think about how we can remove the dependency on Akka by also converting the `ConnectionMonitor`, `TaskMonitor`, `LaunchCoordinator` and the `ReconciliationCoordinator` components. I hope to finish the review next week and finally merge the PR. > Implement FLIP-6 Mesos Resource Manager > --- > > Key: FLINK-6379 > URL: https://issues.apache.org/jira/browse/FLINK-6379 > Project: Flink > Issue Type: Sub-task > Components: Mesos >Reporter: Eron Wright >Assignee: Eron Wright > > Given the new ResourceManager of FLIP-6, implement a new > MesosResourceManager. > The minimal effort would be to implement a new resource manager while > continuing to use the various local actors (launch coordinator, task monitor, > etc.) which implement the various FSMs associated with Mesos scheduling. > The Fenzo library would continue to solve the packing problem of matching > resource offers to slot requests. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (FLINK-6379) Implement FLIP-6 Mesos Resource Manager
[ https://issues.apache.org/jira/browse/FLINK-6379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16058431#comment-16058431 ] ASF GitHub Bot commented on FLINK-6379: --- Github user EronWright commented on the issue: https://github.com/apache/flink/pull/3942 @tillrohrmann thoughts on this? I'm open to anything. > Implement FLIP-6 Mesos Resource Manager > --- > > Key: FLINK-6379 > URL: https://issues.apache.org/jira/browse/FLINK-6379 > Project: Flink > Issue Type: Sub-task > Components: Mesos >Reporter: Eron Wright >Assignee: Eron Wright > > Given the new ResourceManager of FLIP-6, implement a new > MesosResourceManager. > The minimal effort would be to implement a new resource manager while > continuing to use the various local actors (launch coordinator, task monitor, > etc.) which implement the various FSMs associated with Mesos scheduling. > The Fenzo library would continue to solve the packing problem of matching > resource offers to slot requests. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (FLINK-6379) Implement FLIP-6 Mesos Resource Manager
[ https://issues.apache.org/jira/browse/FLINK-6379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16028109#comment-16028109 ] ASF GitHub Bot commented on FLINK-6379: --- Github user tillrohrmann commented on the issue: https://github.com/apache/flink/pull/3942 Thanks for your contribution @EronWright. I will try to review and merge this PR this week. > Implement FLIP-6 Mesos Resource Manager > --- > > Key: FLINK-6379 > URL: https://issues.apache.org/jira/browse/FLINK-6379 > Project: Flink > Issue Type: Sub-task > Components: Mesos >Reporter: Eron Wright >Assignee: Eron Wright > > Given the new ResourceManager of FLIP-6, implement a new > MesosResourceManager. > The minimal effort would be to implement a new resource manager while > continuing to use the various local actors (launch coordinator, task monitor, > etc.) which implement the various FSMs associated with Mesos scheduling. > The Fenzo library would continue to solve the packing problem of matching > resource offers to slot requests. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (FLINK-6379) Implement FLIP-6 Mesos Resource Manager
[ https://issues.apache.org/jira/browse/FLINK-6379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16016733#comment-16016733 ] ASF GitHub Bot commented on FLINK-6379: --- Github user EronWright commented on the issue: https://github.com/apache/flink/pull/3942 This is a fairly straightforward port of the RM code to the FLIP-6 architecture. Follow-ups to occur in future PRs include: - adding support for releasing the worker - compensating for the loss of a worker (beyond state cleanup) - eliminating the use of `RegisteredMesosWorkerNode` > Implement FLIP-6 Mesos Resource Manager > --- > > Key: FLINK-6379 > URL: https://issues.apache.org/jira/browse/FLINK-6379 > Project: Flink > Issue Type: Sub-task > Components: Mesos >Reporter: Eron Wright >Assignee: Eron Wright > > Given the new ResourceManager of FLIP-6, implement a new > MesosResourceManager. > The minimal effort would be to implement a new resource manager while > continuing to use the various local actors (launch coordinator, task monitor, > etc.) which implement the various FSMs associated with Mesos scheduling. > The Fenzo library would continue to solve the packing problem of matching > resource offers to slot requests. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (FLINK-6379) Implement FLIP-6 Mesos Resource Manager
[
https://issues.apache.org/jira/browse/FLINK-6379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16016730#comment-16016730
]
ASF GitHub Bot commented on FLINK-6379:
---
GitHub user EronWright opened a pull request:
https://github.com/apache/flink/pull/3942
FLINK-6379 Mesos ResourceManager (FLIP-6)
- Make the RPC gateway of the ResourceManager extensible to allow for
framework-specific RPC methods
- Introduce FLIP-6 MesosResourceManager w/ tests
- Introduce a Mesos-specific RPC gateway for callbacks from child actors
and from the Mesos scheduler client
- Enhance the persistent Mesos worker store to track the resource profile
associated with a worker
- Convert RegisteredMesosWorkerNode to Java
- Decline TE registration if framework doesn’t recognize the worker
Thanks for contributing to Apache Flink. Before you open your pull request,
please take the following check list into consideration.
If your changes take all of the items into account, feel free to open your
pull request. For more information and/or questions please refer to the [How To
Contribute guide](http://flink.apache.org/how-to-contribute.html).
In addition to going through the list, please provide a meaningful
description of your changes.
- [ ] General
- The pull request references the related JIRA issue ("[FLINK-XXX] Jira
title text")
- The pull request addresses only one issue
- Each commit in the PR has a meaningful commit message (including the
JIRA id)
- [ ] Documentation
- Documentation has been added for new functionality
- Old documentation affected by the pull request has been updated
- JavaDoc for public methods has been added
- [ ] Tests & Build
- Functionality added by the pull request is covered by tests
- `mvn clean verify` has been executed successfully locally or a Travis
build has passed
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/EronWright/flink FLINK-6379
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/3942.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #3942
commit c3d080d07c4f8de5ea8459e9f40921f081051b61
Author: Wright, Eron
Date: 2017-05-19T00:34:16Z
FLINK-6379 Mesos ResourceManager (FLIP-6)
- Make the RPC gateway of the ResourceManager extensible to allow for
framework-specific RPC methods
- Introduce FLIP-6 MesosResourceManager w/ tests
- Introduce a Mesos-specific RPC gateway for callbacks from child actors
and from the Mesos scheduler client
- Enhance the persistent Mesos worker store to track the resource profile
associated with a worker
- Convert RegisteredMesosWorkerNode to Java
- Decline TE registration if framework doesn’t recognize the worker
> Implement FLIP-6 Mesos Resource Manager
> ---
>
> Key: FLINK-6379
> URL: https://issues.apache.org/jira/browse/FLINK-6379
> Project: Flink
> Issue Type: Sub-task
> Components: Mesos
>Reporter: Eron Wright
>Assignee: Eron Wright
>
> Given the new ResourceManager of FLIP-6, implement a new
> MesosResourceManager.
> The minimal effort would be to implement a new resource manager while
> continuing to use the various local actors (launch coordinator, task monitor,
> etc.) which implement the various FSMs associated with Mesos scheduling.
> The Fenzo library would continue to solve the packing problem of matching
> resource offers to slot requests.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
