[
https://issues.apache.org/jira/browse/STORM-2018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16118779#comment-16118779
]
Erik Weathers commented on STORM-2018:
--------------------------------------
Thanks for the quick responses [~revans2] & [~kabhwan]! Here's some
follow-ups:
Firstly, I was a bit unfair in one thing I said:
{quote}
Removal of the ability to launch the daemon like this:
backtype.storm.daemon.supervisor.launch(new MesosSupervisor());
{quote}
That would have been *intentionally* broken when we changed from {{backtype}}
to {{org.apache}} anyways. So my focus on the end portion of the call
({{...supervisor.launch()}} not existing anymore is a bit unfair -- we would
have needed to touch the code anyways!
{quote}
Could you file an issue regarding this? I'd be really appreciated if you can
provide a patch and being a tester. (No one claimed this before so that's a
missing point of testing.)
{quote}
{quote}
We should be able to fix these, but it will take a bit of work to be sure that
it is all correct. Please file a JIRA against us and assign it to me. I'll do
what I can to fix them ASAP.
{quote}
I'll make the change & can test it, but thanks for offering to help! I filed
STORM-2690 towards this end, I'll "@" you two on my PR when it arises. Just
FYI, we've found what we believe is a workaround for the problem in
storm-mesos, so it's not super urgent, but should probably be fixed before more
storm releases percolate out.
{quote}
TBH, it would be ideal to support mesos integration officially. Storm
officially supports only standalone, and storm-yarn and storm-mesos are
supported from community which are hard to catch up Storm development. They're
not considered in release verification phase so it's hard to guarantee
compatibility between Storm release and community's efforts.
{quote}
{quote}
And yes we probably need some step in the release process to look at testing
external projects. The ISupervisor interface is really only used by storm mesos
right now, as storm-yarn is effectively dead, so it is not always something we
pay that much attention to, even though we should.
{quote}
Yeah, I was under the impression that storm-yarn had died on the vine from
previous statements I've read from you Bobby. It would be nice to get
storm-mesos properly integrated into the storm releases themselves -- that will
be a long effort I suspect, so for now we'll just hope the interfaces are
sufficient.
> Simplify Threading Model of the Supervisor
> ------------------------------------------
>
> Key: STORM-2018
> URL: https://issues.apache.org/jira/browse/STORM-2018
> Project: Apache Storm
> Issue Type: New Feature
> Components: storm-core
> Affects Versions: 1.0.0, 2.0.0
> Reporter: Robert Joseph Evans
> Assignee: Robert Joseph Evans
> Fix For: 2.0.0, 1.1.0, 1.0.3
>
> Attachments: Slot.dot, Slot.svg
>
> Time Spent: 45h 20m
> Remaining Estimate: 0h
>
> We have been trying to roll out CGROUP enforcement and right now are running
> into a number of race conditions in the supervisor. When using CGROUPS the
> timing of some operations are different and are exposing issues that we would
> not see without this.
> In order to make progress with testing/deploying CGROUP and RAS we are going
> to try and refactor the supervisor to have a simpler threading model, but
> likely with more threads. We will base the code off of the java code
> currently in master, and may replace that in the 2.0 release, but plan on
> having it be a part of 1.x too, if it truly is more stable.
> I will try to keep this JIRA up to date with what we are doing and the
> architecture to keep the community informed. We need to move quickly to meet
> some of our company goals but will not just shove this in. We welcome any
> feedback on the design and code before it goes into the community.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
