storm-yarn was originally done as a proof of concept. We had plans to take it
further, but the amount of work required to make it production ready on a very
heavily used cluster was more then we were willing to invest at the time. Most
of that work was around network scheduling, isolation and prioritization,
mainly in YARN itself. There has been some work looking into this, but nothing
much has happened with it. At the same time
http://slider.incubator.apache.org/ showed up and is now the preferred way to
run Storm on YARN. To get around the networking issues most people will tag a
subset of their cluster, a few racks, and only schedule storm to run on those
nodes. Long term I really would like to revive storm on yarn, and integrate it
directly into storm. Giving storm and the scheduler the ability to request new
resources with specific constraints opens up a lot of new possibilities. If
you want to help out, or if anyone else wants to help out with this work, I
would be very happy to file some JIRA in open source and help direct what needs
to be done.
- Bobby
On Wednesday, May 27, 2015 4:59 AM, Spico Florin <[email protected]>
wrote:
Hello!I'm interesting in running the storm topologies on yarn. I was looking
at the yahoo project https://github.com/yahoo/storm-yarn, and I could observed
that there is no activity since 7 months ago. Also, the issues and requests
lists are not updated.Therefore I have some questions:1. Is there any plan to
evolve this project?2. Is there any plan to integrate this project in the main
branch?3. Is someone using this approach in production ready mode?
I look forward for your answers. Regards, Florin
