Hi,
Short: Is it better to have one big framework and executor with if statements to select what to do or several smaller framework <-> executors when writing a Mesos app? Longer question: Last week I started a side project based on mesos (using Go), http://blog.fmpwizard.com/blog/web-crawler-using-mesos-and-golang https://github.com/fmpwizard/owlcrawler It's a web crawler written on top of Mesos, The very first version of it had a framework that sent a task to an executor and that single executor would fetch the page, extract links from the html and then send them to a message queue. Then the framework reads that queue and starts again, run the executor, etc, etc. Now I'm splitting fetching the html and extracting links into two separate tasks, and putting those two tasks in the same executor doesn't feel right, so I'm thinking that I need at least two diff executors and one framework, but then I wonder if people more experienced with mesos would normally write several pairs of framework <-> executors to keep the design cleaner. On this particular case, I can see the project growing into even more tasks that can be decoupled. Any feedback on the design would be great and let me know if I should explain this better. Thanks Diego -- Diego Medina Lift/Scala consultant [email protected] http://fmpwizard.telegr.am
