I would suggest a cluster on AWS and a cluster on-prem. Then tooling on top to manage between the 2. It is unlikely that a failure of a task on-prem should have a scheduled replacement on AWS or vise versa. It is likely that you will end up creating constraints to statically partition the clusters anyway IMO. 2 Clusters eliminates most of your proposed questions.
ken > On Jun 30, 2016, at 10:57 AM, Florian Pfeiffer <[email protected]> wrote: > > Hi, > > the last 2 years I managed a mesos cluster with bare-metal on-premise. Now at > my new company, the situation is a little bit different, and I'm wondering if > there are some kind of best practices: > The company is in the middle of a transition from on-premise to AWS. The old > stuff is still running in the DC, the newer micro services are running within > autoscales groups on AWS and other AWS services like DynamoDB, Kinesis and > Lambda are also on the rise. > > So in my naive view of the world (where no problems occur..... never!) I'm > thinking that it would be great to span a hybrid mesos cluster over AWS&DC to > leverage the still available resources in the DC which gets more and more > underutilized over the time. > > Now my naive world view slowly crumbles, and I realize that I'm missing the > experience with AWS. Questions that are already popping up (beside all those > Questions, where I currently don't know that I will have them...) are: > * Is Virtual Private Gateway to my VPC enough, or do I need to aim for a > Direct Connect? > * Put everything into one Account, or use a Multi-Account strategy? (Mainly > to prevent things running amok and drag stuff down while running into an > account wide shared limit?) > * Will e.g. DynamoDb be "fast" enough if it's accessed from the Datacenter. > > I'll appreciate any feedback or lessons learned about that topic :) > > Thanks, > Florian >

