[
https://issues.apache.org/jira/browse/AURORA-1618?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15145400#comment-15145400
]
Benjamin Staffin commented on AURORA-1618:
------------------------------------------
Some brain-dumping:
h5. Instance types:
It depends on your workloads of course, but in general the largest instances of
whatever type you choose will be the most cost effective, since mesos means you
can binpack jobs onto them pretty effectively. T2.large instances are great
for dev clusters that aren't cpu loaded. M4 are a decent default choice for
production clusters, or C4 if your workloads are CPU-bound.
h5. Regions/AZs:
I've been doing region==cluster, and labeling slave nodes with what
Availability Zone they fall in to allow sensible scheduling constraints.
You'll want to distribute your master nodes across AZs if possible. EBS isn't
really necessary, but the instance types you'll probably want to use for new
deploys (M4, maybe T2 for development) are EBS-only.
h5. IPs/Hostnames:
Run everything in a VPC. If you have ec2's auto-assigned hostnames enabled,
and your instances are assigned external IPs, they will have two hostnames:
"public dns" and "private dns". From inside the VPC, the "public dns" hostname
will always resolve to the instance's private IP address, so you can run all
the mesos/aurora components with {{--hostname=<public_dns_hostname>}} and
benefit from the hostnames all ending with .amazonaws.com, which can make
things like VPN configuration simpler.
h5. Finding zookeeper
Since you're on AWS, you should consider launching all your components in
autoscaling groups (with fixed instance counts, not dynamic), including
Zookeeper. To get a predictable hostname for your zookeeper ensemble you can
use ELB (zookeeper works behind a TCP load balancer) or a cronjob that updates
a Route53 dns entry to point at your zookeeper nodes. Exhibitor makes running
zookeeper in autoscaling groups very convenient, and provides an http endpoing
listing the ensemble members, which you can then sync to route53 with something
like https://github.com/benley/exhibitor2dns.
h5. Finding the mesos/aurora masters
Use Zookeeper where possible. If you're using Mesos-DNS, that will give you a
consistent hostname for your mesos masters. You could also use something like
Aurproxy to set up a http reverse proxy for your mesos and aurora master nodes.
If you run your mesos/aurora masters in autoscaling groups too, they could
also potentially go behind ELBs.
> Deploying Aurora on AWS
> -----------------------
>
> Key: AURORA-1618
> URL: https://issues.apache.org/jira/browse/AURORA-1618
> Project: Aurora
> Issue Type: Story
> Components: Documentation
> Reporter: Stephan Erb
> Labels: documentation
>
> AWS tends to be one of the more popular deployment targets these days.
> Especially for new adopters of Aurora (and Mesos), this can be quite tricky.
> We should provide a high level guide covering the basic ground necessary for
> a solid deployment.
> Potential issues/questions that should probably be covered:
> * recommended instance type?
> * mapping of components onto regions and availability zones?
> * do I need EBS for my master and scheduler nodes?
> * recommended way how to bind ip/hostnames of the different components, so
> that they can reliably talk to each other
> * ...
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
