Hi all,
Sorry for the delay in my response - been caught up teaching and just
jumped across Australia today.
The 1GiB is perhaps a little too small - especially on an m1.small host
with no other instance store disks. 10GiB is listed as the maximum for
S3 baked instances... but my rushed testing since Sunday found that the
bundling failed for this using our scripts (ie, I updated the manifest
file).
if you're using the S3 backed image on anything larger than an m1.small
then you may have additional ephemeral/instance-store disks that you can
map (on instance launch), format and mount to their full size, but the
root disk is still limited to being with the 10 GiB upper bound.
http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ComponentsAMIs.html#storage-for-the-root-device
If you're finding IO times on EBS are an issue, reach out to support for
them to have a look; otherwise, have a look at Provisioned IOPs. It's
worth noting that after a certain amount of I/O, turning on PIOPs can be
cheaper than not having it turned on - so if you're doing that level of
I/O its worth looking closer at if disk IO is the issue. Another option
you have is to use EBS for your root file system, but use (a set of)
instance store disks for your application to serve from. You may also
want to RAID 1 mirror multiple instance store disks together to guard
against failure of individual instance store disks (and should any fail,
schedule the instance to be terminated and re-provisioned at your
convenience).
Creating your own AMI should be as easy as checking out from github
(https://github.com/andsens/bootstrap-vz), setting your environment
variables up with a secret key, accesss key - and for an S3 backed AMi
you currently also need a certificate (see IAM) - edit the manifest file
(manifests/ec2-s3-....manifest) to change the size of the disk, and then
run bootstrap-vz manifests/ec2-s3-....manifest)
James
On 11/02/2014 1:01 AM, Alistair Prestidge wrote:
Thanks for that list its pretty much what I thought you would say.
Glad I am not missing something with EBS volumes.
I am for the time being going to perceiver with the instance store
backed AMI's and give it a go.
However I just tried to increase the size of the root partition myself
for ami-80d322f7 but for some reason I cannot create an my own AMI
from this ami
the option in the aws console is greyed out any thoughts?
Also I was wondering how easy is it for me to build my own AMI using
all your amazing scripts? ;)
Where do I start? I cannot find a great deal of documentation on how
to create AMI from scratch.
Or I can find a lot that does not really make a lot of sense to me.
Is there a dummies guide somewhere that you could recommend?
Thanks
On 10 February 2014 08:13, Brian Gupta <[email protected]
<mailto:[email protected]>> wrote:
On Mon, Feb 10, 2014 at 2:38 AM, Alistair Prestidge
<[email protected]
<mailto:[email protected]>> wrote:
> Hi brain thanks for your reply.
>
> You say in almost all cases you would recommend esb backed
device's. Can I
> ask why? Iam looking to migrate to instance store from ebs
mainly for io
> reliability issues.
>
> I am having io issues with ebs backed instances the io can be
very erratic
> and cause my sites to have very slow response times from time to
time.
>
> I have currently put the code and logs on an ephemeral partition
and the
> sites are much more stable.
>
> I have no reason to keep any code or data that is on the server
after reboot
> or termination.
>
> Thanks
Well you may actually have a use case that justifies instance-store,
if cost is a
big concern. Typically I have found EBS more pragmatic for the
following reasons:
1) Can base infrastructure off of a single AMI, and use Block Device
parameters [1]
to change the characteristics of the resulting root volume. (i.e.
- one can specify
block device size and performance characteristics.)
2) One is not limited to a 10GB root volume. (1TB is EBS current
limit.)
3) In the case one needs variant AMIs, it is generally much simpler
with EBS root
volumes, since the process is "snapshot" based, and controllable
at the API layer.
4) One gets many of the benefits of a Storage Area Network, at
reasonable prices.
(Snapshots, volume cloning, etc.)
5) Allows me to prototype on t1.micros, which don't support instance
storage, and
bring them up as a different instance type for production.
6) Data persistence in the face of HW failure.
I like you have also observed that an instance-store root volume has
more consistent
performance than a standard EBS volume. However, the introduction
of Provisioned
IOPS [2] EBS volumes has largely changed this for root volumes, as I
can now crank up
the disk performance, but it can get expensive. (Basically you can pay
extra and get an
SLA on your disk performance).
Also do be aware that EBS volumes have a first-access penalty, that
can be addressed
by prewarming your volume. [3]
Bear in mind you can use an EBS root volume, and still take advantage
of instance-
storage for non-root partitions. (For all instance types except
t1.micros, which come with
no instance storage.)
Currently I believe the best deal on AWS instances for smallish fast
storage, might be the m3
class of instances, where one of the ephemeral (instance-store) block
devices is backed by
local SSD. [4] (Whether instance-store or EBS root)
Cheers,
Brian
[1] -
http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/block-device-mapping-concepts.html
[2] -
http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ebs-io-characteristics.html
[3] -
http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ebs-prewarm.html
[4] -
http://aws.typepad.com/aws/2014/01/aws-update-new-m3-features-reduced-ebs-prices-reduced-s3-prices.html
--
-----------------------------------------
Alistair Prestidge
[email protected] <mailto:[email protected]>
