Re: [PERFORM] Testing in AWS, EBS

[Rayson Ho]

Thanks Yves for the clarification!

It used to be very important to pre-warm EBS before running benchmarks
in order to get consistent results.

Then at re:Invent 2015, the AWS engineers said that it is not needed
anymore, which IMO is a lot less work for us to do benchmarking in
AWS, because pre-warming a multi-TB EBS vol is very time consuming,
and the I/Os were not free.

Rayson

==
Open Grid Scheduler - The Official Open Source Grid Engine
http://gridscheduler.sourceforge.net/
http://gridscheduler.sourceforge.net/GridEngine/GridEngineCloud.html


On Thu, May 26, 2016 at 11:41 AM, Yves Dorfsman  wrote:
> On 2016-05-26 09:03, Artem Tomyuk wrote:
>> Why no? Or you missed something?
>
> I think Rayson is correct, but the double negative makes it hard to read:
>
> "So no EBS pre-warming does not apply to EBS volumes created from snapshots."
>
> Which I interpret as:
> So, "no EBS pre-warming", does not apply to EBS volumes created from 
> snapshots.
>
> Which is correct, you sitll have to warm your EBS when created from sanpshots 
> (to get the data from S3 to the filesystem).
>
>
> --
> http://yves.zioup.com
> gpg: 4096R/32B0F416
>
>
>
> --
> Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-performance


-- 
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance

Re: [PERFORM] Testing in AWS, EBS

[Yves Dorfsman]

On 2016-05-26 09:03, Artem Tomyuk wrote:
> Why no? Or you missed something?

I think Rayson is correct, but the double negative makes it hard to read:

"So no EBS pre-warming does not apply to EBS volumes created from snapshots."

Which I interpret as:
So, "no EBS pre-warming", does not apply to EBS volumes created from snapshots.

Which is correct, you sitll have to warm your EBS when created from sanpshots 
(to get the data from S3 to the filesystem).


-- 
http://yves.zioup.com
gpg: 4096R/32B0F416 



-- 
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance

Re: [PERFORM] Testing in AWS, EBS

[Artem Tomyuk]

Why no? Or you missed something?

It should be done on every EBS restored from snapshot.

Is that from your personal experience, and if so, when did you do the test??

Yes we are using this practice, because as a part of our production load we
are using auto scale groups to create new instances, wheech are created
from AMI, wheech stands on snapshots, so...








2016-05-26 17:54 GMT+03:00 Rayson Ho :

> Thanks Artem.
>
> So no EBS pre-warming does not apply to EBS volumes created from snapshots.
>
> Rayson
>
> ==
> Open Grid Scheduler - The Official Open Source Grid Engine
> http://gridscheduler.sourceforge.net/
> http://gridscheduler.sourceforge.net/GridEngine/GridEngineCloud.html
>
>
> On Thu, May 26, 2016 at 10:52 AM, Artem Tomyuk 
> wrote:
> > Please look at the official doc.
> >
> > "New EBS volumes receive their maximum performance the moment that they
> are
> > available and do not require initialization (formerly known as
> pre-warming).
> > However, storage blocks on volumes that were restored from snapshots
> must be
> > initialized (pulled down from Amazon S3 and written to the volume) before
> > you can access the block"
> >
> > Quotation from:
> > http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ebs-initialize.html
> >
> > 2016-05-26 17:47 GMT+03:00 Rayson Ho :
> >>
> >> On Thu, May 26, 2016 at 10:00 AM, Artem Tomyuk 
> >> wrote:
> >>>
> >>>
> >>> 2016-05-26 16:50 GMT+03:00 Rayson Ho :
> 
>  Amazon engineers said that EBS pre-warming is not needed anymore.
> >>>
> >>>
> >>> but still if you will skip this step you wont get much performance on
> ebs
> >>> created from snapshot.
> >>
> >>
> >>
> >> IIRC, that's not what Amazon engineers said. Is that from your personal
> >> experience, and if so, when did you do the test??
> >>
> >> Rayson
> >>
> >> ==
> >> Open Grid Scheduler - The Official Open Source Grid Engine
> >> http://gridscheduler.sourceforge.net/
> >> http://gridscheduler.sourceforge.net/GridEngine/GridEngineCloud.html
> >>
> >>
> >>
> >
>

Re: [PERFORM] Testing in AWS, EBS

[Rayson Ho]

Thanks Artem.

So no EBS pre-warming does not apply to EBS volumes created from snapshots.

Rayson

==
Open Grid Scheduler - The Official Open Source Grid Engine
http://gridscheduler.sourceforge.net/
http://gridscheduler.sourceforge.net/GridEngine/GridEngineCloud.html


On Thu, May 26, 2016 at 10:52 AM, Artem Tomyuk  wrote:
> Please look at the official doc.
>
> "New EBS volumes receive their maximum performance the moment that they are
> available and do not require initialization (formerly known as pre-warming).
> However, storage blocks on volumes that were restored from snapshots must be
> initialized (pulled down from Amazon S3 and written to the volume) before
> you can access the block"
>
> Quotation from:
> http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ebs-initialize.html
>
> 2016-05-26 17:47 GMT+03:00 Rayson Ho :
>>
>> On Thu, May 26, 2016 at 10:00 AM, Artem Tomyuk 
>> wrote:
>>>
>>>
>>> 2016-05-26 16:50 GMT+03:00 Rayson Ho :

 Amazon engineers said that EBS pre-warming is not needed anymore.
>>>
>>>
>>> but still if you will skip this step you wont get much performance on ebs
>>> created from snapshot.
>>
>>
>>
>> IIRC, that's not what Amazon engineers said. Is that from your personal
>> experience, and if so, when did you do the test??
>>
>> Rayson
>>
>> ==
>> Open Grid Scheduler - The Official Open Source Grid Engine
>> http://gridscheduler.sourceforge.net/
>> http://gridscheduler.sourceforge.net/GridEngine/GridEngineCloud.html
>>
>>
>>
>


-- 
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance

Re: [PERFORM] Testing in AWS, EBS

[Artem Tomyuk]

Please look at the official doc.

"New EBS volumes receive their maximum performance the moment that they are
available and do not require initialization (formerly known as
pre-warming). However, storage blocks on volumes that were restored from
snapshots must be initialized (pulled down from Amazon S3 and written to
the volume) before you can access the block"

Quotation from:
http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ebs-initialize.html

2016-05-26 17:47 GMT+03:00 Rayson Ho :

> On Thu, May 26, 2016 at 10:00 AM, Artem Tomyuk 
> wrote:
>
>>
>> 2016-05-26 16:50 GMT+03:00 Rayson Ho :
>>
>>> Amazon engineers said that EBS pre-warming is not needed anymore.
>>
>>
>> but still if you will skip this step you wont get much performance on ebs
>> created from snapshot.
>>
>
>
> IIRC, that's not what Amazon engineers said. Is that from your personal
> experience, and if so, when did you do the test??
>
> Rayson
>
> ==
> Open Grid Scheduler - The Official Open Source Grid Engine
> http://gridscheduler.sourceforge.net/
> http://gridscheduler.sourceforge.net/GridEngine/GridEngineCloud.html
>
>
>
>

Re: [PERFORM] Testing in AWS, EBS

[Rayson Ho]

On Thu, May 26, 2016 at 10:00 AM, Artem Tomyuk  wrote:

>
> 2016-05-26 16:50 GMT+03:00 Rayson Ho :
>
>> Amazon engineers said that EBS pre-warming is not needed anymore.
>
>
> but still if you will skip this step you wont get much performance on ebs
> created from snapshot.
>


IIRC, that's not what Amazon engineers said. Is that from your personal
experience, and if so, when did you do the test??

Rayson

==
Open Grid Scheduler - The Official Open Source Grid Engine
http://gridscheduler.sourceforge.net/
http://gridscheduler.sourceforge.net/GridEngine/GridEngineCloud.html

Re: [PERFORM] Testing in AWS, EBS

[Artem Tomyuk]

2016-05-26 16:50 GMT+03:00 Rayson Ho :

> Amazon engineers said that EBS pre-warming is not needed anymore.


but still if you will skip this step you wont get much performance on ebs
created from snapshot.

Re: [PERFORM] Testing in AWS, EBS

[Rayson Ho]

On Thu, May 26, 2016 at 9:00 AM, Artem Tomyuk  wrote:
>
> But still strong recommendation to pre-warm your ebs in any case,
especially if they created from snapshot.

That used to be true. However, at AWS re:Invent 2015, Amazon engineers said
that EBS pre-warming is not needed anymore.

Rayson

==
Open Grid Scheduler - The Official Open Source Grid Engine
http://gridscheduler.sourceforge.net/
http://gridscheduler.sourceforge.net/GridEngine/GridEngineCloud.html



> 2016-05-26 15:53 GMT+03:00 Yves Dorfsman :
>>
>> On 2016-05-25 19:08, Rayson Ho wrote:
>> > Actually, when "EBS-Optimized" is on, then the instance gets dedicated
>> > bandwidth to EBS.
>>
>> Hadn't realised that, thanks.
>> Is the EBS bandwidth then somewhat limited depending on the type of
instance too?
>>
>> --
>> http://yves.zioup.com
>> gpg: 4096R/32B0F416
>>
>>
>>
>> --
>> Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org
)
>> To make changes to your subscription:
>> http://www.postgresql.org/mailpref/pgsql-performance
>
>

Re: [PERFORM] Testing in AWS, EBS

[Artem Tomyuk]

Yes, the smaller instance you choose - the slower ebs will be.
EBS lives separately from EC2, they are communicating via network. So small
instance = low network bandwidth = poorer disk performance.
But still strong recommendation to pre-warm your ebs in any case,
especially if they created from snapshot.

2016-05-26 15:53 GMT+03:00 Yves Dorfsman :

> On 2016-05-25 19:08, Rayson Ho wrote:
> > Actually, when "EBS-Optimized" is on, then the instance gets dedicated
> > bandwidth to EBS.
>
> Hadn't realised that, thanks.
> Is the EBS bandwidth then somewhat limited depending on the type of
> instance too?
>
> --
> http://yves.zioup.com
> gpg: 4096R/32B0F416
>
>
>
> --
> Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-performance
>

Re: [PERFORM] Testing in AWS, EBS

[Yves Dorfsman]

On 2016-05-25 19:08, Rayson Ho wrote:
> Actually, when "EBS-Optimized" is on, then the instance gets dedicated
> bandwidth to EBS.

Hadn't realised that, thanks.
Is the EBS bandwidth then somewhat limited depending on the type of instance 
too?

-- 
http://yves.zioup.com
gpg: 4096R/32B0F416



-- 
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance

Re: [PERFORM] Testing in AWS, EBS

[Artem Tomyuk]

Hi.

AWS EBS its a really painful story
How was created volumes for RAID? From snapshots?
If you want to get the best performance from EBS it needs to pre-warmed.

Here is the tutorial how to achieve that:
http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ebs-initialize.html

Also you should read this one if you want to get really great for
performance:
http://hatim.eu/2014/05/24/leveraging-ssd-ephemeral-disks-in-ec2-part-1/

Good luck!

2016-05-26 1:34 GMT+03:00 Tory M Blue :

> We are starting some testing in AWS, with EC2, EBS backed setups.
>
> What I found interesting today, was a single EBS 1TB volume, gave me
> something like 108MB/s throughput, however a RAID10 (4 250GB EBS
> volumes), gave me something like 31MB/s (test after test after test).
>
> I'm wondering what you folks are using inside of Amazon (not
> interested in RDS at the moment).
>
> Thanks
> Tory
>
>
> --
> Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-performance
>

Re: [PERFORM] Testing in AWS, EBS

[Rayson Ho]

Actually, when "EBS-Optimized" is on, then the instance gets dedicated
bandwidth to EBS.

Rayson

==
Open Grid Scheduler - The Official Open Source Grid Engine
http://gridscheduler.sourceforge.net/
http://gridscheduler.sourceforge.net/GridEngine/GridEngineCloud.html



On Wed, May 25, 2016 at 7:56 PM, Yves Dorfsman  wrote:

> Indeed, old-style disk EBS vs new-style SSd EBS.
>
> Be aware that EBS traffic is considered as part of the total "network"
> traffic, and each type of instance has different limits on maximum network
> throughput. Those difference are very significant, do tests on the same
> volume
> between two different type of instances, both with enough cpu and memory
> for
> the I/O to be the bottleneck, you will be surprised!
>
>
> On 2016-05-25 17:02, Rayson Ho wrote:
> > There are many factors that can affect EBS performance. For example, the
> type
> > of EBS volume, the instance type, whether EBS-optimized is turned on or
> not, etc.
> >
> > Without the details, then there is no apples to apples comparsion...
> >
> > Rayson
> >
> > ==
> > Open Grid Scheduler - The Official Open Source Grid Engine
> > http://gridscheduler.sourceforge.net/
> > http://gridscheduler.sourceforge.net/GridEngine/GridEngineCloud.html
> >
> >
> >
> > On Wed, May 25, 2016 at 6:34 PM, Tory M Blue  > > wrote:
> >>
> >> We are starting some testing in AWS, with EC2, EBS backed setups.
> >>
> >> What I found interesting today, was a single EBS 1TB volume, gave me
> >> something like 108MB/s throughput, however a RAID10 (4 250GB EBS
> >> volumes), gave me something like 31MB/s (test after test after test).
> >>
> >> I'm wondering what you folks are using inside of Amazon (not
> >> interested in RDS at the moment).
> >>
> >> Thanks
> >> Tory
> >>
> >>
> >> --
> >> Sent via pgsql-performance mailing list (
> pgsql-performance@postgresql.org
> > )
> >> To make changes to your subscription:
> >> http://www.postgresql.org/mailpref/pgsql-performance
>
>
> --
> http://yves.zioup.com
> gpg: 4096R/32B0F416
>
>
>
> --
> Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-performance
>

Re: [PERFORM] Testing in AWS, EBS

[Yves Dorfsman]

Indeed, old-style disk EBS vs new-style SSd EBS.

Be aware that EBS traffic is considered as part of the total "network"
traffic, and each type of instance has different limits on maximum network
throughput. Those difference are very significant, do tests on the same volume
between two different type of instances, both with enough cpu and memory for
the I/O to be the bottleneck, you will be surprised!


On 2016-05-25 17:02, Rayson Ho wrote:
> There are many factors that can affect EBS performance. For example, the type
> of EBS volume, the instance type, whether EBS-optimized is turned on or not, 
> etc.
> 
> Without the details, then there is no apples to apples comparsion...
> 
> Rayson
> 
> ==
> Open Grid Scheduler - The Official Open Source Grid Engine
> http://gridscheduler.sourceforge.net/
> http://gridscheduler.sourceforge.net/GridEngine/GridEngineCloud.html
> 
> 
> 
> On Wed, May 25, 2016 at 6:34 PM, Tory M Blue  > wrote:
>>
>> We are starting some testing in AWS, with EC2, EBS backed setups.
>>
>> What I found interesting today, was a single EBS 1TB volume, gave me
>> something like 108MB/s throughput, however a RAID10 (4 250GB EBS
>> volumes), gave me something like 31MB/s (test after test after test).
>>
>> I'm wondering what you folks are using inside of Amazon (not
>> interested in RDS at the moment).
>>
>> Thanks
>> Tory
>>
>>
>> --
>> Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org
> )
>> To make changes to your subscription:
>> http://www.postgresql.org/mailpref/pgsql-performance


-- 
http://yves.zioup.com
gpg: 4096R/32B0F416



-- 
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance

Re: [PERFORM] Testing in AWS, EBS

[Rayson Ho]

There are many factors that can affect EBS performance. For example, the
type of EBS volume, the instance type, whether EBS-optimized is turned on
or not, etc.

Without the details, then there is no apples to apples comparsion...

Rayson

==
Open Grid Scheduler - The Official Open Source Grid Engine
http://gridscheduler.sourceforge.net/
http://gridscheduler.sourceforge.net/GridEngine/GridEngineCloud.html



On Wed, May 25, 2016 at 6:34 PM, Tory M Blue  wrote:
>
> We are starting some testing in AWS, with EC2, EBS backed setups.
>
> What I found interesting today, was a single EBS 1TB volume, gave me
> something like 108MB/s throughput, however a RAID10 (4 250GB EBS
> volumes), gave me something like 31MB/s (test after test after test).
>
> I'm wondering what you folks are using inside of Amazon (not
> interested in RDS at the moment).
>
> Thanks
> Tory
>
>
> --
> Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-performance