Re: XFS or EXT4 on Amazon AWS AMIs

[William Bell]

http://edgystuff.tumblr.com/post/81219256714/tips-to-check-and-improve-your-storage-io

Which specifies:  SERVER-13417



You might be right on XFS... We are testing today.

On Thu, Dec 22, 2016 at 1:03 AM, Will Martin  wrote:

> I'd like to see the MongoDB report(?). ext4fs design specifications
> includes support for large files via allocation placement. MongoDB, the
> last time I checked, does pre-allocation which gives it the performance
> benefit of ext4fs multiple design factors (Block and Inode Allocation
> Policy), but the disadvantage of having to rebuild when file lengths are
> being exceeded; at which time the disk fragmentation may prevent ext4fs
> from getting the allocation pattern it was designed for.
>
> That design feature is going to be unavailable with Solr where ext4fs
> dynamic allocation features are less deterministic. Other performance
> factors on ext4fs, and mutexes (even with guard mutexes) are pretty
> standard patterns. The threaded calls sound like the advantages of the
> allocation pattern.
>
> Still those statements, *based on a dated reading of mine*, may be out of
> date with the MongoDB report factors.
>
> "ext4 recognizes (better than ext3, anyway) that data locality is
> generally a desirably quality of a filesystem"
>
> https://ext4.wiki.kernel.org/index.php/Ext4_Disk_Layout#
> Block_and_Inode_Allocation_Policy
>
> For AWS AMI, is there an r4 instance type? The c3 and m3 are superseded
> with *4 types that have notable improvements in IOPs and don't cost more.
>
> http://howto.unixdev.net/Test_LVM_Trim_Ext4.html   -- not an extended
> performance benchmark, but useful to validate discard/TRIM.
>
> On 12/22/2016 1:32 AM, William Bell wrote:
>
> So what are people recommending for SOLR on AWS on Amazon AMI - ext4 or
> xfs?
>
> I saw an article about MongoDB - saying performance on Amazon was better
> due to a mutex issue on ext4 files and threaded calls.
>
> I have been using ext4 for a long time, but I am moving to r3.* instances
> and TRIM / DISCARD support just appears more supported on XFS.
>
>
>
>
>
>
>


-- 
Bill Bell
billnb...@gmail.com
cell 720-256-8076

Re: XFS or EXT4 on Amazon AWS AMIs

[Will Martin]

I'd like to see the MongoDB report(?). ext4fs design specifications includes 
support for large files via allocation placement. MongoDB, the last time I 
checked, does pre-allocation which gives it the performance benefit of ext4fs 
multiple design factors (Block and Inode Allocation Policy), but the 
disadvantage of having to rebuild when file lengths are being exceeded; at 
which time the disk fragmentation may prevent ext4fs from getting the 
allocation pattern it was designed for.

That design feature is going to be unavailable with Solr where ext4fs dynamic 
allocation features are less deterministic. Other performance factors on 
ext4fs, and mutexes (even with guard mutexes) are pretty standard patterns. The 
threaded calls sound like the advantages of the allocation pattern.

Still those statements, *based on a dated reading of mine*, may be out of date 
with the MongoDB report factors.

"ext4 recognizes (better than ext3, anyway) that data locality is generally a 
desirably quality of a filesystem"

https://ext4.wiki.kernel.org/index.php/Ext4_Disk_Layout#Block_and_Inode_Allocation_Policy

For AWS AMI, is there an r4 instance type? The c3 and m3 are superseded with *4 
types that have notable improvements in IOPs and don't cost more.

http://howto.unixdev.net/Test_LVM_Trim_Ext4.html   -- not an extended 
performance benchmark, but useful to validate discard/TRIM.

On 12/22/2016 1:32 AM, William Bell wrote:

So what are people recommending for SOLR on AWS on Amazon AMI - ext4 or xfs?

I saw an article about MongoDB - saying performance on Amazon was better
due to a mutex issue on ext4 files and threaded calls.

I have been using ext4 for a long time, but I am moving to r3.* instances
and TRIM / DISCARD support just appears more supported on XFS.

XFS or EXT4 on Amazon AWS AMIs

[William Bell]

So what are people recommending for SOLR on AWS on Amazon AMI - ext4 or xfs?

I saw an article about MongoDB - saying performance on Amazon was better
due to a mutex issue on ext4 files and threaded calls.

I have been using ext4 for a long time, but I am moving to r3.* instances
and TRIM / DISCARD support just appears more supported on XFS.




-- 
Bill Bell
billnb...@gmail.com
cell 720-256-8076