RE: hypothetical question about data storage

2013-07-30 Thread Johan De Meersman
Rick James wrote: > >When writing a random block, RAID-5 does not need to touch all the >drives, only the one with parity. Suitable XORs will update it >correctly. So, a write hits 2 drives, whether you have RAID-5 or -10. Only if the other blocks happen to be in the cache, otherwise a read is

Re: hypothetical question about data storage

2013-07-29 Thread Manuel Arostegui
2013/7/30 Rick James > Elevator... If the RAID _controller_ does the Elevator stuff, any OS > optimizations are wasted. > And there have been benchmarks backing that up. (Sorry, don't have any > links handy.) > > RAID 5/10 ... The testing I have done shows very little difference. > However, y

Re: hypothetical question about data storage

2013-07-29 Thread Carsten Pedersen
On 30-07-2013 01:16, Rick James wrote: Elevator... If the RAID _controller_ does the Elevator stuff, any OS optimizations are wasted. And there have been benchmarks backing that up. (Sorry, don't have any links handy.) RAID 5/10 ... The testing I have done shows very little difference. ...r

RE: hypothetical question about data storage

2013-07-29 Thread Rick James
Monday, July 29, 2013 3:38 PM > To: Rick James; will...@techservsys.com; mysql@lists.mysql.com > Subject: RE: hypothetical question about data storage > > Rick James wrote: > > > >For MySQL + RAID, a Linux elevator strategy of 'deadline' or 'noop' is &g

RE: hypothetical question about data storage

2013-07-29 Thread Johan De Meersman
Rick James wrote: > >For MySQL + RAID, a Linux elevator strategy of 'deadline' or 'noop' is >optimal. (The default, 'cfq', is not as good.) I should look into those again at some point. Do you have a brief word as to why they're better? >A RAID controller with multiple drives striped (and opt

RE: hypothetical question about data storage

2013-07-29 Thread Rick James
64TB limit per PARTITION. > -Original Message- > From: william drescher [mailto:will...@techservsys.com] > Sent: Saturday, July 27, 2013 4:32 AM > To: mysql@lists.mysql.com > Subject: Re: hypothetical question about data storage > > On 7/26/2013 6:58 PM, Chris Knipe wrote: >

Re: hypothetical question about data storage

2013-07-27 Thread william drescher
On 7/26/2013 6:58 PM, Chris Knipe wrote: The issue that we have identified is caused by seek time - hundreds of clients simultaneously searching for a single file. The only real way to explain this is to run 100 concurrent instances of bonnie++ doing random read/writes... Your disk utilization a

Re: hypothetical question about data storage

2013-07-26 Thread hsv
2013/07/27 00:58 +0200, Chris Knipe I would definately consider the md5 checksum as a PK (char(32) due to the hex nature), Well, not that it greatly matters, but you could convert it to BINARY(16). -- MySQL General Mailing List For list archives: http://lists.mysql.com/mysql

Re: hypothetical question about data storage

2013-07-26 Thread Chris Knipe
will have seeks. >> >>No, adding more RAM won't help much. Here's an argument: >>Suppose your data is 20 times as big as the buffer pool and you are >>doing random fetches (MD5, etc). Then 1/20 of fetches are cached; 95% >>cache miss. Estimated time: 0.95

RE: hypothetical question about data storage

2013-07-26 Thread Johan De Meersman
1/20 of fetches are cached; 95% >cache miss. Estimated time: 0.95 * 100ms = 95ms. >Now you double your RAM. 1/10 cached -> 90% cache miss -> 90ms average >-> Not much improvement over 95. > >> -Original Message----- >> From: ckn...@savage.za.org [mailto:ck

RE: hypothetical question about data storage

2013-07-26 Thread Rick James
ilto:ckn...@savage.za.org] On Behalf Of > Chris Knipe > Sent: Friday, July 26, 2013 12:30 AM > To: Johan De Meersman > Cc: mysql > Subject: Re: hypothetical question about data storage > > Hi All, > > Thanks for the responces, and I do concur. I was taking a stab in the &

Re: hypothetical question about data storage

2013-07-26 Thread Chris Knipe
Hi All, Thanks for the responces, and I do concur. I was taking a stab in the dark so to speak. We are working with our hosting providers currently and will be introducing a multitude of small iSCSI SANs to split the storage structure over a multitude of disks... This is something that needs t

Re: hypothetical question about data storage

2013-07-26 Thread Johan De Meersman
Hey Chris, I'm afraid that this is not what databases are for, and the first thing you'll likely run into is amount of concurrent connections. This is typically something you should really tackle from a systems perspective. Seek times are dramatically improved on SSD or similar storage - think

Re: hypothetical question about data storage

2013-07-25 Thread Vahric Muhtaryan
Hi, Sorry but mysql is not the address of it , use riak instead of mysql With riak which is key and value based , all keys are on memory and just only one seek enough to handle it Consider to use riak VM On 7/26/13 12:53 AM, "Chris Knipe" wrote: >Hi all, > >We run an VERY io intensive file appli