Re: [Maria-discuss] Best way to scale writes

2016-10-16 Thread Daniel Black 


On 15/10/16 03:33, Jon Foster wrote:
> I have a DB scenario that is very write intensive. Essentially its a
> large scale hit counter of sorts. Currently we're running on a single
> 12core server with 6 SSDs in a RAID6 array.

Not using RAID6 would be a good start.

> But we're looking for a way
> to scale out write volume by adding more servers, hopefully as
> conveniently as I might add Apache servers to a website. We see two
> servers laboring so we add a third to the mix and so on.
> 
> I'm still looking at the various technologies available to me and was
> wondering if someone out there had some suggestions on this front.
> Although Galera Cluster says it scales for write loads too

Where? This is dubious for the reasons below (which are the same as why
raid6 is poor).

> it also says that all servers write the same data

(well - *have* the same data rather than have it written)

> and make sure its committed by the time the transaction is completed.
> So it seems like write performance
> would never scale because all servers are doing the same write.
> 
> Maybe I'm missing something.

I don't think so. You seem to have that worked out.

> Anyhow some pointers would be appreciated.

MyRocks will be interesting to keep an eye on.

Answering Justin's question and details to what component in
hardware/MariaDB is the limiting factor is probably critical before you
go much further. Perhaps even getting professional help.


___
Mailing list: https://launchpad.net/~maria-discuss
Post to : maria-discuss@lists.launchpad.net
Unsubscribe : https://launchpad.net/~maria-discuss
More help   : https://help.launchpad.net/ListHelp

Re: [Maria-discuss] Best way to scale writes

2016-10-14 Thread Jon Foster 




Clint Byrum wrote:

  Excerpts from Jon Foster's message of 2016-10-14 09:33:59 -0700:
  
  
I have a DB scenario that is very write intensive. Essentially its a
large scale hit counter of sorts. Currently we're running on a single
12core server with 6 SSDs in a RAID6 array. But we're looking for a way
to scale out write volume by adding more servers [...]
Anyhow some pointers would be appreciated.


  
  
If you're overwhelming one server (you haven't even begun to scale up
btw) you will have to shard. However, before you do that.. consider how
cheap 24 core servers with 10x FusionIO or similar SSDs would be versus
the cost in complexity of sharding.

Maybe take a look at Vitess, which helps shard things but keeps all the
good stuff you like about MariaDB/MySQL:

http://vitess.io/

___
Mailing list: https://launchpad.net/~maria-discuss
Post to : maria-discuss@lists.launchpad.net
Unsubscribe : https://launchpad.net/~maria-discuss
More help   : https://help.launchpad.net/ListHelp
  

Thanks, Clint, we are moving towards the 24core option too. Just
looking to the future. :-) In fact we're moving the DB onto another box
just to determine if the "semaphore freezes" we're experiencing are
hardware. The newer box is a dual processor box! I'll look at Vitess.

THX - Jon
-- 
Sent from my Debian Linux workstation -- http://www.debian.org/intro/about

Jon Foster
JF Possibilities, Inc.
j...@jfpossibilities.com
541-410-2760
Making computers work for you!




___
Mailing list: https://launchpad.net/~maria-discuss
Post to : maria-discuss@lists.launchpad.net
Unsubscribe : https://launchpad.net/~maria-discuss
More help   : https://help.launchpad.net/ListHelp

Re: [Maria-discuss] Best way to scale writes

2016-10-14 Thread Justin Swanhart 
I asked about schema and queries to determine which sharding framework makes 
the most sense to suggest.

Sent from my iPhone

> On Oct 14, 2016, at 1:05 PM, Clint Byrum  wrote:
> 
> Excerpts from Jon Foster's message of 2016-10-14 09:33:59 -0700:
>> I have a DB scenario that is very write intensive. Essentially its a
>> large scale hit counter of sorts. Currently we're running on a single
>> 12core server with 6 SSDs in a RAID6 array. But we're looking for a way
>> to scale out write volume by adding more servers, hopefully as
>> conveniently as I might add Apache servers to a website. We see two
>> servers laboring so we add a third to the mix and so on.
>> 
>> I'm still looking at the various technologies available to me and was
>> wondering if someone out there had some suggestions on this front.
>> Although Galera Cluster says it scales for write loads too it also says
>> that all servers write the same data and make sure its committed by the
>> time the transaction is completed. So it seems like write performance
>> would never scale because all servers are doing the same write.
>> 
>> Maybe I'm missing something.
>> 
>> Anyhow some pointers would be appreciated.
>> 
> 
> If you're overwhelming one server (you haven't even begun to scale up
> btw) you will have to shard. However, before you do that.. consider how
> cheap 24 core servers with 10x FusionIO or similar SSDs would be versus
> the cost in complexity of sharding.
> 
> Maybe take a look at Vitess, which helps shard things but keeps all the
> good stuff you like about MariaDB/MySQL:
> 
> http://vitess.io/
> 
> ___
> Mailing list: https://launchpad.net/~maria-discuss
> Post to : maria-discuss@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~maria-discuss
> More help   : https://help.launchpad.net/ListHelp

___
Mailing list: https://launchpad.net/~maria-discuss
Post to : maria-discuss@lists.launchpad.net
Unsubscribe : https://launchpad.net/~maria-discuss
More help   : https://help.launchpad.net/ListHelp

Re: [Maria-discuss] Best way to scale writes

2016-10-14 Thread Clint Byrum 
Excerpts from Jon Foster's message of 2016-10-14 09:33:59 -0700:
> I have a DB scenario that is very write intensive. Essentially its a
> large scale hit counter of sorts. Currently we're running on a single
> 12core server with 6 SSDs in a RAID6 array. But we're looking for a way
> to scale out write volume by adding more servers, hopefully as
> conveniently as I might add Apache servers to a website. We see two
> servers laboring so we add a third to the mix and so on.
> 
> I'm still looking at the various technologies available to me and was
> wondering if someone out there had some suggestions on this front.
> Although Galera Cluster says it scales for write loads too it also says
> that all servers write the same data and make sure its committed by the
> time the transaction is completed. So it seems like write performance
> would never scale because all servers are doing the same write.
> 
> Maybe I'm missing something.
> 
> Anyhow some pointers would be appreciated.
> 

If you're overwhelming one server (you haven't even begun to scale up
btw) you will have to shard. However, before you do that.. consider how
cheap 24 core servers with 10x FusionIO or similar SSDs would be versus
the cost in complexity of sharding.

Maybe take a look at Vitess, which helps shard things but keeps all the
good stuff you like about MariaDB/MySQL:

http://vitess.io/

___
Mailing list: https://launchpad.net/~maria-discuss
Post to : maria-discuss@lists.launchpad.net
Unsubscribe : https://launchpad.net/~maria-discuss
More help   : https://help.launchpad.net/ListHelp

Re: [Maria-discuss] Best way to scale writes

2016-10-14 Thread Justin Swanhart 
Can you describe your schema and typical queries please?

Sent from my iPhone

> On Oct 14, 2016, at 12:33 PM, Jon Foster  wrote:
> 
> I have a DB scenario that is very write intensive. Essentially its a
> large scale hit counter of sorts. Currently we're running on a single
> 12core server with 6 SSDs in a RAID6 array. But we're looking for a way
> to scale out write volume by adding more servers, hopefully as
> conveniently as I might add Apache servers to a website. We see two
> servers laboring so we add a third to the mix and so on.
> 
> I'm still looking at the various technologies available to me and was
> wondering if someone out there had some suggestions on this front.
> Although Galera Cluster says it scales for write loads too it also says
> that all servers write the same data and make sure its committed by the
> time the transaction is completed. So it seems like write performance
> would never scale because all servers are doing the same write.
> 
> Maybe I'm missing something.
> 
> Anyhow some pointers would be appreciated.
> 
> TIA - Jon
> 
> -- 
> Sent from my Debian Linux workstation -- http://www.debian.org/intro/about
> 
> Jon Foster
> JF Possibilities, Inc.
> j...@jfpossibilities.com
> 541-410-2760
> Making computers work for you!
> 
> 
> ___
> Mailing list: https://launchpad.net/~maria-discuss
> Post to : maria-discuss@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~maria-discuss
> More help   : https://help.launchpad.net/ListHelp

___
Mailing list: https://launchpad.net/~maria-discuss
Post to : maria-discuss@lists.launchpad.net
Unsubscribe : https://launchpad.net/~maria-discuss
More help   : https://help.launchpad.net/ListHelp

[Maria-discuss] Best way to scale writes

2016-10-14 Thread Jon Foster 
I have a DB scenario that is very write intensive. Essentially its a
large scale hit counter of sorts. Currently we're running on a single
12core server with 6 SSDs in a RAID6 array. But we're looking for a way
to scale out write volume by adding more servers, hopefully as
conveniently as I might add Apache servers to a website. We see two
servers laboring so we add a third to the mix and so on.

I'm still looking at the various technologies available to me and was
wondering if someone out there had some suggestions on this front.
Although Galera Cluster says it scales for write loads too it also says
that all servers write the same data and make sure its committed by the
time the transaction is completed. So it seems like write performance
would never scale because all servers are doing the same write.

Maybe I'm missing something.

Anyhow some pointers would be appreciated.

TIA - Jon

-- 
Sent from my Debian Linux workstation -- http://www.debian.org/intro/about

Jon Foster
JF Possibilities, Inc.
j...@jfpossibilities.com
541-410-2760
Making computers work for you!


___
Mailing list: https://launchpad.net/~maria-discuss
Post to : maria-discuss@lists.launchpad.net
Unsubscribe : https://launchpad.net/~maria-discuss
More help   : https://help.launchpad.net/ListHelp