Sharding is a pain in the ass and should be avoided when possible. If its possible, I'd look for another data store that can handle a higher load as a cluster so you don't have to worry about the details of sharding.
On Wed, Feb 26, 2014 at 8:54 AM, masoom alam <[email protected]> wrote: > @Sean: You are right, MYSQL is not configured to handle 1000 events per > second. I will post the results of Batch, which is also slow in our case.I > think we should investigate thoroughly why Batch of for example 1000 is > also slow in our case. > > BTW, How easy it is to configure/implement Shards in MYSQL. Any useful > pointers? > > > On Wed, Feb 26, 2014 at 6:48 PM, Sean Allen > <[email protected]>wrote: > >> Is your mysql set up to handle 1000 writes a second? >> >> I'm going to guess no. If that is the case then Klaus' suggestions are >> good ones. Batch or Shard. >> >> >> On Wed, Feb 26, 2014 at 8:45 AM, masoom alam <[email protected]>wrote: >> >>> 1000 Events per second. >>> >>> >>> >>> >>> On Wed, Feb 26, 2014 at 6:40 PM, Sean Allen <[email protected] >>> > wrote: >>> >>>> How much traffic exactly are you pushing at mysql before the load gets >>>> to high and it starts to buckle under the weight? >>>> >>>> >>>> On Wed, Feb 26, 2014 at 8:38 AM, masoom alam <[email protected]>wrote: >>>> >>>>> Dear All, >>>>> >>>>> Have any body worked on the configurations/optimizations needed >>>>> generally for using STORM with MYSQL. Our scenario stores data in MYSQL >>>>> tables, but as the data rate increases MYSQL starts responding very slow >>>>> (in some cases connection refused error), resulting in DBWriterBolt to >>>>> slowdown. All the Topology is bottleneck by this issue. We cannot increase >>>>> the traffic at source beyond a certain level, the reason we noted is that >>>>> sink (MYSQL) or the bolt adjacent to sink is performing slow. >>>>> >>>>> Any suggestion on how should we proceed will be highly appreciated. >>>>> >>>>> Thanks. >>>>> >>>>> >>>>> >>>> >>>> >>>> -- >>>> >>>> Ce n'est pas une signature >>>> >>> >>> >> >> >> -- >> >> Ce n'est pas une signature >> > > -- Ce n'est pas une signature
