>
> Do you have any feel
> for performance of the cache warmer? Meaning if a shard server fails do
> you have any metrics on performance differences before the cache warmer vs
> without?
I just quickly drafted a prototype to get your views on this. I am yet to
run unit-tests and surely don't have any stats as of now...
My gut feel is that cache-warmups will take around 1-2 hours for
non-trivial cache-sizes {60-70GB} with 2-4 MB/sec read throttle speeds.
Some apps may find no value addition with this.
May be we can make this optional via blur-config? Users observing no
improvements or even detrimental to existing code can switch this off…
--
Ravi
On Tue, May 19, 2015 at 6:38 PM, Aaron McCurry <[email protected]> wrote:
> On Mon, May 18, 2015 at 7:56 AM, Ravikumar Govindarajan <
> [email protected]> wrote:
>
> > Aaron,
> >
> > I've re-worked the first-cut patch to make it more simpler. If you could
> go
> > through it when you find time, it'll be great…
> >
>
> I just read through the code. I like it! Thanks! Do you have any feel
> for performance of the cache warmer? Meaning if a shard server fails do
> you have any metrics on performance differences before the cache warmer vs
> without? Thanks.
>
> Aaron
>
>
> >
> > --
> > Ravi
> >
> > On Thu, May 7, 2015 at 3:43 PM, Aaron McCurry <[email protected]>
> wrote:
> >
> > > Thanks! Will do!
> > >
> > > On Wednesday, May 6, 2015, Ravikumar Govindarajan <
> > > [email protected]> wrote:
> > >
> > > > Done… https://issues.apache.org/jira/browse/BLUR-431
> > > >
> > > > Please do review when you find time…
> > > >
> > > > --
> > > > Ravi
> > > >
> > > > On Wed, May 6, 2015 at 7:30 AM, Aaron McCurry <[email protected]
> > > > <javascript:;>> wrote:
> > > >
> > > > > Hey Ravi,
> > > > >
> > > > > Attachments are removed from the mail list. Perhaps you can create
> > an
> > > > > issue in jira (https://issues.apache.org/jira/browse/BLUR) and
> > attach
> > > > them
> > > > > there? Thanks!
> > > > >
> > > > > Aaron
> > > > >
> > > > > On Tue, May 5, 2015 at 3:42 AM, Ravikumar Govindarajan <
> > > > > [email protected] <javascript:;>> wrote:
> > > > >
> > > > > > Aaron,
> > > > > >
> > > > > > I am attaching files for your reference, with minor changes to
> > > > > > CacheDirectory. This is on 0.2.2…
> > > > > >
> > > > > > Changes look quite simple to plug-in to CacheDirectory &
> > > > BlurIndexWarmup.
> > > > > > Makes me quite suspicious of missing something obvious…
> > > > > >
> > > > > > Please do look at it when you find time...
> > > > > >
> > > > > > --
> > > > > > Ravi
> > > > > >
> > > > > >
> > > > > >
> > > > > > On Thu, Apr 30, 2015 at 8:25 PM, Aaron McCurry <
> [email protected]
> > > > <javascript:;>>
> > > > > wrote:
> > > > > >
> > > > > >> Cool, let us know how it goes!
> > > > > >>
> > > > > >> Aaron
> > > > > >>
> > > > > >> On Thu, Apr 30, 2015 at 3:07 AM, Ravikumar Govindarajan <
> > > > > >> [email protected] <javascript:;>> wrote:
> > > > > >>
> > > > > >> > >
> > > > > >> > > Just so you are aware we are working to allow Blur to move
> > > blocks
> > > > to
> > > > > >> > > datanodes where the shards are being served
> > > > > >> >
> > > > > >> >
> > > > > >> > Great. I remember that you were telling me this must be the
> > > correct
> > > > > >> > approach for locality sensitive apps on hadoop.
> > > > > >> >
> > > > > >> > I am also trying the cache re-load on 0.2.2 itself.. Will
> update
> > > you
> > > > > >> once
> > > > > >> > it shapes up…
> > > > > >> >
> > > > > >> > --
> > > > > >> > Ravi
> > > > > >> >
> > > > > >>
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
