Re: invalid hintfiles after upgrde

2014-01-21 Thread Jan Zahradka - MALL.cz
Hello, Thank You very much. You saved my nerves ;o) One more question. Do I have to wait? Or is it possible to force the Bitcask merge? Jan On 01/18/2014 02:16 AM, Charlie Voiselle wrote: Invalid hint files will be rebuilt automatically during the Bitcask merge process. I believe you are

Re: invalid hintfiles after upgrde

2014-01-21 Thread Jan Zahradka - MALL.cz
Hello everybody, I finally forced the bitcask merge this way. It looks ok ;o) Partitions = [433883298582611803841718934712646521460354973696,1347321821914426127719021955160323408745312813056]. [bitcask:merge(string:concat(/var/lib/riak/bitcask/,P)) || P - Partitions]. Thanks for help Jan

invalid hintfiles after upgrde

2014-01-17 Thread Jan Zahradka - MALL.cz
Hi everybody, I have a 3 node Riak 1.1.1 cluster running on Centos 6.3. I wanted to upgrade it to the most recent version and I was told that first I have to upgrade to version 1.3.2 and then I can upgrade to 1.4.X version. So I followed the instructions in Rolling upgrades guide (stop,

Re: invalid hintfiles after upgrde

2014-01-17 Thread Charlie Voiselle
Invalid hint files will be rebuilt automatically during the Bitcask merge process. I believe you are seeing a one-time consequence of switching to signed hint files during the upgrade. Not to worry, they will be regenerated through normal merge activity. Regards, -cv On Jan 17, 2014 6:56 AM,

Re: Invalid hintfiles

2013-09-29 Thread Toby Corkindale
On 19/09/13 12:02, Seth Thomas wrote: Toby, Although I haven't traced a root cause just yet I've had several reports in IRC of odd behavior when using nginx as a load balancer for Riak CS. In almost every case, removing nginx from the equation and switching to haproxy or pointing directly at

Invalid hintfiles (was: Riak CS 1.4.1 bug? InvalidAccessKeyId reported for deletion calls)

2013-09-18 Thread Toby Corkindale
I found one Riak server was reporting a lot of errors like [error] 0.808.0 Hintfile '/var/lib/riak/bitcask/68507889249886074290797726533575766546371837952/3.bitcask.hint' invalid And the Riak CS logs contained a lot of messages about being unable to retrieve s3 user details because error:

Re: Invalid hintfiles (was: Riak CS 1.4.1 bug? InvalidAccessKeyId reported for deletion calls)

2013-09-18 Thread Luke Bakken
Hi Toby, Invalid hint files won't cause Riak to fail requests - there must have been something else happening. Hint files are used by Riak to speed start time when loading a large key set. You mentioned a load-balancer pool - are you using something like HAProxy to load-balance requests to your

Re: Invalid hintfiles

2013-09-18 Thread Toby Corkindale
On 19/09/13 11:17, Luke Bakken wrote: Hi Toby, Invalid hint files won't cause Riak to fail requests - there must have been something else happening. Hint files are used by Riak to speed start time when loading a large key set. You mentioned a load-balancer pool - are you using something like

Re: Invalid hintfiles

2013-09-18 Thread Seth Thomas
Toby,Although I haven't traced a root cause just yet I've had several reports in IRC of odd behavior when using nginx as a load balancer for Riak CS. In almost every case, removing nginx from the equation and switching to haproxy or pointing directly at the nodes in the application has resolved

Re: Invalid hintfiles

2013-09-18 Thread Toby Corkindale
Hi Andrew, Thanks for that -- so far things are looking stable after making that change. -T On 19/09/13 13:27, Andrew Stone wrote: Hi Toby, Can you try raising the pb_backlog to 128 in your riak app.config on each node. It's likely those disconnect errors are left over from the stampede of

Re: Invalid hintfiles

2013-09-18 Thread Andrew Stone
Good to hear. :) On Wed, Sep 18, 2013 at 11:52 PM, Toby Corkindale toby.corkind...@strategicdata.com.au wrote: Hi Andrew, Thanks for that -- so far things are looking stable after making that change. -T On 19/09/13 13:27, Andrew Stone wrote: Hi Toby, Can you try raising the