So I think I have no real chance to get good numbers. I can see a
little bit through the app monitoring but I'm not sure if I can see
real differences about the 100 -> 170 open_files increase.

I will try to change the value on the already migrated nodes as well to
see if this improves the stuff I can see..

Any other ideas?

Cheers
Simon

On Wed, 11 Dec 2013 15:37:03 -0500
Matthew Von-Maszewski <[email protected]> wrote:

> The real Riak developers have suggested this might be your problem with stats 
> being stuck:
> 
> https://github.com/basho/riak_core/pull/467
> 
> The fix is included in the upcoming 1.4.4 maintenance release (which is 
> overdue so I am not going to bother guessing when it will actually arrive).
> 
> Matthew
> 
> On Dec 11, 2013, at 2:47 PM, Simon Effenberg <[email protected]> 
> wrote:
> 
> > I will do..
> > 
> > but one other thing:
> > 
> > watch Every 10.0s: sudo riak-admin status | grep put_fsm
> > node_put_fsm_time_mean : 2208050
> > node_put_fsm_time_median : 39231
> > node_put_fsm_time_95 : 17400382
> > node_put_fsm_time_99 : 50965752
> > node_put_fsm_time_100 : 59537762
> > node_put_fsm_active : 5
> > node_put_fsm_active_60s : 364
> > node_put_fsm_in_rate : 5
> > node_put_fsm_out_rate : 3
> > node_put_fsm_rejected : 0
> > node_put_fsm_rejected_60s : 0
> > node_put_fsm_rejected_total : 0
> > 
> > this is not changing at all.. so maybe my expectations are _wrong_?! So
> > I will start searching around if there is a "status" bug or I'm
> > looking in the wrong place... maybe there is no problem while searching
> > for one?! But I see that at least the app has some issues on GET and
> > PUT (more on PUT).. so I would like to know how fast the things are..
> > but "status" isn't working.. aaaaargh...
> > 
> > Cheers
> > Simon
> > 
> > 
> > On Wed, 11 Dec 2013 14:32:07 -0500
> > Matthew Von-Maszewski <[email protected]> wrote:
> > 
> >> An additional thought:  if increasing max_open_files does NOT help, try 
> >> removing +S 4:4 from the vm.args.  Typically +S setting helps leveldb, but 
> >> one other user mentioned that the new sorted 2i queries needed more CPU in 
> >> the Erlang layer.
> >> 
> >> Summary:
> >> - try increasing max_open_files to 170
> >>  - helps:  try setting sst_block_size to 32768 in app.config
> >>  - does not help:  try removing +S from vm.args
> >> 
> >> Matthew
> >> 
> >> On Dec 11, 2013, at 1:58 PM, Simon Effenberg <[email protected]> 
> >> wrote:
> >> 
> >>> Hi Matthew,
> >>> 
> >>> On Wed, 11 Dec 2013 18:38:49 +0100
> >>> Matthew Von-Maszewski <[email protected]> wrote:
> >>> 
> >>>> Simon,
> >>>> 
> >>>> I have plugged your various values into the attached spreadsheet.  I 
> >>>> assumed a vnode count to allow for one of your twelve servers to die 
> >>>> (256 ring size / 11 servers).
> >>> 
> >>> Great, thanks!
> >>> 
> >>>> 
> >>>> The spreadsheet suggests you can safely raise your max_open_files from 
> >>>> 100 to 170.  I would suggest doing this for the next server you upgrade. 
> >>>>  If part of your problem is file cache thrashing, you should see an 
> >>>> improvement.
> >>> 
> >>> I will try this out.. starting the next server in 3-4 hours.
> >>> 
> >>>> 
> >>>> Only if max_open_files helps, you should then consider adding 
> >>>> {sst_block_size, 32767} to the eleveldb portion of app.config.  This 
> >>>> setting, given your value sizes, would likely half the size of the 
> >>>> metadata held in the file cache.  It only impacts the files newly 
> >>>> compacted in the upgrade, and would gradually increase space in the file 
> >>>> cache while slowing down the file cache thrashing.
> >>> 
> >>> So I'll do this at the over-next server if the next server is fine.
> >>> 
> >>>> 
> >>>> What build/packaging of Riak do you use, or do you build from source?
> >>> 
> >>> Using the debian packages from the basho site..
> >>> 
> >>> I'm really wondering why the "put" performance is that bad.
> >>> Here are the changes which were introduced/changed only on the new
> >>> upgraded servers:
> >>> 
> >>> 
> >>> +        fsm_limit                 => 50000,
> >>> --- our '+P' is set to 262144 so more than 3x fsm_limit which was
> >>> --- stated somewhere
> >>> +        # after finishing the upgrade this should be switched to v1 !!!
> >>> +        object_format             => '__atom_v0',
> >>> 
> >>> -      '-env ERL_MAX_ETS_TABLES' => 8192,
> >>> +      '-env ERL_MAX_ETS_TABLES'  => 256000, # old package used 8192
> >>> but 1.4.2 raised it to this high number
> >>> +      '-env ERL_MAX_PORTS'       => 64000,
> >>> +      # Treat error_logger warnings as warnings
> >>> +      '+W'                       => 'w',
> >>> +      # Tweak GC to run more often
> >>> +      '-env ERL_FULLSWEEP_AFTER' => 0,
> >>> +      # Force the erlang VM to use SMP
> >>> +      '-smp'                     => 'enable',
> >>> +      #################################
> >>> 
> >>> Cheers
> >>> Simon
> >>> 
> >>> 
> >>>> 
> >>>> Matthew
> >>>> 
> >>>> 
> >>>> 
> >>>> On Dec 11, 2013, at 9:48 AM, Simon Effenberg <[email protected]> 
> >>>> wrote:
> >>>> 
> >>>>> Hi Matthew,
> >>>>> 
> >>>>> thanks for all your time and work.. see inline for answers..
> >>>>> 
> >>>>> On Wed, 11 Dec 2013 09:17:32 -0500
> >>>>> Matthew Von-Maszewski <[email protected]> wrote:
> >>>>> 
> >>>>>> The real Riak developers have arrived on-line for the day.  They are 
> >>>>>> telling me that all of your problems are likely due to the extended 
> >>>>>> upgrade times, and yes there is a known issue with handoff between 1.3 
> >>>>>> and 1.4.  They also say everything should calm down after all nodes 
> >>>>>> are upgraded.
> >>>>>> 
> >>>>>> I will review your system settings now and see if there is something 
> >>>>>> that might make the other machines upgrade quicker.  So three more 
> >>>>>> questions:
> >>>>>> 
> >>>>>> - what is the average size of your keys
> >>>>> 
> >>>>> bucket names are between 5 and 15 characters (only ~ 10 buckets)..
> >>>>> key names are normally something like 26iesj:hovh7egz
> >>>>> 
> >>>>>> 
> >>>>>> - what is the average size of your value (data stored)
> >>>>> 
> >>>>> I have to guess.. but mean is (from Riak) 12kb but 95th percentile is
> >>>>> at 75kb and in theory we have a limit of 1MB (then it will be split up)
> >>>>> but sometimes thanks to sibblings (we have to buckets with allow_mult)
> >>>>> we have also some 7MB in MAX but this will be reduced again (it's a new
> >>>>> feature in our app which has to many parallel wrights below of 15ms).
> >>>>> 
> >>>>>> 
> >>>>>> - in regular use, are your keys accessed randomly across their entire 
> >>>>>> range, or do they contain a date component which clusters older, less 
> >>>>>> used keys
> >>>>> 
> >>>>> normally we don't search but retrieve keys by key name.. and we have
> >>>>> data which is up to 6 months old and normally we access mostly
> >>>>> new/active/hot data and not all the old ones.. besides this we have a
> >>>>> job doing a 2i query every 5mins and another one doing this maybe once
> >>>>> an hour.. both don't work while the upgrade is ongoing (2i isn't
> >>>>> working).
> >>>>> 
> >>>>> Cheers
> >>>>> Simon
> >>>>> 
> >>>>>> 
> >>>>>> Matthew
> >>>>>> 
> >>>>>> 
> >>>>>> On Dec 11, 2013, at 8:43 AM, Simon Effenberg 
> >>>>>> <[email protected]> wrote:
> >>>>>> 
> >>>>>>> Oh and at the moment they are waiting for some handoffs and I see
> >>>>>>> errors in logfiles:
> >>>>>>> 
> >>>>>>> 
> >>>>>>> 2013-12-11 13:41:47.948 UTC [error]
> >>>>>>> <0.7157.24>@riak_core_handoff_sender:start_fold:269 hinted_handoff
> >>>>>>> transfer of riak_kv_vnode from '[email protected]'
> >>>>>>> 468137243207554840987117797979434404733540892672
> >>>>>>> 
> >>>>>>> but I remember that somebody else had this as well and if I recall
> >>>>>>> correctly it disappeared after the full upgrade was done.. but at the
> >>>>>>> moment it's hard to think about upgrading everything at once..
> >>>>>>> (~12hours 100% disk utilization on all 12 nodes will lead to real slow
> >>>>>>> puts/gets)
> >>>>>>> 
> >>>>>>> What can I do?
> >>>>>>> 
> >>>>>>> Cheers
> >>>>>>> Simon
> >>>>>>> 
> >>>>>>> PS: transfers output:
> >>>>>>> 
> >>>>>>> '[email protected]' waiting to handoff 17 partitions
> >>>>>>> '[email protected]' waiting to handoff 19 partitions
> >>>>>>> 
> >>>>>>> (these are the 1.4.2 nodes)
> >>>>>>> 
> >>>>>>> 
> >>>>>>> On Wed, 11 Dec 2013 14:39:58 +0100
> >>>>>>> Simon Effenberg <[email protected]> wrote:
> >>>>>>> 
> >>>>>>>> Also some side notes:
> >>>>>>>> 
> >>>>>>>> "top" is even better on new 1.4.2 than on 1.3.1 machines.. IO
> >>>>>>>> utilization of disk is mostly the same (round about 33%)..
> >>>>>>>> 
> >>>>>>>> but
> >>>>>>>> 
> >>>>>>>> 95th percentile of response time for get (avg over all nodes):
> >>>>>>>> before upgrade: 29ms
> >>>>>>>> after upgrade: almost the same
> >>>>>>>> 
> >>>>>>>> 95th percentile of response time for put (avg over all nodes):
> >>>>>>>> before upgrade: 60ms
> >>>>>>>> after upgrade: 1548ms
> >>>>>>>> but this is only because of 2 of 12 nodes are
> >>>>>>>> on 1.4.2 and are really slow (17000ms)
> >>>>>>>> 
> >>>>>>>> Cheers,
> >>>>>>>> Simon
> >>>>>>>> 
> >>>>>>>> On Wed, 11 Dec 2013 13:45:56 +0100
> >>>>>>>> Simon Effenberg <[email protected]> wrote:
> >>>>>>>> 
> >>>>>>>>> Sorry I forgot the half of it..
> >>>>>>>>> 
> >>>>>>>>> seffenberg@kriak46-1:~$ free -m
> >>>>>>>>>          total       used       free     shared    buffers cached
> >>>>>>>>> Mem:         23999      23759        239          0        184      
> >>>>>>>>> 16183
> >>>>>>>>> -/+ buffers/cache:       7391      16607
> >>>>>>>>> Swap:            0          0          0
> >>>>>>>>> 
> >>>>>>>>> We have 12 servers..
> >>>>>>>>> datadir on the compacted servers (1.4.2) ~ 765 GB
> >>>>>>>>> 
> >>>>>>>>> AAE is enabled.
> >>>>>>>>> 
> >>>>>>>>> I attached app.config and vm.args.
> >>>>>>>>> 
> >>>>>>>>> Cheers
> >>>>>>>>> Simon
> >>>>>>>>> 
> >>>>>>>>> On Wed, 11 Dec 2013 07:33:31 -0500
> >>>>>>>>> Matthew Von-Maszewski <[email protected]> wrote:
> >>>>>>>>> 
> >>>>>>>>>> Ok, I am now suspecting that your servers are either using swap 
> >>>>>>>>>> space (which is slow) or your leveldb file cache is thrashing 
> >>>>>>>>>> (opening and closing multiple files per request).
> >>>>>>>>>> 
> >>>>>>>>>> How many servers do you have and do you use Riak's active 
> >>>>>>>>>> anti-entropy feature?  I am going to plug all of this into a 
> >>>>>>>>>> spreadsheet.
> >>>>>>>>>> 
> >>>>>>>>>> Matthew Von-Maszewski
> >>>>>>>>>> 
> >>>>>>>>>> 
> >>>>>>>>>> On Dec 11, 2013, at 7:09, Simon Effenberg 
> >>>>>>>>>> <[email protected]> wrote:
> >>>>>>>>>> 
> >>>>>>>>>>> Hi Matthew
> >>>>>>>>>>> 
> >>>>>>>>>>> Memory: 23999 MB
> >>>>>>>>>>> 
> >>>>>>>>>>> ring_creation_size, 256
> >>>>>>>>>>> max_open_files, 100
> >>>>>>>>>>> 
> >>>>>>>>>>> riak-admin status:
> >>>>>>>>>>> 
> >>>>>>>>>>> memory_total : 276001360
> >>>>>>>>>>> memory_processes : 191506322
> >>>>>>>>>>> memory_processes_used : 191439568
> >>>>>>>>>>> memory_system : 84495038
> >>>>>>>>>>> memory_atom : 686993
> >>>>>>>>>>> memory_atom_used : 686560
> >>>>>>>>>>> memory_binary : 21965352
> >>>>>>>>>>> memory_code : 11332732
> >>>>>>>>>>> memory_ets : 10823528
> >>>>>>>>>>> 
> >>>>>>>>>>> Thanks for looking!
> >>>>>>>>>>> 
> >>>>>>>>>>> Cheers
> >>>>>>>>>>> Simon
> >>>>>>>>>>> 
> >>>>>>>>>>> 
> >>>>>>>>>>> 
> >>>>>>>>>>> On Wed, 11 Dec 2013 06:44:42 -0500
> >>>>>>>>>>> Matthew Von-Maszewski <[email protected]> wrote:
> >>>>>>>>>>> 
> >>>>>>>>>>>> I need to ask other developers as they arrive for the new day.  
> >>>>>>>>>>>> Does not make sense to me.
> >>>>>>>>>>>> 
> >>>>>>>>>>>> How many nodes do you have?  How much RAM do you have in each 
> >>>>>>>>>>>> node?  What are your settings for max_open_files and cache_size 
> >>>>>>>>>>>> in the app.config file?  Maybe this is as simple as leveldb 
> >>>>>>>>>>>> using too much RAM in 1.4.  The memory accounting for 
> >>>>>>>>>>>> maz_open_files changed in 1.4.
> >>>>>>>>>>>> 
> >>>>>>>>>>>> Matthew Von-Maszewski
> >>>>>>>>>>>> 
> >>>>>>>>>>>> 
> >>>>>>>>>>>> On Dec 11, 2013, at 6:28, Simon Effenberg 
> >>>>>>>>>>>> <[email protected]> wrote:
> >>>>>>>>>>>> 
> >>>>>>>>>>>>> Hi Matthew,
> >>>>>>>>>>>>> 
> >>>>>>>>>>>>> it took around 11hours for the first node to finish the 
> >>>>>>>>>>>>> compaction. The
> >>>>>>>>>>>>> second node is running already 12 hours and is still doing 
> >>>>>>>>>>>>> compaction.
> >>>>>>>>>>>>> 
> >>>>>>>>>>>>> Besides that I wonder because the fsm_put time on the new 1.4.2 
> >>>>>>>>>>>>> host is
> >>>>>>>>>>>>> much higher (after the compaction) than on an old 1.3.1 (both 
> >>>>>>>>>>>>> are
> >>>>>>>>>>>>> running in the cluster right now and another one is doing the
> >>>>>>>>>>>>> compaction/upgrade while it is in the cluster but not directly
> >>>>>>>>>>>>> accessible because it is out of the Loadbalancer):
> >>>>>>>>>>>>> 
> >>>>>>>>>>>>> 1.4.2:
> >>>>>>>>>>>>> 
> >>>>>>>>>>>>> node_put_fsm_time_mean : 2208050
> >>>>>>>>>>>>> node_put_fsm_time_median : 39231
> >>>>>>>>>>>>> node_put_fsm_time_95 : 17400382
> >>>>>>>>>>>>> node_put_fsm_time_99 : 50965752
> >>>>>>>>>>>>> node_put_fsm_time_100 : 59537762
> >>>>>>>>>>>>> node_put_fsm_active : 5
> >>>>>>>>>>>>> node_put_fsm_active_60s : 364
> >>>>>>>>>>>>> node_put_fsm_in_rate : 5
> >>>>>>>>>>>>> node_put_fsm_out_rate : 3
> >>>>>>>>>>>>> node_put_fsm_rejected : 0
> >>>>>>>>>>>>> node_put_fsm_rejected_60s : 0
> >>>>>>>>>>>>> node_put_fsm_rejected_total : 0
> >>>>>>>>>>>>> 
> >>>>>>>>>>>>> 
> >>>>>>>>>>>>> 1.3.1:
> >>>>>>>>>>>>> 
> >>>>>>>>>>>>> node_put_fsm_time_mean : 5036
> >>>>>>>>>>>>> node_put_fsm_time_median : 1614
> >>>>>>>>>>>>> node_put_fsm_time_95 : 8789
> >>>>>>>>>>>>> node_put_fsm_time_99 : 38258
> >>>>>>>>>>>>> node_put_fsm_time_100 : 384372
> >>>>>>>>>>>>> 
> >>>>>>>>>>>>> 
> >>>>>>>>>>>>> any clue why this could/should be?
> >>>>>>>>>>>>> 
> >>>>>>>>>>>>> Cheers
> >>>>>>>>>>>>> Simon
> >>>>>>>>>>>>> 
> >>>>>>>>>>>>> On Tue, 10 Dec 2013 17:21:07 +0100
> >>>>>>>>>>>>> Simon Effenberg <[email protected]> wrote:
> >>>>>>>>>>>>> 
> >>>>>>>>>>>>>> Hi Matthew,
> >>>>>>>>>>>>>> 
> >>>>>>>>>>>>>> thanks!.. that answers my questions!
> >>>>>>>>>>>>>> 
> >>>>>>>>>>>>>> Cheers
> >>>>>>>>>>>>>> Simon
> >>>>>>>>>>>>>> 
> >>>>>>>>>>>>>> On Tue, 10 Dec 2013 11:08:32 -0500
> >>>>>>>>>>>>>> Matthew Von-Maszewski <[email protected]> wrote:
> >>>>>>>>>>>>>> 
> >>>>>>>>>>>>>>> 2i is not my expertise, so I had to discuss you concerns with 
> >>>>>>>>>>>>>>> another Basho developer.  He says:
> >>>>>>>>>>>>>>> 
> >>>>>>>>>>>>>>> Between 1.3 and 1.4, the 2i query did change but not the 2i 
> >>>>>>>>>>>>>>> on-disk format.  You must wait for all nodes to update if you 
> >>>>>>>>>>>>>>> desire to use the new 2i query.  The 2i data will properly 
> >>>>>>>>>>>>>>> write/update on both 1.3 and 1.4 machines during the 
> >>>>>>>>>>>>>>> migration.
> >>>>>>>>>>>>>>> 
> >>>>>>>>>>>>>>> Does that answer your question?
> >>>>>>>>>>>>>>> 
> >>>>>>>>>>>>>>> 
> >>>>>>>>>>>>>>> And yes, you might see available disk space increase during 
> >>>>>>>>>>>>>>> the upgrade compactions if your dataset contains numerous 
> >>>>>>>>>>>>>>> delete "tombstones".  The Riak 2.0 code includes a new 
> >>>>>>>>>>>>>>> feature called "aggressive delete" for leveldb.  This feature 
> >>>>>>>>>>>>>>> is more proactive in pushing delete tombstones through the 
> >>>>>>>>>>>>>>> levels to free up disk space much more quickly (especially if 
> >>>>>>>>>>>>>>> you perform block deletes every now and then).
> >>>>>>>>>>>>>>> 
> >>>>>>>>>>>>>>> Matthew
> >>>>>>>>>>>>>>> 
> >>>>>>>>>>>>>>> 
> >>>>>>>>>>>>>>> On Dec 10, 2013, at 10:44 AM, Simon Effenberg 
> >>>>>>>>>>>>>>> <[email protected]> wrote:
> >>>>>>>>>>>>>>> 
> >>>>>>>>>>>>>>>> Hi Matthew,
> >>>>>>>>>>>>>>>> 
> >>>>>>>>>>>>>>>> see inline..
> >>>>>>>>>>>>>>>> 
> >>>>>>>>>>>>>>>> On Tue, 10 Dec 2013 10:38:03 -0500
> >>>>>>>>>>>>>>>> Matthew Von-Maszewski <[email protected]> wrote:
> >>>>>>>>>>>>>>>> 
> >>>>>>>>>>>>>>>>> The sad truth is that you are not the first to see this 
> >>>>>>>>>>>>>>>>> problem.  And yes, it has to do with your 950GB per node 
> >>>>>>>>>>>>>>>>> dataset.  And no, nothing to do but sit through it at this 
> >>>>>>>>>>>>>>>>> time.
> >>>>>>>>>>>>>>>>> 
> >>>>>>>>>>>>>>>>> While I did extensive testing around upgrade times before 
> >>>>>>>>>>>>>>>>> shipping 1.4, apparently there are data configurations I 
> >>>>>>>>>>>>>>>>> did not anticipate.  You are likely seeing a cascade where 
> >>>>>>>>>>>>>>>>> a shift of one file from level-1 to level-2 is causing a 
> >>>>>>>>>>>>>>>>> shift of another file from level-2 to level-3, which causes 
> >>>>>>>>>>>>>>>>> a level-3 file to shift to level-4, etc … then the next 
> >>>>>>>>>>>>>>>>> file shifts from level-1.
> >>>>>>>>>>>>>>>>> 
> >>>>>>>>>>>>>>>>> The bright side of this pain is that you will end up with 
> >>>>>>>>>>>>>>>>> better write throughput once all the compaction ends.
> >>>>>>>>>>>>>>>> 
> >>>>>>>>>>>>>>>> I have to deal with that.. but my problem is now, if I'm 
> >>>>>>>>>>>>>>>> doing this
> >>>>>>>>>>>>>>>> node by node it looks like 2i searches aren't possible while 
> >>>>>>>>>>>>>>>> 1.3 and
> >>>>>>>>>>>>>>>> 1.4 nodes exists in the cluster. Is there any problem which 
> >>>>>>>>>>>>>>>> leads me to
> >>>>>>>>>>>>>>>> an 2i repair marathon or could I easily wait for some hours 
> >>>>>>>>>>>>>>>> for each
> >>>>>>>>>>>>>>>> node until all merges are done before I upgrade the next 
> >>>>>>>>>>>>>>>> one? (2i
> >>>>>>>>>>>>>>>> searches can fail for some time.. the APP isn't having 
> >>>>>>>>>>>>>>>> problems with
> >>>>>>>>>>>>>>>> that but are new inserts with 2i indices processed 
> >>>>>>>>>>>>>>>> successfully or do
> >>>>>>>>>>>>>>>> I have to do the 2i repair?)
> >>>>>>>>>>>>>>>> 
> >>>>>>>>>>>>>>>> /s
> >>>>>>>>>>>>>>>> 
> >>>>>>>>>>>>>>>> one other good think: saving disk space is one advantage ;)..
> >>>>>>>>>>>>>>>> 
> >>>>>>>>>>>>>>>> 
> >>>>>>>>>>>>>>>>> 
> >>>>>>>>>>>>>>>>> Riak 2.0's leveldb has code to prevent/reduce compaction 
> >>>>>>>>>>>>>>>>> cascades, but that is not going to help you today.
> >>>>>>>>>>>>>>>>> 
> >>>>>>>>>>>>>>>>> Matthew
> >>>>>>>>>>>>>>>>> 
> >>>>>>>>>>>>>>>>> On Dec 10, 2013, at 10:26 AM, Simon Effenberg 
> >>>>>>>>>>>>>>>>> <[email protected]> wrote:
> >>>>>>>>>>>>>>>>> 
> >>>>>>>>>>>>>>>>>> Hi @list,
> >>>>>>>>>>>>>>>>>> 
> >>>>>>>>>>>>>>>>>> I'm trying to upgrade our Riak cluster from 1.3.1 to 1.4.2 
> >>>>>>>>>>>>>>>>>> .. after
> >>>>>>>>>>>>>>>>>> upgrading the first node (out of 12) this node seems to do 
> >>>>>>>>>>>>>>>>>> many merges.
> >>>>>>>>>>>>>>>>>> the sst_* directories changes in size "rapidly" and the 
> >>>>>>>>>>>>>>>>>> node is having
> >>>>>>>>>>>>>>>>>> a disk utilization of 100% all the time.
> >>>>>>>>>>>>>>>>>> 
> >>>>>>>>>>>>>>>>>> I know that there is something like that:
> >>>>>>>>>>>>>>>>>> 
> >>>>>>>>>>>>>>>>>> "The first execution of 1.4.0 leveldb using a 1.3.x or 
> >>>>>>>>>>>>>>>>>> 1.2.x dataset
> >>>>>>>>>>>>>>>>>> will initiate an automatic conversion that could pause the 
> >>>>>>>>>>>>>>>>>> startup of
> >>>>>>>>>>>>>>>>>> each node by 3 to 7 minutes. The leveldb data in "level 
> >>>>>>>>>>>>>>>>>> #1" is being
> >>>>>>>>>>>>>>>>>> adjusted such that "level #1" can operate as an overlapped 
> >>>>>>>>>>>>>>>>>> data level
> >>>>>>>>>>>>>>>>>> instead of as a sorted data level. The conversion is 
> >>>>>>>>>>>>>>>>>> simply the
> >>>>>>>>>>>>>>>>>> reduction of the number of files in "level #1" to being 
> >>>>>>>>>>>>>>>>>> less than eight
> >>>>>>>>>>>>>>>>>> via normal compaction of data from "level #1" into "level 
> >>>>>>>>>>>>>>>>>> #2". This is
> >>>>>>>>>>>>>>>>>> a one time conversion."
> >>>>>>>>>>>>>>>>>> 
> >>>>>>>>>>>>>>>>>> but it looks much more invasive than explained here or 
> >>>>>>>>>>>>>>>>>> doesn't have to
> >>>>>>>>>>>>>>>>>> do anything with the (probably seen) merges.
> >>>>>>>>>>>>>>>>>> 
> >>>>>>>>>>>>>>>>>> Is this "normal" behavior or could I do anything about it?
> >>>>>>>>>>>>>>>>>> 
> >>>>>>>>>>>>>>>>>> At the moment I'm stucked with the upgrade procedure 
> >>>>>>>>>>>>>>>>>> because this high
> >>>>>>>>>>>>>>>>>> IO load would probably lead to high response times.
> >>>>>>>>>>>>>>>>>> 
> >>>>>>>>>>>>>>>>>> Also we have a lot of data (per node ~950 GB).
> >>>>>>>>>>>>>>>>>> 
> >>>>>>>>>>>>>>>>>> Cheers
> >>>>>>>>>>>>>>>>>> Simon
> >>>>>>>>>>>>>>>>>> 
> >>>>>>>>>>>>>>>>>> _______________________________________________
> >>>>>>>>>>>>>>>>>> riak-users mailing list
> >>>>>>>>>>>>>>>>>> [email protected]
> >>>>>>>>>>>>>>>>>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
> >>>>>>>>>>>>>>>>> 
> >>>>>>>>>>>>>>>> 
> >>>>>>>>>>>>>>>> 
> >>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>> Simon Effenberg | Site Ops Engineer | mobile.international 
> >>>>>>>>>>>>>>>> GmbH
> >>>>>>>>>>>>>>>> Fon:     + 49-(0)30-8109 - 7173
> >>>>>>>>>>>>>>>> Fax:     + 49-(0)30-8109 - 7131
> >>>>>>>>>>>>>>>> 
> >>>>>>>>>>>>>>>> Mail:     [email protected]
> >>>>>>>>>>>>>>>> Web:    www.mobile.de
> >>>>>>>>>>>>>>>> 
> >>>>>>>>>>>>>>>> Marktplatz 1 | 14532 Europarc Dreilinden | Germany
> >>>>>>>>>>>>>>>> 
> >>>>>>>>>>>>>>>> 
> >>>>>>>>>>>>>>>> Geschäftsführer: Malte Krüger
> >>>>>>>>>>>>>>>> HRB Nr.: 18517 P, Amtsgericht Potsdam
> >>>>>>>>>>>>>>>> Sitz der Gesellschaft: Kleinmachnow
> >>>>>>>>>>>>>>> 
> >>>>>>>>>>>>>> 
> >>>>>>>>>>>>>> 
> >>>>>>>>>>>>>> --
> >>>>>>>>>>>>>> Simon Effenberg | Site Ops Engineer | mobile.international GmbH
> >>>>>>>>>>>>>> Fon:     + 49-(0)30-8109 - 7173
> >>>>>>>>>>>>>> Fax:     + 49-(0)30-8109 - 7131
> >>>>>>>>>>>>>> 
> >>>>>>>>>>>>>> Mail:     [email protected]
> >>>>>>>>>>>>>> Web:    www.mobile.de
> >>>>>>>>>>>>>> 
> >>>>>>>>>>>>>> Marktplatz 1 | 14532 Europarc Dreilinden | Germany
> >>>>>>>>>>>>>> 
> >>>>>>>>>>>>>> 
> >>>>>>>>>>>>>> Geschäftsführer: Malte Krüger
> >>>>>>>>>>>>>> HRB Nr.: 18517 P, Amtsgericht Potsdam
> >>>>>>>>>>>>>> Sitz der Gesellschaft: Kleinmachnow
> >>>>>>>>>>>>>> 
> >>>>>>>>>>>>>> _______________________________________________
> >>>>>>>>>>>>>> riak-users mailing list
> >>>>>>>>>>>>>> [email protected]
> >>>>>>>>>>>>>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
> >>>>>>>>>>>>> 
> >>>>>>>>>>>>> 
> >>>>>>>>>>>>> --
> >>>>>>>>>>>>> Simon Effenberg | Site Ops Engineer | mobile.international GmbH
> >>>>>>>>>>>>> Fon:     + 49-(0)30-8109 - 7173
> >>>>>>>>>>>>> Fax:     + 49-(0)30-8109 - 7131
> >>>>>>>>>>>>> 
> >>>>>>>>>>>>> Mail:     [email protected]
> >>>>>>>>>>>>> Web:    www.mobile.de
> >>>>>>>>>>>>> 
> >>>>>>>>>>>>> Marktplatz 1 | 14532 Europarc Dreilinden | Germany
> >>>>>>>>>>>>> 
> >>>>>>>>>>>>> 
> >>>>>>>>>>>>> Geschäftsführer: Malte Krüger
> >>>>>>>>>>>>> HRB Nr.: 18517 P, Amtsgericht Potsdam
> >>>>>>>>>>>>> Sitz der Gesellschaft: Kleinmachnow
> >>>>>>>>>>> 
> >>>>>>>>>>> 
> >>>>>>>>>>> --
> >>>>>>>>>>> Simon Effenberg | Site Ops Engineer | mobile.international GmbH
> >>>>>>>>>>> Fon:     + 49-(0)30-8109 - 7173
> >>>>>>>>>>> Fax:     + 49-(0)30-8109 - 7131
> >>>>>>>>>>> 
> >>>>>>>>>>> Mail:     [email protected]
> >>>>>>>>>>> Web:    www.mobile.de
> >>>>>>>>>>> 
> >>>>>>>>>>> Marktplatz 1 | 14532 Europarc Dreilinden | Germany
> >>>>>>>>>>> 
> >>>>>>>>>>> 
> >>>>>>>>>>> Geschäftsführer: Malte Krüger
> >>>>>>>>>>> HRB Nr.: 18517 P, Amtsgericht Potsdam
> >>>>>>>>>>> Sitz der Gesellschaft: Kleinmachnow
> >>>>>>>>> 
> >>>>>>>>> 
> >>>>>>>>> --
> >>>>>>>>> Simon Effenberg | Site Ops Engineer | mobile.international GmbH
> >>>>>>>>> Fon:     + 49-(0)30-8109 - 7173
> >>>>>>>>> Fax:     + 49-(0)30-8109 - 7131
> >>>>>>>>> 
> >>>>>>>>> Mail:     [email protected]
> >>>>>>>>> Web:    www.mobile.de
> >>>>>>>>> 
> >>>>>>>>> Marktplatz 1 | 14532 Europarc Dreilinden | Germany
> >>>>>>>>> 
> >>>>>>>>> 
> >>>>>>>>> Geschäftsführer: Malte Krüger
> >>>>>>>>> HRB Nr.: 18517 P, Amtsgericht Potsdam
> >>>>>>>>> Sitz der Gesellschaft: Kleinmachnow
> >>>>>>>> 
> >>>>>>>> 
> >>>>>>>> --
> >>>>>>>> Simon Effenberg | Site Ops Engineer | mobile.international GmbH
> >>>>>>>> Fon:     + 49-(0)30-8109 - 7173
> >>>>>>>> Fax:     + 49-(0)30-8109 - 7131
> >>>>>>>> 
> >>>>>>>> Mail:     [email protected]
> >>>>>>>> Web:    www.mobile.de
> >>>>>>>> 
> >>>>>>>> Marktplatz 1 | 14532 Europarc Dreilinden | Germany
> >>>>>>>> 
> >>>>>>>> 
> >>>>>>>> Geschäftsführer: Malte Krüger
> >>>>>>>> HRB Nr.: 18517 P, Amtsgericht Potsdam
> >>>>>>>> Sitz der Gesellschaft: Kleinmachnow
> >>>>>>> 
> >>>>>>> 
> >>>>>>> --
> >>>>>>> Simon Effenberg | Site Ops Engineer | mobile.international GmbH
> >>>>>>> Fon:     + 49-(0)30-8109 - 7173
> >>>>>>> Fax:     + 49-(0)30-8109 - 7131
> >>>>>>> 
> >>>>>>> Mail:     [email protected]
> >>>>>>> Web:    www.mobile.de
> >>>>>>> 
> >>>>>>> Marktplatz 1 | 14532 Europarc Dreilinden | Germany
> >>>>>>> 
> >>>>>>> 
> >>>>>>> Geschäftsführer: Malte Krüger
> >>>>>>> HRB Nr.: 18517 P, Amtsgericht Potsdam
> >>>>>>> Sitz der Gesellschaft: Kleinmachnow
> >>>>>> 
> >>>>> 
> >>>>> 
> >>>>> --
> >>>>> Simon Effenberg | Site Ops Engineer | mobile.international GmbH
> >>>>> Fon:     + 49-(0)30-8109 - 7173
> >>>>> Fax:     + 49-(0)30-8109 - 7131
> >>>>> 
> >>>>> Mail:     [email protected]
> >>>>> Web:    www.mobile.de
> >>>>> 
> >>>>> Marktplatz 1 | 14532 Europarc Dreilinden | Germany
> >>>>> 
> >>>>> 
> >>>>> Geschäftsführer: Malte Krüger
> >>>>> HRB Nr.: 18517 P, Amtsgericht Potsdam
> >>>>> Sitz der Gesellschaft: Kleinmachnow
> >>>> 
> >>> 
> >>> 
> >>> -- 
> >>> Simon Effenberg | Site Ops Engineer | mobile.international GmbH
> >>> Fon:     + 49-(0)30-8109 - 7173
> >>> Fax:     + 49-(0)30-8109 - 7131
> >>> 
> >>> Mail:     [email protected]
> >>> Web:    www.mobile.de
> >>> 
> >>> Marktplatz 1 | 14532 Europarc Dreilinden | Germany
> >>> 
> >>> 
> >>> Geschäftsführer: Malte Krüger
> >>> HRB Nr.: 18517 P, Amtsgericht Potsdam
> >>> Sitz der Gesellschaft: Kleinmachnow 
> >> 
> > 
> > 
> > -- 
> > Simon Effenberg | Site Ops Engineer | mobile.international GmbH
> > Fon:     + 49-(0)30-8109 - 7173
> > Fax:     + 49-(0)30-8109 - 7131
> > 
> > Mail:     [email protected]
> > Web:    www.mobile.de
> > 
> > Marktplatz 1 | 14532 Europarc Dreilinden | Germany
> > 
> > 
> > Geschäftsführer: Malte Krüger
> > HRB Nr.: 18517 P, Amtsgericht Potsdam
> > Sitz der Gesellschaft: Kleinmachnow 
> 


-- 
Simon Effenberg | Site Ops Engineer | mobile.international GmbH
Fon:     + 49-(0)30-8109 - 7173
Fax:     + 49-(0)30-8109 - 7131

Mail:     [email protected]
Web:    www.mobile.de

Marktplatz 1 | 14532 Europarc Dreilinden | Germany


Geschäftsführer: Malte Krüger
HRB Nr.: 18517 P, Amtsgericht Potsdam
Sitz der Gesellschaft: Kleinmachnow 

_______________________________________________
riak-users mailing list
[email protected]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

Reply via email to