Oleksiy, Go to step 6: "Compare Step 2 and Step 5 …". There is a link to an Excel spreadsheet at the end of the sentence "The above calculations are automated in this memory model spreadsheet.". Forget the text and use the spreadsheet (memory model spreadsheet).
Much of that text is still related to memory management of 1.2 and 1.3. Seems it did not get updated to 1.4. Hmm, that might be my fault. Answers to your comments/questions below: 1. Step 3 on the page is just wrong with 1.4: open_file_memory = (max_open_files -10) * 4194304 2. average_sst_filesize is not relevant with 1.4. It was used to estimate the size of the bloom filter attached to each .sst file. There is now a fixed maximum of 150,001 bytes for the bloom filter, and it is the typical size for all files in levels 2 through 6. 3. The page is attempting to estimate the total memory usage of one vnode. The spreadsheet does the same. Therefore the maximum memory per either model is the "working memory per vnode" in Step 5, or the "working per vnode" line in the spreadsheet, multiplied by the number of vnodes active on the node (server). Now let me make a related note / sales pitch. The upcoming Riak 2.0 eliminates all the manual calculations / planning. You tell Riak what percentage of memory is allocated to leveldb. leveldb then dynamically adjusts each vnode's allocation as your dataset changes and/or vnodes are moved to and from the node (server). Matthew On Apr 4, 2014, at 5:08 AM, Oleksiy Krivoshey <[email protected]> wrote: > Can someone please suggest how to understand the formula for open_file_memory > on this page: > http://docs.basho.com/riak/latest/ops/advanced/backends/leveldb/#Parameter-Planning > > 1. It definitely lacks some brackets, the correct formula is: > > OPEN_FILE_MEMORY = (max_open_files-10) * (184 + (average_sst_filesize/2048) > * (8+((key_size+value_size)/2048 +1)*0.6)) > > > 2. How to estimate average_sst_filesize? > > > 3. does the result estimate the memory used by a single open file in any > particular vnode? Or by a single vnode with max_open_files open? As > max_open_files is a per vnode parameter then how to estimate the maximum > memory used by leveldb if all vnodes have all max_open_files open? is it > result*ring_size or result*ring_size*max_open_files? > > > Thanks! > > -- > Oleksiy > _______________________________________________ > riak-users mailing list > [email protected] > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
_______________________________________________ riak-users mailing list [email protected] http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
