Alan,
If your d-hosts are paging heavily, they may be at or near their
capacity limits. If so, you should think about adding more d-hosts. You
should measure the current Bytes/fragment utilization, and project
growth for the next 12-24 months.
You can calculate Bytes/fragment from the Forest status page or the
xdmp:forest-status() API call. As a rule of thumb, this "forest memory"
shouldn't be more than 1/3 of main memory (ie, 10.67-GB for 32-GB). Of
the rest, another 1/3 will probably be group-level caches, and the final
1/3 will be used by in-memory stands, queries, merges, and the OS.
That's just a rule of thumb, but it works fairly well for me.
In general, you should cap forest size at the lesser of about 200-GB
on-disk, or when total in-memory size approaches 1/3 of main memory. The
number of forests per host is generally dictated by this, plus a
recommendation of 1 Forest per 2 CPU-cores. If you decide that a given
forest is close to its limit, you can set it to be "delete-only" in the
forest configuration. If a forest is too large, you may want to
physically move it to another host, or perhaps use Corb or XQSync to
migrate documents from one forest to another.
All of the above is somewhat simplified: running a TB-scale database
calls for careful planning and constant monitoring. There is also some
useful leverage to be gained from system tuning, but all that gets a bit
involved for an email thread.
Some of these topics are discussed in the docs:
http://developer.marklogic.com/pubs/4.1/books/cluster.pdf
http://developer.marklogic.com/pubs/4.1/books/performance.pdf
http://developer.marklogic.com/pubs/4.1/books/admin.pdf
-- Mike
On 2009-06-29 18:19, Alan Darnell wrote:
We have a cluster of 6 servers — two e-nodes and 4 d-nodes. All six machines
are configured with 32GB of RAM. The 2 e-nodes behave reasonably, eating up
about 16GB of RAM. All four d-nodes eat up 32GB of RAM quickly after MarkLogic
is started. Two of the machines start eating into swap after about a day of use
and this starts to slow the performance of the whole cluster. What causes
these machines to eat up swap and how can we rebalance forests so that memory
is consumed where it is free?
Alan
_______________________________________________
General mailing list
[email protected]
http://xqzone.com/mailman/listinfo/general
