cziegeler 2003/11/10 01:07:08
Modified: site/xdocs index.xml
Added: site/xdocs store-janitor.xml store-index.xml store-mru.xml
Log:
Restoring store docs, not fully converted yet
Revision Changes Path
1.4 +1 -1 avalon-excalibur/site/xdocs/index.xml
Index: index.xml
===================================================================
RCS file: /home/cvs/avalon-excalibur/site/xdocs/index.xml,v
retrieving revision 1.3
retrieving revision 1.4
diff -u -r1.3 -r1.4
--- index.xml 29 Oct 2003 17:01:30 -0000 1.3
+++ index.xml 10 Nov 2003 09:07:08 -0000 1.4
@@ -126,7 +126,7 @@
protocols such as myxmldatabase://root/documents/test.xml.
</p>
</td></tr>
- <tr><td>Store</td><td>
+ <tr><td><a href="store-index.html">Store</a></td><td>
<p>
The Store Package provides a way to store named objects without
taking care about where and how they are stored.
1.1 avalon-excalibur/site/xdocs/store-janitor.xml
Index: store-janitor.xml
===================================================================
<?xml version="1.0" encoding="UTF-8"?>
<document>
<properties>
<author email="[EMAIL PROTECTED]">Avalon Documentation Team</author>
<title>The Store Janitor</title>
</properties>
<body>
<section name="Goal">
<p>This document describes the usage of the StoreJanitor.</p>
</section>
<section name="Description">
<p>The implementation is quit simple! Every implementation of a Store can register
in the
StoreJanitor. It checks in a configurable interval if memory is running low. If
low,
it greps via Round Robin a victim (Store) and frees xx% of all emlements in this
Store.
After that the StoreJanitor sleeps and waits for the next iteration.</p>
<p>The StoreJanitor is very useful for web applications that use the store as
a in-memory cache. The StoreJanitor helps in avoiding OutOfMemory exceptions.</p>
</section>
<section name="Configuration">
<p>The Store Janitor can be configured with a few options:
</p>
<ul>
<li><em>freememory</em>: How many bytes shall be always free in the JVM
(Default: 1mb)</li>
<li><em>heapsize</em>: Maximum possible size of the JVM memory consumption
(Default: 64mb)</li>
<li><em>cleanupthreadinterval</em>: How often (sec) shall run the cleanup
thread (Default: 10s)</li>
<li><em>adaptivethreadinterval</em> (experimental): Enable adaptive algorithm
to determine thread interval
(Default: false) When true, <code>cleanupthreadinterval</code> defines
the maximum cleanup interval.
Cleanup interval then is determined based on the memory fill rate: the
faster memory is filled in,
and the less free memory is left, the shorter is the cleanup time.</li>
<li><em>threadpriority</em>: priority of the thread (1-10). (Default: 10)</li>
<li><em>percent_to_free</em>: What fraction of the store to free when memory
is low (1-100). (Default: 10%)</li>
<li><em>invokegc</em>: Invoke the gc on low memory first (true|false;
default: false)</li>
</ul>
<p>The right configuration is very important, because wrong settings can
cause a high system load. Let's have a look at a sample configuration.</p>
<subsection name="Example configuration">
<ul><li>Tomcat settings in tomcat.sh or tomcat.bat:</li></ul>
<source><![CDATA[
%_RUNJAVA% %TOMCAT_OPTS% -Dtomcat.home="%TOMCAT_HOME%" \
-Xmx200000000 org.apache.tomcat.startup.Tomcat %2 %3 %4 %5 %6 %7 %8 %9
]]></source>
<ul><li>StoreJanitor settings:</li></ul>
<p>The freememory and heapsize paramter always depends on the Xmx
parameter.</p>
<source><![CDATA[
<!--+
| Store Janitor: the store garbage collector and memory usage controller.
|
| Be careful with the heapsize and freememory parameters. Wrong values
| can cause high cpu usage. Example configuration:
| Jvm settings:
| -Xmx200000000
| store-janitor settings:
| <parameter name="freememory" value="5000000"/>
| <parameter name="heapsize" value="196000000"/>
|
| It is recommended to have heapsize equal to -Xmx, especially on Sun's
| JVM which are unable to shrink its heap once it grows above minimum.
| Freememory should be greater than amount of memory necessary for normal
| application operation.
| BUT: The heap size of the memory of the JVM is a little bit less than
| the value you specify for -Xmx, so you have to set the heapsize
| for the store janitor to a value which is lower (2% less seems
| to be a working value).
+-->
<store-janitor logger="core.store.janitor">
<!-- How much free memory shall be available in the jvm -->
<parameter name="freememory" value="2048000"/>
<!-- Indicates the limit of the jvm memory consumption. The default max
heapsize for Sun's JVM is (almost) 64Mb -->
<parameter name="heapsize" value="66600000"/>
<!-- How often shall the cleanup thread check memory -->
<parameter name="cleanupthreadinterval" value="10"/>
<!-- Experimental adaptive algorithm for cleanup interval
<parameter name="adaptivethreadinterval" value="true"/>
-->
<!-- Indicates the thread priority of the cleanup thread -->
<parameter name="threadpriority" value="5"/>
<!-- How much percent of the elements of each registered Store
shall be removed when low on memory. Default 10% -->
<parameter name="percent_to_free" value="10"/>
<!-- Invoke the garbage collector when low memory is reached -->
<parameter name="invokegc" value="false"/>
</store-janitor>
]]></source>
<p>It is recommended to have <code>heapsize</code> equal to -Xmx, especially
on Sun's JVM which are unable to shrink its heap once it grows above minimum.
<code>freememory</code> should be greater than amount of memory necessary for
normal
application operation. But the heap size of the memory of the JVM is a little
bit less than
the value you specify for -Xmx, so you have to set the heapsize
for the store janitor to a value which is lower (2% less seems
to be a working value)
</p>
<p> The <code>cleanupthreadinterval</code> defines the interval of the
background
thread which checks memory in seconds. Also this paramter should configured
wisely.
A to short interval can cause also a high system load. The
<code>threadpriority</code> defines the priority of the background thread.
1 is lowest level and 10 the highest.</p>
<p>
The <code>percent_to_free</code> parameter describes, how much percent of the
elements of each registered Store shall be removed when low on memory.
</p>
</subsection>
</section>
</body>
</document>
1.1 avalon-excalibur/site/xdocs/store-index.xml
Index: store-index.xml
===================================================================
<?xml version="1.0"?>
<document>
<properties>
<author email="[EMAIL PROTECTED]">Avalon Documentation Team</author>
<title>Excalibur Store - Overview</title>
</properties>
<body>
<section name="Introduction">
<p>
The Store Package provides a way to store named objects without
taking care about where and how they are stored.
You can choose between different implementations that store
the data e.g. in memory or on the file system etc.
</p>
<p>
This project provides some general components for dealing with stores.
It contains different store implementations, like the
<a href="store-mru.html">MRU Store</a> and a montoring component,
the <a href="store-janitor.html">Store Janitor</a>.
</p>
<p>
These components have been donated by the <a
href="http://cocoon.apache.org";>Apache Cocoon project</a>.
</p>
</section>
</body>
</document>
1.1 avalon-excalibur/site/xdocs/store-mru.xml
Index: store-mru.xml
===================================================================
<?xml version="1.0" encoding="UTF-8"?>
<document>
<properties>
<author email="[EMAIL PROTECTED]">Avalon Documentation Team</author>
<title>MRUMemoryStore and Swapping</title>
</properties>
<body>
<section name="Goal">
<p>This document explains how the MRUMemoryStore and Swapping executes.</p>
</section>
<section name="Overview">
<p>The MRUMemoryStore was developed to provide a standard algorithm to
store data in memory. For web-based applications the MRU (Most Recently Used)
algorithm
is very suitable, because the object most frequently accessed is always on
"top".
</p>
<p>If configured, the objects are also swapped to the filesystem if the
MRUMemoryStore reached his configured max. object limit.
</p>
</section>
<section name="Implementation">
<subsection name="MRUMemoryStore">
<p>
The heart of the MRUMemoryStore ist combination of a LinkedList and a HashMap:
</p>
<p>
The LinkedList provides the queue mechanism, and the entries in the LinkedList
contain the key to the data in the HashMap. When caching a new entry in to the
list,
the entry is inserted to the front of the list.
If the list is already full, the oldest data entry is removed from the Cache.
When requesting a entry, the store returns the object by key and inserts the
requested key
on the top of the Cache.
This implementation keeps the most recent used objects in the store and provides
the best
use of the machines memory.
</p>
</subsection>
<subsection name="Swapping">
<p>
When the MRUMemoryStore is full or the JVM is at the heap size limit the
objects in the MRUMemoryStore are swapped to the Filesystem. The default
directory is "cache-dir" in the work-directory.
</p>
<p>
NOTE: The keys are Strings at the moment. Therefor the
filenames of the swapped objects can be very long. Especially Windows OS
flavours have problems with long filenames. Use the JispFilesystemStore to
get rid of it.
</p>
</subsection>
</section>
<section name="Configuration of the MRUMemoryStore in the cocoon.xconf">
<source><![CDATA[
<store class="org.apache.excalibur.store.impl.MRUMemoryStore"
logger="core.store">
<parameter name="maxobjects" value="100"/>
<parameter name="use-persistent-cache" value="true"/>
</store>]]>
</source>
<p>Explanation of the paramters:</p>
<ol>
<li><code><parameter name="maxobjects" value="100"/></code>:
Indicates how many objects will be hold in MRUMemoryStore. When the number
of maxobjects has been reached, the last object in the MRUMemoryStore will be
thrown out.</li>
<li><code><parameter name="use-persistent-cache" value="true"/></code>:
If this switch is set on true, objects are swapped out to the filesystem,
if the MRUMemoryStore has reached his maximum object limit, or the JVM
memory consumption is over the heap size limit. See StoreJanitor user
docs for more information.</li>
</ol>
</section>
</body>
</document>
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
