Aaron,
Thanks for your prompt response!
Here is the results after setting jcs.default.cacheattributes.MaxObjects=0,
and putting two more 3 sec sleeps followed by GCs after the cache is loaded:
100K objects in the disk cache:
TEST STARTING heap size: 2787 KB
Purgatory Size = 89171
TEST: Sleeping 15 sec to let objs roll to disk
Purgatory Size = 0
TEST after cache load: 37772 KB, delta 34985 KB
TEST after 3 sec pause: 37772 KB, delta 34985 KB
TEST after 3 another sec pause: 20216 KB, delta 17429 KB
Purgatory Size = 0
200K objects in the disk cache:
TEST STARTING heap size: 2787 KB
Purgatory Size = 173664
TEST: Sleeping 15 sec to let objs roll to disk
Purgatory Size = 0
TEST after cache load: 57444 KB, delta 54657 KB
TEST after 3 sec pause: 57443 KB, delta 54656 KB
TEST after 3 another sec pause: 39886 KB, delta 37099 KB
Storing no objects in cache reduced the heap, as we'd expect. However,
we were still holding 190 bytes per item in the disk cache (e.g., in the
200K objects case, the delta from the start was 37099 KB, divided by 200K
objects is 190 bytes. For the 100K objects case, it comes out 178 bytes.
I modified my test program to store the same data (TestData) in a simple
HashMap, which resulted in this:
100K objects in-mem in HashMap:
TEST STARTING heap size: 2787 KB
TEST after cache load: 84312 KB, delta 81525 KB
TEST after 3 sec pause: 83903 KB, delta 81116 KB
TEST after 3 another sec pause: 83903 KB, delta 81116 KB
200K objects in-mem in HashMap:
TEST STARTING heap size: 2787 KB
TEST after cache load: 172580 KB, delta 169793 KB
TEST after 3 sec pause: 171600 KB, delta 168813 KB
TEST after 3 another sec pause: 171600 KB, delta 168813 KB
Note that in the case of using JCS, after loading 200K objects all into the
disk cache, JCS reported Data File Length = 140755580. So of the 168 MB
being used by the HashMap test, we should expect about 140 MB of that to be
data (assuming Java serialization is pretty efficient), leaving about 30 MB
for the HashMap infrastructure.
I then modified the HashMap test case to store a simple "new Object()" in the
HashMap instead of a TestData.
TEST STARTING heap size: 2787 KB
TEST after cache load: 16573 KB, delta 13786 KB
TEST after 3 sec pause: 16572 KB, delta 13785 KB
TEST after 3 another sec pause: 16572 KB, delta 13785 KB
TEST STARTING heap size: 2787 KB
TEST after cache load: 33136 KB, delta 30349 KB
TEST after 3 sec pause: 33070 KB, delta 30283 KB
TEST after 3 another sec pause: 33070 KB, delta 30283 KB
So JCS's overhead is really not that bad, compared to a HashMap of
just the keys to Object (about 22% more space).
I also ran the 200K TestData JCS cache with 0 objects in mem case under
the YourKit profiler. It showed 37 MB being used by the IndexedDiskCache,
36.1 MB of that in java.util.HashMap. It reported these as the primary
mem users:
+---------------------------------------------------------------------------+-----------------+-------------------+
| Name |
Objects | Shallow Size |
+---------------------------------------------------------------------------+-----------------+-------------------+
|java.lang.String |
200,429 25 % | 8,017,160 21 % |
|char[] |
200,347 25 % | 7,945,024 21 % |
|java.util.HashMap$Entry |
200,254 25 % | 9,612,192 26 % |
|org.apache.jcs.auxiliary.disk.indexed.IndexedDiskElementDescriptor |
200,000 25 % | 6,400,000 17 % |
The interesting there is that my average String size is 40 bytes, ditto the
char[]. Note that I form the key, and the values in TestData, using
Integer.toString(int). I guess I'll have to look at the code for it, maybe
its really inefficent.
After seeing this result, I went back and changed my code to use
"new Integer(i)" as the key, instead of "Integer.toString(i)". Results:
100K objects in the disk cache:
TEST STARTING heap size: 2787 KB
Purgatory Size = 89646
TEST: Sleeping 15 sec to let objs roll to disk
Purgatory Size = 0
TEST after cache load: 32268 KB, delta 29481 KB
TEST after 3 sec pause: 14712 KB, delta 11925 KB
TEST after 3 another sec pause: 14712 KB, delta 11925 KB
200K objects in the disk cache:
TEST STARTING heap size: 2787 KB
Purgatory Size = 176268
TEST: Sleeping 15 sec to let objs roll to disk
Purgatory Size = 0
TEST after cache load: 46339 KB, delta 43552 KB
TEST after 3 sec pause: 28783 KB, delta 25996 KB
TEST after 3 another sec pause: 28783 KB, delta 25996 KB
TEST after 30 another sec pause: 28783 KB, delta 25996 KB
This gets us down to a per-disk-cached-item overhead of 130 bytes. This may
be the minimum that overhead achievable with the current JCS
implementation of the indexed disk cache.
Nick
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
