Stefan Seelmann wrote:
Hi Alex,
Alex Karasulu schrieb:
Stefan S.,
Do we use a secondary cache for Studio? Just wondering because of the
performance issues someone noted on the user list when dealing with a
very large directory.
no, we don't use a cache for Studio. I tried to use ehcache long time
ago but it was really slow when it swaps entries to the disk and back.
Whatever cache you use, as soon as you hit the disk, performances will
be awfull. The question is : do we accept those terrible performance, or
should we accept an OOM ? Now, with JDBM, you have a middle term
situation. If you use an appropriate size for the JDBM cache, it will
only contain the tree, not the entries, leading to a great perf
improvement. You will still have to hit the disk to get the data, but
you will avoid a lit of disk access.
Of course, it depends on the kind of operation you want to run. We
discussed a lot with Pierre-Arnaud about the idea to integrate ADS
_into_ studio, gaining a direct benefit of being able to play faked
operations locally, and also to use its internal cache.
To be discussed, of course...
Today there we have the following:
- a HashMap<String, IEntry>: with the DN as key and the entry as value
(IEntry is a Studio internal interface with some implementations)
- a HashMap<IEntry, AttributeInfo>: as soon as the attributes of an
entry were loaded an AttributeInfo containing all attributes and other
information is created and put to this map
- a hashMap<IEntry, ChildrenInfo>: as soon as child entries are loaded
a ChildrenInfo containing all child entries is created and put to this
map.
For sure, this does scale well. With the default VM parameters (64MB
heap) you could load about 30,000 entries to get an OutOfMemory :-(((
What about WeakHashap ?
I also think that the switch from the old DN/RDN implementation to the
shared-ldap LdapDN/Rdn implementation costs some memory. We should
consider to do some test for performance and memeory consumtion.
Performance is ok, but we may eat more memory with LdapDN, as we keep
two forms of the DN (user provided and normalized). In your case, I
would say it's unoticable, when compared with the other attributes
(especially big binary values).
If we were to dig, I would not pick this place...
The idea occurred to me that the JDBM code for JdbmTable and
JdbmIndex could potentially be used by Studio to help solve some of
the caching problems. If this is something you think will help we
can move this code into shared so both the server and Studio can
leverage them.
Yeah, that sounds great. You want me to look how the JDBM code works?
I guess we will have some time at ApacheCon for that.
Yes, sure ! This is what AC is for !
--
--
cordialement, regards,
Emmanuel Lécharny
www.iktek.com
directory.apache.org
