Hi !
so I'm to get used with the fact that cursor must be *carfully* closed!
I fucked up here, as I forgot one single close after having modifed
mocally the code. Fixing this magically solved the issue.
Sorry for the noise !
Le 4/26/12 2:08 AM, Selcuk AYA a écrit :
On Wed, Apr 25, 2012 at 4:45 PM, Emmanuel Lécharny<[email protected]> wrote:
Le 4/5/12 1:09 AM, Emmanuel Lécharny a écrit :
Le 4/5/12 12:43 AM, Selcuk AYA a écrit :
On Wed, Apr 4, 2012 at 3:22 PM, Emmanuel Lécharny<[email protected]>
wrote:
It's systematic, and I guess that the fact we now pond the RdnIndex
table
way more often than before (just because we don't call anymore the
OneLevelIndex) cause the cache to get filled and not released fast
enough.
do we hold a cursor open while this code gets stuck? I would think we
hold a cursor open and moduify quite a bit of jdbm btree pages for
this kind of behavior to happen.
I'll check that.
As we don't set any size for the cache, its default size is 1024. For
some
of the tests, this mightnot be enough, as we load a lot of entries
(typically the schema elements) plus many others that get added and
removed
while running tests in revert mode.
If I increase the default size to 65536, the tests are passing.
Ok, now, I have to admit I haven't - yet - looked at the LRUCache code,
and
my analysis is just based on what I saw by quickly looking at the code,
the
stack traces I have added and some few blind guesses.
However, I think we have a serious issue here. As far as I can tel, the
code
itself is probably not responsible for this behaviour, but the way we
use it
is.
Did I missed something ? Is there anything we can do - except increase
the
cache size - to get the tests passing fine ?
I'm more concern about what could occur in real life, when some users
will
load the server up to a point it just stop responding...
to aovid this issue, we can let the writers allocate more cache
pages(rather than keeping the cache size fixed) so that they do not
loop waiting for a replaceable cache. However, I would again suggest
making sure we do not forget the cursor open. If we forget a cursor
open and keep allocating new cache pages for writes, we will have
other problems.
Yeah, I can see how it may affect the tests. I'll definitively investigate
this first, before going any further in another direction.
ATM, I'm using a not committed version of JDBM were the default cache size
has been changed.
Thanks a lot Selcuk !
So I still have the LRUCache size issue, after having removed the SubLevel
index. Once I increased the size to 1<< 16, tests are passing.
The failing tests are the SearchAuthorizationIT class' tests.
What happens is that when I add an entry, I update many elements in the
RdnIndex, as I have to modify the nbDescendant in all its parents. As those
tests are injecting a lot of entries, so they do a lot of modifications in
the RdnIndex.
I checked that all the cursors are correctly closed.
Any clue ?
Can you provide the following details:
-on which jdbm table are you having the problem(rdn index, main table?)?
- approximately how many modifications are you doing on this table
while you are holding a cursor open( even if the cursor is held open
legally). Knowing this number would help a lot.
- is the same problem you had before or did closing the cursors in the
previous case solve your problem?
this kind of problem can currently occur if a thread is holding a
cursor on one table(not necessarily illegally) and the "same" thread
is modifying the "same" table with many add/delete/update operations.
I am wondering whether we have a use case like this now. If we have,
then I can change the code to account for it.
--
Regards,
Cordialement,
Emmanuel Lécharny
www.iktek.com
thanks
Selcuk
--
Regards,
Cordialement,
Emmanuel Lécharny
www.iktek.com
