Quoting Emmanuel Lecharny <[email protected]>:
Hi,
this is the last step to handle the paged search request : cleaning the
remaining paged search when they are not used.
The problem is that a client can launch a paged search, get some few
pages, and then stop requesting the remaining pages, without closing
the request. Obviously, this is bad, bad, bad, but we won't be able to
spank all the clients ;)
The created paged search context will then remain until the session is
closed, keeping all the associated data (cursors, and such) in memory.
Plus the client may launch more than one paged search request, leading
to a quick OOM...
There are a few strategy we can implement to avoid such a problem :
1) first, we can limit the number of paged search request per session.
I guess that a session with more than 5 or 10 opened paged search is
very unlikely to occur. I suggest that we limit the number of opened
paged searches to 5 atm (of course, this should be configurable), and
unlimited for the administartor. When we reach this number, then the
oldest paged search will be removed from the session.
Maybe a per-user limit would also make sense. A user could open
multiple sessions in order to start more paged searches. Also a global
limit could make sense. But we should keep it simple ;-)
Another question is how to define the "oldest" paged search. Do we use
- either the timestamp when the initial search were started and the
first page was requested
- or the timestamp when the last page was requested?
2) we can also manage a timeout for the paged search. That mean a
thread will be in charge of removing all the time-outed paged search
across the sessions.
Do we already have another thread for clean-up stuff? Maybe we could
plug into such an existing thread.
If the clean-up isn't too time consuming maybe we could do such checks
in other requests. e.g. each time when a paged search request comes in
we could check that. This would avoid an extra thread.
3) We can also wait until the session dies, and cleanup the associated
paged search.
In any case, we must do something, as the cursors _have_ to be closed.
Does those 3 proposals make sense to you? Did I forgot something? Do
you see a better strategy ?
The server also implements persistent search. Are there similar
problems with open resources? Maybe we can look how it is solved there?
Regards,
Stefan
