Hi all,
I'm currently writing a server plugin. I need it to make some specialized
queries that are not supported by the standard REST API. The important methods
I expose are 'query' and 'get_next_page', the latter to support results
pagination (i.e. the plugin is stateful).
In 'query', I run my query against the Neo4j backend, and store a Node iterator
to the query results (this is either an iterator originating from
'getAllNodes', or a Lucene IndexHits<Node> instance). In 'get_next_page', I run
through the next N items of the iterator and return these as a
ListRepresentation. The same iterator object is kept across all page
retrievals, but of course stepped forward N steps for every invocation. After
having gone through all pages, the reference to the Node iterator is removed.
Now, as I understand it, all the heap space I should be concerned about using,
is the one I allocate locally in my methods, since the referenced stored to the
iterator object is just a tiny reference, and iterator results are fetched
lazily (i.e., even though the iterator covers a result set greater than the
allotted heap size, I shall be able to page through it within given heap space
if the page size is small enough). But when I run my plugin, this does not seem
to be the case. I can make several successful calls in a row to
'get_next_page', but then after a while bump into "GC overhead limit exceeded"
which I cannot quite understand. I am rather certain the size of each page
returned is within the allotted heap size. For some reason the heap usage seems
to grow with the calls to 'get_next_page' which I cannot understand, given my
understanding of the Node iterators from Neo4j.
How do I avoid hitting this GC overhead limit? Am I missing something?
(And yes, I've tried using different values of the allowed heap space by
fiddling in the conf-files, and sure I can give tons of memory to the instance,
and then it works, but I shouldn't have to give more heap space than what Neo4j
"needs", plus my page size).
Thanks!
Regards,Anders
_______________________________________________
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
