We noticed that under heavy load a ResourceChangeListener doesn't
receive notifications about all the changes in the repository. Is this
expected behavior?
More details:
We have a batch import scheduled job that pulls in a few thousand
records from an external database and creates a new node for each of
them (a Visit node). For each of the new nodes, we have a
ResourceChangeListener that creates additional nodes as needed (one or
two Surveys associated with the Visit), and changes the Visit node to
mark that it has surveys. All the new Visit nodes correctly trigger a
ResourceChange event which is correctly processed by the survey
assignment ResourceChangeListener. The subsequent change of the
hasSurveys property in the Visit node triggers a new ResourceChange
event, which is then caught by another ResourceChangeListener that is
monitoring the status of the visits. And this second listener doesn't
always receive events.
Some debugging:
We only noticed the issue only when our import went from tens of records
in our test environment to thousands of records in production. The first
few hundreds of records don't seem to be affected, then more and more
events are missed, with a total of between 5 and 10% event loss. When I
added a 500ms delay between each imported record, no events were lost.
With a 100ms delay between each imported record, 2% of the events were lost.
There's nothing in the logs (at the INFO level) to indicate that the
events were intentionally dropped.
Any ideas?
Thanks,
--
Sergiu Dumitriu
http://purl.org/net/sergiu
