[
https://issues.apache.org/jira/browse/QPID-949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12664565#action_12664565
]
Aidan Skinner commented on QPID-949:
------------------------------------
There are several changes necessary to implement this. Message payload storage
should be seperated from metadata storage. Persistent messages should be
written to disk immediately as they are recieved.
When memory usages reaches a threshold, in-memory persistant message payloads
could be GC'd from (ideally the back of) queues until pressure has been
relieved. These would be reloaded from disk as they were sent. The existing
weak-ref stuff would work here.
If this dump did not release sufficent memory transient messages would be
written to the disk, again ideally from the back of queues and then reloaded.
Internally, messages would hold a reference to a file in a shallow, wide
hierarchy of files which would contain the message payload. These files would
be deleted when the messages are sent or expired.
On startup, after the metadata had been read to restore the state of the broker
the filesystem would be scanned and files which were not currently referenced
would be deleted. This would clean up any transient messages that had been
written to disk before the broker stopped running but not yet deleted.
> Implement Flow To Disk
> ----------------------
>
> Key: QPID-949
> URL: https://issues.apache.org/jira/browse/QPID-949
> Project: Qpid
> Issue Type: New Feature
> Components: Java Broker
> Affects Versions: M2.1
> Reporter: Marnie McCormack
> Assignee: Rob Godfrey
> Fix For: M5
>
>
> Currently, the Java Broker can do one of two things with a message it has to
> deliver:
> 1. Keep transient messages in memory until delivered
> 2. Write persistent messages to a message store (like BDB) and keep in memory
> until delivery complete
> This means that the broker is not able to avoid OoM exceptions i.e. send
> enough messages to the broker, especially if your consumers are not active,
> and you could bring the broker done once it explodes its available heap.
> RG to add more details and design proposal here please :-)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
