That's good to know. I (wrongly) assumed that what counted towards storeUsage was undeleted messages, not undeleted files. Now that I think about it, I can see why that's the decision that was made, though it's a little confusing to see that we're "using" X bytes when in reality almost all of them are deleted. But it's the lesser of the two evils, I suppose.
I'm frankly surprised I've never heard anyone suggest having KahaDB do periodic compaction of data files in lieu of periodic deletion of entirely unused files, to avoid all the nonsense surrounding this issue. So let me be the first (that I've heard): https://issues.apache.org/jira/browse/AMQ-5905 Tim On Jul 28, 2015 4:56 AM, "Christopher Shannon" < christopher.l.shan...@gmail.com> wrote: > Tim, the problem of KahaDB files not being deleted is visible in the > broker. All of the KahaDB data files are counted against store usage and > I've seen it occur several times where the store usage steadily increases > as reported by the SystemUsage metric the point where the store fills up > and producers get blocked because of scattered messages across data files. > > Now, this may not be the actual issue in this case because the original > post mentions that messages shouldn't be held onto for more than 7 days but > I figured it was something worth looking at as the symptom of a steadily > increasing store usage as the month goes on is definitely a symptom of what > Gary describes in his post. > > On Tue, Jul 28, 2015 at 1:00 AM, Tim Bain <tb...@alumni.duke.edu> wrote: > > > If you examine the broker via JMX, do you see an increasing number of > > messages on the various destinations as the percent usage increases > (which > > would indicate that the deletions are not succeeding)? > > > > Have you confirmed that messages are not building up in the DLQ (dead > > letter queue)? You either need to configure a custom process to go > through > > any messages routed there and do whatever you want with them (delete > them, > > log them for a developer to look at later, whatever) or you need to > > configure the broker to not put them in the DLQ to begin with (via the > > discarding DQL plugin). > > > > Chris, the problem Gary's post solves is one of KahaDB data files not > being > > deleted rather than increasing storeUsage; I don't think that problem > would > > manifest as something visible from within the broker like what Mark has > > described, only from the outside when examining the file system. > > > > Tim > > > > On Mon, Jul 27, 2015 at 3:06 PM, Christopher Shannon < > > christopher.l.shan...@gmail.com> wrote: > > > > > If you are using KahaDB you might be running into a limitation of > mixing > > > destinations with fast message rates vs slow message rates. Gary > wrote a > > > good blog post about it here: > > > > > > http://blog.garytully.com/2011/11/activemq-multiple-kahadb-instances.html > > > Take a look at that and see if it helps you. > > > > > > > > > On Mon, Jul 27, 2015 at 4:47 PM, Scammell < > > > mark.har...@meridianenergy.co.nz> > > > wrote: > > > > > > > Hello, > > > > > > > > I'm using ActiveMQ version 5.10.1. > > > > > > > > Since we started using ActiveMQ a few weeks ago, the store percentage > > for > > > > the broker has been steadily increasing - from 10% at the start of > the > > > > month > > > > to 29% currently. > > > > > > > > In that period, we have processed (sent to queue and subsequently > > > removed) > > > > approximately 5 million records. > > > > > > > > We currently hold on to a ever-rotating amount of ~20,000 messages > for > > a > > > > maximum of 7 days, after which we then delete them. > > > > > > > > We don't seem to be seeing any slow-down in the increase of store > > percent > > > > utilised even though the system has reached a point where the amount > of > > > > messages we are processing each day is not increasing. Could anyone > > > advise > > > > on a strategy for reducing this figure? > > > > > > > > Thanks in advance for any assistance. > > > > > > > > > > > > > > > > -- > > > > View this message in context: > > > > > > http://activemq.2283324.n4.nabble.com/Store-percent-used-tp4699945.html > > > > Sent from the ActiveMQ - User mailing list archive at Nabble.com. > > > > > > > > > >
