On Wed, Jul 23, 2014 at 10:11 AM, David Lang <[email protected]> wrote: > On Tue, 22 Jul 2014, Micah Yoder wrote: > > On 7/22/14, 3:21 PM, David Lang wrote: >> >> Well, that indicates that you have a problem with your dynafile >>> configuration, you don't allow enough files to be open, lookup >>> dynafilecachesize and increase it to be much larger >>> >>> With this sort of problem, your performance is going to be horrid, and >>> as such, your queues will grow. Not seeing your impstats output, but >>> I'll bet the size of the queues is large, and if you mutiply those sizes >>> by the maxmessagesize, you will probably end up accounting for a lot of >>> the memory that you think is 'leaking' >>> >> >> Thanks, I will check that. The "main queue" has size=2, but this one is >> rather different: >> >> Tue Jul 22 09:57:51 2014: action 16 queue: size=438266 enqueued=438268 >> full=0 discarded.full=0 discarded.nf=0 maxqsize=438266 >> >> Any idea how one goes about finding which action line this corresponds to? >> > > well, it's the 16th action in the config file, if you set name='value' in > your action() statements it will replace 'action #' with the name. > depending on how many actions you have, it may be easier to count up from > the bottom of the file. > > > Also, if this accounted for the majority of the memory, each item would >> have to consume 5-10k. Does that sound right? (The vast majority of our >> log lines are much less than that, but it's possible that something >> sends a stream of logs that large.) >> > > the datastructures are based on the max message size, not the size of the > actual message (at least as I understand it),
That's not 100% correct. We have a fixed-size buffer (100 chars in default builds) that is always present. If the message is larger, the buffer is not used and exactly the amount of memory that's needed is allocated. and you have to have both the raw message and all the parsed components, so > it's not unreasonable to hit 5-10K of RAM per message if the default > message size is 4K (I know it's at least 2K and I believe it's 4K) > I think 5 to 10k is a bit much, but it depends on many factors. Note that the MSG property does NOT take up space. But there are many other properties which do. And if (JSON) variables are used, much more memory is used. Still, for regular messages, I would expect the average message object to be around 1 to 2 KiB. > Also, if you are using the default FixedArray size for your main message > queue, it's eating up it's max size even when empty. > > This can be a very good explanation. > while the exact details are murky, it really looks like this is your > problem. In any case, it's clear that you have a big problem with that > action, it's only processed two messages while getting 438K sent to it. > > > ... which sounds extremely strange in itself ;) Rainer _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
