It usually starts from a URL as most of these have been a result of super-long URL requests. The other offender would be the auditd.log, but lately it's the long http requests that have been causing us problems.
When you say "logcollectoring ossec.log" do you mean to ask if we're having OSSEC monitor the ossec.log itself? If so, in most cases we are using OSSEC to monitor the ossec.log, but not specifically looking for that Large message size warning. I'd be hesitant to do that, knowing we could potentially cause the alert.log to grow wildly out of control as well. On Mon, Aug 8, 2011 at 7:49 AM, dan (ddp) <[email protected]> wrote: > Have you traced it back to the beginning? To see what message started the > chain? > And are you logcollectoring ossec.log? > > On Mon, Aug 8, 2011 at 10:31 AM, jplee3 <[email protected]> wrote: > > BTW: The initial cause of this issue seems to be related to requests > > too long for OSSEC to handle. But OSSEC shouldn't grow the logs so > > tremendously huge when it encounters multiple events that are too > > large. I think this might be a bug. This is a snippet of what I see > > when looking at an ossec.log that has grown out of control > > > > "'2011/08/07 23:38:01 ossec-logcollector: Large message size: > > '2011/08/07 23:38:01 ossec-logcollector: Large me > > ssage size: '2011/08/07 23:38:01 ossec-logcollector: Large message > > size: '2011/08/07 23:38:01 ossec-logcollector: > > Large message size: '2011/08/07 23:38:01 ossec-logcollector: Large > > message size: '2011/08/07 23:38:01 ossec-logc > > ollector: Large message size: '2011/08/07 23:38:01 ossec-logcollector: > > Large message size: '2011/08/07 23:38:01 o > > ssec-logcollector: Large message size: '2011/08/07 23:38:01 ossec- > > logcollector: Large message size: '2011/08/07 2 > > 3:38:01 ossec-logcollector: Large message size: '2011/08/07 23:38:01 > > ossec-logcollector: Large message size: '201 > > 1/08/07 23:38:01 ossec-logcollector: Large message size: '2011/08/07 > > 23:38:01 ossec-logcollector: Large message s > > ize: '2011/08/07 23:38:01 ossec-logcollector: Large message size: > > '2011/08/07 23:38:01 ossec-logcollector: Large > > message size: '2011/08/07 23:38:01 ossec-logcollector: Large message > > size: '2011/08/07 23:38:01 ossec-logcollecto > > r: Large message size: '2011/08/07 23:38:01 ossec-logcollector: Large > > message size: '2011/08/07 23:38:01 ossec-lo > > gcollector: Large message size: '2011/08/07 23:38:01 ossec- > > logcollector: Large message size: '2011/08/07 23:38:01 > > ossec-logcollector: Large message size: '2011/08/07 23:38:01 ossec- > > logcollector: Large message size: '2011/08/07 > > 23:38:01 ossec-logcollector: Large message size: '2011/08/07 23:38:01 > > ossec-logcollector: Large message size: '2 > > 011/08/07 23:38:01 ossec-logcollector: Large message size: '2011/08/07 > > 23:38:01 ossec-logcollector: Large message > > size: '2011/08/07 23:38:01 ossec-logcollector: Large message size: > > '2011/08/07 23:38:01 ossec-logcollector: Larg > > e message size: '2011/08/07 23:38:01 ossec-logcollector: Large message > > size: '2011/08/07 23:38:01 ossec-logcollec > > tor: Large message size: '2011/08/07 23:38:01 ossec-logcollector: > > Large message size: '2011/08/07 23:38:01 ossec- > > logcollector: Large message size: '2011/08/07 23:38:01 ossec- > > logcollector: Large message size: '2011/08/07 23:38: > > 01 ossec-logcollector: Large message size: '2011/08/07 23:38:01 ossec- > > logcollector: Large message size: '2011/08/ > > 07 23:38:01 ossec-logcollector: Large message size: '2011/08/07 > > 23:38:01 ossec-logcollector: Large message size: > > '2011/08/07 23:38:01 ossec-logcollector: Large message size: > > '2011/08/07 23:38:01 ossec-logcollector: Large messa > > ge size: '2011/08/07 23:38:01 ossec-logcollector: Large message size: > > '2011/08/07 23:38:01 ossec-logcollector: La > > rge message size: '2011/08/07 23:38:01 ossec-logcollector: Large > > message size: '2011/08/07 23:38:01 ossec-logcoll > > ector: Large message size: '2011/08/07 23:38:01 ossec-logcollector: > > Large message size: '2011/08/07 23:38:01 osse > > c-logcollector: Large message size: '2011/08/07 23:38:01 ossec- > > logcollector: Large message size: '2011/08/07 23:3 > > 8:01 ossec-logcollector: Large message size: '2011/08/07 23:38:01 > > ossec-logcollector: Large message size: '2011/0 > > 8/07 23:38:01 ossec-logcollector: " > > > > > > It seems like it gets into a loop that starts spewing out the "Large > > message size" because it seems to start reading each word as a line at > > some point (maybe when the event is a certain # of words long). I'll > > have to research more, but has anyone else come across this? > > > > On Aug 7, 8:08 am, Jeremy Lee <[email protected]> wrote: > >> Thanks Dan, that seemed to have worked. > >> > >> > >> > >> > >> > >> > >> > >> On Sat, Aug 6, 2011 at 11:48 AM, Jeremy Lee <[email protected]> wrote: > >> > Good idea.... I'll give that a shot. > >> > >> > Sent from my Verizon Wireless Phone > >> > >> > -----Original Message----- > >> > From: dan (ddp) <[email protected]> > >> > Sent: Saturday, August 06, 2011 10:48 AM > >> > To: [email protected] > >> > Subject: Re: [ossec-list] Re: Recompiling ossec-logcollector and full > >> > command? > >> > >> > On Sat, Aug 6, 2011 at 1:15 PM, Jeremy Lee <[email protected]> wrote: > >> > > This is 2.5.1 > >> > > We thought about just upgrading to 2.6 but we need the full_command > >> > > functionality in the agent.conf > >> > > I'm not sure what is different about the install.sh compilation of > >> > > ossec-logcollector, but I know that when I compile from source it > doesn't > >> > > work. > >> > > I basically did this: > >> > > 1) in src, run "make all" (also tried just "make libs") > >> > > 2) in src/logcollector, run "make" > >> > > 3) cp src/logcollector/ossec-logcollector /var/ossec/bin > >> > > 4) restart OSSEC > >> > > 5) OSSEC.log loads only what's in ossec.conf > >> > >> > Why not modify the src and re-run the install.sh? > >> > Also, diff? > >> > >> > > On Sat, Aug 6, 2011 at 9:50 AM, dan (ddp) <[email protected]> wrote: > >> > >> > >> Which version of OSSEC? > >> > >> > >> On Sat, Aug 6, 2011 at 12:14 PM, jplee3 <[email protected]> wrote: > >> > >> > Nevermind my last comment about ossec.conf not being read > properly. I > >> > >> > must have not saved it after editing...doh. > >> > >> > >> > It seems to work fine. But agent.conf doesn't seem to be > processed in > >> > >> > still. > >> > >> > >> > On Aug 5, 4:54 pm, jplee3 <[email protected]> wrote: > >> > >> >> Hi all, > >> > >> > >> >> So we ran into an issue with "Large message size" warnings > filling up > >> > >> >> the ossec.log file and causing the file to grow out of control > and > >> > use > >> > >> >> up disk space. I went ahead and commented out the lines in > >> > >> >> read_syslog.c and read_multiline.c to prevent this from > happening in > >> > >> >> the future, but then noticed after starting OSSEC back up, that > the > >> > >> >> full commands weren't running. > >> > >> > >> >> I made sure to backup the original ossec-logcollector, and when > I > >> > >> >> restored it and restarted OSSEC, the full commands showed up as > >> > >> >> running in the ossec.log > >> > >> > >> >> At first I thought it was the changes I made with commenting out > the > >> > >> >> "Large message size" lines, so I deleted the dir, untarred to a > fresh > >> > >> >> folder, and compiled straight away. Copied the > ossec-logcollector > >> > >> >> over, restarted OSSEC, and no go with full command. > >> > >> > >> >> Is there something I'm missing when compiling in > src/logcollector? I > >> > >> >> noticed that read_fullcommand.c does exist in this directory. >
