Warron, I would also be very interested in seeing any audispatch and rsyslog work you have for audit log data.
Ryan On Wed, Jun 22, 2016 at 3:41 PM, Joe Blow <[email protected]> wrote: > Our reqs are a bit beefier. I disagree on a few things. For the SAN/NAS > stuff, just buy beefy servers (36-72TB systems are dirt cheap now). Iops > won't be an issue that way either. > > Also, instead of logrotate, I use a cron job which kicks off a threaded > compression script, using xz -e -z -9 for beast mode compression using > 16/40 cores max. 1.2B logs is like 20GB / day. Do the math on a 20-30TB > mount and you're laughing. > > I'd love to see the audispatch work you've done. > > Also, in case you were wondering, I have a patent on my log-to-jpg-to-sql > module, and the corresponding OCR extraction tool. > > Cheers, > > JB > > Original Message > From:[email protected] > Sent:June 22, 2016 3:26 PM > To:[email protected] > Reply-to:[email protected] > Subject:Re: [rsyslog] Central logging solution > > I worked on something similar. > > Are your requirements as basic as what you have presented Abhilash? > > Do your requirements require centralizing audit log data? That should be > handled through audispatch and I can advise you on how to do that. In fact > I might even have a fully complete set of instructions for the audispatch > and another for rsyslog. > > As for managing the volume of data (from 2000+ hosts) and data store, that > would require some reaching out to NAS/SAN consultants for optimizing the > storage configuration; and then if you set up the logrotation to be daily > you can then write a script and put it into crontab to maintain the 18month > auto-cropping of data from the volume based on a *find *command. > > That's my input, since I have had to do something similar but not at the > same volume, or with the same storage requirements exactly. > > Let me know if you need more input. > > -------------------------- > Warron French > > > On Mon, Jun 20, 2016 at 2:04 PM, Joe Blow <[email protected]> wrote: > > > I feel like this is a troll....... > > > > 1. Turn on imptcp > > 2. Turn on imudp > > 3. Point your systems to rsyslog > > 4. Profit > > > > Some other pieces of glue you'll need: > > > > Keepalived > > Liblognormalize > > Xz (for beast mode compression) > > Drbd > > Imagination > > Iptables > > > > I would suggest you convert the log lines to jpgs, so that you can store > > the base64 encoded jpg inside a non-searchable relational database of > your > > choice. > > > > Then you can sell your new "customer" an OCR appliance to search your > > jpg-log-storage-appliance. > > > > Hope this helps with your homework. :) > > > > /troll > > > > Cheers, > > > > JB > > > > > > Original Message > > From:[email protected] > > Sent:June 20, 2016 11:55 AM > > To:[email protected] > > Reply-to:[email protected] > > Subject:[rsyslog] Central logging solution > > > > Hi Team, > > > > I am working on a Central Logging solution for our environment using > Linux > > native logging feature(rsyslog). The requirement is to store the logs > from > > all servers(Windows +all UNIX) and retain for 18 months.Our account got > > around 2000 server(1200 Windows+ 800 UNIX(HPUX/AIX/Linux/Soalris) and > > appliances. The customer wants a solution based on rsyslog.We also need > to > > retain the logs for 18 months on tamper proof storage. > > > > Please could anyone guide me ? Also pls share some good documentation > > > > -- > > Abhilash.P.A > > voice :+91 9663375151 > > > > " Known is a drop and unknown is an ocean" > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com/professional-services/ > > What's up with rsyslog? Follow https://twitter.com/rgerhards > > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad > > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you > > DON'T LIKE THAT. > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com/professional-services/ > > What's up with rsyslog? Follow https://twitter.com/rgerhards > > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad > > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you > > DON'T LIKE THAT. > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > What's up with rsyslog? Follow https://twitter.com/rgerhards > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you > DON'T LIKE THAT. > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > What's up with rsyslog? Follow https://twitter.com/rgerhards > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you > DON'T LIKE THAT. > _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
