I've implemented storage of log events in various proprietary products through both the JDBC Appender to MySQL and MsSQL and the MongoDb 3 and 4 Appenders. Works like a charm. The database logging is quite customized using the Log4j map message infrastructure.
Gary On Tue, Aug 3, 2021, 12:50 Matt Sicker <[email protected]> wrote: > Hey all, I have a somewhat practical question related to logging here. > For those of you maintaining a structured event log or audit log of > some sort, what types of event log stores are you using to append them > to? I feel like solutions like Splunk, ELK, etc., are geared toward > diagnostic logs which don't necessarily need retention beyond a > relatively short time period. On the other hand, one of the more > natural append-only storage solutions I can think of is Kafka, though > that, too, isn't really geared toward long term storage (even if I can > theoretically fit the entire audit log on one machine). I've been > considering potentially using Cassandra here for durability and append > speed, but even that seems overkill since I don't want or need to be > able to ever update a log event after it's been stored. I've also > considered having Kafka as a layer in between, but that just feels > like overengineering as I don't expect event logs to populate nearly > as fast as, say, wind turbine sensor data where I last used that > architectural pattern. > > I'm curious if anyone has experience with building their own event log > storage service or using an existing one along with any advice. >
