Instead of forming a new thread to point this out, I'll note it here: the AbstractDatabaseAppender classes may be refactorable or renamable into a sort of AbstractIpcAppender instead as another major category of appenders. Might be worth examining, though it's something we'd help in structuring at least for other appenders and whatnot. If you have any questions on best practices, let us know!
Also, it would be great to submit a JIRA issue for this as well so we have somewhere to track the change. On 6 May 2017 at 13:45, Matt Sicker <[email protected]> wrote: > Oh, I also forgot to mention that regarding batching of log messages, > several appenders already follow a similar pattern, so you may find some > reusable code there, especially if you're modeling this from the > NoSqlAppender/Manager abstract classes as well as the > AbstractDatabaseAppender/Manager classes which are beginning to sound > like they have a too-specific name. :) > > On 6 May 2017 at 13:43, Matt Sicker <[email protected]> wrote: > >> First of all, we love contributions, and this one sounds interesting, too. >> >> 1. I don't have experience with Redis (I've used other distributed memory >> caches and data grid frameworks, though), so I don't have an opinion on >> which library to use for it. >> >> 2. As for the RateLimiter class, I'll note that we have a BurstFilter < >> https://logging.apache.org/log4j/2.x/manual/filters.html#BurstFilter> >> which sounds similar to what you're aiming for. In general, rate limiting >> log messages would be more of a Filter concern than an Appender concern, so >> if that isn't a sufficient filter, you could enhance that or create a new >> one backed by that class if it seems appropriate. >> >> 3. Again, we have AsyncLogger and the AsyncAppender implementations that >> fulfill this niche. Take a look at that to see how well it meets your goals >> here and if there are any needed enhancements. >> >> 4. As for host names, take a look at CassandraAppender for some useful >> parsing bits. There is also the FailoverAppender < >> https://logging.apache.org/log4j/2.x/manual/appenders.html# >> FailoverAppender> for a generic failover mechanism, but I don't see the >> harm in including a list of hosts to use (it's how some other appenders >> work due to their drivers anyways like Cassandra and Kafka). >> >> On 6 May 2017 at 13:30, Volkan Yazıcı <[email protected]> wrote: >> >>> Hello, >>> >>> To the best of my knowledge, there is no Redis support in Log4j 2.x NoSQL >>> Appenders. (Please correct if I'm wrong.) I want to create a JIRA ticket >>> and start working on a RedisAppender. Though I do have some questions and >>> will appreciate your feedback. >>> >>> 1. I plan to use Jedis <https://github.com/xetorthio/jedis> for >>> client >>> and perform no shading/relocation. Objections? >>> 2. I want to introduce throttling via Guava's RateLimiter. Is that ok? >>> 3. I am thinking of a background worker Thread periodically polling an >>> ArrayBlockingQueue with a fixed size. This will help me to facilitate >>> the >>> following features: >>> 1. Queue will act as a buffer addressing temporary produce-consume >>> rate mismatch. >>> 2. Background thread will create batches out of the queue, rather >>> than individual inserts. >>> 3. Periodic polling will ensure that we avoid starvation when not >>> enough messages for a batch. >>> 4. I will allow comma separated multiple <host>:<port> pairs to allow >>> fail-over. That is, I will create a Jedis client for each >>> <host>:<port> >>> pair and try them in given order in case of a failure. >>> >>> I'm planning to make a pitch on Monday to have some *company time* for >>> this >>> story. So please do let me know of what you think. >>> >>> Best. >>> >> >> >> >> -- >> Matt Sicker <[email protected]> >> > > > > -- > Matt Sicker <[email protected]> > -- Matt Sicker <[email protected]>
