Oh, I also forgot to mention that regarding batching of log messages, several appenders already follow a similar pattern, so you may find some reusable code there, especially if you're modeling this from the NoSqlAppender/Manager abstract classes as well as the AbstractDatabaseAppender/Manager classes which are beginning to sound like they have a too-specific name. :)
On 6 May 2017 at 13:43, Matt Sicker <[email protected]> wrote: > First of all, we love contributions, and this one sounds interesting, too. > > 1. I don't have experience with Redis (I've used other distributed memory > caches and data grid frameworks, though), so I don't have an opinion on > which library to use for it. > > 2. As for the RateLimiter class, I'll note that we have a BurstFilter < > https://logging.apache.org/log4j/2.x/manual/filters.html#BurstFilter> > which sounds similar to what you're aiming for. In general, rate limiting > log messages would be more of a Filter concern than an Appender concern, so > if that isn't a sufficient filter, you could enhance that or create a new > one backed by that class if it seems appropriate. > > 3. Again, we have AsyncLogger and the AsyncAppender implementations that > fulfill this niche. Take a look at that to see how well it meets your goals > here and if there are any needed enhancements. > > 4. As for host names, take a look at CassandraAppender for some useful > parsing bits. There is also the FailoverAppender < > https://logging.apache.org/log4j/2.x/manual/appenders. > html#FailoverAppender> for a generic failover mechanism, but I don't see > the harm in including a list of hosts to use (it's how some other appenders > work due to their drivers anyways like Cassandra and Kafka). > > On 6 May 2017 at 13:30, Volkan Yazıcı <[email protected]> wrote: > >> Hello, >> >> To the best of my knowledge, there is no Redis support in Log4j 2.x NoSQL >> Appenders. (Please correct if I'm wrong.) I want to create a JIRA ticket >> and start working on a RedisAppender. Though I do have some questions and >> will appreciate your feedback. >> >> 1. I plan to use Jedis <https://github.com/xetorthio/jedis> for client >> and perform no shading/relocation. Objections? >> 2. I want to introduce throttling via Guava's RateLimiter. Is that ok? >> 3. I am thinking of a background worker Thread periodically polling an >> ArrayBlockingQueue with a fixed size. This will help me to facilitate >> the >> following features: >> 1. Queue will act as a buffer addressing temporary produce-consume >> rate mismatch. >> 2. Background thread will create batches out of the queue, rather >> than individual inserts. >> 3. Periodic polling will ensure that we avoid starvation when not >> enough messages for a batch. >> 4. I will allow comma separated multiple <host>:<port> pairs to allow >> fail-over. That is, I will create a Jedis client for each <host>:<port> >> pair and try them in given order in case of a failure. >> >> I'm planning to make a pitch on Monday to have some *company time* for >> this >> story. So please do let me know of what you think. >> >> Best. >> > > > > -- > Matt Sicker <[email protected]> > -- Matt Sicker <[email protected]>
