Isn't Kafka like a lot faster than HTTP? On 25 November 2015 at 02:47, Mikael Ståldal <mikael.stal...@magine.com> wrote:
> I am using AsyncAppender to wrap KafkaAppender, it works fine. Consider > setting blocking="false" and shutdownTimeout on the AsyncAppender. > > I have not tried async loggers, it might work as well. > > I would recommend to always wrap appenders which make synchronous networks > requests with AsyncAppender or async logger. > > On Tue, Nov 24, 2015 at 7:39 PM, Matt Sicker <boa...@gmail.com> wrote: > > > Would using an async logger combine well with an HTTP appender like that? > > Because making a REST API for this sort of thing really sounds like a > > bottleneck. > > > > On 24 November 2015 at 02:42, Mikael Ståldal <mikael.stal...@magine.com> > > wrote: > > > > > It is good if you can have one component less in your system. > > > > > > On Mon, Nov 23, 2015 at 8:07 PM, David KOCH <dk...@ezakus.com> wrote: > > > > > > > There exists a full-fledged Java API for Elasticsearch. Still, I am > not > > > > sure it's the best idea to lump all the intricacies of efficient > > indexing > > > > into ES into a log4j2 appender. Logstash does this well - there is a > > > reason > > > > it is part of the Elastic family. > > > > > > > > /David > > > > > > > > On 23 November 2015 at 18:32, Matt Sicker <boa...@gmail.com> wrote: > > > > > > > > > ElasticSearch has a REST API for adding data which seems to be a > > rather > > > > > inefficient way to implement an appender. > > > > > > > > > > On 19 November 2015 at 13:36, Matt Sicker <boa...@gmail.com> > wrote: > > > > > > > > > > > I just spoke with my friend who's pretty familiar with both Solr > > and > > > > > > ElasticSearch, and he points out that Solr usage might be a bit > > more > > > > > > convoluted due to how rigidly defined search indexes are compared > > to > > > > ES. > > > > > > It'd be like the difference between SQL and NoSQL. > > > > > > > > > > > > On 19 November 2015 at 12:28, Ralph Goers < > > > ralph.go...@dslextreme.com> > > > > > > wrote: > > > > > > > > > > > >> The only problem I have is with the number of integrations we > > could > > > > do. > > > > > >> ElasticSearch and Solr are similar, and Solr is an Apache > project, > > > so > > > > I > > > > > >> would expect we would want to support both if we support either. > > > > > >> > > > > > >> Ralph > > > > > >> > > > > > >> > On Nov 19, 2015, at 10:07 AM, Gary Gregory < > > > garydgreg...@gmail.com> > > > > > >> wrote: > > > > > >> > > > > > > >> > Amazon has a pretty rich ecosystem of stuff. Should this be > in a > > > new > > > > > >> > log4j-amazon module or in the core module? Does it fit in > > no-sql? > > > > > >> > > > > > > >> > Gary > > > > > >> > On Nov 19, 2015 6:29 AM, "Matt Sicker" <boa...@gmail.com> > > wrote: > > > > > >> > > > > > > >> >> An ElasticSearchAppender does sound useful. We use Log4j2 in > > all > > > > our > > > > > >> >> projects, so we don't really need to add middleware unless > > > > absolutely > > > > > >> >> necessary. > > > > > >> >> > > > > > >> >> On 19 November 2015 at 07:35, Mikael Ståldal < > > > > > >> mikael.stal...@magine.com> > > > > > >> >> wrote: > > > > > >> >> > > > > > >> >>> At my previous work, we first used Log4j 2 and Graylog, and > > > > Logstash > > > > > >> in > > > > > >> >>> between. But then I developed GelfLayout (now part of > > > log4j-core) > > > > > and > > > > > >> we > > > > > >> >>> could configure Log4j 2 to send log events directly to > > Graylog, > > > > and > > > > > >> >>> Logstash became unnecessary. > > > > > >> >>> > > > > > >> >>> Can we do something similar with ELK (so that it become just > > > > "E-K")? > > > > > >> Can > > > > > >> >> we > > > > > >> >>> develop an ElasticSearchAppender for Log4j 2? > > > > > >> >>> > > > > > >> >>> > > > > > >> >>> On Thu, Nov 19, 2015 at 1:07 PM, Mikael Ståldal < > > > > > >> >> mikael.stal...@magine.com > > > > > >> >>>> > > > > > >> >>> wrote: > > > > > >> >>> > > > > > >> >>>> Is is even necessary to add Logstash to the mix? I think > that > > > > > Log4j 2 > > > > > >> >>>> should be able to whatever is necessary by itself without > > > either > > > > > >> Flume > > > > > >> >> or > > > > > >> >>>> Logstash. > > > > > >> >>>> > > > > > >> >>>> > > > > > >> >>>> On Wed, Nov 18, 2015 at 10:56 PM, David KOCH < > > dk...@ezakus.com > > > > > > > > > >> wrote: > > > > > >> >>>> > > > > > >> >>>>> We use log4j2's SyslogAppender and a logstash UDP input > > > source, > > > > > >> works > > > > > >> >> ok > > > > > >> >>>>> for us, no need to add Flume to the mix. > > > > > >> >>>>> > > > > > >> >>>>> /David > > > > > >> >>>>> > > > > > >> >>>>> On 18 November 2015 at 22:35, Matt Sicker < > boa...@gmail.com > > > > > > > > wrote: > > > > > >> >>>>> > > > > > >> >>>>>> Any experience with this? I was thinking of using Flume > and > > > the > > > > > >> >> Flume > > > > > >> >>>>>> appender to route all my log messages to an ELK stack, > but > > I > > > > was > > > > > >> >>>>> wondering > > > > > >> >>>>>> if there were other ways of doing this. > > > > > >> >>>>>> > > > > > >> >>>>>> -- > > > > > >> >>>>>> Matt Sicker <boa...@gmail.com> > > > > > >> >>>>>> > > > > > >> >>>>> > > > > > >> >>>> > > > > > >> >>>> > > > > > >> >>>> > > > > > >> >>>> -- > > > > > >> >>>> [image: MagineTV] > > > > > >> >>>> > > > > > >> >>>> *Mikael Ståldal* > > > > > >> >>>> Senior software developer > > > > > >> >>>> > > > > > >> >>>> *Magine TV* > > > > > >> >>>> mikael.stal...@magine.com > > > > > >> >>>> Grev Turegatan 3 | 114 46 Stockholm, Sweden | > > > www.magine.com > > > > > >> >>>> > > > > > >> >>>> Privileged and/or Confidential Information may be contained > > in > > > > this > > > > > >> >>>> message. If you are not the addressee indicated in this > > message > > > > > >> >>>> (or responsible for delivery of the message to such a > > person), > > > > you > > > > > >> may > > > > > >> >>> not > > > > > >> >>>> copy or deliver this message to anyone. In such case, > > > > > >> >>>> you should destroy this message and kindly notify the > sender > > by > > > > > reply > > > > > >> >>>> email. > > > > > >> >>>> > > > > > >> >>> > > > > > >> >>> > > > > > >> >>> > > > > > >> >>> -- > > > > > >> >>> [image: MagineTV] > > > > > >> >>> > > > > > >> >>> *Mikael Ståldal* > > > > > >> >>> Senior software developer > > > > > >> >>> > > > > > >> >>> *Magine TV* > > > > > >> >>> mikael.stal...@magine.com > > > > > >> >>> Grev Turegatan 3 | 114 46 Stockholm, Sweden | > > > www.magine.com > > > > > >> >>> > > > > > >> >>> Privileged and/or Confidential Information may be contained > in > > > > this > > > > > >> >>> message. If you are not the addressee indicated in this > > message > > > > > >> >>> (or responsible for delivery of the message to such a > person), > > > you > > > > > may > > > > > >> >> not > > > > > >> >>> copy or deliver this message to anyone. In such case, > > > > > >> >>> you should destroy this message and kindly notify the sender > > by > > > > > reply > > > > > >> >>> email. > > > > > >> >>> > > > > > >> >> > > > > > >> >> > > > > > >> >> > > > > > >> >> -- > > > > > >> >> Matt Sicker <boa...@gmail.com> > > > > > >> >> > > > > > >> > > > > > >> > > > > > >> > > > > > >> > > > --------------------------------------------------------------------- > > > > > >> To unsubscribe, e-mail: > log4j-user-unsubscr...@logging.apache.org > > > > > >> For additional commands, e-mail: > > log4j-user-h...@logging.apache.org > > > > > >> > > > > > >> > > > > > > > > > > > > > > > > > > -- > > > > > > Matt Sicker <boa...@gmail.com> > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > Matt Sicker <boa...@gmail.com> > > > > > > > > > > > > > > > > > > > > > -- > > > [image: MagineTV] > > > > > > *Mikael Ståldal* > > > Senior software developer > > > > > > *Magine TV* > > > mikael.stal...@magine.com > > > Grev Turegatan 3 | 114 46 Stockholm, Sweden | www.magine.com > > > > > > Privileged and/or Confidential Information may be contained in this > > > message. If you are not the addressee indicated in this message > > > (or responsible for delivery of the message to such a person), you may > > not > > > copy or deliver this message to anyone. In such case, > > > you should destroy this message and kindly notify the sender by reply > > > email. > > > > > > > > > > > -- > > Matt Sicker <boa...@gmail.com> > > > > > > -- > [image: MagineTV] > > *Mikael Ståldal* > Senior software developer > > *Magine TV* > mikael.stal...@magine.com > Grev Turegatan 3 | 114 46 Stockholm, Sweden | www.magine.com > > Privileged and/or Confidential Information may be contained in this > message. If you are not the addressee indicated in this message > (or responsible for delivery of the message to such a person), you may not > copy or deliver this message to anyone. In such case, > you should destroy this message and kindly notify the sender by reply > email. > -- Matt Sicker <boa...@gmail.com>
