Hi, There is some improvement on the ES7 client so may be we could update the ES appender.
Another solution is that we could detect slow down ingestion and add a temporization but I also like the bulk usage :) regards, François [email protected] Le 18/02/2020 à 09:04, Jean-Baptiste Onofre a écrit : > Hi, > > That’s a good question ;) > > Currently, Decanter Elasticsearch appender sends each event followed by a > refresh. > > It directly uses Elasticsearch REST native client via performRequest. > > A possible improvement is to use bulk but it means that we might have some > delay between when the event actually occurs and when it’s stored in > Elasticsearch. Using bulk would dramatically decrease network usage. > I was thinking to introduce a property on the appender to allow user to have > the choice between bulk or atomic Elasticsearch update. > > Regards > JB > >> Le 18 févr. 2020 à 08:01, dschulten <[email protected]> a écrit : >> >> Hi, >> >> I have combined >> >> decanter-collector-log >> decanter-appender-elasticsearch-rest >> >> and decanter sends the log events to my elasticsearch nicely. However, I am >> a bit concerned about network traffic and blocking rest requests. I am not >> entirely sure, but the code just seems to make blocking http requests to ES. >> >> The ES client benchmark >> https://www.elastic.co/de/blog/benchmarking-rest-client-transport-client >> shows that up to 30.000 ingestions/s could be doable under optimal >> circumstances, that seems not all that much. >> >> How does the ES Rest Appender send the log events? What if ES ingestion >> slows down? >> Does the Rest Appender do anything to cope with heavy load on the network or >> reduce the call frequency? >> Are there any advisable strategies which have been proven to work? >> >> Best >> Dietrich >> >> >> >> -- >> Sent from: http://karaf.922171.n3.nabble.com/Karaf-User-f930749.html
