On Tue, 15 Dec 2015, Ciprian Hacman wrote:
Hi David,
maxMessageSize="10000"
queue.size="10000" - main queue
queue.size="10000" - elasticsearch queue
Based on my calculations this brings me to a max of 200MB of memory, maybe
a little more depending on how maxMessageSize is calculated.
I read logs from a file and push them to Elasticsearch (on the same
network), so TCP is the only possibility. This server has a very simple
setup.
If I don't find the reason for this issue, I might have to go implement the
forwarding to a central location and push to Elasticsearch from there.
There are advantages to sending things to a central server.
it's one place to queue data, so you can either allocate more ram, or go to disk
as needed without impacting other workloads.
it's more efficient, the central server is more likely to have larger batches of
data to feed to ES, and ES only needs to be running one thread processing
inbound data
while it is one more point to have to look at, I think it simplifies
troubleshooting as all the communication to ES (and therefor all the errors for
such communication) happen in one place instead of distributed.
anyway, let's see how things look with 8.15
David Lang
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
