You're welcome, Alec! :) -- Performance Monitoring * Log Analytics * Search Analytics Solr & Elasticsearch Support * http://sematext.com/
On Mon, Nov 23, 2015 at 10:40 PM, Alec Swan <[email protected]> wrote: > Thanks, Radu. Cronjob seems to be the way to purge all data in > elasticsearch. Thanks for the links as well. > > On Mon, Nov 23, 2015 at 12:07 AM, Radu Gheorghe <[email protected]> > wrote: > >> Hi Alec, >> >> I assume you're looking to remove old data from Elasticsearch >> automatically. If so, then rsyslog is not the tool for the job. But I >> see two options: >> - the bad one: use the _ttl field: >> >> https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping-ttl-field.html >> >> you can see that even in the docs (besides the fact that _ttl is >> deprecated) it recommends using time-based indices instead. And you >> seem to already have them, which leads me to the second option: >> >> - remove old time-based indices. The common way of doing this is via >> Elasticsearch Curator: >> >> https://www.elastic.co/guide/en/elasticsearch/client/curator/current/installation.html >> >> Curator is a command-line tool that you can use in your cron to remove >> old indices based on whatever rules you want. >> >> I hope this helps. If you want a more complete tutorial about managing >> the Elasticsearch side of things, we had a a rather detailed >> presentation about it recently - you can check the videos, slides and >> commands here: >> http://blog.sematext.com/2015/10/13/log-analysis-with-elasticsearch/ >> >> Best regards, >> Radu >> -- >> Performance Monitoring * Log Analytics * Search Analytics >> Solr & Elasticsearch Support * http://sematext.com/ >> >> >> On Sat, Nov 21, 2015 at 9:27 PM, Alec Swan <[email protected]> wrote: >> > Hello, >> > >> > Does anybody know how I can specify TTL for elasticsearch index? This is >> > what I currently have. >> > >> > template(name = "logstash-index" type="list") { >> > >> > constant(value = "logstash-") >> > >> > property(name = "timereported" dateFormat="rfc3339" position.from="1" >> > position.to="4") >> > >> > constant(value = ".") >> > >> > property(name = "timereported" dateFormat="rfc3339" position.from="6" >> > position.to="7") >> > >> > constant(value = ".") >> > >> > property(name = "timereported" dateFormat="rfc3339" position.from="9" >> > position.to="10") >> > >> > } >> > >> > action( >> > >> > type = "omelasticsearch" >> > >> > template = "es-payload" >> > >> > dynSearchIndex = "on" >> > >> > searchIndex = "logstash-index" >> > >> > searchType = "cassandra" >> > >> > server = "h1.lab.ppops.net" >> > >> > serverport = "9200" >> > >> > errorFile = "/var/log/rsyslog/ES-error.log" >> > >> > bulkmode = "on" >> > >> > action.resumeretrycount = "-1" >> > >> > ) >> > >> > >> > Thanks, >> > >> > >> > Alec >> > _______________________________________________ >> > rsyslog mailing list >> > http://lists.adiscon.net/mailman/listinfo/rsyslog >> > http://www.rsyslog.com/professional-services/ >> > What's up with rsyslog? Follow https://twitter.com/rgerhards >> > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad >> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >> DON'T LIKE THAT. >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com/professional-services/ >> What's up with rsyslog? Follow https://twitter.com/rgerhards >> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad >> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >> DON'T LIKE THAT. >> > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > What's up with rsyslog? Follow https://twitter.com/rgerhards > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of > sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T > LIKE THAT. _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
