Thanks, Radu. Cronjob seems to be the way to purge all data in elasticsearch. Thanks for the links as well.
On Mon, Nov 23, 2015 at 12:07 AM, Radu Gheorghe <[email protected]> wrote: > Hi Alec, > > I assume you're looking to remove old data from Elasticsearch > automatically. If so, then rsyslog is not the tool for the job. But I > see two options: > - the bad one: use the _ttl field: > > https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping-ttl-field.html > > you can see that even in the docs (besides the fact that _ttl is > deprecated) it recommends using time-based indices instead. And you > seem to already have them, which leads me to the second option: > > - remove old time-based indices. The common way of doing this is via > Elasticsearch Curator: > > https://www.elastic.co/guide/en/elasticsearch/client/curator/current/installation.html > > Curator is a command-line tool that you can use in your cron to remove > old indices based on whatever rules you want. > > I hope this helps. If you want a more complete tutorial about managing > the Elasticsearch side of things, we had a a rather detailed > presentation about it recently - you can check the videos, slides and > commands here: > http://blog.sematext.com/2015/10/13/log-analysis-with-elasticsearch/ > > Best regards, > Radu > -- > Performance Monitoring * Log Analytics * Search Analytics > Solr & Elasticsearch Support * http://sematext.com/ > > > On Sat, Nov 21, 2015 at 9:27 PM, Alec Swan <[email protected]> wrote: > > Hello, > > > > Does anybody know how I can specify TTL for elasticsearch index? This is > > what I currently have. > > > > template(name = "logstash-index" type="list") { > > > > constant(value = "logstash-") > > > > property(name = "timereported" dateFormat="rfc3339" position.from="1" > > position.to="4") > > > > constant(value = ".") > > > > property(name = "timereported" dateFormat="rfc3339" position.from="6" > > position.to="7") > > > > constant(value = ".") > > > > property(name = "timereported" dateFormat="rfc3339" position.from="9" > > position.to="10") > > > > } > > > > action( > > > > type = "omelasticsearch" > > > > template = "es-payload" > > > > dynSearchIndex = "on" > > > > searchIndex = "logstash-index" > > > > searchType = "cassandra" > > > > server = "h1.lab.ppops.net" > > > > serverport = "9200" > > > > errorFile = "/var/log/rsyslog/ES-error.log" > > > > bulkmode = "on" > > > > action.resumeretrycount = "-1" > > > > ) > > > > > > Thanks, > > > > > > Alec > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com/professional-services/ > > What's up with rsyslog? Follow https://twitter.com/rgerhards > > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you > DON'T LIKE THAT. > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > What's up with rsyslog? Follow https://twitter.com/rgerhards > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you > DON'T LIKE THAT. > _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
