Thanks for your help, Jochen. as I'm new to ElasticSearch and Graylog, deleting data without understanding is not safe, I guess. but... seems like there's no solution for it... I'll try it. :)
Thanks. On Thursday, November 3, 2016 at 8:22:26 PM UTC+9, Jochen Schalanda wrote: > > Hi Eugene, > > once you've identified the duplicate documents in Elasticsearch, you can > remove these using the Elasticsearch Delete API ( > https://www.elastic.co/guide/en/elasticsearch/reference/2.4/docs-delete.html) > or the Delete By Query plugin ( > https://www.elastic.co/guide/en/elasticsearch/plugins/2.4/delete-by-query-usage.html > ). > > Cheers, > Jochen > > On Thursday, 3 November 2016 11:54:49 UTC+1, Eugene Gwon wrote: >> >> Hello, Jochen. >> >> I've checked that link but unfortunately, I'm new to ElasticSearch. and >> It seems like that link only shows 'how to check duplicate data'. >> does it mean that I insert data first and check duplicate, and than >> delete duplicated data? >> >> I'm confusing :( >> I'm really sorry but would you please explain more detail? >> >> Thanks for your help, Jochen. >> >> On Thursday, November 3, 2016 at 7:08:29 PM UTC+9, Jochen Schalanda wrote: >>> >>> Hi Eugene, >>> >>> Graylog itself doesn't filter out "duplicate" data. >>> >>> You could check your Elasticsearch indices using the approach laid out >>> in https://qbox.io/blog/minimizing-document-duplication-in-elasticsearch >>> after ingesting the missing data. >>> >>> Cheers, >>> Jochen >>> >>> On Thursday, 3 November 2016 09:48:42 UTC+1, Eugene Gwon wrote: >>>> >>>> Hello all, >>>> I have a problem. please help :( >>>> I'm running 3 node of Elasticsearch. 1 master + data, and 2 data node. >>>> log shipper is fluentd. >>>> Unfortunately, today, I accidentally restarted graylog server. >>>> this caused lost of some data, which sent from another servers. >>>> >>>> so I must re-insert data, but WITHOUT duplication. >>>> Is it possible? >>>> >>>> or, can I just filtering duplicated data when search? >>>> >>>> Thanks for your help. >>>> >>>> -- You received this message because you are subscribed to the Google Groups "Graylog Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/graylog2/1972f102-93e7-4c18-8359-78da7fabc05e%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
