Between getting sick on Friday and becoming fire marshall bill last week... I haven't had any time to work on this. The biggest issue is that I'm not at all familiar with logstash, so I need to figure out the filtering stuff. I still believe the idea is sound; its just a matter of time. I'm most likely going to shelf the reindex portion of our upgrade for the time being and circle back to that when I have a little more time. I'll hit you up with details when I have some though.
On Thursday, June 2, 2016 at 1:12:27 PM UTC-4, Jimmy Chen wrote: > > Good luck with the fires and I'll check back to see how it went. > > On Thursday, June 2, 2016 at 6:03:34 AM UTC-7, Robert Hough wrote: >> >> Well, "out of the box", no that didn't work. I've got faith that it can >> be done using this approach, but we'll also need to utilize Elastic's >> "de_dot" filter plugin. I'm hoping to make some progress with that today, >> and I'll provide an update by the end of the day. I've got about 10 fires >> to put out first... :( >> >> Here's the link to the de_dot documentation: >> >> https://www.elastic.co/guide/en/logstash/current/plugins-filters-de_dot.html >> >> >> In a nutshell: >> >> 1) Logstash pulls in old index data from old ES cluster >> 2) Logstash sends that through filter >> 1a) Match any dots in fields (user.id) >> 2a) Add new field as replacement for old field (user.id == user_id) >> 3a) Populate user_id with value from user.id >> 4a) remove old field (user.id) >> 3) Logstash pushes new index data to new ES cluster >> >> I'm sure I've left out something crucial here. Seems to be par for the >> course, but I'm hopeful. :) >> >> >> >> >> On Wednesday, June 1, 2016 at 3:06:34 PM UTC-4, Jimmy Chen wrote: >>> >>> Did this work for you? I am going to be looking into upgrading our >>> existing cluster to 2.x too. >>> >>> On Tuesday, May 31, 2016 at 5:08:05 PM UTC-7, Robert Hough wrote: >>>> >>>> Came across this: >>>> https://gist.github.com/markwalkom/8a7201e3f6ea4354ae06 >>>> <https://www.google.com/url?q=https%3A%2F%2Fgist.github.com%2Fmarkwalkom%2F8a7201e3f6ea4354ae06&sa=D&sntz=1&usg=AFQjCNE1J3mT8QvKd3suG3jqyBKPZYCGng> >>>> >>>> third time's the charm? :) >>>> >>>> >>>> On Friday, May 27, 2016 at 4:43:18 PM UTC-4, Robert Hough wrote: >>>>> >>>>> Recently built a Graylog 2.x cluster, and that seems to be working >>>>> fine. I had some questions though, but right now the biggest nagging >>>>> question has been... >>>>> >>>>> How do we migrate our existing indexes over to the new system? The >>>>> whole dots in field names issue seems to be what is preventing us from >>>>> pulling this off. How do we correct these, and then import them into the >>>>> our new system? >>>>> >>>> -- You received this message because you are subscribed to the Google Groups "Graylog Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/graylog2/8b100acf-e9b1-4cbc-b00b-571cac245da2%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
