Hi Daniel, doc values don't work for analyzed string fields like "message":
Doc values are supported on almost all field types, with the notable > exception of analyzed string fields. > Unfortunately that's exactly the field which trips the field data cache circuit breaker in Jason's case. Cheers, Jochen On Thursday, 21 April 2016 13:14:46 UTC+2, Daniel Kamiński wrote: > > you can change 'message' mapping template in ES via it's rest api, and add > `"doc_values": true` to some less needed fields, more info or doc values > here: > https://www.elastic.co/guide/en/elasticsearch/reference/current/doc-values.html > > W dniu czwartek, 21 kwietnia 2016 00:48:57 UTC+2 użytkownik Jason Haar > napisał: >> >> Hi there >> >> I tried to do what I thought was a simple search across a week's worth of >> data on a single-box graylog server (ie it also has ES and mongodb on it) >> >> Basically I did a search for "fieldname:value1 OR fieldname:value2 OR >> fieldname:value3" over 7days and graylog just sat there spinning it's >> wheels (before hand I was happily doing searches that weren't causing any >> grief at all) >> >> The CPU on the graylog server went through the roof, graylog error file >> showed no problem, but ES logs showed a bunch of these >> >> indices.breaker.fielddata] [fielddata] New used memory 11155918063 >> [10.3gb] for data of [message] would be larger than configured breaker: >> 10857952051 [10.1gb], breaking >> >> After five minutes of graylog just sitting there, I restarted ES, but >> graylog was now borked. The input channels were still receiving data, but >> nothing was flowing out. So I restarted graylog and all was good again >> >> Is this expected behaviour, and if so, what is needed to stop it? I've >> seen other non-graylog related postings on the ES list about this happening >> with large clusters, so it seems to be an error case for ES, but I'm more >> concerned over how graylog reacted: ie why didn't it give up and give me an >> error page for starters. It looks to me like graylog didn't expect that ES >> search to error out and that caused it to block? (I'm assuming ES generated >> an error - the logs shows that WARN - I dunno what happens next) >> >> >> -- >> Cheers >> >> Jason Haar >> Information Security Manager, Trimble Navigation Ltd. >> Phone: +1 408 481 8171 >> PGP Fingerprint: 7A2E 0407 C9A6 CAF6 2B9F 8422 C063 5EBB FE1D 66D1 >> > -- You received this message because you are subscribed to the Google Groups "Graylog Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/graylog2/a2aee280-56e8-4705-8c7b-4edd39a0d5a8%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
