Looking at the documentation, you cannot remove the "message" field, as it is a required field. As far as I know, the only thing you could do with the pipeline processor, is to set the "message" field content to some dummy value, e.g. "discarded", by using the `set_field` function.
If that is good enough for you, the rule you shared looks otherwise fine, with two remarks: - If you use a JSON extractor with data from the "message" field, you need to ensure that the processors are in the right order. Check the "Message Processors Configuration" section under System -> Configurations - Connecting it to "Incoming messages stream" will drop the "message" field on all messages that are not routed into any stream I hope that helps. Regards, Edmundo > On 10 Jun 2016, at 12:43, nimmie <[email protected]> wrote: > > Hi, > to be honest I tried the pipelines first, but without positive result. > May be I am missing something. > > My pipeline setup: > 1. rule definition > rule "remove" > when > has_field("message") > then > remove_field("message"); > end > > 2. rule linked with new pipeline (stage 0) and the pipeline linked with > "Incoming messages stream". > Is that OK? > > I suppose no graylog restart is needed. How can I make troubleshooting (check > correct work) of this setup? > > Thanks. > > Nimmie > > > On Friday, June 10, 2016 at 11:21:18 AM UTC+2, Edmundo Alvarez wrote: > Hi, > > I know it's not what you asked for, but I think using the new pipeline > processor can help you with that. You can find it in Graylog 2.0, if you want > to try it. Here's some documentation: > http://docs.graylog.org/en/2.0/pages/pipelines.html > > Regards, > Edmundo > > > On 10 Jun 2016, at 10:41, nimmie <[email protected]> wrote: > > > > Hi, > > I would like to ask you how to deal with $SUBJ? > > > > Additional question marks: > > a. I need something like logstash mutate -> remove_field functionality: > > feasible in Graylog? > > b. after some examination: is the Graylog correct/only tool "drools"? > > (pipeline procesing still marked as experimental) > > c. what is exact processing order in Graylog (from my issue point of view): > > 1. input filter with extractor (created via GUI) 2. drools file *.drl? > > d. where is the best starting point for learning drools? not the link in > > graylog docs, but where can I find e.g. list of "methods" (e.g. > > "add.Field")? > > > > My current set-up: > > 1. local file with logs > > 2. logstah for file processing (LS sending json to RabbitMQ) > > 3. RabbiMQ > > 4. Graylog (input with RawAMPQ): used JSON extractor - working, but I need > > to remove "message" field (remove data duplication) > > > > Thanks in advance. > > nimmie > > > > > > -- > > You received this message because you are subscribed to the Google Groups > > "Graylog Users" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [email protected]. > > To view this discussion on the web visit > > https://groups.google.com/d/msgid/graylog2/b49e3a46-3e46-457d-bd04-5cc34942c2dd%40googlegroups.com. > > > > For more options, visit https://groups.google.com/d/optout. > > > -- > You received this message because you are subscribed to the Google Groups > "Graylog Users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/graylog2/e8ad54fa-1ebe-449a-86a5-a95f0b88680b%40googlegroups.com. > For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups "Graylog Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/graylog2/2E822F8F-0440-48FD-8C54-38950F480FF6%40graylog.com. For more options, visit https://groups.google.com/d/optout.
