I don't think it's possible to have graylog pull messages back out of elasticsearch and re-create extractors. The extractors process messages as they arrive through the inputs, very early in the processing, and even before streams are processed. Once the message has left the input and moved on, you can't add any more fields to it.
It would be nice if one could do this, somehow. Perhaps there is a tool that can do it at the elasticsearch level? On Wed, Feb 3, 2016 at 2:03 PM, Roger Guzman <[email protected]> wrote: > Ok, I have seen some interesting behavior in relation to the extractors: > The fields begin to appear in the messages received after the moment of > creation exhaust, therefore no previous messages have new fields. Now I > will look how to make the extractors work for older messages. > > I attached capture messages to see this behavior > ---- > Ok, he visto un comportamiento interesante en relación a los extractores: > Los campos empiezan a mostrarse en los mensajes recibidos luego del momento > de la creación del extractor, por ende los mensajes anteriores no poseen > los nuevos campos. Ahora buscaré como hacer que los extractores funcionen > para mensajes antiguos. > > Les adjunto la captura de los mensajes para que vean este comportamiento > > > El miércoles, 3 de febrero de 2016, 15:03:06 (UTC-4:30), Roger Guzman > escribió: >> >> All extractors have used work. By clicking on the "Try" button (creating >> the extractor) show matching red. As I mentioned earlier and earlier had >> solved this problem through the web interface in another implementation but >> forgot how to do it: '( >> >> I keep looking through the web interface and will comment any advance >> ------- >> Todos los extractores que he usado funcionan. Al hacer click en el botón >> "Try" (creando el extractor) muestran la coincidencia en rojo. Como comenté >> anteriormente ya había resuelto este problema desde la interfaz web en >> otra implementación pero olvidé como hacerlo :'( >> >> >> Seguiré buscando desde la interfaz web y les comentaré cualquier adelanto >> >> >> El miércoles, 3 de febrero de 2016, 14:39:22 (UTC-4:30), Joi Owen >> escribió: >>> >>> Ok, earlier in the thread you had a screenshot of the field creation >>> form, but the match shown was in red, not in green. It needs to be green >>> before it will create the extractor field. So, I don't think the extractor >>> is actually working as you intended. There is more to the form below the >>> portion you captured, and I'm thinking one of those other control fields is >>> causing the problem. IE there is a second field that is sort of a master >>> control match to decide if your more complex regexp will be applied at >>> all. IE I have it set to "apply this extractor only if message contains >>> 'dhcp' in it." That would prevent the extractor from matching even if the >>> regexp to extract the IP address from the message was otherwise correct, >>> but my test message from a radius server instead. >>> >>> >>> >>> On Wed, Feb 3, 2016 at 1:03 PM, Roger Guzman <[email protected]> >>> wrote: >>> >>>> I used the scroll my mouse is not the problem :D >>>> The problem is that when creating the extractor, not shown the field in >>>> the search, even when the message displayed is that was used to generate >>>> the extractor. >>>> ----- >>>> He usado el scroll, mi mouse no es el problema :D >>>> El problema es que al crear el extractor, no se muestra el campo en el >>>> "search tab", aun cuando el mensaje que se muestra es el que sirvió para >>>> generar el extractor. >>>> >>>> El miércoles, 3 de febrero de 2016, 14:26:12 (UTC-4:30), Joi Owen >>>> escribió: >>>>> >>>>> I'm not sure I understand the issue, but... >>>>> >>>>> System/Input/Manage Extractors is where you create the rules that >>>>> create new fields, and where you can manage existing fields. >>>>> >>>>> The image you sent is from a search result, and the list of fields on >>>>> the left is the list of fields which have already been extracted for the >>>>> message you are viewing. If a field is missing there, then there is an >>>>> issue with the extractor defined on that message's input. You can use the >>>>> 'create extractor' item on the right which you circled to create a new >>>>> extractor using that message as a sample. >>>>> >>>>> Alternatively, you can identify the input the message arrived from, >>>>> and use the 'manage extractors' button on the System/Inputs display to >>>>> create a new extractor. >>>>> >>>>> The list of fields on the left has a scroll bar on the right edge, did >>>>> you overlook that scroll bar? >>>>> >>>>> Did I understand your question? >>>>> >>>>> >>>>> On Wed, Feb 3, 2016 at 12:36 PM, Roger Guzman <[email protected]> >>>>> wrote: >>>>> >>>>>> The search result displays, where the "Create extractor for field >>>>>> message" submenu is deployed (Attached image). >>>>>> >>>>>> Previously this had happened to me (in another implemntación of >>>>>> Graylog) and resolved forcing the burden of fields / extractor from the >>>>>> web >>>>>> interface but the truth is I do not remember where did this :S >>>>>> ---------- >>>>>> El resultado de la búsqueda muestra el mensaje desde donde se >>>>>> desplegó el submenú "Create extractor for field message (imagen >>>>>> adjuntada). >>>>>> >>>>>> Previously this had happened to me (in another implementation of >>>>>> Graylog) and resolved forcing the burden of fields/extractor from the web >>>>>> interface but the truth is I do not remember where did this :S >>>>>> >>>>>> El miércoles, 3 de febrero de 2016, 13:37:30 (UTC-4:30), Joi Owen >>>>>> escribió: >>>>>>> >>>>>>> I've seen this happen to me a few times, and its usually one of the >>>>>>> following: >>>>>>> >>>>>>> *) the search result I'm currently looking at doesn't contain any >>>>>>> messages that would have matched those fields. The search tool only >>>>>>> lists >>>>>>> fields that appear in the current search result. IE, if I do a search >>>>>>> for >>>>>>> 'dhcp' and then look at the search tool, the dhcp-related fields are >>>>>>> there, >>>>>>> but the radius-related fields are not. If I do a search for '*' then >>>>>>> the >>>>>>> fields are all there as long as each field appears at least once in the >>>>>>> result set. >>>>>>> >>>>>>> *) the extracted variables are not on the input through which that >>>>>>> set of messages arrived. My graylog has 6 different inputs (2 gelf, 2 >>>>>>> tcp, >>>>>>> 2 udp) and I have to place the desired extractor on all inputs where >>>>>>> that >>>>>>> content might arrive. Find a message that should have displayed such >>>>>>> fields, and compare the input it arrived on with the input where you >>>>>>> created the extractor, and they're often different (because some admin >>>>>>> decided to change their syslog configuration and are now sending their >>>>>>> messages to an unexpected input.) >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> On Wed, Feb 3, 2016 at 10:06 AM, Roger Guzman <[email protected]> >>>>>>> wrote: >>>>>>> >>>>>>>> I have created several extractors and the same work correctly >>>>>>>> (attached image), but the fields created are not shown in the search >>>>>>>> tab. Has anyone had the same problem? >>>>>>>> --- >>>>>>>> He creado varios extractores y los mismos funcionan correctamente >>>>>>>> (adjunto la imagen), pero los campos creados no se muestran en el tab >>>>>>>> search. ¿Alguien ha tenido el mismo inconveniente? >>>>>>>> >>>>>>>> -- >>>>>>>> You received this message because you are subscribed to the Google >>>>>>>> Groups "Graylog Users" group. >>>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>>> send an email to [email protected]. >>>>>>>> To view this discussion on the web visit >>>>>>>> https://groups.google.com/d/msgid/graylog2/d35609e2-d654-4549-bb07-b945c6cf0945%40googlegroups.com >>>>>>>> <https://groups.google.com/d/msgid/graylog2/d35609e2-d654-4549-bb07-b945c6cf0945%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>>>>> . >>>>>>>> For more options, visit https://groups.google.com/d/optout. >>>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> >>>>>>> No matter what we think of Linux versus FreeBSD, etc., the one thing >>>>>>> I >>>>>>> really like about Linux is that it has Microsoft worried. Anything >>>>>>> that kicks a monopoly in the pants has got to be good for something. >>>>>>> - Chris Johnson >>>>>>> >>>>>>> -- >>>>>> You received this message because you are subscribed to the Google >>>>>> Groups "Graylog Users" group. >>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>> send an email to [email protected]. >>>>>> To view this discussion on the web visit >>>>>> https://groups.google.com/d/msgid/graylog2/a0cd90b9-359c-456f-b3dc-21acd42ac1b7%40googlegroups.com >>>>>> <https://groups.google.com/d/msgid/graylog2/a0cd90b9-359c-456f-b3dc-21acd42ac1b7%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>>> . >>>>>> >>>>>> For more options, visit https://groups.google.com/d/optout. >>>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> >>>>> No matter what we think of Linux versus FreeBSD, etc., the one thing I >>>>> really like about Linux is that it has Microsoft worried. Anything >>>>> that kicks a monopoly in the pants has got to be good for something. >>>>> - Chris Johnson >>>>> >>>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "Graylog Users" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to [email protected]. >>>> To view this discussion on the web visit >>>> https://groups.google.com/d/msgid/graylog2/19a30007-2dd2-4f5e-b1a6-4cfcb65bb391%40googlegroups.com >>>> <https://groups.google.com/d/msgid/graylog2/19a30007-2dd2-4f5e-b1a6-4cfcb65bb391%40googlegroups.com?utm_medium=email&utm_source=footer> >>>> . >>>> >>>> For more options, visit https://groups.google.com/d/optout. >>>> >>> >>> >>> >>> -- >>> >>> No matter what we think of Linux versus FreeBSD, etc., the one thing I >>> really like about Linux is that it has Microsoft worried. Anything >>> that kicks a monopoly in the pants has got to be good for something. >>> - Chris Johnson >>> >>> -- > You received this message because you are subscribed to the Google Groups > "Graylog Users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/graylog2/143f9604-bcf9-4ef7-b194-ae1c6d8297ce%40googlegroups.com > <https://groups.google.com/d/msgid/graylog2/143f9604-bcf9-4ef7-b194-ae1c6d8297ce%40googlegroups.com?utm_medium=email&utm_source=footer> > . > > For more options, visit https://groups.google.com/d/optout. > -- No matter what we think of Linux versus FreeBSD, etc., the one thing I really like about Linux is that it has Microsoft worried. Anything that kicks a monopoly in the pants has got to be good for something. - Chris Johnson -- You received this message because you are subscribed to the Google Groups "Graylog Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/graylog2/CAL5rfGVNbqK6SbgnOQN3dE6X8ZT3TsJW%3DwLLHWwb8yutEA3Ghg%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
