>There is no validation in the im_msvistalog module for these at all. If event >subscription fails, > the module will not start. It is not possible to detect for the >module which > channel (or portion of the xml) is invalid. >All the parsing and validation is handled by the windows code and the logic is >quite convoluted as you can usually expect from microsoft. Thanks for the info. Yeah. I was afraid that the im_msvistalog Channel situation was probably limited by your interaction with Windows.
>Regarding the format required by these directives see the following link: >http://msdn.microsoft.com/en-us/library/aa385231.aspx I had actually looked at that. I'm probably just being dense, but what is not obvious in their examples is how one would use that in a "compound" scenario, which is how most folks want to capture Windows logs, e.g. rarely would someone "only want Security logs". More often, then would prefer to get all of the legacy log types (i.e. Security, System, and Application). But Channel (which seems to be optional, by design) and Query are only allowed to be codified once each, and Channel only allowing one parameter, so it's not possible to do this: Channel Security,System,Application Query * And it's not possible to do this: Channel Security Query * Channel System Query * Etc. Again, except in the limited scenario where a person wants to just get one log Channel, it seems like "Channel" is of little value. The only thing that seems to support multiple Channel extractions is using only the Query directive, with multiple Query's embedded within a single QueryList. I'm not clear how that gets used for the Event Subscription process, but when I codify only the Query (in that manner), I seem to get logs from the Query-designed Channels. (I'm assuming that is the correct solution??) >The im_mseventlog module is a different beast as it uses the old eventlog API. >When you explicitly > specify the sources, it will try to open only those one by one. > When this fails, it will retry again, so you should see errors about Rabbit > not found. FWIW, I certainly saw no failure when nxlog started. Maybe it would have generated some error later on? And I did get logs for the valid Sources. >Spaces might be a problem, will need to test with that. OK. I guess a simpler way for me to have addressed that would have been to simply ask, what do I specify to get "DNS Server" logs (or the AD-related logs) from older Windows systems? Maybe I can spin up some of those services tomorrow, and see if I can figure out what works. >Be careful about commenting out a multi line define. Well "crud". That was actually extremely handy, although I must admit it was different than anything I'd encountered previously. I don't suppose that there are any other undocumented comment mechanisms? It would be handy to be able to comment (uncomment) entire blocks, without having to diddle with leading character on each line. Even something like the old : /* Comments In Here */ Construct would be handy, as it would simplify turning on/off entire sections, all at once. Anyway... thanks for the heads-up on the planned change. That certainly would have bitten me when in the future release, as I was already planning to build some commented "chunks", based upon a single comment character. Thanks for your time. -----Original Message----- From: Botond Botyanszki [mailto:[email protected]] Sent: Sunday, June 30, 2013 10:49 AM To: [email protected] Subject: Re: [nxlog-ce-users] im_msvistalog Channel Query issue Hi, Regarding single vs double quotes this is surprising since it shouldn't matter in xml. The im_msvistalog module uses the EvtSubscribe function internally to subscribe to events, see http://msdn.microsoft.com/en-us/library/aa385487.aspx Basically this function has the ChannelPath and Query parameters which correspond to the Channel and Query config directives. When none of these are specified, nxlog will build a query xml internally from all available channels. Regarding the format required by these directives see the following link: http://msdn.microsoft.com/en-us/library/aa385231.aspx There is no validation in the im_msvistalog module for these at all. If event subscription fails, the module will not start. It is not possible to detect for the module which channel (or portion of the xml) is invalid. All the parsing and validation is handled by the windows code and the logic is quite convoluted as you can usually expect from microsoft. The im_mseventlog module is a different beast as it uses the old eventlog API. When you explicitly specify the sources, it will try to open only those one by one. When this fails, it will retry again, so you should see errors about Rabbit not found. Spaces might be a problem, will need to test with that. Be careful about commenting out a multi line define. There is a bug filed in our tracker to fix that. Currently both lines are ignored and treated as a comment for the following: #line1 \ line2 This is not correct and will be fixed sometime soon so that it works as in other languages (e.g. shell). Regards, Botond On Sun, 30 Jun 2013 14:11:12 +0000 Marvin Nipper <[email protected]> wrote: > Thanks for the quick (weekend) reply. We should BOTH be doing > something else, besides working on our weekends! :-) > > OK, I'm testing on a virgin install of "Windows Edition, Windows Web Server > 2008 R2". Believe it or not, it was all about the choice of "quote". I did > what you suggested, and tried pasting the query directly into Event Viewer. > It immediately complained. > > I realized that some of the windows (web site) examples were all > double-quoted, so I replaced the single-quotes from the nxlog manual, and it > immediately accepted that syntax. So, I'm not sure if this is purely a > documentation issue, or if some other Windows versions (or maybe "fully > patched" Windows version) are more liberal in terms of their string syntax. > > Several other things worth noting, and/or (possibly) documenting: > I had hoped that I could define a query that encompassed everything that I'd > want in a typical log selection, _whether they existed, or not_. I guess I'd > appreciate any thoughts/suggestions that you have on that, because everything > that I'm seeing in the im_msvistalog module seems to indicate that will > simply "not work". I had hoped that if some of the logs did not exist, those > would simply generate a log warning, but everything else would be selected, > but within nxlog, any errors results in no output. (Interestingly, if you > do the same query within Windows, you get the opposite result... i.e. it > generates an error, and then gives you _everything_, from all available > Channels.) Anyway, it's probably worth documenting that the query has to > reference Channels that "absolutely must exist" in the target server, or it > will fail; > > Conversely, within im_mseventlog, the opposite is true. It will literally > accept any values in the Sources directive. E.G. even this is accepted: > Sources Application,Security,System,Rabbit > The main point (worth noting) is that a simple "typo" could prevent > you from getting the desired logs, e.g. accidentally typing "DNS > Service" instead of "DNS Server" _will not result in any warnings_, > but will also not retrieve the desired logs. (And I'm assuming that > the Sources directive properly parses the Log Sources that have spaces > in the name. If there are any syntax-related requirements for those, > e.g. that they need to be quoted, that might also be worth > documenting. I haven't yet built a server that has any of those > sources, so I haven't yet verified how that works.); > > I really am unclear as to how the Channel directive is supposed to be used in > the im_msvistalog module. I had assumed that I could avoid using Query > altogether, if I simply coded something like: > Channel System,Security,Application But that doesn't work at > all (for me). The Channel directive seems to work fine, as long as > there is only _one_ parameter defined, but the instant that you try to > code two or more, it generates errors, indicating that those Channels > do not exist. (If there is some "really specific" syntax requirement > for that directive, that supports more than one Channel, it would be > useful to have that in the documentation, as I haven't been able to > figure it out.); > > Lastly, I would suggest enhancing the description of the "define" directive. > You talk about how you can use it to codify some coding snippets, but your > examples are all single-line components. Maybe it is self-evident to most > people, but what wasn't obvious to me (until I tried it, on a whim) is the > capability to define multi-line structures, that can easily be enabled, > disabled by simply commenting the first (define) line. I'm trying to isolate > all of the typical config file modifications at the beginning of the file, to > (mostly) keep people out of the body of the config file. The multi-line > define mechanism allows me to define several (_easy-to-read_) multi-line > %QUERY% options at the front of config file, e.g. one of a normal server, one > for an AD server. Those can be chosen by toggling a single comment character > (that enables/disables the entire, multi-line structure). You could > obviously also code everything on one (really long) line, but it then becomes > impossible to re ad, and even more difficult to modify. > > > Anyway... THANKS again for the weekend reply. It allowed me to try some new > things, and sort out my previous issue. THANKS for your time and help. > > -----Original Message----- > From: Botond Botyanszki [mailto:[email protected]] > Sent: Sunday, June 30, 2013 2:00 AM > To: [email protected] > Subject: Re: [nxlog-ce-users] im_msvistalog Channel Query issue > > Marvin, > > The query below works ok for me on a W2K8R2. > Try setting Loglevel to Debug and check what query nxlog gets actually, look > for "msvistalog query xml" in nxlog.log The query is passed to the windows > eventlog API as is, nxlog does not deal with it in any way. > > For invalid channels the subscription will fail with the following error: > ERROR failed to subscribe to msvistalog events,the channel was not found > [15007]; The specified channel could not be found. Check channel > configuration. > > If you test with event viewer, just copy-paste the query xml from there > replacing line breaks with te backslash. > > Another possible option to filter is using the nxlog language for > that, i.e. Exec if $smth == 'smthelse' drop() > > Regards, > Botond > > > > On Sat, 29 Jun 2013 05:55:36 +0000 > Marvin Nipper <[email protected]> wrote: > > > OK. This is on 2.4.1054, on a W2K8 R2 server. I'm simply trying to begin > > building some default Query's, in order to limit the logging to what I > > really need/want, but cannot even get this initial setup to work. These > > are the opening lines: > > > > <Input im_msvistalog> > > > > Module im_msvistalog > > > > Query <QueryList><Query Id='1'><Select > > Path='Security'>*</Select></Query></QueryList> > > > > > > That Query line always generates this error: > > 2013-06-28 22:39:22 ERROR failed to subscribe to msvistalog > > events,the Query is invalid: [15008] > > > > If I comment the Query, I get a clean startup. > > > > I obviously tried to perform something very basic, using the example from > > the manual, and some of the Forum emails (but to no avail). What am I > > missing? > > > > Also, one other question, once I can figure out how to get this > > working, if I specify a set of default Queries (in the list), as > > part of a "standard" config file, and some of the referenced > > Channels do not exist on some of the targeted servers, will that > > result in a fatal error, or will nxlog imply generate a warning, and > > still process the Channels that do exist? (I'm attempting to build > > a default config file, to avoid the need to customize each conf > > file, for each different server. Obviously, there may be Channels > > that I want to retrieve, "if they exist", but I don't want the whole > > thing to "fall over" at start-up, if those are not present.) > > > > Thanks for your time and help. Sorry if I'm just being brain-dead with > > that query. > > > > Marvin > > > > > > The information transmitted, including any content in this > > communication is confidential, is intended only for the use of the > > intended recipient and is the property of The Western Union Company > > or its affiliates and subsidiaries. If you are not the intended > > recipient, you are hereby notified that any use of the information > > contained in or transmitted with the communication or dissemination, > > distribution, or copying of this communication is strictly prohibited. > > If you have received this communication in error, please notify the > > Western Union sender immediately by replying to this message and > > delete the original message > > > > ---------------------------------------------------------------------- > -------- This SF.net email is sponsored by Windows: > > Build for Windows Store. > > http://p.sf.net/sfu/windows-dev2dev > _______________________________________________ > nxlog-ce-users mailing list > [email protected] > https://lists.sourceforge.net/lists/listinfo/nxlog-ce-users > > The information transmitted, including any content in this > communication is confidential, is intended only for the use of the > intended recipient and is the property of The Western Union Company or > its affiliates and subsidiaries. If you are not the intended > recipient, you are hereby notified that any use of the information > contained in or transmitted with the communication or dissemination, > distribution, or copying of this communication is strictly prohibited. > If you have received this communication in error, please notify the > Western Union sender immediately by replying to this message and > delete the original message > > ------------------------------------------------------------------------------ This SF.net email is sponsored by Windows: Build for Windows Store. http://p.sf.net/sfu/windows-dev2dev _______________________________________________ nxlog-ce-users mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/nxlog-ce-users The information transmitted, including any content in this communication is confidential, is intended only for the use of the intended recipient and is the property of The Western Union Company or its affiliates and subsidiaries. If you are not the intended recipient, you are hereby notified that any use of the information contained in or transmitted with the communication or dissemination, distribution, or copying of this communication is strictly prohibited. If you have received this communication in error, please notify the Western Union sender immediately by replying to this message and delete the original message ------------------------------------------------------------------------------ This SF.net email is sponsored by Windows: Build for Windows Store. http://p.sf.net/sfu/windows-dev2dev _______________________________________________ nxlog-ce-users mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/nxlog-ce-users
