Agreed, we need to consider that many of us see multi-millions of messages on these systems...
On 7/10/06, Claus Lund <[EMAIL PROTECTED]> wrote: > It may be more flexible but one of my number one priorities is that it > has to scale very well. Depending on the number of subscribers and how > many thousand log entries you have per minute then a dynamic/on demand > solution that queries the entire database may just not work. > > -Claus > > On 7/10/06, Freeman, Michael <[EMAIL PROTECTED]> wrote: > > I don't think you really need a separate feed, since the data already > > exists in the database. I think it would be a little more flexible if > > you had a .php script (renamed as .rss or something) building the actual > > feed for you, every time you demanded it. > > > > -----Original Message----- > > From: Claus Lund [mailto:[EMAIL PROTECTED] > > Sent: Monday, July 10, 2006 1:43 PM > > To: Freeman, Michael > > Cc: php-syslog-ng-support@lists.sourceforge.net > > Subject: Re: [Php-syslog-ng-support] Log2rss support? > > > > I think Jason probably has the right idea to somehow handle it on the > > syslog-ng side. Maybe duplicate the log feed through a separate FIFO and > > have a script look at that FIFO and build the various RSS feeds > > according to whatever search criteria your users want to monitor? > > > > -Claus > > > > On 7/10/06, Freeman, Michael <[EMAIL PROTECTED]> wrote: > > > Is there any other way to do it? > > > > > > > > > -----Original Message----- > > > From: Dukes Clayton [mailto:[EMAIL PROTECTED] > > > Sent: Monday, July 10, 2006 1:29 PM > > > To: Freeman, Michael; php-syslog-ng-support@lists.sourceforge.net > > > Cc: [EMAIL PROTECTED] > > > Subject: RE: [Php-syslog-ng-support] Log2rss support? > > > > > > Be aware that when you use %like% syntax on large amounts of data it > > > can slow it down a lot :-) > > > > > > > > > -----Original Message----- > > > From: [EMAIL PROTECTED] > > > [mailto:[EMAIL PROTECTED] On Behalf > > > > > Of Freeman, Michael > > > Sent: Monday, July 10, 2006 1:05 PM > > > To: php-syslog-ng-support@lists.sourceforge.net > > > Cc: [EMAIL PROTECTED]; Dukes Clayton > > > Subject: Re: [Php-syslog-ng-support] Log2rss support? > > > > > > Here are my initial thoughts on the design: > > > > > > I have attached a screenshot of how I think the interface would look > > > that will allow the user to configure their own particular RSS feed. > > > The interface would also allow the user to make the feed public, so > > > others could subscribe to it. > > > > > > The user will be able to build a filter that is essentially going to > > > be the SQL query used for pulling the data out of mySQL and generating > > > > > the RSS with. > > > > > > When a user goes to pull the feed, http://foo.bar/syslog/feed.rss, it > > > is really calling a php script that is doing the SQL query and > > > generating the RSS feed. I'm not sure yet how this will be possibly, > > > probably just telling apache that .rss is really a .php. I'm not sure > > > yet how to build in the authentication system.. Does anyone have any > > > ideas on that? I think if we could utilize someones existing session > > > key to validate who they are then we could also figure out what > > > feed.rss needs to do to build the actual feed. Users settings for how > > > they want their feeds displayed would probably also be stored in > > mySQL. > > > > > > -----Original Message----- > > > From: Dukes Clayton [mailto:[EMAIL PROTECTED] > > > Sent: Monday, July 10, 2006 12:52 PM > > > To: Freeman, Michael; [EMAIL PROTECTED] > > > Cc: php-syslog-ng-support@lists.sourceforge.net > > > Subject: RE: [Php-syslog-ng-support] Log2rss support? > > > > > > I'd have to agree...plus you introduce all the headaches of > > > maintaining someone elses code and security deficiencies... > > > > > > > > > -----Original Message----- > > > From: [EMAIL PROTECTED] > > > [mailto:[EMAIL PROTECTED] On Behalf > > > > > Of Freeman, Michael > > > Sent: Monday, July 10, 2006 9:32 AM > > > To: [EMAIL PROTECTED] > > > Cc: php-syslog-ng-support@lists.sourceforge.net > > > Subject: Re: [Php-syslog-ng-support] Log2rss support? > > > > > > I think that is like trying to kill an ant with a hammer.. I don't > > > think people should be forced into using a full blown CMS if all they > > > want is a front-end to syslog. I'm just trying to fulfill some > > > requirements from my end-users here at my job and get something out to > > them asap. > > > > > > -----Original Message----- > > > From: Jason Taylor [mailto:[EMAIL PROTECTED] > > > Sent: Friday, July 07, 2006 6:51 PM > > > To: Freeman, Michael > > > Cc: [EMAIL PROTECTED]; Claus Lund; > > > php-syslog-ng-support@lists.sourceforge.net > > > Subject: RE: [Php-syslog-ng-support] Log2rss support? > > > > > > I definitely suggest you look into a CMS (Content Management System) > > > such as plone, XOOPS, phpNuke, etc. > > > These various projects already have a well-developed codebase and > > > probably several modules that can be bent to your needs. > > > I looked briefly at making php-syslog-ng into a XOOPS module about a > > > year and a half ago but got side-tracked. > > > > > > The big advantage of using a CMS is that it would really modularize > > > everything and users would be able to only use the features they like. > > > These products will also have a pretty good authentication and > > > permissioning system which could be useful for limiting which hosts a > > > user could query or other such niceties. > > > > > > Though not rich in time these days, I would be glad to help where I > > > can in such a project. > > > > > > Cheers, > > > > > > /Jason > > > > > > -- > > > You can have my Mac when you pry it from my cold, dead hands. > > > e:[EMAIL PROTECTED] v:514-815-8204 > > > > > > <quote who="Freeman, Michael"> > > > > I'd still be interested in having one centralized location for > > > > managing all the different elements of syslog-ng. You shouldn't > > > > necessarily have to go to different places to do this, for > > > > convenience > > > > > > > sake anyways. How it actually gets done I'm not really concerned > > > > with yet, this is more of a requirements gathering phase for me. > > > > > > > > I have a project at my company to replace CiscoWorks RME with > > > > Opsware NAS, and the whole reason I am involved in this > > > > php-syslog-ng stuff now is because NAS didn't have a front-end to > > > > syslog, like RME provided, and now that I have found this tool I > > > > have mitigated that requirement, my next goal is to extend the > > > > syslog analyzer functions to allow people to create event handlers. > > > > So it sounds like this functionality is built-in to syslog-ng > > > > already, I'm interested in wrapping it all into php-syslog-ng, if > > feasible. > > > > > > > > So the RSS stuff and the event handlers are not necessarily > > > intertwined. > > > > I would separate them so you would have a separate area to build > > > > your filters for RSS feeds, and then another area to build your > > > > event handlers for specific events. > > > > > > > > -----Original Message----- > > > > From: Jason Taylor [mailto:[EMAIL PROTECTED] > > > > Sent: Friday, July 07, 2006 2:22 PM > > > > To: Freeman, Michael > > > > Cc: Claus Lund; php-syslog-ng-support@lists.sourceforge.net > > > > Subject: Re: [Php-syslog-ng-support] Log2rss support? > > > > > > > > > > > > Something like this might be easier to implement on the back-end > > > > syslog-ng side, and not by hitting the database. What about tools > > > > like > > > > > > > swatch or watcher? > > > > > > > > How I see it working is creating a second "destination" in the > > > > syslog-ng config file which sends "pre-filtered" log-output to a > > > > file or socket to be u sed as input by one of these log-watcher > > programs. > > > > > > > > When the program sees a hit, it can create the "notify" event which > > > > could easily be configured as an event to add to an RSS feed. > > > > > > > > If you are dead-set on using the database for this, then you are not > > > > > > going to have "real-time" stream processing as in the above case. > > > > You > > > > > > > will likely be running a periodic query on the database and > > > > processing > > > > > > > that output. > > > > > > > > But in either case, what you are after is some kind of alerter > > > service. > > > > At present, php-syslog-ng is nothing more than a log-presentation > > > > service. The alerter stuff can be added on but would be entirely > > > > separate from the front end. It would be just another consumer of > > > > the > > > > > > > syslog-ng database. > > > > > > > > /Jason > > > > > > > > > > > > -- > > > > You can have my Mac when you pry it from my cold, dead hands. > > > > e:[EMAIL PROTECTED] v:514-815-8204 > > > > > > > > > > > > <quote who="Freeman, Michael"> > > > > > > > >> Yeah, my idea was to allow the user to build the type of filter > > > >> they wanted and underneath the hood it would make a SQL query to > > > >> the DB grabbing that stuff and generating the RSS. I'm not sure how > > > > > >> sophisticated my first attempt would be, but I might make a small > > > >> release and see what people think and if they find it useful, add > > > >> some > > > > > > > >> of the additional things. The whole reason behind me wanting to do > > > >> this is I think while a web front end to syslog is great, it still > > > >> requires someone to look at it. I for one use RSS frequently and > > > >> always have an aggregator application open on one of my screens so > > > >> it > > > > > > >> is easy for me to look over and take notice of things I actually > > > >> care > > > > > > >> about, based on the filter/ruleset I defined for my feed. > > > >> > > > >> One could take this concept even further and extend the > > > >> php-syslog-ng > > > > > > >> interface to allow users to create rules and event handlers that > > > >> would > > > > > > > >> be executed by the syslog-ng daemon itself. > > > >> > > > >> -----Original Message----- > > > >> From: Claus Lund [mailto:[EMAIL PROTECTED] > > > >> Sent: Friday, July 07, 2006 10:43 AM > > > >> To: Freeman, Michael > > > >> Subject: Re: [Php-syslog-ng-support] Log2rss support? > > > >> > > > >> > > > >> > > > >> An RSS feed may be useful in some situations ... but I don't know > > > >> how > > > > > > >> well it would work/useful it would be in very "chatty > > environments"? > > > >> Maybe if you can do some filtering on what you subscribe to then it > > > > > >> could be used for alerting on certain situations that you want to > > > >> make > > > > > > > >> sure you catch. > > > >> > > > >> -Claus > > > >> > > > >> > > > >> > > > >> On 7/6/06, Freeman, Michael <[EMAIL PROTECTED]> wrote: > > > >> > > > >> > > > >>> Is anyone else besides myself interested in getting RSS feeds from > > > > > >>> php-syslog-ng? I'm thinking about hacking up the log2rss stuff to > > > >>> support mySQL and use apache instead of it's own HTTP daemon in > > > perl. > > > >>> > > > >>> > > > >>> Netco Government Services has recently acquired Multimax and is > > > >>> > > > >>> > > > >> changing its name to Multimax Inc. > > > >>> Visit http://www.multimax.com for more information. > > > >>> > > > >>> > > > >>> > > > >>> Note: My email address has changed. Please take a moment to update > > > >>> > > > >>> > > > >> your records with my new address. > > > >>> > > > >>> > > > >>> Using Tomcat but need to do more? Need to support web services, > > > >>> > > > >>> > > > >> security? > > > >>> Get stuff done quickly with pre-integrated technology to make your > > > > > >>> job > > > >>> > > > >> > > > >>> easier Download IBM WebSphere Application Server v.1.0.1 based on > > > >>> Apache Geronimo > > > >>> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat= > > > >>> 12 > > > >>> 1 > > > >>> 6 > > > >>> 42 _______________________________________________ > > > >>> Php-syslog-ng-support mailing list > > > >>> Php-syslog-ng-support@lists.sourceforge.net > > > >>> https://lists.sourceforge.net/lists/listinfo/php-syslog-ng-support > > > >>> > > > >>> > > > >>> > > > >> > > > >> Using Tomcat but need to do more? Need to support web services, > > > >> > > > > security? > > > >> Get stuff done quickly with pre-integrated technology to make your > > > >> job easier Download IBM WebSphere Application Server v.1.0.1 based > > > >> on > > > > > > >> Apache Geronimo > > > >> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=1 > > > >> 21 > > > >> 6 > > > >> 42 _______________________________________________ > > > >> Php-syslog-ng-support mailing list > > > >> Php-syslog-ng-support@lists.sourceforge.net > > > >> https://lists.sourceforge.net/lists/listinfo/php-syslog-ng-support > > > >> > > > >> > > > >> > > > > > > > > > > > > > > > > > > > > ---------------------------------------------------------------------- > > > -- > > > - > > > Using Tomcat but need to do more? Need to support web services, > > > security? > > > Get stuff done quickly with pre-integrated technology to make your job > > > > > easier Download IBM WebSphere Application Server v.1.0.1 based on > > > Apache Geronimo > > > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=1216 > > > 42 _______________________________________________ > > > Php-syslog-ng-support mailing list > > > Php-syslog-ng-support@lists.sourceforge.net > > > https://lists.sourceforge.net/lists/listinfo/php-syslog-ng-support > > > > > > > > > ---------------------------------------------------------------------- > > > --- Using Tomcat but need to do more? Need to support web services, > > > security? > > > Get stuff done quickly with pre-integrated technology to make your job > > > > > easier Download IBM WebSphere Application Server v.1.0.1 based on > > > Apache Geronimo > > > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=1216 > > > 42 _______________________________________________ > > > Php-syslog-ng-support mailing list > > > Php-syslog-ng-support@lists.sourceforge.net > > > https://lists.sourceforge.net/lists/listinfo/php-syslog-ng-support > > > > > > > > > ------------------------------------------------------------------------- > > Using Tomcat but need to do more? Need to support web services, security? > > Get stuff done quickly with pre-integrated technology to make your job > > easier > > Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo > > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > > _______________________________________________ > > Php-syslog-ng-support mailing list > > Php-syslog-ng-support@lists.sourceforge.net > > https://lists.sourceforge.net/lists/listinfo/php-syslog-ng-support > > > > > ------------------------------------------------------------------------- > Using Tomcat but need to do more? Need to support web services, security? > Get stuff done quickly with pre-integrated technology to make your job easier > Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > _______________________________________________ > Php-syslog-ng-support mailing list > Php-syslog-ng-support@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/php-syslog-ng-support > -- ======================================= Clayton Dukes Network Consulting Engineer, Advanced Services Cisco Systems, Inc. Office: 919.392.6122 Fax: : 919.869.1580 Cell: 813.545.7373 Email: [EMAIL PROTECTED] MSN: [EMAIL PROTECTED] ======================================= ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 _______________________________________________ Php-syslog-ng-support mailing list Php-syslog-ng-support@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/php-syslog-ng-support
