Re: improving signal to noise ratio from centralized network syslogs

[John Kougoulos]

On Mon, Feb 5, 2018 at 9:27 PM, James Bensley  wrote:

> On 5 February 2018 at 18:57,   wrote:
> > On Mon, 05 Feb 2018 10:49:42 -0800, "Scott Weeks" said:
> >> I have no knowledge of syslog-ng.  Does it do the
> >> real time scrolling like I mention?
> >
> > Use 'tail -f' or similar.
>
> The only problem is that with BASH based solutions is that they are
> slow. They don't scale well.
>
> Some years ago I wrote a script that would periodically (every 5
> minutes by default) grep for interesting events / filter uninteresting
> events from the syslog file and email you the results. It's here if
> anyone is interested: https://null.53bits.co.uk/index.php?page=sysgrep
>
>
>

Last year I found the time to code something similar in perl using
File::Tail ,  here is the outcome in case anyone is interested:
https://github.com/jkougoulos/9to5tail


Regards,
John

Re: improving signal to noise ratio from centralized network syslogs

[Brian Knight]

On 2018-02-03 15:49, Scott Weeks wrote:

Then, you can watch your network in real time
like so (below is all one line):

tail -f /var/log/router.log /var/log/switch.log
| egrep -vi 'term1|term2|termN'

'egrep -v' takes out all the lines you don't
want to see while the syslog messages scroll
across the screen.


Syslog-ng can do regex filtering on messages also.  So instead of doing 
an 'egrep -v' on a huge file after it has been logged, you can put your 
filter right into the syslog-ng configuration, and have those filtered 
messages output to a file (or any other output that syslog-ng supports). 
 The result is a smaller file to search and work with.


We implemented a simple email alerter using this functionality.  In 
syslog-ng, we set up two filters.  One filter does the 'egrep -v':


filter f_email_msg {
not message("%PKT_INFRA-LINEPROTO-.*[0-9/]+\\.")   # filter out 
subinterface up/downs

and not message("%PKT_INFRA-LINEPROTO-.*Multilink")
and not message("%PKT_INFRA-LINEPROTO-.*Serial")
and not message("%PKT_INFRA-LINEPROTO-.*Tunnel")
# etc
};

Another filter applied to the messages filters messages to just our core 
devices:


filter f_email_sources {
host("192.0.2.1")
or host("192.0.2.2")
or host("192.0.2.3")
or host("192.0.2.4")
or host("192.0.2.5")
or host("192.0.2.6")
};

Then those are tied together in a syslog-ng rule that outputs to a file:

destination d_email_log {
file("/var/log/syslog-ng/alert/alerts.log"
  template("$HOST:$MSG\n")
  create_dirs(yes)
);
};
log { source(s_devices); filter(f_email_sources); filter(f_email_msg); 
destination(d_email_log); };


A lightweight Python script that runs as a daemon checks that file once 
every 10 seconds, and if the file length is non-zero, it sends the 
contents of the file in an email to the admins.  A shell script run as a 
cron job would work equally as well.


(Also, for emailed syslogs, there is more incentive for the admin to 
keep her or his message filter up to date, as opposed to a file the 
administrator must manually examine.  Otherwise the admin has a full 
inbox :) )


It's very simple and stable, and has worked better than the commercial 
product we used to use for this purpose.


-Brian

Re: improving signal to noise ratio from centralized network syslogs

[valdis . kletnieks]

On Mon, 05 Feb 2018 20:27:13 +, James Bensley said:
> On 5 February 2018 at 18:57,   wrote:
> > On Mon, 05 Feb 2018 10:49:42 -0800, "Scott Weeks" said:
> >> I have no knowledge of syslog-ng.  Does it do the
> >> real time scrolling like I mention?
> >
> > Use 'tail -f' or similar.
>
> The only problem is that with BASH based solutions is that they are
> slow. They don't scale well.

The basic point was that you need to supply your own solution for monitoring
syslog-ng logs, be it tail or logwatch or whatever - it doesn't come with its 
own.


pgpO7IpPvL61m.pgp
Description: PGP signature

Re: improving signal to noise ratio from centralized network syslogs

[James Bensley]

On 5 February 2018 at 18:57,   wrote:
> On Mon, 05 Feb 2018 10:49:42 -0800, "Scott Weeks" said:
>> I have no knowledge of syslog-ng.  Does it do the
>> real time scrolling like I mention?
>
> Use 'tail -f' or similar.

The only problem is that with BASH based solutions is that they are
slow. They don't scale well.

Some years ago I wrote a script that would periodically (every 5
minutes by default) grep for interesting events / filter uninteresting
events from the syslog file and email you the results. It's here if
anyone is interested: https://null.53bits.co.uk/index.php?page=sysgrep

It's OK for a small network or small number of devices but it doesn't
scale well. Having said that, it's better than nothing and costs $0
(which exactly why I used it in the first place).

Cheers,
James.

Re: improving signal to noise ratio from centralized network syslogs

[valdis . kletnieks]

On Mon, 05 Feb 2018 10:49:42 -0800, "Scott Weeks" said:
> I have no knowledge of syslog-ng.  Does it do the
> real time scrolling like I mention?

Use 'tail -f' or similar.


pgppqrj2ic42P.pgp
Description: PGP signature

Re: improving signal to noise ratio from centralized network syslogs

[Scott Weeks]

--- sh...@short.id.au wrote:

In addition to that, you can use some fancy awk colour 
coding, so you can make it highlight certain lines based 
on content.. I use this for my e-mail logs, but I’m sure 
it could be adapted:

tail -n 1000 -f /var/log/mail-submission.log | grep smtp.*relay | awk '
/sent/ {print "\033[32m" $0 "\033[39m"}
/bounced/ {print "\033[31m" $0 "\033[39m"}
/deferred/ {print "\033[33m" $0 "\033[39m"}



The main thing for me is to find things that 
your network is doing that you weren't aware 
of.  Not normal things you want to see that 
a monitoring system will alert you about.

scott

Re: improving signal to noise ratio from centralized network syslogs

[Scott Weeks]

--- ta...@lanparty.ee wrote:
> This is done with the 'logging facility'
> command on the devices:
> 
> After defining your syslog server's IP
> address and the level of messaging you want
> (I set it to debug because I want to see
> everything):
> 
> on the routers: logging facility local0
> on the switches:  logging facility local1

Alternative, and more universal, way to do it is to use multiple IPs for 
syslog server. Then configure correct syslog server IP on the device.

syslog-ng and others can all do filtering to different destinations 
based on the IP where message was received.



The nice thing about the simple way is you see 
everything that's happening on the network, except
what you 'egrep -v' out, which you already know 
about.  Then you find things you weren't expecting.
  
You don't go looking for stuff.  You just watch the 
network events scroll by in real time ans see what 
shows up.

I have no knowledge of syslog-ng.  Does it do the
real time scrolling like I mention?

scott 

Re: improving signal to noise ratio from centralized network syslogs

[Shane Short]

In addition to that, you can use some fancy awk colour coding, so you can make 
it highlight certain lines based on content.. I use this for my e-mail logs, 
but I’m sure it could be adapted:

tail -n 1000 -f /var/log/mail-submission.log | grep smtp.*relay | awk '
/sent/ {print "\033[32m" $0 "\033[39m"}
/bounced/ {print "\033[31m" $0 "\033[39m"}
/deferred/ {print "\033[33m" $0 "\033[39m"}
'



> On 4 Feb 2018, at 5:49 am, Scott Weeks  wrote:
> 
> 
> --- jmai...@jmaimon.com wrote:
> Centralized logging is a good thing. However, 
> what happens is that every repetitive, annoying 
> but not (usually) important thing fills up the 
> log with reams of what you are not looking for.
> ---
> 
> Apologies, I'm late to the party.  But I just 
> want to add one thing for the archives.  It's 
> along with what Rich Kulawiec said, "it forces 
> you to look at your own data, which is really 
> helpful.  You'll be surprised at what you find 
> if you've never done it before."  This is 
> accurate.  It's fun to see what your network 
> is putting out.
> 
> This is all from memory (I've done it so many 
> times it's in there permanently... :-) as I 
> don't have a unix server or a router in front 
> of me to use, so don't hold me to exact 
> details...
> 
> And it's mainly for the newbies.
> 
> Have all the routers send to one syslog file, 
> switches to another and other devices to a 
> third on a *nix box: For example, send the 
> router messages to /var/log/router.log and 
> the switch messages to /var/log/switch.log  
> This is done with the 'logging facility' 
> command on the devices: 
> 
> After defining your syslog server's IP 
> address and the level of messaging you want
> (I set it to debug because I want to see 
> everything):
> 
> on the routers: logging facility local0
> on the switches:  logging facility local1
> 
> on the logging server in:  /etc/rsyslog.conf
> local0.* /var/log/router.log
> local1.* /var/log/switch.log
> 
> Use logrotate to manage the log files as they 
> can get quite large.
> 
> Then, you can watch your network in real time 
> like so (below is all one line):
> 
> tail -f /var/log/router.log /var/log/switch.log 
> | egrep -vi 'term1|term2|termN'
> 
> 'egrep -v' takes out all the lines you don't 
> want to see while the syslog messages scroll
> across the screen.
> 
> Say there is a battery condition on router1 
> and a duplex mismatch on a switch I don't want 
> to see:
> 
> tail -f /var/log/router.log /var/log/switch.log 
> | egrep -vi 'router1.*battery|switch1.*duplex.*mismatch'
> 
> For me, N can get to 40-50 sometimes, so I put 
> it into a shell script like so:
> 
> vi log.sh
> 
> ---
> #! /bin/sh
> 
> tail -f /var/log/router.log /var/log/switch.log 
> | egrep -v 'term1|term2|termN'
> ---
> 
> then, run it like so: ./log.sh
> 
> It's all netgeek fun-n-games from there on. :)
> 
> scott

Re: improving signal to noise ratio from centralized network syslogs

[Jippen]

I really recommend setting up fluentd, and then routing logging from there
- it makes it very easy to keep auditor-appeasing logs, while also having
important stuff sending pages. Log aggregation, organization, and search is
a hard problem, other people have already done it and provided it as a
service, and chances are its NOT a core competency or secret sauce at your
organization.

Once you get your logs in one routing system, you can do a lot with them,
but stop rolling your own. This is a prime area for most companies to buy
something that works better, for less than the cost of developing in house.
And if you run your own aggregation layer - then you can easily try out a
bunch of different systems and add/remove them easily. :)

Also, you may want to see one level of logs, but your auditors might wanna
see another, and your engineers/sec team might wanna do some analytics on
them. Being able to provide a solution for everyone who needs network logs
at whatever detail level they ask for will make you popular at your
organization.

On Sun, Feb 4, 2018 at 12:21 AM, Tarko Tikan  wrote:

> hey,
>
> This is done with the 'logging facility'
>> command on the devices:
>>
>> After defining your syslog server's IP
>> address and the level of messaging you want
>> (I set it to debug because I want to see
>> everything):
>>
>> on the routers: logging facility local0
>> on the switches:  logging facility local1
>>
>
> Alternative, and more universal, way to do it is to use multiple IPs for
> syslog server. Then configure correct syslog server IP on the device.
>
> syslog-ng and others can all do filtering to different destinations based
> on the IP where message was received.
>
> --
> tarko
>

Re: improving signal to noise ratio from centralized network syslogs

[Tarko Tikan]

hey,


This is done with the 'logging facility'
command on the devices:

After defining your syslog server's IP
address and the level of messaging you want
(I set it to debug because I want to see
everything):

on the routers: logging facility local0
on the switches:  logging facility local1


Alternative, and more universal, way to do it is to use multiple IPs for 
syslog server. Then configure correct syslog server IP on the device.


syslog-ng and others can all do filtering to different destinations 
based on the IP where message was received.


--
tarko

Re: improving signal to noise ratio from centralized network syslogs

[Scott Weeks]

--- jmai...@jmaimon.com wrote:
Centralized logging is a good thing. However, 
what happens is that every repetitive, annoying 
but not (usually) important thing fills up the 
log with reams of what you are not looking for.
---

Apologies, I'm late to the party.  But I just 
want to add one thing for the archives.  It's 
along with what Rich Kulawiec said, "it forces 
you to look at your own data, which is really 
helpful.  You'll be surprised at what you find 
if you've never done it before."  This is 
accurate.  It's fun to see what your network 
is putting out.

This is all from memory (I've done it so many 
times it's in there permanently... :-) as I 
don't have a unix server or a router in front 
of me to use, so don't hold me to exact 
details...

And it's mainly for the newbies.

Have all the routers send to one syslog file, 
switches to another and other devices to a 
third on a *nix box: For example, send the 
router messages to /var/log/router.log and 
the switch messages to /var/log/switch.log  
This is done with the 'logging facility' 
command on the devices: 

After defining your syslog server's IP 
address and the level of messaging you want
(I set it to debug because I want to see 
everything):

on the routers: logging facility local0
on the switches:  logging facility local1

on the logging server in:  /etc/rsyslog.conf
local0.* /var/log/router.log
local1.* /var/log/switch.log

Use logrotate to manage the log files as they 
can get quite large.

Then, you can watch your network in real time 
like so (below is all one line):

tail -f /var/log/router.log /var/log/switch.log 
| egrep -vi 'term1|term2|termN'

'egrep -v' takes out all the lines you don't 
want to see while the syslog messages scroll
across the screen.

Say there is a battery condition on router1 
and a duplex mismatch on a switch I don't want 
to see:

tail -f /var/log/router.log /var/log/switch.log 
| egrep -vi 'router1.*battery|switch1.*duplex.*mismatch'

For me, N can get to 40-50 sometimes, so I put 
it into a shell script like so:

vi log.sh

---
#! /bin/sh

tail -f /var/log/router.log /var/log/switch.log 
| egrep -v 'term1|term2|termN'
---

then, run it like so: ./log.sh

It's all netgeek fun-n-games from there on. :)

scott

Re: improving signal to noise ratio from centralized network syslogs

[George William Herbert]

From the systems side we got HoneycombIO which shifts a bit to calling itself 
events rather than logs management.  I don't know anyone else who's tried using 
it for networks per se but that's on my "interesting tech tools explorations" 
medium length list.


-george 

Sent from my iPhone

> On Jan 31, 2018, at 7:17 AM, Rich Kulawiec  wrote:
> 
>> On Thu, Jan 25, 2018 at 11:10:02PM -0500, Joe Maimon wrote:
>> What I am interested in is an automated zoom-in zoom-out tool to mask the
>> repetition of "normal" events and allow the unusual to stand out.
> 
> This is an approach outlined by Marcus Ranum years ago; he called it
> "artificial stupidity", and it works.  (Of course, an inverse check
> that makes sure routine boring things are still happening is also
> a good idea.)
> 
> You could use any number of elaborate (and sometimes expensive) tools
> to do this, but I recommend rolling your own with Perl or similar.
> This is goodness for two reasons: first, it forces you to look at your
> own data, which is really helpful.  You'll be surprised at what you
> find if you've never done it before.  Second, it lets you customize for
> your environment at every step.
> 
> I have written dozens of these, some as trivial as a few lines of code,
> some quite extensive.  None of them "solve" the problem per se, they just
> all take bites out of it.  But this admittedly-simplistic (and deliberately
> so) approach has flagged a lot of issues, and because it's simple,
> it's easy to connect to other monitoring/alerting plumbing.
> 
> ---rsk

Re: improving signal to noise ratio from centralized network syslogs

[Rich Kulawiec]

On Thu, Jan 25, 2018 at 11:10:02PM -0500, Joe Maimon wrote:
> What I am interested in is an automated zoom-in zoom-out tool to mask the
> repetition of "normal" events and allow the unusual to stand out.

This is an approach outlined by Marcus Ranum years ago; he called it
"artificial stupidity", and it works.  (Of course, an inverse check
that makes sure routine boring things are still happening is also
a good idea.)

You could use any number of elaborate (and sometimes expensive) tools
to do this, but I recommend rolling your own with Perl or similar.
This is goodness for two reasons: first, it forces you to look at your
own data, which is really helpful.  You'll be surprised at what you
find if you've never done it before.  Second, it lets you customize for
your environment at every step.

I have written dozens of these, some as trivial as a few lines of code,
some quite extensive.  None of them "solve" the problem per se, they just
all take bites out of it.  But this admittedly-simplistic (and deliberately
so) approach has flagged a lot of issues, and because it's simple,
it's easy to connect to other monitoring/alerting plumbing.

---rsk

Re: improving signal to noise ratio from centralized network syslogs

[Casey Russell]

 +1 for Graylog, you can pour ALL your syslog data into it, and then
configure what are called streams.  Streams are a way to whittle down the
incoming log flows and see something LESS than everything.  You can create
a stream that only shows these 6 devices, or one that only shows log info
from the RPD daemon on your Juniper routers.

 In your case, you could use the stream rules to create a stream that
filters out the background noise with regex expressions.  You're not losing
anything, you still have the full log data captured, and you can see it in
the portal, but if you click on one of your streams, you see filtered data
based, on your rulesets.  We've been using it for about 2 years now I
think.

 It's open source, easy to set up, supports LDAP, multiple input types
(beyond just udp syslog), and the community is pretty solid.


Sincerely,
Casey Russell
Network Engineer
[image: KanREN] 
[image: phone]785-856-9809
2029 Becker Drive, Suite 282
Lawrence, Kansas 66047
[image: linkedin]

[image:
twitter]  [image: twitter]
 need support? 


On Fri, Jan 26, 2018 at 10:41 AM, Alain Hebert  wrote:

> ELK stack.
>
> Java RAM devoring monster but Kibana makes indexing easy.
>
> -
> Alain Hebertaheb...@pubnix.net
> PubNIX Inc.
> 50 boul. St-Charles
> 
> P.O. Box 26770 Beaconsfield, Quebec H9W 6G7
> Tel: 514-990-5911  http://www.pubnix.netFax: 514-990-9443
>
>
> On 01/26/18 01:01, Michael Loftis wrote:
>
>> On Thu, Jan 25, 2018 at 8:11 PM Joe Maimon  wrote:
>>
>> Hey All,
>>>
>>> Centralized logging is a good thing. However, what happens is that every
>>> repetitive, annoying but not (usually) important thing fills up the log
>>> with reams of what you are not looking for.
>>>
>>> Networks are a noisy place and silencing every logged condition is
>>> impractical and sometimes undesirable.
>>>
>>> What I am interested in is an automated zoom-in zoom-out tool to mask
>>> the repetition of "normal" events and allow the unusual to stand out.
>>>
>>> Add to that an ability to identify gaps in the background noise. (The
>>> dog that didnt bark)
>>>
>>> What I am not interested in are solutions based upon preconfigured
>>> filters and definitions and built in analysis for supported
>>> (prepopulated definitions) platforms, this is all about pattern
>>> mining/masking and should be self discoverable. Ideally a command tool
>>> to generate static versions of the analysis coupled with a web platform
>>> (with zoom +- buttons)  for realtime.
>>>
>>> I made a crude run of it with SLCT, using its generated patterns to grep
>>> -v, and that in and of itself was useful, but needs a bit of work. Also,
>>> its not quite real time.
>>>
>>> Any ideas would be greatly appreciated.
>>>
>>
>> Not cheap, but Splunk comes to mind.
>>
>>
>>> Joe
>>>
>>>
>

Re: improving signal to noise ratio from centralized network syslogs

[Alain Hebert]

    ELK stack.

    Java RAM devoring monster but Kibana makes indexing easy.

-
Alain Hebertaheb...@pubnix.net
PubNIX Inc.
50 boul. St-Charles
P.O. Box 26770 Beaconsfield, Quebec H9W 6G7
Tel: 514-990-5911  http://www.pubnix.netFax: 514-990-9443

On 01/26/18 01:01, Michael Loftis wrote:

On Thu, Jan 25, 2018 at 8:11 PM Joe Maimon  wrote:


Hey All,

Centralized logging is a good thing. However, what happens is that every
repetitive, annoying but not (usually) important thing fills up the log
with reams of what you are not looking for.

Networks are a noisy place and silencing every logged condition is
impractical and sometimes undesirable.

What I am interested in is an automated zoom-in zoom-out tool to mask
the repetition of "normal" events and allow the unusual to stand out.

Add to that an ability to identify gaps in the background noise. (The
dog that didnt bark)

What I am not interested in are solutions based upon preconfigured
filters and definitions and built in analysis for supported
(prepopulated definitions) platforms, this is all about pattern
mining/masking and should be self discoverable. Ideally a command tool
to generate static versions of the analysis coupled with a web platform
(with zoom +- buttons)  for realtime.

I made a crude run of it with SLCT, using its generated patterns to grep
-v, and that in and of itself was useful, but needs a bit of work. Also,
its not quite real time.

Any ideas would be greatly appreciated.


Not cheap, but Splunk comes to mind.



Joe

RE: improving signal to noise ratio from centralized network syslogs

[Edwin Pers]

On Fri, Jan 26, 2018 at 6:30 AM, Steven Miano  wrote:
>either ELK (or any derivative there of such as: Elasticache, Fluentd, Kibana)

I'm partial to graylog - it does some of the heavy lifting of getting a 
logging-centric ELK stack up and running

-Ed

Re: improving signal to noise ratio from centralized network syslogs

[Steven Miano]

Splunk is the obvious solution that most organizations with a mature
security group will likely already have in their portfolio.

Going a step further, and with an abundance of skill, ability, and
forethought: either ELK (or any derivative there of such as: Elasticache,
Fluentd, Kibana), or rsyslog|syslog-ng + database + loganalzyer.

Grep-fu will pay dividends in any of the three options (do nothing, go
proprietary, go open).

~Steven

On Fri, Jan 26, 2018 at 1:01 AM, Michael Loftis  wrote:

> On Thu, Jan 25, 2018 at 8:11 PM Joe Maimon  wrote:
>
> > Hey All,
> >
> > Centralized logging is a good thing. However, what happens is that every
> > repetitive, annoying but not (usually) important thing fills up the log
> > with reams of what you are not looking for.
> >
> > Networks are a noisy place and silencing every logged condition is
> > impractical and sometimes undesirable.
> >
> > What I am interested in is an automated zoom-in zoom-out tool to mask
> > the repetition of "normal" events and allow the unusual to stand out.
> >
> > Add to that an ability to identify gaps in the background noise. (The
> > dog that didnt bark)
> >
> > What I am not interested in are solutions based upon preconfigured
> > filters and definitions and built in analysis for supported
> > (prepopulated definitions) platforms, this is all about pattern
> > mining/masking and should be self discoverable. Ideally a command tool
> > to generate static versions of the analysis coupled with a web platform
> > (with zoom +- buttons)  for realtime.
> >
> > I made a crude run of it with SLCT, using its generated patterns to grep
> > -v, and that in and of itself was useful, but needs a bit of work. Also,
> > its not quite real time.
> >
> > Any ideas would be greatly appreciated.
>
>
> Not cheap, but Splunk comes to mind.
>
> >
> >
> > Joe
> >
> --
>
> "Genius might be described as a supreme capacity for getting its possessors
> into trouble of all kinds."
> -- Samuel Butler
>



-- 
Steven M. Miano
http://stevenmiano.com

Re: improving signal to noise ratio from centralized network syslogs

[Michael Loftis]

On Thu, Jan 25, 2018 at 8:11 PM Joe Maimon  wrote:

> Hey All,
>
> Centralized logging is a good thing. However, what happens is that every
> repetitive, annoying but not (usually) important thing fills up the log
> with reams of what you are not looking for.
>
> Networks are a noisy place and silencing every logged condition is
> impractical and sometimes undesirable.
>
> What I am interested in is an automated zoom-in zoom-out tool to mask
> the repetition of "normal" events and allow the unusual to stand out.
>
> Add to that an ability to identify gaps in the background noise. (The
> dog that didnt bark)
>
> What I am not interested in are solutions based upon preconfigured
> filters and definitions and built in analysis for supported
> (prepopulated definitions) platforms, this is all about pattern
> mining/masking and should be self discoverable. Ideally a command tool
> to generate static versions of the analysis coupled with a web platform
> (with zoom +- buttons)  for realtime.
>
> I made a crude run of it with SLCT, using its generated patterns to grep
> -v, and that in and of itself was useful, but needs a bit of work. Also,
> its not quite real time.
>
> Any ideas would be greatly appreciated.


Not cheap, but Splunk comes to mind.

>
>
> Joe
>
-- 

"Genius might be described as a supreme capacity for getting its possessors
into trouble of all kinds."
-- Samuel Butler

improving signal to noise ratio from centralized network syslogs

[Joe Maimon]

Hey All,

Centralized logging is a good thing. However, what happens is that every 
repetitive, annoying but not (usually) important thing fills up the log 
with reams of what you are not looking for.


Networks are a noisy place and silencing every logged condition is 
impractical and sometimes undesirable.


What I am interested in is an automated zoom-in zoom-out tool to mask 
the repetition of "normal" events and allow the unusual to stand out.


Add to that an ability to identify gaps in the background noise. (The 
dog that didnt bark)


What I am not interested in are solutions based upon preconfigured 
filters and definitions and built in analysis for supported 
(prepopulated definitions) platforms, this is all about pattern 
mining/masking and should be self discoverable. Ideally a command tool 
to generate static versions of the analysis coupled with a web platform 
(with zoom +- buttons)  for realtime.


I made a crude run of it with SLCT, using its generated patterns to grep 
-v, and that in and of itself was useful, but needs a bit of work. Also, 
its not quite real time.


Any ideas would be greatly appreciated.

Joe