Hi risto,
Great thanks for these useful information!
Actually I am in the researching and evaluation phase, Elasticsearch and
logstash are just one option, I also need look around others. My
requirements is: big data security event management, search, correlation,
alert.
You are the expert of such domain, could you give me your prefer,
recommendation and advise for my requirement?
Great thanks!
Best regards,
John
在 2013年5月3日星期五，Risto Vaarandi 写道：
> hi John,
> if you plan to use Logstash for feeding Elasticsearch database, SEC can be
> quite easily connected to it, since Logstash supports receiving data
> through wide variety of inputs. Depending on your system and log data
> volumes, you could have just one SEC instance which correlates all your
> events and then sends them to Logstash --> Elasticsearch, but you could
> also have a number of instances, each handling a part of the event volume
> and accomplishing some more specific task. Since Elasticsearch is about
> storing large volumes of log data, it is likely that you want to send a
> significant amount of log messages directly to Elasticearch, and use SEC
> for adding additional events to stored data. (My own setup looks like this,
> but you could also have very different requirements in your environment.)
> If you are looking for references and published materials about using SEC,
> then I recollect a recent paper written by David Lang (I think it was
> published in a recent USENIX LISA conference).
> Also, if you want to consider fast alternatives to Logstash, then rsyslog
> has a builtin support for Elasticsearch, and since it's written in C, it
> can feed Elasticsearch much more efficiently. Here is a reference about
> creating a basic configuration:
> http://wiki.rsyslog.com/index.php/HOWTO:_rsyslog_%2B_elasticsearch
> It is also fairly easy to configure rsyslog to store log data into
> Elasticsearch in the way Kibana expects it to see (you have to set up the
> same index name for rsyslog and Kibana, and also include couple of
> mandatory fields in each log message).
> kind regards,
> risto
>
>
> 2013/5/2 John Zhang <kingzy...@gmail.com <javascript:_e({}, 'cvml',
> 'kingzy...@gmail.com');>>
>
>> Hi everyone,
>>
>> I am researching the big data security log management, such as Kibana +
>> ElasticSearch + Logstash for my security log management, I need event
>> correlation on this platform, i know SEC(
>> http://simple-evcorr.sourceforge.net/) can do event correlation.
>>
>> Do you have any idea of SEC on such big data security log platform? Any
>> experience, any reference?
>>
>> Any comment, advise will be highly appreciated!
>>
>> Thanks!
>>
>> John
>>
>>
>> ------------------------------------------------------------------------------
>> Introducing AppDynamics Lite, a free troubleshooting tool for Java/.NET
>> Get 100% visibility into your production application - at no cost.
>> Code-level diagnostics for performance bottlenecks with <2% overhead
>> Download for free and get started troubleshooting in minutes.
>> http://p.sf.net/sfu/appdyn_d2d_ap1
>> _______________________________________________
>> Simple-evcorr-users mailing list
>> Simple-evcorr-users@lists.sourceforge.net <javascript:_e({}, 'cvml',
>> 'Simple-evcorr-users@lists.sourceforge.net');>
>> https://lists.sourceforge.net/lists/listinfo/simple-evcorr-users
>>
>>
>
------------------------------------------------------------------------------
Get 100% visibility into Java/.NET code with AppDynamics Lite
It's a free troubleshooting tool designed for production
Get down to code-level detail for bottlenecks, with <2% overhead.
Download for free and get started troubleshooting in minutes.
http://p.sf.net/sfu/appdyn_d2d_ap2
_______________________________________________
Simple-evcorr-users mailing list
Simple-evcorr-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/simple-evcorr-users
