Nevermind my email! I found: src/probabilistic/cardinality-counter.bif
Thanks, Aashish On Mon, May 9, 2016 at 2:29 AM, Aashish Sharma <asha...@lbl.gov> wrote: > Matthias, > > I am encountering some big tables in my scan-detection heuristics and which > grow due to scanners: > > So was thinking of this possibility to use counting bloomfilters instead of > tables and sets. After-all we are still looking for cardinality of tables and > sets for identifying scanners. > > for example: > > 1) global distinct_peers: table[addr] of set[addr] > > then .... > ..... > > if ( orig !in distinct_peers ) > distinct_peers[orig] = set() &mergeable; > > if ( resp !in distinct_peers[orig] ) > add distinct_peers[orig][resp]; > > local n = |distinct_peers[orig]|; > > > and if n > N - its a scanner !!! > > > SO I was wondering can the following be somehow represented as combinations > of counting bloomfilters: > > 1) global distinct_peers: table[addr] of set[addr] > > and/or > > 2) global distinct_backscatter_peers: table[addr] of table[port] of > set[addr] > > Aashish > > > Here is an example proof-of-concept policy of what I am tryig to explore: > > ======================= bloom-scan.bro ========== > > module Scan; > > global src: opaque of bloomfilter ; > global dst_port: opaque of bloomfilter ; > > > event bro_init() > { > > src = bloomfilter_counting_init(3, 128, 100000000); > dst_port = bloomfilter_counting_init(3, 128, 100000000); > } > > > function check_bloom (c: connection) > { > > local orig = c$id$orig_h; > local resp = c$id$resp_h ; > local resp_p = c$id$resp_p ; > > > if (resp_p == 40884/tcp || resp_p == 40876/tcp) > return ; > > bloomfilter_add (src, orig); > bloomfilter_add (dst_port, fmt("%s%s", resp, resp_p)); > > > local src_counts = bloomfilter_lookup(src, orig) ; > local dst_counts = bloomfilter_lookup(dst_port, fmt("%s%s", resp, > resp_p)) ; > > #### idea here is that a remote scanner is going to be hitting a lot > of local hosts > #### so footprint (conn counts of the remote scanner) is going to be > dis-propotionate to > ### footprint of local host+port > > if (src_counts > 30 && dst_counts < 5) > print fmt ("possible_scanner: %s -> %s on %s ( counts: %s, > %s)", orig, resp, resp_p, src_counts, dst_counts); > > > } > > > event partial_connection(c: connection) > { > Scan::check_bloom(c); > } > > event connection_attempt(c: connection) > { > Scan::check_bloom(c); > } > > event connection_half_finished(c: connection) > { > # Half connections never were "established", so do scan-checking here. > Scan::check_bloom(c); > } > > event connection_rejected(c: connection) > { > Scan::check_bloom(c); > } > > event connection_reset(c: connection) > { > Scan::check_bloom(c); > } > > event connection_pending(c: connection) > { > if ( c$orig$state == TCP_PARTIAL && c$resp$state == TCP_INACTIVE ) > Scan::check_bloom(c); > } > > _______________________________________________ bro-dev mailing list bro-dev@bro.org http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev