On May 23, 2014, at 6:53 PM, "BBcan177" <[email protected]> wrote:
>>> >>> I have been working on a script that downloads over 50 different Blocklists >>> and performs a duplication check to reduce the size of the data. It can >>> download .CSV, .TXT, ,GZ, .ZIP files and also scrape from certain websites >>> that post only a web copy of their Blocklists. > > ie : ET, Spamhaus, IBlock, dShield, Atlas, Alienvault etc.. I have been > researching Blocklists for several Months and have found the current list to > be beneficial in Blocking Malicious IPs. > > It utilizes a tool called "Grepcidr" to make the de-duplication work. > > It also looks at the number of IP addresses found in a /24 range and can > condense the list and enter a /24 block instead. This is done in three ways > > 1) Using a "max" variable, if it finds over the Max variable it will perform > a Maxmind Geoip Database lookup and will process a /24 block for configured > Foreign Countries on an individual Blocklist Basis. > 2) Using a "dmax" variable if it finds over the dmax variable it will > perform a Maxmind Geoip Database lookup and will process a /24 block for > configured Foreign Countries at the end of the download process on all of the > Blocklists together. > 3) Using a "pmax" variable, if it finds over the dmax variable it will > process a /24 Block excluding Country Code whitelist at the end of the > download process on all of the Blocklists together. > > So I set max to 5, dmax to 5 and pmax to 50 in my setup. > > Depending on how aggressive / conservative an admin wants to configure the > processes or disable them completely and just use the de-duplication > processes. > > It is currently outputting to a text file in an "x.x.x.x/zz" format but can > be configured for any output format. > > I have been testing it for several weeks in my pfsense pfBlocker Application > which loads the files into Blocklist tables. > > I also found a way to use the Maxmind database to make a Country Code > Specific Blocklist, excluding whitelisted countries. > > If anyone would like to test the script out, I would be more than happy to > release it. (send me an email). > > Yes very much please :) > > -- > > --- > You received this message because you are subscribed to the Google Groups > "ossec-list" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > For more options, visit https://groups.google.com/d/optout. -- --- You received this message because you are subscribed to the Google Groups "ossec-list" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
