On 05/23/2014 05:45 PM, BBcan177 wrote:
I have been working on a script that downloads over *_50
different Blocklists_* and performs a duplication check to
reduce the size of the data. It can download .CSV, .TXT, ,GZ,
.ZIP files and also scrape from certain websites that post only
a web copy of their Blocklists.
Sounds wonderful. Would you be willing to share your threat source list?
ie : ET, Spamhaus, IBlock, dShield, Atlas, Alienvault etc.. I have been
researching Blocklists for several Months and have found the current
list to be beneficial in Blocking Malicious IPs.
It utilizes a tool called "Grepcidr" to make the de-duplication work.
It also looks at the number of IP addresses found in a /24 range and can
condense the list and enter a /24 block instead. This is done in three ways
OSSEC CDB doesn't understand class boundaries. Can it be easily modified
to account for this?
(http://ossec-docs.readthedocs.org/en/latest/manual/rules-decoders/rule-lists.html)
It is currently outputting to a text file in an "x.x.x.x/zz" format but
can be configured for any output format.
My script outputs in key:value format, as required by OSSEC CDB, then
compiles it. I also like to keep the original txt file around for
reference.
One issue I ran into while testing is that one site only allowed a pull
every four hours so I would get banned for 24 hours. I had to ensure
that I didn't run it too often; a better way would have been to include
some sort of state file that gets checked and skips that list if it is
too soon.
--
---
You received this message because you are subscribed to the Google Groups "ossec-list" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
For more options, visit https://groups.google.com/d/optout.
