The following module was proposed for inclusion in the Module List:
modid: RADIUS::XMLParser
DSLIP: bdpOp
description: Radius log file XML convertor
userid: AAMEND (Antoine Amend)
chapterid: 14 (Security_and_Encryption)
communities:
CVS repository
similar:
RADIUS::Dictionary
rationale:
Working for a large telco operator, I have to parse all over the
day thousands of RADIUS log file (300K lines each, - 10Go each) and
store them into a database. Due to massive data set I need to deal
with, I have realized that RADIUS events are actually randomly
displayed in a given file (e.g. STOP event does not immediately
follow a START event) and therefore cannot be processed as-is. Any
events must be first retrieved / parsed and then grouped by their
unique session ID before being loaded as a one into a database /
FS...etc. Besides, this module will convert the Radius proprietary
log file format into a XML format, processable by any data load
process.
In addition, before I internally released this module, most of the
processing time was actually due to a specific event lookup based on
its Session ID. All events (STOP / START / INTERIM) are now reported
on hash table (stored in memory), and the processing time has been
significantly improved. Processing 24h of log file is taking now
around 3h compare to 10-15h as it was on previous versions.
Every developers working for Internet Service Provider companies
should be aware about the waste time spent on analyzing these data.
I have no doubt this module might be really useful for all of them,
as it was for myself.
I sincerely hope this module will be officially released, and I am
available for any question, remarks you might have.
Sincerely, Antoine Amend
enteredby: AAMEND (Antoine Amend)
enteredon: Mon Apr 30 12:52:49 2012 GMT
The resulting entry would be:
RADIUS::
::XMLParser bdpOp Radius log file XML convertor AAMEND
Thanks for registering,
--
The PAUSE
PS: The following links are only valid for module list maintainers:
Registration form with editing capabilities:
https://pause.perl.org/pause/authenquery?ACTION=add_mod&USERID=d3a00000_968d1f12df1420eb&SUBMIT_pause99_add_mod_preview=1
Immediate (one click) registration:
https://pause.perl.org/pause/authenquery?ACTION=add_mod&USERID=d3a00000_968d1f12df1420eb&SUBMIT_pause99_add_mod_insertit=1
Peek at the current permissions:
https://pause.perl.org/pause/authenquery?pause99_peek_perms_by=me&pause99_peek_perms_query=RADIUS%3A%3AXMLParser
