Hi,
Like James, I wrote tools that handle several 10K variables on 5 minute
interval.
I am not at all trying to discourage you in your project. There are some
concern you should keep in mind:
Please always remember that the primary role of the network / network equipment
you try to manage is to transport data.
I mean payload data, not management data. Have you estimated/calculated the
ration of the bandwidth you will consume "just" for management?
(You did not mention how many counters per routers you intend to collect data
about.)
Measurement should not bias measured (or only to a minimal extend).
Switchs and routers are not designed to assist to this extend, the management
station.
SNMP is not the best method (high CPU load, low priority on the equipment) =>
How about Netflow ( nfsen is a wonderful free tool , if you accept 5 min.
period)
There is a big difference between what can be done and what make sense to be
done.
In any way Get-bulk (if several counters par routers) seems more appropriate
than Traps.
There are some questions you should consider and find an answer to:
How are configured the time-outs and the retries of your SNMP requests (you
intend to address some "real" equipment that may respond with latency) ?
Two seconds time-out and 3 retries don't make really sense when polling
every second.
Impact of a "non-responding" device? (a faulty one, defectuous one), of
several faulty devices (let say 10). What happens to the 190 others?
What is the latency of the network that interconnect your 200 routers?
How do you "time-stamp" the collected data ?
Good luck with your work.
JM
-----Message d'origine-----
De : Richard Mayers [mailto:richard.mayer...@gmail.com]
Envoyé : dimanche, 8. mai 2016 12:11
À : j...@mindspring.com; net-snmp-users@lists.sourceforge.net
Objet : Re: Best practice to get counters from a huge amount of routers.
Hi,
Thanks for the replies.
What about using Traps? Can I make the routers send me the counters every
second ? Is it hard to set up?
Kind regards,
Richard
2016-05-07 20:56 GMT+02:00 James Leu <j...@mindspring.com>:
> For my day job we tried open source packages for SNMP polling
> like:
>
> mrtg
> cricket
>
> They were fine for small numbers and long intervals, but will not
> scale to your usage.
>
> I wrote an implementation from scratch that handles 10K variables on
> 5 minute interval. So it can be done.
>
> Issues I think you will run into:
>
> router CPU: hitting a device every second for a number of interfaces may
> consume
> too much CPU time to be practical.
>
> storage IO: if you are successful in retrieving the data, standard DB
> storage even RRD will not be able to handle the IO load
>
> Things to consider:
>
> Bulk SNMP gets
> RMON
> local device scripting, I believe Cisco routers can do local TCL
> scripting and junos based devices have local scripting as well.
>
> Directory based queues with small files storing samples
>
> Good luck with your work.
>
> On Sat, May 07, 2016 at 05:20:53PM +0200, Richard Mayers wrote:
>> Hi folks,
>>
>> For my master thesis I am doing a load balancing project and I have
>> to know the link usage if possible every second. For that I set the
>> refresh interval to 1 second, so every thing is good so far.
>>
>> My problem is that I am working with big topologies and I may have
>> 200 or more routers. If I get the counters polling it takes forever I
>> can not poll the routers one by one, or not even using threads (at
>> some point it would not scale).
>>
>> What would be the best way to get all the counters ?
>>
>> Since I am simulating everything in a single machine I can do a trick
>> and write the counters in a file, however that will not be useful
>> when I test my solution in a real network.
>>
>> Kind regards,
>> Richard
>>
>> ---------------------------------------------------------------------
>> --------- Find and fix application performance issues faster with
>> Applications Manager Applications Manager provides deep performance
>> insights into multiple tiers of your business applications. It
>> resolves application problems quickly and reduces your MTTR. Get your
>> free trial!
>> https://ad.doubleclick.net/ddm/clk/302982198;130105516;z
>> _______________________________________________
>> Net-snmp-users mailing list
>> Net-snmp-users@lists.sourceforge.net
>> Please see the following page to unsubscribe or change other options:
>> https://lists.sourceforge.net/lists/listinfo/net-snmp-users
>
>
> --
> James R. Leu
> j...@mindspring.com
------------------------------------------------------------------------------
Find and fix application performance issues faster with Applications Manager
Applications Manager provides deep performance insights into multiple tiers of
your business applications. It resolves application problems quickly and
reduces your MTTR. Get your free trial!
https://ad.doubleclick.net/ddm/clk/302982198;130105516;z
_______________________________________________
Net-snmp-users mailing list
Net-snmp-users@lists.sourceforge.net
Please see the following page to unsubscribe or change other options:
https://lists.sourceforge.net/lists/listinfo/net-snmp-users
------------------------------------------------------------------------------
Find and fix application performance issues faster with Applications Manager
Applications Manager provides deep performance insights into multiple tiers of
your business applications. It resolves application problems quickly and
reduces your MTTR. Get your free trial!
https://ad.doubleclick.net/ddm/clk/302982198;130105516;z
_______________________________________________
Net-snmp-users mailing list
Net-snmp-users@lists.sourceforge.net
Please see the following page to unsubscribe or change other options:
https://lists.sourceforge.net/lists/listinfo/net-snmp-users
