Re: [c-nsp] Sup2T / EARL8 Netflow oddities

2013-05-28 Thread Chris Welti
Dear Jiri,

we have similar netflow issues with our Sup2T-XL upgrades from the Sup720-3CXL.
In general, all show platform flow commands are incredibly slow and tend to take
minutes. Yes, I have waited longer than 10 minutes for certain show commands
to complete, which were almost instant on the sup720. The CLI in general appears
to be laggy now as well.
Also, it seems that fast aging is not supported on DFC line cards, at least I 
have
found no way to make them do that.
In addition, even on the platform cache of the Sup2T card, the lowest value for
fast aging is 32 seconds compared to 1 second on the Sup720, which leads to
the fact that it tends to fill up a lot faster than on the Sup720.
So, even though the Sup2T is supposed to have more netflow capacity and 
performance,
the contrary seems to be the case.
I wonder if anyone at Cisco has ever tested any of their netflow show commands
on the Sup2T when the netflow table is filled with a few 100K flows. It's 
unbearable.
It would certainly be time to activate that second, unused core of the Sup2T 
CPU and
dedicate it to netflow processing :)

Regards,
Chris

Network Engineer
SWITCH NOC
AS559


Jiri Prochazka jiri.prochazka at superhosting.cz wrote on Tue Mar 26 11:37:00 
EDT 2013

Hi,

after replacing one of our old vs-s720-3cxl and 6708-3cxl combo for a 
new sup2t-xl and 6908-2txl I'm struggling with a really poor netflow 
performance.

In fact, enhanced netflow capacity and capabilities were the major 
reasons for upgrade.

On the old vs-s720-3cxl setup we have used interface-src-dst flowmask. 
With aggresive timing, this setup was able to 'handle' around 6 Gbps of 
strandard Internet traffic (per DFC) without undercounting and 
overwhelming the whole box.


Now, when using sup2t-xl, which has two times bigger netflow table (512k 
for ingress flows) and faster CPU, I'm not able to get it working with 
even with the same level of traffic.


As soon as traffic on ingress reaches aproximately 3 Gbps, and number of 
flows per one cache(card) exceeds 200k, the whole box begins to be 
unresponsive to SNMP polls, timeouts some commands (for example show 
platform flow ip count module x) and the CLI begins to lag.

Furthermore, I get a lot of following messages -

%IPC-DFC2-5-WATERMARK: 2013 messages pending in rcv for the port 
Card2/0:Request(202.7) seat 202
%IPC-DFC2-5-WATERMARK: 2019 messages pending in rcv for the port 
Card2/0:Request(202.7) seat 202


Utilization of CPU either of Sup or linecards is acceptable (under 60%, 
majority is taken by 'NF SE export thr' and 'NF SE Intr Task' processes).


Settings of netflow is following -

flow record SRC-IP-IF-DST-IP-IF-AS
  match ipv4 source address
  match ipv4 destination address
  collect routing source as
  collect routing destination as
  collect routing next-hop address ipv4
  collect interface input
  collect interface output
  collect counter bytes
  collect counter packets
  collect timestamp sys-uptime first
  collect timestamp sys-uptime last


flow monitor LIVEBOX-MONITOR
  description LIVEBOX v9 monitor
  record SRC-IP-IF-DST-IP-IF-AS
  exporter LIVEBOX-EXPORT
  cache timeout inactive 3
  cache timeout active 60

flow exporter LIVEBOX-EXPORT
  destination x.x.x.x
  source Vlanx
  transport udp 9996




Did you notice any REAL perfomance boost compared to older Sup720 with 
B/CXL DFCs?


Thank you!



-- 
Jiri Prochazka
network administrator (AS39392)
SuperNetwork s.r.o.


___
cisco-nsp mailing list  cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/


Re: [c-nsp] Sup2T / EARL8 Netflow oddities

2013-05-05 Thread Simon Leinen
Jeroen van Ingen writes:
 Our university upgraded from Cat6k/Sup720-3B to Cat6k/Sup2TXL a while
 ago. Recently a few researchers who use our NetFlow data noticed that
 the NetFlow exports sometimes contain strange values: there are flow
 records with a negative duration (flow end before flow start time) and
 some exported flows are far (1 month) in the past or future.

 We're currently running IOS 15.1(1)SY. Has anyone else noticed
 something similar?

Yes, in all releases since we got our first Sup 2Ts.  Quite annoying.
No idea whether this was already reported to Cisco as a bug.

If this happens, the start time looks reasonable, but the end time is
typically around 4194 seconds *lower* than the start time.  In my own
code, I fix this by increasing the end time by 4194 seconds (maybe
4195 would be better).

 If anyone wants to check their NetFlow v9 exports: Wireshark will show
 flowsets containing flow records with negative duration when using the
 display filter 'cflow.timedelta  0'.

Great tip, thanks! As an illustration, here's an extract of a decoded
trace of Netflow v9 packets from one of our Sup 2T routers.  It shows
the range of the time differences:

$ tshark -V -d udp.port==9910,cflow -r ce3-flows.pcap 'cflow.timedelta  0' | 
grep 'Duration: -'
[Duration: -4194.05000 seconds]
[Duration: -4194.05000 seconds]
[Duration: -4194.05000 seconds]
[Duration: -4194.05000 seconds]
[Duration: -4194.05000 seconds]
[Duration: -4194.05000 seconds]
[Duration: -4194.05000 seconds]
[Duration: -4194.05000 seconds]
[Duration: -4194.05000 seconds]
[Duration: -4194.1 seconds]
[Duration: -4194.1 seconds]
[Duration: -4194.1 seconds]
[Duration: -4194.05000 seconds]
[Duration: -4194.05000 seconds]
[Duration: -4194.05000 seconds]
[...]
-- 
Simon.
___
cisco-nsp mailing list  cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/


[c-nsp] Sup2T / EARL8 Netflow oddities

2013-04-29 Thread Jeroen van Ingen
Hi,

Our university upgraded from Cat6k/Sup720-3B to Cat6k/Sup2TXL a while
ago. Recently a few researchers who use our NetFlow data noticed that
the NetFlow exports sometimes contain strange values: there are flow
records with a negative duration (flow end before flow start time) and
some exported flows are far (1 month) in the past or future.

We're currently running IOS 15.1(1)SY. Has anyone else noticed something
similar?

If anyone wants to check their NetFlow v9 exports: Wireshark will show
flowsets containing flow records with negative duration when using the
display filter 'cflow.timedelta  0'.


Regards,

Jeroen van Ingen
ICT Service Centre
University of Twente, P.O.Box 217, 7500 AE Enschede, The Netherlands


___
cisco-nsp mailing list  cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/