I would normally agree, and although I dont have all the answers I do know our network is somewhat different. in this environment the heaviest talkers are machines that send tons, and tons of tiny packets (basically exactly what you describe below :) ). What are you using as a collector? As i mentioned maybe ntop can handle this, but using flow-tools and flowscan/cuflow definately couldnt hang, which is all perl based.
On Thu, Feb 12, 2009 at 3:42 PM, Gary Gatten <[email protected]> wrote: > Something is not right! 500Mb/s is "nothing" if the traffic is typical > and not 500Mb/s of 64byte packets all with unique source and destination > info – Ie: some sort of DoS or test environment. > > > > The POS box I have sees peaks of 700Mb/s and still only uses maybe 60% > cpu. What are your rrd configs? Maybe that's what's slowing everything > down? If you're using "full" and saving hosts, interfaces, etc. etc. – that > could be it. You're 2.4GHz system should EASILY handle 500Mb/s using > netflow and not even break a sweat. > > > > G > > > > > ------------------------------ > > *From:* [email protected] [mailto:[email protected]] *On Behalf Of > *Kyle McLerren > *Sent:* Thursday, February 12, 2009 5:30 PM > > *To:* [email protected] > *Subject:* Re: [Ntop] Ntop NetFlow Question > > > > Well hopefully I can look forward to such a feature in the future. I'll > play more with tweaking some settings, but in an environment like ours, > processing flows on over 500Mbps worth of traffic is... interesting :) Even > on a quad-core 2.4ghz collector with 4gb of ram and 10k SAS disks the flows > were taking 10 minutes a piece to process with sampling turned off. > > Might just have to bite the bullet and move to some expensive but robust > solution :) thanks again. > > On Thu, Feb 12, 2009 at 3:20 PM, Gary Gatten <[email protected]> wrote: > > Seems like it would be simple – multiply rx stats by sample rate before > storing/graphing? I don't do much development so who knows. > > > > I would maybe try without sampling, but maybe set your active/inactive > timers to 120/60? It's not as real-time as it could be, but if you have a > ton of dynamic traffic it will slow down the flow rate. > > > > I'm only seeing traffic of around 100Kpps, but I'm on an old P-III and it's > only using 25% during peaks. Surely a fast box can keep up with netflow > exports from really high util – unless every packet is a different "flow" – > like during some sort of DoS attack. > > > > G > > > ------------------------------ > > *From:* [email protected] [mailto:[email protected]] *On Behalf Of > *Kyle McLerren > *Sent:* Thursday, February 12, 2009 4:03 PM > > > *To:* [email protected] > *Subject:* Re: [Ntop] Ntop NetFlow Question > > > > Thanks for the response. We are doing a boat load of traffic. We had to > enable sampling because our previous collector could no longer cope with the > sheer volume of flows it was trying to process. I do admit I dont know if > ntop can process the flows better as I just started to use it. Previous we > were using flow-tools and flowscan/cuflow. I should not have said > "accurate," as they are prefectly accurate. It just others get confuse when > looking at them and it gets old telling people to keep in mind the data is > accurate, its just the numbers are "smaller" then they actually really are > :) > > so it looks like the answer is no, there isnt a way to set the sample > rate.. seems like a really basic and easy feature to implement, would be > great to see it! > > thanks again. > > On Thu, Feb 12, 2009 at 1:54 PM, Gary Gatten <[email protected]> wrote: > > How'd anyone ever function without Google? > > > > http://www.mail-archive.com/[email protected]/msg11605.html > > > > > > > ------------------------------ > > *From:* [email protected] [mailto:[email protected]] *On Behalf Of > *Gary Gatten > *Sent:* Thursday, February 12, 2009 3:49 PM > *To:* [email protected] > *Subject:* Re: [Ntop] Ntop NetFlow Question > > > > I have seen this asked and I think answered a couple times, but since I > don't do sampling I wasn't paying much attention. I'm sure if you searched > the list you'd see some answers from Luca and/or Burton. I searched the FAQ > and man page and couldn't find anything. Also checked the "Preferences" and > couldn't see anything there either. MAYBE check the globals-define.h; > there's lots of cool stuff in there but you have to recompile with changes > L > > > > Of course the easy answer is to NOT sample. Unless you have a $HIT load of > traffic it will be OK, especially if you set the active/inactive export > timers to something reasonable. > > > > BTW: What is "not accurate"? > > > > > > > ------------------------------ > > *From:* [email protected] [mailto:[email protected]] *On Behalf Of > *Kyle McLerren > *Sent:* Thursday, February 12, 2009 3:13 PM > *To:* [email protected] > *Subject:* [Ntop] Ntop NetFlow Question > > > > Hi All, > > Im sure this has been answered before, but I couldnt find an answer > anywhere. I use sampled netflow, and I simply wanted to know if theres an > option with the ntop netflow plugin to configure the sample rate? Otherwise, > my results arent acurate. Im sending 1 out of 100 from my router. > > thanks! > > "This email is intended to be reviewed by only the intended recipient and > may contain information that is privileged and/or confidential. If you are > not the intended recipient, you are hereby notified that any review, use, > dissemination, disclosure or copying of this email and its attachments, if > any, is strictly prohibited. If you have received this email in error, > please immediately notify the sender by return email and delete this email > from your system." > > "This email is intended to be reviewed by only the intended recipient and > may contain information that is privileged and/or confidential. If you are > not the intended recipient, you are hereby notified that any review, use, > dissemination, disclosure or copying of this email and its attachments, if > any, is strictly prohibited. If you have received this email in error, > please immediately notify the sender by return email and delete this email > from your system." > > > _______________________________________________ > Ntop mailing list > [email protected] > http://listgateway.unipi.it/mailman/listinfo/ntop > > > > "This email is intended to be reviewed by only the intended recipient and > may contain information that is privileged and/or confidential. If you are > not the intended recipient, you are hereby notified that any review, use, > dissemination, disclosure or copying of this email and its attachments, if > any, is strictly prohibited. If you have received this email in error, > please immediately notify the sender by return email and delete this email > from your system." > > > _______________________________________________ > Ntop mailing list > [email protected] > http://listgateway.unipi.it/mailman/listinfo/ntop > > > "This email is intended to be reviewed by only the intended recipient > and may contain information that is privileged and/or confidential. If you > are not the intended recipient, you are hereby notified that any review, > use, dissemination, disclosure or copying of this email and its attachments, > if any, is strictly prohibited. If you have received this email in error, > please immediately notify the sender by return email and delete this email > from your system." > > _______________________________________________ > Ntop mailing list > [email protected] > http://listgateway.unipi.it/mailman/listinfo/ntop > >
_______________________________________________ Ntop mailing list [email protected] http://listgateway.unipi.it/mailman/listinfo/ntop
