Hi Paolo and Scott, Your discussion pointed out an important issue - what is the proper design when it comes to high frequently IP-based queries?
Inspired by this thread, my idea is to use postgresql and count on its cidr/inet function. (I am using mysql and store ipv4 as string(15)) This way the IP address is readable and processing IP address is also efficient. It seems worthy to try. I will try this approach later. Any comment is appreciated. -----Original Message----- From: pmacct-discussion [mailto:pmacct-discussion-boun...@pmacct.net] On Behalf Of Paolo Lucente Sent: Thursday, September 18, 2014 8:44 AM To: Scott Pettit - Vorco Cc: pmacct-discussion@pmacct.net Subject: Re: [pmacct-discussion] mySQL ATON/NTOA and pmacct performance Hi Scott, Great. Let me know how your testing goes. Should you hit any bugs or gaps, feel free to contact me privately to solve them. Then we can summarize on the list. Cheers, Paolo On Wed, Sep 17, 2014 at 04:35:05AM +0000, Scott Pettit - Vorco wrote: > In MySQL (any SQL actually), string searches are computationally very > expensive when compared with integers. If you’re dealing with a > relatively small number of rows then the difference isn’t a problem. > > It’s like asking “show me all the rows where the number is between 1 > and 1000” versus “show me all the rows that look like something > between one and one thousand in words”. The second statement is the > same question but requires a lot more processing because the database > has to think about every number as a word. It might only take > fractions of a second longer for one row, but multiply that by 300 > million rows (which is what I’m trying to analyse) and the difference > is huge. PostgreSQL handles this really well as it has support for > IP’s as a column type which presumably converts to integers in the back end. > > I will try pmacct with ipv4 on my test collector with sql_num_hosts > and if it works well then having the same for inet6_aton would be great. > > -Scott > > > On 17/09/14 16:03, "itria30...@itri.org.tw" <itria30...@itri.org.tw> wrote: > > >Hi Scott and Paolo, > > > > May I ask a question per your good experience? > > > > What is the performance difference between using string ipv4 and > >integer ipv4? And potential impact? > > > > I have a PC running sfacctd that collects 2 giga-switches. And it > >collects around 5M records into mysql per day. > > > > I am satisfied with the default string ipv4 for its convenience of > >being readable. And I would like to know the performance drawback. > > ________________________________ > The content of this message and any attachments may be privileged, > confidential or sensitive. Any unauthorised used is prohibited. Views > expressed in this message are those of the individual sender, except where > stated otherwise with appropriate authority. All pricing provided is valid at > the time of writing only and due to factors such as the exchange rate, may > change without notice. Sales are made subject to our Terms & Conditions, > available on our website or on request. > ________________________________ _______________________________________________ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists ==================================================================== 本信件可能包含工研院機密資訊，非指定之收件者，請勿使用或揭露本信件內容，並請銷毀此信件。 This email may contain confidential information. Please do not use or disclose it in any way and delete it if you are not the intended recipient. _______________________________________________ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists
