[pmacct-discussion] pmacct and FreeBSD/OpenBSD

2009-07-29 Thread Paolo Lucente
Dears, I'm in the need of performing some basic pmacct tests on recent FreeBSD and/or OpenBSD (4.3/4.4) operating systems. Reason being double-check the new major release (0.12), which will be out very soon, compiles fine. Any good souls can provide an unpriviledged temporary (24/48 hours) access

Re: [pmacct-discussion] pmacct and FreeBSD/OpenBSD

2009-07-29 Thread Paolo Lucente
Sorry for replying to myself. Thanks to the good souls who replied; I've found the FreeBSD and OpenBSD resources i was looking for. Cheers, Paolo On Wed, Jul 29, 2009 at 10:12:43AM +0100, Paolo Lucente wrote: Dears, I'm in the need of performing some basic pmacct tests on recent FreeBSD

Re: [pmacct-discussion] Flexible aggregation

2009-06-14 Thread Paolo Lucente
Hi Karl, On Sat, Jun 13, 2009 at 04:30:07PM -0500, Karl O. Pinc wrote: A good database should not have problems with simultaneous updates, or is there another reason why synchronization is an issue? No, not really - expecially when it's down to an INSERTs-only scenario. Just an effort to help

Re: [pmacct-discussion] Flexible aggregation

2009-06-14 Thread Paolo Lucente
Hi Chris, On Sun, Jun 14, 2009 at 02:25:10PM +0300, Chris Wilson wrote: I don't think it can. For example, how would we write the configuration? Let's say we just want to zero (not aggregate on) the destination IP for flows less than 1000 bytes. We could try: plugins: mysql[with_dst],

Re: [pmacct-discussion] Flexible aggregation

2009-06-13 Thread Paolo Lucente
Hi Karl, On Sat, Jun 13, 2009 at 03:03:04PM -0500, Karl O. Pinc wrote: What really is the constrained resource here? Is it the number of transactions the database supports or is it something more fundamental to the pmacct performance like cpu or memory constraints? What I'm thinking is

Re: [pmacct-discussion] Maximum Number of Plugin Entries ?

2009-06-08 Thread Paolo Lucente
Hi Axel, The only thing that comes to my mind is that in pmacct-defines.h is statically defined a maximum number of plugins, 32 by default: #define MAX_N_PLUGINS 32 Meaning that shouldn't be the problem (also because the plugin is effectively created and the issue pops up when trying to connect

Re: [pmacct-discussion] input/output interface information in nfacctd?

2009-06-08 Thread Paolo Lucente
Hi Suraj, This information is not immediately available within the database or memory table; but you can match such fields within the Pre-Tagging infrastructure to generate a tag - which can, in turn, be either just used internally for filtering or splitting data among the plugins or can be made

Re: [pmacct-discussion] nfacctd opening multiple database connections (and not closing them)

2009-05-11 Thread Paolo Lucente
Hi Peter, It should be just a matter of configuration. Maybe a SHOW PROCESSLIST within the MySQL client can tell something more; you should find that those processes are lining up to obtain the LOCK to write to the SQL table. Why processes are queueing up? Because writing takes much time;

Re: [pmacct-discussion] How scalable is pmacct

2009-05-11 Thread Paolo Lucente
Hi Brad, Speaking about scalability pmacctd i would recommend to start from reading Q5 of the FAQS document: internal buffering (which i see it's not enabled in your configuration below) and, if not sufficient, resort to: device polling, PF_RING or a mmap() version of libpcap. Perhaps at those

Re: [pmacct-discussion] bandwidth monitoring

2009-05-01 Thread Paolo Lucente
320gb * 5 = 1600Gb. However the numbers above show only 430Gb. Thanks On Wed, Apr 29, 2009 at 1:38 PM, Paolo Lucente pa...@pmacct.net wrote: Hi Anil, In principle what you are doing looks correct: perhaps for a cleaner result i would use filters to separate the traffic, two

Re: [pmacct-discussion] Wildly inaccurate data sometimes - Please Help

2009-05-01 Thread Paolo Lucente
Hi Mitch, I might help but unfortunately the only way sounds to provide access to the box and related data. Provided there is some disk space, and after some analysis of course, it could be an idea to compile pmacct with ad-hoc debugging information around suspicious points and swap the

Re: [pmacct-discussion] Q. about aggregate_filter and nfacctd

2009-05-01 Thread Paolo Lucente
Hi Matt, Good, you already tried out what would have been my first suggestion. Something else i would recommend traffic load permitting: disable buffering (plugin_buffer_size) whenever testing a new configuration: to be sure nothing remains trapped within the buffers giving the feeling something

Re: [pmacct-discussion] bandwidth monitoring

2009-04-29 Thread Paolo Lucente
Hi Anil, In principle what you are doing looks correct: perhaps for a cleaner result i would use filters to separate the traffic, two plugins, two SQL tables - one for incoming the other for outgoing traffic. Anyway, the result of your SQL query is to account for traffic between your servers and

Re: [pmacct-discussion] timestamp rounding bug

2009-04-19 Thread Paolo Lucente
Hi Alex, DST is not supported. Timezones are. The idea behind this was that a backend application (like pmacct is) should ideally work only with UTC (even if timezones are supported) and then front-ends should localize the time as required. DST doesn't introduce further side-effects to pmacct

Re: [pmacct-discussion] Strange SQL-Error

2009-04-13 Thread Paolo Lucente
Hi Johannes, Chris, Thanks to the both of you for progressing troubleshooting of the problem. A lot of interesting elements are already there but there is something not clear yet about the nature of this issue: * The SEGV appears to be in a pthread_getspecific() call while the MySQL plugin

Re: [pmacct-discussion] Dynamic Table not created

2009-04-09 Thread Paolo Lucente
. 2009/4/8 s.kub...@gmail.com Hi Paolo, i've created the table manually and the table is populated by pmacct. Now i get no errors in messages log. mysqld.log shows no errors. The only exisiting table before the manual created acct_20090408 was the standard acct. 2009/4/8 Paolo

[pmacct-discussion] Open documentation effort

2009-04-05 Thread Paolo Lucente
Hi Guys, I would like to introduce you an initiative of the pmacct project which is starting today; i hope it will be successful and you all will help growing: The open documentation effort @ http://wiki.pmacct.net/ Documentation is a key part of every project; we feel alongside official

Re: [pmacct-discussion] Help, nfacctd records no data. :(

2009-03-18 Thread Paolo Lucente
Hi Matt, Effectively with debug enabled, your output should be a little bit more verbose. As a first step i would carry out some basic checks like: a) make sure nfacctd_ip binds to the correct IP address and b) double check there is no firewall rule prevening packets to be delivered to nfacctd

[pmacct-discussion] pmacct-contribs 20090314 released

2009-03-14 Thread Paolo Lucente
VERSION. 20090314 DESCRIPTION. pmacct is a set of network tools to gather, filter and tag IP traffic; it is able to store collected data either into a DB or a memory table. We see any monitoring, billing or accounting environment as a stack where data are picked from the network, get processed

Re: [pmacct-discussion] pmacct-contribs 20090314 released

2009-03-14 Thread Paolo Lucente
Hi Sander, That is correct - my bad. Thanks very much for pointing that out. Cheers, Paolo On Sat, Mar 14, 2009 at 12:53:42PM +0100, Sander Hoentjen wrote: On Sat, 2009-03-14 at 10:13 +, Paolo Lucente wrote: VERSION. 20090314 SNIP DOWNLOAD. http://www.pmacct.net/pmacct

Re: [pmacct-discussion] pmacct weird counters

2009-03-14 Thread Paolo Lucente
Hi Chris, About the SQL INSERT conflict, are you by any chance making use of the sql_dont_try_update directive in your configuration? And are you using 32bit counters? The conjunction of these two conditions might explain. The SQL cache code, while summing up counters, makes a check on whether

Re: [pmacct-discussion] Patch to support comments in networks_file

2009-03-10 Thread Paolo Lucente
Hi Karl, Thanks for contributing this patch. It makes sense, doesn't break anything and hence i've committed it right away into the CVS with the following description: * net_aggr.c: support is introduced for descriptions in the file pointed by a networks_file configuration directive. Thanks to

Re: [pmacct-discussion] increase shared memory size

2009-03-03 Thread Paolo Lucente
Hi Sven, On Tue, Mar 03, 2009 at 07:11:54PM +0100, Sven Schmitt wrote: The average packetsize is about 500 bytes per packet. Is this the information you are asking for? Yes. By looking again at your config, i see you are not applying filters per-plugin but using the networks file at a later

Re: [pmacct-discussion] pmacct/pmacctd client/server across the net

2009-02-28 Thread Paolo Lucente
, Paolo On Sat, Feb 28, 2009 at 07:49:50AM -0600, Karl O. Pinc wrote: On 02/25/2009 04:46:46 PM, Paolo Lucente wrote: All this said, let me just shoot a proposal: as the in-memory table client/server communication is already based on request/reply headers, operation codes, etc. (so let's say

Re: [pmacct-discussion] pmacct/pmacctd client/server across the net

2009-02-25 Thread Paolo Lucente
Hi Karl, I would be myself interested into it. Currently the situation is not too bad in the sense that who wants to separate client and server can still use ssh/rsh as a glue. This brings modularity, keeps everything out of pmacct (read: no dependencies) and possibly offers security if the

Re: [pmacct-discussion] HTTP Virtual Hosts classification

2009-02-19 Thread Paolo Lucente
Hi Chris, While on the topic of hardening the classification process, let me spare a couple of additional thoughts. The need for hardening the classifier can be a strict requirement for specific scenarios, while it becomes a nice to have feature in others. In your case i fully agree with you as

Re: [pmacct-discussion] HTTP Virtual Hosts classification

2009-02-18 Thread Paolo Lucente
Hi Matthias, In concept, and as documentation says, what you want to achieve is feasible and your understanding of the classifier() is correct - you only have to write down your own patterns: re-phrased, regular expressions are typically employed to recognize protocols but they can be of course

Re: [pmacct-discussion] pmacct 0.11.5 not compiling on OpenBSD 4.4

2009-02-06 Thread Paolo Lucente
Hi Karl, Present! The developer has still brainwave and currently trying to refrain his daytime employer to kick him out. Indeed, thanks for flagging the compilation issue; as this is bound to a specific DLT definition, i was actually wondering whether the underlying problem is more a stale

Re: [pmacct-discussion] Problem on using sfacctd with aggregate_filter

2009-02-06 Thread Paolo Lucente
Hi Axel, any chance the captured traffic is VLAN-tagged? If this is the case and the switch is providing such information through sFlow, then the 'aggregate_filter' should be re-written as 'vlan and ...'. Same is if it's MPLS-labelled 'mpls and ...'. If this is not the case, can you please send

Re: [pmacct-discussion] pmacct 0.11.5 not compiling on OpenBSD 4.4

2009-02-06 Thread Paolo Lucente
Hi Karl, On Fri, Feb 06, 2009 at 02:35:28PM -0600, Karl O. Pinc wrote: Unfortunately i don't have access to any OpenBSD at the moment; is that something you can give it a try? What would I look at? As i was suggesting, please download and compile a recent version of libpcap, say, 0.9.x -

Re: [pmacct-discussion] multiple interfaces

2009-01-23 Thread Paolo Lucente
Hi Mariano, maybe irrelevant for your scenario but pmacct version 0.11.5 is able to make the most profit by the fields of the DLT_LINUX_SLL header - which is prepended to the L3 header when using the any interface. This is an extract from the ChangeLog: + pmacctd, the Linux Cooked device

Re: [pmacct-discussion] tcpdump output for snort sensing

2008-10-15 Thread Paolo Lucente
Hi Joe, can you please show an example of what's the output of the sflowtool -t command, which makes snort happy? That can help addressing your question. Also, do you know which sFlow fields are relevant to snort - this is just in case sfacctd is unable to produce a dump as detailed as sflowtool

Re: [pmacct-discussion] MySQL and Duplicate Primary Keys

2008-10-08 Thread Paolo Lucente
Hi Chris, it's very strange - but from the top of my head, i can't recall any issues like the one described in pmacct 0.9.x; so, the problem could be unrelated to the vintage of the version you are using. For the tuples that you have shown in the original email, it would be nice to know whether

Re: [pmacct-discussion] netflow torture - traffic generator for a large network setup

2008-09-23 Thread Paolo Lucente
Hi Fernando, Thanks very much for appreciating the tool. Regarding the warning message: it depends. It tells you that basing on the sequence numbers, pmacct was expecting a NetFlow packet with sequence number '1980' but got one with sequence number '0'. It might be an issue with the NetFlow

Re: [pmacct-discussion] What is the best way to account SIP/RTP traffic with PMACCT

2008-09-03 Thread Paolo Lucente
Hi Sebastien, it might really depend on the scalability versus the accuracy of the solution you need to develop. I see you having pretty much two ways to go: a) Basic behavioural analysis. Assume all packets flying between the known range of RTP ports are RTP packets. If you know in advance the

Re: [pmacct-discussion] What is the best way to account SIP/RTP traffic with PMACCT

2008-09-03 Thread Paolo Lucente
Hi Sebastien, On Wed, Sep 03, 2008 at 08:02:19PM +0200, S?bastien CRAMATTE wrote: a) Basic behavioural analysis. Assume all packets flying between [ ... ] p ... seems to be very complicate ! isn't it ? ;) Well, it's the simplest approach conceptually but i get your point: it's not

Re: [pmacct-discussion] How to aggregate some hosts into networks and keep host info for others?

2008-07-29 Thread Paolo Lucente
Hi John, On Mon, Jul 28, 2008 at 11:00:45PM +, John Rouillard wrote: I assume I would also specify: networks_file[in]: pmnet.lst Correct. One other idea on the filters may be: id=1 filter='dst net 192.168.2.0/24 or dst net 192.168.3.0/24 or dst net 192.168.5.0' id=1

Re: [pmacct-discussion] How to aggregate some hosts into networks and keep host info for others?

2008-07-28 Thread Paolo Lucente
Hi John, thanks very much for your interesting email and for your interest into the pmacct project. Please follow me inline: On Sun, Jul 27, 2008 at 08:07:45AM +, John Rouillard wrote: Just started using pmacct (0.11.5) this evening on a Centos 4.5 machine. I wanted to aggregate a number

Re: [pmacct-discussion] Does nfacctd is compatible with RFlow / MACupd ?

2008-07-06 Thread Paolo Lucente
Hi Sebastien, On Sun, Jul 06, 2008 at 01:16:34PM +0200, S?bastien CRAMATTE wrote: I'm using a wireless route with dd-wrt firmware (www.dd-wrt.org). This firmware has rflow feature but I'm not sure that is compatible with nfacctd ? The best way to answer your question is to try using

Re: [pmacct-discussion] pmacct not sending MAC address

2008-06-27 Thread Paolo Lucente
Hi Ryan, i've reproduced a very simple scenario: spanned port =(libpcap)= pmacctd =(NF v9)= nfacctd. I then queried the memory plugin attached to the nfacctd daemon. I can verify MAC addresses are showing up correctly. Following is the configuration in place: === ! ! pmacctd config !

Re: [pmacct-discussion] Problems debugging netflow handling

2008-06-08 Thread Paolo Lucente
Hi Inge, what i don't get is whether you can't see these flows in the backend at all or you can see them but they come untagged. In the former case, check whether the daemon is reporting (stdout or logfile) any malformed packets and, if possible, send me privately a few packets making it to the

Re: [pmacct-discussion] Pmacct on MySQL 5.1 NDBCLUSTER as engine

2008-06-08 Thread Paolo Lucente
Hi Peter, long time no see - hope you're fine. I received a single report of somebody using pmacct with a MySQL cluster but don't know which version and never tried myself - indeed if anybody has anything to share in this sense, good or bad, please don't be afraid! From the list of constraints

Re: [pmacct-discussion] MySQL and pmacctd

2008-04-18 Thread Paolo Lucente
configurations. A value smaller than 60 secs is not generally suitable unless you have a very good reason for it. Cheers, Paolo On Thu, Apr 17, 2008 at 11:38:30AM +0200, Schultz Consult - [Ren? Madsen] wrote: Paolo Lucente wrote: Hi Rene, i know this may sound a bit simplistic: version 0.10.1

Re: [pmacct-discussion] Sort flow with ACLs

2008-04-15 Thread Paolo Lucente
Hi Denis, sorry for the late reply. You really have two options: - you can tag traffic through the Pre-Tagging infrastructure as you were suggesting. Then, you can select it on a per-plugin basis with the 'pre_tag_filter' directive. - you can use the 'aggregate_filter' directive which

Re: [pmacct-discussion] NO pmacct aggregation bug

2008-04-15 Thread Paolo Lucente
Hi Alex, please read my comments inline: On Fri, Apr 11, 2008 at 07:05:33PM +0300, alex wrote: 1. How i can in my configuration (several 'nfprobe' plugins on one machine and 'nfacctd' collector on another) setting 'agent_id' field in database? Now i use follow configurations: On

Re: [pmacct-discussion] aggregate_filter and tcpdump portrange

2008-03-30 Thread Paolo Lucente
Hi Enrico, sorry for the late reply. Keywords supported by the aggregate_filter directive highly depend on the grammar of the libpcap library pmacct is compiled against. If you have a tcpdump compiled against the same libpcap library, test its behaviour attaching the same filter, ie.: tcpdump

Re: [pmacct-discussion] Duplicate keys

2008-03-09 Thread Paolo Lucente
Hi Sander, you are absolutely correct about both the cause of the issue and its solution: manually modifying the SQL schema makes the trick. This is documented in the sql/README.IPv6 file. It refers to source and destination IP addresses but indeed the same applies to the IP protocol - will

Re: [pmacct-discussion] efficient netflow probe on linux

2008-01-25 Thread Paolo Lucente
Hi Leo, i can't personally speak about fprobe because i don't know the package; i can see your requirements are consistent with the features the 'nfprobe' plugin offers. If you want to go down that way - any feedback or critics will be much appreciated as they are vital part in the optimization

Re: [pmacct-discussion] MAC aggregation not available

2008-01-23 Thread Paolo Lucente
Hi Gregorics, the issue is aggregation of MAC addresses is currently supported only on plain Ethernet interfaces (not PPPoE for example). Would you mind sending me privately a brief capture in libpcap format of a few packets (full payload)? It might be something that could be sorted easily ...

Re: [pmacct-discussion] Sfacctd showing weird byte count

2007-12-20 Thread Paolo Lucente
Hi Amit, having the same packet sampled multiple times is not ideal for having accurate data. Here are a few advices, depending on the scanario some may apply, some may not: * if the same packet could be sampled multiple times by the same switch but in different VLANs, you can introduce VLANs

Re: [pmacct-discussion] in/out/total per IP per month

2007-12-12 Thread Paolo Lucente
Hi Daniel, in scenarios in which UPDATEs are affordable, the advice is to keep hourly and monthly stats in different tables. With proper indexing, such method is lighter compared to making sums. If UPDATEs are not affordable then, yes, there is no other choice but make sums basing on the

Re: [pmacct-discussion] in/out/total per IP per month

2007-12-05 Thread Paolo Lucente
Hi Daniel, your config is missing the imt_path directive for each of the plugins - ie. add to your config the following lines: imt_path[outbound]: /tmp/outbound.pipe imt_path[inbound]: /tmp/inbound.pipe You will be able to query each plugin's memory table with the following commandline: pmacct

Re: [pmacct-discussion] Accounting accuracy

2007-11-07 Thread Paolo Lucente
Hi Mario Antonio, sorry for getting back to you so late. There have been a few reports in the past pretty in line with what you are outlining in your email. The root of the problem seems to relate somehow to bridged interfaces and libpcap on Linux systems - nothing specifically related to pmacct.

Re: [pmacct-discussion] Problems with accounting with pmacct

2007-09-25 Thread Paolo Lucente
Hi Oliver, you should be able to find sflowtool at the following URL: http://www.inmon.com/technology/sflowTools.php Regarding how to debug the sFlow packets; you can use sflowtool itself, just get through the documentation. Another chance could be to capture and analyze them with the likes of

Re: [pmacct-discussion] Flatfiles

2007-08-02 Thread Paolo Lucente
Hi Thomas, sorry for replying late. pmacct doesn't support writing directly to flat files. This is mainly for the little motivation in merely duplicating others work (ie. flow-tools, sflowtool and the likes). However these don't cover the read from libpcap write to a flat file scenario. I would

Re: [pmacct-discussion] Unknown plugin type: mysql. Ignoring.

2007-07-19 Thread Paolo Lucente
Hi Jon, looking at the output below, it seems the make did not have any effect. Is it possible that a simple make clean does the magic? Everything else seems allright. Cheers, Paolo On Thu, Jul 19, 2007 at 04:03:18PM +1000, Jon Hall wrote: Hi, A while ago I used to run pmacct version 10.2

Re: [pmacct-discussion] nfacctd not seeing NetFlow packets

2007-07-19 Thread Paolo Lucente
Hi Paul, i know it's a trivial question, but the only on top of my mind at the moment: are you sure there aren't any firewall rules setup on the box preventing this to work? Ie. the type of socket employed by tcpdump can see packets allright even if they are dropped before reaching the

Re: [pmacct-discussion] Refresh database each second ?

2007-06-28 Thread Paolo Lucente
Hi K.L., which method are you collecting your traffic (ie. libpcap, NetFlow, sFlow)? I see your point and i think it could make sense collecting NetFlow datagrams - and a similar case, ie. each aggregate is written down to the database independly, is handled by the nfacctd_sql_log directive.

Re: [pmacct-discussion] as-path via sFlow

2007-06-28 Thread Paolo Lucente
Hi Daniel, AS-path in sFlow version 5 is not yet supported. The idea is to support it soon but just inside the Pre-Tagging infrastructure: by intercepting specific AS paths, you can write down a tag into the database. Would this match your needs? If not, would you expand them a little bit? Maybe

Re: [pmacct-discussion] Refresh database each second ?

2007-06-28 Thread Paolo Lucente
On Thu, Jun 28, 2007 at 12:24:57PM +0200, K L wrote: On 6/28/07, Paolo Lucente [EMAIL PROTECTED] wrote: Hi K.L., which method are you collecting your traffic (ie. libpcap, NetFlow, sFlow)? I see your point and i think it could make sense collecting NetFlow datagrams - and a similar case, ie. each

Re: [pmacct-discussion] pmacct -r not actually resetting counters

2007-06-14 Thread Paolo Lucente
Hi Ruben, thanks for the valuable inputs. I'm trying to reproduce the issue now, having in mind all your tests. Will come back to you as soon as i'm able to shed any light on that. Cheers, Paolo On Thu, Jun 14, 2007 at 01:08:07PM +0200, Ruben Laban wrote: I cheered too early. After letting my

Re: [pmacct-discussion] plugin_pipe_size and plugin_buffer_size increas

2007-06-11 Thread Paolo Lucente
Hi Daniel, Q5 of the FAQS document briefly outlines some very basic rules of thumb regarding bufferization, buffer values and how they should compare. You can have a try with the following values and eventually scale them downwards/upwards: plugin_buffer_size: 10240 plugin_pipe_size: 1024000

Re: [pmacct-discussion] sflow version 5

2007-05-30 Thread Paolo Lucente
Hi Daniel, no, the correct information on how to parse each packet is inferred by looking into each sFlow packet's header. So you can have mixed sFlow streams collected into a single sfacctd daemon. The same applies to nfacctd and NetFlow. Cheers, Paolo On Tue, May 29, 2007 at 10:20:38PM +0200,

Re: [pmacct-discussion] Random Counter sizes mismatch errors

2007-05-25 Thread Paolo Lucente
Hi, by default pmacct uses 32bits packets/flows/bytes counters. By using the --enable-64bit flag, you make such counters to be 64bits wide. If a pmacct client is compiled with 64bits counters, it can't read a memory table with 32bits counters - and viceversa. Hence, it kicks out that kind of

Re: [pmacct-discussion] pmacct compilation error

2007-05-25 Thread Paolo Lucente
Hi Raj, are you getting the executable out - ie. sfacctd? I can't see anything wrong with the output posted below. Cheers, Paolo On Thu, May 24, 2007 at 08:34:10PM -0400, Murugaraj Suthandiramani wrote: Hello all , Need help. I am getting the below compilation error when i do a make on

Re: [pmacct-discussion] IP billing solution for datacenter

2007-05-08 Thread Paolo Lucente
Daniel, don't know where you are getting such informations. Can you please provide any docs supporting what you are saying? Even sFlow, which intuitively should be the less reliable, can do the job by playing a bit around the error: http://www.inmon.com/pdf/sFlowBilling.pdf NetFlow is then a

Re: [pmacct-discussion] Configuration advice: Prevent missing netflow

2007-05-08 Thread Paolo Lucente
Hi, can you please outline which network device are you exporting your NetFlow from, which NetFlow version are you actually using and what's roughly the rate of the exported flows (or NetFlow packets) per second? Can you please also: a) post your configuration, if using any? b) post the result

Re: [pmacct-discussion] feature request: pre_tag_filter ranges

2007-05-07 Thread Paolo Lucente
Hi Philipp, i'm glad hearing that! I've stacked your feature request onto my todo list - that's something definitely useful and that should not slip out of my mind one of these days. Hope i would be able to include it in the next release. Cheers, Paolo On Sun, May 06, 2007 at 08:28:51PM +0200,

Re: [pmacct-discussion] dual interface configuration

2007-05-07 Thread Paolo Lucente
Ahoy to you, Florian! a single instance of pmacctd can't be bound to multiple interfaces. This is common to many of the tools based on libpcap, mainly driven by performance reasons. Here you have two options, depending on your requirements (btw, can you please explain in which scenario do you

Re: [pmacct-discussion] sfacctd v2/v5?

2007-05-02 Thread Paolo Lucente
Hi Daniel, Which network device are you getting the sFlow datagrams from? Any chance i can have a look to these samples? If yes, can you please post me privately some full-datagrams captured in libpcap/tcpdump format? That message is generated inside sfacctd.c source file and says there is an

[pmacct-discussion] pmacct 0.11.4 released !

2007-04-25 Thread Paolo Lucente
VERSION. 0.11.4 DESCRIPTION. pmacct is a small set of passive network monitoring tools to measure, account, classify, aggregate and export IPv4 and IPv6 traffic; a pluggable and flexible architecture allows to store collected network data into memory tables or SQL (MySQL, SQLite, PostgreSQL)

Re: [pmacct-discussion] Juniper cFlow, sampling and nfacctd handling

2007-04-23 Thread Paolo Lucente
Hey Inge, That's done. The updated version which includes the patch is now in the CVS. Would you give it a try? Version 0.11.4 should come later this week. Cheers, Paolo On Mon, Apr 23, 2007 at 04:26:41PM +0200, Inge Bj?rnvall Arnesen wrote: As no documentation has shown that a sampling_mode

Re: [pmacct-discussion] ip traffic accounted twice

2007-04-14 Thread Paolo Lucente
Hi Andrei, can you establish any criteria for that doubled traffic, ie. what gets doubled and what is counted once, inbound vs. outbound, etc.? Moreover, can you have a look what happens at layer2, any change in src/dst MAC addresses? That would help but because you told that's bridged traffic ...

Re: [pmacct-discussion] ip traffic accounted twice

2007-04-13 Thread Paolo Lucente
Hi Andrei, the most immediate suggestion i can give you is to tap either tcpdump or ethereal on the eth0 and see which traffic the libpcap library is effectively returning; also, try playing with the 'promisc' directive (which defaults to true) - as my understanding is that you are sniffing

Re: [pmacct-discussion] doubled traffic

2007-03-17 Thread Paolo Lucente
Hi Alan, On Fri, Mar 16, 2007 at 09:27:34AM -0700, Alan wrote: I've run tcpdump on both hosts with the filters in it from the pmacct settings above and the bytes that are caught by tcpdump match properly, however the values that are written by pmacctd to the respective databases are

Re: [pmacct-discussion] i see traffic to 0.0.0.0

2007-03-04 Thread Paolo Lucente
Hi Daniel, for brevity, as it has been asked before - archives and Google have the answer. Get a look here: http://www.mail-archive.com/pmacct-discussion@pmacct.net/msg00660.html Cheers, Paolo On Sun, Mar 04, 2007 at 02:59:44AM +0100, Daniel wrote: Hi there, anyone know why i can see

Re: [pmacct-discussion] Question about SUM_HOST or Traffic per IP-Address

2007-02-15 Thread Paolo Lucente
Hi Mirko, On Wed, Feb 14, 2007 at 08:19:52PM +0100, Mirko wrote: If i use sum_host with networks.lst containing 192.168.0.0/16, is it right, that only traffic inside 192.168.x.x will be accounted? For example only inside the local network? 192.168.0.1 -- . --

Re: [pmacct-discussion] Question about SUM_HOST or Traffic per IP-Address

2007-02-08 Thread Paolo Lucente
Hi Mirko, if i got your ideas correctly, you might want to go with a config similar to the following (proposed in the FAQS): ... aggregate[inbound]: dst_host aggregate[outbound]: src_host aggregate_filter[inbound]: dst net 192.168.0.0/16 aggregate_filter[outbound]: src net

Re: [pmacct-discussion] many process?

2007-02-08 Thread Paolo Lucente
Hi Daniel, MySQL plugin is trying to write to your database but it's unable to do so. Those DB Writer processes are locked out and are patiently queuing to get access to the SQL table. It should mean either something external is currently locking the table or that the plugin is unable to write

Re: [pmacct-discussion] pmacct 0.11.2 compile problem

2007-01-15 Thread Paolo Lucente
Hi Inge, thank you for reporting the problem, good spot. The fix (which is basically what you propose) has already been committed to the CVS. Cheers, Paolo On Mon, Jan 08, 2007 at 03:00:33PM +0100, Inge Bj?rnvall Arnesen wrote: /* Need to preprocess data because packet handlers have

Re: [pmacct-discussion] sflow agent address

2007-01-15 Thread Paolo Lucente
Hi Juraj, that's a very good one: thanks for reporting the issue. I've slightly refined your quick and dirty patch (which basically was correctly doing the job but we just need to keep intact the sa structure). The patch has been committed to the CVS - would you check it out and let me know

Re: [pmacct-discussion] networks_file for probe

2007-01-15 Thread Paolo Lucente
Hi Michael, On Tue, Jan 09, 2007 at 01:13:13PM +0100, Muenz, Michael wrote: In networks.server are only ip addresses listed (/32), no networks. My problem is, that my nfacctd writes content from this probe to DB with port information (I aggregate only host on probe) and also, it writes the

Re: [pmacct-discussion] pmacct measures less traffic than ethereal does

2007-01-05 Thread Paolo Lucente
Hi Sebastian, a couple of things come to my mind - let me know if you have any joy with them: - Ethereal could be counting packet sizes differently compared to pmacct. pmacct counts IPv4/6 header's length plus the payload. ie.m L2 and ethernet-related stuff are excluded. If this is the case,

Re: [pmacct-discussion] pmacctd transparent proxy

2006-12-25 Thread Paolo Lucente
Hi Valery, On Mon, Dec 25, 2006 at 12:39:05PM +0200, Valery Kartel wrote: [ ... ] === /etc/pmacct/pretag.map: (1640 lines with all UA-IX networks) ... id=2filter='net 82.144.192.0/19' ... id=2filter='net 195.144.25.0/24' ... [ ... ] So, some hosts are tagged, but not all

Re: [pmacct-discussion] Locking for MySQL

2006-12-19 Thread Paolo Lucente
Hi Chris, On Tue, Dec 19, 2006 at 05:04:07PM +1100, Chris Ricks wrote: As we use InnoDB tables anyway, I'm working on a patch to make locking configurable for MySQL (as it currently is for PostgreSQL). Would anyone else on the list be interested in such a patch? It would be a nice feature to

Re: [pmacct-discussion] pmaact-fe

2006-12-19 Thread Paolo Lucente
Hi Daniel, yes. pmacct-fe supports just PostgreSQL. And, _YES_: support for MySQL there would be greatly appreciated! I received a good while of emails at this propo. Let me know! Cheers, Paolo ___ pmacct-discussion mailing list

Re: [pmacct-discussion] Timestamp

2006-12-18 Thread Paolo Lucente
Hi Daniel, Q9 in FAQS should give useful pointers in regards to your question. It applies to all SQL database backends and all SQL table versions. Cheers, Paolo ___ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists

Re: [pmacct-discussion] what is network 0.0.0.0 ...

2006-12-13 Thread Paolo Lucente
Hi Ian, On Wed, Dec 13, 2006 at 01:43:43PM +1100, IT Officer wrote: Just today I created another .conf file using the src/dst_net aggregate and ran another pmacctd instance. When I display the statistics I get data for 2 networks. One of these is 0.0.0.0. There seems to be a lot of traffic

[pmacct-discussion] pmacct 0.11.2 released !

2006-11-28 Thread Paolo Lucente
VERSION. 0.11.2 DESCRIPTION. pmacct is a small set of passive network monitoring tools to measure, account, classify, aggregate and export IPv4 and IPv6 traffic; a pluggable and flexible architecture allows to store collected network data into memory tables or SQL (MySQL, SQLite, PostgreSQL)

Re: [pmacct-discussion] Classification

2006-11-15 Thread Paolo Lucente
Hi Guys, sorry to join this - interesting, despite Peter's exagerations :-) - thread a bit late, i'm having some terribly busy days. I want just to put a comment to the following lines: On Mon, Nov 13, 2006 at 09:57:09AM +0300, Chris Wilson wrote: I don't think it's as hard as all that. The OS

Re: [pmacct-discussion] nfacctd warnings

2006-11-15 Thread Paolo Lucente
Hi Guys, reviewing quickly the code, seems like there is something not working properly on the nfacctd side - while on the sfacctd everything is reported to be working well. I'll fix that in the next release and actually pleas ignore it. Just to manually double check for any packet loss, when

Re: [pmacct-discussion] Large number of threads

2006-10-18 Thread Paolo Lucente
Hi Chris, On Wed, Oct 18, 2006 at 07:01:07PM +0100, Chris Wilson wrote: of memory (due to Apache I think), pmacctd started spawning more threads to write to the database. I ended up with 73 processes/threads in total, almost all database writers. Is this really a good idea? Wouldn't it

Re: [pmacct-discussion] MySQL ist too slow with Millions of Data-rows

2006-10-12 Thread Paolo Lucente
Hi Simo, if the problem is with an high volume of data, i think just switching to PostgreSQL would not be the ideal solution. At some stage, it will sink as well. The solution should be in handling meaningfully the data: - you can partition data. pmacct allows you to partition data basing over

Re: [pmacct-discussion] IPv6 support broken in pmacct 0.11.0rc2

2006-08-23 Thread Paolo Lucente
Hi Nigel, On Wed, Aug 23, 2006 at 11:55:17AM +1200, Nigel Roberts wrote: I changed line 379 of nfprobe_plugin.c to match the call to the same function in ipv4_to_flowrec and it compiled ok. I'm testing it now. thank you very much for signalling the bug. Indeed, that was the correct solution.

Re: [pmacct-discussion] Problem with sql_history option

2006-08-22 Thread Paolo Lucente
Hi Nicolas, On Tue, Aug 22, 2006 at 10:01:44AM +0700, Nicolas Fournaux wrote: aggregate: src_mac,dst_mac,src_host,dst_host,src_port,dst_port If you use such aggregation, you have to expect many tuples in your database for the same src_host, dst_host. To get started and keep your database

Re: [pmacct-discussion] pmacctd rc3 core dumps

2006-08-22 Thread Paolo Lucente
Hi Gert, take the virgin pmacct package. Configure it with the options you are used to. Don't care of the --enable-debug. Once you get the Makefile files, get through them and replace the following line: CFLAGS = -O2 with CFLAGS = -g Unless you are not using gcc, it will work. Will disable

Re: [pmacct-discussion] mac_src and marc_dst always set to 0:0:0:0:0:0

2006-08-16 Thread Paolo Lucente
Hi Nicolas, how are you actually capturing traffic (ie. libpcap, NetFlow v5, NetFlow v9, sFlow, etc.) ? Posting your configuration might be of help to solve the your issue. BTW, it seems like you have not specified relevant keys in your aggregate configuration directive (ie. aggregate:

Re: [pmacct-discussion] classification with src + dst ip

2006-07-20 Thread Paolo Lucente
Hi Gregory, On Thu, Jul 20, 2006 at 03:16:11PM +0200, Gregory Machin wrote: But now I need to know the source and destination ip that the, of the packets with the applied filters .. How do I do this .. The usual way. If you actually have your 'aggregation' value set to 'class', then switch

[pmacct-discussion] pmacct 0.11.0rc1 released !

2006-07-20 Thread Paolo Lucente
VERSION. 0.11.0rc1 DESCRIPTION. pmacct is a small set of passive network monitoring tools to measure, account, classify, aggregate and export IPv4 and IPv6 traffic; a pluggable and flexible architecture allows to store collected network data into memory tables or SQL (MySQL, SQLite, PostgreSQL)

Re: [pmacct-discussion] pmacct + peer to peer traffic

2006-07-18 Thread Paolo Lucente
Hi Gregory, yes, you can. If using pmacctd you can use classification. Read more about it in the EXAMPLES document; then, check out whether available classifiers at l7-filter homepage fit for you. Cheers, Paolo ___ pmacct-discussion mailing list

<    4   5   6   7   8   9   10   >