[pmacct-discussion] pmacct 0.8.1p1 released !

2005-02-22 Thread Paolo Lucente
VERSION. 0.8.1p1 DESCRIPTION. pmacct is a small set of tools to account and aggregate IPv4 and IPv6 traffic; aggregation revolves around the key concept of primitives (VLAN id, source and destination MAC addresses, hosts, networks, AS numbers, ports, IP protocol and ToS/DSCP field are

Re: [pmacct-discussion] pmacctd only collecting half of the data being passed.

2005-02-24 Thread Paolo Lucente
Hello Dave, can you tell me on which version did you notice this behaviour ? Which OS are you running pmacct ? Did you notice any specific relation between the caught and missed traffic ? For example you are missing just incoming traffic, etc. Said this all, I've never caught in such a behaviour

Re: [pmacct-discussion] pmacctd only collecting half of the databeing passed.

2005-02-24 Thread Paolo Lucente
Hello Dave, with the details in my hands, i've actually no idea. Can you do some simple file transfer of some rather large file and check counters after it has completed ? This should point out what counter increases the correct way. A 40-50Mb transfer should suffice. One note: because the

[pmacct-discussion] First paper about pmacct

2005-03-01 Thread Paolo Lucente
Hello guys, i'm about to release the first paper about pmacct. It will be linked on the pmacct homepage in next couple of days (very final refinements in act), though i'm actually not considering its publication. If anyone wishes to give it a look, any comment, suggestion, idea and correction is

[pmacct-discussion] pmacct 0.8.2 released !

2005-03-08 Thread Paolo Lucente
VERSION. 0.8.2 DESCRIPTION. pmacct is a small set of tools to account and aggregate IPv4 and IPv6 traffic; aggregation revolves around the key concept of primitives (VLAN id, source and destination MAC addresses, hosts, networks, AS numbers, ports, IP protocol and ToS/DSCP field are supported)

Re: SV: [pmacct-discussion] Counter size

2005-03-21 Thread Paolo Lucente
Hello Tobias, On Mon, Mar 21, 2005 at 11:59:28AM +0100, Tobias Bengtsson wrote: I don't use any sql right now, Wops. I was assuming so :) The memory case, yes, some tweakings are needed: 1) imt_plugin.h - struct acc definition - bytes_counter should be modified to 'u_int64_t'. 2)

Re: [pmacct-discussion] ip_proto: ipv6-c

2005-03-29 Thread Paolo Lucente
Hello Steve, On Tue, Mar 29, 2005 at 05:29:03PM +0100, Steve Wright wrote: Running pmacct-0.8.2, nfacctd to MySQL my ip_proto field I see a large GRE (ESP) flow [1] being reported as ipv6-c in the table. [1] sh ip cache flow displays: Gi0/0 xxx.xxx.xxx.209 Gi0/1

Re: [pmacct-discussion] Invalid ports on ports list?

2005-04-07 Thread Paolo Lucente
Hello Bruno, i've been able to verify the issue you've signalled taking the bare 'ports.lst.example'. It's an ugly bug that makes pmacct not deal very much with comments (the one on the second line, for example) only in the 'ports' file. Just deleting the second line makes it work without

[pmacct-discussion] pmacct 0.8.6 released !

2005-05-23 Thread Paolo Lucente
VERSION. 0.8.6 DESCRIPTION. pmacct is a small set of passive network monitoring tools to measure, account and aggregate IPv4 and IPv6 traffic; aggregation revolves around the key concept of primitives (VLAN id, source and destination MAC addresses, hosts, networks, ports, AS numbers, IP

Re: [pmacct-discussion] pmacct 0.8.6 released !

2005-05-23 Thread Paolo Lucente
Hey Andre, On Mon, May 23, 2005 at 02:53:45PM +0200, Andre Berger wrote: I also added -I/usr/local/include/openssl to CFLAGS manually, with no better result. To my eyes it seems a linking problem not an header inclusion one. Try adding to the CFLAGS a '-lssl -L/usr/local/lib' (here i'm just

[pmacct-discussion] pmacct-contribs 20050531 (second release)

2005-05-31 Thread Paolo Lucente
VERSION. 20050531 DESCRIPTION. pmacct is a set of network tools to gather, filter and tag IP traffic; it is able to store collected data either into a DB or a memory table. We see any monitoring, billing or accounting environment as a stack where data are picked from the network, get processed

Re: [pmacct-discussion] ERROR on AMD64 - x86_64 [Resend]

2005-06-13 Thread Paolo Lucente
Hey Daniel, On Mon, Jun 13, 2005 at 12:25:43AM +0200, Daniel Streicher wrote: I am using source tarball, should I try to use the .deb package? I don't think things could change. However, about the access to the box. Let me know the answer. I should be able to have access to a Fedora 3 Opteron

Re: [pmacct-discussion] ERROR on AMD64 - x86_64 [Resend]

2005-06-13 Thread Paolo Lucente
Hello Daniel, On Mon, Jun 13, 2005 at 05:38:50PM +0200, Daniel Streicher wrote: Ok there seems to be one small error in nfacctd: nfacctd logging MAC Addresses always as SRC MACDST MAC 00:00:00:00:00:00 00:00:00:00:00:00. pmacctd logging MAC Addresses always as SRC MAC

[pmacct-discussion] pmacct 0.8.7 released !

2005-06-14 Thread Paolo Lucente
VERSION. 0.8.7 DESCRIPTION. pmacct is a small set of passive network monitoring tools to measure, account and aggregate IPv4 and IPv6 traffic; aggregation revolves around the key concept of primitives (VLAN id, source and destination MAC addresses, hosts, networks, ports, AS numbers, IP

Re: [pmacct-discussion] problem with accounting

2005-06-15 Thread Paolo Lucente
Hello Piotr, On Wed, Jun 15, 2005 at 07:11:43PM +0200, Piotr Szlenk wrote: I did port mirroring on our core switch with tagged vlans. The whole traffic is mirrored on one port and it is analysed by the snort ids. However I would like to have traffic accounting based on src_host and dst_host.

Re: [pmacct-discussion] src_mac aggregation in 0.9.0

2005-08-09 Thread Paolo Lucente
Hello Chris, On Mon, Aug 08, 2005 at 05:27:10PM -0700, Chris Timmons wrote: # pmacctd -f pmacct.conf WARN ( pmacct.conf ): ignoring unknown aggregation method: src_mac. I'm unable to verify this behaviour over a freshly compiled 0.9.0; do you think there is any chance that you have configured

Re: [pmacct-discussion] stamp_inserted and sql_history

2005-08-12 Thread Paolo Lucente
On Sat, Aug 13, 2005 at 02:14:00AM +1000, Jamie Wilkinson wrote: Ok. Does this mean that unless the config options 'sql_history' and 'sql_history_roundoff' exist, then pmacctd will not write time stamps to the database? yes. I've done so, but I've also added these two config options back

Re: [pmacct-discussion] stamp_inserted and sql_history

2005-08-16 Thread Paolo Lucente
Hello Jamie, On Tue, Aug 16, 2005 at 05:07:33PM +1000, Jamie Wilkinson wrote: Actually, I'm still not clear: the FAQ suggests that only sql_history is required, though sql_history_roundoff is advised. Is that true? True. sql_history writes the stamp fields; sql_history_roundoff enables the

[pmacct-discussion] pmacct 0.9.1 released !

2005-08-16 Thread Paolo Lucente
VERSION. 0.9.1 DESCRIPTION. pmacct is a small set of passive network monitoring tools to measure, account and aggregate IPv4 and IPv6 traffic; aggregation revolves around the key concept of primitives (VLAN id, source and destination MAC addresses, hosts, networks, ports, AS numbers, IP protocol

Re: [pmacct-discussion] postgresql connection errors in 0.9.1

2005-08-18 Thread Paolo Lucente
Hey Jamie, On Thu, Aug 18, 2005 at 01:03:09PM +1000, Jamie Wilkinson wrote: pidfile: /var/run/pmacctd.test.pid debug: true aggregate: src_host,dst_host networks_file: /etc/pmacct/networks pcap_filter: vlan and ( net 202.4.224.0/20 or net 203.98.86/24 ) and not ((src net 202.4.224.0/20 or

Re: [pmacct-discussion] stats in 0.9.1

2005-08-22 Thread Paolo Lucente
Hey Jamie, On Mon, Aug 22, 2005 at 11:37:08AM +1000, Jamie Wilkinson wrote: I've got a patch that also puts the interface name in the log, because I run a couple of pmacctds on different interfaces. Patch accepted: it's a very good idea to add interface infos. It will appear next official

Re: [pmacct-discussion] 95 percentile traffic, accounting, billing

2005-11-21 Thread Paolo Lucente
Hello Christian, first of all, thank you for giving a try to pmacct. Secondly, pmacct does not do any 95th percentile itself but, as the ChangeLog mentions, it eases its computation. The ChangeLog speaks about the 'sql_history' directive and applies to the SQL plugins. So, let's start with this

Re: [pmacct-discussion] mysql error

2005-12-04 Thread Paolo Lucente
Hello Ilya, my first hint is to remove the 'sql_host: localhost' line; it avoids the connection through the usual '/tmp/mysql.sock' pipe file and makes pmacct to establish the TCP connection to 'localhost:3306'. If you really need this, try rewriting it as 'sql_host: 127.0.0.1'. Next, check that

Re: [pmacct-discussion] more filter fields

2005-12-05 Thread Paolo Lucente
Hello Nitzan, On Mon, Dec 05, 2005 at 07:51:46PM +0200, Nitzan Tzelniker wrote: 1. flow-nfilter has configuration file where you define filter primitive and filter definition and then in flow-report you only call the filter definition and don't need to specify the all filter over and over.

[pmacct-discussion] pmacct 0.9.5 released !

2005-12-07 Thread Paolo Lucente
VERSION. 0.9.5 DESCRIPTION. pmacct is a small set of passive network monitoring tools to measure, account and aggregate IPv4 and IPv6 traffic; aggregation revolves around the key concept of primitives (VLAN id, source and destination MAC addresses, hosts, networks, ports, AS numbers, IP

Re: [pmacct-discussion] MySQL update performance and possible bug in 0.9.4

2005-12-08 Thread Paolo Lucente
Hey Inge, let me thank you for your very detailed email about the AS numbers MySQL issue which has allowed me to quickly work on the code. There isn't any specific reason behind the choice of unquoted AS numbers; it is definitely the right behaviour for PostgreSQL in typed data mode but wrong

Re: [pmacct-discussion] mysql error

2005-12-12 Thread Paolo Lucente
Hey Ilya, On Mon, Dec 12, 2005 at 07:55:42PM +0300, Molokanov Ilya wrote: Database changed mysql show tables; +---+ | Tables_in_nfacctd | +---+ | nfacctd | +---+ 1 row in set (0.00 sec) [ ... ] sql_table: acct What i did

Re: [pmacct-discussion] mysql error

2005-12-12 Thread Paolo Lucente
Hey Ilya, On Mon, Dec 12, 2005 at 09:18:10PM +0300, Molokanov Ilya wrote: FAILED query follows: INSERT INTO acct (stamp_updated, stamp_inserted, ip_src, ip_dst, src_port, dst_port, ip_proto, mac_src, mac_dst, packets, bytes) VALUES (FROM_UNIXTIME(1134411301), FROM_UNIXTIME(4291806600),

Re: [pmacct-discussion] mysql problems

2006-01-02 Thread Paolo Lucente
Hi Simone, check that everything works correctly by trying to connect to the remove MySQL server via the mysql client tool: shell mysql -u pmacct -h 192.168.11.14 -p Are you able to connect to the server ? This should clear whether there is any connection filtering or permission-related

Re: [pmacct-discussion] mysql backend failing

2006-01-03 Thread Paolo Lucente
Hello David, i've tested your configuration against a MySQL 4.1 server and it seems to work just fine to my eyes. The password you specified is 'pmacct' which is not the default one - 'arealsmartpwd'. The SQL table creation script uses such password by default. Did you change it in the script

Re: [pmacct-discussion] mysql backend failing

2006-01-04 Thread Paolo Lucente
Hello David, On Wed, Jan 04, 2006 at 10:01:31AM -0500, David Filion wrote: The problem I have now is the mac_src, mac_dst, src_port and dst_port fields are always 0 (protocol is ip). I've added a ports file that currently Here the solution is quite simple: your configuration has an

Re: [pmacct-discussion] mysql backend failing

2006-01-04 Thread Paolo Lucente
Hey Inge, On Wed, Jan 04, 2006 at 04:19:56PM +0100, Inge Bjørnvall Arnesen wrote: I don't mind to much about the MAC addresses (also all zero for me), but I've also noted that with Netflow (Cisco Netflow and DD-WRT rFlow) I never get TCP/UDP/ICMP entries in the IP PROTO field - all entries

Re: [pmacct-discussion] Using tag ID's in aggregate_filter

2006-01-20 Thread Paolo Lucente
Hello Inge, aggregate_filter can match any packet/flow primitives with the exception of IDs. This task is accomplished by pre_tag_filter . I'm just figuring out that there aren't examples for such directive around. However, it's use is pretty simple: pre_tag_filter: IDx [, IDy] [, IDz] Cheers,

[pmacct-discussion] pmacct 0.10.0rc1 released !

2006-01-24 Thread Paolo Lucente
VERSION. 0.10.0rc1 DESCRIPTION. pmacct is a small set of passive network monitoring tools to measure, account, classify and aggregate IPv4 and IPv6 traffic; a pluggable and flexible architecture allows to store the collected traffic data into memory tables or SQL (MySQL, SQLite, PostgreSQL)

Re: [pmacct-discussion] ERROR on AMD64 - x86_64 [Resend] - again?

2006-02-02 Thread Paolo Lucente
Hello Daniel, as usual, thank you very much for the bug report ! A ritual question for you: can you tell me whether this problem has appeared just running the 0.10.0rc1 version ? That is, can you confirm me that the previous version (which version ?) you had in use was running fine ? Cheers,

Re: [pmacct-discussion] sfacctd issues (crash on SIGUSR1, no syslogging)

2006-02-07 Thread Paolo Lucente
Hello Ivan, On Mon, Feb 06, 2006 at 05:01:44PM +, Ivan A. Beveridge wrote: This is my current logfile: = daemonize: true pidfile: /var/run/sfacctd.pid sfacctd_port: 6500 sfacctd_ip: 127.0.0.1 plugins: memory[fdrypeer] aggregate[fdrypeer]: src_mac,

Re: [pmacct-discussion] sfacctd issues (crash on SIGUSR1, no syslogging)

2006-02-07 Thread Paolo Lucente
Hello Ivan (and all), i've just made available for download a fresh tarball (as usual, it is a pre-release so pretty no updates to the documentation and, if you decide to give it a try, upgrade as soon as the final release is out):

Re: [pmacct-discussion] sfacctd 0.10.0rc1 crash (memory plugin)

2006-02-11 Thread Paolo Lucente
Hey Ivan, On Sat, Feb 11, 2006 at 11:19:39AM +, Ivan A. Beveridge wrote: Can I do this by creating a corefile aswell (and then do a backtrace on the corefile)? This is what I've done before, but those programs/daemons didn't have separately-running 'modules' (like the memory module).

Re: [pmacct-discussion] sfacctd 0.10.0rc1 crash (memory plugin)

2006-02-13 Thread Paolo Lucente
Hello Ivan, On Mon, Feb 13, 2006 at 02:01:40PM +, Ivan A. Beveridge wrote: Hrmph - that was useless. Any reason why I'd not be seeing anything useful in the trace? I thought it only looked like that if the binary was stripped :( I compiled with '-g': Fully agree with you. I've just got a

[pmacct-discussion] pmacct 0.10.0rc2 released !

2006-02-14 Thread Paolo Lucente
VERSION. 0.10.0rc2 DESCRIPTION. pmacct is a small set of passive network monitoring tools to measure, account, classify and aggregate IPv4 and IPv6 traffic; a pluggable and flexible architecture allows to store the collected traffic data into memory tables or SQL (MySQL, SQLite, PostgreSQL)

Re: [pmacct-discussion] 0.0.0.0 ip_src ip_dst

2006-02-23 Thread Paolo Lucente
Hello Prakash, can i ask you to verify that flow-send is actually filling the src_as/dst_as fields of the NetFlow packets ? If this is the case, can you please send me privately a chunk of your savefile as i can replay it and investigate the trouble ? Howvever, I see in your configuration you use

Re: [pmacct-discussion] 0.0.0.0 ip_src ip_dst

2006-03-01 Thread Paolo Lucente
Hey Ivan and Prakash, On Wed, Mar 01, 2006 at 11:02:34AM +, Ivan A. Beveridge wrote: To the best of my knowledge (I'm pretty certain) netflow is layer3+ so will never show layer2 (eg MAC addresses). MAC addresses are shown in sflow (picked up by sfacctd), and probably also picked up by

[pmacct-discussion] pmacct 0.10.0rc3 released !

2006-03-01 Thread Paolo Lucente
VERSION. 0.10.0rc3 DESCRIPTION. pmacct is a small set of passive network monitoring tools to measure, account, classify and aggregate IPv4 and IPv6 traffic; a pluggable and flexible architecture allows to store the collected traffic data into memory tables or SQL (MySQL, SQLite, PostgreSQL)

Re: [pmacct-discussion] [patch] don't lock the table when we're only using insert

2006-03-07 Thread Paolo Lucente
Hey Jamie, thank you very much for the patch, it makes sense to me. And i've just committed it in mainstream code. However, taking apart the concurrent access, do we have a way to measure the speed of things ? I suspect EXPLAIN is not our friend here (ie. to measure speed of transactions rather

Re: [pmacct-discussion] sfacctd and -D not daemonizing

2006-03-10 Thread Paolo Lucente
Hey Ivan, On Fri, Mar 10, 2006 at 02:23:42AM +, Ivan A. Beveridge wrote: I try the -D in either place and it makes no difference (as expected): /usr/local/sbin/sfacctd -f /path/to/config/file -D /usr/local/sbin/sfacctd -D -f /path/to/config/file This is bizarre ... I could have sworn

Re: [pmacct-discussion] sfacctd and -D not daemonizing

2006-03-11 Thread Paolo Lucente
Hey Ivan and all, i've made available for download an updated pmacct snapshot at the following URL: http://www.ba.cnr.it/~paolo/pmacct/pmacct-0.10.0-20060311.tar.gz As any intermidiate tarball, documentation has not already been updated. Among the other things, two (included) changes to notice

Re: [pmacct-discussion] flow and packet agregation

2006-03-13 Thread Paolo Lucente
Hello Peter, it shows the total number of bytes, ie. it's not an average and you don't have to multiply by the number of packets and/or flows. Cheers, Paolo On Mon, Mar 13, 2006 at 12:26:25PM +0200, Peter Nixon wrote: # pmacct -s SRC_IP DST_IP SRC_PORT DST_PORT

Re: [pmacct-discussion] pmacct 0.10.0 uploaded to Debian

2006-04-05 Thread Paolo Lucente
Hey Jamie, that's a good news ! Thank you for the great work. Cheers, Paolo

Re: [pmacct-discussion] PostgreSQL performance

2006-04-10 Thread Paolo Lucente
Hey Sven, digging through the SQL scripts i've just noticed something bad: primary keys of default PostgreSQL tables v2-v5 are just missing the 'vlan' field (while it correctly appears in equivalent MySQL/SQLite 3.x schemas). This may explain the slowness. To verify this, can you please modify the

Re: [pmacct-discussion] network port monitoring

2006-04-18 Thread Paolo Lucente
Hi Ben, so, port refers to TCP/UDP port. Ok. About the tutorial per-se, graphing port data rather than network data requires a) intercepting occurrences of src|dst_net in the document and b) replacing them with src|dst_port. Now, if you need just a per-port breakdown it's feaasible: generating

Re: [pmacct-discussion] pmacct problems on x86_64?

2006-04-19 Thread Paolo Lucente
Hi Peter, i was wondering whether it's something related with VLANs and the aggregate_filter directives: traffic seen on tunnel0 is tagged (and doesn't match the filter -- pcap filter need to match the vlan layer, ie. vlan and src net ...) while the one on the eth0 isn't (thus, matching it). Do

Re: [pmacct-discussion] PostgreSQL performance

2006-05-01 Thread Paolo Lucente
Hey Sven, On Fri, Apr 28, 2006 at 02:31:43PM +0200, Sven Anderson wrote: it seems to be known, that a default MySQL performs a lot better than a default PostgreSQL. One difference is, that PostgreSQL is doing an COMMIT Not to open a possible flame, but i fully agree with this view. However,

Re: [pmacct-discussion] pmacct's place in the network... passive host?

2006-05-01 Thread Paolo Lucente
Hi Duncan, On Fri, Apr 28, 2006 at 10:45:24AM -0500, Duncan Shannon wrote: Can I run pmacct on my (Fedora) system that is seeing all in/out traffic (for snort) that is on a passive Ethernet tap? Just to fully agree with Sven's reply and to add that both Snort and pmacctd are libpcap-based:

Re: [pmacct-discussion] memory table extraction lossless?

2006-05-03 Thread Paolo Lucente
Hi Sven, On Wed, May 03, 2006 at 10:51:08AM +0200, Sven Anderson wrote: But to make sure, that no packets are counted in the wrong bin, the memory table has to be locked as long as the port list is processed. Otherwise packets that arrive during processing with, for example, port 80, would

Re: [pmacct-discussion] memory table extraction lossless?

2006-05-03 Thread Paolo Lucente
Hi Sven, On Wed, May 03, 2006 at 03:28:22PM +0200, Sven Anderson wrote: Is there a case, when using the -r flag, where you don't want locking? Or do you mean, that -l could be useful in case of not using -r? The latter. You can think at it like a macro-transaction. Now, multiple queries (ie.

Re: [pmacct-discussion] memory table extraction lossless?

2006-05-03 Thread Paolo Lucente
Ciao Sven, On Wed, May 03, 2006 at 06:55:16PM +0200, Sven Anderson wrote: Ok, I'm a little bit confused now, so let me resume: There IS a locking, but only if the -r flag is used. So this race condition cannot appear in that case. When I'm using -r I don't have to worry, that port 80 packets

Re: [pmacct-discussion] beginning

2006-05-10 Thread Paolo Lucente
Hi Cedric, On Wed, May 10, 2006 at 10:26:36AM +0200, Cédric Delaunay wrote: First : which machine should I use ? I want collect about 9000 flows per minute. (this is a campus network). It should be nice if I could collect, store data in a sql database and create graphs on the same computer.

Re: [pmacct-discussion] beginning

2006-05-12 Thread Paolo Lucente
Hey Cedric, On Fri, May 12, 2006 at 12:19:44PM +0200, Cédric Delaunay wrote: aggregate[in]: src_host,dst_host aggregate[out]: src_host,dst_host I would correct the above few lines in: aggregate[in]: dst_host aggregate[out]: src_host This is very likely to stop the error message you are

Re: [pmacct-discussion] beginning

2006-05-12 Thread Paolo Lucente
Hey Peter, On Thu, May 11, 2006 at 09:13:09AM +0300, Peter Nixon wrote: I would love to see an SNMP agent however so that the in memory tables could be queried remotely via SNMP. This would allow trivial integration with any number of SNMP graphing tools :-) That's a good point. Seems like

Re: [pmacct-discussion] nfacctd and mysql

2006-05-16 Thread Paolo Lucente
Hi Cedric, On Tue, May 16, 2006 at 02:37:46PM +0200, Cédric Delaunay wrote: I tried a few solutions and understood that I have to use mysql plugin. I would generalize and say SQL plugin. The winning solution should be better evaluated after getting preliminar results (ie. on the field).

[pmacct-discussion] pmacct 0.10.2 released !

2006-05-17 Thread Paolo Lucente
VERSION. 0.10.2 DESCRIPTION. pmacct is a small set of passive network monitoring tools to measure, account, classify and aggregate IPv4 and IPv6 traffic; a pluggable and flexible architecture allows to store the collected traffic data into memory tables or SQL (MySQL, SQLite, PostgreSQL)

Re: [pmacct-discussion] Signalling dropped packets

2006-05-17 Thread Paolo Lucente
Hi Antonin, On Wed, May 17, 2006 at 05:29:07PM +0200, Antonin Kral wrote: I want to use ipacct for part of our university network monitoring. Is there any option / way how to get clue about state, when pmacct drops any packets? Basicaly I need some way to get information, that system is

Re: [pmacct-discussion] Putting inbound and outbound packets into same table

2006-05-19 Thread Paolo Lucente
Hi Peter, good point (again). pmacct actually lacks of such thing. But it could be good idea to implement it. The method will need some extra cares in order to make things work smoothly: a) writes need to be interleaved in order to avoid one plugin to lock out the other while racing for the same

Re: [pmacct-discussion] Putting inbound and outbound packets into same table

2006-05-19 Thread Paolo Lucente
Hi Peter, On Fri, May 19, 2006 at 02:38:37PM +0300, Peter Nixon wrote: The only question that remains, is how to handle bytes? Ideally I think the schema should be changed/extended to have an bytesin and bytesout column.. What do you think? I think that while it would be the best ever

Re: [pmacct-discussion] Multiple sflow exporters sending data to sfacctd

2006-06-02 Thread Paolo Lucente
Hi Gregor, let me briefly summarize your (good!) point: dealing with large networks - where multiple sensors, dynamic routing, etc. come into play - requires a kind of spanning-tree method to consolidate collected data. This is a point on which i'm spending more and more thoughts recently. While

Re: [pmacct-discussion] inbound data collection not working, outbound OK

2006-06-13 Thread Paolo Lucente
Hi Karl, i've never been involved with a bridged interface on my own, so some basic hints (configuration looks ok to my eyes): - Having an idea of what pmacct gets and what doesn't, do you see any relevant compatible/different behaviour spawning on the same interface, say, tcpdump ? This can

[pmacct-discussion] pmacct 0.10.3 released !

2006-06-21 Thread Paolo Lucente
VERSION. 0.10.3 DESCRIPTION. pmacct is a small set of passive network monitoring tools to measure, account, classify and aggregate IPv4 and IPv6 traffic; a pluggable and flexible architecture allows to store the collected traffic data into memory tables or SQL (MySQL, SQLite, PostgreSQL)

Re: [pmacct-discussion] [BUG] aggregate filter filters out everything if interface is set to any

2006-06-23 Thread Paolo Lucente
Hi Zhuang, good catch! I've verified the trouble and it has never been noticed before. I've also managed to fix it. Below you will find a link to a fresh snapshot containing the fix (it's basically 0.10.3 + this fix): http://www.pmacct.net/pmacct-0.10.4-20060623.tar.gz Let me know whether it

Re: [pmacct-discussion] Differentiating multiple netflow exporters

2006-06-26 Thread Paolo Lucente
Hi Jon, the solution lies in Pre-Tagging; then making the plugin to accept the tags; tags will discriminate data per router; see the fragments below: nfacctd.conf: === ... pre_tag_map: /path/to/pretag.map aggregate: tag,whatever ... === /path/to/pretag.map: === id=1000 ip=Router 1 id=1001

Re: [pmacct-discussion] pmacct version update plans?

2006-07-04 Thread Paolo Lucente
Hi Ivan, there isn't a new version on the way. There should be one by the end of July. So, would say ... go ahead. I'm not Gentoo fellow, but from their http://www.gentoo-portage.com/net-analyzer/pmacct/ page i can see there should be 0.10.2 and 0.10.3 versions available aswell. Marcelo Goes (his

Re: [pmacct-discussion] sfacctd/nfacctd security

2006-07-04 Thread Paolo Lucente
Hi Peter, nfacctd_allow_file and sfacctd_allow_file config directives are aimed precisely to this. The idea behind them is the same as for hosts.allow. They are both listed in CONFIG-KEYS. Cheers, Paolo ___ pmacct-discussion mailing list

Re: [pmacct-discussion] pmacct does not show much information/Help with configuration needed

2006-07-04 Thread Paolo Lucente
Hi Melitta, it may depend on the content of the networks.def file. Did you forget to insert any network in your networks_file ? When either src|sum|dst_net primitives, if pmacct is unable to determine to which defined network the actual src|dst host belongs to, then zeroes it. Zeroed addresses are

Re: [pmacct-discussion] pmacctd - netflow/sflow export

2006-07-04 Thread Paolo Lucente
Hi Ivan and Peter, while the idea of integrating a kind of sFlow/NetFlow probe has been already considered (i remember some thoughts recently exchanged with Sven Anderson about this), i'm somewhat not fully convinced. In a first instance it will take time as it's absolutely not trivial; this has

Re: [pmacct-discussion] raw data - question

2006-07-10 Thread Paolo Lucente
Hello Dian, thank you very much for the WC greetings, hehe! On Mon, Jul 10, 2006 at 11:28:57AM +0200, Dian Baltadzhiev wrote: now, my question: is there an option, so that i can configure nfacctd to send src and dst addresses in raw format, i mean as a decimal number and not in dotted

Re: [pmacct-discussion] nfacctd stamp_inserted

2006-07-14 Thread Paolo Lucente
Hello Dan, Look at Q8 in FAQS. You are looking for historical accouting and 'sql_history' configuration directive fits for you. Furthermore, you can get more infos about its syntax by browsing CONFIG-KEYS doc. To get started, try inserting the following lines in your configuration: sql_history:

Re: [pmacct-discussion] Help, nfacctd cannot read Netflow v9 data from Cisco router

2006-07-17 Thread Paolo Lucente
Hi Jeremy, giving a look to the ChangeLog, NetFlow v9 support has been introduced in pmacct 0.8.0. So, you will not be able to collect such flows in earlier versions, as 0.7.4. Try upgrading to some more fresh version, ie. 0.10.3. It will work. Cheers, Paolo

Re: [pmacct-discussion] pmacct + peer to peer traffic

2006-07-18 Thread Paolo Lucente
Hi Gregory, yes, you can. If using pmacctd you can use classification. Read more about it in the EXAMPLES document; then, check out whether available classifiers at l7-filter homepage fit for you. Cheers, Paolo ___ pmacct-discussion mailing list

Re: [pmacct-discussion] Templates (was: Help, nfacctd cannot read Netflow v9 data from Cisco router)

2006-07-18 Thread Paolo Lucente
Hi Sven, On Tue, Jul 18, 2006 at 04:19:45PM +0200, Sven Anderson wrote: Paolo Lucente, 17.07.2006 22:57: BTW.: AFAIK Netflow v9 also uses templates to define flows. What happens so far with a template and the according data, which contain flow keys that don't exists in the pmacct flow

Re: [pmacct-discussion] classification with src + dst ip

2006-07-20 Thread Paolo Lucente
Hi Gregory, On Thu, Jul 20, 2006 at 03:16:11PM +0200, Gregory Machin wrote: But now I need to know the source and destination ip that the, of the packets with the applied filters .. How do I do this .. The usual way. If you actually have your 'aggregation' value set to 'class', then switch

[pmacct-discussion] pmacct 0.11.0rc1 released !

2006-07-20 Thread Paolo Lucente
VERSION. 0.11.0rc1 DESCRIPTION. pmacct is a small set of passive network monitoring tools to measure, account, classify, aggregate and export IPv4 and IPv6 traffic; a pluggable and flexible architecture allows to store collected network data into memory tables or SQL (MySQL, SQLite, PostgreSQL)

Re: [pmacct-discussion] HowTo: Increase performence and decrease a number of rows inside the database.

2006-07-25 Thread Paolo Lucente
Hi Marek, combining inbound+outbound traffic in one row is not possible; but you can use two different tables for inbound and outbound traffic; you will be able to achieve precisely your goal by looking at the following configuration fragment (which should be quite intuitive): === ... daemonize:

Re: [pmacct-discussion] reliability of dst_host aggregate?

2006-08-08 Thread Paolo Lucente
Hi Aaron, can you please post your configuration ? I was wondering whether you are actually filtering things out somewhere (ie. sfacctd_allow_file, *_filter, networks_file). Cheers, Paolo ___ pmacct-discussion mailing list

[pmacct-discussion] pmacct 0.11.0rc2 released !

2006-08-08 Thread Paolo Lucente
VERSION. 0.11.0rc2 DESCRIPTION. pmacct is a small set of passive network monitoring tools to measure, account, classify, aggregate and export IPv4 and IPv6 traffic; a pluggable and flexible architecture allows to store collected network data into memory tables or SQL (MySQL, SQLite, PostgreSQL)

Re: [pmacct-discussion] data link not supported

2006-08-13 Thread Paolo Lucente
Hi Gert, did you give a try binding pmacct straight to the underlying ethernet interface ? In case this fails, can you please send me (privately) a brief capture from the tun interface so that i can replay it on my side - libpcap/tcpdump format would be fine ? This would greatly speed up

Re: [pmacct-discussion] mac_src and marc_dst always set to 0:0:0:0:0:0

2006-08-16 Thread Paolo Lucente
Hi Nicolas, how are you actually capturing traffic (ie. libpcap, NetFlow v5, NetFlow v9, sFlow, etc.) ? Posting your configuration might be of help to solve the your issue. BTW, it seems like you have not specified relevant keys in your aggregate configuration directive (ie. aggregate:

Re: [pmacct-discussion] Problem with sql_history option

2006-08-22 Thread Paolo Lucente
Hi Nicolas, On Tue, Aug 22, 2006 at 10:01:44AM +0700, Nicolas Fournaux wrote: aggregate: src_mac,dst_mac,src_host,dst_host,src_port,dst_port If you use such aggregation, you have to expect many tuples in your database for the same src_host, dst_host. To get started and keep your database

Re: [pmacct-discussion] pmacctd rc3 core dumps

2006-08-22 Thread Paolo Lucente
Hi Gert, take the virgin pmacct package. Configure it with the options you are used to. Don't care of the --enable-debug. Once you get the Makefile files, get through them and replace the following line: CFLAGS = -O2 with CFLAGS = -g Unless you are not using gcc, it will work. Will disable

Re: [pmacct-discussion] IPv6 support broken in pmacct 0.11.0rc2

2006-08-23 Thread Paolo Lucente
Hi Nigel, On Wed, Aug 23, 2006 at 11:55:17AM +1200, Nigel Roberts wrote: I changed line 379 of nfprobe_plugin.c to match the call to the same function in ipv4_to_flowrec and it compiled ok. I'm testing it now. thank you very much for signalling the bug. Indeed, that was the correct solution.

Re: [pmacct-discussion] pm-acct on debian sarge

2006-08-28 Thread Paolo Lucente
Hi Thomas, On Mon, Aug 28, 2006 at 06:43:26PM +0200, Thomas Stegbauer wrote: so first i tried to run the nfacctd from sarge (version 0.7.4) but starting the process with; nfacctd -D -c src_host,dst_host -l 3 -P pgsql -f /etc/pmacct/nfacctd.conf In old releases, the use of commandline

[pmacct-discussion] pmacct 0.11.0rc3 released !

2006-08-30 Thread Paolo Lucente
VERSION. 0.11.0rc3 DESCRIPTION. pmacct is a small set of passive network monitoring tools to measure, account, classify, aggregate and export IPv4 and IPv6 traffic; a pluggable and flexible architecture allows to store collected network data into memory tables or SQL (MySQL, SQLite, PostgreSQL)

Re: [pmacct-discussion] pmacct_v6 doesn't get all values

2006-08-30 Thread Paolo Lucente
Hi Thomas, On Wed, Aug 30, 2006 at 04:44:58PM +0200, Thomas Stegbauer wrote: ./nfacctd -c src_host,dst_host -f /etc/pmacct/nfacctd.conf (i give -c on commandline, cause i dont know how to write it correctly to the logfile.) Well, '-c' corresponds to 'aggregate' configuration directive. You

Re: [pmacct-discussion] Capure from multiple devices

2006-09-05 Thread Paolo Lucente
Hi Gert, On Mon, Sep 04, 2006 at 07:45:43PM +0200, Gert Burger wrote: This is where my problem lies, I can run pmacctd on tun0 because the traffic is NATted, so I cannot see where it comes from. And on dc0 I cannot see the bandwidth the server is using. Can I combine the data from two

Re: [pmacct-discussion] nfacctd and flow filters?

2006-09-28 Thread Paolo Lucente
Hi Matt, happy to hear it works fine! All SQL options can be both global and local. This also applies to all other directives not including either NO_GLOBAL or GLOBAL flags - which are the vast majority. GLOBAL means the directive can't be local (this is most likely because it is of no interest

Re: [pmacct-discussion] Problems with shared mem on mac os x

2006-09-28 Thread Paolo Lucente
Hi Stoffer, On Thu, Sep 28, 2006 at 06:49:09PM +0200, stoffer wrote: That is a possibility. /etc/rc contains the following: sysctl -w kern.sysv.shmmax=4194304 kern.sysv.shmmin=1 kern.sysv.shmmni=32 kern.sysv.shmseg=8 kern.sysv.shmall=1024 Is it setting a limit of 4MB, right? Can you

Re: [pmacct-discussion] MySQL ist too slow with Millions of Data-rows

2006-10-12 Thread Paolo Lucente
Hi Simo, if the problem is with an high volume of data, i think just switching to PostgreSQL would not be the ideal solution. At some stage, it will sink as well. The solution should be in handling meaningfully the data: - you can partition data. pmacct allows you to partition data basing over

Re: [pmacct-discussion] MySQL ist too slow with Millions of Data-rows

2006-10-12 Thread Paolo Lucente
Hi Jaime, just a brief follow-up as you brought valid and very interesting points to the discussion: On Thu, Oct 12, 2006 at 04:18:16PM +0200, Jaime Nebrera wrote: 2) Reduce the volume of data without affecting precision too much. You can apply sampling or even better intelligent sampling

Re: [pmacct-discussion] Classification

2006-10-18 Thread Paolo Lucente
Hi Chris, On Wed, Oct 18, 2006 at 02:31:48PM +0100, Chris Wilson wrote: I'd be interested to know if anyone has combined layer 7 classification with pmacct's traffic aggregation. For example, I would like to combine all Kazaa traffic (per minute) into a single counter. It's already there,

Re: [pmacct-discussion] Large number of threads

2006-10-18 Thread Paolo Lucente
Hi Chris, On Wed, Oct 18, 2006 at 07:01:07PM +0100, Chris Wilson wrote: of memory (due to Apache I think), pmacctd started spawning more threads to write to the database. I ended up with 73 processes/threads in total, almost all database writers. Is this really a good idea? Wouldn't it

Re: [pmacct-discussion] Classification

2006-11-15 Thread Paolo Lucente
Hi Guys, sorry to join this - interesting, despite Peter's exagerations :-) - thread a bit late, i'm having some terribly busy days. I want just to put a comment to the following lines: On Mon, Nov 13, 2006 at 09:57:09AM +0300, Chris Wilson wrote: I don't think it's as hard as all that. The OS

Re: [pmacct-discussion] nfacctd warnings

2006-11-15 Thread Paolo Lucente
Hi Guys, reviewing quickly the code, seems like there is something not working properly on the nfacctd side - while on the sfacctd everything is reported to be working well. I'll fix that in the next release and actually pleas ignore it. Just to manually double check for any packet loss, when

  1   2   3   4   5   6   7   8   9   10   >