Re: [pmacct-discussion] Pmacct data inconsistencies between tables.

2010-02-19 Thread Paolo Lucente
Hi Daniel, I see the 1 minute table contains duplicates - it would be better to say that _everything_ extracted is repeated twice. It could be handy to add tags to the aggregation method and assign a different 'post_tag' to each plugin so to identify who is generating them. I also wonder: how

Re: [pmacct-discussion] Pmacct data inconsistencies between tables.

2010-02-19 Thread Chris Wilson
Hi Paolo and Daniel, (please allow me to jump in as I may be able to help here, despite currently being in country working on a project.) On Fri, 19 Feb 2010, Paolo Lucente wrote: I also wonder: how does the primary key of the 1 min table look like? Is it any different from the 1 hour

Re: [pmacct-discussion] Pmacct data inconsistencies between tables.

2010-02-19 Thread Karl O. Pinc
On 02/19/2010 07:42:08 AM, Chris Wilson wrote: Hi Paolo and Daniel, I deleted the primary key from that table because it should not be necessary (there should not be any duplicates if everything is configured correctly) and it makes inserts extremely slow (by a factor of 10-100) when

Re: [pmacct-discussion] Pmacct data inconsistencies between tables.

2010-02-19 Thread Chris Wilson
Hi Karl, On Fri, 19 Feb 2010, Karl O. Pinc wrote: On 02/19/2010 07:42:08 AM, Chris Wilson wrote: I deleted the primary key from that table because it should not be necessary (there should not be any duplicates if everything is configured correctly) and it makes inserts extremely slow

Re: [pmacct-discussion] Pmacct data inconsistencies between tables.

2010-02-19 Thread Daniel Levy
Thanks, Paolo, for your help. Sorry about the attachments. I have updated the pmacctd.conf file and removed a couple of the plugins to prevent data replication. Regards -- Daniel Levy Aptivate | http://www.aptivate.org/ | +44 (0)1223 760887 The Humanitarian Centre, Fenner's, Gresham Road,

Re: [pmacct-discussion] Pmacct data inconsistencies between tables.

2010-02-19 Thread Karl O. Pinc
On 02/19/2010 10:24:57 AM, Chris Wilson wrote: Hi Karl, On Fri, 19 Feb 2010, Karl O. Pinc wrote: FWIW, the automatic sequential key generation speed is unrelated to table size when using postgresql. There is no sequence to generate as far as I know. The problem is the size of the

Re: [pmacct-discussion] Pmacct data inconsistencies between tables.

2010-02-16 Thread Paolo Lucente
Hi Daniel, Getting through the data and compare traffic figures is, IHMO, the more practical approach - compared to trying to reproduce the issue in a controlled environment. Once you discover a descrepancy, it would be great to receive the contributing data of each report to see where the issue

Re: [pmacct-discussion] Pmacct data inconsistencies between tables.

2010-02-15 Thread Daniel Levy
Hi Paolo, Thanks for getting back to me. The version of pmacct being used is 0.9.1-1ubuntu1. I'm not sure how the problem was discovered, but I have asked to person who found the problem to tell me and I will forward you the response. As for the reports, I'm not entirely sure what you need. I am

Re: [pmacct-discussion] Pmacct data inconsistencies between tables.

2010-02-12 Thread Paolo Lucente
Hi Daniel, Unfortunately the configuration doesn't make evident where the issue can be. The 'sql_dont_try_update' very well protects against duplicate tuples - so i'm rather inclined to exclude that reason. Which version are you using? How you did discover the issue - ie. did you upgrade