Hi Chris, The 'failsafe' mechanism (ie. writing to a backup database or to log files) kicks in upon receipt of an error code from the RDBMS API. So what you see in the log file should never be already in the database.
Your specific configuration is tricky because you write to the RDBMS every 60 seconds but aggregate on 5 minutes time-bins and hence it's not possible to determine for sure whether a record is contributing or not to a certain aggregate. Only work-around i can think of for your case is to permanenly enable debug in pmacct (or logging in the RDBMS) so that it's possible, at any moment, to know which queries have been performed. This at the expense of slightly more resources. If using default SQL schemas and an INSERT-only scenario, to protect against duplicates, you could have simply appended the '-i' option to pmpgplay to disable UPDATE SQL queries. Logics is explained here: http://wiki.pmacct.net/CustomizingTheSqlIndexes Cheers, Paolo On Fri, Jul 23, 2010 at 11:53:53AM +1000, Chris wrote: > Hi, > > I'm using pmacct-0.12.0rc3 with postgresql. > > I've been trying to work out what pmpgplay does when it looks at a log > file. I couldn't find much in my searches that go into the details of > how it does things. > > It seems it generates a bunch of sql, but does it check to see if that > record exists already before applying the change? > > I'm worried about replaying the log and it double-recording data (ie the > data is already recorded in the db, and then replaying the log does it > again), or will it only apply new records that it can't find in the db? > > I have a missing chunk of data from my database (about 1.5 hrs worth) > that I can see in the logs, but the log is surrounded by data that could > be in the database (I haven't checked each individual log entry). > > Relevant pmacct.conf info: > > plugin_pipe_size:1024000 > plugin_buffer_size:8192 > imt_buckets: 65537 > imt_mem_pools_size: 1024000 > > sql_table_version:1 > sql_refresh_time: 60 > sql_history: 5m > > Any information is much appreciated, thanks! > > -- > Postgresql & php tutorials > http://www.designmagick.com/ > > > _______________________________________________ > pmacct-discussion mailing list > http://www.pmacct.net/#mailinglists _______________________________________________ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists
