[pmacct-discussion] pmacct can't create mysql table, but I can with the same schema.

2009-12-22 Thread Jeff Welling
Hi
Im running into a strange problem where I'm trying to get pmacct to
dynamically create the database tables, something that I understand it
can do, with the simple pmacct-create-table_v7.sqlite3 file, with the
table names updated.

The error that I'm seeing from pmacct in my syslog is this

Dec 22 02:40:05 Gir pmacctd[9922]: ERROR ( incoming/mysql ): PRIMARY
'mysql' backend trouble.
Dec 22 02:40:05 Gir pmacctd[9922]: ERROR ( incoming/mysql ): The SQL
server says: You have an error in your SQL syntax; check the manual that
corresponds to your MySQL server version for the right syntax to use
near ';  CREATE TABLE pmacct_v7_incoming_200912 ( ^Iagent_id INT(4) NOT
NULL DEFAULT 0,' at line 1
Dec 22 02:40:06 Gir pmacctd[9922]: ERROR ( incoming/mysql ): PRIMARY
'mysql' backend trouble.
Dec 22 02:40:06 Gir pmacctd[9922]: ERROR ( incoming/mysql ): The SQL
server says: Table 'pmacct.pmacct_v7_incoming_200912' doesn't exist


And the schema file that it is using to generate it it this.



CREATE TABLE pmacct_v7_incoming_200901 (
  agent_id INT(4) NOT NULL DEFAULT 0,
  class_id CHAR(16) NOT NULL DEFAULT ' ',
  mac_src CHAR(17) NOT NULL DEFAULT '0:0:0:0:0:0',
  mac_dst CHAR(17) NOT NULL DEFAULT '0:0:0:0:0:0',
  vlan INT(4) NOT NULL DEFAULT 0,
as_src INT(4) NOT NULL DEFAULT 0,
as_dst INT(4) NOT NULL DEFAULT 0,
  ip_src CHAR(15) NOT NULL DEFAULT '0.0.0.0',
  ip_dst CHAR(15) NOT NULL DEFAULT '0.0.0.0',
  src_port INT(4) NOT NULL DEFAULT 0,
  dst_port INT(4) NOT NULL DEFAULT 0,
  tcp_flags INT(4) NOT NULL DEFAULT 0,
  ip_proto CHAR(6) NOT NULL DEFAULT 0,
  tos INT(4) NOT NULL DEFAULT 0,
packets INT NOT NULL,
  bytes BIGINT NOT NULL,
  flows INT NOT NULL DEFAULT 0,
  stamp_inserted DATETIME NOT NULL DEFAULT '-00-00 00:00:00',
  stamp_updated DATETIME,
  PRIMARY KEY (agent_id, class_id, mac_src, mac_dst, vlan, as_src,
as_dst, ip_sr
c, ip_dst, src_port, dst_port, ip_proto, tos, stamp_inserted)
);





When I do a simple `mysql -p pmacct  v7_incoming.schema` it goes off
and adds the table without a hitch, yet for some reason pmacct throws
that error.   I can't figure out why?  Any ideas?

My config for that one is




! pmacctd configuration
!
!
!
daemonize: true
pidfile: /var/run/pmacctd.pid
syslog: daemon
sql_host: 192.168.24.20
sql_passwd: aggg...@!@#
sql_refresh_time: 300
sql_optimize_clauses: true
sql_history: 5m
sql_table_version: 7
sql_history_roundoff: mhd
sql_cache_entries: 256000
sql_table_schema[incoming]: /etc/pmacct/v7_incoming.schema
sql_table_schema[outgoing]: /etc/pmacct/v7_outgoing.schema
sql_recovery_logfile: /var/lib/pmacct/recovery_log
sql_table[incoming]: pmacct_v7_incoming_%Y%m
sql_table[outgoing]: pmacct_v7_outgoing_%Y%m

interface: eth2
plugin_buffer_size: 16384
plugin_pipe_size: 16777216
networks_file: /etc/pmacct/networks.def
plugins: mysql[incoming], mysql[outgoing], memory[in], memory[out]
aggregate[in]: dst_host
aggregate[out]: src_host
aggregate[incoming]:
src_host,dst_host,src_mac,dst_mac,vlan,src_port,dst_port,tos,proto,flows
aggregate[outgoing]:
src_host,dst_host,src_mac,dst_mac,vlan,src_port,dst_port,tos,proto,flows
aggregate_filter[in]: dst host 75.155.123.182
aggregate_filter[out]: src host 75.155.123.182
aggregate_filter[incoming]: dst host 75.155.123.182
aggregate_filter[outgoing]: src host 75.155.123.182

imt_path[in]: /tmp/pmacct_eth2_in.pipe
imt_path[out]: /tmp/pmacct_eth2_out.pipe



If your wondering about any values that look strange to you, *shrugs* it
works for me.  It took a lot to get it to just do what it does right
now, had to deal with errors about running out of memory and such.
Fiddling with various numbers prevented it from throwing nasty errors in
my syslog and segfaulting, and this is what my config is today as a
result.  My setup is kind of complex, I can try and explain if you want,
and Im trying to get the dynamic tables working on the other one as well
but posting all that extra log info just for the other daemon seemed
superfluous.

Thanks in advance!
Jeff.

___
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists


Re: [pmacct-discussion] pmacct can't create mysql table, but I can with the same schema

2009-12-22 Thread Jeff Welling
Hi again,
So it seems I spoke too soon  .  It looks like its accumulating
information in the database in the right tables, but I don't know that
the information is correct or why its trying to create the table even
though it already exists..?

I'm seeing this in my syslogs, one per table but here's the output from
just one;

Dec 22 03:35:15 Gir pmacctd[10365]: ERROR ( incoming/mysql ): PRIMARY
'mysql' backend trouble.
Dec 22 03:35:15 Gir pmacctd[10365]: ERROR ( incoming/mysql ): The SQL
server says: Table 'pmacct_v7_incoming_200912' already exists

It shows up, I'm guessing, every time it tries to update the sql
database.  Any idea what I did wrong?
Anything at all will be appreciated :)
Thanks,
Jeff.


___
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists


Re: [pmacct-discussion] pmacct can't create mysql table, but I can with the same schema

2009-12-22 Thread Paolo Lucente
Hi Jeff,

If i'm getting this correctly, you see everything working except for
the table creation error you get back. 

If this is the case, i wouldn't say you are doing something wrong;
pmacct tries to create the table every time the SQL cache scanner kicks
in (sql_refresh_time interval); table name is a string, just it gets
compiled in at runtime with dynamic tables. So, the failure to create
the table derives from the fact the table name changes monthly in your
case whereas data is inserted every 5 minutes. In summary: this is an
harmless and not impacting issue.

Why in the end this log message is labelled by pmacct as ERROR? pmacct
just reports what the SQL library says - it has no clue of the severity
of the content itself. This should really be an informational one.

Cheers,
Paolo


On Tue, Dec 22, 2009 at 03:44:16AM -0800, Jeff Welling wrote:
 Hi again,
 So it seems I spoke too soon  .  It looks like its accumulating
 information in the database in the right tables, but I don't know that
 the information is correct or why its trying to create the table even
 though it already exists..?
 
 I'm seeing this in my syslogs, one per table but here's the output from
 just one;
 
 Dec 22 03:35:15 Gir pmacctd[10365]: ERROR ( incoming/mysql ): PRIMARY
 'mysql' backend trouble.
 Dec 22 03:35:15 Gir pmacctd[10365]: ERROR ( incoming/mysql ): The SQL
 server says: Table 'pmacct_v7_incoming_200912' already exists
 
 It shows up, I'm guessing, every time it tries to update the sql
 database.  Any idea what I did wrong?
 Anything at all will be appreciated :)
 Thanks,
 Jeff.


___
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists