----- Original Message -----
From: "Stefan Bodewig" <[email protected]>
To: <[email protected]>
Sent: Tuesday, September 08, 2009 2:00 AM
Subject: Re: Problem running Apache Gump [vmgump-public]
On 2009-09-08, <[email protected]> wrote:
Failed to build project #[(704, 704)] : [test-ant], state:Failed
SQL Warning:<class '_mysql_exceptions.OperationalError'>:(2006, 'MySQL
server has gone away')
We keep getting this since a few weeks over and over again. I guess it
started when somebody upgraded vmgump's OS (many thanks for that!) and
either a newer MySQL or python-mysqldb isn't working as expected.
What I can tell by now is that the database is still there (executed a
few minutes after the emial arrived):
It has a correlation of nearly 1 with commons-lang-2.x failing. I'm
guessing that since the later event hoses all M2 builds, that Gump is
temporarily running out of connections with so many projects failing
quickly. It's only a theory, but it is the one that I based splitting the
tests out of commons-lang-2.x.
,----
| r...@vmgump:/var/log# /etc/init.d/mysql status
| * /usr/bin/mysqladmin Ver 8.41 Distrib 5.0.51a, for debian-linux-gnu
on i486
| Copyright (C) 2000-2006 MySQL AB
| This software comes with ABSOLUTELY NO WARRANTY. This is free software,
| and you are welcome to modify and redistribute it under the GPL license
|
| Server version 5.0.51a-3ubuntu5.4
| Protocol version 10
| Connection Localhost via UNIX socket
| UNIX socket /var/run/mysqld/mysqld.sock
| Uptime: 1 day 23 hours 11 min 10 sec
|
| Threads: 1 Questions: 6286 Slow queries: 0 Opens: 34 Flush tables: 1
Open tables: 28 Queries per second avg: 0.037
`----
and hasn't restarted during the Gump run.
<http://dev.mysql.com/doc/refman/5.0/en/gone-away.html> hints at various
reasons why this might happen. The server side timeout is at eight
hours and we shouldn't be hitting that.
It seems as if the python wrapper was not enabling automatic reconnect
and unfortunately the python-mysqldb version installed by hardy (1.2.2)
doesn't have the ping(reconnect) method later versions have added (by
looking through python-mysqldb's svn logs, that is),
mysql error logs don't say anything (or I just don't find them, quite
possible).
My next step would be to wrap each execute() going against the database
in a try/catch that clears the connection and creates a new one if an
OperationalError is caught. But before I get my feet wet with another
area of Python I've never used, maybe anybody knows of a better
approach.
BTW, after this type of error occurs we get Gump runs where bcel and
many other "early building mvn projects" fail. This is because the mvn
proxy of the previous run is still up and thinks it could serve
commons-lang but the jar has already been removed by the sync operation
(and thus mvn fails because it cannot download commons-lang). This
needs to be fixed in a separate way, either by making sure the mvn proxy
is shut down at the end or by clearing it at the start (the later is
probably easier to implement).
Stefan
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
