----- Original Message ----- From: "Stefan Bodewig" <bode...@apache.org>
To: <general@gump.apache.org>
Sent: Tuesday, September 08, 2009 2:00 AM
Subject: Re: Problem running Apache Gump [vmgump-public]


On 2009-09-08, <g...@vmgump.apache.org> wrote:

Failed to build project #[(704, 704)] : [test-ant], state:Failed
SQL Warning:<class '_mysql_exceptions.OperationalError'>:(2006, 'MySQL server has gone away')

We keep getting this since a few weeks over and over again.  I guess it
started when somebody upgraded vmgump's OS (many thanks for that!) and
either a newer MySQL or python-mysqldb isn't working as expected.

What I can tell by now is that the database is still there (executed a
few minutes after the emial arrived):


It has a correlation of nearly 1 with commons-lang-2.x failing. I'm guessing that since the later event hoses all M2 builds, that Gump is temporarily running out of connections with so many projects failing quickly. It's only a theory, but it is the one that I based splitting the tests out of commons-lang-2.x.

,----
| r...@vmgump:/var/log# /etc/init.d/mysql status
| * /usr/bin/mysqladmin Ver 8.41 Distrib 5.0.51a, for debian-linux-gnu on i486
| Copyright (C) 2000-2006 MySQL AB
| This software comes with ABSOLUTELY NO WARRANTY. This is free software,
| and you are welcome to modify and redistribute it under the GPL license
|
| Server version          5.0.51a-3ubuntu5.4
| Protocol version        10
| Connection              Localhost via UNIX socket
| UNIX socket             /var/run/mysqld/mysqld.sock
| Uptime:                 1 day 23 hours 11 min 10 sec
|
| Threads: 1 Questions: 6286 Slow queries: 0 Opens: 34 Flush tables: 1 Open tables: 28 Queries per second avg: 0.037
`----

and hasn't restarted during the Gump run.

<http://dev.mysql.com/doc/refman/5.0/en/gone-away.html> hints at various
reasons why this might happen.  The server side timeout is at eight
hours and we shouldn't be hitting that.

It seems as if the python wrapper was not enabling automatic reconnect
and unfortunately the python-mysqldb version installed by hardy (1.2.2)
doesn't have the ping(reconnect) method later versions have added (by
looking through python-mysqldb's svn logs, that is),

mysql error logs don't say anything (or I just don't find them, quite
possible).

My next step would be to wrap each execute() going against the database
in a try/catch that clears the connection and creates a new one if an
OperationalError is caught.  But before I get my feet wet with another
area of Python I've never used, maybe anybody knows of a better
approach.

BTW, after this type of error occurs we get Gump runs where bcel and
many other "early building mvn projects" fail.  This is because the mvn
proxy of the previous run is still up and thinks it could serve
commons-lang but the jar has already been removed by the sync operation
(and thus mvn fails because it cannot download commons-lang).  This
needs to be fixed in a separate way, either by making sure the mvn proxy
is shut down at the end or by clearing it at the start (the later is
probably easier to implement).

Stefan

---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscr...@gump.apache.org
For additional commands, e-mail: general-h...@gump.apache.org




---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscr...@gump.apache.org
For additional commands, e-mail: general-h...@gump.apache.org

Reply via email to