Re: [base] analysis table already exists
one thing that comes to my mind and that could explain both the 'table exists' error and the duplication issue is that each job is for some reason executed twice (and at the same time). It shouldn't happen, but I remember that we had some problems when running test programs and still having the web server's job queue active. Every now and then the web server would start executing a job that was also started by our test program. Some times the one executing on the web server would fail and some times the test program. So, do you have any job agents installed, or multiple web applications, or anything else that is executing jobs from the job queue? yes we do have some prototype web apps run from separate tomcats which use instances of the BASE application. I have killed them and did another filtering and got only one result - no more duplication! On starting one tomcat and BASE-related webapp again I still don't get duplication. I'll post back here again if we see duplication again and can recreate the conditions. In the meantime, all seems OK again. I think this is a problem with the how the job queue start jobs. We'll check if we can reproduce it here as well. Having multiple web servers connecting to the same database could be a useful setup in some situations. In the meantime, the recommendation must be to disable the internal job queue in all but one of the Tomcat instances. http://base.thep.lu.se/ticket/887 /Nicklas - Check out the new SourceForge.net Marketplace. It's the best place to buy or sell services for just about anything Open Source. http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace ___ The BASE general discussion mailing list basedb-users@lists.sourceforge.net unsubscribe: send a mail with subject unsubscribe to [EMAIL PROTECTED]
Re: [base] analysis table already exists
Nicklas Nordborg writes: I'll post back here again if we see duplication again and can recreate the conditions. In the meantime, all seems OK again. I think this is a problem with the how the job queue start jobs. We'll check if we can reproduce it here as well. Having multiple web servers connecting to the same database could be a useful setup in some situations. In the meantime, the recommendation must be to disable the internal job queue in all but one of the Tomcat instances. http://base.thep.lu.se/ticket/887 Thanks Nicklas, I was also wondering if it was something to do with polling intervals (as you suggest on the ticket). We actually had 3 instances running when the problem arose (more chance of two synchronising). At the moment we use a carbon copy of base.config for each instance, as this saves a lot of hassle when upgrading BASE on our test server. For production purposes we could figure out some way of maintaining different versions of base.config (with jobqueue.internal.enabled = false) for all but the main web instance. cheers, Bob. -- Bob MacCallum | VectorBase Developer | Kafatos/Christophides Groups | Division of Cell and Molecular Biology | Imperial College London | Phone +442075941945 | Email [EMAIL PROTECTED] - Check out the new SourceForge.net Marketplace. It's the best place to buy or sell services for just about anything Open Source. http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace ___ The BASE general discussion mailing list basedb-users@lists.sourceforge.net unsubscribe: send a mail with subject unsubscribe to [EMAIL PROTECTED]
Re: [base] analysis table already exists
Hi Nicklas,
Nicklas Nordborg writes:
I think the 'table exists' problem can happen in rare cases if two jobs
are executed at the same time, the time window between checking if a
table exists and creating it if it doesn't is small but not zero. This
means that two different jobs may not see the table and both will try to
create it... The last one will fail. I guess it is possible to make the
code more robust in this case, but figured it would maybe not happen
very often (if at all) since the tables are usually created at the same
time as the first root bioassay set in an experiment is created. In any
case, you don't need to worry about database being corrupted.
thanks for the explanation/reassurance (and also for the reply to
the other mail)
The duplication is a bit more surprising. Are the bioassay sets created
by the same or different jobs? You can check the jobs table (View -
Jobs) if you can't find it out from the experiment analysis page. The
well I stupidly deleted them and emptied the trash, so can't check again,
sorry!
but... I just made a filtered bioassayset (raw('flags') == 0) and get this
duplication again! (no errors reported).
so to answer your question above about the jobs:
View-Jobs says that one JEP Filter Plugin job was completed at 12:07:32
And the two duplicate bioassaysets have exactly the same timestamp.
one thing that comes to my mind and that could explain both the 'table
exists' error and the duplication issue is that each job is for some
reason executed twice (and at the same time). It shouldn't happen, but I
remember that we had some problems when running test programs and still
having the web server's job queue active. Every now and then the web
server would start executing a job that was also started by our test
program. Some times the one executing on the web server would fail and
some times the test program. So, do you have any job agents installed,
or multiple web applications, or anything else that is executing jobs
from the job queue?
yes we do have some prototype web apps run from separate tomcats which use
instances of the BASE application. I have killed them and did another
filtering and got only one result - no more duplication!
On starting one tomcat and BASE-related webapp again I still don't get
duplication.
I'll post back here again if we see duplication again and can recreate the
conditions. In the meantime, all seems OK again.
cheers,
Bob.
/Nicklas
Bob MacCallum wrote:
Dear BASE team,
I just got the following error while creating a root bioassayset for a
small
test experiment. My hunch is that it's something bad like a corrupted
db...
I plan to get 2.5 installed ASAP, but was hoping to do a demo with 2.4.5 to
my colleagues later this week.
Well I am in luck, I just repeated exactly the same thing and it worked
fine.
But the odd thing is that I now see that there are three identical-looking
bioassaysets in the experiment, even though I only ran the Formula
intensity
calculator plugin twice. From the Date column it is the latter analysis
which got duplicated (and the original one seems to work ok). I am not
usually a double-clicker, but that might be one explanation.
Strange indeed.
Here's the stack trace in any case.
cheers,
Bob.
net.sf.basedb.core.BaseException: Table 'D13Pos' already exists
at
net.sf.basedb.core.HibernateUtil.createVirtualTable(HibernateUtil.java:605)
at net.sf.basedb.core.VirtualDb.createTables(VirtualDb.java:290)
at
net.sf.basedb.core.PositionBatcher.buildInsertSelectSql(PositionBatcher.java:303)
at net.sf.basedb.core.PositionBatcher.insert(PositionBatcher.java:246)
at
net.sf.basedb.util.IntensityCalculatorUtil.createRootBioAssaySet(IntensityCalculatorUtil.java:204)
at
net.sf.basedb.plugins.IntensityCalculatorPlugin.run(IntensityCalculatorPlugin.java:249)
at
net.sf.basedb.core.PluginExecutionRequest.invoke(PluginExecutionRequest.java:89)
at
net.sf.basedb.core.InternalJobQueue$JobRunner.run(InternalJobQueue.java:421)
at java.lang.Thread.run(Thread.java:619)
Caused by: com.mysql.jdbc.exceptions.MySQLSyntaxErrorException: Table
'D13Pos' already exists
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:936)
at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:2934)
at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:1616)
at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:1708)
at com.mysql.jdbc.Connection.execSQL(Connection.java:3249)
at com.mysql.jdbc.Statement.executeUpdate(Statement.java:1350)
at com.mysql.jdbc.Statement.executeUpdate(Statement.java:1266)
at
com.mchange.v2.c3p0.impl.NewProxyStatement.executeUpdate(NewProxyStatement.java:64)
at
net.sf.basedb.core.HibernateUtil.createVirtualTable(HibernateUtil.java:581)
... 8 more
[base] analysis table already exists
Dear BASE team, I just got the following error while creating a root bioassayset for a small test experiment. My hunch is that it's something bad like a corrupted db... I plan to get 2.5 installed ASAP, but was hoping to do a demo with 2.4.5 to my colleagues later this week. Well I am in luck, I just repeated exactly the same thing and it worked fine. But the odd thing is that I now see that there are three identical-looking bioassaysets in the experiment, even though I only ran the Formula intensity calculator plugin twice. From the Date column it is the latter analysis which got duplicated (and the original one seems to work ok). I am not usually a double-clicker, but that might be one explanation. Strange indeed. Here's the stack trace in any case. cheers, Bob. net.sf.basedb.core.BaseException: Table 'D13Pos' already exists at net.sf.basedb.core.HibernateUtil.createVirtualTable(HibernateUtil.java:605) at net.sf.basedb.core.VirtualDb.createTables(VirtualDb.java:290) at net.sf.basedb.core.PositionBatcher.buildInsertSelectSql(PositionBatcher.java:303) at net.sf.basedb.core.PositionBatcher.insert(PositionBatcher.java:246) at net.sf.basedb.util.IntensityCalculatorUtil.createRootBioAssaySet(IntensityCalculatorUtil.java:204) at net.sf.basedb.plugins.IntensityCalculatorPlugin.run(IntensityCalculatorPlugin.java:249) at net.sf.basedb.core.PluginExecutionRequest.invoke(PluginExecutionRequest.java:89) at net.sf.basedb.core.InternalJobQueue$JobRunner.run(InternalJobQueue.java:421) at java.lang.Thread.run(Thread.java:619) Caused by: com.mysql.jdbc.exceptions.MySQLSyntaxErrorException: Table 'D13Pos' already exists at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:936) at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:2934) at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:1616) at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:1708) at com.mysql.jdbc.Connection.execSQL(Connection.java:3249) at com.mysql.jdbc.Statement.executeUpdate(Statement.java:1350) at com.mysql.jdbc.Statement.executeUpdate(Statement.java:1266) at com.mchange.v2.c3p0.impl.NewProxyStatement.executeUpdate(NewProxyStatement.java:64) at net.sf.basedb.core.HibernateUtil.createVirtualTable(HibernateUtil.java:581) ... 8 more Version BASE 2.4.6pre (build #3938; schema #40) Web server Apache Tomcat/5.5.20 Database Server MySQL 5.0.21-max-log Database Dialectorg.hibernate.dialect.MySQLInnoDBDialect JDBC Driver com.mysql.jdbc.Driver (version 5.0) Java runtimeJava(TM) SE Runtime Environment (1.6.0-b105), Sun Microsystems Inc. Operating systemLinux amd64 2.6.16.53-0.16-smp Memory Total: 474.6 MB Free: 238.2 MB Max: 910.3 MB -- Bob MacCallum | VectorBase Developer | Kafatos/Christophides Groups | Division of Cell and Molecular Biology | Imperial College London | Phone +442075941945 | Email [EMAIL PROTECTED] - Check out the new SourceForge.net Marketplace. It's the best place to buy or sell services for just about anything Open Source. http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace ___ The BASE general discussion mailing list basedb-users@lists.sourceforge.net unsubscribe: send a mail with subject unsubscribe to [EMAIL PROTECTED]
Re: [base] analysis table already exists
I think the 'table exists' problem can happen in rare cases if two jobs are executed at the same time, the time window between checking if a table exists and creating it if it doesn't is small but not zero. This means that two different jobs may not see the table and both will try to create it... The last one will fail. I guess it is possible to make the code more robust in this case, but figured it would maybe not happen very often (if at all) since the tables are usually created at the same time as the first root bioassay set in an experiment is created. In any case, you don't need to worry about database being corrupted. The duplication is a bit more surprising. Are the bioassay sets created by the same or different jobs? You can check the jobs table (View - Jobs) if you can't find it out from the experiment analysis page. The one thing that comes to my mind and that could explain both the 'table exists' error and the duplication issue is that each job is for some reason executed twice (and at the same time). It shouldn't happen, but I remember that we had some problems when running test programs and still having the web server's job queue active. Every now and then the web server would start executing a job that was also started by our test program. Some times the one executing on the web server would fail and some times the test program. So, do you have any job agents installed, or multiple web applications, or anything else that is executing jobs from the job queue? /Nicklas Bob MacCallum wrote: Dear BASE team, I just got the following error while creating a root bioassayset for a small test experiment. My hunch is that it's something bad like a corrupted db... I plan to get 2.5 installed ASAP, but was hoping to do a demo with 2.4.5 to my colleagues later this week. Well I am in luck, I just repeated exactly the same thing and it worked fine. But the odd thing is that I now see that there are three identical-looking bioassaysets in the experiment, even though I only ran the Formula intensity calculator plugin twice. From the Date column it is the latter analysis which got duplicated (and the original one seems to work ok). I am not usually a double-clicker, but that might be one explanation. Strange indeed. Here's the stack trace in any case. cheers, Bob. net.sf.basedb.core.BaseException: Table 'D13Pos' already exists at net.sf.basedb.core.HibernateUtil.createVirtualTable(HibernateUtil.java:605) at net.sf.basedb.core.VirtualDb.createTables(VirtualDb.java:290) at net.sf.basedb.core.PositionBatcher.buildInsertSelectSql(PositionBatcher.java:303) at net.sf.basedb.core.PositionBatcher.insert(PositionBatcher.java:246) at net.sf.basedb.util.IntensityCalculatorUtil.createRootBioAssaySet(IntensityCalculatorUtil.java:204) at net.sf.basedb.plugins.IntensityCalculatorPlugin.run(IntensityCalculatorPlugin.java:249) at net.sf.basedb.core.PluginExecutionRequest.invoke(PluginExecutionRequest.java:89) at net.sf.basedb.core.InternalJobQueue$JobRunner.run(InternalJobQueue.java:421) at java.lang.Thread.run(Thread.java:619) Caused by: com.mysql.jdbc.exceptions.MySQLSyntaxErrorException: Table 'D13Pos' already exists at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:936) at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:2934) at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:1616) at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:1708) at com.mysql.jdbc.Connection.execSQL(Connection.java:3249) at com.mysql.jdbc.Statement.executeUpdate(Statement.java:1350) at com.mysql.jdbc.Statement.executeUpdate(Statement.java:1266) at com.mchange.v2.c3p0.impl.NewProxyStatement.executeUpdate(NewProxyStatement.java:64) at net.sf.basedb.core.HibernateUtil.createVirtualTable(HibernateUtil.java:581) ... 8 more Version BASE 2.4.6pre (build #3938; schema #40) Web serverApache Tomcat/5.5.20 Database Server MySQL 5.0.21-max-log Database Dialect org.hibernate.dialect.MySQLInnoDBDialect JDBC Driver com.mysql.jdbc.Driver (version 5.0) Java runtime Java(TM) SE Runtime Environment (1.6.0-b105), Sun Microsystems Inc. Operating system Linux amd64 2.6.16.53-0.16-smp MemoryTotal: 474.6 MB Free: 238.2 MB Max: 910.3 MB - Check out the new SourceForge.net Marketplace. It's the best place to buy or sell services for just about anything Open Source. http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace ___ The BASE general discussion mailing list basedb-users@lists.sourceforge.net unsubscribe: send a mail with subject unsubscribe to [EMAIL PROTECTED]
