On Thursday 19 December 2002 01:13 pm, Joost Diepenmaat wrote:
> Hi all,
>
> I've been working on an application that need to import about 47000
> nodes (including relations) and apart from taking quite a long time to
> import all of them (about 1 hour on a dual 1Gz PIII machine) I sometimes
> see the following or similar errors in the mmbase log:
>
> mmbase.module.database.MultiPool - KILLED SQL SELECT * FROM install_link
> WHERE number=1241 time 1040233606 because it took too long
>
> mmbase.module.database.MultiPool - KILLED SQL INSERT INTO
> install_categorylink (number, otype, owner, snumber, dnumber, rnumber,
> dir, longname) VALUES (?, ?, ?, ?, ?, ?, ?, ?) time 1040058537 because
> it took too long
>
> We are using a Postgresql 7.2.1 database. After some tuning of the
> database the amount of errors went down to about 1 to 2 errors per
> import when the machine is not doing anything else (and we gained about
> 5 minutes import time :-)
>
first:
How can a query take so long?
does it realy take that long to do a select on an index table?
there must be something else going on .. or the error message is just wrong.
> I've got 2 questions:
>
> 1. Will this make the imported data inconsistent?
when using postgress the chances your data wil be inconsitent wil be smaller.
but still when importing a node 2 things happen
-1 the node get's imported
-2 a node is insterted in the syncnodes table
both actions don't take place in the same transaction so yes your data might
be inconsistent.
The current app1 import doesn't detect if only one of the nodes is inserted.
So if you run the import again i should fail..or import your node for a
second time.
I think your data will alway's be
> 2. Is there a way of changing this behaviour i.e. by upping the timeout
> interval or switching timouts off?
the 120 seconds magic time is hardcoded in the database
but it's possible to "almoost" disable the probe by setting the probeTime
Property in jdbc.ml
<!--
sometime connections take a long time, every now and then the
database connection pooling has to check/kill queries that take to long
probetime is the time in seconds between the checks. Connections that are
running for more then 120 seconds will be killed
-->
<property name="probetime">30</property>
The hole database pooling stuff is known to have problems. with bugs
that have been fixed a number of times.. but still not realy..
if you have some time left cloud you look into the postgress logs to see if
the "KILLED query" is realy the KILLED query?
>
> Thanks for your time,
I hope this helps.
>
> Joost Diepenmaat
> Framfab Nederland.
