On Wed, 2010-02-10 at 09:25 +0100, Steffen Mueller wrote: Sorry for top posting, but further to this:
make test on trunk tonight showed this: t/82-plugin-manager.t ...... 1/29 DESTROY ignored because DBI dbh handle (DBD::SQLite::db) is owned by thread 8ce2008 not current thread 9f86080 t/82-plugin-manager.t ...... ok Pete > Hi Adam, hi all, > > Adam Kennedy wrote: > > There is an expected but never before observed bug to do with locking > > and threads. > > Yummy. > > > ORLite in default mode runs every SQL statement as a separate > > connection. The connectionless nature means we don't have to care > > about the problem of DBI being incompatible with threads. > > > > Unfortunately, now we have this great locking system. > > > > And the locking system is going to run persistant connections to the > > database, which might end up surviving across the thread fork, and > > thus exploding in the manner you see. > > DBI isn't really considered thread safe by Tim Bunce, either. > > > Because Padre only uses transactions as a convenience to speed up > > statements and control when the disk blocking will occur, we can use > > the following solution. > > > > 1. Make the Task Manager aware that the locking system exists. > > > > 2. Add special code to the lock manager so that when the Task Manager > > is about to fork off a thread, any current database connection is > > committed, and then disconnected. > > Note: New threads should be spawned fairly rarely: o(tens/hundreds of > seconds). And doing so is already hideously expensive. > > > 3. After the thread is spawned, the Task Manager should return the DB > > lock to the state it was before the thread spawn (either connected, or > > not connected). > > > > 4. The above should have very very tight so there's no chance for > > anything other than the thread spawn to occur (other code might screw > > up the lock state). > > I sense danger. Let me get a few bits straight: > > 1) Can threads other than the main thread access the DB? I think yes. > Barring them from using the configuration would be a problem. > > => Therefore, your point 4. isn't enough. It's asking for strange race > conditions. > > 2) Can ORLite have multiple connections to the same DB? > => I guess not. > 3) Can SQLite handle multiple connections (i.e. from different > processes) to the same DB? > => I guess yes. > 4) Can DBD::SQLite handle multiple connections from the same process to > the same DB? > => No idea. > > I was thinking that maybe each thread should/could handle its own DB > connection. ORLite is pure Perl, so by just spawning off a new ithread, > it should be safe unless it holds a db handle. The db handle itself is > an opaque XS object (DBI, I presume). Now, let me go on a wild rampage: > If we create a class that trivially inherits from DBI and use that class > in place of DBI everywhere. Then we can use XS to add a thread cloning > hook to our new class. When a new thread is spawned, this hook simply > creates a new DB connection for the new thread -- an entirely new DBI > handle. > Now, there are a few drawbacks to this plan: > a) It's a lot of work and potentially fragile. > b) DBI docs IIRC say something like "should be fine in threads, but > having handles to the same DB in different threads might be weird", i.e. > warn about this scenario specifically. > c) DBD's are usually not thread-safe. This could manifest in that they > use some C-level storage for the db handles. This in turn would make it > impossible to have separate connections to the same per thread. I have > no idea about DBD::SQLite, but if in doubt, it's probably not up to par > with the likes of DBD::Pg/mysql/Oracle. > > Okay, so let's consider this a thought experiment for now and move on. > Back to Adam' suggestion > > One could make this thread-safe-ish by using a global semaphore > (Thread::Semaphore or whatever) in the Locking system to synchronize all > interaction with the database. This may be a problem for applications > that hit the DB a lot, but feels safe for us. > > Suppose I want to access the config DB. What's the way this works? Is it: > > a) I get a lock on it > b) I use the DB > c) lock goes out of scope > d) other bits of code could use the DB? > > If so, we're in luck. Just add the afortmentioned semaphore to the DB > locking. AND, of course, we need to implement Adam's plan. > > > 5. The database operations should occur WITHOUT changing the > > underlying state (lock counts etc) of the locking system. > > Of course. > > I won't be able to do any hacking before the week-end and even then, it > looks shaky. > > Best regards, > Steffen > _______________________________________________ > Padre-dev mailing list > [email protected] > http://mail.perlide.org/mailman/listinfo/padre-dev
_______________________________________________ Padre-dev mailing list [email protected] http://mail.perlide.org/mailman/listinfo/padre-dev
