Important news! 1) The new DAL (dal.py) passes all the tests I have. 2) The new DAL has integrated GAE support (datastore) without need for contrib/gql.py 3) The new DAL is better because more customizable, smaller (10%), and more readable 4) The new DAL is one single file and it does not depend on web2py. You can use it with other frameworks out of the box without tweaking.
TODO: - test test test and more tests - benckmark old dal vs new dal - replace old dal with new dal in stable web2py I may keep working on this make more corrections/improvements. Massimo On Dec 6, 1:12 pm, ron_m <[email protected]> wrote: > Ok that worked for starting the welcome app. As Jonathan suggests the > self.uri in ConnectionPool.pools[self.uri] needs to be instance.uri as > well. > > I will look to see why the application I am working on won't connect > to postgresql - timeout after 5 tries and get back if I find the > cause. > > Thanks > Ron > > On Dec 6, 9:25 am, mdipierro <[email protected]> wrote: > > > please try replace > > > @staticmethod > > def close_all_instances(action): > > """ to close cleanly databases in a multithreaded environment > > """ > > if not hasattr(thread,'instances'): > > return > > while thread.instances: > > instance = thread.instances.pop() > > action(instance) > > # ## if you want pools, recycle this > > connection > > really = True > > if instance.pool_size: > > sql_locker.acquire() > > pool = ConnectionPool.pools[self.uri] > > if len(pool) < instance.pool_size: > > pool.append(self.connection) > > really = False > > sql_locker.release() > > if really: > > self.connection.close() > > return > > > with > > > @staticmethod > > def close_all_instances(action): > > """ to close cleanly databases in a multithreaded environment > > """ > > if not hasattr(thread,'instances'): > > return > > while thread.instances: > > instance = thread.instances.pop() > > action(instance) > > # ## if you want pools, recycle this > > connection > > really = True > > if instance.pool_size: > > sql_locker.acquire() > > pool = ConnectionPool.pools[self.uri] > > if len(pool) < instance.pool_size: > > pool.append(instance.connection) > > really = False > > sql_locker.release() > > if really: > > instance.connection.close() > > return > > > I think this is global replace error. > > > On Dec 6, 11:13 am, ron_m <[email protected]> wrote: > > > > I did an update of trunk using hg and copied dal.py over sql.py in > > > gluon and start the web2py.py server and still get > > > > Traceback (most recent call last): > > > File "/home/camcentral/Dev/web2py_hg/gluon/main.py", line 446, in > > > wsgibase > > > BaseAdapter.close_all_instances(BaseAdapter.commit) > > > File "/home/camcentral/Dev/web2py_hg/gluon/sql.py", line 276, in > > > close_all_instances > > > self.connection.close() > > > NameError: global name 'self' is not defined > > > > main is calling BaseAdapter.close_all_instances() as a class method so > > > the self reference at line 276 blows up because there is no instance > > > of a class. I see from the latest post we can test dal.py standalone, > > > are we not to be testing as part of web2py yet because of integration > > > issues? > > > > Ron > > > > On Dec 5, 7:11 pm, mdipierro <[email protected]> wrote: > > > > > started work integrating with gluon/contrib/gql.py do > > > > > please keep testing dal.py > > > > > now it no longer requires web2py and you can do > > > > > % python>>> from dal import DAL, Field > > > > >>> db=DAL('sqlite://file.sqlite') > > > > >>> db.define_table('person',Field('name')) > > > > > etc etc. > > > > > you ONLY need dal.py > > > > > On Dec 5, 1:56 pm, mdipierro <[email protected]> wrote: > > > > > > in trunk! > > > > > > On Dec 5, 1:38 pm, "mr.freeze" <[email protected]> wrote: > > > > > > > Auth.define_tables doesn't have a fake_migrate attribute so I added > > > > > > one (want a patch?). Once I did, my .table files were re-created > > > > > > when > > > > > > migrate=True and fake_migrate=True. It still works once I set > > > > > > fake_migrate to False. I will test it against the new DAL now. > > > > > > > On Dec 5, 1:24 pm, mdipierro <[email protected]> wrote: > > > > > > > > can you try migrate=True, fake_migrate=True? > > > > > > > > On Dec 5, 1:05 pm, "mr.freeze" <[email protected]> wrote: > > > > > > > > > I tried and the file modification times of the .table files do > > > > > > > > not > > > > > > > > change. I deleted them (safely backed up) and they are not > > > > > > > > re-created. > > > > > > > > > On Dec 5, 12:56 pm, mdipierro <[email protected]> wrote: > > > > > > > > > > It is the same. try without delete them. > > > > > > > > > > On Dec 5, 12:53 pm, "mr.freeze" <[email protected]> wrote: > > > > > > > > > > > I reverted to the old DAL and did the steps you outlined. > > > > > > > > > > Same result. > > > > > > > > > > Should I delete the .table files first? > > > > > > > > > > > On Dec 5, 12:41 pm, mdipierro <[email protected]> > > > > > > > > > > wrote: > > > > > > > > > > > > for the troublesome table (auth_user) set migrate=False, > > > > > > > > > > > fake_migrate=True reload, then migrate=True and remove > > > > > > > > > > > fake_migrate. > > > > > > > > > > > It should fix your broken .table. > > > > > > > > > > > > Should work with old and with new dal. > > > > > > > > > > > > On Dec 5, 11:47 am, "mr.freeze" <[email protected]> > > > > > > > > > > > wrote: > > > > > > > > > > > > > I seem to remember this failing before though. > > > > > > > > > > > > Something about > > > > > > > > > > > > my .TABLE files being out of sync. What is the > > > > > > > > > > > > procedure to re-sync > > > > > > > > > > > > them? > > > > > > > > > > > > > On Dec 5, 10:45 am, "mr.freeze" <[email protected]> > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > I have a global variable migrate_db which is passed > > > > > > > > > > > > > to all > > > > > > > > > > > > > define_table functions. I switched it to false and > > > > > > > > > > > > > get this: > > > > > > > > > > > > > Traceback (most recent call last): > > > > > > > > > > > > > File "C:\web2py\gluon\restricted.py", line 188, in > > > > > > > > > > > > > restricted > > > > > > > > > > > > > exec ccode in environment > > > > > > > > > > > > > File "C:/web2py/applications/main/models/db.py", > > > > > > > > > > > > > line 34, in > > > > > > > > > > > > <module> > > > > > > > > > > > > > readable=False, default=""),migrate=migrate_db) > > > > > > > > > > > > > File "C:\web2py\gluon\sql.py", line 1406, in > > > > > > > > > > > > > define_table > > > > > > > > > > > > > obj = str(obj) > > > > > > > > > > > > > File "C:\web2py\gluon\sql.py", line 1858, in _create > > > > > > > > > > > > > 'boolean': 'CHAR(1)', > > > > > > > > > > > > > File "C:\web2py\gluon\sql.py", line 1024, in > > > > > > > > > > > > > <lambda> > > > > > > > > > > > > > 'string': 'CHAR(%(length)s)', > > > > > > > > > > > > > File > > > > > > > > > > > > > "C:\Python26\lib\site-packages\MySQLdb\cursors.py", > > > > > > > > > > > > > line 166, > > > > > > > > > > > > > in execute > > > > > > > > > > > > > self.errorhandler(self, exc, value) > > > > > > > > > > > > > File > > > > > > > > > > > > > "C:\Python26\lib\site-packages\MySQLdb\connections.py", > > > > > > > > > > > > > line > > > > > > > > > > > > > 35, in defaulterrorhandler > > > > > > > > > > > > > raise errorclass, errorvalue > > > > > > > > > > > > > OperationalError: (1050, "Table 'auth_user' already > > > > > > > > > > > > > exists") > > > > > > > > > > > > > > On Dec 5, 10:37 am, "mr.freeze" > > > > > > > > > > > > > <[email protected]> wrote: > > > > > > > > > > > > > > > Let me backup the web2pyslice.com database and I > > > > > > > > > > > > > > will try. > > > > > > > > > > > > > > > On Dec 5, 10:36 am, mdipierro > > > > > > > > > > > > > > <[email protected]> wrote: > > > > > > > > > > > > > > > > can you try migrate=True (add a dummy field and > > > > > > > > > > > > > > > the remove it) with > > > > > > > > > > > > > > > mysql? > > > > > > > > > > > > > > > > On Dec 5, 10:20 am, "mr.freeze" > > > > > > > > > > > > > > > <[email protected]> wrote: > > > > > > > > > > > > > > > > > Working with mysql also. I tested with > > > > > > > > > > > > > > > > migrate=False. Nice work! > > > > > > > > > > > > > > > > > On Dec 5, 10:12 am, "mr.freeze" > > > > > > > > > > > > > > > > <[email protected]> wrote: > > > > > > > > > > > > > > > > > > Okay, web2pyslices.com source runs against it > > > > > > > > > > > > > > > > > with a sqlite database. > > > > > > > > > > > > > > > > > I will test with mysql. > > > > > > > > > > > > > > > > > > On Dec 4, 11:02 pm, mdipierro > > > > > > > > > > > > > > > > > <[email protected]> wrote: > > > > > > > > > > > > > > > > > > > One more test please. I do not have mysql > > > > > > > > > > > > > > > > > > installed here so I did not > > > > > > > > > > > > > > > > > > test pooling. This helps a lot. thanks. > > > > > > > > > > > > > > > > > > > On Dec 4, 10:33 pm, "mr.freeze" > > > > > > > > > > > > > > > > > > <[email protected]> wrote: > > > > > > > > > > > > > > > > > > > > Looking closer, there is a syntax error > > > > > > > > > > > > > > > > > > > on line 189: COnnectionPool > > > > > > > > > > > > > > > > > > > After I fixed that, I get: > > > > > > > > > > > > > > > > > > > Traceback (most recent call last): > > > > > > > > > > > > > > > > > > > File "C:\web2py\gluon\main.py", line > > > > > > > > > > > > > > > > > > > 488, in wsgibase > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > BaseAdapter.close_all_instances(BaseAdapter.rollback) > > > > > > > > > > > > > > > > > > > File "C:\web2py\gluon\sql.py", line > > > > > > > > > > > > > > > > > > > 163, in close_all_instances > > > > > > > > > > > > > > > > > > > pool = > > > > > > > > > > > > > > > > > > > ConnectionPool._pools[instance._uri] > > > > > > > > > > > > > > > > > > > AttributeError: 'SQLiteAdapter' object > > > > > > > > > > > > > > > > > > > has no attribute '_uri' > > > > > > > > > > > > > > > > > > > > On Dec 4, 10:26 pm, "mr.freeze" > > > > > > > > > > > > > > > > > > > <[email protected]> wrote: > > > > > > > > > > > > > > > > > > > > > I get this now: > > > > > > > > > > > > > > > > > > > > Traceback (most recent call last): > > > > > > > > > > > > > > > > > > > > File "C:\web2py\gluon\restricted.py", > > > > > > > > > > > > > > > > > > > > line 188, in restricted > > > > > > > > > > > > > > > > > > > > exec ccode in environment > > > > > > > > > > > > > > > > > > > > File > > > > > > > > > > > > > > > > > > > > "C:/web2py/applications/main/models/db.py", > > > > > > > > > > > > > > > > > > > > line 22, in > > > > > > > > > > > > > > > > > > > > <module> > > > > > > > > > > > > > > > > > > > > db = > > > > > > > > > > > > > > > > > > > > SQLDB(dal_connection,pool_size=10) > > > > > > > > > > > > > > > > > > > > File "C:\web2py\gluon\sql.py", line > > > > > > > > > > > > > > > > > > > > 2405, in __init__ > > > > > > > > > > > > > > > > > > > > raise RuntimeError, "Failure to > > > > > > > > > > > > > > > > > > > > connect, tried 5 times" > > ... > > read more »
