Re: [Repoze-dev] bfg concurrency problem with SqlAlchemy, SOLVED

2010-08-18 Thread Chris Withers
Iain Duncan wrote:
> I was erroneously trying to re-create mappers on each request when my 
> abstract model was being created. I had reasons for this that I won't 
> detail here, but suffice it to say it was a terrible idea because 
> mappers in SA are globa, so if you make a call to clear_mappers in one 
> thread you kill them in another concurrently requesting threadl. I need 
> to make a utility that globally makes my mappers at server startup and 
> leave them alone.

Yes, you generally do this in something akin to a models.py and the code 
gets executed at import time.

I am curious how you got to a position where you thought you needed to 
dynamically define mappers. Care to share?

Chris

-- 
Simplistix - Content Management, Batch Processing & Python Consulting
- http://www.simplistix.co.uk
___
Repoze-dev mailing list
Repoze-dev@lists.repoze.org
http://lists.repoze.org/listinfo/repoze-dev


Re: [Repoze-dev] bfg concurrency problem with SqlAlchemy, SOLVED

2010-08-18 Thread Iain Duncan
Solved, posting in case someone googles this one day.

I was erroneously trying to re-create mappers on each request when my
abstract model was being created. I had reasons for this that I won't detail
here, but suffice it to say it was a terrible idea because mappers in SA are
globa, so if you make a call to clear_mappers in one thread you kill them in
another concurrently requesting threadl. I need to make a utility that
globally makes my mappers at server startup and leave them alone.

hope that helps someone sometime.
Iain

On Wed, Aug 18, 2010 at 9:19 PM, Iain Duncan wrote:

> I'm sure this is my fault, but hoping someone has helpful suggestions.
>  have figured out that my weird issue happens when two requests hit the
> paster server serving my app too quickly. When I put a breakpoint in my
> javascript debugger, and manually separate the ajax requests in time, I
> don't have an issue. When the two come in quickly, my logging from my bfg
> app shows that they are overlapping, ie, I get beginning of the method
> statements from both before end of the method statements from either. In
> that circumstance I then get the internal server error from bfg and the
> traceback that I will post below. I've got a really bad feeling that this
> means my attempt to make an SQLAlchemy abstract model that gets created on
> each request is using some thread shared code or something in SA and hitting
> problems. I'm wondering if anyone has encountered anything like this before
> or would know what to look for at a glance at this traceback. Ugh! =(
>
> thanks
> Iain
>
> (you can see the interleaved logging statements in the below too,)
> 21:16:59,778 INFO  [irmacs.views.bookings] BookingView.__init__()
> 21:16:59,779 INFO  [irmacs.views.bookings] BookingView.by_date() date:
> 2010-08-18
> 21:16:59,851 INFO  [irmacs.views.bookings] BookingView.__init__()
> 21:16:59,852 INFO  [irmacs.views.bookings] BookingView.by_date() date:
> 2009-12-18
> 
> Exception happened during processing of request from ('127.0.0.1', 52483)
> Traceback (most recent call last):
>   File
> "/home/irmacs/irmacs_trunk/eggs/Paste-1.7.4-py2.6.egg/paste/httpserver.py",
> line 1068, in process_request_in_thread
> 21:16:59,881 INFO  [irmacs.views.bookings]  - returning json of: []
> self.finish_request(request, client_address)
>   File "/usr/lib/python2.6/SocketServer.py", line 320, in finish_request
> self.RequestHandlerClass(request, client_address, self)
>   File "/usr/lib/python2.6/SocketServer.py", line 615, in __init__
> self.handle()
>   File
> "/home/irmacs/irmacs_trunk/eggs/Paste-1.7.4-py2.6.egg/paste/httpserver.py",
> line 442, in handle
> BaseHTTPRequestHandler.handle(self)
>   File "/usr/lib/python2.6/BaseHTTPServer.py", line 329, in handle
> self.handle_one_request()
>   File
> "/home/irmacs/irmacs_trunk/eggs/Paste-1.7.4-py2.6.egg/paste/httpserver.py",
> line 437, in handle_one_request
> self.wsgi_execute()
>   File
> "/home/irmacs/irmacs_trunk/eggs/Paste-1.7.4-py2.6.egg/paste/httpserver.py",
> line 287, in wsgi_execute
> self.wsgi_start_response)
>   File
> "/home/irmacs/irmacs_trunk/eggs/repoze.who-1.0.18-py2.6.egg/repoze/who/middleware.py",
> line 107, in __call__
> app_iter = app(environ, wrapper.wrap_start_response)
>   File
> "/home/irmacs/irmacs_trunk/eggs/repoze.bfg-1.2.1-py2.6.egg/repoze/bfg/router.py",
> line 130, in __call__
> response = view_callable(context, request)
>   File
> "/home/irmacs/irmacs_trunk/eggs/repoze.bfg-1.2.1-py2.6.egg/repoze/bfg/configuration.py",
> line 1665, in _bfg_class_view
> response = getattr(inst, attr)()
>   File "/home/irmacs/irmacs_trunk/Irmacs/irmacs/views/bookings.py", line
> 48, in by_date
> bookings = self.model.list( Booking, {'date':isodate} )
>   File
> "/home/irmacs/irmacs_trunk/framework_src/xornot.model/xornot/model/__init__.py",
> line 208, in list
> items = query.all()
>   File
> "/home/irmacs/irmacs_trunk/eggs/SQLAlchemy-0.5.8-py2.6.egg/sqlalchemy/orm/query.py",
> line 1267, in all
> return list(self)
>   File
> "/home/irmacs/irmacs_trunk/eggs/SQLAlchemy-0.5.8-py2.6.egg/sqlalchemy/orm/query.py",
> line 1422, in instances
> rows = [process[0](context, row) for row in fetch]
>   File
> "/home/irmacs/irmacs_trunk/eggs/SQLAlchemy-0.5.8-py2.6.egg/sqlalchemy/orm/query.py",
> line 2032, in main
> return _instance(row, None)
>   File
> "/home/irmacs/irmacs_trunk/eggs/SQLAlchemy-0.5.8-py2.6.egg/sqlalchemy/orm/mapper.py",
> line 1751, in _instance
> state._run_on_load(instance)
>   File
> "/home/irmacs/irmacs_trunk/eggs/SQLAlchemy-0.5.8-py2.6.egg/sqlalchemy/orm/state.py",
> line 119, in _run_on_load
> self.manager.events.run('on_load', instance)
> AttributeError: 'NoneType' object has no attribute 'run'
> 
>
>
___
Repoze-dev mailing list
Repoze-dev@lists.repoze.org
http://lists.repoze.org/listinfo/repoze-dev