Here's what I ended up with for persistent thread-local Durus
connections in Pylons. It works in my test, using a TCP socket to the
database server and doing multiple requests until a thread is reused.
Are there any flaws in the coding?
Is CONFIG["global_conf"]{"__file__"] the best way to get the config
file's path? Will it always be defined?
To test if the socket is still alive, I do a makeshift "version"
command to the server. I think Durus ClientStorage needs a dedicated
method for this. (.ping() or .is_connected()) (Note: this catches
IOError if the server has been killed or restarted.)
Is there a way to do the equivalent of thread.get_ident() from the
threading module or Thread objects? This seems to be the only way to
get a thread ID.
Is there a way to get an application instance ID to distinguish
between multiple instances of the same application? This is not the
same as id(the_controller), which would change every request.
--
Mike Orr <[EMAIL PROTECTED]>
"""Durus database interface for Pylons.
Usage:
# In your_pylons_project.lib.base
from pylons_durus import get_connection
class BaseController(WSGIController):
def __call__(self, environ, start_response):
self.db_connection = get_connection("durus.main")
self.db_connection.abort() # Roll back any uncommitted transaction.
return WSGIController(self, environ, start_response)
The user needs an entry in the "[app.main]" section of their config file:
# TCP connection to Durus server.
durus.main = client://localhost:5001
# Unix domain socket to Durus server.
durus.main = unix:///my_db.sock
durus.main = unix:////var/run/my_db.sock
# Open Durus database directly. !!! SAFE ONLY IF readonly=1 !!!
durus.main = file:///relative/path/from/config/file/my_db.durus?readonly=1
durus.main = file:////absolute/path/needs/four/slashes/db.durus?readonly=1
Three slashes name a path relative to the config file. Four slashes name an
absolute path. This was chosen to be consistent with SQLAlchemy's SQLite
behavior, and because I'm using sqlalchemy.engine.make_url() to parse the URL.
Query parameters for all schemes:
cache_size : int (default 1000000).
Number of Persistent objects to cache. NOT byte size of cache.
Query parameters for file: scheme:
readony: 1 or 0 (default 0). Open the database read-only. USE 1!!!
repair: 1 or 0 (default 0). Repair the database on open if corrupted.
The connections are stored in a thread-local dict keyed by URL.
An application may connect to multiple databases by specifying a different
config entry for each (e.g., "durus.connection2").
An application may connect to multiple databases by specifying a different
config key and URL for each. The connections themselves are stored in a
thread-local dict keyed by URL, so identical URLs map to the same connection.
"""
import os
import thread
import threading
from durus.client_storage import ClientStorage
from durus.connection import Connection
from durus.file_storage import FileStorage
from durus.storage_server import recv, sendall
from durus.storage_server import StorageServer
from paste.deploy import CONFIG
from sqlalchemy.engine.url import make_url as make_U
__all__ = ["get_connection"]
localdata = None
def get_connection(config_key):
global localdata
if localdata is None:
localdata = threading.local()
if not hasattr(localdata, "connections"):
localdata.connections = {}
# 'connections' is a thread-local dict of URL : Durus connection.
err_suffix = "config key '%s'" % config_key
url = CONFIG["app_conf"][config_key] # Raises KeyError.
conn = None
if url in localdata.connections:
conn = localdata.connections[url]
if not is_connected(conn):
conn = None
if conn is None:
conn = create_connection(url)
localdata.connections[url] = conn
#print "Thread %s using %s" % (thread.get_ident(), conn)
return conn
def is_connected(conn):
"""Is the connection still alive?"""
if not isinstance(conn.storage, ClientStorage):
return True
# Based on durus.client_storage.ClientStorage.__init__()
protocol = StorageServer.protocol
assert len(protocol) == 4
try:
sendall(conn.storage.s, "V" + protocol)
recv(conn.storage.s, 4) # Discard response.
except IOError:
return False
return True
def create_connection(url):
"""Return a new Durus connection based on the URL."""
paths_relative_to = os.path.dirname(CONFIG["global_conf"]["__file__"])
U = make_U(url)
database = U.database
if paths_relative_to is not None and database is not None:
database = os.path.join(paths_relative_to, U.database)
cache_size = int(U.query.pop("cache_size", 1000000))
if U.drivername == "file":
readonly = bool(int(U.query.pop("readonly", 0)))
repair = bool(int(U.query.pop("repair", 0)))
storage = FileStorage(database, readonly, repair)
elif U.drivername == "client":
storage = ClientStorage(U.host, U.port)
elif U.drivername == "unix": # Unix domain socket.
storage = ClientStorage(address=database)
else:
msg = "unknown Durus scheme '%s' %s"
tup = U.drivername, err_suffix
raise ValueError(msg % tup)
if U.query:
msg = "illegal query parameters for '%s' database connection %s: %s"
tup = U.drivername, err_suffix, ", ".join(U.query.iterkeys())
raise ValueError(msg % tup)
return Connection(storage, cache_size)
_______________________________________________
Paste-users mailing list
[email protected]
http://webwareforpython.org/cgi-bin/mailman/listinfo/paste-users
