[sqlalchemy] MapperOptions...
...I think I may want to use session.query(...).options(...) to work with the versioning stuff I'm working on to express queries along the lines of: - tell me what the state of play was at 2pm yesterday - give me back new revisions of all objects between 2pm and 4pm yesterday - give me one revision per object that indicates a diff between what the case was at 2pm yesterday and at what it is now Sound like a plan? If so, where can I find out more about MapperOptions and how they work? cheers, Chris -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalch...@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en.
Re: [sqlalchemy] doing stuff in MapperExtension's before_insert and before_update
Michael Bayer wrote: The recursion overflow is fixed and make_transient now removes expiration flags. In the latest tip you can now say: session.expire(object, ['id']) make_transient(object) or alternatively: make_transient(object) object.id = None and the object, when re-added back to a Session, will generate an INSERT and create a new primary key value. Cool, will this make it into 0.6.2? If so, when's that likely to be released? Chris -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalch...@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en.
[sqlalchemy] mappers and non_primary arg
Hi, I'm trying to use non_primary arg of function 'mapper'. Currently I have a GUI widget to browse/edit tables that is based on introspection of the mapper. My goal would be to reuse all the machinary of table browsing even when browsing tables tat where built w/o primary key. I'm ready to understand that I can't use persistence but I thought that flagging as a non primary mapper I could assemble a mapper even from a table missing a primary_key, that's what I thought was the meaning of: Construct a Mapper that will define only the selection of instances, not their persistence. What I get is: ipdb mapper(X, table, non_primary=True) *** ArgumentError: Mapper Mapper|lavori_class|lavori|non-primary could not assemble any primary key columns for mapped table 'lavori' So wht's the real meaning of primary_key? thanks sandro *:-) -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalch...@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en.
[sqlalchemy] models in different packages, often declaratively defined
Hi All, Suppose I have packageA that defines: from sqlalchemy.ext.declarative import declarative_base Base = declarative_base() class User(Base) __tablename__ = 'user' ... Now, I have a packageB that defines: from sqlalchemy.ext.declarative import declarative_base Base = declarative_base() class Something(Base) ... I want Something to have a foreign key that points at User. How do I do that? The tables for packageA and packageB should exist in the same database (the extraction of User into packageA is just so that all our projects that need users get the same schema for the 'user' and related tables and functionality for users). I guess things could be engineered such that one MetaData instance is shared between all the bases (how would that be done though? I'd need to get the MetaData instance into each of the packages before declarative_base is called...) Moreover, how do I get all the Base's to share a _decl_class_registry? (I'm still hazy on why the information in _decl_class_registry can't go into MetaData, rather than having two registries...) Any ideas gratefully received... Chris -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalch...@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en.
[sqlalchemy] Blob streaming
Hi list, I am new user of sqlalchemy. I play with python and MySQL for several years now and I discover sqlalchemy 1 year ago, it change my life. So thank you for this fantastic work. I develop a project of analysing big experimental dataset of intra/extra cellular recordings : OpenElectrophy. I highly use database for this, schema are really simple but table and row are really heavy because I use BLOB to store big raw data signal. I usually play with 1Go to 300Go databases. Playing with SQL is very util but problems are coming because signals get bigger and bigger. After a walk on google, a solution could come with streaming Binary field. In short to be able to take a field chunk by chunk. PBXT engine for mysql seems to umplement it: http://www.blobstreaming.org/ Postgree SQL seems to natively support sort of streaming. I don't known for SQLite. It seems that there is not an official way to stream big binary fields in SQL world. Correct me if I am wrong I am a newbie in this fields. At the end my questions : 1 - Does sqlalchemy plan to implement a unified layer for playing with stream-able field ? I really can't figure out the amount of work to achieve that, sorry if it is naive. 2 - Could you give me somes tips to deal with streaming with theses backend : MySQL, SQLite, postgree Thanks a lot. Samuel -- ~ Samuel Garcia Laboratoire de Neurosciences Sensorielles, Comportement, Cognition. CNRS - UMR5020 - Universite Claude Bernard LYON 1 Equipe logistique et technique 50, avenue Tony Garnier 69366 LYON Cedex 07 FRANCE Tél : 04 37 28 74 24 Fax : 04 37 28 76 01 http://olfac.univ-lyon1.fr/unite/equipe-07/ http://neuralensemble.org/trac/OpenElectrophy ~ -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalch...@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en.
Re: [sqlalchemy] Blob streaming
On Jul 1, 2010, at 4:35 AM, Samuel GARCIA wrote: Hi list, I am new user of sqlalchemy. I play with python and MySQL for several years now and I discover sqlalchemy 1 year ago, it change my life. So thank you for this fantastic work. I develop a project of analysing big experimental dataset of intra/extra cellular recordings : OpenElectrophy. I highly use database for this, schema are really simple but table and row are really heavy because I use BLOB to store big raw data signal. I usually play with 1Go to 300Go databases. Playing with SQL is very util but problems are coming because signals get bigger and bigger. After a walk on google, a solution could come with streaming Binary field. In short to be able to take a field chunk by chunk. PBXT engine for mysql seems to umplement it: http://www.blobstreaming.org/ Postgree SQL seems to natively support sort of streaming. I don't known for SQLite. It seems that there is not an official way to stream big binary fields in SQL world. Correct me if I am wrong I am a newbie in this fields. At the end my questions : 1 - Does sqlalchemy plan to implement a unified layer for playing with stream-able field ? I really can't figure out the amount of work to achieve that, sorry if it is naive. 2 - Could you give me somes tips to deal with streaming with theses backend : MySQL, SQLite, postgree its not something DBAPI supports, and to my knowledge only cx_oracle has an actual streamable blob type - im not aware of one for MySQL, PG, etc. So there are no plans for cross-DBAPI support of streaming blobs in SQLAlchemy right now since DBAPI doesn't provide it for us (we have some support for cx_oracle's feature).My advice for now would be that storing BLOBs are probably not a good idea for streams of data larger than a few megs - I'd use the filesystem instead. -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalch...@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en.
[sqlalchemy] Column order with declarative base
I'm trying to do some DDL creation with declarative base. THe problem I am running into is that I'm using a mixin, and it seems that the order the columns are being created in is different than the order they're declared with. Is there any way to control this? Thanks, Mike Lewis -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalch...@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en.
Re: [sqlalchemy] Column order with declarative base
Mike Lewis wrote: I'm trying to do some DDL creation with declarative base. THe problem I am running into is that I'm using a mixin, and it seems that the order the columns are being created in is different than the order they're declared with. Is there any way to control this? Please provide a simple, small example of your problem :-) Also, is there a reason the order of column creation matters? Chris -- Simplistix - Content Management, Batch Processing Python Consulting - http://www.simplistix.co.uk -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalch...@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en.
[sqlalchemy] Re: Column order with declarative base
Please provide a simple, small example of your problem :-) Also, is there a reason the order of column creation matters? Chris -- Simplistix - Content Management, Batch Processing Python Consulting -http://www.simplistix.co.uk class Foo(object): id = Column(Integer, primary_key=True) foo = Column(Integer, nullable=True) class Bar(Base, object): __tablename__ = 'bar' then on creation of bar, foo might be first. I believe this is because DeclarativeBase adds new documns with dir() and that returns stuff in an arbitrary order. I'm trying to use __metaclass__ to make a metaclass for Foo where it retains order of the dictionary, but I am getting stumped. Thanks, Mike -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalch...@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en.
Re: [sqlalchemy] Re: Column order with declarative base
On Jul 1, 2010, at 6:35 PM, Mike Lewis wrote: Please provide a simple, small example of your problem :-) Also, is there a reason the order of column creation matters? Chris -- Simplistix - Content Management, Batch Processing Python Consulting -http://www.simplistix.co.uk class Foo(object): id = Column(Integer, primary_key=True) foo = Column(Integer, nullable=True) class Bar(Base, object): __tablename__ = 'bar' then on creation of bar, foo might be first. I believe this is because DeclarativeBase adds new documns with dir() and that returns stuff in an arbitrary order. I'm trying to use __metaclass__ to make a metaclass for Foo where it retains order of the dictionary, but I am getting stumped. The Column object contains a sort key when constructed, against a single global value, that is used as a sort key when the Table is generated. This is to get around the fact that the attribute dictionary of the declarative class is unordered. The mixin columns should copy their sort key over, or it should somehow be tailored in the declarative base so that the order of the two columns stays relatively the same, and perhaps is also tailored to be at the same position relative to the other columns in the ultimate table. I'd welcome any patches in this regard since I don't usually deal with the mixin feature. -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalch...@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en.
Re: [sqlalchemy] Re: Column order with declarative base
On Jul 1, 2010, at 8:24 PM, Michael Bayer wrote: The Column object contains a sort key when constructed, against a single global value, that is used as a sort key when the Table is generated. This is to get around the fact that the attribute dictionary of the declarative class is unordered. The mixin columns should copy their sort key over, or it should somehow be tailored in the declarative base so that the order of the two columns stays relatively the same, and perhaps is also tailored to be at the same position relative to the other columns in the ultimate table. I'd welcome any patches in this regard since I don't usually deal with the mixin feature. fine, how about Chris works up the unit test for it: diff -r af4bdd33564e lib/sqlalchemy/ext/declarative.py --- a/lib/sqlalchemy/ext/declarative.py Thu Jul 01 16:57:02 2010 -0400 +++ b/lib/sqlalchemy/ext/declarative.py Thu Jul 01 20:25:59 2010 -0400 @@ -673,7 +673,8 @@ if name not in dict_ and not ( '__table__' in dict_ and name in dict_['__table__'].c ): - potential_columns[name]=column_copies[obj]=obj.copy() +potential_columns[name] = column_copies[obj] = obj.copy() +column_copies[obj]._creation_order = obj._creation_order elif isinstance(obj, RelationshipProperty): raise exceptions.InvalidRequestError( relationships are not allowed on -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalch...@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en.
[sqlalchemy] ObjectDeletedError on orm.Query.delete
We're getting a strange ObjectDeletedError that we've been trying to debug for a large part of the day. The problem occurs when we keep a (hard) reference to an ORM object, drop_all, create_all, then delete() on a query over that ORM class (using the same scoped_session). Unfortunately, just the above steps aren't enough; despite our efforts so far, we haven't been able to reproduce this outside of our (large) application. Any help would be tremendously appreciated. Traceback (most recent call last): ... File /home/yang/work/app.py, line 3115, in foo count = q.delete() File /usr/local/lib/python2.6/dist-packages/sqlalchemy/orm/query.py, line 1858, in delete if issubclass(cls, target_cls) and eval_condition(obj)] File /usr/local/lib/python2.6/dist-packages/sqlalchemy/orm/evaluator.py, line 60, in evaluate value = sub_evaluate(obj) File /usr/local/lib/python2.6/dist-packages/sqlalchemy/orm/evaluator.py, line 60, in evaluate value = sub_evaluate(obj) File /usr/local/lib/python2.6/dist-packages/sqlalchemy/orm/evaluator.py, line 60, in evaluate value = sub_evaluate(obj) File /usr/local/lib/python2.6/dist-packages/sqlalchemy/orm/evaluator.py, line 60, in evaluate value = sub_evaluate(obj) File /usr/local/lib/python2.6/dist-packages/sqlalchemy/orm/evaluator.py, line 60, in evaluate value = sub_evaluate(obj) File /usr/local/lib/python2.6/dist-packages/sqlalchemy/orm/evaluator.py, line 60, in evaluate value = sub_evaluate(obj) File /usr/local/lib/python2.6/dist-packages/sqlalchemy/orm/evaluator.py, line 60, in evaluate value = sub_evaluate(obj) File /usr/local/lib/python2.6/dist-packages/sqlalchemy/orm/evaluator.py, line 82, in evaluate left_val = eval_left(obj) File /usr/local/lib/python2.6/dist-packages/sqlalchemy/orm/evaluator.py, line 42, in lambda return lambda obj: get_corresponding_attr(obj) File /usr/local/lib/python2.6/dist-packages/sqlalchemy/orm/attributes.py, line 159, in __get__ return self.impl.get(instance_state(instance), instance_dict(instance)) File /usr/local/lib/python2.6/dist-packages/sqlalchemy/orm/attributes.py, line 377, in get value = callable_(passive=passive) File /usr/local/lib/python2.6/dist-packages/sqlalchemy/orm/state.py, line 268, in __call__ self.manager.deferred_scalar_loader(self, toload) File /usr/local/lib/python2.6/dist-packages/sqlalchemy/orm/mapper.py, line 2097, in _load_scalar_attributes raise orm_exc.ObjectDeletedError(Instance '%s' has been deleted. % state_str(state)) ObjectDeletedError: Instance 'Blah at 0x33032d0' has been deleted. When echoing the SQL, we see toward the end: ... sqlalchemy.engine.base.Engine.0x...1f90: INFO: DELETE FROM blah WHERE blah.actor_id = ? AND blah.atype = ? AND blah.domain = ? AND ... sqlalchemy.engine.base.Engine.0x...1f90: INFO: (read-only buffer for 0x4aaece0, size -1, offset 0 at 0x4807170, 2, u'fake.com', ...) sqlalchemy.engine.base.Engine.0x...1f90: INFO: SELECT blah.id AS blah_id, blah.actor_id AS blah_actor_id, blah.domain AS blah_domain, ... FROM blah WHERE blah.id = ? sqlalchemy.engine.base.Engine.0x...1f90: INFO: (426,) sqlalchemy.engine.base.Engine.0x...1f90: INFO: ROLLBACK -- Yang Zhang http://yz.mit.edu/ -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalch...@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en.
Re: [sqlalchemy] ObjectDeletedError on orm.Query.delete
On Jul 1, 2010, at 9:40 PM, Yang Zhang wrote: We're getting a strange ObjectDeletedError that we've been trying to debug for a large part of the day. The problem occurs when we keep a (hard) reference to an ORM object, drop_all, create_all, then delete() on a query over that ORM class (using the same scoped_session). Unfortunately, just the above steps aren't enough; despite our efforts so far, we haven't been able to reproduce this outside of our (large) application. Any help would be tremendously appreciated. The object you're holding onto previously is no longer valid. You're dropping all the tables in between, the row is gone. If that old object gets involved in a new session, an attribute gets hit, it goes to the database to refresh, and that's the error you get.In this case it seems that your delete() is seeing that the old object, present in the session, is part of what would have been deleted, and seeks to mark it as deleted- but it's state is incongruent with the changes that have occurred without the Session's knowledge. ORM loaded objects contain state which link them to the row they originated from. There's any number of options to proceed here. You could send the correct arguments to your delete() such that the evaluator isn't used. You could prevent the old object from being expired. You could manipulate the state of the old object or remove it from the session. The best option of all would be, if you're dropping tables, you definitely should be closing out all Sessions beforehand - what you're doing here doesn't make any sense. Traceback (most recent call last): ... File /home/yang/work/app.py, line 3115, in foo count = q.delete() File /usr/local/lib/python2.6/dist-packages/sqlalchemy/orm/query.py, line 1858, in delete if issubclass(cls, target_cls) and eval_condition(obj)] File /usr/local/lib/python2.6/dist-packages/sqlalchemy/orm/evaluator.py, line 60, in evaluate value = sub_evaluate(obj) File /usr/local/lib/python2.6/dist-packages/sqlalchemy/orm/evaluator.py, line 60, in evaluate value = sub_evaluate(obj) File /usr/local/lib/python2.6/dist-packages/sqlalchemy/orm/evaluator.py, line 60, in evaluate value = sub_evaluate(obj) File /usr/local/lib/python2.6/dist-packages/sqlalchemy/orm/evaluator.py, line 60, in evaluate value = sub_evaluate(obj) File /usr/local/lib/python2.6/dist-packages/sqlalchemy/orm/evaluator.py, line 60, in evaluate value = sub_evaluate(obj) File /usr/local/lib/python2.6/dist-packages/sqlalchemy/orm/evaluator.py, line 60, in evaluate value = sub_evaluate(obj) File /usr/local/lib/python2.6/dist-packages/sqlalchemy/orm/evaluator.py, line 60, in evaluate value = sub_evaluate(obj) File /usr/local/lib/python2.6/dist-packages/sqlalchemy/orm/evaluator.py, line 82, in evaluate left_val = eval_left(obj) File /usr/local/lib/python2.6/dist-packages/sqlalchemy/orm/evaluator.py, line 42, in lambda return lambda obj: get_corresponding_attr(obj) File /usr/local/lib/python2.6/dist-packages/sqlalchemy/orm/attributes.py, line 159, in __get__ return self.impl.get(instance_state(instance), instance_dict(instance)) File /usr/local/lib/python2.6/dist-packages/sqlalchemy/orm/attributes.py, line 377, in get value = callable_(passive=passive) File /usr/local/lib/python2.6/dist-packages/sqlalchemy/orm/state.py, line 268, in __call__ self.manager.deferred_scalar_loader(self, toload) File /usr/local/lib/python2.6/dist-packages/sqlalchemy/orm/mapper.py, line 2097, in _load_scalar_attributes raise orm_exc.ObjectDeletedError(Instance '%s' has been deleted. % state_str(state)) ObjectDeletedError: Instance 'Blah at 0x33032d0' has been deleted. When echoing the SQL, we see toward the end: ... sqlalchemy.engine.base.Engine.0x...1f90: INFO: DELETE FROM blah WHERE blah.actor_id = ? AND blah.atype = ? AND blah.domain = ? AND ... sqlalchemy.engine.base.Engine.0x...1f90: INFO: (read-only buffer for 0x4aaece0, size -1, offset 0 at 0x4807170, 2, u'fake.com', ...) sqlalchemy.engine.base.Engine.0x...1f90: INFO: SELECT blah.id AS blah_id, blah.actor_id AS blah_actor_id, blah.domain AS blah_domain, ... FROM blah WHERE blah.id = ? sqlalchemy.engine.base.Engine.0x...1f90: INFO: (426,) sqlalchemy.engine.base.Engine.0x...1f90: INFO: ROLLBACK -- Yang Zhang http://yz.mit.edu/ -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalch...@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en. -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalch...@googlegroups.com. To unsubscribe from this group, send email to
