On Fri, Aug 24, 2018 at 3:12 AM, Chris Wilson <[email protected]> wrote: > Dear Michael, > > > > I have discovered that under very particular circumstances, SQLAlchemy > forgets that an object was newly created in a transaction, and if that > transaction rolls back, the object’s state is not reset from persistent to > transient.
right off I see it is attempting to use SAVEPOINT with SQLite which won't work unless you apply: http://docs.sqlalchemy.org/en/latest/dialects/sqlite.html#pysqlite-serializable. but that shouldn't affect the ORM's sense of if the object is persistent or not unless it is retrieved fresh. your example doesn't cut and paste cleanly but after looking for a bit the problem is that because you are losing the identity of fido after create() is done, you then ask it to fetch fido fresh into memory all over again, then roll back the transaction, but the ORM has no idea that this object was part of this transaction because you let it get lost. the ORM doesn't track states that are garbage collected because it is just as likely the program is running millions of states through the transaction in batches and holding onto them when the program has decided it doesn't care about them anymore would constitute a memory leak. The .persistent attribute does not incur a refresh, however if you access any of the normal attributes on the object like .id, *and* you've also put the SAVEPOINT, you'll get: "Instance '<A at 0x7f0651cc2f98>' has been deleted, or its row is otherwise not present.". > > > > It seems to rely on the following: > > > > · Open an outer transaction > > · Open an inner transaction, add object to session, close transaction > to commit. The object is now persistent. > > · Clear all references to the object, allowing it to be GC’ed and > removed from the session._new weakref map > > · Load the object from the database again. > > · Raise an exception to rollback the outer transaction. > > · SQLAlchemy does not realise that the object is new, and fails to > reset its state. > > · Object is still “persistent”, but no longer in the database. > > > > Here is an example that reproduces the problem: > > > > from sqlalchemy import Column, Integer, Text, create_engine, inspect > > from sqlalchemy.ext.declarative import declarative_base > > from sqlalchemy.orm import sessionmaker > > > > Base = declarative_base() > > > > class Dog(Base): > > __tablename__ = 'dog' > > > > id = Column(Integer, primary_key=True) > > name = Column(Text) > > > > def create(): > > global session > > with session.begin(nested=True): > > fido = Dog(name="fido") > > session.add(fido) > > assert not inspect(fido).persistent, "should not yet be persisted to > database" > > assert inspect(fido).persistent, "should now be persisted to database, > but transaction not yet committed" > > > > def fetch(): > > global session > > global fido > > fido = session.query(Dog).filter_by(name="fido").one() > > assert inspect(fido).persistent, "should have been retrieved from the > database, therefore persistent" > > > > def main(): > > engine = create_engine('sqlite://') > > engine.echo = True > > Base.metadata.create_all(engine) > > > > DBSession = sessionmaker(bind=engine) > > > > global session > > session = DBSession(autocommit=True) > > > > try: > > with session.begin(subtransactions=True) as trans: > > create() > > print(dict(trans._new)) > > fetch() > > print(dict(trans._new)) > > raise Exception("force transaction to roll back") > > except Exception: > > pass > > > > global fido > > assert not inspect(fido).persistent, "after rollback, should no longer > be in the database" > > > > if __name__ == '__main__': > > main() > > > > Which fails the last assertion: > > > > AssertionError: after rollback, should no longer be in the database > > > > I know that this is a minor issue, but I found it quite confusing while > debugging. > > > > I suspect that we might need to keep a strongly referenced list of “newly > added” PKs in the outer session, instead of using _new. > > > > Thanks, Chris. > > > > > > > > > ________________________________ > This email is confidential. If you are not the intended recipient, please > advise us immediately and delete this message. The registered name of > Cantab- part of GAM Systematic is Cantab Capital Partners LLP. See - > http://www.gam.com/en/Legal/Email+disclosures+EU for further information on > confidentiality, the risks of non-secure electronic communication, and > certain disclosures which we are required to make in accordance with > applicable legislation and regulations. If you cannot access this link, > please notify us by reply message and we will send the contents to you. > > GAM Holding AG and its subsidiaries (Cantab – GAM Systematic) will collect > and use information about you in the course of your interactions with us. > Full details about the data types we collect and what we use this for and > your related rights is set out in our online privacy policy at > https://www.gam.com/en/legal/privacy-policy. Please familiarise yourself > with this policy and check it from time to time for updates as it > supplements this notice > ________________________________ > > -- > SQLAlchemy - > The Python SQL Toolkit and Object Relational Mapper > > http://www.sqlalchemy.org/ > > To post example code, please provide an MCVE: Minimal, Complete, and > Verifiable Example. See http://stackoverflow.com/help/mcve for a full > description. > --- > You received this message because you are subscribed to the Google Groups > "sqlalchemy" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To post to this group, send email to [email protected]. > Visit this group at https://groups.google.com/group/sqlalchemy. > For more options, visit https://groups.google.com/d/optout. -- SQLAlchemy - The Python SQL Toolkit and Object Relational Mapper http://www.sqlalchemy.org/ To post example code, please provide an MCVE: Minimal, Complete, and Verifiable Example. See http://stackoverflow.com/help/mcve for a full description. --- You received this message because you are subscribed to the Google Groups "sqlalchemy" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at https://groups.google.com/group/sqlalchemy. For more options, visit https://groups.google.com/d/optout.
