dewey <[email protected]> wrote:
> While the docs are pretty good at describing flush dependencies, side-effects > and how to issue, it, there is not a lot of clarity around what all it > actually does. For a deep dive, if the ORM tutorial isn’t doing it for you (from what I can tell, your questions are all answered within it) i recommend my video “The SQLAlchemy Session - In Depth” for more insight: http://www.sqlalchemy.org/library.html#thesqlalchemysessionindepth > > I can surmise that it pushes all non-saved session state to the DB > (committing or not depending upon whether a transaction is currently in > process), but it's not clear if it auto-loads any created PKeys for those > rows. Take a look at the ORM tutorial under “Adding New Objects”: http://docs.sqlalchemy.org/en/rel_0_9/orm/tutorial.html#adding-new-objects This explains that primary key values are in fact made available after the flush where you see "If we look at Ed’s id attribute, which earlier was None, it now has a value:”. > I also believe it leaves those objects in the identity map (rather than > expiring them) for further use, but it does not state this explicitly. Expiration and identity map are two separate concepts. Keeping in mind that I’ve just put up new links for these sections as of yesterday (larger pages have been broken up), expiration is described here: http://docs.sqlalchemy.org/en/rel_0_9/orm/session_state_management.html#refreshing-expiring and does not involve removing those objects from the identity map. There is an auto-expiration on commit, described here: http://docs.sqlalchemy.org/en/rel_0_9/orm/session_basics.html#committing, which is also configurable. The "adding new objects” section again makes a good explanation of when something is in the identity map and when it’s not. > > Whether or not you agree the docs could be clarified about such points, > here's a summary of what i'm trying to do: > > We have much data loaded in batch to many tables. Any given batch is tied to > one specific time window. We keep a "time-window-id" on every row, in every > table. This allows us to delete and reload all data related to a given > time-batch at any future point. You can think of it as a persisted > transaction id. We want this "time-window" record created AND ON DISK > before adding a lot of other data to the session. This wish is driven a bit > by paranoia, but also by the fact that we're manually controlling (not > letting the ORM do it) FKey relationships in a few special circumstances. So > we need that ID in memory for records in which the ORM is not tracking the > relationships. > > Is "flush" the right way to make sure this record is persisted and reloaded > NEAR the beginning of our batch transaction? When you say “and on disk”, there’s a little bit of interpretation there. With relational databases, we’re thinking in terms of transactions, so when you emit some SQL within a transaction, but that transaction is not committed, I’m not sure whether one would consider the data to be “on disk” or not. The data may technically be on the disk but this is somewhat subject to how the DB works. The guarantee of durability is typically looking at after the transaction is committed. So If you are just looking for the data to be committed, then you’d call session.commit(). If the auto-expiration at that point is getting in the way, turn it off with expire_on_commit=False. If OTOH you are looking for the fact that the SQL has been emitted and the database is aware of the rows at within an ongoing transaction, then you only need to call flush, you can keep the transaction open. > > Thanks for all tips! > D > > > > > -- > You received this message because you are subscribed to the Google Groups > "sqlalchemy" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected] > <mailto:[email protected]>. > To post to this group, send email to [email protected] > <mailto:[email protected]>. > Visit this group at http://groups.google.com/group/sqlalchemy > <http://groups.google.com/group/sqlalchemy>. > For more options, visit https://groups.google.com/d/optout > <https://groups.google.com/d/optout>. -- You received this message because you are subscribed to the Google Groups "sqlalchemy" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/sqlalchemy. For more options, visit https://groups.google.com/d/optout.
