Re: [sqlalchemy] Looks like there is no event to catch "before" a rollback happens

[mike bayer]

On 03/11/2017 10:15 AM, mike bayer wrote:



On 03/10/2017 11:12 AM, Alessandro Molina wrote:



On Fri, Mar 10, 2017 at 3:40 PM, mike bayer > wrote:

If this is truly, "unexpected error but we need to do things",
perhaps you can use before_flush() to memoize the details you need
for a restore inside of session.info .

An event hook can be added but it would need to be carefully
considered what the specific use case for this hook is.   For
example I'm not sure "before_rollback()" is really what this should
be, it likely should be "on flush exception" similar to how engine
does it.


My specific need is related to https://github.com/amol-/depot/issues/36

DEPOT allows loading files associated to database data.
In case of a rollback DEPOT deletes the files that got uploaded.

That works in case of `.flush()` + `.rollback()` because it gathers the
history of the entity and the changed files in `before_flush`, but if a
rollback is issued without a flush it currently lacks an event from
which it can get the state of the objects and their history before the
rollback.



Looking at the code for how the events are called I would say that
moving the after_rollback() event call to be before the attributes are
reset might be more appropriate.   The after_commit() event works this
way, that is, inside of after_commit() things have not yet been expired.
  This would be an example of how critical it is that we consider really
deeply how these events are implemented.   Throwing them in quickly is
how these design mistakes occur.



propose after_rollback() emit before the snapshot is cleared at 
https://bitbucket.org/zzzeek/sqlalchemy/issues/3934/call-after_rollback-before-accounting. 
  If you can evaluate the patch at 
https://gerrit.sqlalchemy.org/#/c/334/ that would be helpful.  Note that 
you still cannot emit SQL in this event.












--
SQLAlchemy -
The Python SQL Toolkit and Object Relational Mapper

http://www.sqlalchemy.org/

To post example code, please provide an MCVE: Minimal, Complete, and
Verifiable Example. See http://stackoverflow.com/help/mcve for a full
description.
---
You received this message because you are subscribed to the Google
Groups "sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to sqlalchemy+unsubscr...@googlegroups.com
.
To post to this group, send email to sqlalchemy@googlegroups.com
.
Visit this group at https://groups.google.com/group/sqlalchemy.
For more options, visit https://groups.google.com/d/optout.


--
SQLAlchemy - 
The Python SQL Toolkit and Object Relational Mapper


http://www.sqlalchemy.org/

To post example code, please provide an MCVE: Minimal, Complete, and Verifiable 
Example.  See  http://stackoverflow.com/help/mcve for a full description.
--- 
You received this message because you are subscribed to the Google Groups "sqlalchemy" group.

To unsubscribe from this group and stop receiving emails from it, send an email 
to sqlalchemy+unsubscr...@googlegroups.com.
To post to this group, send email to sqlalchemy@googlegroups.com.
Visit this group at https://groups.google.com/group/sqlalchemy.
For more options, visit https://groups.google.com/d/optout.

Re: [sqlalchemy] Looks like there is no event to catch "before" a rollback happens

[mike bayer]

On 03/10/2017 11:12 AM, Alessandro Molina wrote:



On Fri, Mar 10, 2017 at 3:40 PM, mike bayer > wrote:

If this is truly, "unexpected error but we need to do things",
perhaps you can use before_flush() to memoize the details you need
for a restore inside of session.info .

An event hook can be added but it would need to be carefully
considered what the specific use case for this hook is.   For
example I'm not sure "before_rollback()" is really what this should
be, it likely should be "on flush exception" similar to how engine
does it.


My specific need is related to https://github.com/amol-/depot/issues/36

DEPOT allows loading files associated to database data.
In case of a rollback DEPOT deletes the files that got uploaded.

That works in case of `.flush()` + `.rollback()` because it gathers the
history of the entity and the changed files in `before_flush`, but if a
rollback is issued without a flush it currently lacks an event from
which it can get the state of the objects and their history before the
rollback.



Looking at the code for how the events are called I would say that 
moving the after_rollback() event call to be before the attributes are 
reset might be more appropriate.   The after_commit() event works this 
way, that is, inside of after_commit() things have not yet been expired. 
  This would be an example of how critical it is that we consider 
really deeply how these events are implemented.   Throwing them in 
quickly is how these design mistakes occur.









--
SQLAlchemy -
The Python SQL Toolkit and Object Relational Mapper

http://www.sqlalchemy.org/

To post example code, please provide an MCVE: Minimal, Complete, and
Verifiable Example. See http://stackoverflow.com/help/mcve for a full
description.
---
You received this message because you are subscribed to the Google
Groups "sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to sqlalchemy+unsubscr...@googlegroups.com
.
To post to this group, send email to sqlalchemy@googlegroups.com
.
Visit this group at https://groups.google.com/group/sqlalchemy.
For more options, visit https://groups.google.com/d/optout.


--
SQLAlchemy - 
The Python SQL Toolkit and Object Relational Mapper


http://www.sqlalchemy.org/

To post example code, please provide an MCVE: Minimal, Complete, and Verifiable 
Example.  See  http://stackoverflow.com/help/mcve for a full description.
--- 
You received this message because you are subscribed to the Google Groups "sqlalchemy" group.

To unsubscribe from this group and stop receiving emails from it, send an email 
to sqlalchemy+unsubscr...@googlegroups.com.
To post to this group, send email to sqlalchemy@googlegroups.com.
Visit this group at https://groups.google.com/group/sqlalchemy.
For more options, visit https://groups.google.com/d/optout.

Re: [sqlalchemy] Looks like there is no event to catch "before" a rollback happens

[Simon King]

On Fri, Mar 10, 2017 at 4:12 PM, Alessandro Molina
 wrote:
>
>
> On Fri, Mar 10, 2017 at 3:40 PM, mike bayer 
> wrote:
>>
>> If this is truly, "unexpected error but we need to do things", perhaps you
>> can use before_flush() to memoize the details you need for a restore inside
>> of session.info.
>>
>> An event hook can be added but it would need to be carefully considered
>> what the specific use case for this hook is.   For example I'm not sure
>> "before_rollback()" is really what this should be, it likely should be "on
>> flush exception" similar to how engine does it.
>
>
> My specific need is related to https://github.com/amol-/depot/issues/36
>
> DEPOT allows loading files associated to database data.
> In case of a rollback DEPOT deletes the files that got uploaded.
>
> That works in case of `.flush()` + `.rollback()` because it gathers the
> history of the entity and the changed files in `before_flush`, but if a
> rollback is issued without a flush it currently lacks an event from which it
> can get the state of the objects and their history before the rollback.
>

Could you collect the necessary data using the before_attach or
after_attach events, rather than before_flush?

Simon

-- 
SQLAlchemy - 
The Python SQL Toolkit and Object Relational Mapper

http://www.sqlalchemy.org/

To post example code, please provide an MCVE: Minimal, Complete, and Verifiable 
Example.  See  http://stackoverflow.com/help/mcve for a full description.
--- 
You received this message because you are subscribed to the Google Groups 
"sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sqlalchemy+unsubscr...@googlegroups.com.
To post to this group, send email to sqlalchemy@googlegroups.com.
Visit this group at https://groups.google.com/group/sqlalchemy.
For more options, visit https://groups.google.com/d/optout.

Re: [sqlalchemy] Looks like there is no event to catch "before" a rollback happens

[Alessandro Molina]

On Fri, Mar 10, 2017 at 3:40 PM, mike bayer 
wrote:

> If this is truly, "unexpected error but we need to do things", perhaps you
> can use before_flush() to memoize the details you need for a restore inside
> of session.info.
>
> An event hook can be added but it would need to be carefully considered
> what the specific use case for this hook is.   For example I'm not sure
> "before_rollback()" is really what this should be, it likely should be "on
> flush exception" similar to how engine does it.
>

My specific need is related to https://github.com/amol-/depot/issues/36

DEPOT allows loading files associated to database data.
In case of a rollback DEPOT deletes the files that got uploaded.

That works in case of `.flush()` + `.rollback()` because it gathers the
history of the entity and the changed files in `before_flush`, but if a
rollback is issued without a flush it currently lacks an event from which
it can get the state of the objects and their history before the rollback.

-- 
SQLAlchemy - 
The Python SQL Toolkit and Object Relational Mapper

http://www.sqlalchemy.org/

To post example code, please provide an MCVE: Minimal, Complete, and Verifiable 
Example.  See  http://stackoverflow.com/help/mcve for a full description.
--- 
You received this message because you are subscribed to the Google Groups 
"sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sqlalchemy+unsubscr...@googlegroups.com.
To post to this group, send email to sqlalchemy@googlegroups.com.
Visit this group at https://groups.google.com/group/sqlalchemy.
For more options, visit https://groups.google.com/d/optout.

Re: [sqlalchemy] Looks like there is no event to catch "before" a rollback happens

[mike bayer]

On 03/10/2017 01:57 AM, Alessandro Molina wrote:

I have been looking for  a way to know what's going to be rolled back in
SQLAlchemy so that I can know what was changed and restore other
database unrelated things to their previous state.

By http://docs.sqlalchemy.org/en/latest/orm/events.html#session-events
it looks like it's available an after_soft_rollback event, but in that
even the objects already got rolled back and so their history is gone.
In the most common scenario users do Session.flush() and then
Session.rollback(), and in that case I have before_flush that can tell
me everything that changed (than I can restore the state of related
things into after_soft_rollback which can benefit from the knowledge
that I gathered in before_flush) but if the user does a direct
Session.rollback() without a flush I couldn't find an event I could
attach to know what changed an so what is going to be rolled back.

Not sure if that can be achieved or a new event would be needed.


When an integrity error or something like that happens, often the 
database transaction is unusable anyway, and no further SELECT can be 
emitted.  See 
http://docs.sqlalchemy.org/en/latest/faq/sessions.html#but-why-does-flush-insist-on-issuing-a-rollback 
for detail.   An event that is added before this rollback occurs would 
provide an environment that can't work consistently because the database 
connection may or may not be usable depending on specifics.   So if I 
add that event, then I get the endless parade of "I can't do X in the 
before_rollback event but only when Q, P, R exist and I'm using backend Z!".


The idiomatic way to be able to rollback an operation but still continue 
to work with the data is to use savepoints.


If this is truly, "unexpected error but we need to do things", perhaps 
you can use before_flush() to memoize the details you need for a restore 
inside of session.info.


An event hook can be added but it would need to be carefully considered 
what the specific use case for this hook is.   For example I'm not sure 
"before_rollback()" is really what this should be, it likely should be 
"on flush exception" similar to how engine does it.






--
SQLAlchemy -
The Python SQL Toolkit and Object Relational Mapper

http://www.sqlalchemy.org/

To post example code, please provide an MCVE: Minimal, Complete, and
Verifiable Example. See http://stackoverflow.com/help/mcve for a full
description.
---
You received this message because you are subscribed to the Google
Groups "sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to sqlalchemy+unsubscr...@googlegroups.com
.
To post to this group, send email to sqlalchemy@googlegroups.com
.
Visit this group at https://groups.google.com/group/sqlalchemy.
For more options, visit https://groups.google.com/d/optout.


--
SQLAlchemy - 
The Python SQL Toolkit and Object Relational Mapper


http://www.sqlalchemy.org/

To post example code, please provide an MCVE: Minimal, Complete, and Verifiable 
Example.  See  http://stackoverflow.com/help/mcve for a full description.
--- 
You received this message because you are subscribed to the Google Groups "sqlalchemy" group.

To unsubscribe from this group and stop receiving emails from it, send an email 
to sqlalchemy+unsubscr...@googlegroups.com.
To post to this group, send email to sqlalchemy@googlegroups.com.
Visit this group at https://groups.google.com/group/sqlalchemy.
For more options, visit https://groups.google.com/d/optout.