Re: Feature request: post/pre commit/rollback signals

[Carl Meyer]

On 01/18/2014 01:19 PM, Anssi Kääriäinen wrote:
> On Saturday, January 18, 2014 9:02:41 PM UTC+2, Aymeric Augustin wrote:
> 
> On 18 janv. 2014, at 19:50, Carl Meyer  > wrote:
> 
> > Those interested in this feature may find django-transaction-hooks
> [1] useful. It is my attempt to implement the commonly-useful case
> (run some code after the current transaction successfully commits,
> providing the code wasn't registered during a savepoint which was
> later rolled back.) I think the API provided by
> django-transaction-hooks is actually better for that use case than
> transaction signals are, since it does the work of handling some
> situations correctly (savepoints, closed connections), which in the
> case of signals are left up to the signal receiver to handle (or
> more likely, not bother to handle.)
> >
> > Testing and feedback welcome. If this becomes widely used and
> seems to fill a need, I think there is a chance it could become part
> of Django core. (See #21803.)
> 
> I think this is the right approach. I would use that package. I
> support adding it to Django.
> 
> 
> If documentation warns clearly that there is no guarantee whatsoever
> that the hook is always ran after commit +1 for addition in Django. The
> most obvious case of abuse for this sort of hook is doing something
> business critical in the post-commit hook. It must be clear that this is
> not a substitute for two phase commit or other ways of avoiding
> crash-just-after-commit problem.

Right. I have this warning in the current docs for
django-transaction-hooks, though it should probably be more prominent.
I'll make sure it gets into the eventual Django patch, too.

Carl



signature.asc
Description: OpenPGP digital signature

Re: Feature request: post/pre commit/rollback signals

[Anssi Kääriäinen]

On Saturday, January 18, 2014 9:02:41 PM UTC+2, Aymeric Augustin wrote:
>
> On 18 janv. 2014, at 19:50, Carl Meyer  
> wrote: 
>
> > Those interested in this feature may find django-transaction-hooks [1] 
> useful. It is my attempt to implement the commonly-useful case (run some 
> code after the current transaction successfully commits, providing the code 
> wasn't registered during a savepoint which was later rolled back.) I think 
> the API provided by django-transaction-hooks is actually better for that 
> use case than transaction signals are, since it does the work of handling 
> some situations correctly (savepoints, closed connections), which in the 
> case of signals are left up to the signal receiver to handle (or more 
> likely, not bother to handle.) 
> > 
> > Testing and feedback welcome. If this becomes widely used and seems to 
> fill a need, I think there is a chance it could become part of Django core. 
> (See #21803.) 
>
> I think this is the right approach. I would use that package. I support 
> adding it to Django. 
>

If documentation warns clearly that there is no guarantee whatsoever that 
the hook is always ran after commit +1 for addition in Django. The most 
obvious case of abuse for this sort of hook is doing something business 
critical in the post-commit hook. It must be clear that this is not a 
substitute for two phase commit or other ways of avoiding 
crash-just-after-commit problem.

 - Anssi

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/574600c8-2aba-4ac6-bb19-f28e0a13a46d%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Feature request: post/pre commit/rollback signals

[Andrey Antukh]

Hi!

2014/1/18 Aymeric Augustin 

> On 18 janv. 2014, at 19:50, Carl Meyer  wrote:
>
> > Those interested in this feature may find django-transaction-hooks [1]
> useful. It is my attempt to implement the commonly-useful case (run some
> code after the current transaction successfully commits, providing the code
> wasn't registered during a savepoint which was later rolled back.) I think
> the API provided by django-transaction-hooks is actually better for that
> use case than transaction signals are, since it does the work of handling
> some situations correctly (savepoints, closed connections), which in the
> case of signals are left up to the signal receiver to handle (or more
> likely, not bother to handle.)
> >
> > Testing and feedback welcome. If this becomes widely used and seems to
> fill a need, I think there is a chance it could become part of Django core.
> (See #21803.)
>
> I think this is the right approach. I would use that package. I support
> adding it to Django.
>

You are absolutely right! This is much best approach than using signals (I
also don't like signals, my approach was only for show a use case).
And I think that this is much less intrusive (and without signals), and
would be awesome if it be added to django. This use case is very common in
web development.


> Since it doesn’t do anything until after the transaction has successfully
> committed, it cannot interfere with the transaction management code. That’s
> my most important requirement and the reason why I’m so strongly opposed to
> pre-commit/rollback signals.
>
> If you look at the first email in this thread, the use case is “do
> something when a transaction is committed successfully”. In the later email
> that gives an example, it only uses post_commit. Generic transaction
> signals look like a collective hallucination to me, or at least a pretty
> bad case of YAGNI.
>
> --
> Aymeric.


Thanks!

Greetings.
Andrey


-- 
Andrey Antukh - Андрей Антух -  / 
http://www.niwi.be 
https://github.com/niwibe

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAKn%3DmONTxxVA_HMHuMzxabL6zgG83f1NpUQyaoqV13LY5y%3DZ1A%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Feature request: post/pre commit/rollback signals

[Aymeric Augustin]

On 18 janv. 2014, at 19:50, Carl Meyer  wrote:

> Those interested in this feature may find django-transaction-hooks [1] 
> useful. It is my attempt to implement the commonly-useful case (run some code 
> after the current transaction successfully commits, providing the code wasn't 
> registered during a savepoint which was later rolled back.) I think the API 
> provided by django-transaction-hooks is actually better for that use case 
> than transaction signals are, since it does the work of handling some 
> situations correctly (savepoints, closed connections), which in the case of 
> signals are left up to the signal receiver to handle (or more likely, not 
> bother to handle.)
> 
> Testing and feedback welcome. If this becomes widely used and seems to fill a 
> need, I think there is a chance it could become part of Django core. (See 
> #21803.)

I think this is the right approach. I would use that package. I support adding 
it to Django.

Since it doesn’t do anything until after the transaction has successfully 
committed, it cannot interfere with the transaction management code. That’s my 
most important requirement and the reason why I’m so strongly opposed to 
pre-commit/rollback signals.

If you look at the first email in this thread, the use case is “do something 
when a transaction is committed successfully”. In the later email that gives an 
example, it only uses post_commit. Generic transaction signals look like a 
collective hallucination to me, or at least a pretty bad case of YAGNI.

-- 
Aymeric.



-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/A8A2614C-FF80-4978-9DE6-B85A41C9FE99%40polytechnique.org.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Feature request: post/pre commit/rollback signals

[Carl Meyer]

Hi,

Those interested in this feature may find django-transaction-hooks [1] 
useful. It is my attempt to implement the commonly-useful case (run some 
code after the current transaction successfully commits, providing the code 
wasn't registered during a savepoint which was later rolled back.) I think 
the API provided by django-transaction-hooks is actually better for that 
use case than transaction signals are, since it does the work of handling 
some situations correctly (savepoints, closed connections), which in the 
case of signals are left up to the signal receiver to handle (or more 
likely, not bother to handle.)

Testing and feedback welcome. If this becomes widely used and seems to fill 
a need, I think there is a chance it could become part of Django core. (See 
#21803.)

Carl

  [1] https://github.com/carljm/django-transaction-hooks

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/200b56e4-6577-451f-889d-d50678e77496%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Feature request: post/pre commit/rollback signals

[Andrey Antukh]

Thanks for your package and clear explanation ;)

Greetings.
Andrey.


2014/1/11 Aymeric Augustin 

> On 11 janv. 2014, at 11:05, Andrey Antukh  wrote:
>
> > In my opinion, "the default case are solved with atomic block" seems to
> be a workaround instead of a solution.
>
> As a short reminder, we’re talking about inconsistencies that arise when:
>
> 1) you do something in the database;
> 2) you do something related in another system — send an email, enqueue a
> task, cache a value;
> 3) the database transaction is rolled back, canceling the effects of 1)
> but not those of 2).
>
> I may have been unclear when I closed the ticket. I’m not saying that
> atomic blocks solve the problem. I’m saying that the problem doesn’t happen
> in autocommit mode, which is the default in Django >= 1.6.
>
> If you’re using transactions (i.e. ATOMIC_REQUESTS or atomic blocks), you
> may encounter the problem. But then I’m positive that transaction signals
> aren’t the solution.
>
> > I understand that signals in general are evil and register global
> callbacks is not a very good solution, but I think, that a orm should give
> some generic facility to attach code to execute when a transaction is
> successfully committed or rolled back.
>
> I’m not having this argument once more, so, there you go:
> https://github.com/aaugustin/django-transaction-signals
> https://pypi.python.org/pypi/django-transaction-signals-do-not-use
>
> You may also be interested in:
> https://github.com/davehughes/django-transaction-signals
> https://pypi.python.org/pypi/django_transaction_signals
>
> (The name conflict is unfortunate; I didn’t check before writing my
> package; at least that forced me to include a warning in the name.)
>
> Jokes aside, transaction signals are just a (wrong) means to an
> (legitimate) end. I’ve documented the solutions I’m aware of:
> https://github.com/aaugustin/django-transaction-signals#alternatives
>
> Additional ideas welcome!
>
> --
> Aymeric.
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at http://groups.google.com/group/django-developers.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/989FB6EA-12A2-4D54-9821-FB8473F3AF29%40polytechnique.org
> .
> For more options, visit https://groups.google.com/groups/opt_out.
>



-- 
Andrey Antukh - Андрей Антух -  / 
http://www.niwi.be 
https://github.com/niwibe

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAKn%3DmONL6KGBtgZR7fwUN7iLEmjhJa0xP23Pdq3MHHaHMcJ%2BTg%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Feature request: post/pre commit/rollback signals

[Aymeric Augustin]

On 11 janv. 2014, at 11:05, Andrey Antukh  wrote:

> In my opinion, "the default case are solved with atomic block" seems to be a 
> workaround instead of a solution.

As a short reminder, we’re talking about inconsistencies that arise when:

1) you do something in the database;
2) you do something related in another system — send an email, enqueue a task, 
cache a value;
3) the database transaction is rolled back, canceling the effects of 1) but not 
those of 2).

I may have been unclear when I closed the ticket. I’m not saying that atomic 
blocks solve the problem. I’m saying that the problem doesn’t happen in 
autocommit mode, which is the default in Django >= 1.6.

If you’re using transactions (i.e. ATOMIC_REQUESTS or atomic blocks), you may 
encounter the problem. But then I’m positive that transaction signals aren’t 
the solution. 

> I understand that signals in general are evil and register global callbacks 
> is not a very good solution, but I think, that a orm should give some generic 
> facility to attach code to execute when a transaction is successfully 
> committed or rolled back.

I’m not having this argument once more, so, there you go:
https://github.com/aaugustin/django-transaction-signals
https://pypi.python.org/pypi/django-transaction-signals-do-not-use

You may also be interested in:
https://github.com/davehughes/django-transaction-signals
https://pypi.python.org/pypi/django_transaction_signals

(The name conflict is unfortunate; I didn’t check before writing my package; at 
least that forced me to include a warning in the name.)

Jokes aside, transaction signals are just a (wrong) means to an (legitimate) 
end. I’ve documented the solutions I’m aware of:
https://github.com/aaugustin/django-transaction-signals#alternatives

Additional ideas welcome!

-- 
Aymeric.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/989FB6EA-12A2-4D54-9821-FB8473F3AF29%40polytechnique.org.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Feature request: post/pre commit/rollback signals

[Andrey Antukh]

Hi!

In my opinion, "the default case are solved with atomic block" seems to be
a workaround instead of a solution. I understand that signals in general
are evil and register global callbacks is not a very good solution, but I
think, that a orm should give some generic facility to attach code to
execute when a transaction is successfully committed or rolled back.

For example, one possible use case can be this:

import threading

from django.dispatch import receiver
from django.core.signals import request_started

from .signals import post_commit

_local = threading.local()
_local.post_commit_tasks = []

@receiver(post_commit, dispatch_uid="_run_defered_tasks")
def _run_defered_tasks(sender, **kwargs):
while _local.post_commit_tasks:
task = _local.post_commit_tasks.pop(0)
task[0](*task[1], **task[2])

@receiver(request_started, dispatch_uid="_clean_tasks")
def _clean_tasks(sender, **kwargs):
_local.post_commit_tasks = []

def delay_until_transaction(func, *args, **kwargs):
_local.post_commit_tasks.append((func, args, kwargs))

This is one of much other examples that try to solve this (problem/use
case) out of django, and I think that django/django-orm should provide some
facility to defer execution some code after a transaction is commited or
rolled back.

In the last aaugustin comment, he suggest one approach without using global
signals, and it seems  very interesting approach. Can these  approach to be
generalized and included in django?

As Jesus said, it not seems a isolated problem, and would be awesome if
django provides some generic and good way to do this.

Greetings.
Andrey.




2014/1/10 Andreas Pelme 

>
> On 10 jan 2014, at 09:19, Jesús Espino  wrote:
>
> > Hi folks!
> >
> > The propose is add post and pre signals for commit and rollback actions
> (as sqlalchemy orm events). This allows attach some code when a transaction
> is committed or rolled back.
> >
> > I have some problem, such as send email only when a transaction is
> committed successfully, and it's can be done with simple monkey patching
> the db backend. But it not appear to be an isolated problem, and would be
> awesome if these signals are included in core.
> >
> > If a purpose is accepted, I can take care of making the issue +
> pull-request.
> >
> > Greetings.
> > Jesus.
>
>
> Hi,
>
> This is a tricky issue that has been discussed earlier, you will probably
> want to check out ticket #14051:
>
> https://code.djangoproject.com/ticket/14051
>
> —
> Andreas Pelme
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at http://groups.google.com/group/django-developers.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/948D5F1E-F86D-44A0-9EEA-F1E9A6A4A382%40pelme.se
> .
> For more options, visit https://groups.google.com/groups/opt_out.
>



-- 
Andrey Antukh - Андрей Антух -  / 
http://www.niwi.be 
https://github.com/niwibe

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAKn%3DmOOq-dE8Nc9b604DnUkriRp2mhW90nx5nfc9PUuNyHn4dA%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Feature request: post/pre commit/rollback signals

[Andreas Pelme]

On 10 jan 2014, at 09:19, Jesús Espino  wrote:

> Hi folks!
> 
> The propose is add post and pre signals for commit and rollback actions (as 
> sqlalchemy orm events). This allows attach some code when a transaction is 
> committed or rolled back.
> 
> I have some problem, such as send email only when a transaction is committed 
> successfully, and it's can be done with simple monkey patching the db 
> backend. But it not appear to be an isolated problem, and would be awesome if 
> these signals are included in core.
> 
> If a purpose is accepted, I can take care of making the issue + pull-request.
> 
> Greetings.
> Jesus.


Hi,

This is a tricky issue that has been discussed earlier, you will probably want 
to check out ticket #14051:

https://code.djangoproject.com/ticket/14051

—
Andreas Pelme

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/948D5F1E-F86D-44A0-9EEA-F1E9A6A4A382%40pelme.se.
For more options, visit https://groups.google.com/groups/opt_out.

Feature request: post/pre commit/rollback signals

[Jesús Espino]

Hi folks!

The propose is add post and pre signals for commit and rollback actions (as
sqlalchemy orm events). This allows attach some code when a transaction is
committed or rolled back.

I have some problem, such as send email only when a transaction is
committed successfully, and it's can be done with simple monkey patching
the db backend. But it not appear to be an isolated problem, and would be
awesome if these signals are included in core.

If a purpose is accepted, I can take care of making the issue +
pull-request.

Greetings.
Jesus.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CALRMi60kK-u%2BVKmJbp4tJNYwYYukgDnEvEvs6o7zZx5tqAwaTw%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.