Re: Add signals for QuerySet bulk operations such as `delete`, `update, `bulk_create`
I guess this would not be too different from how the `m2m_changed` signal works. The arguments would (at a bare minimum) include the `sender`, `action`, and `using`, then depending on the `action` other arguments may be passed in. I would imagine if this approach would be taken, migrating existing signals over would follow the normal deprecation policy. But it could be introduced for the bulk-level operations initially without breaking any compatibility. On Monday, March 26, 2012 4:00:14 AM UTC-4, Anssi Kääriäinen wrote: > > On 03/26/2012 06:34 AM, Byron Ruth wrote: > > Sure I guess you _could_ take a generic signals approach to handling > > all modify operations, but for each operation context is important. > > The state of the object(s) matters when performing operations. The > > `pre_save` and `post_save` signals for example have a `created` flag > > which allows for filtering out the new objects versus existing ones. > > Likewise, having a distinct pair of delete signals ensures you don't > > make the mistake of performing operations on objects that are now > deleted. > > The pre/post_modify signal would have the context available. (parameters > action='save/delete/bulk_create/update' and action_args). It would be > kind of all the current signals compressed into one. If that is a good > design or not is a good question. I think it would be useful, but I > would not be surprised if other core developers have different opinion > of such a signal. > > The reason the generic signal would be useful is that more often than > not you are interested in _all_ the data modifying operations done to a > model, not just in the save or delete or update subset. Hence, one mount > point for all the operations. What is done in the signal could be very > different for the .update() case than for the .save() case, but on the > other hand there are cases where the generic signal handler would reduce > the amount of work. For search engine reindexing (Haystack for example), > you would just do reindex(modified_objects) for all cases except delete. > Similar for cache invalidation. So, such a signal could simplify some > common operations. > > Having signals for all data modifying operations is in my opinion > important. It is of course possible to achieve this without generic > signals by just adding pre/post update / bulk_create signals. The > bulk_create signal is problematic because you do not have the auto-pk > values available even in post_bulk_create signal. For PostgreSQL and > Oracle having the PKs would be possible using the "RETURNING ID" syntax. > SQLite could be hacked to support returning the IDs (there are no > concurrent transactions, hence you know what the IDs will be, or at > least I think this is the case). But for MySQL I do not know of any way > of returning the IDs. Except for locking the table for the duration of > the insert... > > - Anssi > > > -- You received this message because you are subscribed to the Google Groups "Django developers" group. To view this discussion on the web visit https://groups.google.com/d/msg/django-developers/-/16BAPV5h3FsJ. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Add signals for QuerySet bulk operations such as `delete`, `update, `bulk_create`
On 03/26/2012 06:34 AM, Byron Ruth wrote: Sure I guess you _could_ take a generic signals approach to handling all modify operations, but for each operation context is important. The state of the object(s) matters when performing operations. The `pre_save` and `post_save` signals for example have a `created` flag which allows for filtering out the new objects versus existing ones. Likewise, having a distinct pair of delete signals ensures you don't make the mistake of performing operations on objects that are now deleted. The pre/post_modify signal would have the context available. (parameters action='save/delete/bulk_create/update' and action_args). It would be kind of all the current signals compressed into one. If that is a good design or not is a good question. I think it would be useful, but I would not be surprised if other core developers have different opinion of such a signal. The reason the generic signal would be useful is that more often than not you are interested in _all_ the data modifying operations done to a model, not just in the save or delete or update subset. Hence, one mount point for all the operations. What is done in the signal could be very different for the .update() case than for the .save() case, but on the other hand there are cases where the generic signal handler would reduce the amount of work. For search engine reindexing (Haystack for example), you would just do reindex(modified_objects) for all cases except delete. Similar for cache invalidation. So, such a signal could simplify some common operations. Having signals for all data modifying operations is in my opinion important. It is of course possible to achieve this without generic signals by just adding pre/post update / bulk_create signals. The bulk_create signal is problematic because you do not have the auto-pk values available even in post_bulk_create signal. For PostgreSQL and Oracle having the PKs would be possible using the "RETURNING ID" syntax. SQLite could be hacked to support returning the IDs (there are no concurrent transactions, hence you know what the IDs will be, or at least I think this is the case). But for MySQL I do not know of any way of returning the IDs. Except for locking the table for the duration of the insert... - Anssi -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
RE: Add signals for QuerySet bulk operations such as `delete`, `update, `bulk_create`
A somewhat different proposal is in ticket #17824: Add generic pre/post_modify signal. I think the generic "object modified" signals would fit your use case very well. The idea is that there would be just one signal which would be fired for any data modifying ORM operation. The arguments for it: - fired for every operation modifying data - one signal to listen to for all data modifications The likely counter-arguments: - duplicates the existing signals - the callbacks end up being a big "switch statement", and thus you end up separating save, delete etc anyways. - the API isn't good enough >From performance perspective there should be no big problems: the signal is >given an iterable as "objs_modified" argument. For .update() for example, where you don't want to fetch all the objects for performance reasons, you could just pass qs.filter(update_filters) as the modified objects. This way there would be no performance penalty, except if there is actual use of the signal. I would like to see a generic pre/post modify signal, as I think it is much easier to use than using the pre/post save/delete + m2m_changed signals. However, I do not feel strongly at all about this, just something I would find useful. I believe having total control of all data modifying operations using Django signals would be a welcome addition for many users. - Anssi From: django-developers@googlegroups.com [django-developers@googlegroups.com] On Behalf Of Byron Ruth [bjr...@gmail.com] Sent: Sunday, March 25, 2012 17:46 To: django-developers@googlegroups.com Subject: Add signals for QuerySet bulk operations such as `delete`, `update, `bulk_create` My use case is for regenerating aggregate data cache at a table level. Simply calling a single signal after a bulk operation is complete would enable invalidating such aggregate cache. There is not a very clean alternate solution to this problem unless using database triggers which calls an external script that invalidates the cache. -- You received this message because you are subscribed to the Google Groups "Django developers" group. To view this discussion on the web visit https://groups.google.com/d/msg/django-developers/-/DAaTRIau8h8J. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en. -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Add signals for QuerySet bulk operations such as `delete`, `update, `bulk_create`
My use case is for regenerating aggregate data cache at a table level. Simply calling a single signal after a bulk operation is complete would enable invalidating such aggregate cache. There is not a very clean alternate solution to this problem unless using database triggers which calls an external script that invalidates the cache. -- You received this message because you are subscribed to the Google Groups "Django developers" group. To view this discussion on the web visit https://groups.google.com/d/msg/django-developers/-/DAaTRIau8h8J. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
