Looks like this issue isn't affecting lots of folks, since nobody is piping up, so I'll just add that if anybody arrives here in the future, we'll be tracking this in a public issue here:
https://github.com/freelawproject/courtlistener/issues/3359 My theory is it has something to do with race conditions/timings and async, since I *think* it started around the time we started switching to async, but others on the team tell me that's a red herring (it's not the cause). Anyhow, if anybody ever arrives here in the future, we fixed it <https://github.com/grossir/courtlistener/commit/91b38620fcc837be5f848f0f594d7ae968439a1f> by overriding the save method to catch IntegrityErrors. It's pretty lame to have code like this in our save method, but it seems to be working: try: # Without a transaction wrapper, a failure will invalidate outer transactions with transaction.atomic(): super(Docket, self).save( update_fields=update_fields, *args, **kwargs ) except IntegrityError: # Temporary patch while we solve #3359 # If the error is not related to `date_modified` it will raise again self.date_modified = timezone.now() super(Docket, self).save( update_fields=update_fields, *args, **kwargs ) Anyhow, thanks for reading, everybody. On Monday, March 18, 2024 at 7:08:22 AM UTC-7 Matthew Pava wrote: > Hi Bill, > > We ended up using a package called Django-audit-log: > https://pypi.org/project/django-audit-log/. > > It’s outdated now, but we used the source code for their > CreatingUserField, LastUserField, CreationDateTimeField, and > ModificationDateTimeField. More modern packages may have enhanced features > than these. > > > > In your case, I may just add a save method to the abstract model and add > timezone.now() to the corresponding fields. If there’s not a pk, then > populate the created date field. > > > > You are right, though. Perhaps the Django community can explain the issue > between auto_now and auto_now_add fields. > > > > *From:* 'William Palin' via Django developers (Contributions to Django > itself) <django-d...@googlegroups.com> > *Sent:* Sunday, March 17, 2024 9:59 AM > *To:* Django developers (Contributions to Django itself) < > django-d...@googlegroups.com> > *Subject:* Intermittent IntegrityError on Model Save with auto_now and > auto_now_add Fields > > > > Hello Django community, > > > > We are reaching out after encountering a persistent and elusive issue that > manifests as an IntegrityError during the save operation of a Django model. > This problem has been sporadically occurring since last year and has > successfully stumped four seasoned Django developers on our team. The error > seems to involve the non-update of auto_now fields upon model save, leading > us to suspect a deeper issue within Django, though we are cautious about > drawing premature conclusions. Given the complex and intermittent nature of > this bug, we are seeking insights, advice, or any form of guidance from the > community. > > > > *Issue Overview:* > > > > Our model inherits date_created and date_modified fields from an abstract > base class designed to standardize these timestamps across our models. Here > is how the abstract base class is defined: > > > > > > class AbstractDateTimeModel(models.Model): > > """An abstract base class for most models""" > > > > date_created = models.DateTimeField( > > help_text="The moment when the item was created.", > > auto_now_add=True, > > db_index=True, > > ) > > date_modified = models.DateTimeField( > > help_text="The last moment when the item was modified. A value in > year" > > " 1750 indicates the value is unknown", > > auto_now=True, > > db_index=True, > > ) > > > > class Meta: > > abstract = True > > > > *The Problem:* > > > > Intermittently, the .save() method triggers an IntegrityError, apparently > because the auto_now field (date_modified) does not get created. A specific > instance of this error showed that while date_created is correctly > populated, date_modified remains null, which directly leads to the > IntegrityError upon attempting to insert the record: > > > > [datetime.datetime(2023, 10, 2, 14, 33, 49, 99833, > tzinfo=datetime.timezone.utc), None, ...] > > 'INSERT INTO "search_docket" ("date_created", "date_modified",... ] > > > > *What We've Tried:* > > > > - Investigated the possibility of an issue with update_fields being > non-null and excluding date_modified during .save(), but confirmed through > Sentry logs that update_fields was indeed None in all instances of the > error. > - Attempted to reproduce the issue in a controlled environment without > success, leaving us without a clear direction for a solution. > > > > *Request for Help:* > > > > We're wondering if this could point to an undocumented edge case in > Django's auto_now and auto_now_add implementation or a specific database > behavior under certain conditions. Any advice on further debugging steps, > experiences with similar issues, or knowledge of potential Django nuances > that we might not be considering would be incredibly valuable. > > > > We appreciate your time and any feedback or suggestions you can offer. > > > > Thanks > > > > Bill > > > > django v5.0.2 > > python 3.12.2 > > db is postgres > > > > -- > You received this message because you are subscribed to the Google Groups > "Django developers (Contributions to Django itself)" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to django-develop...@googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/django-developers/a788c065-3a96-472c-9b41-7c2aebca6967n%40googlegroups.com > > <https://groups.google.com/d/msgid/django-developers/a788c065-3a96-472c-9b41-7c2aebca6967n%40googlegroups.com?utm_medium=email&utm_source=footer> > . > -- You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/5e4a7fa2-2770-44f0-a09b-395cce933c54n%40googlegroups.com.
