#17096: Strengthen the makemessages command's safe-guarding of po files
-------------------------------------+-------------------------------------
Reporter: julien | Owner: nobody
Type: | Status: new
Cleanup/optimization | Version: 1.3
Component: Core | Keywords:
(Management commands) | Has patch: 0
Severity: Normal | Needs tests: 0
Triage Stage: | Easy pickings: 0
Unreviewed |
Needs documentation: 0 |
Patch needs improvement: 0 |
UI/UX: 0 |
-------------------------------------+-------------------------------------
It is common practice to set up a cron job to regularly run the
`makemessages` command in order to update the 'django.po' file with the
latest available translations. To limit the risk of data loss, the command
first creates a temporary file, but then it simply copies the resulting
content into the original file `[1]`. It seems there is still a risk to
run into a race condition where another process would update that file at
the same time and override the content initially populated by the
`makemessages` command, or vice versa.
I've discussed this with Alex Gaynor and Benjamin Peterson (Python core
dev) about using some file-locking or safe-guarding strategies. Quoting
Benjamin:
"On posix systems, you can simply use os.rename to replace translations
file with the temporary file; the operation is guaranteed to be
atomic, so any Django process accessing it will be guaranteed to see
either only the old translation file or the new one.
Windows is far more painful, since it has no guarantees of atomic
renaming. On Vista and after, there is a MoveFileTransacted. This has
the problem of not supporting older versions of Windows and not being
exposed directly by Python (though it is in pywin32). Usually then,
you have to do something in the writing process like creating a lock
file, writing to the main file, then deleting the lock file. The
reader then has to check to make sure there is no lock file before
commencing reading.
Unittesting is not so fun either. You can take the approach of
creating a bunch of readers and a bunch of writers and running them
against each other for a few seconds. You can also mock out the file
system calls and ensure that the writer and the reader and performing
operations in the correct sequence."
So, to strengthen the safe-guarding of po files, we could perhaps use a
combination of a) comparing the file's before&after modification date-
times in case it has simultaneously been modified by another process — if
the date-time is different then the `makemessages` process should stop
with a warning; and b) using `os.rename` to replace the destination file
by a temporary file instead of directly writing into the destination file.
If implementing the same support for Windows appears to be too hard, then
maybe we could leave that off and revert to the current system, since the
issue described here is most likely to occur in production environments
and I've personally never heard of live Django sites deployed on Windows-
based platforms.
`[1]`
https://code.djangoproject.com/browser/django/trunk/django/core/management/commands/makemessages.py?rev=16864#L297
--
Ticket URL: <https://code.djangoproject.com/ticket/17096>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
--
You received this message because you are subscribed to the Google Groups
"Django updates" group.
To post to this group, send email to [email protected].
To unsubscribe from this group, send email to
[email protected].
For more options, visit this group at
http://groups.google.com/group/django-updates?hl=en.
