Hi all, I've had experience playing around with watchdog[1] before - I've only ever used it on OS X but it should work cross platform. I've thrown together a rough proof of concept[2] for runserver that uses watchdog to watch for changes/additions/deletions to python files recursively in the same directory as manage.py and restarts the server accordingly. It could easily be extended to handle translations.
To avoid changing a lot of code I've just set it up to use watchdog if available and fall back to the current method silently if not. I'd be happy to turn this into a more complete solution if people think it's the right route to go. We can choose to: - make watchdog required - it's a simple pip install - make watchdog required if you want runserver to autoreload - fall back to the current implementation if watchdog is not on the python path (and potentially tell the user about watchdog) Do you think it's worth carrying on down this path? Sam [1] https://pythonhosted.org/watchdog/ [2] https://github.com/django/django/compare/master...sdcooke:watchdog (ignore the code quality - it was thrown together quickly!) On Sun, 9 Aug 2015 at 22:22 Daniel Moisset <[email protected]> wrote: > The idea sounds awesome. A couple of corner cases that should be taken in > consideration: > > - I think it is common for small simple projects (and people doing the > tutorial) to have the sqlite db file in the same project directory. If > we're watching for writes in every file in the project directory, every > save() could trigger a reload > - It is also usual to have the media/ directory based on filesystem > storage inside the project dir. So any project with file uploads could > trigger a reload on each file upload. > > Possibly some exclusion rules with sane defaults could help here > > Regards, > > D. > > > > On Sat, Aug 8, 2015 at 6:53 PM, Aymeric Augustin < > [email protected]> wrote: > >> Hello, >> >> While writing some horrific code [1] related to the development server’s >> auto-reloading, I realized that its design is fundamentally flawed and I >> think that we should reconsider it. >> >> The current implementation walks through the list of all imported Python >> modules, attempts to figure out which file they come from and builds a list >> of these files. Then it watches all these files for changes by checking if >> their mtime changed every second or, since 1.7, with inotify on Linux. I >> tried to do it with kqueue on BSD (read: OS X) but I failed. This stuff is >> hard. >> >> In addition to the source of imported Python modules, the autoreloader >> watches .mo files in order to catch translation changes and source files >> for modules that failed to import — it extracts them from exception >> tracebacks. This shows the limits of basing the list on imported Python >> modules. Even with such hacks, the auto-reloader will never handle all >> cases. Two examples: >> >> - It doesn’t survive a syntax error in the settings module. I have >> reasons to believe that this would be extremely messy to fix. >> - If a module reads a configuration file on disk at startup and caches >> it, the auto-reloader doesn’t detect changes to that file. >> >> So we’re stuck with cases where the development server doesn’t reload. I >> can’t say they’re edge cases; who never made a typo in a settings file? >> People who run tutorials report that it’s a very common cause of >> frustration for beginners. >> >> It's also wasteful. There’s little need to watch every file of every >> dependency in the virtualenv for changes. >> >> I think it would be much better to remove the auto-reloading logic from >> Django and have an external process watch the current working directory for >> changes. (When virtualenv is considered appropriate for people with exactly >> 0 days of Django experience under their belt, I think we can require a >> dependency for the development server’s auto-reloading.) >> >> The behavior would be much easier to describe: if you change something in >> your source tree, it reloads automatically; if you change something >> somewhere else, you must reload manually. It would also be quite easy to >> customize with a standard include / exclude mechanism to select which files >> should be watched. >> >> We aren’t in the business of writing production web servers, neither are >> we in the business of writing a filesystem watcher :-) Watchman [2] appears >> to be a robust solution. However I would prefer a pure-Python >> implementation that could easily be installed in a virtualenv. Does someone >> have experience in this area? >> >> I’m not making a concrete proposal as I haven’t studied the details. It >> this point I’d just like to hear what you think of the general concept. >> >> Thanks! >> >> -- >> Aymeric. >> >> [1] https://github.com/django/django/pull/5106 >> [2] https://github.com/facebook/watchman >> [3] >> http://tutorial.djangogirls.org/en/django_installation/index.html#virtual-environment >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Django developers (Contributions to Django itself)" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> To post to this group, send email to [email protected]. >> Visit this group at http://groups.google.com/group/django-developers. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/django-developers/8E08AB59-2450-40D5-A52B-C8803DA39D5F%40polytechnique.org >> <https://groups.google.com/d/msgid/django-developers/8E08AB59-2450-40D5-A52B-C8803DA39D5F%40polytechnique.org?utm_medium=email&utm_source=footer> >> . >> For more options, visit https://groups.google.com/d/optout. >> > > > > -- > Daniel F. Moisset - Technical Leader > www.machinalis.com > Skype: @dmoisset > > -- > You received this message because you are subscribed to the Google Groups > "Django developers (Contributions to Django itself)" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To post to this group, send email to [email protected]. > Visit this group at http://groups.google.com/group/django-developers. > To view this discussion on the web visit > https://groups.google.com/d/msgid/django-developers/CALuYSZX1X8D3BNO9Mt_DuHHA34FTO2PM%3DUMYw87ueeGLnbnk_A%40mail.gmail.com > <https://groups.google.com/d/msgid/django-developers/CALuYSZX1X8D3BNO9Mt_DuHHA34FTO2PM%3DUMYw87ueeGLnbnk_A%40mail.gmail.com?utm_medium=email&utm_source=footer> > . > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/django-developers. To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CAMyx3A098k2%3DBJSXPsOg-54KV8QZRZf7XiGmruuzH5nZLKfjHg%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
