Definitely a +1 from me.
-- Horst
On Thu, Sep 17, 2009 at 10:25 AM, Simon Willison
<si...@simonwillison.net> wrote:
>
> I think we should add logging to Django in version 1.2, implemented as
> a light-weight wrapper around the Python logging module
> (django.core.log maybe?) plus code to write errors to the Apache error
> log under the mod_python handler and environ['wsgi.errors'] under WSGI
> (meaning mod_wsgi will write to the Apache error log as well).
>
> Benefits of logging as a core Django service
> ============================================
>
> Adding logging to Django core would provide the following benefits:
>
> 1. We'll be able to de-emphasise the current default "e-mail all
> errors to someone" behaviour, which doesn't scale at all well.
>
> 2. Having logging in the core framework will mean people start
> actually using it, which will make it easier for people to debug their
> Django apps. Right now adding "print" statements to a Django app is a
> common debugging technique, but it's messy (you have to remember to
> take them out again) and error prone - some production environments
> throw errors if an app attempts to write to stdout. It's also not
> obvious - many developers are surprised when I show them the
> technique.
>
> 3. Logging in Django core rather than a 3rd party app will encourage
> reusable applications to log things in a predictable way, standard
> way.
>
> 4. 3rd party debugging tools such as the debug toolbar will be able to
> hook in to Django's default logging behaviour. This could also lead to
> plenty of additional 3rd party innovation - imagine a tool that looks
> out for logged SQL that took longer than X seconds, or one that groups
> together similar log messages, or streams log messages to IRC...
>
> 5. Built-in support for logging reflects a growing reality of modern
> Web development: more and more sites have interfaces with external web
> service APIs, meaning there are plenty of things that could go wrong
> that are outside the control of the developer. Failing gracefully and
> logging what happened is the best way to deal with 3rd party problems
> - much better than throwing a 500 and leaving no record of what went
> wrong.
>
> 6. Most importantly from my point of view, when a sysadmin asks where
> Django logs errors in production we'll have a good answer for them!
>
> 7. As a general rule, I believe you can never have too much
> information about what's going on with your web application. I've
> never thought to myself "the problem with this bug is I've got too
> much information about it". As for large log files, disk space is
> cheap - and pluggable backends could ensure logs were sensibly
> rotated.
>
> Places logging would be useful
> ==============================
>
> - Unhandled exceptions that make it up to the top of the Django stack
> (and would cause a 500 error to be returned in production)
> - The development web server could use logging for showing processed
> requests (where currently these are just printed to stdout).
> - Failed attempts at signing in to the admin could be logged, making
> security audits easier.
> - We could replace (or complement) django.connection.queries with a
> log of executed SQL. This would make the answer to the common question
> "how do I see what SQL is being executed" much more obvious.
> - Stuff that loads things from INSTALLED_APPS could log what is being
> loaded, making it much easier to spot and debug errors caused by code
> being incorrectly loaded.
> - Likewise, the template engine could log which templates are being
> loaded from where, making it easier to debug problems stemming from an
> incorrectly configured TEMPLATE_DIRS setting.
> - We could use logging to address the problems with the template
> engine failing silently - maybe some template errors (the ones more
> likely to be accidental than just people relying on the fail-silent
> behaviour deliberately) should be logged as warnings.
>
> Most of the above would be set to a low log level which by default
> would not be handled, displayed or stored anywhere (logging.info or
> similar). Maybe "./manage.py runserver --loglevel=info" could cause
> such logs to be printed to the terminal while the development server
> is running.
>
> Problems and challenges
> =======================
>
> 1. The Python logging module isn't very nicely designed - its Java
> heritage shines through, and things like logging.basicConfig behave in
> unintuitive ways (if you call basicConfig twice the second call fails
> silently but has no effect). This is why I suggest wrapping it in our
> own higher level interface.
>
> 2. There may be some performance overhead, especially if we replace
> mechanisms like django.connection.queries with logging. This should be
> negligble: here's a simple benchmark:
>
> # ("hello " * 100) gives a 600 char string, long enough for a SQL
> statement
>>>> import timeit, logging
>>>> t = timeit.Timer('logging.info("hello " * 100)', 'import logging')
>>>> t.timeit(number=100) # one hundred statements
> 0.00061702728271484375
>>>> t.timeit(number=1000000) # one million statements
> 6.458014965057373
>
> That's 0.0006 of a second overhead for a page logging 100 SQL
> statements. The performance overhead will go up if you attach a
> handler, but that's fine - the whole point of a framework like
> 'logging' is that you can log as much as you like but only act on
> messages above a certain logging level.
>
> 3. We risk people using logging where signals would be more
> appropriate.
>
> 4. We might go too far, and make Django a "noisy" piece of software
> which logs almost everything that happens within it. Let's be tasteful
> about this.
>
> 5. People might leave logging on, then find their server disk has
> filled up with log files and caused their site to crash.
>
> 6. Logging levels are confusing - what exactly is the difference
> between warn, info, error, debug, critical and fatal? We would need to
> document this and make decisions on which ones get used for what
> within the framework.
>
> What would it look like?
> ========================
>
> Here's what I'm thinking at the moment (having given the API very
> little thought). In your application code:
>
> from django.core import log
> # Log to the default channel:
> log.debug('Retrieving RSS feed from %s' % url)
> # Log to a channel specific to your app:
> log.debug('Retrieving RSS feed from %s' % url, channel='myapp.rss')
> try:
> feed = httpfetch(url, timeout=3)
> except socket.timeout:
> log.info('Timeout fetching feed %s' % url)
>
> In settings.py:
>
> MIDDLEWARE_CLASSES = (
> ...
> 'django.middleware.LogErrorsToWSGI', # write exceptions to
> wsgi.errors
> )
> LOGGING_MIDDLEWARE_LEVEL = 'info'
>
> # If you want custom log handlers - not sure how these would interact
> with
> # channels and log levels yet
> LOG_HANDLERS = (
> 'django.core.log.handlers.LogToDatabase',
> 'django.core.log.handlers.LogToEmail',
> )
>
> What do people think? I'd be happy to flesh this out in to a full spec
> with running code.
>
> >
>
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/django-developers?hl=en
-~----------~----~----~----~------~----~------~--~---
