On Thursday, December 8, 2011 at 7:33 PM, Russell Keith-Magee wrote: > On Thu, Dec 8, 2011 at 8:42 PM, Sam Berry <samkbe...@googlemail.com > (mailto:samkbe...@googlemail.com)> wrote: > > Hello there, > > > > I am currently running a number of small sites using logging via email > > to notify me of 404s and server errors. The volume of email produced > > is perfectly manageable due to the low level of traffic. > > > > > First off, I'd point out that the 404 emails aren't sent every time > your server generates a 404 -- they're only generated as a result of a > 404 *from an internal link on your site*. If John Q Hacker hits > http://yoursite.com/this_page_doesnt_exist, you won't get a 404 email. > >
This isn't exactly accurate. It sends them for any 404 response where the HTTP_REFERER header is not in the ignorable urls setting, is not empty, and does not contain a ? This is also trivial to get around and if someone is attempting to trigger a boatload of 404 emails this check is only going to prevent someone who knows nothing about Django (in which case how do they know about 404 email sending?). > > If you're getting *any* 404 emails, it means there are internal links > on your site that don't resolve. If this is something that is > happening systematically, at a rate that has given you cause for > concern about the potential for DOS, I'd suggest you have a bigger > problem with your site. > > > It does occur to me that it would be very easy to disrupt the site by > > simply sending repeated requests which trigger a 404 Not Found. I'm > > sure 10 404s a second would completely shutdown the process to normal > > requests due to the time taken to send the log emails. I'm not using a > > message queueing system. > > > > I would like to keep the option of receiving logging emails. Is there > > anything I can do to throttle or limit the email sending without > > writing my own logging handler? > > > > > Interestingly, 404 emails aren't handled by a logging handler (unlike > 500 messages) -- they're handled as a direct mail to site managers. > This is something that is probably worth a ticket in itself; using the > logging framework for 404 mails would allow for much more flexibility. > > Agreed, 404 should be configurable via Logging. > > So - what you want here isn't a custom logging handler, but a custom > *mail* handler -- django-mailer [1] is a good candidate here. This > makes "sending" an email a very short lived operation from the > perspective of your web server -- all the web request does is queue > the email for sending. The actual sending of mail is handled out of > the request-response cycle. This removes the possibility of "DOS > attack by 404" because the expensive operation is taken out of the > request-response cycle. > > This is True. You could also move email sending to a celery task or something similar. > > Of course, you could still end up being flooded with email -- so you > either need to (a) monitor the size of your mail queue to make sure it > isn't getting flooded, or (b) put a processor in place to > merge/throttle the contents of the mail queue. This could also be > handled with a custom mail handler; however, I can't point you at a > ready-to-use candidate for this. > > Another option is to use a mechanism other than the 404 emails to log > and handle errors. Django-sentry [2] and Arecibo [3] are two > candidates here, both of which are easy to install, and provide much > richer analytics than a full mailbox :-) > > This is, in my opinion, the best option if you wish to track 404's. > > [1] https://github.com/jtauber/django-mailer/ > [2] https://github.com/dcramer/django-sentry/ > [3] http://areciboapp.com/ > > Yours, > Russ Magee %-) > > -- > You received this message because you are subscribed to the Google Groups > "Django users" group. > To post to this group, send email to django-users@googlegroups.com > (mailto:django-users@googlegroups.com). > To unsubscribe from this group, send email to > django-users+unsubscr...@googlegroups.com > (mailto:django-users+unsubscr...@googlegroups.com). > For more options, visit this group at > http://groups.google.com/group/django-users?hl=en. > > -- You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com. To unsubscribe from this group, send email to django-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-users?hl=en.
