Re: Consistent exception handling in templates.
Could we summarize in few words? | DEBUG (development) | not DEBUG (production) TEMPLATE_DEBUG | raise TemplateSyntaxError | ? not TEMPLATE_DEBUG| ? | ? On Sat, Jul 9, 2011 at 12:53 PM, Tai Lee wrote: > > > On Jul 9, 12:24 pm, Karen Tracey wrote: >> I'm strongly against that idea. Swallowing errors makes it incredibly >> difficult to debug errors. I would prefer for the `TEMPLATE_DEBUG` setting to >> turn off much of the error-silencing currently done in template processing. >> Yes, that means different behavior during development compared to >> production. But running with DEBUG (template or otherwise) on IS different >> from production, that's a given. And it this case it would allow for more >> effective development since errors wouldn't be hidden away in nearly >> impossible-to-find places. In my mind that's a tradeoff well worth making. >> >> See ticket #13167 for one example of where "template errors" have not been >> silenced since before 1.0. If that case is to be be "fixed" now, I would >> very much want that silencing to ONLY happen when TEMPLATE_DEBUG is False. >> In my opinion silencing these during development would be a huge step >> backwards. > > I think we are in agreement here. Swallowing errors makes it > incredible difficult to debug errors. I would like to see the > circumstances when exceptions are swallowed be more clearly defined, > and ensure that only exceptions that are expected to be swallowed, are > swallowed. If we can do that, then there should be no need for the > behaviour to change based on the `TEMPLATE_DEBUG` setting. > > The problem with the inconsistent behaviour associated with this > setting is exceptions that are expected to be swallowed, that we want > to be swallowed, are not being swallowed in some cases when > `TEMPLATE_DEBUG = True`. This can make it impossible to access some > URLs on a website where such a condition is present, even though there > is no actual error and the exception is expected to be silenced. It > also results in some exceptions being transformed into > `TemplateSyntaxError` sometimes, and the true exception raised other > times. > > See ticket #16147 for an example where `TemplateDoesNotExist` is > raised when the template DOES exist, because a template tag inside the > template being loaded raises `TemplateDoesNotExist` at compile time, > only when `TEMPLATE_DEBUG = True`. > > This can make it impractical to develop locally with `TEMPLATE_DEBUG = > True`, because it can prevent access to working pages, which means you > miss out on all the enhanced debugging that `TEMPLATE_DEBUG = True` > provides when you encounter an actual error that needs developer > attention. > > With the intent being that template authors should not be able to > trigger HTTP 500 errors on a production site, couldn't we achieve that > by silencing exceptions raised when an invalid argument was passed to > a template tag (e.g. `ValueError` or `TypeError` in the template tag > code), but allowing exceptions raised in the code for the object that > was passed in to the template tag as an argument? > > For example, if an object that calculates and returns a number when > evaluated is passed to `{% widthratio %}`, but there is a bug in the > code of that object and it raises some kind of exception when > evaluated, that should not be silenced. It should bubble up so that > the developer is alerted to a problem in the code for that object (not > an error that the template author has made by passing an invalid > argument). However, if there was no exception raised in evaluating the > object, but it simply returned something other than a number, then > that should behave the same as if an empty string or missing template > variable was passed in and no exception should be raised. > > I think that a `TemplateSyntaxError` should be raised at compile time > if an invalid template tag is used (unknown, too few or too many > arguments). The `render()` function for a template tag should silence > exceptions raised when arguments with the incorrect type or value are > provided, but should not silence exceptions raised when evaluating the > arguments themselves (or properties on the arguments). > > Cheers. > Tai. > > -- > You received this message because you are subscribed to the Google Groups "Django developers" group. > To post to this group, send email to django-developers@googlegroups.com. > To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. > For more options, visit this group at http://groups.google.com/group/django-developers?hl=en. > > -- Best regards, Yuri V. Baburov, Skype: yuri.baburov, MSN: bu...@live.com -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr.
Re: Consistent exception handling in templates.
On Jul 9, 12:24 pm, Karen Tracey wrote: > I'm strongly against that idea. Swallowing errors makes it incredibly > difficult to debug errors. I would prefer for the `TEMPLATE_DEBUG` setting to > turn off much of the error-silencing currently done in template processing. > Yes, that means different behavior during development compared to > production. But running with DEBUG (template or otherwise) on IS different > from production, that's a given. And it this case it would allow for more > effective development since errors wouldn't be hidden away in nearly > impossible-to-find places. In my mind that's a tradeoff well worth making. > > See ticket #13167 for one example of where "template errors" have not been > silenced since before 1.0. If that case is to be be "fixed" now, I would > very much want that silencing to ONLY happen when TEMPLATE_DEBUG is False. > In my opinion silencing these during development would be a huge step > backwards. I think we are in agreement here. Swallowing errors makes it incredible difficult to debug errors. I would like to see the circumstances when exceptions are swallowed be more clearly defined, and ensure that only exceptions that are expected to be swallowed, are swallowed. If we can do that, then there should be no need for the behaviour to change based on the `TEMPLATE_DEBUG` setting. The problem with the inconsistent behaviour associated with this setting is exceptions that are expected to be swallowed, that we want to be swallowed, are not being swallowed in some cases when `TEMPLATE_DEBUG = True`. This can make it impossible to access some URLs on a website where such a condition is present, even though there is no actual error and the exception is expected to be silenced. It also results in some exceptions being transformed into `TemplateSyntaxError` sometimes, and the true exception raised other times. See ticket #16147 for an example where `TemplateDoesNotExist` is raised when the template DOES exist, because a template tag inside the template being loaded raises `TemplateDoesNotExist` at compile time, only when `TEMPLATE_DEBUG = True`. This can make it impractical to develop locally with `TEMPLATE_DEBUG = True`, because it can prevent access to working pages, which means you miss out on all the enhanced debugging that `TEMPLATE_DEBUG = True` provides when you encounter an actual error that needs developer attention. With the intent being that template authors should not be able to trigger HTTP 500 errors on a production site, couldn't we achieve that by silencing exceptions raised when an invalid argument was passed to a template tag (e.g. `ValueError` or `TypeError` in the template tag code), but allowing exceptions raised in the code for the object that was passed in to the template tag as an argument? For example, if an object that calculates and returns a number when evaluated is passed to `{% widthratio %}`, but there is a bug in the code of that object and it raises some kind of exception when evaluated, that should not be silenced. It should bubble up so that the developer is alerted to a problem in the code for that object (not an error that the template author has made by passing an invalid argument). However, if there was no exception raised in evaluating the object, but it simply returned something other than a number, then that should behave the same as if an empty string or missing template variable was passed in and no exception should be raised. I think that a `TemplateSyntaxError` should be raised at compile time if an invalid template tag is used (unknown, too few or too many arguments). The `render()` function for a template tag should silence exceptions raised when arguments with the incorrect type or value are provided, but should not silence exceptions raised when evaluating the arguments themselves (or properties on the arguments). Cheers. Tai. -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Consistent exception handling in templates.
On Fri, Jul 8, 2011 at 9:55 PM, Tai Lee wrote: > Secondly, there is the inconsistent behaviour between production and > development environments related to the `TEMPLATE_DEBUG` setting. I > think that if an exception is intentionally silenced in production, it > should also be intentionally silenced with `TEMPLATE_DEBUG = True`, to > allow for consistent behaviour during development. > I'm strongly against that idea. Swallowing errors makes it incredibly difficult to debug errors. I would prefer for the TEMPLATE_DEBUG setting to turn off much of the error-silencing currently done in template processing. Yes, that means different behavior during development compared to production. But running with DEBUG (template or otherwise) on IS different from production, that's a given. And it this case it would allow for more effective development since errors wouldn't be hidden away in nearly impossible-to-find places. In my mind that's a tradeoff well worth making. See ticket #13167 for one example of where "template errors" have not been silenced since before 1.0. If that case is to be be "fixed" now, I would very much want that silencing to ONLY happen when TEMPLATE_DEBUG is False. In my opinion silencing these during development would be a huge step backwards. Karen -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Consistent exception handling in templates.
On Jul 8, 11:31 pm, Jacob Kaplan-Moss wrote: > > [...] but some of Django's own template tags (e.g. `widthratio`) don't > > adhere to > > this policy. > > These are bugs, and should be fixed. Thanks for this clarification. I will try to review the built-in tags and submit some patches to fix these bugs. For added clarity, should a template tag behave the same when a string or variable is passed as it's argument? E.g. if the first argument to a template tag is expected to be a number, we can validate that and raise a `TemplateSyntaxError` at compile time if it is passed in as a literal, but we can only validate it and fail silently at render time if it is passed in as a template variable. I believe that this behaviour is inconsistent and confusing, and can lead to unexpected and difficult to trace problems. To avoid this, should the validation of arguments to template tags that can be passed in as either literals or template variables be limited to checking that an argument was passed, and delay type or value checking until render time, which will always fail silently? > I'm against this, for a whole host of reasons — the crux of which > comes down to the philosophy that templates are *not* a programming > language, and introducing a try/except concept into them is about > seven steps too far. > > The point of the template language is to give front end designers and > developers the space to explore without the persnicketiness a > so-called "real" programming language introduces. What you're asking > for here is to fundamentally change the philosophy of the template > language. If you disagree that fundamentally with the ideas behind the > template language I really think you'll be better off finding another > one than trying to swim upstream. I agree that what I suggested would be a fundamental change, and not likely to be adopted in full. However, I would still like to hear other ideas from the community about smaller steps that we could take that would make exception handling more consistent and selective, without changing the fundamental philosophy of the template language. Firstly, Jonathan Slenders makes the point that perhaps not ALL exceptions should be silenced. Secondly, there is the inconsistent behaviour between production and development environments related to the `TEMPLATE_DEBUG` setting. I think that if an exception is intentionally silenced in production, it should also be intentionally silenced with `TEMPLATE_DEBUG = True`, to allow for consistent behaviour during development. If an exception IS worth raising when `TEMPLATE_DEBUG = True`, it should also raise in production, and these types of exceptions should be caused by an underlying problem in the template tag or the evaluation of an object passed to the template tag, rather than something that a template author is likely to trigger because they are not programmers. Does anybody else have ideas or suggestions about these two issues, or other small improvements we could make to exception handling in templates without fundamentally changing the philosophy of the template language? Cheers. Tai -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Consistent exception handling in templates.
On 07/08/2011 05:24 PM, Jonathan Slenders wrote: My feeling is that we should silence TypeError, AttributeError and KeyError in _resolve_lookup, but only when they occur one frame deep. I agree with what you propose but AFAIK it's not easy to solve without doing frame analysis (I'd say that feature is not worth doing such hacks). That's because of the way `hasattr` is implemented: def hasattr(obj, attr): try: getattr(obj, attr) return True except AttributeError: return False This code directly mirrors the code found in CPython and implies that /hasattr "executes" properties/. So there's no way to figure out whether an attribute exists without executing the code of properties. Jonas -- Django + MongoDB: http://django-mongodb.org -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Consistent exception handling in templates.
Something related I was wondering about... Errors in templates should be silenced -- I agree with that -- but only when they are caused by an error in the template. Variable.resolve can fail silent because that's caused by the template author, who is referring to non-existing variables. However, sometimes, it happens to me that the template calls a @property of some object, which acesses at his turn an non existing attribute of another object. An AttributeError is raised, but because that's somewhere in Variable.resolve, it's silently ignored. It's clear that at that point, it's not something in the template, but a hard-to-find bug in the @property. My feeling is that we should silence TypeError, AttributeError and KeyError in _resolve_lookup, but only when they occur one frame deep. Maybe I am wrong here, it's also just an opinion, and I have no real solution... (Except for writing better unit-tests for the application logic.) -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Consistent exception handling in templates.
On Thu, Jul 7, 2011 at 8:37 PM, Tai Lee wrote: > The docs say that when writing custom template tags, the `render()` > function should never raise `TemplateSyntaxError`, [...] This rule is the correct one, and I'm very much against changing it. > [...] but some of Django's own template tags (e.g. `widthratio`) don't adhere > to > this policy. These are bugs, and should be fixed. > I'd like to see a move towards consistent exception handling in > templates for template tags and filters. I'd like to see no exceptions > silenced by default, but provide a way for template authors to > conditionally silence exceptions in fragments of their templates. > Perhaps with a block tag. I'm against this, for a whole host of reasons — the crux of which comes down to the philosophy that templates are *not* a programming language, and introducing a try/except concept into them is about seven steps too far. The point of the template language is to give front end designers and developers the space to explore without the persnicketiness a so-called "real" programming language introduces. What you're asking for here is to fundamentally change the philosophy of the template language. If you disagree that fundamentally with the ideas behind the template language I really think you'll be better off finding another one than trying to swim upstream. > Should built-in template tags follow the advice given in the docs, and > never raise `TemplateSyntaxError` in their `render()` function? Yes; the two instances you mentioned are bugs and should be fixed. Jacob -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Consistent exception handling in templates.
I'd like to raise the topic of exception handling in templates, to see what people think of the current state and discuss possible improvements. I personally find it confusing to use and create template tags at times, due to exception handling. Sometimes exceptions are supposed to be silenced, and sometimes not. The docs say that when writing custom template tags, the `render()` function should never raise `TemplateSyntaxError`, but some of Django's own template tags (e.g. `widthratio`) don't adhere to this policy. I have also had difficulty with inconsistent behaviour between development and production environments, due to the `TEMPLATE_DEBUG` setting. For example, if `TEMPLATE_DEBUG` is `False`, the `include` template tag will fail silently when passed a string or context variable. If `TEMPLATE_DEBUG` is `True` (often the case on local development servers) it will raise an exception at compile time only when passed a literal string argument, otherwise it will raise an exception at render time (again, against the advice in the docs) when passed a context variable. In order to mimic production behaviour in a development environment, you must set `TEMPLATE_DEBUG` to `False`, and then you miss out on useful debug information. When it comes time to test, and you have a template tag that does happen to raise an unhandled exception (e.g. `ValueError`), it will be transformed into `TemplateSyntaxError` if `TEMPLATE_DEBUG` is `True`. So you need to test for either exception in your tests, or override the `TEMPLATE_DEBUG` setting so that you can expect one exception or the other, or add a try/except block to the `render()` function of your template tag and either raise `TemplateSyntaxError` all the time (against the advice of the docs) or silence all exceptions, with either option possibly hiding problems that exist elsewhere in the code (in a function called by the template tag). I'd like to see a move towards consistent exception handling in templates for template tags and filters. I'd like to see no exceptions silenced by default, but provide a way for template authors to conditionally silence exceptions in fragments of their templates. Perhaps with a block tag. This should allow for consistent behaviour, make it easier to spot errors during development, and give developers the option of silencing specific exceptions when they need to do something conditionally, e.g. include a template that may or may not exist. I imagine that this would be a huge backwards incompatible change. Are there any less severe alternative steps we could take to improve the situation generally? Would it be possible to make such a change by following a deprecation path? Should built-in template tags follow the advice given in the docs, and never raise `TemplateSyntaxError` in their `render()` function? Cheers. -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.