Re: Add custom autoreload file tracking options setting

2017-01-05 Thread Florian Apolloner


On Thursday, January 5, 2017 at 11:14:08 PM UTC+1, Josh Smeaton wrote:
>
> > I am -0 to -1 for the debugger -- I've seen to many sites out there 
> running with DEBUG=True, enabling RCE ootb seems to be pretty horrible.
>
> But it's so incredibly useful. And we already show the django debug page 
> for errors with DEBUG=True that exposes enough secrets to allow a 
> sufficient attacker to gain access.
>

What exactly? Last time I checked SECRET_KEY and other dangerous stuff was 
blanked out as good as possible. Do not get me wrong, it is certainly not 
"safe" to show the debug page, but leaking information versus RCE is a 
different story… Even if the debug page leaks enough information to login 
as admin, you do not neccessarily compromise the OS, whereas the werkzeug 
debugger gives you at least user access to the OS.

And truth to be told, I can count the instance where the werkzeug debugger 
would have been useful on one hand -- the traces are usually more than 
enough.
 

> If we could, by default, block the debugger in a similar way that django 
> debug toolbar does, would that be appropriate? That is, checks for DEBUG 
> and HOST etc?
>

I am not deploying debug toolbar anywhere, so I cannot tell. That said, 
people have DEBUG=True etc in prod, if we open those installations for RCE, 
that would still suck (defense in depth and everything). Also 
https://labs.detectify.com/2015/10/02/how-patreon-got-hacked-publicly-exposed-werkzeug-debugger/
 
-- even though it is old, it is a nice sign that it happens…

Cheers,
Florian

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/91eb120a-e777-4bbe-ad63-345d6983cfba%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Re: Add custom autoreload file tracking options setting

2017-01-05 Thread Josh Smeaton
> I am -0 to -1 for the debugger -- I've seen to many sites out there 
running with DEBUG=True, enabling RCE ootb seems to be pretty horrible.

But it's so incredibly useful. And we already show the django debug page 
for errors with DEBUG=True that exposes enough secrets to allow a 
sufficient attacker to gain access. If we could, by default, block the 
debugger in a similar way that django debug toolbar does, would that be 
appropriate? That is, checks for DEBUG and HOST etc?

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/65ab0096-d7e9-4817-8fa2-0d2296f97671%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Re: Django 2.0 Python version support (Python 3.6+ only?)

2017-01-05 Thread Florian Apolloner
Hi Asif,

On Thursday, January 5, 2017 at 9:10:40 PM UTC+1, Asif Saifuddin wrote:
>
> django 2.0 will be released in december 2017 and ubuntu 18.04 will be 
> released in april 2018 which will default atleast 3.6, so I think this 
> should also be taken as consideration while deciding.
>

What comes out __after__ the release of Django is the least of my concerns. 
The more important thing here are the still supported LTS releases and how 
disruptive that change would be. OS releases after Django's release are in 
general not an issue (aside from new bugs here and there, but that is 
nothing anyone can predict).

Cheers,
Florian

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/4374e0c9-83bf-4bee-ac4a-9beae00fbfec%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Re: Django 2.0 Python version support (Python 3.6+ only?)

2017-01-05 Thread Asif Saifuddin
Hi,

django 2.0 will be released in december 2017 and ubuntu 18.04 will be 
released in april 2018 which will default atleast 3.6, so I think this 
should also be taken as consideration while deciding.

Thanks 

On Wednesday, January 4, 2017 at 1:00:00 AM UTC+6, Tim Graham wrote:
>
> August 2016: PyPy gets funding from Mozilla for Python 3.5 support
> "Within the next year, we plan to use the money to pay four core PyPy 
> developers half-time to work on the missing features and on some of the big 
> performance and cpyext issues. This should speed up the progress of 
> catching up with Python 3.x significantly. "
>
> https://morepypy.blogspot.com/2016/08/pypy-gets-funding-from-mozilla-for.html
>
> According to http://pypy.org/py3donate.html, it seems that anyone who 
> cares can donate to the effort of porting PyPy to Python 3. Django 
> 1.11/Python 2 will be supported until 2020 anyway.
>
> On Tuesday, January 3, 2017 at 1:04:02 PM UTC-5, Florian Apolloner wrote:
>
>> Mhm, just thought about the fact that this means we are also dropping 
>> support for PyPy and Jython -- not sure about the Jyton usage, but loosing 
>> PyPy sounds sad, how far along are there python 3 efforts? It looks like it 
>> is/was close to 3.3 according to 
>> https://morepypy.blogspot.co.at/2016/08/pypy-gets-funding-from-mozilla-for.html
>>
>> Cheers,
>> Florian
>>
>> On Tuesday, December 27, 2016 at 11:03:22 PM UTC+1, Tim Graham wrote:
>>>
>>> Yes, Django 1.11 is the last version to support Python 2.7. This is 
>>> documented in the 1.11 release notes, in 
>>> https://www.djangoproject.com/download/#supported-versions, and 
>>> elsewhere. 
>>>
>>> On Tuesday, December 27, 2016 at 4:37:06 PM UTC-5, MMeent wrote:

 I won't mind dropping support for Python versions that are not 
 supported up to the end of the support period of the next LTS (2.2 in this 
 case). If you want to use long-term stability and/or support for current 
 Python versions, you should use the current django LTS version, which will 
 be 1.11. I am perfectly fine with django dropping support for a python 
 version that won't be supported for over 1 1/2 years of that (major) 
 versions support cycle.

 Noting that python 2.x also has an EOL in 2020, this one being half a 
 year earlier (March 16th vs September 13th), will django 2.0 drop 
 python 2.7 support, or will the 2.x series continue support for 2.7? I 
 cant 
 really find definite docs on that. 
 (https://www.djangoproject.com/weblog/2015/jun/25/roadmap/ talks about 
 it but is not completely clear)

 If django drops 2.7 for django 2.x, a lot of code will probably be 
 reworked, and seeing the 3.6 features I would love to see those available 
 directly while removing/refactoring the compat-layer. e.g. f-strings 
 instead of "{}".format or %-formatting, as it is less prone to random 
 bugs like https://code.djangoproject.com/ticket/6343 .


 -Matthias

 On 27 Dec 2016 21:25, "Florian Apolloner"  wrote:

> Imo we should not drop Python versions overeagerly. After all I do not 
> wanna compile our own python for djangoproject.com :D Given that 
> Redhat is on Python 3.4 for the foreseeable future, I'd actually even 
> like 
> to see 3.4 still supported in Django 2.0 unless there is a good reason to 
> drop it. Fwiw, Ubuntu Trusty which is LTS and still supported also is on 
> Python 3.4. So unless there are compelling arguments to drop 3.4, lets 
> keep 
> it as long as it is not too much work.
>
> Either way, I am completely against dropping Python 3.5 now -- lets 
> make the Django 2.0 migration not more painful than it has to be (ie I do 
> not want to force people to upgrade existing supported systems just to 
> get 
> the latest python and therefor Django).
>
> Cheers,
> Florian
>
> On Tuesday, December 27, 2016 at 4:12:57 PM UTC+1, Tim Graham wrote:
>>
>> When I drafted the 1.11 release notes in May, I wrote, "The next 
>> major release, Django 2.0, will only support Python 3.5+."
>>
>> Our Python version support policy is "Typically, we will support a 
>> Python version up to and including the first Django LTS release whose 
>> security support ends after security support for that version of Python 
>> ends."
>>
>> Python 3.5's EOL is September 2020 which I think is sufficiently 
>> close to Django 1.11's EOL of April 2020 that we could say Django 2.0 is 
>> Python 3.6+. The alternative is not to drop Python 3.5 compatibility 
>> until 
>> Django 2.2 LTS which is supported until April 2022. I don't see much 
>> advantage to that. Any objections?
>>
>> p.s. There is already a ticket suggesting to take advantage of a 
>> Python 3.6 feature:
>> https://code.djangoproject.com/ticket/27635* - *django.utils.crypto 
>> 

Re: Template handling of undefined variables

2017-01-05 Thread Tim Martin


On Thursday, 5 January 2017 04:15:31 UTC, Carl Meyer wrote:
>
> Hi Tim, 
>
> On 01/04/2017 03:39 PM, Tim Martin wrote: 
>  
> > 1) There are test cases where we have templates that should treat "x 
> >is y" as True where x and y are both undefined. 
>
> Really? Is this an accidental side effect of some historical change, or 
> really the intent of the test? That behavior seems buggy to me; more 
> likely to be a bug-magnet than useful. 
>

There are a couple of test cases in test_if.py: 
test_if_is_both_variables_missing and 
test_if_is_not_both_variables_missing. These are verifying this specific 
case, so they haven't been introduced by accident. I haven't got as far as 
figuring out whether it was a fully thought through decision or whether 
someone just created these cases for completeness.
 

> ... 
> >I don't see any way to satisfy both these requirements. Is it 
> >reasonable to relax the requirement that "x is y" should be treated 
> >as True when both variables are undefined? 
>
> Seems reasonable to me. 
>
> > 2) There appears to be an inconsistency in the default_if_none 
> >modifier.
>
[...] 

> >I think this is just a bug, does anyone think otherwise? 
>
> I agree. 
>

OK. I think I'll open this as a separate bug and solve that first.

Tim 

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/4f1a359c-909b-48a7-b854-273b77066b19%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Re: Methodology for increasing the number of PBKDF2 iterations

2017-01-05 Thread Martin Koistinen
Slightly off-topic, this presents a really nice case for switching to 
Argon2 via argon2_cffi (supported in Django 1.10+). Its super fast (C-lib) 
and resistant to GPU/ASIC brute-forcing. So, where as an attacker's 8-GPU 
hashing machine would probably have something on the order of 24,000X more 
hashing capability for SHA256 than a typical Django server, I estimate that 
the same hardware (8 GPUs) would only have about 20-30X more hashing 
capability than a typical server. (Note, the anecdotal evidence across the 
internet supporting this is pretty thin).


On Wednesday, January 4, 2017 at 2:13:09 PM UTC-5, Martin Koistinen wrote:
>
> I think this is a pretty solid guess. Bear in mind this was a direct 
> install from Python.org.
>
> The important thing here is, this demonstrates that we cannot just assume 
> that all Python 3 installs have a "fast" PBKDF2 implementation =/
>
> On Wednesday, January 4, 2017 at 11:33:17 AM UTC-5, Tobias McNulty wrote:
>
>> ... 
>>
> Martin, is it possible your version of Python 3 is not linked against 
>> OpenSSL and hence is missing the fast version of pbkdf2_hmac? I haven't had 
>> a chance to try your benchmark yet, but in a quick test I don't see any 
>> difference between Python 3.5.2 and Python 2.7.12 on a Mac.
>>
>> Tobias
>>
>
>  
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/7a1b926b-d75c-4f45-b3be-d5d8b7b8a7e5%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Re: Add custom autoreload file tracking options setting

2017-01-05 Thread Stratos Moros
I’m -0 on the change. I could move to +0 if I understood why the use 
case described here requires watching additional files.


A different use case we've run into is non-python configuration files. 
Our settings.py reads a few variables off a toml file and it would be 
nice if we could configure runserver to reload automatically when that 
toml file changes.


That said, I agree that auto reloading should probably live outside 
Django.


--
You received this message because you are subscribed to the Google Groups "Django 
developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/D8E34EDB-0BF8-4D78-A567-FB0EE372E631%40gmail.com.
For more options, visit https://groups.google.com/d/optout.


Re: Changing {% include %} to strip a trailing newline

2017-01-05 Thread Alasdair Nicol
Would adding the future include tag to the builtins in the template OPTIONS 
work? If it did, then you'd only have to make the change to the TEMPLATES 
setting instead of every template.

OPTIONS={
'builtins': ['django.template.future_include'],  # put new include 
in it's own module so that we don't add anything else in future.
}

I'm not familiar with that code, so I don't know whether that would 
override the old include tag. 

Cheers,
Alasdair

On Thursday, 5 January 2017 14:49:58 UTC, Tim Graham wrote:
>
> Hi Simon, I mentioned that idea offhand in an earlier mail, however, I 
> feel that it would be really disruptive to force all projects to go through 
> that deprecation and require adding {% load include from future %} to every 
> template that uses the tag to silence warnings (and then remove those lines 
> sometime after the deprecation ends). (More disruptive than just making a 
> backwards-incompatible change that's likely to affect a (hopefully small) 
> subset of users.) I guess I'm open to it if others think that's the way to 
> go though.
>
> On Thursday, January 5, 2017 at 9:20:36 AM UTC-5, charettes wrote:
>>
>> Hey Tim,
>>
>> I just wanted to mention that another approach for a transition plan 
>> could be
>> to use {% load include from future %} like we did with the {% url %} tag.
>>
>> That doesn't answer the question of whether or not we should make the 
>> change
>> but I think it offers a better upgrade path than a global template option 
>> or
>> a keep_trailing_newline kwarg.
>>
>> Simon
>>
>> Le mercredi 4 janvier 2017 14:58:42 UTC-5, Tim Graham a écrit :
>>>
>>> Shortly after template widget rendering was merged, an issue about extra 
>>> newlines appearing in the rendered output was reported [0]:
>>>
>>> For example, from django-money:
>>>
>>> UIC-Franc
>>> \n\n
>>> US Dollar
>>> \n\n
>>>
>>> The newlines aren't useful and they break assertions like this:
>>>
>>> US Dollar in form.as_p
>>>
>>> --- end report---
>>>
>>> The reporter suggested removing the trailing newline in the attrs.html 
>>> template but Adam Johnson reported: "POSIX states that all text files 
>>> should end with a newline, and some tools break when operating on text 
>>> files missing the final newline. That's why git has the warning \ No 
>>> newline at end of file and Github has a warning symbol for the missing 
>>> newline."
>>>
>>> I suggested that perhaps {% include %} could do .rstrip('\n') on 
>>> whatever it renders.
>>>
>>> Preston pointed out that Jinja2 does something similar:
>>>
>>> http://jinja.pocoo.org/docs/dev/templates/#whitespace-control
>>>
>>>- a single trailing newline is stripped if present
>>>- other whitespace (spaces, tabs, newlines etc.) is returned 
>>>unchanged
>>>
>>> I noted that the issue of {% include %} adding extra newlines was raised 
>>> in #9198 [1] but Malcolm said, "For backwards compatibility reasons we 
>>> cannot change the default behaviour now (there will be templates that rely 
>>> on the newline being inserted)." I was skeptical this would be a burdensome 
>>> change.
>>>
>>> Carl replied:  "It seems quite likely to me that there are templates in 
>>> the wild relying on preservation of the final newline. People render 
>>> preformatted text (e.g. text emails) using DTL. I probably have some text 
>>> email templates in older projects myself that would break with that change. 
>>>
>>> We could add a new option to {% include %}, though." Adam also said, "I 
>>> too have used DTL for text emails that would break under that behaviour. 
>>> New option to include sounds good to me."
>>>
>>>
>>> Me again: In the long run, having {% include %} remove the trailing 
>>> newline seems like a more sensible default. For example, I wouldn't expect 
>>> this code to have a newline inserted between it:
>>>
>>>
>>> {% include "foo.txt" %}
>>> {% include "bar.txt" %}
>>>  
>>>
>>> An option for {% include %} seems superfluous given that if you were 
>>> writing
>>>
>>>
>>> {% include "foo.txt" %}{% include "bar.txt" %}
>>>  
>>>
>>> now you can write the first thing which is much more intuitive.
>>>
>>>
>>> How about a keep_trailing_newline TEMPLATES option for backwards 
>>> compatibility for those who don't want to adapt their templates for the new 
>>> behavior? Jinja2 has that option.
>>>
>>> Carl replied: An engine option may be better than an option to {% 
>>> include %}, though it doesn't allow us to ensure that we strip the 
>>> newline in the specific case of attrs.
>>>
>>> How we default the engine option I guess just depends on how seriously 
>>> we take backwards compatibility. If we default it to strip and make people 
>>> add the config to preserve the old behavior, that's not really backwards 
>>> compatible. Historically (as seen in Malcolm's comment) we would choose to 
>>> err on the side of actual backwards compatibility in a case like this, even 
>>> if it didn't result in the ideal future behavior. But the adaptation isn't 
>>> 

Re: Changing {% include %} to strip a trailing newline

2017-01-05 Thread Tim Graham
Hi Simon, I mentioned that idea offhand in an earlier mail, however, I feel 
that it would be really disruptive to force all projects to go through that 
deprecation and require adding {% load include from future %} to every 
template that uses the tag to silence warnings (and then remove those lines 
sometime after the deprecation ends). (More disruptive than just making a 
backwards-incompatible change that's likely to affect a (hopefully small) 
subset of users.) I guess I'm open to it if others think that's the way to 
go though.

On Thursday, January 5, 2017 at 9:20:36 AM UTC-5, charettes wrote:
>
> Hey Tim,
>
> I just wanted to mention that another approach for a transition plan could 
> be
> to use {% load include from future %} like we did with the {% url %} tag.
>
> That doesn't answer the question of whether or not we should make the 
> change
> but I think it offers a better upgrade path than a global template option 
> or
> a keep_trailing_newline kwarg.
>
> Simon
>
> Le mercredi 4 janvier 2017 14:58:42 UTC-5, Tim Graham a écrit :
>>
>> Shortly after template widget rendering was merged, an issue about extra 
>> newlines appearing in the rendered output was reported [0]:
>>
>> For example, from django-money:
>>
>> UIC-Franc
>> \n\n
>> US Dollar
>> \n\n
>>
>> The newlines aren't useful and they break assertions like this:
>>
>> US Dollar in form.as_p
>>
>> --- end report---
>>
>> The reporter suggested removing the trailing newline in the attrs.html 
>> template but Adam Johnson reported: "POSIX states that all text files 
>> should end with a newline, and some tools break when operating on text 
>> files missing the final newline. That's why git has the warning \ No 
>> newline at end of file and Github has a warning symbol for the missing 
>> newline."
>>
>> I suggested that perhaps {% include %} could do .rstrip('\n') on 
>> whatever it renders.
>>
>> Preston pointed out that Jinja2 does something similar:
>>
>> http://jinja.pocoo.org/docs/dev/templates/#whitespace-control
>>
>>- a single trailing newline is stripped if present
>>- other whitespace (spaces, tabs, newlines etc.) is returned unchanged
>>
>> I noted that the issue of {% include %} adding extra newlines was raised 
>> in #9198 [1] but Malcolm said, "For backwards compatibility reasons we 
>> cannot change the default behaviour now (there will be templates that rely 
>> on the newline being inserted)." I was skeptical this would be a burdensome 
>> change.
>>
>> Carl replied:  "It seems quite likely to me that there are templates in 
>> the wild relying on preservation of the final newline. People render 
>> preformatted text (e.g. text emails) using DTL. I probably have some text 
>> email templates in older projects myself that would break with that change. 
>>
>> We could add a new option to {% include %}, though." Adam also said, "I 
>> too have used DTL for text emails that would break under that behaviour. 
>> New option to include sounds good to me."
>>
>>
>> Me again: In the long run, having {% include %} remove the trailing 
>> newline seems like a more sensible default. For example, I wouldn't expect 
>> this code to have a newline inserted between it:
>>
>>
>> {% include "foo.txt" %}
>> {% include "bar.txt" %}
>>  
>>
>> An option for {% include %} seems superfluous given that if you were 
>> writing
>>
>>
>> {% include "foo.txt" %}{% include "bar.txt" %}
>>  
>>
>> now you can write the first thing which is much more intuitive.
>>
>>
>> How about a keep_trailing_newline TEMPLATES option for backwards 
>> compatibility for those who don't want to adapt their templates for the new 
>> behavior? Jinja2 has that option.
>>
>> Carl replied: An engine option may be better than an option to {% 
>> include %}, though it doesn't allow us to ensure that we strip the 
>> newline in the specific case of attrs.
>>
>> How we default the engine option I guess just depends on how seriously we 
>> take backwards compatibility. If we default it to strip and make people add 
>> the config to preserve the old behavior, that's not really backwards 
>> compatible. Historically (as seen in Malcolm's comment) we would choose to 
>> err on the side of actual backwards compatibility in a case like this, even 
>> if it didn't result in the ideal future behavior. But the adaptation isn't 
>> hard in this case, so I won't object if the choice is to break back-compat.
>>
>>
>> If it's not a per-include choice, of course, we have to break overall 
>> back-compat to preserve form-attr-rendering back-compat.
>> 
>>
>> What do you think?
>>
>> [0] https://github.com/django/django/pull/7769
>> [1] https://code.djangoproject.com/ticket/9198
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send 

Re: Add custom autoreload file tracking options setting

2017-01-05 Thread Michael Manfre
On Thu, Jan 5, 2017 at 8:06 AM Florian Apolloner 
wrote:

>
>
> On Thursday, January 5, 2017 at 1:38:44 PM UTC+1, Sam Willis wrote:
>
> Could one options be to replace the current devserver with the one from
> Werkzeug? It already uses watchdog (similar to watchman) for monitoring
> file system events and is well maintained. With Django now allowing
> dependancies, this seems like something that doesn't necessarily need to be
> developed internally.
>
>
> Watchdog seems to require a .c extension file on MacOSX at least, so as
> long as there are no wheels, that does not seem to be an option.
>
> The Werkzeug devserver also has some niceties like an interactive debugger
> and ssl hosting with an automatically issued self signed certificate. It
> could be implemented behind the current management command
>
>
> Self-Signed SSL sounds somewhat nice, I am -0 to -1 for the debugger --
> I've seen to many sites out there running with DEBUG=True, enabling RCE
> ootb seems to be pretty horrible.
>

I think anyone running devserver in prod deserves a breakpoint induced
outage.

Regards,
Michael Manfre

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAGdCwBsshYVoNS04Rii2Xm1qi6J4mQPaBrNja%3DGwLVhQQ4RqNw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


Re: Changing {% include %} to strip a trailing newline

2017-01-05 Thread charettes
Hey Tim,

I just wanted to mention that another approach for a transition plan could 
be
to use {% load include from future %} like we did with the {% url %} tag.

That doesn't answer the question of whether or not we should make the change
but I think it offers a better upgrade path than a global template option or
a keep_trailing_newline kwarg.

Simon

Le mercredi 4 janvier 2017 14:58:42 UTC-5, Tim Graham a écrit :
>
> Shortly after template widget rendering was merged, an issue about extra 
> newlines appearing in the rendered output was reported [0]:
>
> For example, from django-money:
>
> UIC-Franc
> \n\n
> US Dollar
> \n\n
>
> The newlines aren't useful and they break assertions like this:
>
> US Dollar in form.as_p
>
> --- end report---
>
> The reporter suggested removing the trailing newline in the attrs.html 
> template but Adam Johnson reported: "POSIX states that all text files 
> should end with a newline, and some tools break when operating on text 
> files missing the final newline. That's why git has the warning \ No 
> newline at end of file and Github has a warning symbol for the missing 
> newline."
>
> I suggested that perhaps {% include %} could do .rstrip('\n') on whatever 
> it renders.
>
> Preston pointed out that Jinja2 does something similar:
>
> http://jinja.pocoo.org/docs/dev/templates/#whitespace-control
>
>- a single trailing newline is stripped if present
>- other whitespace (spaces, tabs, newlines etc.) is returned unchanged
>
> I noted that the issue of {% include %} adding extra newlines was raised 
> in #9198 [1] but Malcolm said, "For backwards compatibility reasons we 
> cannot change the default behaviour now (there will be templates that rely 
> on the newline being inserted)." I was skeptical this would be a burdensome 
> change.
>
> Carl replied:  "It seems quite likely to me that there are templates in 
> the wild relying on preservation of the final newline. People render 
> preformatted text (e.g. text emails) using DTL. I probably have some text 
> email templates in older projects myself that would break with that change. 
>
> We could add a new option to {% include %}, though." Adam also said, "I 
> too have used DTL for text emails that would break under that behaviour. 
> New option to include sounds good to me."
>
>
> Me again: In the long run, having {% include %} remove the trailing 
> newline seems like a more sensible default. For example, I wouldn't expect 
> this code to have a newline inserted between it:
>
>
> {% include "foo.txt" %}
> {% include "bar.txt" %}
>  
>
> An option for {% include %} seems superfluous given that if you were 
> writing
>
>
> {% include "foo.txt" %}{% include "bar.txt" %}
>  
>
> now you can write the first thing which is much more intuitive.
>
>
> How about a keep_trailing_newline TEMPLATES option for backwards 
> compatibility for those who don't want to adapt their templates for the new 
> behavior? Jinja2 has that option.
>
> Carl replied: An engine option may be better than an option to {% include 
> %}, though it doesn't allow us to ensure that we strip the newline in the 
> specific case of attrs.
>
> How we default the engine option I guess just depends on how seriously we 
> take backwards compatibility. If we default it to strip and make people add 
> the config to preserve the old behavior, that's not really backwards 
> compatible. Historically (as seen in Malcolm's comment) we would choose to 
> err on the side of actual backwards compatibility in a case like this, even 
> if it didn't result in the ideal future behavior. But the adaptation isn't 
> hard in this case, so I won't object if the choice is to break back-compat.
>
>
> If it's not a per-include choice, of course, we have to break overall 
> back-compat to preserve form-attr-rendering back-compat.
> 
>
> What do you think?
>
> [0] https://github.com/django/django/pull/7769
> [1] https://code.djangoproject.com/ticket/9198
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/0dff767b-6a27-45d3-a055-25bbebad9f18%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Re: Add custom autoreload file tracking options setting

2017-01-05 Thread Florian Apolloner


On Thursday, January 5, 2017 at 1:38:44 PM UTC+1, Sam Willis wrote:
>
> Could one options be to replace the current devserver with the one from 
> Werkzeug? It already uses watchdog (similar to watchman) for monitoring 
> file system events and is well maintained. With Django now allowing 
> dependancies, this seems like something that doesn't necessarily need to be 
> developed internally.
>

Watchdog seems to require a .c extension file on MacOSX at least, so as 
long as there are no wheels, that does not seem to be an option.

The Werkzeug devserver also has some niceties like an interactive debugger 
> and ssl hosting with an automatically issued self signed certificate. It 
> could be implemented behind the current management command 
>

Self-Signed SSL sounds somewhat nice, I am -0 to -1 for the debugger -- 
I've seen to many sites out there running with DEBUG=True, enabling RCE 
ootb seems to be pretty horrible.

Cheers,
Florian

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/93d787ad-dd98-4ae5-8251-8eddd4b97d3d%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Re: Add custom autoreload file tracking options setting

2017-01-05 Thread Sam Willis
Could one options be to replace the current devserver with the one from 
Werkzeug? It already uses watchdog (similar to watchman) for monitoring 
file system events and is well maintained. With Django now allowing 
dependancies, this seems like something that doesn't necessarily need to be 
developed internally.

The Werkzeug devserver also has some niceties like an interactive debugger 
and ssl hosting with an automatically issued self signed certificate. It 
could be implemented behind the current management command relatively 
easily.

There is already an implementation as part of django-extentions that I 
believe is well used and battle tested 
- http://django-extensions.readthedocs.io/en/latest/runserver_plus.html 


On Thursday, January 5, 2017 at 9:11:53 AM UTC, Aymeric Augustin wrote:
>
> On 4 Jan 2017, at 23:31, Bobby Mozumder  
> wrote: 
>
> > I guess I could just use Watchman to restart the Django development 
> server as needed? 
>
>
> If you find a way to tell watchman to run `django-admin runserver 
> --noreload` and restart it whenever a file in the current directory 
> changes, that should do the job. 
>
> Unfortunately the APIs exposed by watchman don’t make this trivial. 
> They’re intended to run a build process that will terminate, while the 
> development server will keep running. 
>
> -- 
> Aymeric. 
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/6d346a13-b096-42f8-a15b-a2814a81eef5%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Re: django-formtools is neglected/unmaintained

2017-01-05 Thread Romain Garrigues
Hi guys,

I'm currently investing some efforts to clean and make some django packages 
up-to-date.
After django-dirtyfields (https://github.com/romgar/django-dirtyfields), 
I'm now beginning to work on django-model-utils 
(https://github.com/carljm/django-model-utils) where there is some work to 
update it, but after this, and if nobody else had enough time to have a 
look, I can try to contribute on django-formtools.

Romain.

Le mercredi 4 janvier 2017 23:20:40 UTC, eric@datalyticsolutions.com a 
écrit :
>
> Thanks a lot, Tim! If you think I could help, then let me know. Otherwise, 
> I'll just stay out of your way :)
>
> On Wednesday, January 4, 2017 at 4:02:29 PM UTC-7, Tim Graham wrote:
>>
>> According to https://github.com/django/django-formtools/issues/75, 
>> there's a change in master that's needed for 1.10 compatibility.
>>
>> I'll try to do a release if I can get access to the PyPI record. I pinged 
>> the owner (jezdez) in #django-dev about it.
>>
>> On Wednesday, January 4, 2017 at 5:51:56 PM UTC-5, Adam Johnson wrote:
>>>
>>> 1) Tim added testing on Django 1.10 in July 2016, it seems to work? 
>>> https://github.com/django/django-formtools/commits/master
>>> 2) New contributors are always welcome
>>> 3) I don't know of other packages, you can check 
>>> https://djangopackages.org/
>>> 4) It's true that many websites are built with pure JS frontends these 
>>> days, there are probably more recently developed tools there
>>>
>>> On 4 January 2017 at 22:42,  wrote:
>>>
 Hi Tim,

 You make some good points. Basically, my situation is that I want to 
 use some features of Django v1.10 for a project at my company, but I am 
 unable to because I use django-formtools for a FormPreview, and it is not 
 compatible with v.1.10. I am also worried about using Django for future 
 projects given that there seem to be problems finding people to contribute 
 to what seems to be an important package. Whether I contribute depends on 
 the answers to several questions: (1) How much work is necessary to get 
 formtools compatible with v1.10? (2) Would my contribution make a 
 difference, given that I would be a new contributor, and I don't have a 
 ton 
 of time to give? (3) Is there another package that people are using for 
 FormPreviews that I'm not aware of? I couldn't find anything else that was 
 more up-to-date than django-formtools. (4) Is this a sign that people 
 simply aren't using Django for this task, and are maybe instead using some 
 kind of Javascript library instead? If you have any information these 
 questions, please let me know.

 Thanks,
 Eric

 On Tuesday, January 3, 2017 at 6:04:59 PM UTC-7, Tim Graham wrote:
>
> Is the situation bad enough that you would volunteer to maintain the 
> project?
>
> The Django team is a collection of volunteers (excepting me) who work 
> on Django. It seems that no one on the team or reading this mailing list 
> has time or interest in maintaining the project. We need to clarify the 
> maintenance status, but that doesn't necessarily mean continuing the 
> project in its current form if there's little interest.
>
> Absent volunteers, another idea could be for an interested freelancer 
> to start a kickstarter-style fundraiser to raise money to fund some time 
> to 
> work on some issue in the backlog. At least it would help determine if 
> any 
> users think the project is worth paying for.
>
> On Tuesday, January 3, 2017 at 7:28:59 PM UTC-5, 
> eric@datalyticsolutions.com wrote:
>>
>> This is really bad. django-formtools used to be part of the core of 
>> Django. Is this getting the attention it deserves from the Django 
>> Foundation?
>>
>> On Monday, November 28, 2016 at 9:55:48 PM UTC-7, Asif Saifuddin 
>> wrote:
>>>
>>> Hi Tim,
>>>
>>> In case there is lack of active maintainers for the project then you 
>>> could add me to the list of maintainers. I do contribute to django eco 
>>> system regularly.
>>>
>>> my github activities:
>>>
>>> github.com/auvipy
>>>
>>>
>>> And about moving to jazzband, I do take part in django-admin2 
>>> maintenance there with the two other old maintainers, The fact is 
>>> moving it 
>>> to jazzband haven't increased the active co maintainers.
>>>
>>> Thanks,
>>>
>>> Asif
>>>
>>> On Tuesday, November 29, 2016 at 5:45:59 AM UTC+6, Tim Graham wrote:

 Hi,

 django-formtools seems neglected. It's been several months since 
 the release of Django 1.10 but a compatible version of formtools 
 hasn't 
 been released to PyPI.

 Personally, I don't know if it's important to maintain it as an 
 "official project." An alternative could 

Re: Add custom autoreload file tracking options setting

2017-01-05 Thread Aymeric Augustin
On 4 Jan 2017, at 23:31, Bobby Mozumder  wrote:

> I guess I could just use Watchman to restart the Django development server as 
> needed?


If you find a way to tell watchman to run `django-admin runserver --noreload` 
and restart it whenever a file in the current directory changes, that should do 
the job.

Unfortunately the APIs exposed by watchman don’t make this trivial. They’re 
intended to run a build process that will terminate, while the development 
server will keep running.

-- 
Aymeric.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/A7512959-281B-4679-9E04-93EEB78C33A1%40polytechnique.org.
For more options, visit https://groups.google.com/d/optout.


Re: Changing {% include %} to strip a trailing newline

2017-01-05 Thread Aymeric Augustin
Hello,

I’m not going to veto whitespace control in Django templates. Mostly I’m 
uncomfortable with rushing this feature to fix the much narrower problem 
discussed here.

If we want to provide whitespace control in templates, we should review use 
cases, other implementations, and figure out something reasonable.

I hope this clarifies my position,

-- 
Aymeric.

> On 4 Jan 2017, at 23:53, Tim Graham  wrote:
> 
> Aymeric, we have a difference of opinion. I feel that if {% include %} 
> removed the trailing newline, it would result in far more intuitive 
> whitespace control (which may be needed in plain text email templates, for 
> example). I think the change has merits outside the context of this issue. 
> For example, I checked view-source on a djangoproject.com 
>  page that uses a {% for ... %}{% include %}{% 
> endfor %} construct and noticed that it's much shorter without all those 
> extra blank lines. I'm not advocating for additional controls to craft well 
> formatted HTML but I think there's some advantages to not having blank lines 
> everywhere. (I don't like the keep_trailing_newline option either, I'd rather 
> call it a bugfix and make a backwards-incompatible change but my projects are 
> unaffected as far as I checked.)
> 
> There are many questions on Stackoverflow about how to suppress newlines from 
> {% include %} and even a django-include-strip-tag third-party app that offers 
> the feature.
> 
> In the context of multiple template engines, I think it would be useful if 
> DTL templates could be written in a similar style to Jinja2.
> 
> I'd like to know from Carl, Adam, and others, how much effort would be 
> required to adapt the templates in your project for the new behavior. I 
> imagine a script could be written to add newlines after all {% include %} in 
> all plain text templates, for example.
> 
> It won't be the end of the world if we can't get consensus to change the 
> behavior -- I'm okay with removing the trailing newlines in the source files 
> as needed, although it will add some inconvenience.
> 
> On Wednesday, January 4, 2017 at 4:26:43 PM UTC-5, Aymeric Augustin wrote:
> If I understand correctly, we have to choose between:
> 
> 1. breaking backwards compatibility for {% include %}
> 2. breaking backwards compatibility for widgets HTML
> 3. having a handful of single-line, non-newline-terminated files
> 
> I don’t think option 1 is reasonable. Whitespace control in templates is an 
> exercise in frustration. In my experience more flexible tools such as {%-, 
> {%+, -%}, and +%} in Jinja2 increase the frustration. I don’t think Django 
> should get itself into this mess just for fixing the problem discussed here.
> 
> Option 2 seems acceptable to me. This is the "do nothing” option. I don’t 
> think the exact HTML string rendered by widgets is considered a public API. 
> We made changes to that output in the past, for example when we switched to 
> HTML5.
> 
> Option 3 also seems reasonable to me. Files could end with {# no newline, see 
> issue #xxx #} and no newline. A test could validate that the rendered output 
> doesn’t contain a newline. This would be enough to catch accidental 
> regressions caused by editors automatically adding the newline.
> 
> The quote from the POSIX standard says “should”, not “must”. If we interpret 
> it as if it was written in capitals in a RFC, this means we can make an 
> exception if we have a good reason.
> 
> (At this point surely someone will suggest that we can write an editorconfig 
> to specify that these files mustn’t end with a newline. Whatever works.)
> 
> I don’t have a strong preference for 2 or 3, however, I have a strong 
> distaste for anything more complicated.
> 
> Best regards,
> 
> -- 
> Aymeric.
> 
>> On 4 Jan 2017, at 20:58, Tim Graham gmail.com 
>> > wrote:
>> 
>> Shortly after template widget rendering was merged, an issue about extra 
>> newlines appearing in the rendered output was reported [0]:
>> 
>> For example, from django-money:
>> 
>> UIC-Franc
>> \n\n
>> US Dollar
>> \n\n
>> 
>> The newlines aren't useful and they break assertions like this:
>> 
>> US Dollar in form.as_p
>> 
>> --- end report---
>> 
>> 
>> The reporter suggested removing the trailing newline in the attrs.html 
>> template but Adam Johnson reported: "POSIX states that all text files should 
>> end with a newline, and some tools break when operating on text files 
>> missing the final newline. That's why git has the warning \ No newline at 
>> end of file and Github has a warning symbol for the missing newline."
>> 
>> I suggested that perhaps {% include %} could do .rstrip('\n') on whatever it 
>> renders.
>> 
>> Preston pointed out that Jinja2 does something similar:
>> 
>> http://jinja.pocoo.org/docs/dev/templates/#whitespace-control 
>> 
>> a single trailing newline is stripped if