Re: Window expressions, #26608

2017-02-20 Thread Mads Jensen
I hit somewhat of a wall, and would like some input/help.

* Import of window functions from `django.db.model.functions` seems to be 
the
  convention - `DEFAULT_INDEX_TABLESPACE` causes some headaches, because of
  XField being explicitly set in _output_field variable. Probably the 
functions need some extra fine-tuning in some way. So
  far, most of the functions don't have explicit arguments in the 
constructor,
  because they either have a varying number of arguments (Lead and Lag, for
  instance) or don't take any (rank and row_number; however, those can take 
an
  argument in case somebody wants to add support for WITHIN 
GROUP-expression). I
  deleted the "_output_field = Field" from the functions that do take an
  argument, such as Lead and Lag, would it be required to explicit about the
  output_field, such as overriding the resolve_output_field-field? In those
  cases, it's always the type of the first argument or the default-argument 
in
  case it's provided/available. Tests are running locally without the 
explicit
  _output_field. And just to avoid shuffling around things later, any 
arguments
  against django.db.models.functions.window being the right place for this?

* `__repr__` and `__str__` methods and testing. What's a good general 
convention
  for something like this? I don't have access to the connection in 
__repr__,
  and __str__-methods, and thus just imported from `django.db` in the tests,
  which I suppose is not the ideal solution for this. At least, it doesn't
  really feel that way.

* Are there more cases which should fail apart from filtering (supported by
  filterable) and use in UPDATE (can_be_reffed_in_update)?

Thank you.

Kind regards,
Mads

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/d81ddfe0-f8f2-4ec0-99c9-96cd4b619246%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


[ANNOUNCE] Django 1.11 beta 1 released

2017-02-20 Thread Tim Graham
We've made the second release on the way to Django's next major release, 
Django 1.11! With a month and a half until the final release, we'll need 
timely testing from the community to ensure an on-time and stable release. 
Check out the blog post:

https://www.djangoproject.com/weblog/2017/feb/20/django-111-beta-1-released/

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/6da1ca0c-4d2c-462c-a729-f86279e9dc3a%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


"Ask the Django Team Anything" IRC session - Wed Feb 22 18:00 UTC

2017-02-20 Thread Tim Graham
In partnership with the freenode community team, the Django team will hold 
an "Ask Me Anything" session this week.

See http://freenode.net/news/django-ama for details.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/b5234f7e-0528-4323-9c04-5940fd4d76e6%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Re: Some thoughts about improving migration squashing

2017-02-20 Thread Patryk Zawadzki
W dniu poniedziałek, 20 lutego 2017 11:49:38 UTC+1 użytkownik Markus 
Holtermann napisał:
>
> On Thu, Feb 16, 2017 at 03:25:16PM -0800, rap...@makeleaps.com 
>  wrote: 
> >RemoveField('A', 'foo') also references A and foo, but does it reference 
> B? 
> >if it does, then it' s hard to have optimizations that pass through this, 
> >because this field could be referencing any model (theoretically). 
>
> No, that field can not reference any model. It reference exactly one 
> model (that even holds for FK to abstract models as fields from abstract 
> models are inlined in the concrete models in migrations). However, 
> RemoveField doesn't have the information to "just" figure out the 
> referenced model. RemoveField would need to look into the from_state's 
> apps and actually even look into the actual field that's referred. 
> ForeignKeys have an implicit to_field attribute, so, having 
>
>   AddField('A', 'foo', ForeignKey('B', to_field='bar')) 
>
> a 
>
>   RemoveField('A', 'foo') 
>
> references exactly one field on one particular model. Not more and not 
> less. The issue here is that RemoveField needs to take that information 
> from the state and not from one of its attributes. 
>

Technically it references _some_ model named "B" that was created not 
sooner than the current migration's explicit dependencies. It may be the 
model you saw when you created that migration or it may be some other 
model. You can tell your migration that it has to be executed no sooner 
than after another migration is complete but there is no way to say "but 
before model B is modified any further".

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/b9964408-8dff-49ec-aa08-89826d35503c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Re: Template handling of undefined variables

2017-02-20 Thread Marc Tamlyn
+1 to not having to add (and then remove later) a {% load %} tag to every
template - that was seriously dull with the URL change.

Marc

On 20 February 2017 at 11:42, Luke Plant  wrote:

> On 19/02/17 12:55, Adam Johnson wrote:
>
> +1 for more obvious errors, silently changing the behaviour could indeed
> lead to unconsidered security holes like
>
> {% if user is None %}
> non-sensitive information
> {% else %}
> sensitive information
> {% endif %}
>
> ...which doesn't seem like an unrealistic template snippet. We all know
> variables can go missing in refactorings.
>
> Another option, perhaps not feasible to implement, would be deprecating
> the old behaviour, similar to the previous change in url with something
> like:
>
> {% load undefined_vars from future %}
>
>
> I agree there are a lot of potential security/correctness issues with
> this, it is potentially quite a big change (though very helpful IMO).
>
> A different approach to introducing it might be a setting, possibly an
> option to the Django template engine instead - https://docs.djangoproject.
> com/en/1.10/ref/settings/#std:setting-TEMPLATES-OPTIONS . I think this
> would be more appropriate for something that is more of a global behaviour
> issue, more practical than having to add hundreds of 'load from future'
> tags, plus it would then parallel other similar settings like
> 'string_if_invalid'. In the next version of Django the option would default
> to False (i.e. old behaviour), but raise a deprecation warning, in future
> versions it would simply be True, and raise an error if someone tries to
> pass False (but allow True, for the sake of apps that are spanning multiple
> Django versions).
>
> This would allow people to test their site with the new mechanism and have
> time to fix issues, which can be especially important for 3rd party Django
> apps.
>
> Ideally there would be some way to instrument code and see if the output
> would be different with the new behaviour, but I can't think of an easy way
> to do this.
>
> Luke
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at https://groups.google.com/group/django-developers.
> To view this discussion on the web visit https://groups.google.com/d/
> msgid/django-developers/496fa4a8-b902-657a-92c2-9766bfff8a26%40cantab.net
> 
> .
>
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAMwjO1GmpDk8ByyEW2tjpstV9OpfPNokfGKNgufEmDWFhi27hQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


Re: Template handling of undefined variables

2017-02-20 Thread Luke Plant

On 19/02/17 12:55, Adam Johnson wrote:
+1 for more obvious errors, silently changing the behaviour could 
indeed lead to unconsidered security holes like


{% if user is None %}
non-sensitive information
{% else %}
sensitive information
{% endif %}

...which doesn't seem like an unrealistic template snippet. We all 
know variables can go missing in refactorings.


Another option, perhaps not feasible to implement, would be 
deprecating the old behaviour, similar to the previous change in 
url with something like:


{% load undefined_vars from future %}


I agree there are a lot of potential security/correctness issues with 
this, it is potentially quite a big change (though very helpful IMO).


A different approach to introducing it might be a setting, possibly an 
option to the Django template engine instead - 
https://docs.djangoproject.com/en/1.10/ref/settings/#std:setting-TEMPLATES-OPTIONS 
. I think this would be more appropriate for something that is more of a 
global behaviour issue, more practical than having to add hundreds of 
'load from future' tags, plus it would then parallel other similar 
settings like 'string_if_invalid'. In the next version of Django the 
option would default to False (i.e. old behaviour), but raise a 
deprecation warning, in future versions it would simply be True, and 
raise an error if someone tries to pass False (but allow True, for the 
sake of apps that are spanning multiple Django versions).


This would allow people to test their site with the new mechanism and 
have time to fix issues, which can be especially important for 3rd party 
Django apps.


Ideally there would be some way to instrument code and see if the output 
would be different with the new behaviour, but I can't think of an easy 
way to do this.


Luke

--
You received this message because you are subscribed to the Google Groups "Django 
developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/496fa4a8-b902-657a-92c2-9766bfff8a26%40cantab.net.
For more options, visit https://groups.google.com/d/optout.


Re: Some thoughts about improving migration squashing

2017-02-20 Thread Markus Holtermann

On Thu, Feb 16, 2017 at 03:25:16PM -0800, raph...@makeleaps.com wrote:

When you have AddField('A', 'foo', ForeignKey('B')), this operation
references A and foo, but also references B.


correct.


RemoveField('A', 'foo') also references A and foo, but does it reference B?
if it does, then it' s hard to have optimizations that pass through this,
because this field could be referencing any model (theoretically).


No, that field can not reference any model. It reference exactly one
model (that even holds for FK to abstract models as fields from abstract
models are inlined in the concrete models in migrations). However,
RemoveField doesn't have the information to "just" figure out the
referenced model. RemoveField would need to look into the from_state's
apps and actually even look into the actual field that's referred.
ForeignKeys have an implicit to_field attribute, so, having

 AddField('A', 'foo', ForeignKey('B', to_field='bar'))

a

 RemoveField('A', 'foo')

references exactly one field on one particular model. Not more and not
less. The issue here is that RemoveField needs to take that information
from the state and not from one of its attributes.

/Markus



But if we assert that RemoveField doesn't refer to any models referenced to
by its field, then our optimizer can take a couple more liberties.

Raphael

On Friday, February 17, 2017 at 2:15:47 AM UTC+9, Markus Holtermann wrote:


I'm not sure if it's related or not wo what you're investigating,
RemoveField cannot "just" optimized through, as you might have another
AddField operation afterwards adding another field with the same name.

/Markus

On Thu, Feb 16, 2017 at 08:19:01AM -0800, rap...@makeleaps.com
 wrote:
>Hey Simon,
>
> I looked through your PR and added a couple comments. The main thing is
I
>think we can actually ignore the field context on "RemoveField", if only
>because the executor doesn't need it. Even though the field might be
>pointing to a related model, that doesn't prevent being optimized
through.
>
> This is hard to explain, but intuitively, each "RemoveField" is paired
>with an "AddField" or "CreateModel" that *does *depend on the related
>model. So if we have a potentially dangerous optimization, those initial
>operations will "protect" the causal order, not "RemoveField".
>
> Raphael
>
>--
>You received this message because you are subscribed to the Google Groups
"Django developers  (Contributions to Django itself)" group.
>To unsubscribe from this group and stop receiving emails from it, send an
email to django-develop...@googlegroups.com .
>To post to this group, send email to django-d...@googlegroups.com
.
>Visit this group at https://groups.google.com/group/django-developers.
>To view this discussion on the web visit
https://groups.google.com/d/msgid/django-developers/9dfdcec6-b98c-44f2-86af-99aaa8857cc9%40googlegroups.com.

>For more options, visit https://groups.google.com/d/optout.




--
You received this message because you are subscribed to the Google Groups "Django 
developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/7515c909-a015-451d-bdaa-f040e6322166%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


--
You received this message because you are subscribed to the Google Groups "Django 
developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/20170220104923.GA17288%40inel.local.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: PGP signature


Re: Modifying django.db.models.expressions.Func.as_sql() to allow `**extra_context` to be more useful

2017-02-20 Thread Adam Johnson
Josh is right, cloning is better

On 20 February 2017 at 03:00, Josh Smeaton  wrote:

> Avoid modifying self in `as_` methods if possible, even if you are going
> to restore state. See how some of the others work around needing to switch
> order etc here: https://github.com/django/django/blob/
> 75f0070a54925bc8d10b1f5a2b6a50fe3a1f7f50/django/db/models/
> functions/base.py#L58
>
> Short version: clone self (performance cost I admit), then manipulate the
> clone before rendering it. I think that's enough for the patch you're
> working on. If we want to tackle clone performance then we can do so in
> another patch.
>
>
> On Monday, 20 February 2017 04:51:18 UTC+11, Adam Johnson wrote:
>>
>> I don't understand the implications of moving the extra_context line,
>> but is it not possible to reverse the order of the params in as_mysql with
>> something like the following?
>>
>> def as_mysql(self, compiler, connection):
>> self.source_expressions = reversed(self.source_expressions)
>> sql, params = self.as_sql(compiler, connection, function='LOCATE')
>> self.source_expressions = reversed(self.source_expressions)
>> return sql, params
>>
>> It's a bit hacky but I don't see why it wouldn't work. May be better with
>> a context manager to do the temporary reversal.
>>
>> On 19 February 2017 at 08:04,  wrote:
>>
>>> I'm busy working on a patch to add a string index function to the
>>> default database functions: https://code.djangoproject.com/ticket/27834
>>>
>>> As part of this patch, the arguments passed to the LOCATE MySQL function
>>> need to be in reverse order to all of the other backend implementations.
>>> I've been toying with a few ways to make this as small a code change as
>>> possible, but I don't think that the Func API allows for modification of
>>> something like expression order by the time the backend is known (i.e. in
>>> `as_mysql()`.
>>>
>>> What are everyone's thoughts on moving this line:
>>> https://github.com/django/django/blob/master/django/db/model
>>> s/expressions.py#L552 to just before the return statement on line 563?
>>> Are there any repercussions?
>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "Django developers (Contributions to Django itself)" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to django-develop...@googlegroups.com.
>>> To post to this group, send email to django-d...@googlegroups.com.
>>> Visit this group at https://groups.google.com/group/django-developers.
>>> To view this discussion on the web visit https://groups.google.com/d/ms
>>> gid/django-developers/3975696a-1846-4a20-9c9b-a1cd25625e2a%
>>> 40googlegroups.com
>>> 
>>> .
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>
>>
>>
>> --
>> Adam
>>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at https://groups.google.com/group/django-developers.
> To view this discussion on the web visit https://groups.google.com/d/
> msgid/django-developers/748f3067-66f7-4b31-b71a-
> bfad290e3399%40googlegroups.com
> 
> .
>
> For more options, visit https://groups.google.com/d/optout.
>



-- 
Adam

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAMyDDM24gVn9tUjH194HcKL2-5vjZrQj%2BqWCtMe0s7AJTzLF8w%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.