Re: Composite key development

[Roger Gammans]

On Mon, 2017-02-27 at 21:03 -0800, Asif Saifuddin wrote:
> Hi Julian,
> > > I have been also reviewing and studying the previous works, deps,
discussions, codes and tickets. I have also been working to prepare a
new dep based on the previous works.
> 

I haven't had a chance to review Michael's code (is there a github
link?) but I think the composite field approach is almost certainly a
sensible route. Which is a shame because it adds quite a bit of extrope
tha refactor work for me. ;-).

The code I have got[2], does not take that route[1], shows the switch
between single field and composite primary adds a lot of complexity to
far too many (IMHO) code routes. If it was a handful I wouldn't worry a
great deal, but there is a lot of complexities which fall on the admin
code which don't quite fall out as neatly as I would like. 

Hence I hope that by providing a named composite field, then we
mitigate the issues above as arguable we still only have single field
PK, but that that field is itself composite which may well mitigate the
issues I have been seeing.


[1] We took a patch from somewhere which allowed multiple
primary_key=True specifications in the model.
[2] Initially developed by by one of by colleagues based on some other
patches he found  - We may unfortunately have lost the attribution.
-- 
Roger Gammans 

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/1488267322.2524.18.camel%40gammascience.co.uk.
For more options, visit https://groups.google.com/d/optout.

Re: Composite key development

[Asif Saifuddin]

Hi Julian,

I have been also reviewing and studying the previous works, deps, 
discussions, codes and tickets. I have also been working to prepare a new 
dep based on the previous works.

Like what Michal said, from my observation, I found the works and 
approaches of Michal is quite better and it's possible to break the work 
down into smaller parts to implement gradually.

I am not quite sure how much work or which approaches of  Thomas Stephenson 
in 2015 could be mixed or needed with Michal's approach. In my humble 
opinion Michal spent more time in working on this and I also found his 
insights and suggestions on this topic more sensible.

I would also like to work on this. My Dep is not yet ready for a push.

You could certainly review and give you input on the Dep. Some core members 
suggested me to finalize a Dep before div into code.

Let me know what your thoughts.

Regards,

Asif

I have also contact with 

On Tuesday, November 29, 2016 at 6:10:38 PM UTC+6, Craig Kerstiens wrote:
>
> Hi all,
>
> My company (Citus Data) is interested in sponsoring some Django work. In 
> particular work on support for composite primary keys. From what I 
> understand this wouldn't be the first time the work has been explored and 
> it sounds like it has a number of intricacies to it (
> https://github.com/django/deps/blob/master/draft/0191-composite-fields.rst 
> and 
> https://github.com/django/deps/blob/master/draft/0192-standalone-composite-fields.rst).
>  
> Our hope here is that it would be done in line with something that could 
> eventually become part of an official Django release vs. a one-off work 
> around. 
>
> While we know there's no guarantee of it being accepted, we'd love to find 
> someone with some knowledge of all the existing areas that would have to be 
> touched and has some experience contributing to Django core to improve that 
> likelihood. As part of the work, we would want the two existing DEPs to be 
> completed and work towards getting those accepted by the broader community. 
> And hopefully it goes without saying, but we'd fully expect all the work to 
> done in the open similar to other Django development. 
>
> If you're interested in doing this work please reach out as we'd love to 
> discuss further. And if we have enough interest we'll be doing a full CFP 
> for the work to try to keep the process as fair as possible. 
>
> ~ Craig
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/cc29383b-b930-4d4c-9f48-51fa10909ecd%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Template handling of undefined variables

[Shai Berger]

On Sunday 26 February 2017 11:16:54 Florian Apolloner wrote:
> As much as I'd like variables to raise an error if they cannot resolve, I
> am with Karen here -- the backwards compatibility considerations should
> take priority here. At least we should have a possibility in the templates
> to check if variables are undefined before we start raising exceptions.
> 

I concur.

Adam's {% load undefined_vars from future %} sounds like a good idea -- with 
two caveats:

1) We shouldn't make it part of a deprecation plan (at least, not yet)

2) We should consider how to enable it to affect many templates from one place. 
My first thought: If included in a template, it should apply (recursively) to 
any template which {% extends %} it, and to any {% include %}'d template.

Then it's not "modify each template" to use, and it isn't all-or-nothing 
unless you want it to be.

My 2 cents,
Shai

Re: Google Summer of Code 2017

[Tim Graham]

Django is accepted as a mentor organization. The student application period 
opens March 20 ends on April 3. Students playing to apply should share 
drafts of their proposal on this mailing list well before the deadline to 
get feedback and find possible mentors.

On Friday, January 20, 2017 at 3:57:33 PM UTC-5, Tim Graham wrote:
>
> I've submitted our organization application for this year.
>
> Anyone interested in mentoring, please indicate your interest 
> https://code.djangoproject.com/wiki/SummerOfCode2017. Any experienced 
> contributors are welcome to mentor. We also need project ideas on that 
> page. I've copied the ideas from past years, but they may not be the best. 
> If you have an idea in mind, feel free to start a new thread on this 
> mailing list to see if others think it's suitable.
>
> Any students interested in participating, the time to starting 
> contributing is now! The chances of developing a successful proposal 
> without any past contributions to Django is very low.
>
> If you have a question, let me know.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/13f1cef9-7d35-49e4-98cf-256427d4bf1c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Composite key development

[Michal Petrucha]

Hi everyone,

I can't speak for Thomas' implementation, as I haven't had the time
and energy to look into it too much, but I can try to clarify some
aspects of what I did years ago.

On Mon, Feb 27, 2017 at 11:10:55AM -0800, Julien Phalip wrote:
> Composite primary keys (GSoC)
> 
> =
> 
> 
> Michal Petrucha participated in two Google Summer of Code (GSoC) projects 
> in 2011 and 2013, during which some extensive code, tests and documentation 
> were written. Unfortunately this project didn't reach the point where it 
> could be merged into core and it eventually fell behind. Some relevant 
> resources about these projects can be found here:
> 
> - GSoC'2013 proposal: https://gist.github.com/konk/5408673
> 
> - GSoC'2013 status updates: 
> https://groups.google.com/forum/#!topic/django-developers/CD7OrkJ63zc
> 
> - Discussion about some implementation details: 
> https://github.com/django/django/pull/1407
> 
> - Status update from Michal from last year: 
> https://groups.google.com/forum/#!topic/django-developers/SunyXc_BTVM
> 
> 
> Michal's implementation work is divided in two separate branches 
> ("composite-fields" and "foreignkey-refactor"), presented below with their 
> corresponding list of commits:
> 
> 
> "composite-fields" branch:
> 
> - Branch: https://github.com/konk/django/tree/soc2013/composite-fields
> 
> - Commits: 
> https://github.com/django/django/compare/041a076dadce547d450cf73d97401d63cde8891d...konk:soc2013/composite-fields
> 
> 
> "foreign-refactor" branch:
> 
> - Branch: 
> https://github.com/konk/django/tree/soc2013/foreignkey-refactor
> 
> - Commits: 
> https://github.com/django/django/compare/041a076dadce547d450cf73d97401d63cde8891d...konk:soc2013/foreignkey-refactor
> 
> 
> I took a quick stab at merging current master into those branches but 
> didn't make it too far as it turned out to be a little tedious. The main 
> reason is that since 2013 there have been extensive changes in the ORM 
> internals, in particular around the Meta api. So I'm not sure if it's worth 
> doing that merge at this point. However, it'd be a shame to let all that 
> great work go to waste and hopefully some of it can still be reused.

Sadly, I don't see how a merge of current master would be feasible
with those branches. This all happened before migrations, the explicit
AppRegistry, and all the meta changes were merged, all of which cause
quite some conflicts. It should be feasible to port significant
portions of my old work onto master, though, commit by commit.

> Some questions:
> 
> - Are there any portions of code or some solutions from Michal's 
> implementation that could be ported over to the DEPs and to Thomas' more 
> recent implementation?

I would hope so, but unfortunately, can't say with any certainty. Even
if I just look at the interface, Thomas' approach seems quite
different from mine, but without going through the code, it's hard for
me to tell what parts could be relevant.

> - What is the difference or relationship between the "composite-fields" and 
> "foreignkey-refactor" branches?

Right, so the gist of it is that if we ever want to support composite
foreign keys, it will be necessary to turn ForeignKey into a virtual
field, which is exactly what I attempted to achieve in the
“foreignkey-refactor” branch. This would also have the neat side
effect of clearly defining what even is a virtual field in the ORM,
since that was definitely not a very well defined concept at the time.
(I think it's gotten better with the formalization of _meta, but it's
still very heavily influenced by the early approach of “any aspects of
GenericForeignKey that don't closely match regular fields”.)

My thinking was that I'd first try to get the refactoring part as
solid as possible, and only then start to build on that by introducing
the new concept of composite fields. Unfortunately, I seem to have
crammed too much into the GSoC project, so I couldn't really stop and
take the time to get the FK refactor over the finish line, and instead
focused on getting as much of the new-feature code finished as
possible.

So yeah, foreignkey-refactor is the part where I turned ForeignKey
into a virtual field, and composite-fields is the part based on the
first branch where I added composite fields to the equation.

FWIW, there's https://code.djangoproject.com/ticket/16508 where I've
tried to introduce some prerequisites for the ForeignKey refactor in
smaller chunks.

> - Should a new DEP be submitted to specifically tackle composite primary 
> keys and foreign keys?

Good question. At a glance, Thomas' DEPs do not mention composite
foreign keys at all, so I guess if we want support for those at some
point, the complexity of this task would definitely warrant a DEP.

Anyway, most of this email focuses on virtual FKs, rather than
composite fields and composite primary keys. In my mind, that's the
order in which those things make sense, but I may be overly ambitious
here (an

Re: RGSoC'17

[Tim Graham]

Yes, I think a collection of tickets from the "Testing framework"
component in Trac (or whatever ideas you might have) would be a
suitable project. If you want any feedback about what tickets would be
more useful than others, just let me know.

https://code.djangoproject.com/query?status=assigned&status=new&component=Testing+framework&stage=Accepted&col=id&col=summary&col=status&col=owner&col=type&col=version&desc=1&order=id

On Mon, Feb 27, 2017 at 3:02 PM, Саша Ершова
 wrote:
> Hi Tim,
>
> We're thinking about working on test framework cleanup, do you think it
> would be more useful than NoSQL idea?
>
> Alexandra and Alexandra
>
> пн, 27 февр. 2017 г. в 15:30, Tim Graham :
>>
>> Hi Alexandra and Alexandra,
>>
>> I'm not sure what exactly the NoSQL idea would entail but I personally
>> don't see it as a big priority. There are a number of third-party packages
>> offering NoSQL support, see https://djangopackages.org/grids/g/nosql/ for
>> some examples. I'd recommend to choose some other idea unless you have a
>> specific use case and idea in mind, as well as a motivated coach.
>>
>>
>> On Sunday, February 26, 2017 at 6:28:56 PM UTC-5, Александра Мартынова
>> wrote:
>>>
>>> Dear all,
>>>
>>> We are a team looking to participate in Rails Girls Summer of Code 2017,
>>> and we would like to contribute to Django for our internship. We are Russian
>>> CS students with 4-year experience in Python (both 2 and 3) and OOP, as well
>>> as algorithms and database design.
>>>
>>> Among the projects suggested for GSoC'17 we think the most suitable for
>>> us would be SQLAlchemy / NoSQL integration, because we have some experience
>>> with both SQL and NoSQL. There are no issues related to this project, so
>>> could you please give us some test task?
>>>
>>> Best wishes,
>>> Alexandra Ershova (https://github.com/religofsil)
>>> and Alexandra Martynova (https://github.com/melanoya)

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAD-rxRC4A53umUud_qBieDCQA1tSvTr-9ia5hN%2BTcfwo3%3DqRHg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Composite key development

[Julien Phalip]

On Tuesday, November 29, 2016 at 4:10:38 AM UTC-8, Craig Kerstiens wrote:
>
> My company (Citus Data) is interested in sponsoring some Django work. In 
> particular work on support for composite primary keys. From what I 
> understand this wouldn't be the first time the work has been explored and 
> it sounds like it has a number of intricacies to it (
> https://github.com/django/deps/blob/master/draft/0191-composite-fields.rst 
> and 
> https://github.com/django/deps/blob/master/draft/0192-standalone-composite-fields.rst).
>  
> Our hope here is that it would be done in line with something that could 
> eventually become part of an official Django release vs. a one-off work 
> around. 
>
> While we know there's no guarantee of it being accepted, we'd love to find 
> someone with some knowledge of all the existing areas that would have to be 
> touched and has some experience contributing to Django core to improve that 
> likelihood. As part of the work, we would want the two existing DEPs to be 
> completed and work towards getting those accepted by the broader community. 
> And hopefully it goes without saying, but we'd fully expect all the work to 
> done in the open similar to other Django development. 
>
> If you're interested in doing this work please reach out as we'd love to 
> discuss further. And if we have enough interest we'll be doing a full CFP 
> for the work to try to keep the process as fair as possible.
>

Thank you Craig for bringing this up and for offering support. This is a 
long-wanted feature —the original ticket is 12 (!) years old— and as you 
know there has been extensive work done on this in the past, so it'd be 
great to find a way to push this all the way across the finish line.


I just took a bit of time and tried to review all past discussions and 
implementations on the subject. See my recap and questions below. If anyone 
here on this list has any answers or thoughts about this, please jump in! I 
will then compile all new feedback and try to help move this forward.


Composite fields DEPs

==


Currently the consensus as a first step is to finalize these two draft DEPs:


https://github.com/django/deps/blob/master/draft/0191-composite-fields.rst

https://github.com/django/deps/blob/master/draft/0192-standalone-composite-fields.rst


Those drafts were written by Thomas Stephenson in 2015 and were discussed 
on this thread: 
https://groups.google.com/forum/#!topic/django-developers/MZUcOE6-7GY as 
well as in this pull request: https://github.com/django/deps/pull/12


It seems that some extra topics might need to be included and fleshed out 
in those DEPs, in particular:

- Inspectdb integration

- Migration implications

- Form fields handling

- Admin integration

- Serialization/deserialization


Some questions:

- Do you agree that all the topics above should be covered in the DEPs?

- Should other topics also be covered?

- Are the current design details (observer pattern, lookup functions, 
constraints management, etc) already accepted or do they still need more 
discussion?


Thomas had also started an implementation for these DEPs in this branch: 
https://github.com/ovangle/django/tree/dep_191


I've just merged current master into that branch and pushed it to my fork. 
You can see the details here: 
https://github.com/django/django/compare/eb97af0402536a86d5d8c916859986288bb88dc9...jphalip:dep_191


Some questions:

- What is the status of this implementation work?

- What are the pieces that still need to be implemented in order to fulfill 
the existing DEPs?

- Is this implementation at a stage where it can formally be reviewed? If 
so I can create a PR to initiate a more detailed discussion.


Composite primary keys (GSoC)

=


Michal Petrucha participated in two Google Summer of Code (GSoC) projects 
in 2011 and 2013, during which some extensive code, tests and documentation 
were written. Unfortunately this project didn't reach the point where it 
could be merged into core and it eventually fell behind. Some relevant 
resources about these projects can be found here:

- GSoC'2013 proposal: https://gist.github.com/konk/5408673

- GSoC'2013 status updates: 
https://groups.google.com/forum/#!topic/django-developers/CD7OrkJ63zc

- Discussion about some implementation details: 
https://github.com/django/django/pull/1407

- Status update from Michal from last year: 
https://groups.google.com/forum/#!topic/django-developers/SunyXc_BTVM


Michal's implementation work is divided in two separate branches 
("composite-fields" and "foreignkey-refactor"), presented below with their 
corresponding list of commits:


"composite-fields" branch:

- Branch: https://github.com/konk/django/tree/soc2013/composite-fields

- Commits: 
https://github.com/django/django/compare/041a076dadce547d450cf73d97401d63cde8891d...konk:soc2013/composite-fields


"foreign-refactor" branch:

- Branch: 
https://github.com/konk/django/tree/soc20

Re: Template handling of undefined variables

[Luke Plant]

On 05/01/17 02:39, Tim Martin wrote:

2) There appears to be an inconsistency in the default_if_none
   modifier. If I have a template with

   x|default_if_none:y = {{x|default_if_none:y}}
   {% if x|default_if_none:y %}
   x|default_if_none:y is apparently True
   {% endif %}

   with y defined to True and x undefined, then it produces:

   x|default_if_none:y =
   x|default_if_none:y is apparently True

   IOW it appears that the default_if_none doesn't cause y's value to
   be used in the case where the variable is being printed, even
   though it causes the expression to have y's value when evaluated in
   an if. I think this is something to do with the fact that the two
   ways of handling failures (ignore_failures and string_if_invalid)
   do different things.

   I don't see a way to make backward compatible code here.

   I think this is just a bug, does anyone think otherwise?


This seems to be working exactly as it should.

In the context of template variable interpolation i.e. `{{ }}` syntax, 
it is defined that `x` in this case will get converted to the empty 
string (or your 'string_if_invalid' setting) - 
https://docs.djangoproject.com/en/dev/ref/templates/api/#how-invalid-variables-are-handled 
- unlike the case for `if` tags.


When this value (the empty string) is passed to `default_if_none`, the 
value is not changed, since it is not None. See 
https://docs.djangoproject.com/en/1.10/ref/templates/builtins/#default-if-none


The behaviour here follows 100% logically from the documented behaviour 
of the two components.


Regards,

Luke




Sorry for the lengthy email. Thanks for any input.

Tim
--
You received this message because you are subscribed to the Google 
Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send 
an email to django-developers+unsubscr...@googlegroups.com 
.
To post to this group, send email to 
django-developers@googlegroups.com 
.

Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/b9d7b557-5826-4c7e-a064-5f8b59312faa%40googlegroups.com 
.

For more options, visit https://groups.google.com/d/optout.


--
You received this message because you are subscribed to the Google Groups "Django 
developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/a7dc3b9e-195f-e4e4-2692-4b07de68c0cd%40cantab.net.
For more options, visit https://groups.google.com/d/optout.

Re: Template handling of undefined variables

[Luke Plant]

On 26/02/17 00:44, Karen Tracey wrote:
On Sat, Feb 25, 2017 at 2:10 PM, Tim Graham > wrote:


I think any use of undefined template variables should raise an
exception. In the long run, keeping a setting to allow some other
behavior seems confusing and, considering the case of templates
that might be reused in different projects with different
settings, even dangerous.


I think I'm confused...Django templates have allowed use of undefined 
variables and documented their use as evaluating to the empty string 
for as long as I recall. Wouldn't a change to instead raise exceptions 
be a major backwards-incompatibility?


https://docs.djangoproject.com/en/1.7/topics/templates/#variables said 
"If you use a variable that doesn’t exist, the template system will 
insert the value of the TEMPLATE_STRING_IF_INVALID setting, which is 
set to '' (the empty string) by default."


This behaviour applies only to what happens when the variable is 
rendered. In the context of `if` tags, undefined variables get converted 
to `None`. This behaviour is documented - 
https://docs.djangoproject.com/en/dev/ref/templates/api/#how-invalid-variables-are-handled 
but perhaps not in as much detail as necessary.


The question is about changing this, especially due to templates like this:

{% if user.role == roles.staff %}
[sensitive information here]
{% endif %}

If:
1) you forget to provide both user and role to template context or
2) 'role' or 'staff' are invalid attributes, or
3) 'role' returns None but 'staff' is an invalid attribute, for example,
4) various combinations of these

then this results in the sensitive info being shown, because `None == 
None`. The proposal is to introduce a value that doesn't not compare 
equal to itself and avoid this kind of issue.


Having thought about it more, I've realised that this really isn't going 
to work at all.


First attempt:

class Undefined1(object):
def __eq__(self, other):
return False

If we use this, then consider the following template code:

{% if user.role != roles.staff %}
This info is private, sorry
{% else %}
[sensitive information here]
{% endif %}


The default implementation of != means we will again get the sensitive 
data shown for some of the error situations given above (e.g. 1 and 2). 
Second attempt:


class Undefined2(object):
def __eq__(self, other):
return False
def __new__(self, other):
return False

This object is neither equals nor not-equals to itself or anything else. 
But consider this template:



{% if user.role == roles.customer %}
This info is private, sorry
{% else %}
[sensitive information here]
{% endif %}

With `Undefined2` being used for invalid attributes, we again get the 
sensitive information being shown in the case of developer error and 
missing attributes. Plus, we now have the situation that `{% if a != b 
%}` is not the same as `{% if not a == b %}`, which is very confusing 
behaviour for most people.


In other words, we are getting into the territory of needing a value 
that behaves like NULL in SQL. Even if there were no implementation 
issues (and there will be lots, because operators like 'or' 'and' etc. 
would need to cope with this, not to mention 3rd party code), and there 
were no backwards compatibility issues (there are lots), this would be 
extremely dubious to me, because ternary logic is extremely tricky.


The only sensible alternative to current behaviour is to raise 
exceptions, but huge backwards incompatibility issues I think rule this 
out. If I were to start again, I think I would make the template system 
not silence any errors you would normally get in Python, but just have 
some special syntax for converting non-existent data or attributes into 
None. But I don't have that time machine.


Regards,

Luke

--
You received this message because you are subscribed to the Google Groups "Django 
developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/4b0a8ec9-c588-de8d-76ba-0d3b55b8fe20%40cantab.net.
For more options, visit https://groups.google.com/d/optout.