Re: Displaying (some) M2M fields with through models specified

[Russell Keith-Magee]

Hi Alex,

Short version - Broadly speaking, this is a feature we've discussed many
times over many years, and we've accepted as a good idea in principle. It's
logged as ticket #9475 [1]. There is some additional discussion on the
original ticket that introduced m2m through models (#6095 [2]).

The devil is in the details. Your analysis is largely correct, but there
are a couple of details in your analysis that need tweaking.

On Mon, May 26, 2014 at 1:55 PM, Alexander Hill  wrote:

> Hi all,
>
> Currently, only M2M fields without custom intermediary models can use the
> normal Django M2M form machinery. If you define your own intermediary
> models, you need to use inlines instead.
>
> I would like to allow fields with custom through models to use the regular
> M2M field/widget automatically, if the through model meets the requirement
> that all its extra fields either have defaults or are nullable. I'll refer
> to these as "auto-compatible" models.
>

It's not *just* about the admin interface, however. Admin only works
because there's an underlying API that can be used; we need an API proposal
to underpin the m2m widget used by admin. This was the sticking point that
prevented this feature being added 7 years ago.

In your case, you're punting on the API case where extra m2m fields aren't
optional, but the underlying issues are fundamentally the same.


> Making this work right now is as easy as adding "auto_created = True" to
> the intermediary model's Meta. If your model is not auto-compatible, you'll
> get a error when you try to add to the relation.
>

If this works, it's accidental, and I'd be *very* surprised if the final
fix is as simple as this. auto_created is involved with the process that
determines whether a table is synchronised; so while it probably works if
you add `auto_created = True` to a table that has already been
synchronised, you'll probably find that it breaks if you do this on an
unsynchronised model (with "can't find table" type errors).


> I would like to add official support for this. My suggestion would be:
>
> 1. Add a model Meta attribute that better describes these models, e.g.
> "simple_m2m". Auto-created intermediary models should have this set to True.
>

This should be possible to auto detect - a manual flag shouldn't be
required. There's enough data in the Options object on a model to determine
if an m2m instance can be saved without any data beyond the PKs of the two
related objects.


> 2. Wherever auto_created appears in a condition, decide for each whether
> or not auto-compatible models should also pass the test - if so, replaced
> with a check to "simple_m2m".
> 3. Add code to validate that user-created models with simple_m2m True
> actually do meet the above requirements.
>

If the flag can be auto detected, this bit isn't required.


> As a note to step 2, if there is no test of auto_created that
> auto-compatible models shouldn't also pass, auto_created could simply be
> renamed. I doubt that though.
>

As do I :-)


> Thoughts?
>
>
In summary - you're on the right path. There's a few edge cases that need
to be addressed, and from a "greedy core developer" point of view, it would
be nice to see the *whole* problem solved, not just the subset you've
restricted yourself to. However, I'm sure we'll take whatever patch we can
get :-)

[1] https://code.djangoproject.com/ticket/9475
[2] https://code.djangoproject.com/ticket/6095

Yours,
Russ Magee %-)

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAJxq8491ONjTtTUBTsEgohRpV5hr9nYK4WAkEKAnTqqhsk6-XQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Displaying (some) M2M fields with through models specified

[Alexander Hill]

Hi all,

Currently, only M2M fields without custom intermediary models can use the
normal Django M2M form machinery. If you define your own intermediary
models, you need to use inlines instead.

I would like to allow fields with custom through models to use the regular
M2M field/widget automatically, if the through model meets the requirement
that all its extra fields either have defaults or are nullable. I'll refer
to these as "auto-compatible" models.

Making this work right now is as easy as adding "auto_created = True" to
the intermediary model's Meta. If your model is not auto-compatible, you'll
get a error when you try to add to the relation.

I would like to add official support for this. My suggestion would be:

1. Add a model Meta attribute that better describes these models, e.g.
"simple_m2m". Auto-created intermediary models should have this set to True.
2. Wherever auto_created appears in a condition, decide for each whether or
not auto-compatible models should also pass the test - if so, replaced with
a check to "simple_m2m".
3. Add code to validate that user-created models with simple_m2m True
actually do meet the above requirements.

As a note to step 2, if there is no test of auto_created that
auto-compatible models shouldn't also pass, auto_created could simply be
renamed. I doubt that though.

Thoughts?

Alex

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CA%2BKBOKykoNVUZLR_DGdYx%2Bfd87ZcpGWku25ZR8japaPG2fdReg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: [GSOC] Weekly update

[Daniel Pyrathon]

Hi All,

Just to make you know, I have put up the current _meta API documentation 
here:
http://162.219.6.191:8000/ref/models/meta.html?highlight=_meta
As always, feel free to ask questions.

Daniel

On Monday, May 26, 2014 1:26:27 AM UTC+2, Daniel Pyrathon wrote:
>
> Hi Josh,
>
> The meta API specified in the docs (
> https://github.com/PirosB3/django/blob/meta_documentation/docs/ref/models/meta.txt)
>  
> is the current API. I have documented this in order to understand more of 
> the current implementation and it will be good to show a comparison when a 
> new meta API will be approved.
>
> My current proposal (https://gist.github.com/PirosB3/371704ed40ed093d5a82) 
> and it will be discussed tomorrow with Russell. I will post as soon as I 
> have updates.
>
> Daniel Pyrathon 
>
> On Saturday, May 24, 2014 10:37:49 AM UTC+2, Josh Smeaton wrote:
>>
>> Hi Daniel,
>>
>> Nice work putting that document together. Is the meta document you put 
>> together the current API or is it the API you are proposing? If the latter, 
>> a few suggestions (and if others disagree, please point that out):
>>
>> - Remove all mention of caching. That should be an implementation detail 
>> only, and not a requirement for other implementations.
>> - the *_with_model methods really rub me up the wrong way. I would prefer 
>> always returning the _with_model variant, and letting the caller discard 
>> the model if they don't need it.
>> - I'm not a fan of virtual and concrete fields, though I have to admit 
>> I'm not sure how they're different, especially in the context of different 
>> implementations.
>> - Not sure that m2m should be differentiated from related.
>> - init_name_map should be an implementation detail.
>> - normalize_together should be an implementation detail.
>>
>> Regards,
>>
>> Josh
>>
>> On Saturday, 24 May 2014 05:05:02 UTC+10, Daniel Pyrathon wrote:
>>>
>>> Hi all,
>>>
>>> In the last days I have built a documentation of the current state of 
>>> Options. Based on feedback and prototyping I have thought of a potential 
>>> interface for _meta that can solve the issues currently present, such as 
>>> redundancy (in code and in caching systems). The interface has also been 
>>> thought to be maintainable and is a base that can be used to create custom 
>>> meta stores.
>>> Obviously this is far from perfect, It will need many iterations and 
>>> maybe it is too complex. I would really love to gain as much feedback as 
>>> possible so it can be discussed with Russell and the community on Monday.
>>>
>>> The documentation of _meta can be found here: 
>>> https://github.com/PirosB3/django/blob/meta_documentation/docs/ref/models/meta.txt
>>> I will be refining the document in the next days, I will also be 
>>> publishing the docs on a webserver and will be linking a URL soon.
>>>
>>> My proposal has been published here:
>>> https://gist.github.com/PirosB3/371704ed40ed093d5a82
>>> In the next days I will be iterating over the feedback gained, and based 
>>> on one very interesting suggestion on IRC, I will try to see how my API 
>>> syntax looks in modelforms.py.
>>>
>>> As said previously, and feedback is greatly appreciated.
>>>
>>> Hi from Pycon IT!
>>>
>>> Daniel Pyrathon
>>>
>>> On Tuesday, May 20, 2014 3:25:45 PM UTC+2, Josh Smeaton wrote:

 Best of luck!

 On Tuesday, 20 May 2014 03:56:06 UTC+10, Daniel Pyrathon wrote:
>
> Hi All,
>
> Today I will be starting my weekly updates on my SoC project: 
> refactoring Meta to a stable API. For anyone who missed out, you will be 
> able to view it here: 
> https://docs.google.com/document/d/1yp2_skqkxyrc0egdRv6ofnRGCI9nmvxDFBkCXgy0Jwo/edit
>
> This week is the first official week of SoC. Me and my mentor 
> (Russell) are initially approaching the work in the following way:
>
>- *Document the existing Meta API*
>For each endpoint, document the following:
>  - Input parameters and return type
>  - Caching pattern used
>  - Where it's called from (internally and externally to Meta)
>  - Why is it being called
>  - When is it being called
>
>- *Propose an initial refactor plan*
>Once the documentation has been done, I should have a better idea 
>of the current implementation. This will allow me to mock a proposed 
>implementation that will be reviewed at my next update call, on Monday.
>
> My next update will be posted on Friday, just to make sure the 
> community is informed of my progress. For any major updates that require 
> community approval, I will be creating separate threads.
> My name on the internet is pirosb3, so if you want to have a chat 
> about my progress feel free to contact me! The branch I am currently 
> working on is 
> https://github.com/PirosB3/django/tree/meta_documentation
>
> Regards,
> Daniel Pyrathon
>

>>> On Tuesday, May 20, 2014 3:25:

Re: [GSOC] Weekly update

[Daniel Pyrathon]

Hi Josh,

The meta API specified in the docs 
(https://github.com/PirosB3/django/blob/meta_documentation/docs/ref/models/meta.txt)
 
is the current API. I have documented this in order to understand more of 
the current implementation and it will be good to show a comparison when a 
new meta API will be approved.

My current proposal (https://gist.github.com/PirosB3/371704ed40ed093d5a82) 
and it will be discussed tomorrow with Russell. I will post as soon as I 
have updates.

Daniel Pyrathon 

On Saturday, May 24, 2014 10:37:49 AM UTC+2, Josh Smeaton wrote:
>
> Hi Daniel,
>
> Nice work putting that document together. Is the meta document you put 
> together the current API or is it the API you are proposing? If the latter, 
> a few suggestions (and if others disagree, please point that out):
>
> - Remove all mention of caching. That should be an implementation detail 
> only, and not a requirement for other implementations.
> - the *_with_model methods really rub me up the wrong way. I would prefer 
> always returning the _with_model variant, and letting the caller discard 
> the model if they don't need it.
> - I'm not a fan of virtual and concrete fields, though I have to admit I'm 
> not sure how they're different, especially in the context of different 
> implementations.
> - Not sure that m2m should be differentiated from related.
> - init_name_map should be an implementation detail.
> - normalize_together should be an implementation detail.
>
> Regards,
>
> Josh
>
> On Saturday, 24 May 2014 05:05:02 UTC+10, Daniel Pyrathon wrote:
>>
>> Hi all,
>>
>> In the last days I have built a documentation of the current state of 
>> Options. Based on feedback and prototyping I have thought of a potential 
>> interface for _meta that can solve the issues currently present, such as 
>> redundancy (in code and in caching systems). The interface has also been 
>> thought to be maintainable and is a base that can be used to create custom 
>> meta stores.
>> Obviously this is far from perfect, It will need many iterations and 
>> maybe it is too complex. I would really love to gain as much feedback as 
>> possible so it can be discussed with Russell and the community on Monday.
>>
>> The documentation of _meta can be found here: 
>> https://github.com/PirosB3/django/blob/meta_documentation/docs/ref/models/meta.txt
>> I will be refining the document in the next days, I will also be 
>> publishing the docs on a webserver and will be linking a URL soon.
>>
>> My proposal has been published here:
>> https://gist.github.com/PirosB3/371704ed40ed093d5a82
>> In the next days I will be iterating over the feedback gained, and based 
>> on one very interesting suggestion on IRC, I will try to see how my API 
>> syntax looks in modelforms.py.
>>
>> As said previously, and feedback is greatly appreciated.
>>
>> Hi from Pycon IT!
>>
>> Daniel Pyrathon
>>
>> On Tuesday, May 20, 2014 3:25:45 PM UTC+2, Josh Smeaton wrote:
>>>
>>> Best of luck!
>>>
>>> On Tuesday, 20 May 2014 03:56:06 UTC+10, Daniel Pyrathon wrote:

 Hi All,

 Today I will be starting my weekly updates on my SoC project: 
 refactoring Meta to a stable API. For anyone who missed out, you will be 
 able to view it here: 
 https://docs.google.com/document/d/1yp2_skqkxyrc0egdRv6ofnRGCI9nmvxDFBkCXgy0Jwo/edit

 This week is the first official week of SoC. Me and my mentor (Russell) 
 are initially approaching the work in the following way:

- *Document the existing Meta API*
For each endpoint, document the following:
  - Input parameters and return type
  - Caching pattern used
  - Where it's called from (internally and externally to Meta)
  - Why is it being called
  - When is it being called

- *Propose an initial refactor plan*
Once the documentation has been done, I should have a better idea 
of the current implementation. This will allow me to mock a proposed 
implementation that will be reviewed at my next update call, on Monday.

 My next update will be posted on Friday, just to make sure the 
 community is informed of my progress. For any major updates that require 
 community approval, I will be creating separate threads.
 My name on the internet is pirosb3, so if you want to have a chat about 
 my progress feel free to contact me! The branch I am currently working on 
 is https://github.com/PirosB3/django/tree/meta_documentation

 Regards,
 Daniel Pyrathon

>>>
>> On Tuesday, May 20, 2014 3:25:45 PM UTC+2, Josh Smeaton wrote:
>>>
>>> Best of luck!
>>>
>>> On Tuesday, 20 May 2014 03:56:06 UTC+10, Daniel Pyrathon wrote:

 Hi All,

 Today I will be starting my weekly updates on my SoC project: 
 refactoring Meta to a stable API. For anyone who missed out, you will be 
 able to view it here: 
 https://docs.google.com/document/d/1yp2_skqkxyrc0egdRv6ofnRGCI9nmvx

Re: [GSOC] Weekly update

[Daniel Pyrathon]

Hi Chris,

Oh sorry about that! big typo over there. Modifying the gist.

Thanks,
Daniel Pyrathon

On Saturday, May 24, 2014 7:44:15 AM UTC+2, Chris Beaven wrote:
>
> Hi Daniel,
>
> The proposal looks interesting - I've only skimmed it so far but one 
> question: you mention User.get_model() several times -- do you mean 
> User.get_meta()?
>
> On Saturday, May 24, 2014 7:05:02 AM UTC+12, Daniel Pyrathon wrote:
>>
>> Hi all,
>>
>> In the last days I have built a documentation of the current state of 
>> Options. Based on feedback and prototyping I have thought of a potential 
>> interface for _meta that can solve the issues currently present, such as 
>> redundancy (in code and in caching systems). The interface has also been 
>> thought to be maintainable and is a base that can be used to create custom 
>> meta stores.
>> Obviously this is far from perfect, It will need many iterations and 
>> maybe it is too complex. I would really love to gain as much feedback as 
>> possible so it can be discussed with Russell and the community on Monday.
>>
>> The documentation of _meta can be found here: 
>> https://github.com/PirosB3/django/blob/meta_documentation/docs/ref/models/meta.txt
>> I will be refining the document in the next days, I will also be 
>> publishing the docs on a webserver and will be linking a URL soon.
>>
>> My proposal has been published here:
>> https://gist.github.com/PirosB3/371704ed40ed093d5a82
>> In the next days I will be iterating over the feedback gained, and based 
>> on one very interesting suggestion on IRC, I will try to see how my API 
>> syntax looks in modelforms.py.
>>
>> As said previously, and feedback is greatly appreciated.
>>
>> Hi from Pycon IT!
>>
>> Daniel Pyrathon
>>
>> On Tuesday, May 20, 2014 3:25:45 PM UTC+2, Josh Smeaton wrote:
>>>
>>> Best of luck!
>>>
>>> On Tuesday, 20 May 2014 03:56:06 UTC+10, Daniel Pyrathon wrote:

 Hi All,

 Today I will be starting my weekly updates on my SoC project: 
 refactoring Meta to a stable API. For anyone who missed out, you will be 
 able to view it here: 
 https://docs.google.com/document/d/1yp2_skqkxyrc0egdRv6ofnRGCI9nmvxDFBkCXgy0Jwo/edit

 This week is the first official week of SoC. Me and my mentor (Russell) 
 are initially approaching the work in the following way:

- *Document the existing Meta API*
For each endpoint, document the following:
  - Input parameters and return type
  - Caching pattern used
  - Where it's called from (internally and externally to Meta)
  - Why is it being called
  - When is it being called

- *Propose an initial refactor plan*
Once the documentation has been done, I should have a better idea 
of the current implementation. This will allow me to mock a proposed 
implementation that will be reviewed at my next update call, on Monday.

 My next update will be posted on Friday, just to make sure the 
 community is informed of my progress. For any major updates that require 
 community approval, I will be creating separate threads.
 My name on the internet is pirosb3, so if you want to have a chat about 
 my progress feel free to contact me! The branch I am currently working on 
 is https://github.com/PirosB3/django/tree/meta_documentation

 Regards,
 Daniel Pyrathon

>>>
>> On Tuesday, May 20, 2014 3:25:45 PM UTC+2, Josh Smeaton wrote:
>>>
>>> Best of luck!
>>>
>>> On Tuesday, 20 May 2014 03:56:06 UTC+10, Daniel Pyrathon wrote:

 Hi All,

 Today I will be starting my weekly updates on my SoC project: 
 refactoring Meta to a stable API. For anyone who missed out, you will be 
 able to view it here: 
 https://docs.google.com/document/d/1yp2_skqkxyrc0egdRv6ofnRGCI9nmvxDFBkCXgy0Jwo/edit

 This week is the first official week of SoC. Me and my mentor (Russell) 
 are initially approaching the work in the following way:

- *Document the existing Meta API*
For each endpoint, document the following:
  - Input parameters and return type
  - Caching pattern used
  - Where it's called from (internally and externally to Meta)
  - Why is it being called
  - When is it being called

- *Propose an initial refactor plan*
Once the documentation has been done, I should have a better idea 
of the current implementation. This will allow me to mock a proposed 
implementation that will be reviewed at my next update call, on Monday.

 My next update will be posted on Friday, just to make sure the 
 community is informed of my progress. For any major updates that require 
 community approval, I will be creating separate threads.
 My name on the internet is pirosb3, so if you want to have a chat about 
 my progress feel free to contact me! The branch I am currently working on 
 i

Re: Why not Single Table Inheritance?

[Craig de Stigter]

> If you ignore STI, I think it is quite straightforward to solve this with 
a 
parent model class which adds a type field, and manager methods to add the 
select_related calls and "interpret" the type field properly; so I don't 
see an 
immediate need for inclusion in core. 

Well, you don't need select_related calls at all, if you're actually 
storing things in one table like "single-table inheritance" implies.

I too was surprised to find Django doesn't do this, and was unable to find 
a good third-party app that does it.

So I wrote my own: https://github.com/craigds/django-typed-models/

It works well and we have been using it in production for a couple years.

It does rely on a few hacks that Django doesn't officially support, like 
proxy models with their own fields, which has unfortunately been broken in 
django 1.7 . I'd love to see 
better support for this in Django core.


Regards
Craig de Stigter

On Thursday, 22 May 2014 21:02:48 UTC+12, Anssi Kääriäinen wrote:
>
> On 05/22/2014 11:13 AM, Shai Berger wrote: 
> >> Any thoughts on this idea? 
> >> 
> > Instinctively -- isn't it possible to achieve the same things today by 
> > overriding __new__ ? 
> My understanding is that achieving all the same things isn't possible. 
> The problem is that inside __new__ it is impossible to know if the call 
> to __new__ was made from database loading or from user code. It also 
> seems that it is impossible to alter the args and kwargs passed to 
> __init__(). In addition if one wants for some reason (speed, or not 
> invoking __setattr__) to assign values directly to the __dict__ of the 
> new class, then __new__() doesn't seem to offer any way to do that. 
>
> It is true that STI is likely possible with usage of __new__() as long 
> as you don't want to change the arguments to the __init__ call of the 
> created object. 
>
> As a side note I think direct assignment to __dict__ on model loading 
> would be a better design than the current __init__ call. For example 
> Django needs to do some pretty crazy stuff  in __init__() to support 
> deferred field loading. With direct __dict__ assignment deferred model 
> creation is trivial. Also, loading from the database is a form of 
> deserialization, and when deserializing you want to load the model as it 
> were saved. The way to do this is to avoid __init__, __setattr__ and 
> descriptor __set__ calls. To avoid those the values should be assigned 
> directly to the __dict__ of the object. This is also used by Python's 
> deserialization. Of course, thinking about this is mostly academic. 
> Changing the way model loading from database is done has severe 
> backwards compatibility issues. Even django-core relies on descriptor 
> calls in some case. As an example to_python() method of custom fields is 
> called through a descriptor. 
>
>   - Anssi 
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/ffb11c40-c231-48d5-898a-71684536f55e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [cx-oracle-users] cx_Oracle 5.1.3

[Josh Smeaton]

Thanks for pointing this out - I was just about to start a new project with 
oracle 12c and python 3.4, both of which are only now supported. Great 
timing!

On Sunday, 25 May 2014 16:20:21 UTC+10, Shai Berger wrote:
>
> Hi all, 
>
> This release of cx_Oracle includes fixes the segfaults encountered with 
> Python3, and includes significant performance improvements for 64-bit 
> platforms 
> (where long is 64-bits, so, specifically, Windows not included). 
>
> Strongly recommended for all Oracle users. 
>
> Have fun, 
> Shai. 
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/cd97c8f8-dd03-41e7-a5e4-383ebb0ec7b1%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.