Re: Adding a database-agnostic JSONField into Django

[Adam Johnson]

Dan we don't need a user-operated flag, instead we would introspect the
database version in use and have a database backend feature. For example
here's where the MySQL backend determines if the "over" clause is supported
for window functions:
https://github.com/django/django/blob/master/django/db/backends/mysql/features.py#L86

On Tue, 19 Feb 2019 at 19:45, Dan Davis  wrote:

> I'm for this.   My only advice is that only some versions of Oracle have a
> native JSON type. The oracle backend should probably use some query to
> determine whether the Oracle instance supports JSON field, or there could
> be a flag in OPTIONS about tihs.
>
> On Tuesday, February 19, 2019 at 7:44:40 AM UTC-5, nikesh...@nituk.ac.in
> wrote:
>>
>> Hello ,
>> I would like to work on this. I recently started working on django. I
>> want to participate in GSoC so this might work for me.
>> Would you mind helping me.
>>
>>>
>>> --
> You received this message because you are subscribed to the Google Groups
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at https://groups.google.com/group/django-developers.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/c308887d-bd65-4883-b7f1-65d6a36491d5%40googlegroups.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>


-- 
Adam

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAMyDDM2GeHwt%3D5mEQnW8gFoZdwHgfS4%2Bh-mGBoUgPvkbLdnqGQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Will Django ever CompositePrimaryKeys?

[Josh Smeaton]

To add - there's definitely appetite for this feature, but it's a difficult 
one, and no one has stepped up to do it.

There are DEP drafts that cover pieces:

https://github.com/django/deps/blob/master/draft/0191-composite-fields.rst
https://github.com/django/deps/blob/master/draft/0192-standalone-composite-fields.rst

Citus Data was looking to sponsor development for this feature, and still, 
no one with the technical capabilities was willing (or able) to step up: 
https://groups.google.com/forum/#!searchin/django-developers/composite|sort:date/django-developers/wakEPFMPiyQ/ke5OwgOPAQAJ

I would strongly advise not just running off and attempting something on 
your own. Look over the previous attempts and DEPs, and then document your 
way forward either by editing the DEP, and by initiating a thread on this 
list.

Cheers

On Wednesday, 20 February 2019 04:29:41 UTC+11, Dan Davis wrote:
>
> James,
>
> As a Django user I've had this problem often. My best practice ways to 
> handle this is as follows:
>
>
>- If the table is read-only, then create a database-level view that 
>manufactures a primary key by concatenating the primary key columns 
>together.   Lie to Django and say this combined column is the primary key, 
>and keep the Database-level view's model as managed = False at the Django 
>level.   However, keep the SQL for the view in your git repository, and 
>when it changes, build a manual migration - manage.py makemigration fooapp 
>--empty -n create_replace_bar_view.
>
>
>- If the table is read-write, then either (a) create a view with 
>instead of update/instead of insert triggers that manage an underlying 
>table, or (b) just add a unique ID and make the existing primary key a 
>unique together constraint and index.  With the trigger to add a new ID, 
>other users of that table shouldn't notice any issues.
>
>
>- Maybe my statement that Django should manage migrations to the 
>schema seems unworkable. However, one of the biggest gains you can get 
> from 
>Django with old schemas like this is to get their DDL into git. Even if 
> git 
>doesn't do it, make sure some code does it, and it isn't left unmanaged. 
>That's been a key challenge and opportunity for me.
>
>
> As a Django developer, I'm a bit green to volunteer to handle this issue, 
> but this is exactly the kind of issue that leads me to be a developer - I 
> see the ORM and the all-bells included nature of Django as the killer 
> combo.  I don't have to go outside the farm to have database migrations, 
> url routing, etc.  It is almost all there.  So, please ping back in 3 
> months to see whether I'm up to it.
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/f2a3663c-3d0a-4f97-9699-f016c6bc00cd%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Add an defer=True option for model fields

[Dan Davis]

What I mean by the below:
> I was not sure whether to tell him to implement a ModelManager with a
get_queryset() method that defers the field,

Of course this works, but I'm not going to maintain this code, and that
sort of sophistication creates a need for more sophisticated maintenance.


On Tue, Feb 19, 2019 at 12:43 PM Dan Davis  wrote:

> I have a developer who stores the binary copy of a file in his table.  In
> ColdFusion, this was acceptable, because he was writing every query by
> hand, and could simply exclude that field.  However, with the Django ORM it
> is a bit of a problem.   The primary table he uses is just for the file,
> and has a file_name, file_type, file_size, and BinaryField.
>
> The problem is that he has a database-level view that incorporates this
> field, and it may be that he needs to keep this because other schemas in
> our big-office Oracle use the view as an exported synonym.
>
> What I advised him to do was to take the BinaryField out of the
> database-level view, to protect the ORM from reading these large files into
> memory, as in:
>
>  [obj for obj in LicensesDBView.objects.all()]
>
> Or, if he cannot do that, to simply defer the field:
>
>  [obj for obj in
> LicensesDBView.objects.defer('scanned_license').all()]
>
> I was not sure whether to tell him to implement a ModelManager with a
> get_queryset() method that defers the field, but it made me wonder whether
> we should have a concept of an "initially deferred" field.
> That is, this is a field that starts deferred, and can be pulled into the
> select using a values iterator or a call to only() or defer(), e.g. the one
> that cancels prior defers.   The concept of "initially deferred" fields
> would certainly require a new queryset method, such as "nodefer" which is
> sort of like only but doesn't cause only those fields to load, or rather
> defer could accept a syntax like defer('-scanned_license') to cancel that
> previous deferred loading field.
>
> I'm afraid I probably don't understand all the implications of this
> feature, so I thought I'd bring it up on the list before filing any sort of
> issue. Its likely this has been discussed before; I cannot do a historical
> search all the time, especially when ancient history may not be today's
> read on this issue.
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at https://groups.google.com/group/django-developers.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/ee5c04e5-69d6-42f9-95ff-c01d553b24c1%40googlegroups.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAFzonYaFjKv%3DW7V4RpSrpLhpA5iWUjbYgSdH4-Pb6517BdKWTg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Adding a database-agnostic JSONField into Django

[Dan Davis]

I'm for this.   My only advice is that only some versions of Oracle have a 
native JSON type. The oracle backend should probably use some query to 
determine whether the Oracle instance supports JSON field, or there could 
be a flag in OPTIONS about tihs.

On Tuesday, February 19, 2019 at 7:44:40 AM UTC-5, nikesh...@nituk.ac.in 
wrote:
>
> Hello ,
> I would like to work on this. I recently started working on django. I want 
> to participate in GSoC so this might work for me.
> Would you mind helping me. 
>
>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/c308887d-bd65-4883-b7f1-65d6a36491d5%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Add an defer=True option for model fields

[Dan Davis]

I have a developer who stores the binary copy of a file in his table.  In 
ColdFusion, this was acceptable, because he was writing every query by 
hand, and could simply exclude that field.  However, with the Django ORM it 
is a bit of a problem.   The primary table he uses is just for the file, 
and has a file_name, file_type, file_size, and BinaryField.

The problem is that he has a database-level view that incorporates this 
field, and it may be that he needs to keep this because other schemas in 
our big-office Oracle use the view as an exported synonym.

What I advised him to do was to take the BinaryField out of the 
database-level view, to protect the ORM from reading these large files into 
memory, as in:

 [obj for obj in LicensesDBView.objects.all()] 

Or, if he cannot do that, to simply defer the field:

 [obj for obj in 
LicensesDBView.objects.defer('scanned_license').all()] 

I was not sure whether to tell him to implement a ModelManager with a 
get_queryset() method that defers the field, but it made me wonder whether 
we should have a concept of an "initially deferred" field.
That is, this is a field that starts deferred, and can be pulled into the 
select using a values iterator or a call to only() or defer(), e.g. the one 
that cancels prior defers.   The concept of "initially deferred" fields 
would certainly require a new queryset method, such as "nodefer" which is 
sort of like only but doesn't cause only those fields to load, or rather 
defer could accept a syntax like defer('-scanned_license') to cancel that 
previous deferred loading field.

I'm afraid I probably don't understand all the implications of this 
feature, so I thought I'd bring it up on the list before filing any sort of 
issue. Its likely this has been discussed before; I cannot do a historical 
search all the time, especially when ancient history may not be today's 
read on this issue.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/ee5c04e5-69d6-42f9-95ff-c01d553b24c1%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Will Django ever CompositePrimaryKeys?

[Dan Davis]

James,

As a Django user I've had this problem often. My best practice ways to 
handle this is as follows:


   - If the table is read-only, then create a database-level view that 
   manufactures a primary key by concatenating the primary key columns 
   together.   Lie to Django and say this combined column is the primary key, 
   and keep the Database-level view's model as managed = False at the Django 
   level.   However, keep the SQL for the view in your git repository, and 
   when it changes, build a manual migration - manage.py makemigration fooapp 
   --empty -n create_replace_bar_view.


   - If the table is read-write, then either (a) create a view with instead 
   of update/instead of insert triggers that manage an underlying table, or 
   (b) just add a unique ID and make the existing primary key a unique 
   together constraint and index.  With the trigger to add a new ID, other 
   users of that table shouldn't notice any issues.


   - Maybe my statement that Django should manage migrations to the schema 
   seems unworkable. However, one of the biggest gains you can get from Django 
   with old schemas like this is to get their DDL into git. Even if git 
   doesn't do it, make sure some code does it, and it isn't left unmanaged. 
   That's been a key challenge and opportunity for me.


As a Django developer, I'm a bit green to volunteer to handle this issue, 
but this is exactly the kind of issue that leads me to be a developer - I 
see the ORM and the all-bells included nature of Django as the killer 
combo.  I don't have to go outside the farm to have database migrations, 
url routing, etc.  It is almost all there.  So, please ping back in 3 
months to see whether I'm up to it.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/e2b080d8-3b0d-49bf-b0b7-d5f1883e98fa%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Will Django ever CompositePrimaryKeys?

[James Bennett]

On Tue, Feb 19, 2019 at 4:44 AM 'Lance Ellinghaus' via Django developers
(Contributions to Django itself)  wrote:

> Should I consider his statements to be the final statement from the Django
> core developers?
>
> You should take it as someone giving their opinion on the users list.

This is the current roster of "Django core":

https://www.djangoproject.com/foundation/teams/

As to your actual question, the main issues were also discussed in the
thread:

* Someone sufficiently motivated needs to sit down and write the code and
do it in a way that integrates well into Django
* That person needs to commit, or find someone willing to commit, to
maintaining this code in Djano

To date, nobody has stepped up and complete those things. If you'd like to
make an attempt because you need the feature, you're certainly welcome to,
and I don't think anyone will try to stop you (there would almost certainly
be constructive feedback on API design, but that's about guiding something
to completion, not stopping it).

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAL13Cg9e6ojROwOb%2BjFPdAsAQ5g1pQVOZ0%3DVwNbvY9ACri9uyw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Adding a database-agnostic JSONField into Django

[nikesh . cse17]

Hello ,
I would like to work on this. I recently started working on django. I want 
to participate in GSoC so this might work for me.
Would you mind helping me. 

On Thursday, June 23, 2016 at 3:27:07 PM UTC+5:30, Raphael Hertzog wrote:
>
> Hello, 
>
> in almost all projects I work on, I end up using a JSONField. Since 
> I value being able to run with any database, I'm not relying on 
> django.contrib.postgres.fields.JSONField. So I have been using 
> pypi's django-jsonfield maintained by Matthew Schinckel: 
> https://bitbucket.org/schinckel/django-jsonfield 
> (I have also packaged this for Debian) 
>
> I have recently discovered pypi's "jsonfield" maintained by Brad Jasper: 
> https://github.com/bradjasper/django-jsonfield 
>
> Both projects are very similar (and use the same python package name) and 
> both projects are actually looking for a new maintainer... since I rely on 
> something like this, I would be willing to try to merge the best of both 
> modules into a possible django.contrib.jsonfield or directly into the 
> core. 
>
> We could use this opportunity to let the newly-integrated field use 
> DjangoJSONEncoder by default (see recent discussion about this) and 
> django.contrib.postgres could register its additionals lookups into the 
> generic field (assuming we use "jsonb" as underlying type for postgresql). 
>
> What do you think of this? 
>
> If inclusion into Django is not desired, then maybe we could aim to 
> at least merge both of those projects in a single "blessed" third-party 
> module that could be maintained in 
> https://github.com/django/django-jsonfield? 
>
> Cheers, 
> -- 
> Raphaël Hertzog ◈ Writer/Consultant ◈ Debian Developer 
>
> Discover the Debian Administrator's Handbook: 
> → http://debian-handbook.info/get/ 
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/dc26a007-5fd0-4b9d-9193-3361ba4bf891%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Will Django ever CompositePrimaryKeys?

['Lance Ellinghaus' via Django developers (Contributions to Django itself)]

I have a number of legacy databases that I need to access from Django and 
they do not have a single field primary key. They have composite primary 
keys. There is no easy way to add the single field primary key because 
other programs create records and Django would not be the application that 
maintains and modifies these tables.

I asked about this on the Django Users list and Simone Federici said:

sincerely, nobody take care about that feature because that's needed just 
for legacy db. 

The number of applications with on legacy db is less than 0.01% of the 
django applications.

There are some workaround for that applications, for example use SQL 
Alchemy.

 

I implemented a CompositeKey Dajngo feature for Django 1.4.  It is 
completly working also for the Generic Keys, concatening escaped keys. 
There was also a draft to implement the indexes. However, Django core team 
is evolving the platform so quickly (good for the framework) and the 
library is not mantained anymore. On Dajngo 1.6 I had should rewrote it 
completly.

 
In the end, now I understand why the core team don't care about the 
feature. Forget it. 

Should I consider his statements to be the final statement from the Django 
core developers?

Thank you,

Lance Ellinghaus

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/01043d62-a18e-4741-bcef-b3bc0a58543a%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.