Re: Default to BigAutoField

[Tim Graham]

I'm not sure how this could work with migrations. In a sense, it would 
involve making the auto-generated primary key "swappable", including 
foreign keys that point to it. This sounds like a headache.

I haven't thought of a possible solution since Kenneth floated this idea in 
#django-dev yesterday.

On Friday, June 9, 2017 at 7:11:05 PM UTC-4, Curtis Maloney wrote:
>
> I know people hate settings, but what about one for auto id field type?
>
> It would let you handle backwards compatibility, uuid, and bigint...
>
> --
> C
>
>
> On 10 June 2017 5:42:42 AM AEST, Jacob Kaplan-Moss  > wrote:
>>
>> I think this would be a good improvement, and I'd like to see it. I've 
>> been bitten by integers overflowing at least twice I can remember in my 
>> career, which is two times too many.
>>
>> However, a major thing we'd have to work out is the upgrade path Consider 
>> a simple model:
>>
>> class Person(Model):
>> name = CharField()
>>
>> In Django 1.11, this actually generates a model with an integer `id` 
>> field. But in we change it, in Django vNext, that `id` field would "turn 
>> into" a bigint magically without the underlying table changes. That'd be 
>> confusing: you'd expect the model to be "fixed" by pugrading to vNext, but 
>> it wouldn't be. I think the migrations engine would detect this as a 
>> migration (?), so perhaps that's the path forward, but it could still be 
>> super-confusing. We've never shipped a release of Django that required a 
>> migration to _all_ your models.
>>
>> Have you thought about what the upgrade path should look like, Kenneth?
>>
>> Jacob
>>
>> On Fri, Jun 9, 2017 at 11:24 AM, Kenneth Reitz > > wrote:
>>
>>> Dear Django Dev,
>>>
>>>  
>>>
>>> At Heroku, we have the privilege of seeing an extremely broad range of 
>>> customers utilizing tools like Django to build their applications and 
>>> companies. One of the things that we’ve seen customers hit, time and time 
>>> again when using tools like Django, is integer overflows for primary keys. 
>>> Their application starts behaving unpredictably once they reach the 
>>> overflow, not even knowing such a constraint exists, and they often think 
>>> the problem is with their database provider, rather than with their schema. 
>>> Once they realize what is wrong, it’s a relatively trivial fix, but a 
>>> migration can take several hours to complete, which results in unacceptable 
>>> amounts of downtime.
>>>
>>>  
>>>
>>> Because of this, Heroku, as a company, would like to encourage bigints 
>>> as a sane reasonable default for primary keys for application models. If 
>>> the Django project agrees with this idea, that would mean that Django would 
>>> provide BigAutoField as the default for ‘id’ instead of AutoField. 
>>>
>>>  
>>>
>>> Rails made this change recently 
>>> , and has seen 
>>> success in doing so. 
>>>
>>>  
>>>
>>> I’m happy to provide the code to do this, but I wanted to discuss it 
>>> here before doing so, to hear what the general consensus was to the 
>>> proposal of such a change. 
>>>
>>>  
>>>
>>>  
>>>
>>> Pros:
>>>
>>>- 
>>>
>>>Users of Django, following the docs, won’t accidentally hit the int 
>>>overflow barrier. 
>>>- 
>>>
>>>Encourages best-practices from the beginning. 
>>>- 
>>>
>>>Bigints don’t take up much more storage than ints when using 
>>>Postgres. 
>>>- 
>>>
>>>In-line with other frameworks moving forward on this issue, like 
>>>Rails .
>>>
>>>  
>>>
>>> Cons:
>>>
>>>- 
>>>
>>>Backwards compatibility would need to be considered. 
>>>
>>>  
>>> Why not UUID?
>>>
>>>  
>>>
>>> I agree! I love using UUID for my primary keys, and I think a patch to 
>>> Django which provides an AutoUUIDField would be wonderful. However, there 
>>> are a few major drawbacks to making this the new default:
>>>
>>>  
>>>
>>>1. 
>>>
>>>It’s confusing to new users, would make onboarding process more 
>>>difficult. 
>>>2. 
>>>
>>>UUID is difficult to implement in MySQL. 
>>>3. 
>>>
>>>UUID has larger storage requirements. 
>>>4. 
>>>
>>>Incrementing IDs are actually useful. 
>>>
>>>  
>>>
>>>
>>> So, my proposal is to simply lift the int barrier to a bigint barrier 
>>> for new Django applications, and I think it will save a lot of developers a 
>>> lot of pain in the long run. 
>>>
>>>  
>>>
>>> Many thanks,
>>>
>>>  
>>>
>>> Kenneth Reitz
>>>
>>> Heroku Python
>>>
>>> -- 
>>> You received this message because you are subscribed to the Google 
>>> Groups "Django developers (Contributions to Django itself)" group.
>>> To unsubscribe from this group and stop receiving emails from it, send 
>>> an email to django-develop...@googlegroups.com .
>>> To post to this group, send email to django-d...@googlegroups.com 
>>> .
>>> 

Re: Default to BigAutoField

[Curtis Maloney]

I know people hate settings, but what about one for auto id field type?

It would let you handle backwards compatibility, uuid, and bigint...

--
C


On 10 June 2017 5:42:42 AM AEST, Jacob Kaplan-Moss  wrote:
>I think this would be a good improvement, and I'd like to see it. I've
>been
>bitten by integers overflowing at least twice I can remember in my
>career,
>which is two times too many.
>
>However, a major thing we'd have to work out is the upgrade path
>Consider a
>simple model:
>
>class Person(Model):
>name = CharField()
>
>In Django 1.11, this actually generates a model with an integer `id`
>field.
>But in we change it, in Django vNext, that `id` field would "turn into"
>a
>bigint magically without the underlying table changes. That'd be
>confusing:
>you'd expect the model to be "fixed" by pugrading to vNext, but it
>wouldn't
>be. I think the migrations engine would detect this as a migration (?),
>so
>perhaps that's the path forward, but it could still be super-confusing.
>We've never shipped a release of Django that required a migration to
>_all_
>your models.
>
>Have you thought about what the upgrade path should look like, Kenneth?
>
>Jacob
>
>On Fri, Jun 9, 2017 at 11:24 AM, Kenneth Reitz 
>wrote:
>
>> Dear Django Dev,
>>
>>
>>
>> At Heroku, we have the privilege of seeing an extremely broad range
>of
>> customers utilizing tools like Django to build their applications and
>> companies. One of the things that we’ve seen customers hit, time and
>time
>> again when using tools like Django, is integer overflows for primary
>keys.
>> Their application starts behaving unpredictably once they reach the
>> overflow, not even knowing such a constraint exists, and they often
>think
>> the problem is with their database provider, rather than with their
>schema.
>> Once they realize what is wrong, it’s a relatively trivial fix, but a
>> migration can take several hours to complete, which results in
>unacceptable
>> amounts of downtime.
>>
>>
>>
>> Because of this, Heroku, as a company, would like to encourage
>bigints as
>> a sane reasonable default for primary keys for application models. If
>the
>> Django project agrees with this idea, that would mean that Django
>would
>> provide BigAutoField as the default for ‘id’ instead of AutoField.
>>
>>
>>
>> Rails made this change recently
>> , and has seen
>success
>> in doing so.
>>
>>
>>
>> I’m happy to provide the code to do this, but I wanted to discuss it
>here
>> before doing so, to hear what the general consensus was to the
>proposal of
>> such a change.
>>
>>
>>
>>
>>
>> Pros:
>>
>>-
>>
>>Users of Django, following the docs, won’t accidentally hit the
>int
>>overflow barrier.
>>-
>>
>>Encourages best-practices from the beginning.
>>-
>>
>>Bigints don’t take up much more storage than ints when using
>Postgres.
>>-
>>
>>In-line with other frameworks moving forward on this issue, like
>Rails
>>.
>>
>>
>>
>> Cons:
>>
>>-
>>
>>Backwards compatibility would need to be considered.
>>
>>
>> Why not UUID?
>>
>>
>>
>> I agree! I love using UUID for my primary keys, and I think a patch
>to
>> Django which provides an AutoUUIDField would be wonderful. However,
>there
>> are a few major drawbacks to making this the new default:
>>
>>
>>
>>1.
>>
>>It’s confusing to new users, would make onboarding process more
>>difficult.
>>2.
>>
>>UUID is difficult to implement in MySQL.
>>3.
>>
>>UUID has larger storage requirements.
>>4.
>>
>>Incrementing IDs are actually useful.
>>
>>
>>
>>
>> So, my proposal is to simply lift the int barrier to a bigint barrier
>for
>> new Django applications, and I think it will save a lot of developers
>a lot
>> of pain in the long run.
>>
>>
>>
>> Many thanks,
>>
>>
>>
>> Kenneth Reitz
>>
>> Heroku Python
>>
>> --
>> You received this message because you are subscribed to the Google
>Groups
>> "Django developers (Contributions to Django itself)" group.
>> To unsubscribe from this group and stop receiving emails from it,
>send an
>> email to django-developers+unsubscr...@googlegroups.com.
>> To post to this group, send email to
>django-developers@googlegroups.com.
>> Visit this group at
>https://groups.google.com/group/django-developers.
>> To view this discussion on the web visit https://groups.google.com/d/
>> msgid/django-developers/6fe3401c-4404-4bd8-9d22-
>> 58df95cd1348%40googlegroups.com
>>
>
>> .
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>-- 
>You received this message because you are subscribed to the Google
>Groups "Django developers  (Contributions to Django itself)" group.
>To unsubscribe from this group and stop receiving emails 

Re: Default to BigAutoField

[Curtis Maloney]

I don't see it helping once you hit the problem, but like custom user its a 
recommendable setting for pre planning

--
C


On 10 June 2017 11:37:04 AM AEST, Tim Graham  wrote:
>I'm not sure how this could work with migrations. In a sense, it would 
>involve making the auto-generated primary key "swappable", including 
>foreign keys that point to it. This sounds like a headache.
>
>I haven't thought of a possible solution since Kenneth floated this
>idea in 
>#django-dev yesterday.
>
>On Friday, June 9, 2017 at 7:11:05 PM UTC-4, Curtis Maloney wrote:
>>
>> I know people hate settings, but what about one for auto id field
>type?
>>
>> It would let you handle backwards compatibility, uuid, and bigint...
>>
>> --
>> C
>>
>>
>> On 10 June 2017 5:42:42 AM AEST, Jacob Kaplan-Moss
>> > wrote:
>>>
>>> I think this would be a good improvement, and I'd like to see it.
>I've 
>>> been bitten by integers overflowing at least twice I can remember in
>my 
>>> career, which is two times too many.
>>>
>>> However, a major thing we'd have to work out is the upgrade path
>Consider 
>>> a simple model:
>>>
>>> class Person(Model):
>>> name = CharField()
>>>
>>> In Django 1.11, this actually generates a model with an integer `id`
>
>>> field. But in we change it, in Django vNext, that `id` field would
>"turn 
>>> into" a bigint magically without the underlying table changes.
>That'd be 
>>> confusing: you'd expect the model to be "fixed" by pugrading to
>vNext, but 
>>> it wouldn't be. I think the migrations engine would detect this as a
>
>>> migration (?), so perhaps that's the path forward, but it could
>still be 
>>> super-confusing. We've never shipped a release of Django that
>required a 
>>> migration to _all_ your models.
>>>
>>> Have you thought about what the upgrade path should look like,
>Kenneth?
>>>
>>> Jacob
>>>
>>> On Fri, Jun 9, 2017 at 11:24 AM, Kenneth Reitz >> > wrote:
>>>
 Dear Django Dev,

  

 At Heroku, we have the privilege of seeing an extremely broad range
>of 
 customers utilizing tools like Django to build their applications
>and 
 companies. One of the things that we’ve seen customers hit, time
>and time 
 again when using tools like Django, is integer overflows for
>primary keys. 
 Their application starts behaving unpredictably once they reach the
>
 overflow, not even knowing such a constraint exists, and they often
>think 
 the problem is with their database provider, rather than with their
>schema. 
 Once they realize what is wrong, it’s a relatively trivial fix, but
>a 
 migration can take several hours to complete, which results in
>unacceptable 
 amounts of downtime.

  

 Because of this, Heroku, as a company, would like to encourage
>bigints 
 as a sane reasonable default for primary keys for application
>models. If 
 the Django project agrees with this idea, that would mean that
>Django would 
 provide BigAutoField as the default for ‘id’ instead of AutoField. 

  

 Rails made this change recently 
 , and has seen 
 success in doing so. 

  

 I’m happy to provide the code to do this, but I wanted to discuss
>it 
 here before doing so, to hear what the general consensus was to the
>
 proposal of such a change. 

  

  

 Pros:

- 

Users of Django, following the docs, won’t accidentally hit the
>int 
overflow barrier. 
- 

Encourages best-practices from the beginning. 
- 

Bigints don’t take up much more storage than ints when using 
Postgres. 
- 

In-line with other frameworks moving forward on this issue, like
>
Rails .

  

 Cons:

- 

Backwards compatibility would need to be considered. 

  
 Why not UUID?

  

 I agree! I love using UUID for my primary keys, and I think a patch
>to 
 Django which provides an AutoUUIDField would be wonderful. However,
>there 
 are a few major drawbacks to making this the new default:

  

1. 

It’s confusing to new users, would make onboarding process more 
difficult. 
2. 

UUID is difficult to implement in MySQL. 
3. 

UUID has larger storage requirements. 
4. 

Incrementing IDs are actually useful. 

  


 So, my proposal is to simply lift the int barrier to a bigint
>barrier 
 for new Django applications, and I think it will save a lot of
>developers a 
 lot of pain in the long run. 

  

 Many thanks,

  

 Kenneth Reitz

Re: Default to BigAutoField

[Collin Anderson]

I might be wrong, but if the default changes, won't the migrations detect
it and migrate it just fine, including foreign keys?

All of my migrations have this:
('id', models.AutoField(auto_created=True, primary_key=True,
serialize=False, verbose_name='ID')),

So everyone would either need to manually specify the AutoField to keep the
old behavior, or run makemigrations to auto-generate migrations to
BigAutoField. This seems similar to increasing the max_length of
EmailField, user.username, and user.last_name, though would affect a lot
more models in this case.

(I'm not sure what I think about the setting idea.)

(While we're at it, maybe we could make it a (new) PositiveBigAutoField to
help out the mysql folks and close the oldest open ticket:
https://code.djangoproject.com/ticket/56 :)



On Fri, Jun 9, 2017 at 9:37 PM, Tim Graham  wrote:

> I'm not sure how this could work with migrations. In a sense, it would
> involve making the auto-generated primary key "swappable", including
> foreign keys that point to it. This sounds like a headache.
>
> I haven't thought of a possible solution since Kenneth floated this idea
> in #django-dev yesterday.
>
> On Friday, June 9, 2017 at 7:11:05 PM UTC-4, Curtis Maloney wrote:
>>
>> I know people hate settings, but what about one for auto id field type?
>>
>> It would let you handle backwards compatibility, uuid, and bigint...
>>
>> --
>> C
>>
>>
>> On 10 June 2017 5:42:42 AM AEST, Jacob Kaplan-Moss 
>> wrote:
>>>
>>> I think this would be a good improvement, and I'd like to see it. I've
>>> been bitten by integers overflowing at least twice I can remember in my
>>> career, which is two times too many.
>>>
>>> However, a major thing we'd have to work out is the upgrade path
>>> Consider a simple model:
>>>
>>> class Person(Model):
>>> name = CharField()
>>>
>>> In Django 1.11, this actually generates a model with an integer `id`
>>> field. But in we change it, in Django vNext, that `id` field would "turn
>>> into" a bigint magically without the underlying table changes. That'd be
>>> confusing: you'd expect the model to be "fixed" by pugrading to vNext, but
>>> it wouldn't be. I think the migrations engine would detect this as a
>>> migration (?), so perhaps that's the path forward, but it could still be
>>> super-confusing. We've never shipped a release of Django that required a
>>> migration to _all_ your models.
>>>
>>> Have you thought about what the upgrade path should look like, Kenneth?
>>>
>>> Jacob
>>>
>>> On Fri, Jun 9, 2017 at 11:24 AM, Kenneth Reitz 
>>> wrote:
>>>
 Dear Django Dev,



 At Heroku, we have the privilege of seeing an extremely broad range of
 customers utilizing tools like Django to build their applications and
 companies. One of the things that we’ve seen customers hit, time and time
 again when using tools like Django, is integer overflows for primary keys.
 Their application starts behaving unpredictably once they reach the
 overflow, not even knowing such a constraint exists, and they often think
 the problem is with their database provider, rather than with their schema.
 Once they realize what is wrong, it’s a relatively trivial fix, but a
 migration can take several hours to complete, which results in unacceptable
 amounts of downtime.



 Because of this, Heroku, as a company, would like to encourage bigints
 as a sane reasonable default for primary keys for application models. If
 the Django project agrees with this idea, that would mean that Django would
 provide BigAutoField as the default for ‘id’ instead of AutoField.



 Rails made this change recently
 , and has seen
 success in doing so.



 I’m happy to provide the code to do this, but I wanted to discuss it
 here before doing so, to hear what the general consensus was to the
 proposal of such a change.





 Pros:

-

Users of Django, following the docs, won’t accidentally hit the int
overflow barrier.
-

Encourages best-practices from the beginning.
-

Bigints don’t take up much more storage than ints when using
Postgres.
-

In-line with other frameworks moving forward on this issue, like
Rails .



 Cons:

-

Backwards compatibility would need to be considered.


 Why not UUID?



 I agree! I love using UUID for my primary keys, and I think a patch to
 Django which provides an AutoUUIDField would be wonderful. However, there
 are a few major drawbacks to making this the new default:



1.

It’s confusing to new users, would make 

Re: Allow extra blank rows for adding new records when using list_editable in admin change list view.

[John-Paul Jorissen]

I would love to see this change implemented as well. I have a few models 
that would benefit from having a quick entryline at the bottom of the 
list_display. 

Personally, I think that the first possibility, where only the items that 
are in the 'list_editable' tuple would be editable in the  blank row makes 
more sense; only fields that have been designated as list editable should 
be editable from that view, and all others would be read-only of the 
default value. This would make more sense for the sake of consistency, 
because there may be some fields that are intentionally excluded.

On Monday, January 9, 2017 at 11:26:34 AM UTC-6, Adonys wrote:
>
> This ticket #11574  is open 
> from many years ago. I'm interested to contribute with this new feature 
> but, if this ticket has a lot of time opened, and is very possible that 
> there are some reservations and consideration about this feature. Before 
> use time to build a solution, i think that will be useful discuss about 
> this topic with the community.
> Similar to `InlineModelAdmin`, I consider that this feature won't be 
> useful for all models, is really useful for models with few fields. But, 
> adding new records directly in the `changelist` would be util like the 
> feature `InlineModelAdmin`, for little models. For a big model won't be 
> impossible, but the admin form will be more usable. 
> I have an idea to complete this task and is the following:
>
>1. Add a new attribute to the class `ModelAdmin`, to specify how many 
>blank rows will be shown by default when the `changelist` is rendered. The 
>new attribute would be named `list_editable_extra`, as was proposed in the 
>ticket description. This attribute will be used basically to set as 
> `extra` 
>value in ModelAdmin.get_changelist_formset 
>
> 
>. 
>2. As was described in the ticket; "Adapt the `result_list` template 
>tag in `admin_list.py` to show the extra forms. This can be done by 
>calling `items_for_result` with an empty model for each empty form". I 
>tested that suggestion and works good. 
>3. Avoid to render the `checkbox` "action-select" for the blank rows 
>in the `changelist` action column, in order to avoid problems with 
>changelist's actions.
>4. Add functionality similar to `InlineModelAdmin`, in order to add or 
>to delete blank rows.
>5. Add tests for the solution, including `Selenium` tests.
>6. Add documentation about the new feature.
>
> I evaluated 2 possibilities to reach that, both possibilities with 
> different impact in the Django's code.
>
>- Render in the blank row as editable fields, only the `list_editable` 
>fields. With this solution we need to modify less code's lines to render 
>the blank rows. Is necessary to be clear in the documentation that this 
>feature only will be able if the `list_editable` has been set, and could 
> be 
>edited only the fields added to `list_editable` tuple.
>- Render in the blank row as editable fields, all `list_display` 
>fields. With this solution we need to make more adjusts in order to render 
>the blank rows successfully. Is necessary to be clear in the documentation 
>that this feature only will be able if the `list_display` has been set, 
> and 
>could be edited this time only the fields added to `list_display` tuple. 
>
> I know that there are some obstacles that I will go finding in the way to 
> reach this goal, but it's a simple resume of my idea. I'm open to better 
> ideas.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/e2fd9de0-1b75-4931-8d8c-4ac245e8cc28%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: On ASGI...

[Tom Christie]

Figure I may as well show the sort of thing I'm thinking wrt. a more 
constrained consumer callable interface...

* A callable, taking two arguments, 'message' & 'channels'
* Message being JSON-serializable python primitives.
* Channels being a dictionary of str:channel
* Channel instances expose `.send()`, `.receive()` and `.name` interfaces.

Extensions such as groups/statistics/flush would get expressed instead as 
channel interfaces,
eg. a chat example...

def ws_connected(message, channels):
channels['reply'].send({'accept': True})
channels['groups'].send({
'group': 'chat',
'add': channels['reply'].name
})

def ws_receive(message, channels):
channels['groups'].send({
'group': 'chat',
'send': message['text']
})

def ws_disconnect(message, channels):
channels['groups'].send({
'group': 'chat',
'discard': channels['reply'].name
})

My thinking at the moment is that there isn't any great way of supporting 
both asyncio and sync implementations under the same interface.
If you're in asyncio land, it makes sense to *only* expose awaitable 
channel operations as you don't ever want to be able to block the task pool.

I think the best you can really do is express two distinct modes of 
interface.

sync: (callable interface, blocking send/receive interface)
asyncio (coroutine interface, coroutine send/receive interface)

Presumably the equivalent would be true of eg. twisted.

(There's a couple of diff things you can do to bridge from the asyncio 
interface -> sync interface if that's useful)

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/c6cb3dc6-7c30-4eca-a389-326552ca4c36%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Default to BigAutoField

[Kenneth Reitz]

Dear Django Dev,

 

At Heroku, we have the privilege of seeing an extremely broad range of 
customers utilizing tools like Django to build their applications and 
companies. One of the things that we’ve seen customers hit, time and time 
again when using tools like Django, is integer overflows for primary keys. 
Their application starts behaving unpredictably once they reach the 
overflow, not even knowing such a constraint exists, and they often think 
the problem is with their database provider, rather than with their schema. 
Once they realize what is wrong, it’s a relatively trivial fix, but a 
migration can take several hours to complete, which results in unacceptable 
amounts of downtime.

 

Because of this, Heroku, as a company, would like to encourage bigints as a 
sane reasonable default for primary keys for application models. If the 
Django project agrees with this idea, that would mean that Django would 
provide BigAutoField as the default for ‘id’ instead of AutoField. 

 

Rails made this change recently 
, and has seen success 
in doing so. 

 

I’m happy to provide the code to do this, but I wanted to discuss it here 
before doing so, to hear what the general consensus was to the proposal of 
such a change. 

 

 

Pros:

   - 
   
   Users of Django, following the docs, won’t accidentally hit the int 
   overflow barrier. 
   - 
   
   Encourages best-practices from the beginning. 
   - 
   
   Bigints don’t take up much more storage than ints when using Postgres. 
   - 
   
   In-line with other frameworks moving forward on this issue, like Rails 
   .
   
 

Cons:

   - 
   
   Backwards compatibility would need to be considered. 
   
 
Why not UUID?

 

I agree! I love using UUID for my primary keys, and I think a patch to 
Django which provides an AutoUUIDField would be wonderful. However, there 
are a few major drawbacks to making this the new default:

 

   1. 
   
   It’s confusing to new users, would make onboarding process more 
   difficult. 
   2. 
   
   UUID is difficult to implement in MySQL. 
   3. 
   
   UUID has larger storage requirements. 
   4. 
   
   Incrementing IDs are actually useful. 
   
 


So, my proposal is to simply lift the int barrier to a bigint barrier for 
new Django applications, and I think it will save a lot of developers a lot 
of pain in the long run. 

 

Many thanks,

 

Kenneth Reitz

Heroku Python

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/6fe3401c-4404-4bd8-9d22-58df95cd1348%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Default to BigAutoField

[Jacob Kaplan-Moss]

I think this would be a good improvement, and I'd like to see it. I've been
bitten by integers overflowing at least twice I can remember in my career,
which is two times too many.

However, a major thing we'd have to work out is the upgrade path Consider a
simple model:

class Person(Model):
name = CharField()

In Django 1.11, this actually generates a model with an integer `id` field.
But in we change it, in Django vNext, that `id` field would "turn into" a
bigint magically without the underlying table changes. That'd be confusing:
you'd expect the model to be "fixed" by pugrading to vNext, but it wouldn't
be. I think the migrations engine would detect this as a migration (?), so
perhaps that's the path forward, but it could still be super-confusing.
We've never shipped a release of Django that required a migration to _all_
your models.

Have you thought about what the upgrade path should look like, Kenneth?

Jacob

On Fri, Jun 9, 2017 at 11:24 AM, Kenneth Reitz  wrote:

> Dear Django Dev,
>
>
>
> At Heroku, we have the privilege of seeing an extremely broad range of
> customers utilizing tools like Django to build their applications and
> companies. One of the things that we’ve seen customers hit, time and time
> again when using tools like Django, is integer overflows for primary keys.
> Their application starts behaving unpredictably once they reach the
> overflow, not even knowing such a constraint exists, and they often think
> the problem is with their database provider, rather than with their schema.
> Once they realize what is wrong, it’s a relatively trivial fix, but a
> migration can take several hours to complete, which results in unacceptable
> amounts of downtime.
>
>
>
> Because of this, Heroku, as a company, would like to encourage bigints as
> a sane reasonable default for primary keys for application models. If the
> Django project agrees with this idea, that would mean that Django would
> provide BigAutoField as the default for ‘id’ instead of AutoField.
>
>
>
> Rails made this change recently
> , and has seen success
> in doing so.
>
>
>
> I’m happy to provide the code to do this, but I wanted to discuss it here
> before doing so, to hear what the general consensus was to the proposal of
> such a change.
>
>
>
>
>
> Pros:
>
>-
>
>Users of Django, following the docs, won’t accidentally hit the int
>overflow barrier.
>-
>
>Encourages best-practices from the beginning.
>-
>
>Bigints don’t take up much more storage than ints when using Postgres.
>-
>
>In-line with other frameworks moving forward on this issue, like Rails
>.
>
>
>
> Cons:
>
>-
>
>Backwards compatibility would need to be considered.
>
>
> Why not UUID?
>
>
>
> I agree! I love using UUID for my primary keys, and I think a patch to
> Django which provides an AutoUUIDField would be wonderful. However, there
> are a few major drawbacks to making this the new default:
>
>
>
>1.
>
>It’s confusing to new users, would make onboarding process more
>difficult.
>2.
>
>UUID is difficult to implement in MySQL.
>3.
>
>UUID has larger storage requirements.
>4.
>
>Incrementing IDs are actually useful.
>
>
>
>
> So, my proposal is to simply lift the int barrier to a bigint barrier for
> new Django applications, and I think it will save a lot of developers a lot
> of pain in the long run.
>
>
>
> Many thanks,
>
>
>
> Kenneth Reitz
>
> Heroku Python
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at https://groups.google.com/group/django-developers.
> To view this discussion on the web visit https://groups.google.com/d/
> msgid/django-developers/6fe3401c-4404-4bd8-9d22-
> 58df95cd1348%40googlegroups.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAK8PqJELsQdpKEcg-a_iEKKwoYkoGmc8pikFM-hgG4g0CZE8XQ%40mail.gmail.com.
For more options, visit 

Re: Default to BigAutoField

[Tom Forbes]

How would this work with generic foreign keys as well? Those migrations
couldn't be automatic.

On 9 Jun 2017 20:43, "Jacob Kaplan-Moss"  wrote:

> I think this would be a good improvement, and I'd like to see it. I've
> been bitten by integers overflowing at least twice I can remember in my
> career, which is two times too many.
>
> However, a major thing we'd have to work out is the upgrade path Consider
> a simple model:
>
> class Person(Model):
> name = CharField()
>
> In Django 1.11, this actually generates a model with an integer `id`
> field. But in we change it, in Django vNext, that `id` field would "turn
> into" a bigint magically without the underlying table changes. That'd be
> confusing: you'd expect the model to be "fixed" by pugrading to vNext, but
> it wouldn't be. I think the migrations engine would detect this as a
> migration (?), so perhaps that's the path forward, but it could still be
> super-confusing. We've never shipped a release of Django that required a
> migration to _all_ your models.
>
> Have you thought about what the upgrade path should look like, Kenneth?
>
> Jacob
>
> On Fri, Jun 9, 2017 at 11:24 AM, Kenneth Reitz  wrote:
>
>> Dear Django Dev,
>>
>>
>>
>> At Heroku, we have the privilege of seeing an extremely broad range of
>> customers utilizing tools like Django to build their applications and
>> companies. One of the things that we’ve seen customers hit, time and time
>> again when using tools like Django, is integer overflows for primary keys.
>> Their application starts behaving unpredictably once they reach the
>> overflow, not even knowing such a constraint exists, and they often think
>> the problem is with their database provider, rather than with their schema.
>> Once they realize what is wrong, it’s a relatively trivial fix, but a
>> migration can take several hours to complete, which results in unacceptable
>> amounts of downtime.
>>
>>
>>
>> Because of this, Heroku, as a company, would like to encourage bigints as
>> a sane reasonable default for primary keys for application models. If the
>> Django project agrees with this idea, that would mean that Django would
>> provide BigAutoField as the default for ‘id’ instead of AutoField.
>>
>>
>>
>> Rails made this change recently
>> , and has seen
>> success in doing so.
>>
>>
>>
>> I’m happy to provide the code to do this, but I wanted to discuss it here
>> before doing so, to hear what the general consensus was to the proposal of
>> such a change.
>>
>>
>>
>>
>>
>> Pros:
>>
>>-
>>
>>Users of Django, following the docs, won’t accidentally hit the int
>>overflow barrier.
>>-
>>
>>Encourages best-practices from the beginning.
>>-
>>
>>Bigints don’t take up much more storage than ints when using
>>Postgres.
>>-
>>
>>In-line with other frameworks moving forward on this issue, like Rails
>>.
>>
>>
>>
>> Cons:
>>
>>-
>>
>>Backwards compatibility would need to be considered.
>>
>>
>> Why not UUID?
>>
>>
>>
>> I agree! I love using UUID for my primary keys, and I think a patch to
>> Django which provides an AutoUUIDField would be wonderful. However, there
>> are a few major drawbacks to making this the new default:
>>
>>
>>
>>1.
>>
>>It’s confusing to new users, would make onboarding process more
>>difficult.
>>2.
>>
>>UUID is difficult to implement in MySQL.
>>3.
>>
>>UUID has larger storage requirements.
>>4.
>>
>>Incrementing IDs are actually useful.
>>
>>
>>
>>
>> So, my proposal is to simply lift the int barrier to a bigint barrier for
>> new Django applications, and I think it will save a lot of developers a lot
>> of pain in the long run.
>>
>>
>>
>> Many thanks,
>>
>>
>>
>> Kenneth Reitz
>>
>> Heroku Python
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Django developers (Contributions to Django itself)" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to django-developers+unsubscr...@googlegroups.com.
>> To post to this group, send email to django-developers@googlegroups.com.
>> Visit this group at https://groups.google.com/group/django-developers.
>> To view this discussion on the web visit https://groups.google.com/d/ms
>> gid/django-developers/6fe3401c-4404-4bd8-9d22-58df95cd1348%
>> 40googlegroups.com
>> 
>> .
>> For more options, visit https://groups.google.com/d/optout.
>>
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to 

Re: Default to BigAutoField

[Kenneth Reitz]

My initial thought was to just have this apply to *new* Django applications, if 
that's possible.

Running this migration could take quite some time on some larger apps, and 
would require a non-trivial amount of time to execute. 

--
Kenneth Reitz

> On Jun 9, 2017, at 3:53 PM, Tom Forbes  wrote:
> 
> How would this work with generic foreign keys as well? Those migrations 
> couldn't be automatic.
> 
> On 9 Jun 2017 20:43, "Jacob Kaplan-Moss"  > wrote:
> I think this would be a good improvement, and I'd like to see it. I've been 
> bitten by integers overflowing at least twice I can remember in my career, 
> which is two times too many.
> 
> However, a major thing we'd have to work out is the upgrade path Consider a 
> simple model:
> 
> class Person(Model):
> name = CharField()
> 
> In Django 1.11, this actually generates a model with an integer `id` field. 
> But in we change it, in Django vNext, that `id` field would "turn into" a 
> bigint magically without the underlying table changes. That'd be confusing: 
> you'd expect the model to be "fixed" by pugrading to vNext, but it wouldn't 
> be. I think the migrations engine would detect this as a migration (?), so 
> perhaps that's the path forward, but it could still be super-confusing. We've 
> never shipped a release of Django that required a migration to _all_ your 
> models.
> 
> Have you thought about what the upgrade path should look like, Kenneth?
> 
> Jacob
> 
> On Fri, Jun 9, 2017 at 11:24 AM, Kenneth Reitz  > wrote:
> Dear Django Dev,
>  
> At Heroku, we have the privilege of seeing an extremely broad range of 
> customers utilizing tools like Django to build their applications and 
> companies. One of the things that we’ve seen customers hit, time and time 
> again when using tools like Django, is integer overflows for primary keys. 
> Their application starts behaving unpredictably once they reach the overflow, 
> not even knowing such a constraint exists, and they often think the problem 
> is with their database provider, rather than with their schema. Once they 
> realize what is wrong, it’s a relatively trivial fix, but a migration can 
> take several hours to complete, which results in unacceptable amounts of 
> downtime.
>  
> Because of this, Heroku, as a company, would like to encourage bigints as a 
> sane reasonable default for primary keys for application models. If the 
> Django project agrees with this idea, that would mean that Django would 
> provide BigAutoField as the default for ‘id’ instead of AutoField. 
>  
> Rails made this change recently 
> , and has seen success in 
> doing so. 
>  
> I’m happy to provide the code to do this, but I wanted to discuss it here 
> before doing so, to hear what the general consensus was to the proposal of 
> such a change. 
>  
>  
> Pros:
> Users of Django, following the docs, won’t accidentally hit the int overflow 
> barrier. 
> Encourages best-practices from the beginning. 
> Bigints don’t take up much more storage than ints when using Postgres. 
> In-line with other frameworks moving forward on this issue, like Rails 
> .
>  
> Cons:
> Backwards compatibility would need to be considered. 
>  
> Why not UUID?
>  
> I agree! I love using UUID for my primary keys, and I think a patch to Django 
> which provides an AutoUUIDField would be wonderful. However, there are a few 
> major drawbacks to making this the new default:
>  
> It’s confusing to new users, would make onboarding process more difficult. 
> UUID is difficult to implement in MySQL. 
> UUID has larger storage requirements. 
> Incrementing IDs are actually useful. 
>  
> 
> So, my proposal is to simply lift the int barrier to a bigint barrier for new 
> Django applications, and I think it will save a lot of developers a lot of 
> pain in the long run. 
>  
> Many thanks,
>  
> Kenneth Reitz
> Heroku Python
> 
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-developers+unsubscr...@googlegroups.com 
> .
> To post to this group, send email to django-developers@googlegroups.com 
> .
> Visit this group at https://groups.google.com/group/django-developers 
> .
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/django-developers/6fe3401c-4404-4bd8-9d22-58df95cd1348%40googlegroups.com
>  
> .
> For more 

Re: Default to BigAutoField

[Kenneth Reitz]

However, it should also be noted that those same larger applications are the 
ones that are likely to run into this problem eventually, so perhaps forcing 
the migration is the best path moving forward. 

Interested in hearing thoughts about this.

--
Kenneth Reitz

> On Jun 9, 2017, at 3:54 PM, Kenneth Reitz  wrote:
> 
> My initial thought was to just have this apply to *new* Django applications, 
> if that's possible.
> 
> Running this migration could take quite some time on some larger apps, and 
> would require a non-trivial amount of time to execute. 
> 
> --
> Kenneth Reitz
> 
>> On Jun 9, 2017, at 3:53 PM, Tom Forbes > > wrote:
>> 
>> How would this work with generic foreign keys as well? Those migrations 
>> couldn't be automatic.
>> 
>> On 9 Jun 2017 20:43, "Jacob Kaplan-Moss" > > wrote:
>> I think this would be a good improvement, and I'd like to see it. I've been 
>> bitten by integers overflowing at least twice I can remember in my career, 
>> which is two times too many.
>> 
>> However, a major thing we'd have to work out is the upgrade path Consider a 
>> simple model:
>> 
>> class Person(Model):
>> name = CharField()
>> 
>> In Django 1.11, this actually generates a model with an integer `id` field. 
>> But in we change it, in Django vNext, that `id` field would "turn into" a 
>> bigint magically without the underlying table changes. That'd be confusing: 
>> you'd expect the model to be "fixed" by pugrading to vNext, but it wouldn't 
>> be. I think the migrations engine would detect this as a migration (?), so 
>> perhaps that's the path forward, but it could still be super-confusing. 
>> We've never shipped a release of Django that required a migration to _all_ 
>> your models.
>> 
>> Have you thought about what the upgrade path should look like, Kenneth?
>> 
>> Jacob
>> 
>> On Fri, Jun 9, 2017 at 11:24 AM, Kenneth Reitz > > wrote:
>> Dear Django Dev,
>>  
>> At Heroku, we have the privilege of seeing an extremely broad range of 
>> customers utilizing tools like Django to build their applications and 
>> companies. One of the things that we’ve seen customers hit, time and time 
>> again when using tools like Django, is integer overflows for primary keys. 
>> Their application starts behaving unpredictably once they reach the 
>> overflow, not even knowing such a constraint exists, and they often think 
>> the problem is with their database provider, rather than with their schema. 
>> Once they realize what is wrong, it’s a relatively trivial fix, but a 
>> migration can take several hours to complete, which results in unacceptable 
>> amounts of downtime.
>>  
>> Because of this, Heroku, as a company, would like to encourage bigints as a 
>> sane reasonable default for primary keys for application models. If the 
>> Django project agrees with this idea, that would mean that Django would 
>> provide BigAutoField as the default for ‘id’ instead of AutoField. 
>>  
>> Rails made this change recently 
>> , and has seen success in 
>> doing so. 
>>  
>> I’m happy to provide the code to do this, but I wanted to discuss it here 
>> before doing so, to hear what the general consensus was to the proposal of 
>> such a change. 
>>  
>>  
>> Pros:
>> Users of Django, following the docs, won’t accidentally hit the int overflow 
>> barrier. 
>> Encourages best-practices from the beginning. 
>> Bigints don’t take up much more storage than ints when using Postgres. 
>> In-line with other frameworks moving forward on this issue, like Rails 
>> .
>>  
>> Cons:
>> Backwards compatibility would need to be considered. 
>>  
>> Why not UUID?
>>  
>> I agree! I love using UUID for my primary keys, and I think a patch to 
>> Django which provides an AutoUUIDField would be wonderful. However, there 
>> are a few major drawbacks to making this the new default:
>>  
>> It’s confusing to new users, would make onboarding process more difficult. 
>> UUID is difficult to implement in MySQL. 
>> UUID has larger storage requirements. 
>> Incrementing IDs are actually useful. 
>>  
>> 
>> So, my proposal is to simply lift the int barrier to a bigint barrier for 
>> new Django applications, and I think it will save a lot of developers a lot 
>> of pain in the long run. 
>>  
>> Many thanks,
>>  
>> Kenneth Reitz
>> Heroku Python
>> 
>> 
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "Django developers (Contributions to Django itself)" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to django-developers+unsubscr...@googlegroups.com 
>> .
>> To post to this group, send email to django-developers@googlegroups.com 
>>