Re: Default to BigAutoField

2017-08-31 Thread Adam Johnson
I agree with Tim. I also think the rename has potential to mess with historical migrations, or other uses of the field classes, since the output of deconstruct() will change the class name. On 31 August 2017 at 16:13, Tim Graham wrote: > Glancing at the PR, one thing I'm not sure about is renami

Re: Default to BigAutoField

2017-08-31 Thread Tim Graham
Glancing at the PR, one thing I'm not sure about is renaming AutoField to SmallAutoField. I think that's going to cause needless additional complexity and confusion among people who've worked with Django a long time. For example, you might think that "Small" there has a similar meaning as Small

Re: Default to BigAutoField

2017-08-27 Thread Adam Johnson
I don't think "primary key size" is something that's even within consideration for 99% of django apps. Sure a few more bytes are going to be wasted here, but you could argue the same that the default AutoField was already too big for most models that have 100's of instances and could use even a one

Re: Default to BigAutoField

2017-08-18 Thread Andrew Godwin
On Fri, Aug 18, 2017 at 5:43 AM, Markus Holtermann wrote: > > I'm don't fully agree with the approach. This essentially forces 3rd > party package authors to make the call about the primary key field size. > While for small to medium size projects BigAutoField is unlikely > required and only comes

Re: Default to BigAutoField

2017-08-18 Thread Markus Holtermann
Thanks for taking the effort to work on this, Kenneth! I'm don't fully agree with the approach. This essentially forces 3rd party package authors to make the call about the primary key field size. While for small to medium size projects BigAutoField is unlikely required and only comes with additi

Re: Default to BigAutoField

2017-08-17 Thread Andrew Godwin
To elaborate on the solution we eventually came up with - we default models to use a new BigAutoField that migrations will pick up on and generate migrations to alter columns to, but for safety reasons for those that don't read release notes, made the migration autodetector ask you if you want to m

Re: Default to BigAutoField

2017-08-17 Thread Kenneth Reitz
I have opened a pull request: https://github.com/django/django/pull/8924 Andrew and I came up with a good solution for migrations, together at DjangoCon. On Wednesday, June 14, 2017 at 7:36:36 AM UTC-7, Melvyn Sopacua wrote: > > On Friday 09 June 2017 15:59:50 Kenneth Reitz wrote: > > > Howeve

Re: Default to BigAutoField

2017-06-14 Thread Melvyn Sopacua
On Friday 09 June 2017 15:59:50 Kenneth Reitz wrote: > However, it should also be noted that those same larger applications > are the ones that are likely to run into this problem eventually, so > perhaps forcing the migration is the best path moving forward. Existing models are the problem. Then

Re: Default to BigAutoField

2017-06-12 Thread Ash Christopher
I really like this idea. We created a custom BigAutoField for our systems with sharded databases, and fast growing data. I understand the desire to fix it for new projects moving forward, but this smells a little like what happened with custom User models - it was introduced for new projects, b

Re: Default to BigAutoField

2017-06-11 Thread emorley
On Saturday, 10 June 2017 10:33:35 UTC+1, Claude Paroz wrote: > > Another idea is to leverage the system check framework (--deploy part) to > warn when the max id is over 50% of available range. > I was about to suggest the same. Seems like something worth doing regardless of whether we change t

Re: Default to BigAutoField

2017-06-10 Thread Claude Paroz
Le samedi 10 juin 2017 14:25:49 UTC+2, Curtis Maloney a écrit : > On 10/06/17 22:21, Claude Paroz wrote: > > Another idea would be to offer variants of models.Model which models > > could inherit from, like models.BigIntModel or models.UUIDModel. > > Ah, well... now you're talking. But then, y

Re: Default to BigAutoField

2017-06-10 Thread Curtis Maloney
On 10/06/17 22:21, Claude Paroz wrote: Le samedi 10 juin 2017 11:40:42 UTC+2, Curtis Maloney a écrit : Right, hence my point of having a global setting to say "the default auto-field is ..." I see, but this conforms to the pattern "use the same id field type for all models of my pro

Re: Default to BigAutoField

2017-06-10 Thread Claude Paroz
Le samedi 10 juin 2017 11:40:42 UTC+2, Curtis Maloney a écrit : > > Right, hence my point of having a global setting to say "the default > auto-field is ..." > > This would: > a) ... > I see, but this conforms to the pattern "use the same id field type for all models of my project". I'm not su

Re: Default to BigAutoField

2017-06-10 Thread Tom Forbes
I really like the idea of a global configurable setting. One problem with a setting is that it's not always changeable, which settings kind of imply (IMO). Going from int->bigint is always possible, but going backwards may not be, nor might going from int->guid. I attempted an int->guid migration

Re: Default to BigAutoField

2017-06-10 Thread Curtis Maloney
f) let MySQL users opt for PositiveBigAutoField if they want... On 10/06/17 19:40, Curtis Maloney wrote: Right, hence my point of having a global setting to say "the default auto-field is ..." This would: a) let people continue to use AutoField b) let people opt for BigAutoField c) let 3rd par

Re: Default to BigAutoField

2017-06-10 Thread Curtis Maloney
Right, hence my point of having a global setting to say "the default auto-field is ..." This would: a) let people continue to use AutoField b) let people opt for BigAutoField c) let 3rd party projects be agnostic d) let people use UUIDField(default=uuid.uuid4) e) possibly make it feasible to cha

Re: Default to BigAutoField

2017-06-10 Thread Claude Paroz
I think we should first discuss if it makes sense to set a BigAutoField by default for all tables. I'm not convinced of that yet. I looked at my projects with this perspective and found for example a case where I have many small lookup tables (containing between 2-20 entries) for which I know I

Re: Default to BigAutoField

2017-06-10 Thread Andrew Godwin
As long as you changed the actual field that ended up under the "id" column, then yes, migrations should detect it and apply all the changes during an upgrade, including to foreign keys. Generic foreign keys are another problem, however, not to mention the prospect of how many migrations and ALTER

Re: Default to BigAutoField

2017-06-09 Thread Collin Anderson
I might be wrong, but if the default changes, won't the migrations detect it and migrate it just fine, including foreign keys? All of my migrations have this: ('id', models.AutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')), So everyone would either need to manual

Re: Default to BigAutoField

2017-06-09 Thread Curtis Maloney
I don't see it helping once you hit the problem, but like custom user its a recommendable setting for pre planning -- C On 10 June 2017 11:37:04 AM AEST, Tim Graham wrote: >I'm not sure how this could work with migrations. In a sense, it would >involve making the auto-generated primary key "s

Re: Default to BigAutoField

2017-06-09 Thread Tim Graham
I'm not sure how this could work with migrations. In a sense, it would involve making the auto-generated primary key "swappable", including foreign keys that point to it. This sounds like a headache. I haven't thought of a possible solution since Kenneth floated this idea in #django-dev yesterd

Re: Default to BigAutoField

2017-06-09 Thread Curtis Maloney
I know people hate settings, but what about one for auto id field type? It would let you handle backwards compatibility, uuid, and bigint... -- C On 10 June 2017 5:42:42 AM AEST, Jacob Kaplan-Moss wrote: >I think this would be a good improvement, and I'd like to see it. I've >been >bitten by i

Re: Default to BigAutoField

2017-06-09 Thread Kenneth Reitz
However, it should also be noted that those same larger applications are the ones that are likely to run into this problem eventually, so perhaps forcing the migration is the best path moving forward. Interested in hearing thoughts about this. -- Kenneth Reitz > On Jun 9, 2017, at 3:54 PM, Ke

Re: Default to BigAutoField

2017-06-09 Thread Kenneth Reitz
My initial thought was to just have this apply to *new* Django applications, if that's possible. Running this migration could take quite some time on some larger apps, and would require a non-trivial amount of time to execute. -- Kenneth Reitz > On Jun 9, 2017, at 3:53 PM, Tom Forbes wrote:

Re: Default to BigAutoField

2017-06-09 Thread Tom Forbes
How would this work with generic foreign keys as well? Those migrations couldn't be automatic. On 9 Jun 2017 20:43, "Jacob Kaplan-Moss" wrote: > I think this would be a good improvement, and I'd like to see it. I've > been bitten by integers overflowing at least twice I can remember in my > care

Re: Default to BigAutoField

2017-06-09 Thread Jacob Kaplan-Moss
I think this would be a good improvement, and I'd like to see it. I've been bitten by integers overflowing at least twice I can remember in my career, which is two times too many. However, a major thing we'd have to work out is the upgrade path Consider a simple model: class Person(Model):

Default to BigAutoField

2017-06-09 Thread Kenneth Reitz
Dear Django Dev, At Heroku, we have the privilege of seeing an extremely broad range of customers utilizing tools like Django to build their applications and companies. One of the things that we’ve seen customers hit, time and time again when using tools like Django, is integer overflows f