Re: Proposed Field API additions

[Andrew Godwin]

On 11/06/12 10:27, Anssi Kääriäinen wrote:

> All of the above sounds good - my main worry was that if you subclass
> a field, then it will not get a rule match as the module path prefix
> will be different than the parent field's. I don't know if this is an
> issue even in South... But if the rules are part of the field API,
> then there is no worry. Are the rules even needed after the
> introduction of "get_init_args()"?.

Nope, the whole point of introducing this stuff is to get rid of the
rules. That will make me a happy man.

> The only possible problem with using the module path as the identifier
> is the inability for the user to replace removed fields with copied
> backwards compatibility code. As this is probably a non-issue (and
> possible to work around) I will surprise you all and not complain
> about this :)

I think that the case of a field being removed completely is rare enough
that this is acceptable (having to use a workaround). Most times a field
becomes unimportable it will likely have moved, in which case a good
number of apps would keep the old import working (as code probably
depends on it).

But yes, I'll stop now before you complain again :)

Andrew

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Proposed Field API additions

[Anssi Kääriäinen]

On 11 kesä, 11:49, Andrew Godwin  wrote:
> On 10/06/12 21:54, Anssi K ri inen wrote:
>
> > I agree the needed_fields.py idea was bad. One part of the idea was
> > that you would get immediately broken migrations if you remove a
> > field. Do all the migrations break if you have a broken field
> > reference, or only when you actually use a migration referencing the
> > field?
>
> In the new system all the migrations in the same app would immediately
> break (and it would print out a nice explanation showing exactly where
> the issue was) - that's a side-effect of deducing the current model
> state from the previous migrations, and probably better than only
> failing at some mysterious future point.
>
> > Is there anything else tied to the fields identifier than just how to
> > get the field class back when unfreezing? The rules in South seem to
> > reference the module path in some way which is why I am asking.
>
> Well, the module path is in there so you know where to get the field
> from - as Luke suggested, that's probably just as good a way of doing it
> as registration, and involves not introducing another namespace.
>
> South's rules also include provision for saying which fields are safe or
> unsafe to freeze - that's only because it has to enact this system via
> entirely third-party means. Once it's a part of the Field API, checking
> if it's there becomes a lot simpler.

All of the above sounds good - my main worry was that if you subclass
a field, then it will not get a rule match as the module path prefix
will be different than the parent field's. I don't know if this is an
issue even in South... But if the rules are part of the field API,
then there is no worry. Are the rules even needed after the
introduction of "get_init_args()"?.

The only possible problem with using the module path as the identifier
is the inability for the user to replace removed fields with copied
backwards compatibility code. As this is probably a non-issue (and
possible to work around) I will surprise you all and not complain
about this :)

 - Anssi

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Proposed Field API additions

[Andrew Godwin]

On 10/06/12 21:54, Anssi Kääriäinen wrote:
> I agree the needed_fields.py idea was bad. One part of the idea was
> that you would get immediately broken migrations if you remove a
> field. Do all the migrations break if you have a broken field
> reference, or only when you actually use a migration referencing the
> field?

In the new system all the migrations in the same app would immediately
break (and it would print out a nice explanation showing exactly where
the issue was) - that's a side-effect of deducing the current model
state from the previous migrations, and probably better than only
failing at some mysterious future point.

> Is there anything else tied to the fields identifier than just how to
> get the field class back when unfreezing? The rules in South seem to
> reference the module path in some way which is why I am asking.

Well, the module path is in there so you know where to get the field
from - as Luke suggested, that's probably just as good a way of doing it
as registration, and involves not introducing another namespace.

South's rules also include provision for saying which fields are safe or
unsafe to freeze - that's only because it has to enact this system via
entirely third-party means. Once it's a part of the Field API, checking
if it's there becomes a lot simpler.

Andrew

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Proposed Field API additions

[Anssi Kääriäinen]

On 9 kesä, 13:37, Andrew Godwin  wrote:
> Eugh, autogenerated python files full of imports? Plus, you'd
> potentially need more than one (for different migration versions), and I
> don't see what improvement this is over full paths, which is what Luke
> proposed, what South currently does, and it works reasonably well.
>
> Remember, if a path does get hilariously outdated then there'll be a new
> option to collapse old migrations into one new initial migration, wiping
> out the outdated references in the process.

I agree the needed_fields.py idea was bad. One part of the idea was
that you would get immediately broken migrations if you remove a
field. Do all the migrations break if you have a broken field
reference, or only when you actually use a migration referencing the
field?

Is there anything else tied to the fields identifier than just how to
get the field class back when unfreezing? The rules in South seem to
reference the module path in some way which is why I am asking.

> > I am afraid the unfreeze can't be made totally robust by better core
> > support. There simply can be no guarantees of having the code
> > dependencies around.
>
> Oh, I entirely agree, I'm just trying to improve it. There's going to be
> no solution that works 100%, and part of using any migrations library
> will be a documented set of limitations like this. However, they'll work
> for 99% of cases, and I think that's a big improvement.
>
> > I would like there to be alternative solution for those who want to
> > record just the upgrade SQL. It doesn't need freezed model states at
> > all, and is thus more robust. If there is core migrations support,
> > then there should be option for this, even if all the SQL needs to be
> > hand written.
>
> That'll definitely be there - South has never needed the frozen ORM, and
> this won't either. If you want, you can just use sql: and depend:
> stanzas in the new migration files, and write everything manually - and
> you'll still get the benefits, like state tracking, transaction
> management around DDL and dependency resoution. Complete raw SQL support
> was one of the main points on my design notes for this.

I'm glad to hear this.

 - Anssi

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Proposed Field API additions

[Andrew Godwin]

On 08/06/12 20:01, Anssi Kääriäinen wrote:
> On 8 kesä, 19:42, Carl Meyer  wrote:
>> Yuck. I am not at all convinced that this cure isn't worth than the
>> disease. In every case where Django has introduced flattened
>> pseudo-namespaces in place of Python's existing namespace system, I think
>> it's come back to bite us later.
>>
>> How bad is it really to do nothing here? If I understand correctly, it
>> would simply mean that you have to keep a Field class always importable
>> from the same location, or else manually fix the import location in older
>> migrations. Frankly, given how rarely I've seen this issue in practice with
>> South, I think this limitation is perfectly fine, and much, much, better
>> than introducing a whole new registration machinery and flattened
>> pseudo-namespace for Fields.
> 
> Why not just import the needed fields explicitly in "needed_fields.py"
> as part of the migrations - the file would be autogenerated by the
> migrations machinery.
>
> This isn't a big change to what currently happens in South - only
> change is that the import is done explicitly in a .py file, instead of
> writing the import path in the freeze file. But, this way it would be
> pretty obvious that the migrations need the fields in their original
> position. If something is deleted or moved you will get an import
> error. The user can fix it manually. You would still have problems if
> a field is updated in incompatible ways, but I don't see any way
> around that issue except of freezing the whole Python path for each
> migration.

Eugh, autogenerated python files full of imports? Plus, you'd
potentially need more than one (for different migration versions), and I
don't see what improvement this is over full paths, which is what Luke
proposed, what South currently does, and it works reasonably well.

Remember, if a path does get hilariously outdated then there'll be a new
option to collapse old migrations into one new initial migration, wiping
out the outdated references in the process.

> 
> I am afraid the unfreeze can't be made totally robust by better core
> support. There simply can be no guarantees of having the code
> dependencies around.
> 

Oh, I entirely agree, I'm just trying to improve it. There's going to be
no solution that works 100%, and part of using any migrations library
will be a documented set of limitations like this. However, they'll work
for 99% of cases, and I think that's a big improvement.

> I would like there to be alternative solution for those who want to
> record just the upgrade SQL. It doesn't need freezed model states at
> all, and is thus more robust. If there is core migrations support,
> then there should be option for this, even if all the SQL needs to be
> hand written.

That'll definitely be there - South has never needed the frozen ORM, and
this won't either. If you want, you can just use sql: and depend:
stanzas in the new migration files, and write everything manually - and
you'll still get the benefits, like state tracking, transaction
management around DDL and dependency resoution. Complete raw SQL support
was one of the main points on my design notes for this.

Andrew

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Proposed Field API additions

[Andrew Godwin]

On 08/06/12 17:42, Carl Meyer wrote:
> Hi Andrew,
> 
> On Thursday, June 7, 2012 11:17:51 AM UTC-6, Andrew Godwin wrote:
> 
>  - Requiring that all fields expose a method which says how to
> reconstruct them.
> 
> Essentially, it returns the positional and keyword arguments you would
> have to pass into the constructor to make the field again (as
> there's no
> way to get them directly). For those familiar with south_field_triple,
> it would be a bit like that.
> 
> There will have to be some restrictions on what can be returned as
> valid
> values - no values that are pre-made anonymous functions, for example -
> but that can be cleared up later.
> 
> 
> If you're allowing named callables, but not anonymous ones, that implies
> that you'll be looking up the named ones by import path? Does this not
> present the same issue as with Fields - if the callable moves, the
> migration breaks? Or am I misunderstanding?

See my reply to Jacob about the same situation - yes, that's the case,
but it's mitigated somewhat by an easy way of collapsing old migrations
(with bad callable references) down. This proposed one would be just
like how fields currently work in South.

> 
>  - Requiring all fields to be accessible by only their app label and
> class name/other unique name.
> 
> This means either having to register custom fields (like admin classes,
> for example), or requiring fields to live in a fields.py or fields
> package (like models and models.py). This is to provide for a
> less-fragile way of referring to them than their full module path
> (which
> might change based on project name or package location).
> 
> Neither of these two options is perfect - registration means a little
> more "boilerplate" code, while requiring them to be in a certain module
> is going to hurt apps that don't have them there already. For that
> reason, I prefer the registration approach - it will only be one extra
> line per field, it's a pattern already used for admin classes and
> filters/tags, and it will allow for field classes to be renamed while
> keeping the same registered name (if someone wants).
> 
> 
> Yuck. I am not at all convinced that this cure isn't worth than the
> disease. In every case where Django has introduced flattened
> pseudo-namespaces in place of Python's existing namespace system, I
> think it's come back to bite us later.
> 
> How bad is it really to do nothing here? If I understand correctly, it
> would simply mean that you have to keep a Field class always importable
> from the same location, or else manually fix the import location in
> older migrations. Frankly, given how rarely I've seen this issue in
> practice with South, I think this limitation is perfectly fine, and
> much, much, better than introducing a whole new registration machinery
> and flattened pseudo-namespace for Fields.

You're entirely correct that that's the restriction, and given that I'm
imposing precisely the same restrictions on default callables, you may
well be right here - after all, "Namespaces are one honking great idea
-- let's do more of those!".

I think, unless anyone has an objection I can't think of, we should do
what you suggest, and just use full paths; after all, third-party apps
will have to keep aliases to fields in the right place so imports don't
break - registration would just be giving them one more thing to add an
alias to.

Andrew

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Proposed Field API additions

[Andrew Godwin]

On 08/06/12 16:45, Jacob Kaplan-Moss wrote:
> Hi Andrew --
> 
> Generally I'm +1, and I think I see the point pretty clearly. Just a
> couple of questions:
> 
> On Thu, Jun 7, 2012 at 7:17 PM, Andrew Godwin  wrote:
>>  - Requiring that all fields expose a method which says how to
>> reconstruct them.
>>
>> Essentially, it returns the positional and keyword arguments you would
>> have to pass into the constructor to make the field again (as there's no
>> way to get them directly). For those familiar with south_field_triple,
>> it would be a bit like that.
>>
>> There will have to be some restrictions on what can be returned as valid
>> values - no values that are pre-made anonymous functions, for example -
>> but that can be cleared up later.
> 
> I'm not sure I follow this limitation -- you're not saying that things
> like `datetime.now` would be verboten, right? At least for defaults,
> callables are pretty common.

No, that'd be allowed (and already is in South) - it's just edge cases
like, for example, def'ing a function inside the method that returns the
arguments and then using that as a default - there's no way to serialise
that other than by using its code object.

The basic restriction will be that you're allowed callables, but they
must be importable by another module and that they're available as long
as the migration they're in exists. That second point will be helped by
a much easier way of collapsing old migrations and removing out-of-date
references.

> 
>>
>>  - Requiring all fields to be accessible by only their app label and
>> class name/other unique name.
>>
>> This means either having to register custom fields (like admin classes,
>> for example), or requiring fields to live in a fields.py or fields
>> package (like models and models.py). This is to provide for a
>> less-fragile way of referring to them than their full module path (which
>> might change based on project name or package location).
> 
> Can't this be done by auto-discovering subclasses of Field
> (Field.__subclasses__)?

It could - however, that would mean that we'd have to enforce unique
naming rules for fields inside an app (not really a problem), and that
people making fields inside a function or other weird situation might
have those fields detected (could cause some weird issues).

The other problem with that is the inability to change a field's class
name without breaking old migrations (whereas with a template-tag-like
registering approach you could optionally provide a different name to
register it as, thus allowing it to exist under two names).

Still, it's a hard balance to strike, really - registering is still a
little clunky, and we can't (yet) rely on things like class decorators
to make it nicer.

Andrew

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Proposed Field API additions

[Andrew Godwin]

On 08/06/12 16:16, Anssi Kääriäinen wrote:
> I did a little digging into South code, and I think I now understand
> the needs. Basically, when you migrate a model, you might need to read
> the database data by using the old model definitions. You can't use
> the currently installed model definition because it does not match the
> database state. So, you read in by using the unfreezed old model
> definition, do needed changes to data, do actual schema migrations,
> then continue to next freezed state, do changes to data, migrate
> schema and so on. Even if you don't need the ORM for data migrations,
> you might need the old field definition: how to generate a column's
> SQL for a field in multidb friendly way if you don't have the field
> available?

Precisely.

> The problem with the above approach is that if for example a field
> __init__ definition is changed (or the user deletes unneeded field
> code) there goes your ability to do migrations. The migrations do not
> contain all the needed data to do the migrations. You should have the
> fields available for your whole migration history, and the fields
> should not change in ways that break the unfreeze. This is also the
> reason why identifying fields uniquely is so important - to unfreeze,
> you need the original field class. If you can't get the original field
> definition back, your migration is nearly worthless.
> 
> The dependency on having the field code available is by design. This
> is somewhat fragile no matter what you do, because you might need
> historical code to be able to run your migrations. But, it seems there
> is no way around this if you want to have multidb capabilities for
> custom fields, or any kind of versioned ORM capability available.

Right. There are obviously limits to any approach, as (as I've seen
people do) you can generate models dynamically at runtime as the
application is loaded - that said, migrations are not going to be a
required feature, and so some limits on how you're allowed to arrange
your models seems sensible.

> 
> I favor the explicit registrations API with free format strings. The
> idea would be that the field author will do:
> migrations.register('mypackage:somefield', SomeField)
> If he needs to later do incompatible changes to the field, he could
> change the registration to
> migrations.register('mypackage:somefield:v2', SomeField)
> Likely field versioning isn't commonly needed. At some point old
> migrations lose their value, and incompatible changes in fields aren't
> that common to begin with. Still, if the registration string is just a
> string, then the ability to have the ':v2' in there comes for free.
> For example, Django's XMLField could be resurrected for migrations by
> downloading its code, and registering it to 'django:xmlfield' (or
> whatever the key was previously).

Yes, I'd say versioning is unnecessary, as you can just use a different
name - the new migrations stuff will also include a much easier way to
delete older migrations from the history and thus wipe out-of-date
references, while still keeping things the same in the database.

> 
> The __init__ args and kwargs needed for unfreeze should be asked
> directly from the Field, just as Andrew suggested in his original
> mail. If a field is registered to migrations, then it will need to
> provide "get init arguments" method.
> 
> I think I am finally up to speed with this discussion... In short: +1
> for field registering, -0 for automatic fields.py introspection, +1
> for "get init arguments".
> 
> To me it seems there would be room for different kind of migrations,
> too. For example, just record the raw SQL needed. This is the method I
> am using (manually) currently. This is totally multidb unfriendly, but
> it doesn't matter for some use cases. It seems South migrations files
> could be used in this way, too, but there isn't support for recording
> raw SQL changesets between current database state, and current
> models.py.

There's no way of doing a raw SQL method without extra code as Django
will never emit ALTER TABLE statements by itself (otherwise, we could
just capture it from the cursor during syncdb or something).

The new format will include provision for just having a big block of raw
SQL, though - so if someone wants to fall back to doing that it's pretty
easy.

Andrew

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Proposed Field API additions

[Andrew Godwin]

On 08/06/12 18:16, Michael Manfre wrote:
> 
> 
> On Thursday, June 7, 2012 4:16:12 PM UTC-4, Alex Ogier wrote:
> 
> This isn't particularly robust. The SQL string generated by a
> particular backend isn't considered part of any API, and might change
> formatting or semantics on minor updates. Some SQL is quite
> complicated and dubiously related to the particular field, such as
> extra INDEX or UNIQUE constraint statements. Certain backends elide
> some arguments, performing constraint checking in the application (the
> sqlite backend for example just drops many constraints). In general,
> you want the migrations to be DB-agnostic, and checking SQL output
> loses this feature.
> 
> 
> While the entire raw SQL generated by a field may be too unstable to
> inspect, the underlying SQL shouldn't be completely ignored. At the very
> least, the datatype needs to be inspected when freezing to detect if the
> rug has been pulled out from under the ORM.
> 
> A real world use case. Django-mssql uses the datetime datatype for
> datetime fields. Several years ago, Microsoft made the recommendation
> that all new work should use datetime2. A future version of django-mssql
> will switch to using datetime2. It would be great if the proposed
> migration framework handled that sort of switch, or at the very least
> displayed a warning about the need for manual user intervention.
> 
> The same type of problem could be encountered if a database backend
> becomes unsupported and users are forced to switch to another backend
> where the maintainer follows a different set of "best practices". There
> are many ways of storing the same data that are all completely valid.
> The most likely problem field will be IPAddressField due to varchar(15)
> vs. unsigned int.

I'm not sure it's going to be sensible to have the migration framework
change things when the apps themselves have not changed, only the
backend - Django already does a reasonable job of abstracting types, and
that will need to be slightly improved but not radically changed for
this work.

The situation where someone changes a database backend completely and
points it at the exact same database is going to be rare - far more
common (I suspect) is someone changing database server and database
backend in sync, and for that you DO want to ignore the SQL completely -
if I go from PostgreSQL to MySQL (heaven forbid) I don't want South
thinking it should still try and use an IP column type.

Andrew

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Proposed Field API additions

[Anssi Kääriäinen]

On 8 kesä, 19:42, Carl Meyer  wrote:
> Yuck. I am not at all convinced that this cure isn't worth than the
> disease. In every case where Django has introduced flattened
> pseudo-namespaces in place of Python's existing namespace system, I think
> it's come back to bite us later.
>
> How bad is it really to do nothing here? If I understand correctly, it
> would simply mean that you have to keep a Field class always importable
> from the same location, or else manually fix the import location in older
> migrations. Frankly, given how rarely I've seen this issue in practice with
> South, I think this limitation is perfectly fine, and much, much, better
> than introducing a whole new registration machinery and flattened
> pseudo-namespace for Fields.

Why not just import the needed fields explicitly in "needed_fields.py"
as part of the migrations - the file would be autogenerated by the
migrations machinery.

This isn't a big change to what currently happens in South - only
change is that the import is done explicitly in a .py file, instead of
writing the import path in the freeze file. But, this way it would be
pretty obvious that the migrations need the fields in their original
position. If something is deleted or moved you will get an import
error. The user can fix it manually. You would still have problems if
a field is updated in incompatible ways, but I don't see any way
around that issue except of freezing the whole Python path for each
migration.

I am afraid the unfreeze can't be made totally robust by better core
support. There simply can be no guarantees of having the code
dependencies around.

I would like there to be alternative solution for those who want to
record just the upgrade SQL. It doesn't need freezed model states at
all, and is thus more robust. If there is core migrations support,
then there should be option for this, even if all the SQL needs to be
hand written.

 - Anssi

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Proposed Field API additions

[Michael Manfre]

On Thursday, June 7, 2012 4:16:12 PM UTC-4, Alex Ogier wrote:
>
> This isn't particularly robust. The SQL string generated by a 
> particular backend isn't considered part of any API, and might change 
> formatting or semantics on minor updates. Some SQL is quite 
> complicated and dubiously related to the particular field, such as 
> extra INDEX or UNIQUE constraint statements. Certain backends elide 
> some arguments, performing constraint checking in the application (the 
> sqlite backend for example just drops many constraints). In general, 
> you want the migrations to be DB-agnostic, and checking SQL output 
> loses this feature. 
>
>
While the entire raw SQL generated by a field may be too unstable to 
inspect, the underlying SQL shouldn't be completely ignored. At the very 
least, the datatype needs to be inspected when freezing to detect if the 
rug has been pulled out from under the ORM.

A real world use case. Django-mssql uses the datetime datatype for datetime 
fields. Several years ago, Microsoft made the recommendation that all new 
work should use datetime2. A future version of django-mssql will switch to 
using datetime2. It would be great if the proposed migration framework 
handled that sort of switch, or at the very least displayed a warning about 
the need for manual user intervention.

The same type of problem could be encountered if a database backend becomes 
unsupported and users are forced to switch to another backend where the 
maintainer follows a different set of "best practices". There are many ways 
of storing the same data that are all completely valid. The most likely 
problem field will be IPAddressField due to varchar(15) vs. unsigned int.

Regards,
Michael Manfre

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/django-developers/-/OzJ6JJwJxsYJ.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Proposed Field API additions

[Carl Meyer]

Hi Andrew,

On Thursday, June 7, 2012 11:17:51 AM UTC-6, Andrew Godwin wrote:
>
>  - Requiring that all fields expose a method which says how to 
> reconstruct them. 
>
> Essentially, it returns the positional and keyword arguments you would 
> have to pass into the constructor to make the field again (as there's no 
> way to get them directly). For those familiar with south_field_triple, 
> it would be a bit like that. 
>
> There will have to be some restrictions on what can be returned as valid 
> values - no values that are pre-made anonymous functions, for example - 
> but that can be cleared up later. 
>

If you're allowing named callables, but not anonymous ones, that implies 
that you'll be looking up the named ones by import path? Does this not 
present the same issue as with Fields - if the callable moves, the 
migration breaks? Or am I misunderstanding?

 - Requiring all fields to be accessible by only their app label and 
> class name/other unique name. 
>
> This means either having to register custom fields (like admin classes, 
> for example), or requiring fields to live in a fields.py or fields 
> package (like models and models.py). This is to provide for a 
> less-fragile way of referring to them than their full module path (which 
> might change based on project name or package location). 
>
> Neither of these two options is perfect - registration means a little 
> more "boilerplate" code, while requiring them to be in a certain module 
> is going to hurt apps that don't have them there already. For that 
> reason, I prefer the registration approach - it will only be one extra 
> line per field, it's a pattern already used for admin classes and 
> filters/tags, and it will allow for field classes to be renamed while 
> keeping the same registered name (if someone wants). 
>

Yuck. I am not at all convinced that this cure isn't worth than the 
disease. In every case where Django has introduced flattened 
pseudo-namespaces in place of Python's existing namespace system, I think 
it's come back to bite us later. 

How bad is it really to do nothing here? If I understand correctly, it 
would simply mean that you have to keep a Field class always importable 
from the same location, or else manually fix the import location in older 
migrations. Frankly, given how rarely I've seen this issue in practice with 
South, I think this limitation is perfectly fine, and much, much, better 
than introducing a whole new registration machinery and flattened 
pseudo-namespace for Fields.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/django-developers/-/hTlDL9sLsJMJ.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Proposed Field API additions

[Marty Alchin]

On Fri, Jun 8, 2012 at 8:45 AM, Jacob Kaplan-Moss wrote:

> Can't this be done by auto-discovering subclasses of Field
> (Field.__subclasses__)?


Unfortunately, __subclasses__() doesn't work down through the whole
hierarchy, just one level deep. So unless we plan to walk the tree to find
them all, some other form of registration would probably be better.

-Marty

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Proposed Field API additions

[Jacob Kaplan-Moss]

Hi Andrew --

Generally I'm +1, and I think I see the point pretty clearly. Just a
couple of questions:

On Thu, Jun 7, 2012 at 7:17 PM, Andrew Godwin  wrote:
>  - Requiring that all fields expose a method which says how to
> reconstruct them.
>
> Essentially, it returns the positional and keyword arguments you would
> have to pass into the constructor to make the field again (as there's no
> way to get them directly). For those familiar with south_field_triple,
> it would be a bit like that.
>
> There will have to be some restrictions on what can be returned as valid
> values - no values that are pre-made anonymous functions, for example -
> but that can be cleared up later.

I'm not sure I follow this limitation -- you're not saying that things
like `datetime.now` would be verboten, right? At least for defaults,
callables are pretty common.

>
>  - Requiring all fields to be accessible by only their app label and
> class name/other unique name.
>
> This means either having to register custom fields (like admin classes,
> for example), or requiring fields to live in a fields.py or fields
> package (like models and models.py). This is to provide for a
> less-fragile way of referring to them than their full module path (which
> might change based on project name or package location).

Can't this be done by auto-discovering subclasses of Field
(Field.__subclasses__)?

Jacob

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Proposed Field API additions

[Anssi Kääriäinen]

n 8 kesä, 00:20, Andrew Godwin  wrote:
> On 07/06/12 21:56, Anssi K ri inen wrote:
>
> > Is the reason for this to be able to track changes to field by
> > checking if its init arguments have changed? Why is it not possible to
> > track changes by checking the SQL output the field will generate
> > instead? This is guaranteed to be a string, and should be readily
> > available for any field. I am just trying to get up to speed here...
>
> No, it is so a model can be instantiated for any point in time
> (essentially a versioned ORM). This is needed so that any custom
> python/ORM code in migrations see the models as they were when they were
> created, and thus won't get weirded out by the columns not matching.
>
> There was a decent explanation about why and how models are resurrected
> like this on south-users ages ago, but I think that post may have been
> lost to the voids of time.

I did a little digging into South code, and I think I now understand
the needs. Basically, when you migrate a model, you might need to read
the database data by using the old model definitions. You can't use
the currently installed model definition because it does not match the
database state. So, you read in by using the unfreezed old model
definition, do needed changes to data, do actual schema migrations,
then continue to next freezed state, do changes to data, migrate
schema and so on. Even if you don't need the ORM for data migrations,
you might need the old field definition: how to generate a column's
SQL for a field in multidb friendly way if you don't have the field
available?

The problem with the above approach is that if for example a field
__init__ definition is changed (or the user deletes unneeded field
code) there goes your ability to do migrations. The migrations do not
contain all the needed data to do the migrations. You should have the
fields available for your whole migration history, and the fields
should not change in ways that break the unfreeze. This is also the
reason why identifying fields uniquely is so important - to unfreeze,
you need the original field class. If you can't get the original field
definition back, your migration is nearly worthless.

The dependency on having the field code available is by design. This
is somewhat fragile no matter what you do, because you might need
historical code to be able to run your migrations. But, it seems there
is no way around this if you want to have multidb capabilities for
custom fields, or any kind of versioned ORM capability available.

I favor the explicit registrations API with free format strings. The
idea would be that the field author will do:
migrations.register('mypackage:somefield', SomeField)
If he needs to later do incompatible changes to the field, he could
change the registration to
migrations.register('mypackage:somefield:v2', SomeField)
Likely field versioning isn't commonly needed. At some point old
migrations lose their value, and incompatible changes in fields aren't
that common to begin with. Still, if the registration string is just a
string, then the ability to have the ':v2' in there comes for free.
For example, Django's XMLField could be resurrected for migrations by
downloading its code, and registering it to 'django:xmlfield' (or
whatever the key was previously).

The __init__ args and kwargs needed for unfreeze should be asked
directly from the Field, just as Andrew suggested in his original
mail. If a field is registered to migrations, then it will need to
provide "get init arguments" method.

I think I am finally up to speed with this discussion... In short: +1
for field registering, -0 for automatic fields.py introspection, +1
for "get init arguments".

To me it seems there would be room for different kind of migrations,
too. For example, just record the raw SQL needed. This is the method I
am using (manually) currently. This is totally multidb unfriendly, but
it doesn't matter for some use cases. It seems South migrations files
could be used in this way, too, but there isn't support for recording
raw SQL changesets between current database state, and current
models.py.

 - Anssi

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Proposed Field API additions

[Andrew Godwin]

On 07/06/12 21:56, Anssi Kääriäinen wrote:
> Is the reason for this to be able to track changes to field by
> checking if its init arguments have changed? Why is it not possible to
> track changes by checking the SQL output the field will generate
> instead? This is guaranteed to be a string, and should be readily
> available for any field. I am just trying to get up to speed here... 

No, it is so a model can be instantiated for any point in time
(essentially a versioned ORM). This is needed so that any custom
python/ORM code in migrations see the models as they were when they were
created, and thus won't get weirded out by the columns not matching.

There was a decent explanation about why and how models are resurrected
like this on south-users ages ago, but I think that post may have been
lost to the voids of time.

(Also worth pointing out: fields don't have just one SQL definition,
they can have many - not only for different database backends, but also
different ones for creation-with-the-table and being-added-later).

> As an idea, why not mimic the models.Model.__new__. You could have
> Field.__new__ which does the registration. A field could have
> app_label and "field name" as Meta (or just base level arguments). If
> these are not supplied, they will be generated by the __new__ from the
> field's class.__name__ and module. Once backwards compatibility period
> is over the meta variables are required.
>
> Any field which is never imported will never get registered, but I
> don't see that as a problem - if the field is not in use in the
> project, why should it need to be registered?

As some of the other posts argue, that's also quite fragile - I'm not
sure there's a guarantee all field classes will be unique (especially as
fields have been around for ages). It also needs metaclasses - you can't
do it in Field.__new__, you need to do it in the metaclasses' __new__,
and if there's one thing I want to avoid it's adding more metaclasses.

Andrew

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Proposed Field API additions

[Anssi Kääriäinen]

On Jun 7, 11:44 pm, Anssi Kääriäinen  wrote:
> I don't see the difference between doing the registration
> automatically by Field.__new__() and automatically registering
> anything living in fields.py or fields module.

OK, now I get it - you will see in which app the field lives by in
which app the fields.py lives, and thus know the correct app_label for
the field. Then class name would uniquely identify the field inside
that app, right?

 - Anssi

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Proposed Field API additions

[Anssi Kääriäinen]

On Jun 7, 11:16 pm, Alex Ogier  wrote:
> This isn't particularly robust. The SQL string generated by a
> particular backend isn't considered part of any API, and might change
> formatting or semantics on minor updates. Some SQL is quite
> complicated and dubiously related to the particular field, such as
> extra INDEX or UNIQUE constraint statements. Certain backends elide
> some arguments, performing constraint checking in the application (the
> sqlite backend for example just drops many constraints). In general,
> you want the migrations to be DB-agnostic, and checking SQL output
> loses this feature.

Ah, so you will save the current state somewhere, but not necessarily
the DB being migrated. And yes, I can see how upgrading Django from
1.4 to 1.4.1 could change the SQL in subtle ways...

If you track changes by __init__ args, wouldn't then 'label' change
mark the field as changed? Is this a problem?

> >>  - Requiring all fields to be accessible by only their app label and
> >> class name/other unique name.
>
> >> This means either having to register custom fields (like admin classes,
> >> for example), or requiring fields to live in a fields.py or fields
> >> package (like models and models.py). This is to provide for a
> >> less-fragile way of referring to them than their full module path (which
> >> might change based on project name or package location).
>
> > As an idea, why not mimic the models.Model.__new__. You could have
> > Field.__new__ which does the registration. A field could have
> > app_label and "field name" as Meta (or just base level arguments). If
> > these are not supplied, they will be generated by the __new__ from the
> > field's class.__name__ and module. Once backwards compatibility period
> > is over the meta variables are required.
>
> > Any field which is never imported will never get registered, but I
> > don't see that as a problem - if the field is not in use in the
> > project, why should it need to be registered?
>
> Adding additional invariants to Fields like "the (trailing module
> path, ClassName) tuple must be unique over all fields" is arguably
> backwards incompatible. Registering has several benefits. For example,
> it doesn't impose any constraints until you decide that you want to
> use migrations, at which point a nice error can be thrown: "Model Foo
> has an unknown field: `field_name` of type: `project.app.CustomField`.
> Please register this field with
> `django.db.migrations.register_field()` before creating migrations on
> the Foo model." Also, you can support third-party fields by
> registering them in your own modules if you need to instead of hacking
> on their code if they haven't updated it.

I don't see the difference between doing the registration
automatically by Field.__new__() and automatically registering
anything living in fields.py or fields module.

 - Anssi

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Proposed Field API additions

[Alex Ogier]

On Thu, Jun 7, 2012 at 3:56 PM, Anssi Kääriäinen
 wrote:
> On Jun 7, 8:17 pm, Andrew Godwin  wrote:
>> Hi everyone,
>>
>> As part of my planning for adding schema alteration/migrations into
>> Django proper, I need to make a few changes to Fields to allow for
>> better serialisation of model definitions (pretty much a requirement for
>> any change-detecting migrations system).
>>
>> In particular, I propose:
>>
>>  - Requiring that all fields expose a method which says how to
>> reconstruct them.
>>
>> Essentially, it returns the positional and keyword arguments you would
>> have to pass into the constructor to make the field again (as there's no
>> way to get them directly). For those familiar with south_field_triple,
>> it would be a bit like that.
>
> Is the reason for this to be able to track changes to field by
> checking if its init arguments have changed? Why is it not possible to
> track changes by checking the SQL output the field will generate
> instead? This is guaranteed to be a string, and should be readily
> available for any field. I am just trying to get up to speed here...
>

This isn't particularly robust. The SQL string generated by a
particular backend isn't considered part of any API, and might change
formatting or semantics on minor updates. Some SQL is quite
complicated and dubiously related to the particular field, such as
extra INDEX or UNIQUE constraint statements. Certain backends elide
some arguments, performing constraint checking in the application (the
sqlite backend for example just drops many constraints). In general,
you want the migrations to be DB-agnostic, and checking SQL output
loses this feature.

>>  - Requiring all fields to be accessible by only their app label and
>> class name/other unique name.
>>
>> This means either having to register custom fields (like admin classes,
>> for example), or requiring fields to live in a fields.py or fields
>> package (like models and models.py). This is to provide for a
>> less-fragile way of referring to them than their full module path (which
>> might change based on project name or package location).
>
> As an idea, why not mimic the models.Model.__new__. You could have
> Field.__new__ which does the registration. A field could have
> app_label and "field name" as Meta (or just base level arguments). If
> these are not supplied, they will be generated by the __new__ from the
> field's class.__name__ and module. Once backwards compatibility period
> is over the meta variables are required.
>
> Any field which is never imported will never get registered, but I
> don't see that as a problem - if the field is not in use in the
> project, why should it need to be registered?
>

Adding additional invariants to Fields like "the (trailing module
path, ClassName) tuple must be unique over all fields" is arguably
backwards incompatible. Registering has several benefits. For example,
it doesn't impose any constraints until you decide that you want to
use migrations, at which point a nice error can be thrown: "Model Foo
has an unknown field: `field_name` of type: `project.app.CustomField`.
Please register this field with
`django.db.migrations.register_field()` before creating migrations on
the Foo model." Also, you can support third-party fields by
registering them in your own modules if you need to instead of hacking
on their code if they haven't updated it.

Best,
Alex Ogier

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Proposed Field API additions

[Anssi Kääriäinen]

On Jun 7, 8:17 pm, Andrew Godwin  wrote:
> Hi everyone,
>
> As part of my planning for adding schema alteration/migrations into
> Django proper, I need to make a few changes to Fields to allow for
> better serialisation of model definitions (pretty much a requirement for
> any change-detecting migrations system).
>
> In particular, I propose:
>
>  - Requiring that all fields expose a method which says how to
> reconstruct them.
>
> Essentially, it returns the positional and keyword arguments you would
> have to pass into the constructor to make the field again (as there's no
> way to get them directly). For those familiar with south_field_triple,
> it would be a bit like that.

Is the reason for this to be able to track changes to field by
checking if its init arguments have changed? Why is it not possible to
track changes by checking the SQL output the field will generate
instead? This is guaranteed to be a string, and should be readily
available for any field. I am just trying to get up to speed here...

>  - Requiring all fields to be accessible by only their app label and
> class name/other unique name.
>
> This means either having to register custom fields (like admin classes,
> for example), or requiring fields to live in a fields.py or fields
> package (like models and models.py). This is to provide for a
> less-fragile way of referring to them than their full module path (which
> might change based on project name or package location).

As an idea, why not mimic the models.Model.__new__. You could have
Field.__new__ which does the registration. A field could have
app_label and "field name" as Meta (or just base level arguments). If
these are not supplied, they will be generated by the __new__ from the
field's class.__name__ and module. Once backwards compatibility period
is over the meta variables are required.

Any field which is never imported will never get registered, but I
don't see that as a problem - if the field is not in use in the
project, why should it need to be registered?

 - Anssi

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Proposed Field API additions

[Andrew Godwin]

On 07/06/12 20:14, Alex Gaynor wrote:
>
>
> On Thu, Jun 7, 2012 at 12:17 PM, Andrew Godwin  > wrote:
>
>
> In particular, I propose:
>
>  - Requiring that all fields expose a method which says how to
> reconstruct them.
>
> Essentially, it returns the positional and keyword arguments you would
> have to pass into the constructor to make the field again (as
> there's no
> way to get them directly). For those familiar with south_field_triple,
> it would be a bit like that.
>
>
> This sounds similar to pickling's __getinitargs__, is there anyway we
> can reuse some of:
> http://docs.python.org/library/pickle.html#object.__getinitargs__

Hmm, it's close, but by not allowing keywords it's going to be very
unsightly for something with optional arguments. Additionally, we're
going to need to restrict the output from any function like this to
values we can serialise as text (basically, repr plus a few custom
handlers for things like model instances), and getinitargs doesn't
enforce that either.

Andrew

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Proposed Field API additions

[Dalton Barreto]

2012/6/7 Andrew Ingram :
> On 7 June 2012 18:17, Andrew Godwin  wrote:
>> This means either having to register custom fields (like admin classes,
>> for example), or requiring fields to live in a fields.py or fields
>> package (like models and models.py). This is to provide for a
>> less-fragile way of referring to them than their full module path (which
>> might change based on project name or package location).
>>
>> Neither of these two options is perfect - registration means a little
>> more "boilerplate" code, while requiring them to be in a certain module
>> is going to hurt apps that don't have them there already. For that
>> reason, I prefer the registration approach - it will only be one extra
>> line per field, it's a pattern already used for admin classes and
>> filters/tags, and it will allow for field classes to be renamed while
>> keeping the same registered name (if someone wants).
>
> I prefer the registration approach because it's more explicit. But
> there's also the third option of using both, auto-register anything in
> fields.py but allow explicit registration for anything found
> elsewhere.
>

Continuing on the auto-register approach. Would be a good idea to have
just one more
option on settings.py that would make all fields auto-registered?
Despite being an "all or nothing"
option, I don't know if it's that common choose to auto-migrate just
part of the application database.

This would also make the path more smooth to people wanting to change
their app to be auto-migratable.

Thanks,

-- 
Dalton Barreto
http://daltonmatos.com

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Proposed Field API additions

[Alex Gaynor]

On Thu, Jun 7, 2012 at 12:17 PM, Andrew Godwin  wrote:

> Hi everyone,
>
> As part of my planning for adding schema alteration/migrations into
> Django proper, I need to make a few changes to Fields to allow for
> better serialisation of model definitions (pretty much a requirement for
> any change-detecting migrations system).
>
> In particular, I propose:
>
>  - Requiring that all fields expose a method which says how to
> reconstruct them.
>
> Essentially, it returns the positional and keyword arguments you would
> have to pass into the constructor to make the field again (as there's no
> way to get them directly). For those familiar with south_field_triple,
> it would be a bit like that.
>
>
This sounds similar to pickling's __getinitargs__, is there anyway we can
reuse some of:
http://docs.python.org/library/pickle.html#object.__getinitargs__

Alex


> There will have to be some restrictions on what can be returned as valid
> values - no values that are pre-made anonymous functions, for example -
> but that can be cleared up later.
>
>
>  - Requiring all fields to be accessible by only their app label and
> class name/other unique name.
>
> This means either having to register custom fields (like admin classes,
> for example), or requiring fields to live in a fields.py or fields
> package (like models and models.py). This is to provide for a
> less-fragile way of referring to them than their full module path (which
> might change based on project name or package location).
>
> Neither of these two options is perfect - registration means a little
> more "boilerplate" code, while requiring them to be in a certain module
> is going to hurt apps that don't have them there already. For that
> reason, I prefer the registration approach - it will only be one extra
> line per field, it's a pattern already used for admin classes and
> filters/tags, and it will allow for field classes to be renamed while
> keeping the same registered name (if someone wants).
>
> I'd appreciate feedback on these general ideas - a more concrete API
> proposal will come later along with details about how I plan to approach
> the rest of the problem, but this is one of the few direct changes to
> Django's core and so needs dicussion first.
>
> Andrew
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers" group.
> To post to this group, send email to django-developers@googlegroups.com.
> To unsubscribe from this group, send email to
> django-developers+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/django-developers?hl=en.
>
>


-- 
"I disapprove of what you say, but I will defend to the death your right to
say it." -- Evelyn Beatrice Hall (summarizing Voltaire)
"The people's good is the highest law." -- Cicero

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Proposed Field API additions

[Andrew Ingram]

On 7 June 2012 18:17, Andrew Godwin  wrote:
> This means either having to register custom fields (like admin classes,
> for example), or requiring fields to live in a fields.py or fields
> package (like models and models.py). This is to provide for a
> less-fragile way of referring to them than their full module path (which
> might change based on project name or package location).
>
> Neither of these two options is perfect - registration means a little
> more "boilerplate" code, while requiring them to be in a certain module
> is going to hurt apps that don't have them there already. For that
> reason, I prefer the registration approach - it will only be one extra
> line per field, it's a pattern already used for admin classes and
> filters/tags, and it will allow for field classes to be renamed while
> keeping the same registered name (if someone wants).

I prefer the registration approach because it's more explicit. But
there's also the third option of using both, auto-register anything in
fields.py but allow explicit registration for anything found
elsewhere.

- Andy

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Proposed Field API additions

[Andrew Godwin]

Hi everyone,

As part of my planning for adding schema alteration/migrations into
Django proper, I need to make a few changes to Fields to allow for
better serialisation of model definitions (pretty much a requirement for
any change-detecting migrations system).

In particular, I propose:

 - Requiring that all fields expose a method which says how to
reconstruct them.

Essentially, it returns the positional and keyword arguments you would
have to pass into the constructor to make the field again (as there's no
way to get them directly). For those familiar with south_field_triple,
it would be a bit like that.

There will have to be some restrictions on what can be returned as valid
values - no values that are pre-made anonymous functions, for example -
but that can be cleared up later.


 - Requiring all fields to be accessible by only their app label and
class name/other unique name.

This means either having to register custom fields (like admin classes,
for example), or requiring fields to live in a fields.py or fields
package (like models and models.py). This is to provide for a
less-fragile way of referring to them than their full module path (which
might change based on project name or package location).

Neither of these two options is perfect - registration means a little
more "boilerplate" code, while requiring them to be in a certain module
is going to hurt apps that don't have them there already. For that
reason, I prefer the registration approach - it will only be one extra
line per field, it's a pattern already used for admin classes and
filters/tags, and it will allow for field classes to be renamed while
keeping the same registered name (if someone wants).

I'd appreciate feedback on these general ideas - a more concrete API
proposal will come later along with details about how I plan to approach
the rest of the problem, but this is one of the few direct changes to
Django's core and so needs dicussion first.

Andrew

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.