Re: Composite Primary Keys

2016-02-22 Thread Michal Petrucha 
On Tue, Feb 16, 2016 at 05:38:12PM -0800, Cristiano Coelho wrote:
> Hello there,
> 
> What's the status for this? This 
> (https://code.djangoproject.com/wiki/MultipleColumnPrimaryKeys) is 3 years 
> old (last edit) and the links on it are even older. Googling around only 
> gave me some very old projects so it wasn't good neither.

Hi Cristiano,

I personally have spent in total perhaps about a year (or even more)
working on this (which includes two Summer of Code programs).
Unfortunately, my work was spread over more than three years, which
means a significant chunk of effort went into keeping up with changes
in Django and, ultimately, that's the part at which I failed. Twice.
(With a patch of this magnitude, all you need is to fall behind by
three weeks, and it gets very difficult and/or time-consuming to merge
new changes into it.)

The latest effort was launched by Thomas Stephenson, and resulted in
the following DEP:
https://github.com/django/deps/blob/master/draft/0191-composite-fields.rst,

There was even a pull request, but eventually, Thomas withdrew his
patch. You can find the corresponding thread on this mailing list
here:
https://groups.google.com/d/msgid/django-developers/ed326cce-6784-429b-869b-f6f66d3c77fd%40googlegroups.com

I wasn't around at that particular time, and I haven't yet found the
time to carefully read that thread and let it sink in, but I can give
a summary of my observations from 2013, when I last worked on this.

If you're interested, you can take a look at the wall of text I wrote
as my GSoC proposal in 2013: https://gist.github.com/konk/5408673.
You might also want to search this mailing list for previous threads
about this topic.


I seem to recall that just the most basic functionality of
CompositeField was the easy part. At that time (before migrations
landed), it meant mostly just making some parts of Django play nicer
with virtual fields (which was an almost entirely unexplored concept
in Django back then), some hackery for query lookups (since we didn't
have the nice API for custom lookups and transforms), and a fancy
class to represent composite values in model instances. Whipping up
something like this takes a few days, at most weeks, of focused work,
and you can have something sort of usable, with many sharp edges that
you have to keep in mind.

Things start to get ugly when you try to make CompositeField play nice
with the rest of the ORM, and the most obvious troublemaker is
ForeignKey. Composite ForeignKey is something that requires a big
refactor of the way related fields work, but even with a simple
ForeignKey to a unique column on a model with a composite primary key,
you'd start stumbling on errors with Python-level delete cascades,
prefetching and such -- those require an overhaul of how __in lookups
are handled. Then there are things like generic foreign keys, database
migrations, introspection, and potentially other features that would
need to be taken into account. Most of them could be postponed to a
later time, but some of them would most likely have to be part of the
initial patch.

Anyway, with the recent _meta refactor, reimplementing ForeignKey as a
virtual field should hopefully be easier now, although no one will
know for sure unless someone actually steps up and tries to do it. I
certainly hope that I'll be able to tackle this once again, but I
cannot make any promises.


Cheers,

Michal

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/20160222102153.GH880%40konk.org.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: Digital signature

Re: Composite Primary Keys

2016-02-17 Thread bliyanage 
Sounds cool, I think we would totally use that. 

On Wednesday, February 17, 2016 at 1:55:15 AM UTC-8, Roger Gammans wrote:
>
> Hi, 
>
>
> We've got some patches that we are using in production, against 
> 1.8 . Out long term aim would be to get them ready for a PR , but I they 
> are missing all sorts of things, such unit tests and migrations (we 
> don't use migrations at the moment - long story, so these haven't been 
> tested). The other thing which I think is  maybe is some of the 
> prefetch_related API, I've had difficulties with it and just worked 
> around it in my app. 
>
> And it really needs another user at least to kick the tires which we 
> aren't exercising - we mainly use because we have a lot of intersection 
> tables of natural / 'foreign' keys and a legacy model which didn't add a 
> serialized Pk for these. 
>
> I'd also like to know how interested the core developers would be in 
> merging some support for this. 
>
>
>
> On Tue, 2016-02-16 at 17:38 -0800, Cristiano Coelho wrote: 
> > Hello there, 
> > 
> > What's the status for this? This 
> > (https://code.djangoproject.com/wiki/MultipleColumnPrimaryKeys) is 3 
> > years old (last edit) and the links on it are even older. Googling 
> > around only gave me some very old projects so it wasn't good neither. 
> > 
> > -- 
> > You received this message because you are subscribed to the Google 
> > Groups "Django developers (Contributions to Django itself)" group. 
> > To unsubscribe from this group and stop receiving emails from it, send 
> > an email to django-develop...@googlegroups.com . 
> > To post to this group, send email to 
> > django-d...@googlegroups.com . 
> > Visit this group at https://groups.google.com/group/django-developers. 
> > To view this discussion on the web visit 
> > 
> https://groups.google.com/d/msgid/django-developers/b2873ece-39bb-4536-b23d-d988c7122204%40googlegroups.com.
>  
>
> > For more options, visit https://groups.google.com/d/optout. 
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/098addaa-a397-4016-913c-e1a3037d34e1%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Composite Primary Keys

2016-02-17 Thread Roger Gammans 
Hi,


We've got some patches that we are using in production, against
1.8 . Out long term aim would be to get them ready for a PR , but I they
are missing all sorts of things, such unit tests and migrations (we
don't use migrations at the moment - long story, so these haven't been
tested). The other thing which I think is  maybe is some of the
prefetch_related API, I've had difficulties with it and just worked
around it in my app. 

And it really needs another user at least to kick the tires which we
aren't exercising - we mainly use because we have a lot of intersection
tables of natural / 'foreign' keys and a legacy model which didn't add a
serialized Pk for these.

I'd also like to know how interested the core developers would be in
merging some support for this.



On Tue, 2016-02-16 at 17:38 -0800, Cristiano Coelho wrote:
> Hello there,
> 
> What's the status for this? This
> (https://code.djangoproject.com/wiki/MultipleColumnPrimaryKeys) is 3
> years old (last edit) and the links on it are even older. Googling
> around only gave me some very old projects so it wasn't good neither.
> 
> -- 
> You received this message because you are subscribed to the Google
> Groups "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to
> django-developers@googlegroups.com.
> Visit this group at https://groups.google.com/group/django-developers.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/b2873ece-39bb-4536-b23d-d988c7122204%40googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.


-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/1455702835.27233.10.camel%40gammascience.co.uk.
For more options, visit https://groups.google.com/d/optout.

Re: Composite primary keys

2011-03-31 Thread Michal Petrucha 
On Thu, Mar 31, 2011 at 09:54:49AM -0700, Christophe Pettus wrote:
> What concerns me about composite fields is that they seem to be a
> lot of Python machinery just to accomplish the goal of allowing this
> annotation.  If they were super-useful in their own right, that
> would be one thing, but I'm not sure that I see the utility of them
> absent indexes and foreign keys.  I'm also bothered, perhaps
> excessively, about having two different ways of getting at the same
> field in the model just to support this.

Just an observation, there already are two ways of getting at the same
field in case of primary keys...

> So, another proposal:
> 
> In the foreign key case, just extending the ForeignKey syntax to
> allow for multiple related fields makes the most sense:
> 
>   overThere = models.ForeignKey(OtherModel, to_field=('first_name', 
> 'last_name', ))
> 
> For indexes on the table for the model, include the declaration in
> the Meta class, since that's the obvious place to stick indexing:
> 
>   class SomeModel:
> 
>   class Meta:
>   primary_key = 'some_field'
>   indexes = ['some_field', 'some_other_field', ('field1', 
> '-field2', ), ]
>   raw_indexes = [ 'some_invariant_function(some_field)' ]
> 
> (This was proposed by someone else, and isn't original to me;
> apologies that I can't find the email to give credit.)
> 
> Of course, the existing syntax would still work as a shortcut for
> primary_key and indexes.
> 
> Thoughts?

One thing I'm missing in this proposal is the behavior of the pk
property. Since the primary key is a tuple, I can't imagine any other
representation in .pk than some flavor of a tuple. And to make things
like SomeModel.objects.get(pk=some_value) possible, we'd still have to
implement most of the functionality of a composite field.

Michal Petrucha


signature.asc
Description: Digital signature

Re: Composite primary keys

2011-03-31 Thread Christophe Pettus 

On Mar 21, 2011, at 12:20 PM, Jacob Kaplan-Moss wrote:

> I think we're talking slightly different concerns here: I'm mostly
> interested in the Python-side API, and to my eyes a composite field
> matches more closely what's happening on the Python side of things.

I agree 100%!  I think I'm just drawing a different conclusion from that point, 
which is that indexes are more metadata on the database rather than a critical 
part of the Python API: In an imaginary perfect database (like, say, the SQL 
spec envisions), we wouldn't need to talk about indexes as all.

The more I think about it, the less I like including this directly in the field 
declaration part of the model, including my Index type proposal.  It just 
doesn't seem to belong there.

What concerns me about composite fields is that they seem to be a lot of Python 
machinery just to accomplish the goal of allowing this annotation.  If they 
were super-useful in their own right, that would be one thing, but I'm not sure 
that I see the utility of them absent indexes and foreign keys.  I'm also 
bothered, perhaps excessively, about having two different ways of getting at 
the same field in the model just to support this.

So, another proposal:

In the foreign key case, just extending the ForeignKey syntax to allow for 
multiple related fields makes the most sense:

overThere = models.ForeignKey(OtherModel, to_field=('first_name', 
'last_name', ))

For indexes on the table for the model, include the declaration in the Meta 
class, since that's the obvious place to stick indexing:

class SomeModel:

class Meta:
primary_key = 'some_field'
indexes = ['some_field', 'some_other_field', ('field1', 
'-field2', ), ]
raw_indexes = [ 'some_invariant_function(some_field)' ]

(This was proposed by someone else, and isn't original to me; apologies that I 
can't find the email to give credit.)

Of course, the existing syntax would still work as a shortcut for primary_key 
and indexes.

Thoughts?

--
-- Christophe Pettus
   x...@thebuild.com

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Composite primary keys

2011-03-24 Thread Michal Petrucha 
On Mon, Mar 21, 2011 at 02:20:41PM -0500, Jacob Kaplan-Moss wrote:
> On Mon, Mar 21, 2011 at 2:33 AM, Christophe Pettus  wrote:
> > I'd like to make one more pitch for a slightly different
> > implementation here.  My concern with CompositeField isn't based
> > on the fact that it doesn't map one-to-one with a field in the
> > table; it's that it doesn't have any of the semantics that are
> > associated with a field.  In particular, it can't be:
> >
> > - Assigned to.
> > - Iterated over.
> > - Or even have a value.
> 
> Obviously there's no code here yet, so we don't know exactly. I'd also
> be -1 on an implementation of a CompositeField that didn't have those
> values. However, it's reasonably easy to come up with a CompositeField
> that is assignable, iterable, and has values. Here's the basics::
> 
[snippet]
> 
> Fields even sorta get saved correctly to the DB with just these few
> lines of code. Of course there's a lot missing here to correctly
> handle actual composite keys -- this tiny example won't work in
> querysets, for example -- but the basics of the Python-side behavior's
> right there.

This is exactly where I would start. One question though: should I use
namedtuple in here or should we try to keep compatibility with python
< 2.6? Maybe a module like django.utils.namedtuplecompat providing a
fallback implementation?

> > My suggestion is to create an Index type that can be included in a
> > class just like a field can.
> 
> I think we're talking slightly different concerns here: I'm mostly
> interested in the Python-side API, and to my eyes a composite field
> matches more closely what's happening on the Python side of things.
> Python's not generating an index, after all, so using something called
> "index" for compositing multiple attributes together seems weird to
> me. But at the DB level, "index" makes perfect sense. Thing is, we've
> always tried to make Django's APIs behave well in Python *first*, and
> then think about the DB concerns. And again, to me "composite field"
> matches more closely the behavior we want out of the Python side of
> things.
> 
> All that said, there's a lot to like about your Index proposal.
> Perhaps there's a way we can merge these two things together somehow?

The composite index could be achieved by the standard field option
Field.db_index applied to a CompositeField. This would still be
consistent with the rest of the API since it would not require any new
construct to do this thing.

To sum up, I believe the CompositeField could be a general solution to
both composite primary keys and composite indexes.

Michal Petrucha


signature.asc
Description: Digital signature

Re: Composite primary keys

2011-03-21 Thread Jacob Kaplan-Moss 
On Mon, Mar 21, 2011 at 2:33 AM, Christophe Pettus  wrote:
> I'd like to make one more pitch for a slightly different implementation here. 
>  My concern with CompositeField isn't based on the fact that it doesn't map 
> one-to-one with a field in the table; it's that it doesn't have any of the 
> semantics that are associated with a field.  In particular, it can't be:
>
> - Assigned to.
> - Iterated over.
> - Or even have a value.

Obviously there's no code here yet, so we don't know exactly. I'd also
be -1 on an implementation of a CompositeField that didn't have those
values. However, it's reasonably easy to come up with a CompositeField
that is assignable, iterable, and has values. Here's the basics::

class CompositeField(object):
def __init__(self, *fields):
self.fields = fields

def contribute_to_class(self, cls, name):
nt_name = "%s_%s" % (cls.__name__, name)
nt_fields = " ".join(f.name for f in self.fields)
self.nt = collections.namedtuple(nt_name, nt_fields)
setattr(cls, name, self)

def __get__(self, instance, owner):
if instance:
return self.nt._make(getattr(instance, f, None) for f
in self.nt._fields)
raise AttributeError("Composite fields only work on instances.")

def __set__(self, instance, value):
for (field, val) in zip(self.nt._fields, value):
setattr(instance, field, val)

It works, too::

class Person(models.Model):
first_name = models.CharField(max_length=100)
last_name = models.CharField(max_length=100)
name = CompositeField(first_name, last_name)

# ...

>>> p = Person(first_name="Jacob", last_name="KM")

>>> p.name
  : Person_name(first_name='Jacob', last_name='KM')

>>> p.name.last_name
  : 'KM'

>>> p.name = ("John", "Doe")

>>> p.last_name
  : 'Doe'

>>> for f in p.name:
..: print f
..:
..:
John
Doe

Fields even sorta get saved correctly to the DB with just these few
lines of code. Of course there's a lot missing here to correctly
handle actual composite keys -- this tiny example won't work in
querysets, for example -- but the basics of the Python-side behavior's
right there.

> My suggestion is to create an Index type that can be included in a class just 
> like a field can.

I think we're talking slightly different concerns here: I'm mostly
interested in the Python-side API, and to my eyes a composite field
matches more closely what's happening on the Python side of things.
Python's not generating an index, after all, so using something called
"index" for compositing multiple attributes together seems weird to
me. But at the DB level, "index" makes perfect sense. Thing is, we've
always tried to make Django's APIs behave well in Python *first*, and
then think about the DB concerns. And again, to me "composite field"
matches more closely the behavior we want out of the Python side of
things.

All that said, there's a lot to like about your Index proposal.
Perhaps there's a way we can merge these two things together somehow?

Jacob

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Composite primary keys

2011-03-21 Thread akaariai 
On Mar 21, 1:20 pm, Michal Petrucha  wrote:
> > My suggestion is to create an Index type that can be included in a
> > class just like a field can.  The example we've been using would
> > then look like:
>
> > class Foo(Model):
> >    x = models.FloatField()
> >    y = models.FloatField()
> >    a = models.ForeignKey(A)
> >    b = models.ForeignKey(B)
>
> >    coords = models.CompositeIndex((x, y))
> >    pair = models.CompositeIndex((a, b), primary_key=True)
>
> > We could have FieldIndex (the equivalent of the current
> > db_index=True), CompositeIndex, and RawIndex, for things like
> > expression indexes and other things that can be specified just as a
> > raw SQL string.
>
> > I think this is a much better contract to offer in the API than one
> > based on field which would have to throw exceptions left and right
> > for most of the common field operations.
>
> I don't see how ForeignKeys would be possible this way.
>

In much the same way:

class FooBar(Model):
a = models.ForeignKey(A)
b = models.ForeignKey(B)
pair = models.ForeignKey(Foo, fields=(a, b))

Note that this is very close to what SQL does. If you have a composite
unique index or composite foreign key you define the fields and then
the index / foreign key. Though I don't know how much value that
argument has in this discussion.

You could add some DRY and allow a shortcut:
class FooBar(Model):
pair = models.ForeignKey(Foo)
# a and b are created automatically.

Now, to make things work consistently pair should be a field. But on
the other hand when using a ModelForm, the pair should probably not be
a field of that form. This is more clear in an example having a (city,
state, country) primary key. These should clearly be separate fields
in a form.

In my opinion, if the composite structures are called fields or
something else isn't that important. There are cases where composite
structures behave like a field and some cases where they do not. The
main problem is how the composite structures should behave in
ModelForms and serialization, should they be assignable, how the
relate to model __init__ method, should they be in model fields
iterators, how they are used in QuerySets and so on. When these
questions are answered it is probably easier to answer if the
composite structures should be called fields or something else.

 - Anssi

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Composite primary keys

2011-03-21 Thread Michal Petrucha 
On Mon, Mar 21, 2011 at 12:33:01AM -0700, Christophe Pettus wrote:
> I'd like to make one more pitch for a slightly different
> implementation here.  My concern with CompositeField isn't based on
> the fact that it doesn't map one-to-one with a field in the table;
> it's that it doesn't have any of the semantics that are associated
> with a field.  In particular, it can't be:
> 
> - Assigned to.
> - Iterated over.
> - Or even have a value.
I disagree. The CompositeField would need to have a value to be able
to implement a ForeignKey pointing to a modedel with a composite
primary key.

The CompositeField itself would be just a proxy to the actual atomic
fields. You should be able to assign a tuple (or namedtuple) to it,
specifying the actual values for the fields. Similarly, you'll be able
to retrieve its value which is a tuple or a namedtuple.

This way, the following code would work for composite primary keys the
same way it works for simple keys:

class CompositeModel(models.Model):
a = models.IntegerField()
b = models.IntegerField()
key = models.CompositeField((a, b), primary_key=True)

class ReferencingModel(models.Model):
cm = models.ForeignKey(CompositeModel)

cminstance = CompositeModel.objects.get(something)
newref = ReferencingModel()
newref.cm = cminstance.pk

> My suggestion is to create an Index type that can be included in a
> class just like a field can.  The example we've been using would
> then look like:
> 
> class Foo(Model):
>x = models.FloatField()
>y = models.FloatField()
>a = models.ForeignKey(A)
>b = models.ForeignKey(B)
> 
>coords = models.CompositeIndex((x, y))
>pair = models.CompositeIndex((a, b), primary_key=True)
> 
> We could have FieldIndex (the equivalent of the current
> db_index=True), CompositeIndex, and RawIndex, for things like
> expression indexes and other things that can be specified just as a
> raw SQL string.
> 
> I think this is a much better contract to offer in the API than one
> based on field which would have to throw exceptions left and right
> for most of the common field operations.
I don't see how ForeignKeys would be possible this way.

Michal Petrucha


signature.asc
Description: Digital signature

Re: Composite primary keys

2011-03-21 Thread Johannes Dollinger 

Am 21.03.2011 um 08:33 schrieb Christophe Pettus:

> I'd like to make one more pitch for a slightly different implementation here. 
>  My concern with CompositeField isn't based on the fact that it doesn't map 
> one-to-one with a field in the table; it's that it doesn't have any of the 
> semantics that are associated with a field.  In particular, it can't be:
> 
> - Assigned to.
> - Iterated over.
> - Or even have a value.

You would be able to use composite fields normally (as in "normal django 
field"):

>>> foo = Foo.objects.create(coords=(0, 0))
>>> foo.coords
(0, 0)
>>> foo.coords = (4, 2)
>>> foo.coords.x # == foo.x == foo.coords[0]
4 

Sidenote:: Subclassing the default implementation of composite field values 
should be easy:

>>> type(foo.coords)

>>> foo.coords.length
4.4721359549995796
>>> foo.coords += foo.velocity

__
Johannes

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Composite primary keys

2011-03-21 Thread Christophe Pettus 
I'd like to make one more pitch for a slightly different implementation here.  
My concern with CompositeField isn't based on the fact that it doesn't map 
one-to-one with a field in the table; it's that it doesn't have any of the 
semantics that are associated with a field.  In particular, it can't be:

- Assigned to.
- Iterated over.
- Or even have a value.

My suggestion is to create an Index type that can be included in a class just 
like a field can.  The example we've been using would then look like:

class Foo(Model):
   x = models.FloatField()
   y = models.FloatField()
   a = models.ForeignKey(A)
   b = models.ForeignKey(B)

   coords = models.CompositeIndex((x, y))
   pair = models.CompositeIndex((a, b), primary_key=True)

We could have FieldIndex (the equivalent of the current db_index=True), 
CompositeIndex, and RawIndex, for things like expression indexes and other 
things that can be specified just as a raw SQL string.

I think this is a much better contract to offer in the API than one based on 
field which would have to throw exceptions left and right for most of the 
common field operations.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Composite primary keys

2011-03-17 Thread Michal Petrucha 
On Thu, Mar 17, 2011 at 09:33:43AM -0500, Jacob Kaplan-Moss wrote:
> On Wed, Mar 16, 2011 at 4:24 AM, Johannes Dollinger
>  wrote:
> > I would be nice if support for composite primary keys would be
> > implemented as a special case of general composite fields. There
> > would be no need for new Meta options:
> >
> > class Foo(Model):
> >    x = models.FloatField()
> >    y = models.FloatField()
> >    coords = models.CompositeField((x, y), db_index=True)
> >    a = models.ForeignKey(A)
> >    b = models.ForeignKey(B)
> >    pair = models.CompositeField((a, b), primary_key=True)
> >
> > A CompositeField descriptor would then return a namedtuple of its
> > values and would support queries:
> >
> >    filter(coords__x=42)
> >    filter(coords=(1,2))
> >
> > Adding the individual fields may be optional, e.g,
> > CompositeField((FloatField(), FloatField()), db_index=True).
> >
> > This has been proposed before:
> > http://groups.google.com/group/django-developers/browse_thread/thread/32f861c8bd5366a5
I must have overlooked this thread before...

> I like this quite a bit. Of all the various syntaxes proposed here so
> far, this is the first one that feels like it "fits" with the rest of
> Django, and the first one I'm +1 on.
I agree as well. This approach looks much cleaner from the design
perspective. At least the syntax is more consistent than having the
same information scattered throughout the individual fields and
several Meta attributes in different cases.

However, we'd either have a different API for specifying unique
constraints for sets of fields than for composite keys or we'd have
two options for the unique thing. (Or we'd lose backwards
compatibility.) The current API could be simulated by creating an
implicit CompositeField for each unique tuple with a reasonable
name.

A ForeignKey referencing a model with a primary CompositeField could
then act as a CompositeField itself, creating implicit fields unless
explicitly specified. Would this be a good idea? It would make it
easier to mess with the values of the fields directly, possibly
breaking the references to other rows, however, this is possible even
with the way it is now.

There is one thing though that's bothering me a little bit... At a
first glance this looks to me like a lot more work than my original
proposal. Now I'm not sure whether I should try to squeeze it all into
a single project with the primary key support and ForeignKey and
everything or rather just do the CompositeField with proper queryset
support and save the primary keys for a later time. Thoughts?

> I'm sensitive to Christophe's point that a "composite field" doesn't
> map to a relational concept very well, but quite frankly that ship has
> sailed. We've got ManyToManyFields, GenericForeignKeys, and once you
> branch out into the ecosystem you find TagFields, PickleFields (ugh)
> and so forth.
Just a note, my first approach at the composite ForeignKey field would
fall into this category anyway. I think we can't avoid that with this
kind of functionality...

Michal Petrucha


signature.asc
Description: Digital signature

Re: Composite primary keys

2011-03-17 Thread Łukasz Rekucki 
On 17 March 2011 16:18, Mike Axiak  wrote:
> Just to be clear, for this to be valid syntax doesn't this idea have to be
> written as::
>    class Foo(Model):
>       x = models.FloatField()
>       y = models.FloatField()
>       coords = models.CompositeField(('x', 'y'), db_index=True)
>       a = models.ForeignKey(A)
>       b = models.ForeignKey(B)
>       pair = models.CompositeField(('a', 'b'), primary_key=True)
> (Note the quotes around the field names.)

Actually, it works without the quotes as long as the fields are
defined in the same class before the CompositeField:
http://ideone.com/LPNzS

--
Łukasz Rekucki

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Composite primary keys

2011-03-17 Thread Mike Axiak 
Just to be clear, for this to be valid syntax doesn't this idea have to be
written as::

   class Foo(Model):
  x = models.FloatField()
  y = models.FloatField()
  coords = models.CompositeField(('x', 'y'), db_index=True)
  a = models.ForeignKey(A)
  b = models.ForeignKey(B)
  pair = models.CompositeField(('a', 'b'), primary_key=True)

(Note the quotes around the field names.)

Not that it matters too much, but I think any discussion of syntax should
have valid python.

Cheers,
Mike

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Composite primary keys

2011-03-17 Thread Jacob Kaplan-Moss 
On Wed, Mar 16, 2011 at 4:24 AM, Johannes Dollinger
 wrote:
> I would be nice if support for composite primary keys would be implemented as 
> a special case of general composite fields. There would be no need for new 
> Meta options:
>
> class Foo(Model):
>    x = models.FloatField()
>    y = models.FloatField()
>    coords = models.CompositeField((x, y), db_index=True)
>    a = models.ForeignKey(A)
>    b = models.ForeignKey(B)
>    pair = models.CompositeField((a, b), primary_key=True)
>
> A CompositeField descriptor would then return a namedtuple of its values and 
> would support queries:
>
>    filter(coords__x=42)
>    filter(coords=(1,2))
>
> Adding the individual fields may be optional, e.g, 
> CompositeField((FloatField(), FloatField()), db_index=True).
>
> This has been proposed before: 
> http://groups.google.com/group/django-developers/browse_thread/thread/32f861c8bd5366a5

I like this quite a bit. Of all the various syntaxes proposed here so
far, this is the first one that feels like it "fits" with the rest of
Django, and the first one I'm +1 on.

I'm sensitive to Christophe's point that a "composite field" doesn't
map to a relational concept very well, but quite frankly that ship has
sailed. We've got ManyToManyFields, GenericForeignKeys, and once you
branch out into the ecosystem you find TagFields, PickleFields (ugh)
and so forth.

Jacob

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Composite primary keys

2011-03-16 Thread Christophe Pettus 

On Mar 16, 2011, at 9:13 AM, Carl Meyer wrote:

> I'm not expressing an opinion one way or another on composite primary
> key syntax, but I don't agree here that a Django model "field" must
> map one-to-one to a database column.

That's fair, but a composite index lacks some of the characteristics of a field 
(assignability, for example).  Most DBs don't have functions that explicitly 
iterate over indexes, so such a thing isn't really readable, either.

It might be appealing to have a models.Index base class that represents an 
index on a table, and have db_index=True be a shortcut to creating one.  That 
might be more machinery than we want just for composite primary keys though.

--
-- Christophe Pettus
   x...@thebuild.com

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Composite primary keys

2011-03-16 Thread Carl Meyer 
On Mar 16, 11:43 am, Christophe Pettus  wrote:
> On Mar 16, 2011, at 2:24 AM, Johannes Dollinger wrote:
>
> > I would be nice if support for composite primary keys would be implemented 
> > as a special case of general composite fields.
>
> It's appealing, but the reality is that no existing back-end actually has 
> such an animal as a composite field.  In all of these cases, what we're 
> really creating is a composite index on a set of standard fields.  
> Introducing a more powerful index-creation syntax into Django isn't a bad 
> idea, but we shouldn't call it a "field" if it is not.

I'm not expressing an opinion one way or another on composite primary
key syntax, but I don't agree here that a Django model "field" must
map one-to-one to a database column. It already does not, in the case
of ManyToManyField, and at some point I would like to introduce
(irrespective of composite primary keys) a more general ORM
abstraction for composite fields (i.e. model Fields that map to more
than one database column) as a path to cleaning up the implementation
of GenericForeignKey.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Composite primary keys

2011-03-16 Thread Christophe Pettus 

On Mar 16, 2011, at 2:24 AM, Johannes Dollinger wrote:

> I would be nice if support for composite primary keys would be implemented as 
> a special case of general composite fields.

It's appealing, but the reality is that no existing back-end actually has such 
an animal as a composite field.  In all of these cases, what we're really 
creating is a composite index on a set of standard fields.  Introducing a more 
powerful index-creation syntax into Django isn't a bad idea, but we shouldn't 
call it a "field" if it is not.

--
-- Christophe Pettus
   x...@thebuild.com

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Composite primary keys

2011-03-16 Thread Johannes Dollinger 
I would be nice if support for composite primary keys would be implemented as a 
special case of general composite fields. There would be no need for new Meta 
options:

class Foo(Model):
x = models.FloatField()
y = models.FloatField()
coords = models.CompositeField((x, y), db_index=True)
a = models.ForeignKey(A)
b = models.ForeignKey(B)
pair = models.CompositeField((a, b), primary_key=True)

A CompositeField descriptor would then return a namedtuple of its values and 
would support queries:

filter(coords__x=42)
filter(coords=(1,2))

Adding the individual fields may be optional, e.g, 
CompositeField((FloatField(), FloatField()), db_index=True).

This has been proposed before: 
http://groups.google.com/group/django-developers/browse_thread/thread/32f861c8bd5366a5

__
Johannes

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Composite primary keys

2011-03-16 Thread Yishai Beeri 
On Wed, 16 Mar 2011 07:49:17 +0200, Michal Petrucha  
 wrote:



On Tue, Mar 15, 2011 at 09:15:29PM -0500, Javier Guerra Giraldez wrote:
On Tue, Mar 15, 2011 at 7:14 PM, Christophe Pettus   
wrote:

> A concern here is that composite indexes, like unique, are
> sensitive to the ordering of the fields, which means that the
> ordering of the fields in the class declaration becomes important.


This is the same reason a new Meta flag has been agreed upon in the
past. (That, however, does not mean it has to be that way.)


a simplistic proposal:

the order of the fields on a composite index is determined by the
exact value given to the primary_key argument.

that way, just setting primary_key=True on a few fields won't
guarantee order, but something like:

class City (models.Model):
country = models.CharField (max_length=2, primary_key=1)
state = models.CharField (max_length=2, primary_key=2)
city = models.CharField (max_length=3, primary_key=2)
Name = models.CharField (max_length=20)

would set the (country,state,city) primary key in the obvious order.

in short: giving any non-falsy value to primary_key would add the
field to the key, the exact value would determine ordering.


I like this proposal. It might even be easier to implement than
fiddling with new Meta flags and everything.

One minor detail though, just setting primary_key=True on multiple
fields would still have to guarantee some ordering since the primary
key would be represented by a tuple. If you don't know for sure in
which order the values are, you can't really use the .pk property.


This feels like something that wants a named-tuple (or a full blown dict).  
Alternatively, provide a method on the model class that takes the  
name=value arguments  (queryset style) and returns the right pk tuple.  
Otherwise, the exact ordering of the fields in the pk tuple becomes yet  
another implicit(!) part of the model's contract - and any code that wants  
to use this model will be that much more brittle.




This would require a thorough explanation in the docs, that if you
supply values that compare equal to primary_key, the order of fields
will be the same as in the model definition.

Michal Petrucha


--
You received this message because you are subscribed to the Google Groups "Django 
developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Composite primary keys

2011-03-15 Thread Michal Petrucha 
On Tue, Mar 15, 2011 at 09:15:29PM -0500, Javier Guerra Giraldez wrote:
> On Tue, Mar 15, 2011 at 7:14 PM, Christophe Pettus  wrote:
> > A concern here is that composite indexes, like unique, are
> > sensitive to the ordering of the fields, which means that the
> > ordering of the fields in the class declaration becomes important.

This is the same reason a new Meta flag has been agreed upon in the
past. (That, however, does not mean it has to be that way.)

> a simplistic proposal:
> 
> the order of the fields on a composite index is determined by the
> exact value given to the primary_key argument.
> 
> that way, just setting primary_key=True on a few fields won't
> guarantee order, but something like:
> 
> class City (models.Model):
> country = models.CharField (max_length=2, primary_key=1)
> state = models.CharField (max_length=2, primary_key=2)
> city = models.CharField (max_length=3, primary_key=2)
> Name = models.CharField (max_length=20)
> 
> would set the (country,state,city) primary key in the obvious order.
> 
> in short: giving any non-falsy value to primary_key would add the
> field to the key, the exact value would determine ordering.

I like this proposal. It might even be easier to implement than
fiddling with new Meta flags and everything.

One minor detail though, just setting primary_key=True on multiple
fields would still have to guarantee some ordering since the primary
key would be represented by a tuple. If you don't know for sure in
which order the values are, you can't really use the .pk property.

This would require a thorough explanation in the docs, that if you
supply values that compare equal to primary_key, the order of fields
will be the same as in the model definition.

Michal Petrucha


signature.asc
Description: Digital signature

Re: Composite primary keys

2011-03-15 Thread Javier Guerra Giraldez 
On Tue, Mar 15, 2011 at 7:14 PM, Christophe Pettus  wrote:
> A concern here is that composite indexes, like unique, are sensitive to the 
> ordering of the fields, which means that the ordering of the fields in the 
> class declaration becomes important.

a simplistic proposal:

the order of the fields on a composite index is determined by the
exact value given to the primary_key argument.

that way, just setting primary_key=True on a few fields won't
guarantee order, but something like:

class City (models.Model):
country = models.CharField (max_length=2, primary_key=1)
state = models.CharField (max_length=2, primary_key=2)
city = models.CharField (max_length=3, primary_key=2)
Name = models.CharField (max_length=20)

would set the (country,state,city) primary key in the obvious order.

in short: giving any non-falsy value to primary_key would add the
field to the key, the exact value would determine ordering.


-- 
Javier

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Composite primary keys

2011-03-15 Thread Christophe Pettus 

On Mar 15, 2011, at 5:06 PM, Russell Keith-Magee wrote:

> And if you mark
> multiple fields, then you have a composite primary key composed of
> those fields.

A concern here is that composite indexes, like unique, are sensitive to the 
ordering of the fields, which means that the ordering of the fields in the 
class declaration becomes important.  That could, potentially, be surprising.

--
-- Christophe Pettus
   x...@thebuild.com

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Composite primary keys

2011-03-15 Thread Russell Keith-Magee 
On Tue, Mar 15, 2011 at 11:11 PM, Andrew Godwin  wrote:
> On 14/03/11 21:14, Michal Petrucha wrote:
>>
>> Good evening (or whatever it is in everyone's timezone).
>>
>> I'm an undergrad computer science student at the Faculty of
>> Mathematics, Physics and Informatics, Commenius University,
>> Bratislava, Slovakia and I'm willing to participate in this year's
>> GSoc. I'm interested in fixing the six-year-old open ticket in trac
>> concerning the subject, http://code.djangoproject.com/ticket/373
>
> Firstly, thanks for proposing a GSoC project - it's always nice to have more
> students. Bear in mind that the list of accepted organisations hasn't been
> published yet, but we're pretty hopeful Django will make it in again this
> year.

I'd like to second what Andrew has said. This is definitely a project
with enough meat to fill a GSoC, and it's something that has been on
Django's wish list for a long time.

I also agree with Andrew's answers to your questions, with one exception:

>>  - The composite primary key would be specified as a tuple of strings
>>    in a primary_key attribute inside a model's Meta class instead of
>>    having a field with primary_key=True.
>
> That seems to match with our current handling of things like unique, so that
> seems fairly reasonable.

The difference between primary_key and unique is that you can have
multiple unique conditions, including a number of individual field
uniques *plus* a series of grouped uniques. However you can only have
one primary key field (or field set).

I'd like to suggest an alternate representation.

At present, you can make any field a primary key by setting
primary_key=True on that field. The Meta class then validates to
ensure that there is only one field that has that attribute set (and
inserts an AutoField if no field has it set). Another way to represent
primary keys would be to relax this validation constraint. If you mark
no field as primary_key=True, then an AutoField is added. If you mark
one field, then that field is the primary key. And if you mark
multiple fields, then you have a composite primary key composed of
those fields.

This removes the need to introduce a new Meta flag, but more
importantly, it means you won't have to resolve discrepancies between
fields with primary_key and a Meta.primary_key option.

Yours,
Russ Magee %-)

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Composite primary keys

2011-03-15 Thread Andrew Godwin 

On 14/03/11 21:14, Michal Petrucha wrote:

Good evening (or whatever it is in everyone's timezone).

I'm an undergrad computer science student at the Faculty of
Mathematics, Physics and Informatics, Commenius University,
Bratislava, Slovakia and I'm willing to participate in this year's
GSoc. I'm interested in fixing the six-year-old open ticket in trac
concerning the subject, http://code.djangoproject.com/ticket/373


Firstly, thanks for proposing a GSoC project - it's always nice to have 
more students. Bear in mind that the list of accepted organisations 
hasn't been published yet, but we're pretty hopeful Django will make it 
in again this year.



Before I dig deeper into the issue I would like to know whether there
is interest in having this fixed and whether it is worth a full GSoC
project. Also, is there already any work regarding this issue
underway? If so, would it be reasonable for me to go on with this
project?

Anyway, if I'm to take on the task, there are quite a few design
considerations to be taken care of.

For starters, I've read through David Cramer's work on this from
two-three years ago. I'd stick to the API skeleton that was agreed on
back then, however, there remain lots of other unresolved questions.

The following list is in no way meant to be exhaustive, it's just a
list of things that came to my mind during the past hour. In fact, I'd
appreciate other issues that would need to be kept in mind I forgot to
mention.

  - The composite primary key would be specified as a tuple of strings
in a primary_key attribute inside a model's Meta class instead of
having a field with primary_key=True.


That seems to match with our current handling of things like unique, so 
that seems fairly reasonable.



  - The pk property of model instances would be a tuple instead of a
single value for composite key models.


So the property will vary type based on the model? Obviously this is 
needed for backwards-compatability, but it's still going to affect e.g. 
library authors, so we'd need to make sure everything that previously 
accepted a PK also accepts a tuple.



  - The admin could reference composite keys using some kind of smart
string escaping, for example escaping the , (comma) and using it as
a delimiter.


That should work, though it might be a bit ugly; again, I can't think of 
a better alternative.



  - This could maybe even be used in generic relations. Is it
reasonable to support this in generic relations? The following post
suggests it might not be the best idea:
http://groups.google.com/group/django-developers/msg/dea0e360c6cd37a6


Generic relations are ugly and inefficient as it is; I'm not sure that 
including them in something like this is strictly necessary, much as 
Malcolm suggests, and probably outside the scope of a GSoC project 
(something of this scale is already touching a lot of places in the 
model layer as it is)



  - The managers and querysets would have to be updated to handle
composite primary keys correctly.

  - Consequently, there would need to be added support in the SQL
compiler.

  - The same holds for syncdb, inspectdb would be also nice.


I'd say inspectdb isn't strictly necessary - not very many people use 
it. I'll also be looking at potentally merging in some schema alteration 
code (i.e. moving parts of the South database backends) over the next 
few months, so it's going to affect that, too, though I'm more than 
happy to help fix that part (the existing creation code will definitely 
need changing, at least).



  - ForeignKeys would have to be backed by multiple database columns.
How should they be named by default? How should their names be
overriden? Should db_column expect a tuple of strings? Should there
be another db_column_prefix option to prefix the names with a
common string?


These are some very good questions, and something we need a good 
discussion here about; I'd personally say doing "fkname_remotecolname" 
is best for the columns (which then means "author_id" still matches most 
DBs, and you could also have something like "passport_country_id, 
passport_number").


I'm also -0 on the idea of a db_column_prefix, but db_column is going to 
need to take tuples/sequences for multi-column FKs.



  - What about the ForeignKey field's attname? Should it be a tuple of
names or should it be a single string pointing to a tuple of
attributes?


I'd say a single string pointing to a tuple - I don't like fields which 
'magically' make more attributes than the name they were assigned. That 
also makes sense if you think of the foreign key as a single entity 
composed of several values.



  - How should a person trying to add a ForeignKey field pointing to
its own model into the primary_key be punished?


Preferably with a model validation error.


  - Does it make sense to make a subset of columns created by a
ForeignKey part of a primary key?


Provided they're then

Re: Composite Primary Keys

2008-10-30 Thread David Cramer 
Well GenericRelation's or whatever are an attempt to provide some composite
space, but we need an expandable space, not something tied to content
type's. We need to say mykey = CompositeField(field, field, field) or
something.

No one wanted to discuss it at DjangoCon, so no.

On Thu, Oct 30, 2008 at 3:57 PM, Joey Wilhelm <[EMAIL PROTECTED]> wrote:

> That would be great. The project I am working on now won't be doing
> anything too terribly complex just yet; I mainly need the admin support to
> make my life a little easier.
>
> As to the API, I saw several proposals earlier along on this thread, but
> obviously nothing solid. Did anything ever come from DjangoCon on this
> topic? What issues still need to be addressed in this design?
>
> On Thu, Oct 30, 2008 at 13:46, David Cramer <[EMAIL PROTECTED]> wrote:
>
>> It allows you to use them, automatically creates them, and has some of the
>> admin handling done. However, there's still no API design around
>> multi-column fields (no one seems to want to talk about it) so I'm pretty
>> much stopped working on it.
>>
>> e.g. You can't say field1 = this, field2 = that, and then say compositekey
>> = field1,field2 you instead are forced to do key1=blah, key2=blah in all
>> your lookups, and no easy foreignkey properties.
>>
>> I'm running this on production environments, so it works fine, but I can
>> up SVN and fix any conflicts and post a patch again.
>>
>> --
>> David Cramer
>> Director of Technology
>> iBegin
>> http://www.ibegin.com/
>>
>>
>>
>
> >
>


-- 
David Cramer
Director of Technology
iBegin
http://www.ibegin.com/

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Composite Primary Keys

2008-10-30 Thread Joey Wilhelm 
That would be great. The project I am working on now won't be doing anything
too terribly complex just yet; I mainly need the admin support to make my
life a little easier.

As to the API, I saw several proposals earlier along on this thread, but
obviously nothing solid. Did anything ever come from DjangoCon on this
topic? What issues still need to be addressed in this design?

On Thu, Oct 30, 2008 at 13:46, David Cramer <[EMAIL PROTECTED]> wrote:

> It allows you to use them, automatically creates them, and has some of the
> admin handling done. However, there's still no API design around
> multi-column fields (no one seems to want to talk about it) so I'm pretty
> much stopped working on it.
>
> e.g. You can't say field1 = this, field2 = that, and then say compositekey
> = field1,field2 you instead are forced to do key1=blah, key2=blah in all
> your lookups, and no easy foreignkey properties.
>
> I'm running this on production environments, so it works fine, but I can up
> SVN and fix any conflicts and post a patch again.
>
> --
> David Cramer
> Director of Technology
> iBegin
> http://www.ibegin.com/
>
> >
>

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Composite Primary Keys

2008-10-30 Thread David Cramer 
It allows you to use them, automatically creates them, and has some of the
admin handling done. However, there's still no API design around
multi-column fields (no one seems to want to talk about it) so I'm pretty
much stopped working on it.

e.g. You can't say field1 = this, field2 = that, and then say compositekey =
field1,field2 you instead are forced to do key1=blah, key2=blah in all your
lookups, and no easy foreignkey properties.

I'm running this on production environments, so it works fine, but I can up
SVN and fix any conflicts and post a patch again.

On Thu, Oct 30, 2008 at 3:40 PM, Joey Wilhelm <[EMAIL PROTECTED]> wrote:

>
> David,
>
> What is the current status of this patch? I'm starting up a new
> project which pretty much desperately needs this support as well. I
> could work around it, but the thought of adding AutoFields to all of
> these models which really -do not- need them, makes me a bit ill.
>
> I would be more than willing to help test your implementation if there
> is anything usable yet. This is one of the pieces that's getting me
> all twitchy waiting for it.
> >
>


-- 
David Cramer
Director of Technology
iBegin
http://www.ibegin.com/

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Composite Primary Keys

2008-10-30 Thread Joey Wilhelm 

David,

What is the current status of this patch? I'm starting up a new
project which pretty much desperately needs this support as well. I
could work around it, but the thought of adding AutoFields to all of
these models which really -do not- need them, makes me a bit ill.

I would be more than willing to help test your implementation if there
is anything usable yet. This is one of the pieces that's getting me
all twitchy waiting for it.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Composite Primary Keys

2008-10-04 Thread David Cramer 
What we hope to achieve here is full support within the Django core, as
opposed to specifying some kind of model. The only thing I have left to do
is implement composite fields, but there's still no API for it.

On Sat, Oct 4, 2008 at 12:03 PM, Eric <[EMAIL PROTECTED]> wrote:

>
> Hi,
> i just discover this thread, I am working on this problem; you may
> take a look at
> http://kenai.com/projects/django-trac/pages/LegacyModule
>
> legacy is a module in my "django hacks trac" (or djac) project; it
> aims to deal with
> tables with no primary key or with composite pk. It provides 2
> methods:
>
> - use of oid field (works on sqlite, oracle, postgres <= 8)
> - composite pk (for mysql that provides no oid field)
>
> cheers,
> Eric
> >
>


-- 
David Cramer
Director of Technology
iBegin
http://www.ibegin.com/

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Composite Primary Keys

2008-10-04 Thread Eric 

Hi,
i just discover this thread, I am working on this problem; you may
take a look at
http://kenai.com/projects/django-trac/pages/LegacyModule

legacy is a module in my "django hacks trac" (or djac) project; it
aims to deal with
tables with no primary key or with composite pk. It provides 2
methods:

- use of oid field (works on sqlite, oracle, postgres <= 8)
- composite pk (for mysql that provides no oid field)

cheers,
Eric
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Composite Primary Keys

2008-09-04 Thread Malcolm Tredinnick 


On Wed, 2008-09-03 at 23:22 -0700, David Cramer wrote:
> This is one of those things that I really need to get hammered out
> (for our platform). I'm willing to do all the work, but I need a
> design around it to where the patch won't get rejected :)

No question. And for that, we have this very handy mailing list with
archives that will store all the discussion. Meanwhile, this weekend, we
have a two day conference with a very full schedule,  no scheduled
downtime and a limited attendance group. This is all at the end of a
long process of nothing but dawn to dusk bug fixing for a group of us,
so some of us will be relaxing. So by all means have a conversation with
some people, that's a given. But you'll still need to bring the results
to the mailing list in a coherent format so that it can be looked at
when we start to consider the patch.

This is a serious proposal. It's being taken seriously. But realise the
broader things going on as well. If you're hoping/expecting to get this
resolved in a couple of days, you are, frankly, optimistic.
 
Regards,
Malcolm



--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Composite Primary Keys

2008-09-04 Thread David Cramer 

This is one of those things that I really need to get hammered out
(for our platform). I'm willing to do all the work, but I need a
design around it to where the patch won't get rejected :)

On Sep 3, 11:08 am, Malcolm Tredinnick <[EMAIL PROTECTED]>
wrote:
> On Tue, 2008-09-02 at 22:02 -0700, David Cramer wrote:
> > For anyone who's interested, it'd be great to meetup at DjangoCon to
> > go over a good design approach to composite fields.
>
> Take notes. There's going to be a lot going on at DjangoCon (including
> celebrating), so there will be a group who are putting off thinking
> about this until quieten down who'll likely be busy throughout the
> weekend. In a few weeks we can start looking at this more concretely on
> the list and working out how it might fit into the whole.
>
> Regards,
> Malcolm
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Composite Primary Keys

2008-09-03 Thread David Cramer 

For anyone who's interested, it'd be great to meetup at DjangoCon to
go over a good design approach to composite fields.

On Sep 2, 2:44 pm, Rock <[EMAIL PROTECTED]> wrote:
> One use case for Composite Primary Keys is for setting up database
> partitions. In my case I am using Range-Hash partitions with the range
> determined by an IntegerField called "ISOweek" and the hash working
> off of the "id" field supplied by Django. To allow this partitioning
> to work, the primary key must be a composite primary key incorporating
> the "ISOweek" and the "id" fields. My versions of the sqlreset and
> reset management functions do this while also ensuring that "id" is
> marked as unique even though it is not the primary key. This allows a
> ForeignKey pointed at my partitioned model to work correctly by
> setting "id" as the to_field. (If "id" is not set as unique, Django
> and/or the database will fail in its' attempt to set up the full
> foreign key relationship.)
>
> The initial version of Composite Primary Keys should not preclude this
> scenario, however full support for setting up and managing partitioned
> models need not be included at this time. (I plan to help add that
> later.) The interesting point is that support for related fields for
> the Composite Primary Key is not required in order to support this
> particular use case.
>
> Rock
>
> On Aug 28, 8:05 pm, "David Cramer" <[EMAIL PROTECTED]> wrote:
>
> > I'm not quite sure how that relates to Composite Primary Keys?
>
> > A ForeignKey would point to multiple internal fields, but it should look
> > like it's a single field. At the same time, this would open up the
> > possibility for Composite Foreign Keys, which would mean it could point to
> > multiple public fields.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Composite Primary Keys

2008-09-03 Thread Malcolm Tredinnick 


On Tue, 2008-09-02 at 22:02 -0700, David Cramer wrote:
> For anyone who's interested, it'd be great to meetup at DjangoCon to
> go over a good design approach to composite fields.

Take notes. There's going to be a lot going on at DjangoCon (including
celebrating), so there will be a group who are putting off thinking
about this until quieten down who'll likely be busy throughout the
weekend. In a few weeks we can start looking at this more concretely on
the list and working out how it might fit into the whole.

Regards,
Malcolm




--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Composite Primary Keys

2008-09-02 Thread Rock 


One use case for Composite Primary Keys is for setting up database
partitions. In my case I am using Range-Hash partitions with the range
determined by an IntegerField called "ISOweek" and the hash working
off of the "id" field supplied by Django. To allow this partitioning
to work, the primary key must be a composite primary key incorporating
the "ISOweek" and the "id" fields. My versions of the sqlreset and
reset management functions do this while also ensuring that "id" is
marked as unique even though it is not the primary key. This allows a
ForeignKey pointed at my partitioned model to work correctly by
setting "id" as the to_field. (If "id" is not set as unique, Django
and/or the database will fail in its' attempt to set up the full
foreign key relationship.)

The initial version of Composite Primary Keys should not preclude this
scenario, however full support for setting up and managing partitioned
models need not be included at this time. (I plan to help add that
later.) The interesting point is that support for related fields for
the Composite Primary Key is not required in order to support this
particular use case.

Rock


On Aug 28, 8:05 pm, "David Cramer" <[EMAIL PROTECTED]> wrote:
> I'm not quite sure how that relates to Composite Primary Keys?
>
> A ForeignKey would point to multiple internal fields, but it should look
> like it's a single field. At the same time, this would open up the
> possibility for Composite Foreign Keys, which would mean it could point to
> multiple public fields.
>
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Composite Primary Keys

2008-08-28 Thread David Cramer 
I'm not quite sure how that relates to Composite Primary Keys?

A ForeignKey would point to multiple internal fields, but it should look
like it's a single field. At the same time, this would open up the
possibility for Composite Foreign Keys, which would mean it could point to
multiple public fields.

On Thu, Aug 28, 2008 at 5:36 PM, Rock <[EMAIL PROTECTED]> wrote:

>
> To be clear, the syntax is:
>
> myfkey = models.ForeignKey(SomeClass,to_field="id")
>
>
> >
>


-- 
David Cramer
Director of Technology
iBegin
http://www.ibegin.com/

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Composite Primary Keys

2008-08-28 Thread Rock 

To be clear, the syntax is:

myfkey = models.ForeignKey(SomeClass,to_field="id")


--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Composite Primary Keys

2008-08-28 Thread Rock 

Ha! It turns out that a to_field option already exists for ForeignKey.
(I did not know that yesterday.) I have just verified that
to_field(SomeClass,"id")
works fine even if the PRIMARY KEY uses multiple columns. However, and
this
is the key point, the id field has to be marked as UNIQUE.

To prove all of this I have created a sqlresetpartition and
resetpartition command
that looks for classes to partition within your settings. It sets up
the required
multi-column primary key, marks the ID field as UNIQUE, and also does
the other
magic to create partitioned tables. (It also creates partitioned
indexes as required.)

This only works for Oracle and the solution is special cased for my
needs, but it
points the way forward. I will look over the partitioning logic for
MySQL when I get
a chance. Maybe by Django 1.5 or so we can include direct support for
partitioned models.

BTW, once I have the models properly created with partitions, none of
my other
django code requires any changes at all. Besides improved performance,
I must
say that it is a thrill to delete hundreds of thousands of old rows in
less than a
second. This is the key feature I needed for my django app.


On Aug 27, 6:27 pm, Rock <[EMAIL PROTECTED]> wrote:
> Well for one thing, if one of the columns happens to be named "ID", we
> should use that for the relatedfields lookup column and that is that.
> (BTW, does your approach allow the Django supplied ID field to be
> combined with some other field(s) to make a multi-column key? This
> would be bang up for future partitioning support.)
>
> Next I would suggest adding a meta model column designation like
> "id_field" to specify a field to use for related classes. This might
> be a good "80/20" solution that could serve for an initial test
> version.
>
> On Aug 27, 5:27 pm, "David Cramer" <[EMAIL PROTECTED]> wrote:
>
>
>
> > Really I'm stuck at an architectural point.
>
> > I have database validation and synchronization done, and the admin is
> > working.
>
> > What is left is more or less handling relatedfield lookups. The issue is,
> > that field's are designed to reference more than one field, so it's a tough
> > design deicision to make on how that should be approached.- Hide quoted 
> > text -
>
> - Show quoted text -
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Composite Primary Keys

2008-08-27 Thread Rock 


Well for one thing, if one of the columns happens to be named "ID", we
should use that for the relatedfields lookup column and that is that.
(BTW, does your approach allow the Django supplied ID field to be
combined with some other field(s) to make a multi-column key? This
would be bang up for future partitioning support.)

Next I would suggest adding a meta model column designation like
"id_field" to specify a field to use for related classes. This might
be a good "80/20" solution that could serve for an initial test
version.

On Aug 27, 5:27 pm, "David Cramer" <[EMAIL PROTECTED]> wrote:
> Really I'm stuck at an architectural point.
>
> I have database validation and synchronization done, and the admin is
> working.
>
> What is left is more or less handling relatedfield lookups. The issue is,
> that field's are designed to reference more than one field, so it's a tough
> design deicision to make on how that should be approached.
>

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Composite Primary Keys

2008-08-27 Thread David Cramer 
What I had briefly discussed with malcom was using ordered tuples but
switching up the defaults to use actualy field lookups.

MyModel.objects.get(pk=(1, 2)) or MyModel.objects.get(foo=1, bar=2)

If we could come up with some design for multi-column fields I'm wiling to
put in the work.

On Wed, Aug 27, 2008 at 6:15 PM, Alberto García Hierro <[EMAIL PROTECTED]>wrote:

>
>
> El 28/08/2008, a las 0:27, David Cramer escribió:
>
> > Really I'm stuck at an architectural point.
> >
> > I have database validation and synchronization done, and the admin
> > is working.
> >
> > What is left is more or less handling relatedfield lookups. The
> > issue is, that field's are designed to reference more than one
> > field, so it's a tough design deicision to make on how that should
> > be approached.
>
>   I think the best (and the only one right) solution involves adding
> multicolumn fields to Django and doing lookups with some syntax like
> Model.objects.get(pk=('foo', 1)). There are other hackish approaches,
> like using hash(tuple(pk[0], pk[1], ..., pk[n])) as foreign key, that
> could work. However, I won't rely on them, since I'm not sure if
> hash() implementation is guaranteed to be kept as is.
>
> On other related point, what's the status of multicolumn fields?
>
> Regards,
> Alberto
>
> >
>


-- 
David Cramer
Director of Technology
iBegin
http://www.ibegin.com/

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Composite Primary Keys

2008-08-27 Thread Alberto García Hierro 


El 28/08/2008, a las 0:27, David Cramer escribió:

> Really I'm stuck at an architectural point.
>
> I have database validation and synchronization done, and the admin  
> is working.
>
> What is left is more or less handling relatedfield lookups. The  
> issue is, that field's are designed to reference more than one  
> field, so it's a tough design deicision to make on how that should  
> be approached.

  I think the best (and the only one right) solution involves adding  
multicolumn fields to Django and doing lookups with some syntax like  
Model.objects.get(pk=('foo', 1)). There are other hackish approaches,  
like using hash(tuple(pk[0], pk[1], ..., pk[n])) as foreign key, that  
could work. However, I won't rely on them, since I'm not sure if  
hash() implementation is guaranteed to be kept as is.

On other related point, what's the status of multicolumn fields?

Regards,
Alberto

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Composite Primary Keys

2008-08-27 Thread David Cramer 
Really I'm stuck at an architectural point.

I have database validation and synchronization done, and the admin is
working.

What is left is more or less handling relatedfield lookups. The issue is,
that field's are designed to reference more than one field, so it's a tough
design deicision to make on how that should be approached.

On Wed, Aug 27, 2008 at 5:23 PM, Rock <[EMAIL PROTECTED]> wrote:

>
> Any progress on this patch David? I would be happy to take a look at
> whatever you have and perhaps help out with completing the patch.
>
> >
>


-- 
David Cramer
Director of Technology
iBegin
http://www.ibegin.com/

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Composite Primary Keys

2008-08-27 Thread Rock 

Any progress on this patch David? I would be happy to take a look at
whatever you have and perhaps help out with completing the patch.

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Composite Primary Keys

2008-05-20 Thread Jacob Kaplan-Moss 

On Thu, May 8, 2008 at 7:57 PM, David Cramer <[EMAIL PROTECTED]> wrote:
> I swear I saw something about work being done on this. Has anyone
> begun? If not I'll gladly throw up a patch to get it into trunk.

Please do. If you make sure it solves all three problems I outlined in
#373 (http://code.djangoproject.com/ticket/373#comment:3) I'll make
sure it gets in before the next release.

Jacob

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Composite Primary Keys

2008-05-08 Thread Russell Keith-Magee 

On Fri, May 9, 2008 at 8:57 AM, David Cramer <[EMAIL PROTECTED]> wrote:
>
> I swear I saw something about work being done on this. Has anyone
> begun? If not I'll gladly throw up a patch to get it into trunk.

The topic comes up fairly regularly, but I'm not aware of anyone that
is actively working on it.

The most recent reference I am aware of is this one where it was
proposed as a GSOC project; the comment from Malcolm is the most
enlightening:

http://groups.google.com/group/django-developers/browse_thread/thread/647009a9a4439f0b

Yours,
Russ Magee %-)

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---