Re: Generated Field

['schinckel' via Django developers (Contributions to Django itself)]

I believe there are a bunch of similarities between the requirements of 
generated fields and my project 
django-shared-property: https://django-shared-property.readthedocs.io/en/latest/

On Sunday, December 25, 2022 at 10:23:10 AM UTC+10:30 jeremy...@gmail.com 
wrote:

> I'd love to help implement this if we can find a rough syntax! I've made a 
> proof of concept in Postgres, and there are two outstanding limitations to 
> address:
>
> - The generated field value is not set until the model is reloaded from 
> the database
> - The `GENERATED ALWAYS` expression requires an argument to be injected in 
> the the sql expression, but this is not currently possible
>
> *from django.db.backends.utils import CursorWrapper*
>
> *from django.db.models import Expression, Field*
>
> *from django.db.models.sql import Query*
>
>
>
> *class GeneratedField(Field):*
>
> *"""*
>
> *Wrapper field used to support generated columns in postgres.*
>
> *"""*
>
>
> *def __init__(self, expression: Expression, db_collation: str = None, 
> *args, **kwargs):*
>
> *"""*
>
> *:param expression: DB expression used to calculate the 
> auto-generated field value*
>
> *"""*
>
>
> *self.expression = expression*
>
> *self.db_collation = db_collation*
>
>
> *kwargs['editable'] = False  # This field can never be edited*
>
> *kwargs['blank'] = True  # This field never requires a value to be 
> set*
>
> *kwargs['null'] = True  # This field never requires a value to be 
> set*
>
>
> *super().__init__(*args, **kwargs)*
>
>
> *def _compile_expression(self, cursor: CursorWrapper, sql: str, 
> params: dict):*
>
> *"""*
>
> *Compiles SQL and its associated parameters into a full SQL query. 
> Usually sql params are kept*
>
> *separate until `cursor.execute()` is called, but this is not 
> possible since this function*
>
> *must return a single sql string.*
>
> *"""*
>
>
> *return cursor.mogrify(sql, params).decode()*
>
>
> *def db_type(self, connection):*
>
> *"""*
>
> *Called when calculating SQL to create DB column (e.g. DB 
> migrations)*
>
> *
> https://docs.djangoproject.com/en/4.1/ref/models/fields/#django.db.models.Field.db_type
>  
> *
>
> *"""*
>
>
> *db_type = 
> self.expression.output_field.db_type(connection=connection)*
>
>
> *# Convert any F() references to concrete field names*
>
> *query = Query(model=self.model, alias_cols=False)*
>
> *expression = self.expression.resolve_expression(query, 
> allow_joins=False)*
>
>
> *# Compile expression into SQL*
>
> *expression_sql, params = expression.as_sql(*
>
> *compiler=connection.ops.compiler('SQLCompiler')(*
>
> *query, connection=connection, using=None*
>
> *),*
>
> *connection=connection,*
>
> *)*
>
>
> *with connection.cursor() as cursor:*
>
> *expression_sql = self._compile_expression(*
>
> *cursor=cursor, sql=expression_sql, params=params*
>
> *)*
>
>
> *return f'{db_type} GENERATED ALWAYS AS ({expression_sql}) STORED'*
>
>
> *def rel_db_type(self, connection):*
>
> *"""*
>
> *Called when calculating SQL to reference DB column*
>
> *
> https://docs.djangoproject.com/en/4.1/ref/models/fields/#django.db.models.Field.rel_db_type
>  
> *
>
> *"""*
>
> *return 
> self.expression.output_field.db_type(connection=connection)*
>
>
> *def deconstruct(self):*
>
> *"""*
>
> *Add custom field properties to allow migrations to deconstruct 
> field*
>
> *
> https://docs.djangoproject.com/en/4.1/ref/models/fields/#django.db.models.Field.deconstruct
>  
> *
>
> *"""*
>
> *name, path, args, kwargs = super().deconstruct()*
>
> *kwargs['expression'] = self.expression*
>
> *if self.db_collation is not None:*
>
> *kwargs['db_collation'] = self.db_collation*
>
> *return name, path, args, kwargs*
>
>
>
> *class GeneratedFieldQuerysetMixin:*
>
> *"""*
>
> *Must be added to queryset classes*
>
> *"""*
>
>
> *def _insert(self, objs, fields, *args, **kwargs):*
>
> *if getattr(self.model, '_generated_fields', None) and fields:*
>
> *# Don't include generated fields when performing a 
> `model.objects.bulk_create()`*
>
> *fields = [f for f in fields if f not in 
> self.model._generated_fields()]*
>
>
> *return super()._insert(objs, fields, *args, **kwargs)*
>
>
>
> *class GeneratedFieldModelMixin:*
>
> *"""*
>
> *Must be added 

Re: Changing the order of SQL generation

[schinckel]

Hey Luke,

Yeah, I had seen hybrid attributes. I think I came across them after 
starting this, but I think they are pretty much the same idea. Not sure if 
their hybrid methods are new, but that looks fun too ;)

I'll be more than happy to look at a PR to include this in Django, once 
I've got a completely solid proof of concept.


With respect to the logical ordering: Django already does things in a 
different order to that, as it requires the SELECT to be prepared first, 
and has a comment as to why. Thinking a little further about that, you 
might be able to make an argument that the preparation of the SQL statement 
should be taking those steps in the reverse order, as things from each 
phase there are available to the following phases.

Cheers,

Matt.
On Wednesday, July 28, 2021 at 7:36:26 PM UTC+9:30 Luke Plant wrote:

> I haven't had chance to dig into this, but django-shared-property looks 
> very interesting. IMO it would be great to be able to support it well, or 
> even include that kind of functionality in Django itself.
>
> The closest SQLAlchemy equivalent to this seems to be hybrid attributes 
> <https://docs.sqlalchemy.org/en/14/orm/extensions/hybrid.html> - a 
> feature I'm very jealous of. In almost every Django project I've come 
> across the need for computed properties that should be available both at DB 
> level (filtering/searching) and at instance level, and it is a constant 
> source of duplication of logic.
>
> Relating to your actual post, and without thinking it through, I'm 
> guessing the conceptual order of SQL evaluation is relevant here - e.g. 
>
>
> https://stackoverflow.com/questions/21693208/sql-conceptual-order-evaluation
>
>
> https://docs.microsoft.com/en-us/sql/t-sql/queries/select-transact-sql?redirectedfrom=MSDN=sql-server-ver15#logical-processing-order-of-the-select-statement
>
> Luke
>
>
> On 28/07/2021 05:02, Matthew Schinckel wrote:
>
> Greetings, 
>
> For a couple of things I’ve been working on, I wound up hitting exactly 
> the same problem with respect to SQL generation.
>
> It seems that the `FROM` clauses are generated after the SELECT, but 
> before WHERE.
>
> This means that any .as_sql() code that is run only on things that appear 
> in a WHERE clause but not in a SELECT clause can reference any of the 
> aliases that it needs to generate it’s SQL, but is not able to “ref” the 
> alias (ie, indicate that it has actually used it).
>
> This means that there are certain situations where a table is referenced 
> but the SQL compiler does not know about that reference, and the table is 
> stripped out.
>
> I have a fairly simple PR, that is passing tests, to move the FROM 
> generation until after any WHERE clauses have been executed.
>
> https://github.com/django/django/pull/14683
>
> Does anyone have any suggestions as to why the FROM needs to be generated 
> before the WHERE?
>
> For background the things I’ve hit this same issue in are:
>
> Django Implied Relationship 
> <https://schinckel.net/2021/07/14/django-implied-relationship/> - 
> creating implicit relationships between objects that don’t otherwise have a 
> direct relationship
>
> https://github.com/schinckel/django-shared-property - which is woefully 
> under-documented, but basically allows defining an expression as something 
> that should (a) always be annotated onto a query, but (b) also be “live”, 
> and changes to the dependencies affect the value the property returns. 
> https://github.com/schinckel/django-shared-property/blob/main/tests/models.py 
> has 
> some nice examples.
>
> Cheers,
>
> Matt.
>
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-develop...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/django-developers/9F101363-110C-498A-8C6C-554138F8D673%40schinckel.net
>  
> <https://groups.google.com/d/msgid/django-developers/9F101363-110C-498A-8C6C-554138F8D673%40schinckel.net?utm_medium=email_source=footer>
> .
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/096cb1c6-0a02-4779-b650-8db408bcc3c0n%40googlegroups.com.

Changing the order of SQL generation

[Matthew Schinckel]

Greetings,

For a couple of things I’ve been working on, I wound up hitting exactly the 
same problem with respect to SQL generation.

It seems that the `FROM` clauses are generated after the SELECT, but before 
WHERE.

This means that any .as_sql() code that is run only on things that appear in a 
WHERE clause but not in a SELECT clause can reference any of the aliases that 
it needs to generate it’s SQL, but is not able to “ref” the alias (ie, indicate 
that it has actually used it).

This means that there are certain situations where a table is referenced but 
the SQL compiler does not know about that reference, and the table is stripped 
out.

I have a fairly simple PR, that is passing tests, to move the FROM generation 
until after any WHERE clauses have been executed.

https://github.com/django/django/pull/14683

Does anyone have any suggestions as to why the FROM needs to be generated 
before the WHERE?

For background the things I’ve hit this same issue in are:

Django Implied Relationship 
<https://schinckel.net/2021/07/14/django-implied-relationship/> - creating 
implicit relationships between objects that don’t otherwise have a direct 
relationship

https://github.com/schinckel/django-shared-property 
<https://github.com/schinckel/django-shared-property> - which is woefully 
under-documented, but basically allows defining an expression as something that 
should (a) always be annotated onto a query, but (b) also be “live”, and 
changes to the dependencies affect the value the property returns. 
https://github.com/schinckel/django-shared-property/blob/main/tests/models.py 
<https://github.com/schinckel/django-shared-property/blob/main/tests/models.py> 
has some nice examples.

Cheers,

Matt.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/9F101363-110C-498A-8C6C-554138F8D673%40schinckel.net.

Re: Help with implementing calculated fields (#28822)

[schinckel]

Hi Tobias.

I've done a bit of stuff on this, and found there were actually limited 
changes that need to be made to django to got this to work.

https://schinckel.net/2020/09/15/django-properties-from-expressions%2C-or-computedfield-part-2/

I haven't touched it much since then (although I'm keen to revisit it, 
actually), but it's actually possible that we can do it without any changes 
to Django.

https://github.com/schinckel/django-shared-property

Matt.

On Sunday, March 14, 2021 at 11:46:15 PM UTC+10:30 Tobias Bengfort wrote:

> Hi,
>
> a while back there has been a discussion about calculated model fields:
>
> https://groups.google.com/g/django-developers/c/ADSuUUuZp3Q/m/5o02Lp_jCwAJ
>
> The main benefit would be that these would be available in cases where 
> annotations are currently not available, e.g. related queries or admin 
> lists (see #14336).
>
> Note that the topic of fields that have both a DB and python 
> implementation was discussed and the consensus was that the python 
> equivalents could be added by a separate library, if required. So I am 
> only talking about the database part here.
>
> The linked discussion was mainly about *what* is desired, not *how* it 
> could be done. I also noticed that the fellows did not participate in 
> that discussion.
>
> I would like to have a shot at this topic. So I am interested in:
>
> - Do you think this is even possible/worth the effort?
> - How could this be implemented and what would be potential challenges?
> - Is it possible to implement this in a third party library or does it 
> require changes to django itself?
>
> I would probably start with a non-editable, non-concrete model field 
> (similar to GenericForeignKey or ForeignObjRel). But I have no clue yet 
> how to integrate that with QuerySet/Query.
>
> thanks
> tobias
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/8d325f4e-9c20-4268-9c46-6f682911ac5dn%40googlegroups.com.

Re: Subquery join support

[schinckel]

Great work on this, Alexandr. I've been thinking a lot about doing joins in 
Django to subqueries. As you point out, when you are doing several subquery 
annotations of the same subquery, just with different columns, that can 
really hurt performance. Currently, I've been recommending doing a JSON 
object as the subquery column.

I really like the idea of an explicit join to a subquery, as it can perform 
significantly better - in the case where most of the rows share the same 
subquery, the database can perform the subquery once per distinct row 
joined, rather than once for each row.

I think, instead of using a `JoinedSubquery`, we should just make it that a 
`Subquery`, containing unresolved OuterRef instances, can also be resolved 
by doing a `QuerySet.join(Subquery(...))`.

We could also have a lateral argument that allowed a LATERAL JOIN to a 
subquery, but perhaps that's later down the track.

(Actually, now I think about it, a QuerySet should probably just act as a 
Subquery when it's used as one, but perhaps that's too magic).

Matt.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/2bec9c0d-574c-493a-bbe2-fa58ab56%40googlegroups.com.

Re: Proposal for better managed raw SQL migrations

[schinckel]

On Wednesday, March 25, 2020 at 1:45:48 AM UTC+10:30, Petr Přikryl wrote:
>
> Hi Adam,
> thank you for your reply.
>
> We usually have few indices, functions and triggers. But the most used 
> database object is view. We used them for data synchronizing from some 
> third party databases. These databases have complex schema which we want 
> simplify. So we are building low-level DB API via views. Then we create 
> Django models for these views. Then is easy to use ORM for data access or 
> sync operations. 
>
>
Hi Petr,

I too have a bunch of database Raw SQL. I came up with a mechanism for 
doing this that allows for/generates numbered versions of each file.

https://schinckel.net/2017/06/07/versioning-complex-database-migrations/

There's no way to hook in to the migrations framework to get this to happen 
during `makemigrations`, but I have used the checks framework to examine 
the project for any files that appear to be out of date, and moan about 
those, as well as a custom management command that generates the migrations 
for any that have changed, as well as the versions.

There's also a command that copies the current version of the file to the 
"newest" migration which is useful for development.

I'll try to publish the actual code that does it soon.

Matt.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/2226e719-785b-481c-90f7-fde346399cd7%40googlegroups.com.

Re: Cross-DB JSONField ready for review

[schinckel]

Hi Ole,

I'm interested in what you are trying to do with JSONExtract. I have a 
subclass of Func called JSONBExtractPathText that I use with great success 
to extract parts of a JSONB object.

Also, as of Django 3.0, you can filter directly on an expression (that has 
an output_field of BooleanField).

Thus, you could write your first example as:

MyModel.objects.filter(JSONBExtractPathText('field', Value('Foo the bar?'), 
output_field=models.BooleanField())

I think you could possibly do the other stuff using either an 
ExpressionWrapper, or at worst a Case(When()).

(I hang out on #django on IRC if you want to discuss this in a more 
interactive manner).

Matt.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/45fb5bf5-c1d8-44eb-973b-56914264fcc3%40googlegroups.com.

SubQuery without using RawSQL

[schinckel]

I started some work late last night on attempting to replace some RawSQL() 
calls that do a sub query with a new Expression.

It actually worked!

Even in the cold light of day, it doesn't seem _too_ bad, so after working 
on it a bit today with jarshwah, I decided to stick up a WIP PR.

https://github.com/django/django/pull/6478

I'd love to have some further input.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/f11385f8-9147-4f3b-8f5d-5b9000fbecd4%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: CHECK Constraints and migrations

[schinckel]

That's the approach I've used: either have a migration operation that you 
can run (included in a 3rd party or local app), or a management command 
that generates a migration file with the relevant custom operations.

Matt.

On Friday, August 28, 2015 at 11:37:55 AM UTC+9:30, Andrew Godwin wrote:
>
> Hi Gavin,
>
> Hooking into the migration autodetector is currently not really possible - 
> it's quite inextensible, which is something I'd love to get ironed out in 
> the long term but isn't really possible right now to modify it from 
> third-party code.
>
> You can, however, supply custom migration Operation classes that people 
> could put into their migrations manually, or potentially have some other 
> way of outputting those migration files with operations in them (a lot of 
> the code to write them out is re-useable). Hopefully someone else has a 
> better idea!
>
> Andrew
>
> On Thu, Aug 27, 2015 at 6:27 PM, Gavin Wahl  > wrote:
>
>> I'm interested in writing a third-party app that allows users to 
>> declaratively write check constraints and have them automatically be added 
>> and removed by migrations. I'm envisioning being able to attach a list of Q 
>> objects to your model, something like:
>>
>>
>> class Meta:
>> checks = [
>> Q(end_date__isnull=True) | Q(end_date__gt=F('start_date'))
>> ]
>>
>> Now that we have migrations, formalized Meta, and custom lookups, is it 
>> possible for a third-party app to manage this seamlessly?
>>
>> I'm having trouble figuring out where it could hook in to migrations to 
>> detect changes to the check constraints, and automatically generate the 
>> migrations with operations to update the constraints. What code I should be 
>> reading to figure out how to do it?
>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "Django developers (Contributions to Django itself)" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to django-develop...@googlegroups.com .
>> To post to this group, send email to django-d...@googlegroups.com 
>> .
>> Visit this group at http://groups.google.com/group/django-developers.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/django-developers/CACPudh1gRB_Jov_bH-maDGr4EBRugFuV3Y%2BBzYM9w0yyki7zkg%40mail.gmail.com
>>  
>> 
>> .
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/dedacf17-865e-4900-832c-2135039437ba%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: The hypothetical models.MultiField: is there a rationale to avoid it ? Or could it make it into future releases ?

[schinckel]

As it turns out, this is the approach I've settled on for my Postgres 
Composite Type/Field implementation.

https://bitbucket.org/schinckel/django-postgres/src/6a6078d8a2cc30bfb61d093354b8228f76484a0c/postgres/fields/composite_field.py?at=default

Some issues I have identified are:

* it's not really possible at this stage to have something that triggers a 
migration operation to create the type. In practice I've just been writing 
that migration as a RunSQL operation, but it would be nice to have this 
included in the project state in some way. 

* There is a fair bit of duplicated code between the composite field 
metaclass and the django model metaclass. It's possible there are better 
ways to handle this.

I haven't done anything with querying/lookups at this stage, but that's an 
exciting possible feature.

I also blogged about this last year:

http://schinckel.net/2014/09/24/using-postgres-composite-types-in-django/


Matt.


On Friday, August 21, 2015 at 8:41:23 PM UTC+9:30, Anssi Kääriäinen wrote:
>
> I've been thinking how we could move forward with composite fields without 
> requiring too much changes in one go (large changes tend to never happen).
>
> It seems the first step forward would be to introduce a composite field 
> with the following properties:
>
> class Foo(models.Model):
> x = models.IntegerField()
> y = models.IntegerField()
> point = models.CompositeField((x, y))
>
> foo.point would always return a named tuplet. You could customize the 
> return value to a full fledged object by implementing a custom subclass of 
> CompositeField, but Django will not have any opinion on what should happen 
> when you do:
> foo.x = 1
> foo.y = 2
> foo.point.x = 2
> foo.point.x == foo.x ???
>
> this was one of the main concerns in the DEP.
>
> In addition, on query side you should be able to use 
> .filter(point__x__lte=...), and you should be able to select 
> .values('point'). We also need migrations support, and likely changes in a 
> lot of other places in Django.
>
> The draft DEPs for this feature shouldn't be used as definite resource 
> when implementing this feature.
>
>  - Anssi
>
> On Thursday, August 20, 2015 at 6:49:11 PM UTC+3, Aron Podrigal wrote:
>>
>> Have a look at [1] it is a composite field implementation.
>>
>> [1] 
>> https://groups.google.com/forum/m/#!msg/django-developers/MZUcOE6-7GY/sZkBaHvC8wgJ
>> [2] 
>> https://github.com/django/deps/blob/master/draft/0191-composite-fields.rst
>> [3] 
>> https://github.com/django/deps/blob/master/draft/0192-standalone-composite-fields.rst
>> On Aug 20, 2015 10:31 AM, <boito...@gmail.com> wrote:
>>
>>>
>>>
>>> Le mardi 18 août 2015 01:36:28 UTC+2, Tim Graham a écrit :
>>>>
>>>> I think the general idea is captured in ticket #5929 -- Allow Fields to 
>>>> use multiple db columns (complex datatypes). Is that the gist of your 
>>>> proposal?
>>>>
>>>
>>> Thank you for this link! It seems to discuss the same end result as what 
>>> I tried to present in my first message: the ability to have a single 
>>> models.Field managing an arbitrary number of DB columns under the hood.
>>>
>>> The proposed approach is perhaps a bit different: if I understood the 
>>> ticked correctly, it proposes to change the base Field class to make it 
>>> possible, when deriving from it, to manage one or several DB columns. My 
>>> first idea was more to mimic the composite pattern implementation already 
>>> in use with forms.MultiValueField:
>>> * The models.Field *leaf* classes would still manage a single DB column.
>>> * Introduce a models.MultiField class, which is a container of 
>>> models.Field classes (be it leaf classes or other MultiField classes). This 
>>> container would address the multiple columns indirectly, through the 
>>> interface of the composing fields. And, to the eyes of the rest of the 
>>> code, it would behave as a normal field, notably offering the to_python() 
>>> feature, hiding the composition in its implementation details.
>>>
>>> I did not take time yet to try and assemble a prototype of this idea; In 
>>> fact, I first wanted to confirm if such approach has not already been 
>>> rejected in the past, before investing work in it ;) 
>>>
>>> Does it sound like a feasible/interesting idea ? Or is there a good 
>>> reason not to do it / too many obvious technical complications that I did 
>>> not foresee ?
>>>
>>> Thank you for reading,
>>>   Ad
>>>
>>>
>>>> https://cod

Re: JsonResponse and list values

[schinckel]

My attempts to reproduce it (I have quite a lot of code that returns JSON 
arrays) on the smattering of browsers I had handy resulted in no apparent 
vulnerabilities.

On Tuesday, February 17, 2015 at 10:30:20 AM UTC+10:30, Tom Christie wrote:
>
> I haven't dug out sources on this, but I think that vulnerability had been 
> dealt with a long time ago, and isn't an issue on browsers with any real 
> market share today. 
>
> Would need to double check though - don't remember where I came across it 
> previously.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/df61d6a0-48d7-49fc-bb94-227e73e83c48%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Migration challenges

[schinckel]

On Wednesday, September 24, 2014 8:24:28 AM UTC+9:30, Shai Berger wrote:
>
> Hi all, 
>
> I gave a talk to a local user-group about the migrations in 1.7, and some 
> people in the audience raised things they would like to be able to do, and 
> are 
> not supported by the current framework; thought it would be nice to bring 
> them 
> here. 
>
> Two issues were about handling migrations at the project level rather than 
> app 
> level: 
>
> 1) Cross-app migrations. The act of moving a model from one app to another 
> should be a common-enough refactoring; especially for beginners, breaking 
> a 
> piece of an app out to a separate app is something a user is very likely 
> to 
> want. Current migrations, being app-oriented, make this possible, but a 
> little 
> awkward. 
>

I had a need to do something similar to this: basically adding an extra 
attribute
to a contrib app (and the database column too). It is possible to write a 
custom
migration operation that allows you to alter a model that is not part of 
the same
app, but, yes, it's a pain in the proverbial.

I guess the 'changing app of a model' is something that could appear.

However, it's going to require having a db_table entry in Meta at the least,
or migrations to rename the table. I guess you could replace the creation
migration with one that renames the other one.
 

> 2) Roll back project to point in history. This is requested by people who 
> want 
> to work on feature branches (or any separate branches, each with its own 
> migrations). It is a bit of a hard problem I've run int myself, and I 
> suspect 
> it requires some integration with source-control systems to be done right. 
> The 
> solution I recommended (and used) was to keep separate databases for 
> separate 
> branches, but that is quite cumbersome in a large project. 
>

It's no different to what exists with south though, or any other migration 
system.

The backwards migrations need to be done while the checked out version
contains them. You could have a pre-update hook that examines if the 
changeset
you are checking out is missing any migrations, and warn that reverse 
migrations
may need to be run first.

Matt.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/dad40f8c-92b7-47d4-8fd0-5d06ecbab42f%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Proposal: Database Constraints

[schinckel]

On Wednesday, April 2, 2014 3:35:24 AM UTC+10:30, Andrew Godwin wrote:
>
> Hmm, I'm not sure about this. On the one hand, during the migrations work 
> I refactored check constraints for the existing stuff (e.g. 
> PostitiveIntegerField) out of the type string and into a separate check 
> constraints string, so some of the work is already there, especially around 
> making sure they're present on field alteration.
>
> On the other hand, check constraints vary wildly from database to database 
> - MySQL doesn't have them, SQLite has a basic version in later versions 
> only and parses but ignores them in earlier versions, PostgreSQL has full 
> support and Oracle has them too but with a different set of functions to 
> PostgreSQL, so it's unlikely anything other than simple comparison would be 
> portable.
>

Thanks Andrew. I haven't touched other databases for a long time, so it's 
good to get that feedback.
 

> For that reason, I wouldn't be the fan of the exploded syntax you've 
> given; I think I'd rather just be able to supply a list of strings for my 
> constraints that are SQL themselves; there's no real benefit to abstraction 
> here, and removing it doubtless removes a lot of code and bugs. Otherwise, 
> I think I'd be +0.
>

Pushing on from there: it's not much work from doing "constraints = ('check 
start < finish',)" to just writing a RunSQL migration operation. The 
exploded syntax was more me working things through late at night anyway. 
And it seems like there's not really much else to do other than raw SQL for 
exclude constraints.

So, it's actually possible to do _most_ of what I want already - just using 
a migration. The per-field check constraint could/should also be 
implemented as a validator. Exclude constraints are another kettle of fish: 
as I mentioned before, it's not actually possible to validate these 
constraints without hitting the database, and even then that's not alway 
enough. You still need to handle IntegrityErrors if an update between when 
you checked and when you actually try to save causes a conflict.

Matt.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/378962de-7ae6-496e-8a85-88c964d43e78%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Proposal: Database Constraints

[schinckel]

Some years ago, I discussed adding database-level check constraints into 
django: https://groups.google.com/forum/#!topic/django-developers/OJN5kpcseAg

There is also an issue: https://code.djangoproject.com/ticket/11964

I'm thinking about revisiting this, and wanted to get some discussion going 
about if this is a viable thing to do, and what it might look like.


Django already uses check constraints with PositiveIntegerField and 
friends, and at first blush I thought we might be able to co-opt some of 
that (indeed, I've got an internal monkey-patch that does, with some level 
of success). However, other than the fact there is already a 
'sql_create_check' string, the actual python code that creates the check 
constraint probably isn't that usable in this context. Also, I like the 
idea of having more complex constraints (postgres has EXCLUDE constraints, 
but I don't know if there is an equivalent for other backends).


The approach that I am thinking of could see a syntax similar to:


class MyModel(models.Model):
start = models.DateField()
finish = models.DateField(constraints=[('check', '> start')])
user = models.ForeignKey('auth.User')

This maps directly to creating a check constraint on the table:

 ALTER TABLE "myapp_mymodel" ADD CONSTRAINT CHECK (finish > start)

And, on the same model, a more complex constraint could look like:


class Meta:
constraints = [
('exclude', [
('overlaps', ['start','finish']),
('equal', 'user')
])
   ]

I'm still unsure of the best way to describe this: it's supposed to mean:

   ALTER TABLE "myapp_mymodel" ADD EXCLUDE USING gist (daterange(start, 
finish) WITH &&, user_id WITH =)

(but the python syntax is obviously immaterial at this stage).


Obviously, we can't just rely on the database to do the validation for us: 
it will just raise DatabaseErrors when something fails to validate anyway, 
so we would want to handle this stuff in django's validation framework.

One possibility, at least with the field-based check constraint, would be 
to automatically add a field validator in the case of a CHECK constraint, 
however in the case of an EXCLUDE constraint, we can't really validate 
_without_ hitting the database. Does this mean we should treat EXCLUDE-type 
constraints as something that is beyond the scope of Django?



With the new migrations framework, it's actually trivial to write a 
migration to add this type of constraint (or the check constraint), and a 
pre_save signal handler in conjunction with that would get 90% of the way, 
but it's still going to be open to a race condition anyway - the only way 
to actually do that is to try to save the object and see what the database 
says.


So, I guess I'm asking: is it worth pursuing this further, either in part 
or full?

Matt.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/250ed272-198a-478f-b2ce-920afcf533a2%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: #20824 - Email-based auth contrib module; some help required

[schinckel]

On Friday, February 28, 2014 12:55:31 PM UTC+10:30, waylan wrote:
>
> On Thursday, February 27, 2014 6:50:38 PM UTC-5, Camilo Torres wrote:
>>
>> normalize_email will indeed allow both us...@example.com and 
>> us...@example.com to be different entities. From the user perspective, 
>> this is an error. Most probably the user enters some day their email in all 
>> upper case because he pressed the CapsLock key, or copy pasted a transcript 
>> of his email, etc., the user could not be able to figure out the difference 
>> and simply could not log in.
>>
>
>
[snip]
 

>  
> The point is that the user will inadvertently type 
> u...@example.com on 
> their mobile device after registering with us...@example.com  on 
> their desktop/laptop. And they won't understand why they can't log in if it 
> is case sensitive. Given that both addresses will deliver email to the same 
> email account, it seems to me that both should be valid for the same 
> account as an email based username on a sign-in form.
>

But there's the rub. Whilst for _most_ email servers, this will indeed send 
mail to the same account, there's nowhere that says this _must_ happen.

Relevant: 
http://stackoverflow.com/questions/9807909/are-email-addresses-case-sensitive

Matt. 

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/ab6fb338-252e-42f5-a83f-fd414d49432c%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

db_type is called more than documented.

[schinckel]

I maintain the django-jsonfield module, and I have, as the documentation 
suggests, some relatively expensive code in JSONField.db_type.

The documentation is pretty clear on this as being _the_ place to put a 
test like this (that checks to see if the database can handle a json field 
type, or if we should just store it as text).


https://docs.djangoproject.com/en/dev/howto/custom-model-fields/#custom-database-types

"The 
db_type()
 method 
is only called by Django when the framework constructs the CREATE TABLE 
statements 
for your application – that is, when you first create your tables. It’s not 
called at any other time, so it can afford to execute slightly complex 
code, such as the connection.settings_dict check in the above example."

However, I noticed that the SQL query I was running to check for if I 
should use a json field was running way more than expected.

It turns out that this method is called at:


https://github.com/django/django/blob/master/django/db/models/sql/where.py#L368

All other calls to this method appear to be in CREATE statements.

Is this something that just slipped under the radar? Is it something that 
should be fixed in documentation? Any suggestions?

Regards,

Matt.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/3b0084b2-da32-420b-a9b1-dc4bcd4ffe46%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Model field metaclass magic (or lack thereof)

[schinckel]

Hi,

I was wondering if there was any reason why fields such as
models.DateField() do not use the SubFieldBase metaclass
trick to ensure they always contain instances of the correct
class?

I'm referring to the description 
from 
https://docs.djangoproject.com/en/dev/howto/custom-model-fields/#modelforms-and-custom-fields

I had a look though the tickets, but was unable to find any discussion on 
this.

Is there a rationalé for not using this?

Matt.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/21d71a45-f9d6-41bb-bba9-fce0a35b41ae%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: json vs simplejson

[schinckel]

The other thing this breaks is using **kwargs with something loaded from 
JSON.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/django-developers/-/ynPxXsJZUB8J.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Multi-tenant Django

[schinckel]

I played around a bit with using this, and got something quite workable 
going on.

https://bitbucket.org/schinckel/django-multi-schema

It's really only an exploration, but does show that it can be done.

The problems actually really only got hard once you factor south into the 
equation.

There are a few little hacky things that needed to be done (like setting 
the search path),
and it probably only works on postgres, but as a concept I still feel like 
it has legs.

There are also a couple of other approaches out there, too.

Matt.


On Thursday, May 24, 2012 3:36:58 AM UTC+9:30, Anssi Kääriäinen wrote:
>
> On May 23, 3:16 am, Anthony Briggs <anthony.bri...@gmail.com> wrote: 
> > I did a similar thing for some of my projects - the problem is that you 
> > can't reuse any non-multitenant apps without hacking the multitenant 
> stuff 
> > in first, ie. it's a bit of a hack, plus it makes things harder to 
> maintain 
> > going forwards. 
> > 
> > One database per tenant should (in theory) make life much simpler. 
>
> One schema per tenant would also be pretty good. It makes it much 
> easier to do cross-db queries, or use a public schema for common 
> stuff. This should be doable if/when the support for database schemas 
> is included. I got pretty far in that feature in ticket #6148, but it 
> turns out to be pretty complex to support introspection and creation 
> on all the databases for both production and testing. So, it remains 
> to see if we want to add all that complexity. Still, multitenancy 
> support seems to be one use case where the database schemas support 
> could be very useful in Django. 
>
>  - Anssi

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/django-developers/-/YNGondmretoJ.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Awkwardness of overriding View.dispatch for method-agnostic request-handling

[schinckel]

I agree: but this means that the actual dispatcher (that, according to the 
comments,
"dispatch[es] to the right method", is called handle(), rather than 
dispatch.

Perhaps the assignation of request/args/kwargs could happen before dispatch 
is 
called?

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/django-developers/-/s6wUo1ejoFoJ.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: [GSOC 2012] Customizable serialization

[schinckel]

I am quite interested in this, as most of my coding work has been 
developing APIs that get consumed by non-web-browser software.

Lately, I've taken the approach that a Form is the appropriate tool to use 
for (de)serialisation: it's already used extensively by the template 
rendering (which is indeed serialisation of data).

The workflow I use is:

Model instance -> Model form -> pre-serialised (python) data structure -> 
serialised data

That way, the only bits of code that are required are the pre-serialiser: 
which takes a Form object (possibly a ModelForm, but not necessarily), and 
returns a python dict. This handles the nested pre-serialisation, for 
instance.

Then, for each serialisation type, you need a serialiser that converts that 
dict into the relevant type. For vanilla JSON, this is simply json.dumps(), 
and it is trivial to get a simple XML output too.


As for going back the other way: the data that comes in can be 
deserialised, and then passed directly to the form(), just as you would 
with request.POST.


Matt.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/django-developers/-/12mcu7EJmDQJ.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: JsonField

[schinckel]

I'm doing just this in several cases: I packaged up a JSONField into a 
re-usable 
application: https://bitbucket.org/schinckel/django-jsonfield/overview

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/django-developers/-/XrB-bbFjvZcJ.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: I want to pass the Exception info from Http404 to handler404 view and consequently the 404.html

[schinckel]

The way I am handling this type of thing (and this is getting into 
django-users territory), is to either:

- have a middleware that catches exceptions, and if they are a sub-class of 
HttpError, then return them,

- have a framework which handles this, often by having the CBV handle it in 
it's dispatch method.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/django-developers/-/KgRMAnoaz7wJ.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Django test documentation example not respecting xUnit convention

[schinckel]

Yeah, I saw that later when I wrote some tests.

I'm sure I saw the style of failure message with some django app tests I 
wrote ages ago: maybe my brain has failed that test and it was a ruby unit 
test.

Matt.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/django-developers/-/vZFC4lC3hUIJ.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Django test documentation example not respecting xUnit convention

[schinckel]

It isn't 'enforced' by Python at a language level, but as dmoisset stated, 
it makes the failure messages actually make sense:

"Expected 'foo', got 'bar'".

(paraphrasing failure message: don't have any failing tests to look at right 
now. YAY! :)

Matt.

>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/django-developers/-/3B4s7zenKmIJ.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Should user passwords be more strict?

[schinckel]

Attached is a post that requires all passwords to be:

'correct horse battery staple'

:)

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/django-developers/-/GBAQF7IoHhYJ.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: django.contrib.admin and null=True

[schinckel]

On Dec 9, 1:57 pm, schinckel <m...@schinckel.net> wrote:
> On Dec 8, 2:02 pm, nasp <charett...@gmail.com> wrote:
>
> > You might consider 
> > readinghttp://docs.djangoproject.com/en/dev/ref/models/fields/#null.
>
> Thanks: that was the link I needed.
>
> However, I do take exception with the comment:
>
>     If a string-based field has null=True, that means it has two
> possible values for “no data”: NULL, and the empty string.
>
> An empty string means something different in the context of a
> relational database than a NULL value (as -RAX- and Andrew hint at
> below).  It goes a bit deeper than just unique constraints (although I
> have hit this several times), but also impacts upon relational
> algebra. (NULL and TRUE => NULL, for instance). Even just as a string,
> an empty string is different to NULL. An empty string means "I know
> what the value is, and it is a string of no length", compared to one
> use of a NULL, as "I don't know what the value is (yet)."
>
> In addition, NULL behaves vastly differently to an empty string when
> using COALESCE() or BOOL().
>
>     SELECT COALESCE(NULL, '', 'FOO'); => ''
>     SELECT BOOL(''); => ERROR
>
> Now, if you stay in django-land, this is probably not going to bite
> you too much (unless you want an optional IPAddress field in a
> Postgres db), but if you sometimes have to hand-tune queries (or
> *gasp* create stored procedures in your database), then you lose the
> ability to use COALESCE, for instance.

Having said all of that, I've just implemented a solution (using
pre_save), but there is an issue: it is not possible to know from the
admin interface returned value if the user meant to have a None value,
or an empty string.

Matt.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: django.contrib.admin and null=True

[schinckel]

On Dec 8, 9:24 pm, -RAX-  wrote:
> By default Admin saves an empty string in those TextFields and
> CharFields defined as null = True.
>
> Whatever the semantic reasons are, this default behavior creates
> problems in those fields which are both unique = True and null = True
> because by saving an empty string they do not respect that null !=
> null.

Exactly!

> A work around of would be to override the form used by the admin
> interface replacing the empty strings with None.

Yeah, that's what I was doing. I do like Andrew's solution more
though: a global pre_save hook.

Matt.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: django.contrib.admin and null=True

[schinckel]

On Dec 8, 2:02 pm, nasp  wrote:
> You might consider 
> readinghttp://docs.djangoproject.com/en/dev/ref/models/fields/#null.

Thanks: that was the link I needed.

However, I do take exception with the comment:

If a string-based field has null=True, that means it has two
possible values for “no data”: NULL, and the empty string.

An empty string means something different in the context of a
relational database than a NULL value (as -RAX- and Andrew hint at
below).  It goes a bit deeper than just unique constraints (although I
have hit this several times), but also impacts upon relational
algebra. (NULL and TRUE => NULL, for instance). Even just as a string,
an empty string is different to NULL. An empty string means "I know
what the value is, and it is a string of no length", compared to one
use of a NULL, as "I don't know what the value is (yet)."

In addition, NULL behaves vastly differently to an empty string when
using COALESCE() or BOOL().

SELECT COALESCE(NULL, '', 'FOO'); => ''
SELECT BOOL(''); => ERROR

Now, if you stay in django-land, this is probably not going to bite
you too much (unless you want an optional IPAddress field in a
Postgres db), but if you sometimes have to hand-tune queries (or
*gasp* create stored procedures in your database), then you lose the
ability to use COALESCE, for instance.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: django.contrib.admin and null=True

[schinckel]

On Dec 8, 9:28 pm, Andrew Godwin <and...@aeracode.org> wrote:
> On 07/12/10 23:26, schinckel wrote:
>
>
>
>
>
> > I haven't been able to find any documentation about this, but would be
> > happy to be pointed in the right direction.
>
> > When you use null=True in a field, and then use that model in the
> > admin, it will not save NULL to the database, but will instead save an
> > empty string (or attempt to).
>
> > I think this is broken behaviour: NULL values are semantically
> > different to empty strings, and in certain cases (I think it was
> > IPAddressField, but I will have another look later), it would fail to
> > write to the database.
>
> > Is there a reason that the admin interface saves what should be a NULL
> > value as an empty string? Do I report this as a bug?
>
> > Matt.
>
> Further to the other two replies, this is a known issue, and affects
> things like unique constraints too. The relevant bug 
> -http://code.djangoproject.com/ticket/9590- was closed as WONTFIX a
> while ago.

Yes, but don't we have new world order now? (GRIN).


> General opinion last time I asked about this (it bites me occasionally)
> is that there's a divide between people who use NULL, and between people
> who think there should be one empty value, but even if fixed it'd
> probably break backwards compatibility in so many subtle ways, it's
> unlikely to occur.

> My suggested workaround is to add a pre_save hook that manually fixes
> the fields up. Someone I know has one that runs against all saves on all
> models, and auto-corrects any "" values in nullable fields to None
> before saving them.

This looks like a good solution. I was just cleaning up the data in
the clean() method on a form,
but this is nicer. And global.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

django.contrib.admin and null=True

[schinckel]

I haven't been able to find any documentation about this, but would be
happy to be pointed in the right direction.

When you use null=True in a field, and then use that model in the
admin, it will not save NULL to the database, but will instead save an
empty string (or attempt to).

I think this is broken behaviour: NULL values are semantically
different to empty strings, and in certain cases (I think it was
IPAddressField, but I will have another look later), it would fail to
write to the database.

Is there a reason that the admin interface saves what should be a NULL
value as an empty string? Do I report this as a bug?

Matt.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Bug #348 and Safari.

[schinckel]

It seems Safari (Mac, at least) is still plagued by what seems to be
the behaviour described by http://code.djangoproject.com/ticket/348 -
when you have a ManyToMany, and have turned it into a javascript-
enabled filter box pair (SelectFilter2.js), then you get funky
behaviour.

This is fine in Firefox/Camino, but still has duplicates (and no
pattern to them that I could find).

Is this reason enough to reopen this ticket?

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: why last_login in django.contrib.auth.models.User cannot be null?

[Matt Schinckel]

On Dec 15, 8:59 am, Sergiy Kuzmenko  wrote:
> I wonder if there is a particular reason why last_login field of  is not
> defined as "null=True"? It makes sense to me to have it as null which would
> mean that the user never logged in. Could there be any dependencies relying
> on this field not being null?
>

What isn't clear is that this value must be after 1900: I had used a
value of datetime(1,1,1,0,0) as "never logged in", but this fails with
password reset.

Matt.

--

You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: why last_login in django.contrib.auth.models.User cannot be null?

[Matt Schinckel]

On Dec 15, 8:59 am, Sergiy Kuzmenko  wrote:
> I wonder if there is a particular reason why last_login field of  is not
> defined as "null=True"? It makes sense to me to have it as null which would
> mean that the user never logged in. Could there be any dependencies relying
> on this field not being null?

I came across one today:
contrib.auth.tokens.PasswordResetTokenGenerator
has a method _make_token_with_timestamp, which uses last_login to
create the
token. This means that if someone generates a password reset request,
the token
will be invalidated if that user then logs in. This could occur if a
person creates
password reset requests for a user that is not themself.

Matt.

--

You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Feature Request: Reinteract

[Matt Schinckel]

On Nov 27, 10:07 pm, noel  wrote:
> I really love this application Reinteract. Its an enhancement to
> Python Interactive Shell. And it would be lovely if I can use
> Reinteract with manage.py shell.

Have a look at bpython.  I have a command set up where I can run

./manage.py bshell

And I get a bpython shell with all of the models already imported.

--

You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Check Constraints for databases that support them

[Matt Schinckel]

I'm very interested in the idea of expanding the database level
constraints that can be supplied to a database schema, and also
automatically apply the same constraints to the model/view in django.

The various backends that support them, I believe already apply such
constraints for PositiveIntegerField() and so on.

I would like to see the ability to define extra constraints as part of
the model definition, and have these applied in a similar manner to
the unique and unique_together constraints.

I am more familiar with PostgreSQL than anything else, but I believe
the syntax I am about to propose will work in several databases.

Check constraints can be divided into two types: column level and
table level.  With PostgreSQL, a column level constraint can reference
another column, but I would suggest that a constraint that does so
should be a table level constraint.

Column level constraints could be declared as a keyword argument to
the field constructor:

column = models.IntegerField(constraint='< 100')

Table level constraints could be declared as an attribute on the Meta
class of the model:

check_constraints = ('start < finish', )

Validation of the models should pick up when an incorrect constraint
has been declared. For instance, not using a valid operator ('# 100')
or not supplying a valid comparison value ('< 1e' for an IntegerField)
would be invalid column constraints, and comparing a Date to an
Integer would be an invalid table level constraint.

The code that should be generated would add "CHECK ('column' < 100)"
to the column definition in the first instance, and "CONSTRAINT
app_table_start_finish CHECK ('start' < 'finish')" to the table
definition in the second case.

I have already written code that fulfils these requirements, and some
tests for validation errors.  I have also written additions to
BaseModelForm which validates according to the defined constraint(s).

At this stage, I have not allowed for more complicated relationships,
such as ('first < second + 65'), but this is planned.

Is there anyone else interested in this?  Should I put this into a
ticket and attach my current diff?

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Check Constraints for databases that support them

[Matt Schinckel]

On Sep 26, 10:27 pm, Tim Chase  wrote:
> > Is there anyone else interested in this?
>
> yes, I'd be interested in seeing some sort of database-level
> CHECK constraint as part of Django.  I had been sitting on my
> thoughts until I see the GSoC work done on model-validation wend
> its way towards trunk.  My hope had been to see model validation
> incorporate some DB-level CHECK constraints where feasible.

I did have a brief look at the django-check-constraint project on
google code, but it was overkill for what I needed.

> One of the other difficulties involves database expression
> differences.  For the simple cases such as you suggest, it's not
> as bad as they're fairly standard.  However, when functions are
> involved, each DB seems to have its own family of functions.
> E.g. if you want to assert the length of a string is 10
> characters ("len" vs. "strlen"?) or the time is during business
> hours ("hour(fieldname) between 8 and 17"...extracting
> time-portions varies across DB engines).

Yeah, for the time being I was only interested in simple cases.
Comparing against a static value in the case of column constraints,
and comparing two columns in the table-level case.  I don't even allow
for 'start < finish + 1' in the code I have written so far.

> I currently just add the CHECK constraints manually (well, in
> post-syncdb code).  Having them in a declarative fashion would
> help keep them in the right place.

Not to mention that it will allow the same code to control the db-
level constraint, and the django validation (admin, form).  Which is
DRY, and one of the reasons I am moving projects to django.

I also looked a little bit at south, which I have just started using
for migrations.  It doesn't have anything in there that is that
useful, other than raw SQL.  Which can do those type of things, but it
is nicer to move into the declaration.

Matt.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---