Fwd: Re : Validate that IntegerField is a valid Signed value

[Cal Leeming [Simplicity Media Ltd]]

Done

[Django] #15939 :
SignedIntegerField? 
 / 
UnsignedIntegerField?
as
part of the core fields.py

[Django] #15940 : Patch database
documentation to explain why using sql_mode=strict is important




On Sun, May 1, 2011 at 2:19 PM, Mathieu AGOPIAN
wrote:

> I really can't tell if those two new fields would be integrated in
> core, either wait for feedback here, and/or give it a try with a
> ticket (and i'm pretty sure it'll land in "Design Decision Needed").
>
> On May 1, 2:42 pm, "Cal Leeming [Simplicity Media Ltd]"
>  wrote:
> > Yeah, subclassing sounds the way forward. I was thinking something like:
> >
> > SignedIntegerField = IntegerField(min_value=–2147483648, max_value=
> > 2147483647)
> > UnsignedIntegerField = IntegerField(min_value=0, max_value=4294967295)
> >
> > Do you think there would be much chance in having these two field types
> > included in the core? If so, should I create a patch for fields.py and
> > submit??
> >
> > As for the MySQL strict mode, I'll submit a documentation patch, and see
> if
> > it gets accepted :)
> >
> > On Sun, May 1, 2011 at 11:58 AM, Mathieu AGOPIAN
> > wrote:
> >
> >
> >
> >
> >
> >
> >
> > > For reference, this discussion is linked to
> >
> > >http://groups.google.com/group/django-developers/browse_thread/thread.
> ..
> > > and to the tickethttp://code.djangoproject.com/ticket/15923
> >
> > > Cal,
> >
> > > The only thing that i could imagine regarding "fixing" this issue
> > > would indeed be a documentation addition in the MySQL (or MySQLdb)
> > > part of thehttp://docs.djangoproject.com/en/dev/ref/databases/page,
> > > explaining the possible issue with integers that are too large to fit
> > > in the appropriate mysql column (though i'm not sure how all this
> > > works, as i'm far from a MySQL or ORM expert).
> > > Specifically, explaining how to set the sql_mode to "traditional" as
> > > explained by kmtracey (http://code.djangoproject.com/ticket/
> > > 15923#comment:10) I believe.
> >
> > > Let's see if there's anyone reading this mailing list showing interest
> > > about this, and/or feel free to create a feature request on trac, and
> > > see if it's accepted.
> >
> > > On a side note, as python doesn't seem to have any issue with large
> > > integers, i guess you could subclass the IntegerField, and add to it's
> > > validation a check to see if the resulting integer can be stored in a
> > > 32 bits using either a simple comparison, like the following:
> > >-int('1' * 31, 2) < int(field_value) < int('1' * 31, 2)
> >
> > > Mathieu
> >
> > > On Apr 30, 8:31 pm, "Cal Leeming [Simplicity Media Ltd]"
> > >  wrote:
> > > > Hey Mathieu,
> >
> > > > Thanks for taking the time to reply. I'm starting to see now why the
> core
> > > > devs are reluctant to modify IntegerField.
> >
> > > > I'm wondering if maybe Django should have a SignedIntegerField and
> > > > UnsignedIntegerField as part of the core (for those that wish to have
> > > > enforced 32-bit integers), with the same INT_MIN and INT_MAX from
> > > limits.h (http://en.wikipedia.org/wiki/Limits.h). But there again,
> would
> > > this be
> > > > considered un-pythonic or against the ethics of Django?
> >
> > > > I guess really it should be up to MySQL to have strict mode by
> default.
> > > But,
> > > > as this is unlikely to happen, could we perhaps consider having a
> > > commented
> > > > out entry in the settings.py file that allows you to set strict mode
> for
> > > all
> > > > SQL connections? Or, perhaps a documentation change, which explains
> > > easily
> > > > to the user how to do it (Kinda like the storage_engine thing
> onhttp://
> > > docs.djangoproject.com/en/dev/ref/databases/#creating-your-tables) .
> >
> > > > Let me know your thoughts :)
> >
> > > > Cal
> >
> > > > On Sat, Apr 30, 2011 at 6:32 PM, Mathieu AGOPIAN
> > > > wrote:
> >
> > > > > Hello,
> >
> > > > > I'm afraid there isn't such a thing as "a valid signed value", if
> we're
> > > > > still talking about "size wise".
> >
> > > > > For django (python), the integer you gave in the ticket is
> perfectly
> > > valid.
> > > > > Here's a way for you to check that :
> > > > > >>> s = '351760125423456632454565345363453423453465345453'
> > > > > >>> int(s)
> > > > > 351760125423456632454565345363453423453465345453L
> >
> > > > > And indeed, an IntegerField validates that the content of the field
> can
> > > be
> > > > > converted to an int this way (check django/forms/fields.py line
> 230).
> >
> > > > > So definitely, as Alex pointed, this is an issue on MySQL's side,
> not
> > > > > Django's.
> > > > > I believe this can't (shan't?) be fixed at Django's level, as
> there's

Re: Re : Validate that IntegerField is a valid Signed value

[Mathieu AGOPIAN]

I really can't tell if those two new fields would be integrated in
core, either wait for feedback here, and/or give it a try with a
ticket (and i'm pretty sure it'll land in "Design Decision Needed").

On May 1, 2:42 pm, "Cal Leeming [Simplicity Media Ltd]"
 wrote:
> Yeah, subclassing sounds the way forward. I was thinking something like:
>
> SignedIntegerField = IntegerField(min_value=–2147483648, max_value=
> 2147483647)
> UnsignedIntegerField = IntegerField(min_value=0, max_value=4294967295)
>
> Do you think there would be much chance in having these two field types
> included in the core? If so, should I create a patch for fields.py and
> submit??
>
> As for the MySQL strict mode, I'll submit a documentation patch, and see if
> it gets accepted :)
>
> On Sun, May 1, 2011 at 11:58 AM, Mathieu AGOPIAN
> wrote:
>
>
>
>
>
>
>
> > For reference, this discussion is linked to
>
> >http://groups.google.com/group/django-developers/browse_thread/thread...
> > and to the tickethttp://code.djangoproject.com/ticket/15923
>
> > Cal,
>
> > The only thing that i could imagine regarding "fixing" this issue
> > would indeed be a documentation addition in the MySQL (or MySQLdb)
> > part of thehttp://docs.djangoproject.com/en/dev/ref/databases/page,
> > explaining the possible issue with integers that are too large to fit
> > in the appropriate mysql column (though i'm not sure how all this
> > works, as i'm far from a MySQL or ORM expert).
> > Specifically, explaining how to set the sql_mode to "traditional" as
> > explained by kmtracey (http://code.djangoproject.com/ticket/
> > 15923#comment:10) I believe.
>
> > Let's see if there's anyone reading this mailing list showing interest
> > about this, and/or feel free to create a feature request on trac, and
> > see if it's accepted.
>
> > On a side note, as python doesn't seem to have any issue with large
> > integers, i guess you could subclass the IntegerField, and add to it's
> > validation a check to see if the resulting integer can be stored in a
> > 32 bits using either a simple comparison, like the following:
> >    -int('1' * 31, 2) < int(field_value) < int('1' * 31, 2)
>
> > Mathieu
>
> > On Apr 30, 8:31 pm, "Cal Leeming [Simplicity Media Ltd]"
> >  wrote:
> > > Hey Mathieu,
>
> > > Thanks for taking the time to reply. I'm starting to see now why the core
> > > devs are reluctant to modify IntegerField.
>
> > > I'm wondering if maybe Django should have a SignedIntegerField and
> > > UnsignedIntegerField as part of the core (for those that wish to have
> > > enforced 32-bit integers), with the same INT_MIN and INT_MAX from
> > limits.h (http://en.wikipedia.org/wiki/Limits.h). But there again, would
> > this be
> > > considered un-pythonic or against the ethics of Django?
>
> > > I guess really it should be up to MySQL to have strict mode by default.
> > But,
> > > as this is unlikely to happen, could we perhaps consider having a
> > commented
> > > out entry in the settings.py file that allows you to set strict mode for
> > all
> > > SQL connections? Or, perhaps a documentation change, which explains
> > easily
> > > to the user how to do it (Kinda like the storage_engine thing onhttp://
> > docs.djangoproject.com/en/dev/ref/databases/#creating-your-tables) .
>
> > > Let me know your thoughts :)
>
> > > Cal
>
> > > On Sat, Apr 30, 2011 at 6:32 PM, Mathieu AGOPIAN
> > > wrote:
>
> > > > Hello,
>
> > > > I'm afraid there isn't such a thing as "a valid signed value", if we're
> > > > still talking about "size wise".
>
> > > > For django (python), the integer you gave in the ticket is perfectly
> > valid.
> > > > Here's a way for you to check that :
> > > > >>> s = '351760125423456632454565345363453423453465345453'
> > > > >>> int(s)
> > > > 351760125423456632454565345363453423453465345453L
>
> > > > And indeed, an IntegerField validates that the content of the field can
> > be
> > > > converted to an int this way (check django/forms/fields.py line 230).
>
> > > > So definitely, as Alex pointed, this is an issue on MySQL's side, not
> > > > Django's.
> > > > I believe this can't (shan't?) be fixed at Django's level, as there's
> > no
> > > > "size" limitation for the IntegerField, as you would have on a
> > CharField
> > > > with the *max_length* attribute.
>
> > > > And no, limiting the length of the string won't work, as "2147483647"
> > isn't
> > > > the same length as "-2147483647", but is the same length as
> > "99" (if
> > > > we're taking the example of 2^32-1 as the max SIGNED INT value).
>
> > > > my two cents ;)
>
> > > > --
> > > > You received this message because you are subscribed to the Google
> > Groups
> > > > "Django developers" group.
> > > > To post to this group, send email to
> > django-developers@googlegroups.com.
> > > > To unsubscribe from this group, send email to
> > > > 

Re : Validate that IntegerField is a valid Signed value

[Cal Leeming [Simplicity Media Ltd]]

Yeah, subclassing sounds the way forward. I was thinking something like:

SignedIntegerField = IntegerField(min_value=–2147483648, max_value=
2147483647)
UnsignedIntegerField = IntegerField(min_value=0, max_value=4294967295)

Do you think there would be much chance in having these two field types
included in the core? If so, should I create a patch for fields.py and
submit??

As for the MySQL strict mode, I'll submit a documentation patch, and see if
it gets accepted :)

On Sun, May 1, 2011 at 11:58 AM, Mathieu AGOPIAN
wrote:

> For reference, this discussion is linked to
>
> http://groups.google.com/group/django-developers/browse_thread/thread/f0b8ddbda03a2d8e
> and to the ticket http://code.djangoproject.com/ticket/15923
>
> Cal,
>
> The only thing that i could imagine regarding "fixing" this issue
> would indeed be a documentation addition in the MySQL (or MySQLdb)
> part of the http://docs.djangoproject.com/en/dev/ref/databases/ page,
> explaining the possible issue with integers that are too large to fit
> in the appropriate mysql column (though i'm not sure how all this
> works, as i'm far from a MySQL or ORM expert).
> Specifically, explaining how to set the sql_mode to "traditional" as
> explained by kmtracey (http://code.djangoproject.com/ticket/
> 15923#comment:10) I believe.
>
> Let's see if there's anyone reading this mailing list showing interest
> about this, and/or feel free to create a feature request on trac, and
> see if it's accepted.
>
> On a side note, as python doesn't seem to have any issue with large
> integers, i guess you could subclass the IntegerField, and add to it's
> validation a check to see if the resulting integer can be stored in a
> 32 bits using either a simple comparison, like the following:
>-int('1' * 31, 2) < int(field_value) < int('1' * 31, 2)
>
> Mathieu
>
> On Apr 30, 8:31 pm, "Cal Leeming [Simplicity Media Ltd]"
>  wrote:
> > Hey Mathieu,
> >
> > Thanks for taking the time to reply. I'm starting to see now why the core
> > devs are reluctant to modify IntegerField.
> >
> > I'm wondering if maybe Django should have a SignedIntegerField and
> > UnsignedIntegerField as part of the core (for those that wish to have
> > enforced 32-bit integers), with the same INT_MIN and INT_MAX from
> limits.h (http://en.wikipedia.org/wiki/Limits.h). But there again, would
> this be
> > considered un-pythonic or against the ethics of Django?
> >
> > I guess really it should be up to MySQL to have strict mode by default.
> But,
> > as this is unlikely to happen, could we perhaps consider having a
> commented
> > out entry in the settings.py file that allows you to set strict mode for
> all
> > SQL connections? Or, perhaps a documentation change, which explains
> easily
> > to the user how to do it (Kinda like the storage_engine thing onhttp://
> docs.djangoproject.com/en/dev/ref/databases/#creating-your-tables) .
> >
> > Let me know your thoughts :)
> >
> > Cal
> >
> > On Sat, Apr 30, 2011 at 6:32 PM, Mathieu AGOPIAN
> > wrote:
> >
> >
> >
> >
> >
> >
> >
> > > Hello,
> >
> > > I'm afraid there isn't such a thing as "a valid signed value", if we're
> > > still talking about "size wise".
> >
> > > For django (python), the integer you gave in the ticket is perfectly
> valid.
> > > Here's a way for you to check that :
> > > >>> s = '351760125423456632454565345363453423453465345453'
> > > >>> int(s)
> > > 351760125423456632454565345363453423453465345453L
> >
> > > And indeed, an IntegerField validates that the content of the field can
> be
> > > converted to an int this way (check django/forms/fields.py line 230).
> >
> > > So definitely, as Alex pointed, this is an issue on MySQL's side, not
> > > Django's.
> > > I believe this can't (shan't?) be fixed at Django's level, as there's
> no
> > > "size" limitation for the IntegerField, as you would have on a
> CharField
> > > with the *max_length* attribute.
> >
> > > And no, limiting the length of the string won't work, as "2147483647"
> isn't
> > > the same length as "-2147483647", but is the same length as
> "99" (if
> > > we're taking the example of 2^32-1 as the max SIGNED INT value).
> >
> > > my two cents ;)
> >
> > > --
> > > You received this message because you are subscribed to the Google
> Groups
> > > "Django developers" group.
> > > To post to this group, send email to
> django-developers@googlegroups.com.
> > > To unsubscribe from this group, send email to
> > > django-developers+unsubscr...@googlegroups.com.
> > > For more options, visit this group at
> > >http://groups.google.com/group/django-developers?hl=en.
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers" group.
> To post to this group, send email to django-developers@googlegroups.com.
> To unsubscribe from this group, send email to
> django-developers+unsubscr...@googlegroups.com.
> For more options, visit this 

Re: Re : Validate that IntegerField is a valid Signed value

[Mathieu AGOPIAN]

For reference, this discussion is linked to
http://groups.google.com/group/django-developers/browse_thread/thread/f0b8ddbda03a2d8e
and to the ticket http://code.djangoproject.com/ticket/15923

Cal,

The only thing that i could imagine regarding "fixing" this issue
would indeed be a documentation addition in the MySQL (or MySQLdb)
part of the http://docs.djangoproject.com/en/dev/ref/databases/ page,
explaining the possible issue with integers that are too large to fit
in the appropriate mysql column (though i'm not sure how all this
works, as i'm far from a MySQL or ORM expert).
Specifically, explaining how to set the sql_mode to "traditional" as
explained by kmtracey (http://code.djangoproject.com/ticket/
15923#comment:10) I believe.

Let's see if there's anyone reading this mailing list showing interest
about this, and/or feel free to create a feature request on trac, and
see if it's accepted.

On a side note, as python doesn't seem to have any issue with large
integers, i guess you could subclass the IntegerField, and add to it's
validation a check to see if the resulting integer can be stored in a
32 bits using either a simple comparison, like the following:
-int('1' * 31, 2) < int(field_value) < int('1' * 31, 2)

Mathieu

On Apr 30, 8:31 pm, "Cal Leeming [Simplicity Media Ltd]"
 wrote:
> Hey Mathieu,
>
> Thanks for taking the time to reply. I'm starting to see now why the core
> devs are reluctant to modify IntegerField.
>
> I'm wondering if maybe Django should have a SignedIntegerField and
> UnsignedIntegerField as part of the core (for those that wish to have
> enforced 32-bit integers), with the same INT_MIN and INT_MAX from limits.h 
> (http://en.wikipedia.org/wiki/Limits.h). But there again, would this be
> considered un-pythonic or against the ethics of Django?
>
> I guess really it should be up to MySQL to have strict mode by default. But,
> as this is unlikely to happen, could we perhaps consider having a commented
> out entry in the settings.py file that allows you to set strict mode for all
> SQL connections? Or, perhaps a documentation change, which explains easily
> to the user how to do it (Kinda like the storage_engine thing 
> onhttp://docs.djangoproject.com/en/dev/ref/databases/#creating-your-tables) .
>
> Let me know your thoughts :)
>
> Cal
>
> On Sat, Apr 30, 2011 at 6:32 PM, Mathieu AGOPIAN
> wrote:
>
>
>
>
>
>
>
> > Hello,
>
> > I'm afraid there isn't such a thing as "a valid signed value", if we're
> > still talking about "size wise".
>
> > For django (python), the integer you gave in the ticket is perfectly valid.
> > Here's a way for you to check that :
> > >>> s = '351760125423456632454565345363453423453465345453'
> > >>> int(s)
> > 351760125423456632454565345363453423453465345453L
>
> > And indeed, an IntegerField validates that the content of the field can be
> > converted to an int this way (check django/forms/fields.py line 230).
>
> > So definitely, as Alex pointed, this is an issue on MySQL's side, not
> > Django's.
> > I believe this can't (shan't?) be fixed at Django's level, as there's no
> > "size" limitation for the IntegerField, as you would have on a CharField
> > with the *max_length* attribute.
>
> > And no, limiting the length of the string won't work, as "2147483647" isn't
> > the same length as "-2147483647", but is the same length as "99" (if
> > we're taking the example of 2^32-1 as the max SIGNED INT value).
>
> > my two cents ;)
>
> > --
> > You received this message because you are subscribed to the Google Groups
> > "Django developers" group.
> > To post to this group, send email to django-developers@googlegroups.com.
> > To unsubscribe from this group, send email to
> > django-developers+unsubscr...@googlegroups.com.
> > For more options, visit this group at
> >http://groups.google.com/group/django-developers?hl=en.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Re : Validate that IntegerField is a valid Signed value

[Cal Leeming [Simplicity Media Ltd]]

Hey Mathieu,

Thanks for taking the time to reply. I'm starting to see now why the core
devs are reluctant to modify IntegerField.

I'm wondering if maybe Django should have a SignedIntegerField and
UnsignedIntegerField as part of the core (for those that wish to have
enforced 32-bit integers), with the same INT_MIN and INT_MAX from limits.h (
http://en.wikipedia.org/wiki/Limits.h). But there again, would this be
considered un-pythonic or against the ethics of Django?

I guess really it should be up to MySQL to have strict mode by default. But,
as this is unlikely to happen, could we perhaps consider having a commented
out entry in the settings.py file that allows you to set strict mode for all
SQL connections? Or, perhaps a documentation change, which explains easily
to the user how to do it (Kinda like the storage_engine thing on
http://docs.djangoproject.com/en/dev/ref/databases/#creating-your-tables ) .

Let me know your thoughts :)

Cal


On Sat, Apr 30, 2011 at 6:32 PM, Mathieu AGOPIAN
wrote:

> Hello,
>
> I'm afraid there isn't such a thing as "a valid signed value", if we're
> still talking about "size wise".
>
> For django (python), the integer you gave in the ticket is perfectly valid.
> Here's a way for you to check that :
> >>> s = '351760125423456632454565345363453423453465345453'
> >>> int(s)
> 351760125423456632454565345363453423453465345453L
>
> And indeed, an IntegerField validates that the content of the field can be
> converted to an int this way (check django/forms/fields.py line 230).
>
> So definitely, as Alex pointed, this is an issue on MySQL's side, not
> Django's.
> I believe this can't (shan't?) be fixed at Django's level, as there's no
> "size" limitation for the IntegerField, as you would have on a CharField
> with the *max_length* attribute.
>
> And no, limiting the length of the string won't work, as "2147483647" isn't
> the same length as "-2147483647", but is the same length as "99" (if
> we're taking the example of 2^32-1 as the max SIGNED INT value).
>
> my two cents ;)
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers" group.
> To post to this group, send email to django-developers@googlegroups.com.
> To unsubscribe from this group, send email to
> django-developers+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/django-developers?hl=en.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re : Validate that IntegerField is a valid Signed value

[Mathieu AGOPIAN]

Hello,

I'm afraid there isn't such a thing as "a valid signed value", if we're 
still talking about "size wise".

For django (python), the integer you gave in the ticket is perfectly valid. 
Here's a way for you to check that :
>>> s = '351760125423456632454565345363453423453465345453'
>>> int(s)
351760125423456632454565345363453423453465345453L

And indeed, an IntegerField validates that the content of the field can be 
converted to an int this way (check django/forms/fields.py line 230).

So definitely, as Alex pointed, this is an issue on MySQL's side, not 
Django's.
I believe this can't (shan't?) be fixed at Django's level, as there's no 
"size" limitation for the IntegerField, as you would have on a CharField 
with the *max_length* attribute.

And no, limiting the length of the string won't work, as "2147483647" isn't 
the same length as "-2147483647", but is the same length as "99" (if 
we're taking the example of 2^32-1 as the max SIGNED INT value).

my two cents ;)

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re : Validate that IntegerField is a valid Signed value

[Mathieu AGOPIAN]

Hello,

I'm afraid there isn't such a thing as "a valid signed value", if we're 
still talking about "size wise".

For django (python), the integer you gave in the ticket is perfectly valid. 
Here's a way for you to check that :
>>> s = '351760125423456632454565345363453423453465345453'
>>> int(s)
351760125423456632454565345363453423453465345453L

And indeed, an IntegerField validates that the content of the field can be 
converted to an int this way (check django/forms/fields.py line 230).

So definitely, as Alex pointed, this is an issue on MySQL's side, not 
Django's.
I believe this can't (shan't?) be fixed at Django's level, as there's no 
"size" limitation for the IntegerField, as you would have on a CharField 
with the *max_length* attribute.

And no, limiting the length of the string won't work, as "2147483647" isn't 
the same length as "-2147483647", but is the same length as "99" (if 
we're taking the example of 2^32-1 as the max SIGNED INT value).

my two cents ;)

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.