Re: [Django] #36913: Optimise ChoiceField / MultipleChoiceField handling of duplicate submissions

[Django]

#36913: Optimise ChoiceField / MultipleChoiceField handling of duplicate
submissions
-+-
 Reporter:  Jake Howard  |Owner:  Afenomamy
 Type:   |   Status:  closed
  Cleanup/optimization   |
Component:  Forms|  Version:  6.0
 Severity:  Normal   |   Resolution:  fixed
 Keywords:   | Triage Stage:  Ready for
 |  checkin
Has patch:  1|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  0
Easy pickings:  0|UI/UX:  0
-+-
Comment (by Jacob Walls ):

 In [changeset:"8b7ea2bcdd7297941ca6beb72d93b9568e187349" 8b7ea2bc]:
 {{{#!CommitTicketReference repository=""
 revision="8b7ea2bcdd7297941ca6beb72d93b9568e187349"
 Refs #36913 -- Maintained error message determinism in
 MultipleChoiceField.validate().

 Used Django's OrderedSet datastructure instead of set() in
 MultipleChoiceField.validate()
 to prevent submission ordering from being discarded during validation.

 Thanks to Jacob Walls, JaeHyuck Sa, Jake Howard and Simon Charette for
 the reviews.
 }}}
-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To view this discussion visit 
https://groups.google.com/d/msgid/django-updates/0107019d2b97d65e-86c1b1a7-13f4-4bc3-b77b-bad7dc2ffda4-00%40eu-central-1.amazonses.com.

Re: [Django] #36913: Optimise ChoiceField / MultipleChoiceField handling of duplicate submissions

[Django]

#36913: Optimise ChoiceField / MultipleChoiceField handling of duplicate
submissions
-+-
 Reporter:  Jake Howard  |Owner:  Afenomamy
 Type:   |   Status:  closed
  Cleanup/optimization   |
Component:  Forms|  Version:  6.0
 Severity:  Normal   |   Resolution:  fixed
 Keywords:   | Triage Stage:  Ready for
 |  checkin
Has patch:  1|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  0
Easy pickings:  0|UI/UX:  0
-+-
Changes (by Jacob Walls ):

 * resolution:   => fixed
 * status:  assigned => closed

Comment:

 In [changeset:"26f8929f16c514bd9967cd78d8dcd7760b82cc92" 26f8929]:
 {{{#!CommitTicketReference repository=""
 revision="26f8929f16c514bd9967cd78d8dcd7760b82cc92"
 Fixed #36913 -- Optimized MultipleChoiceField.validate().
 }}}
-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To view this discussion visit 
https://groups.google.com/d/msgid/django-updates/0107019d2529e233-5ce09057-e283-4324-b372-a736125a5d98-00%40eu-central-1.amazonses.com.

Re: [Django] #36913: Optimise ChoiceField / MultipleChoiceField handling of duplicate submissions

[Django]

#36913: Optimise ChoiceField / MultipleChoiceField handling of duplicate
submissions
-+-
 Reporter:  Jake Howard  |Owner:  Afenomamy
 Type:   |   Status:  assigned
  Cleanup/optimization   |
Component:  Forms|  Version:  6.0
 Severity:  Normal   |   Resolution:
 Keywords:   | Triage Stage:  Ready for
 |  checkin
Has patch:  1|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  0
Easy pickings:  0|UI/UX:  0
-+-
Changes (by Jake Howard):

 * stage:  Accepted => Ready for checkin

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To view this discussion visit 
https://groups.google.com/d/msgid/django-updates/0107019d1ac8e0fd-51a7674e-168f-4923-b0e6-248e798e412f-00%40eu-central-1.amazonses.com.

Re: [Django] #36913: Optimise ChoiceField / MultipleChoiceField handling of duplicate submissions

[Django]

#36913: Optimise ChoiceField / MultipleChoiceField handling of duplicate
submissions
-+-
 Reporter:  Jake Howard  |Owner:  Afenomamy
 Type:   |   Status:  assigned
  Cleanup/optimization   |
Component:  Forms|  Version:  6.0
 Severity:  Normal   |   Resolution:
 Keywords:   | Triage Stage:  Accepted
Has patch:  1|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  0
Easy pickings:  0|UI/UX:  0
-+-

Old description:

> When a ChoiceField / MultipleChoiceField has 5 possible choices, but the
> form submits 25 values, the `choices` values are compared once per
> submitted value. If the submitted values are duplicates, the validation
> doesn't terminate early, but can still spend a lot of time unnecessarily
> validating values. This can be very slow when large (~30k) numbers of
> values are submitted.
>
> A suggested fix is to only validate the unique submitted values (for
> example `for val in set(value)`).
>
> This issue was reported to the Security Team, but deemed not a security
> issue due to the minimal impact when given reasonable input (in the
> bounds of the security policy).
>
> sugessted PR : https://github.com/django/django/pull/20960

New description:

 When a ChoiceField / MultipleChoiceField has 5 possible choices, but the
 form submits 25 values, the `choices` values are compared once per
 submitted value. If the submitted values are duplicates, the validation
 doesn't terminate early, but can still spend a lot of time unnecessarily
 validating values. This can be very slow when large (~30k) numbers of
 values are submitted.

 A suggested fix is to only validate the unique submitted values (for
 example `for val in set(value)`).

 This issue was reported to the Security Team, but deemed not a security
 issue due to the minimal impact when given reasonable input (in the bounds
 of the security policy).

--
Comment (by Jake Howard):

 [https://github.com/django/django/pull/20960 PR]
-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To view this discussion visit 
https://groups.google.com/d/msgid/django-updates/0107019d19fa02e7-6099c8f7-0eac-4e91-9f6a-ed137f62c24e-00%40eu-central-1.amazonses.com.

Re: [Django] #36913: Optimise ChoiceField / MultipleChoiceField handling of duplicate submissions

[Django]

#36913: Optimise ChoiceField / MultipleChoiceField handling of duplicate
submissions
-+-
 Reporter:  Jake Howard  |Owner:  Afenomamy
 Type:   |   Status:  assigned
  Cleanup/optimization   |
Component:  Forms|  Version:  6.0
 Severity:  Normal   |   Resolution:
 Keywords:   | Triage Stage:  Accepted
Has patch:  1|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  0
Easy pickings:  0|UI/UX:  0
-+-
Description changed by Afenomamy:

Old description:

> When a ChoiceField / MultipleChoiceField has 5 possible choices, but the
> form submits 25 values, the `choices` values are compared once per
> submitted value. If the submitted values are duplicates, the validation
> doesn't terminate early, but can still spend a lot of time unnecessarily
> validating values. This can be very slow when large (~30k) numbers of
> values are submitted.
>
> A suggested fix is to only validate the unique submitted values (for
> example `for val in set(value)`).
>
> This issue was reported to the Security Team, but deemed not a security
> issue due to the minimal impact when given reasonable input (in the
> bounds of the security policy).

New description:

 When a ChoiceField / MultipleChoiceField has 5 possible choices, but the
 form submits 25 values, the `choices` values are compared once per
 submitted value. If the submitted values are duplicates, the validation
 doesn't terminate early, but can still spend a lot of time unnecessarily
 validating values. This can be very slow when large (~30k) numbers of
 values are submitted.

 A suggested fix is to only validate the unique submitted values (for
 example `for val in set(value)`).

 This issue was reported to the Security Team, but deemed not a security
 issue due to the minimal impact when given reasonable input (in the bounds
 of the security policy).

 sugessted PR : https://github.com/django/django/pull/20960

--
-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To view this discussion visit 
https://groups.google.com/d/msgid/django-updates/0107019d10a167f1-a0174929-b680-411c-8974-b74e4f6986ee-00%40eu-central-1.amazonses.com.

Re: [Django] #36913: Optimise ChoiceField / MultipleChoiceField handling of duplicate submissions

[Django]

#36913: Optimise ChoiceField / MultipleChoiceField handling of duplicate
submissions
-+-
 Reporter:  Jake Howard  |Owner:  Afenomamy
 Type:   |   Status:  assigned
  Cleanup/optimization   |
Component:  Forms|  Version:  6.0
 Severity:  Normal   |   Resolution:
 Keywords:   | Triage Stage:  Accepted
Has patch:  1|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  0
Easy pickings:  0|UI/UX:  0
-+-
Changes (by Afenomamy):

 * has_patch:  0 => 1

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To view this discussion visit 
https://groups.google.com/d/msgid/django-updates/0107019d108d039c-5f16fff9-1af1-4ee1-b30f-449ae726e60d-00%40eu-central-1.amazonses.com.

Re: [Django] #36913: Optimise ChoiceField / MultipleChoiceField handling of duplicate submissions

[Django]

#36913: Optimise ChoiceField / MultipleChoiceField handling of duplicate
submissions
-+-
 Reporter:  Jake Howard  |Owner:  Afenomamy
 Type:   |   Status:  assigned
  Cleanup/optimization   |
Component:  Forms|  Version:  6.0
 Severity:  Normal   |   Resolution:
 Keywords:   | Triage Stage:  Accepted
Has patch:  0|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  0
Easy pickings:  0|UI/UX:  0
-+-
Changes (by Afenomamy):

 * owner:  Sarah Boyce => Afenomamy

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To view this discussion visit 
https://groups.google.com/d/msgid/django-updates/0107019cd1b25c91-608798cb-dbea-43ea-997c-5a2f728333b0-00%40eu-central-1.amazonses.com.

Re: [Django] #36913: Optimise ChoiceField / MultipleChoiceField handling of duplicate submissions

[Django]

#36913: Optimise ChoiceField / MultipleChoiceField handling of duplicate
submissions
-+-
 Reporter:  Jake Howard  |Owner:  Sarah
 Type:   |  Boyce
  Cleanup/optimization   |   Status:  assigned
Component:  Forms|  Version:  6.0
 Severity:  Normal   |   Resolution:
 Keywords:   | Triage Stage:  Accepted
Has patch:  0|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  0
Easy pickings:  0|UI/UX:  0
-+-
Comment (by anjaniacatus):

 Hello, I am Aina (Team Saturn) from Djangonaut Space. I will be looking at
 this ticket
-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To view this discussion visit 
https://groups.google.com/d/msgid/django-updates/0107019ccd334d1e-4fb330fc-a26e-4f46-8558-e7223e538e49-00%40eu-central-1.amazonses.com.

Re: [Django] #36913: Optimise ChoiceField / MultipleChoiceField handling of duplicate submissions

[Django]

#36913: Optimise ChoiceField / MultipleChoiceField handling of duplicate
submissions
-+-
 Reporter:  Jake Howard  |Owner:  Sarah
 Type:   |  Boyce
  Cleanup/optimization   |   Status:  assigned
Component:  Forms|  Version:  6.0
 Severity:  Normal   |   Resolution:
 Keywords:   | Triage Stage:  Accepted
Has patch:  0|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  0
Easy pickings:  0|UI/UX:  0
-+-
Comment (by Natalia Bidart):

 Replying to [comment:3 Abhimanyu Singh Negi]:
 > I’d like to work on this issue.
 >
 > From what I understand the slowdown happens because duplicate submitted
 values are validated again and again. I’m thinking of validating only the
 unique values internally while keeping the original list unchanged so
 behaviour stays the same. I’ll add some tests too to make sure duplicates
 and invalid values are still handled correctly.
 >
 > If this sounds reasonable and no one is working on this, i would like to
 assign the issue to myself.

 This ticket has been assigned already and has been reserved to evaluate
 under the Djangonaut Space umbrella. Sorry!
-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To view this discussion visit 
https://groups.google.com/d/msgid/django-updates/0107019c4e9f5630-96b09d4b-ab64-4e91-b7fd-88d7432fcaad-00%40eu-central-1.amazonses.com.

Re: [Django] #36913: Optimise ChoiceField / MultipleChoiceField handling of duplicate submissions

[Django]

#36913: Optimise ChoiceField / MultipleChoiceField handling of duplicate
submissions
-+-
 Reporter:  Jake Howard  |Owner:  Sarah
 Type:   |  Boyce
  Cleanup/optimization   |   Status:  assigned
Component:  Forms|  Version:  6.0
 Severity:  Normal   |   Resolution:
 Keywords:   | Triage Stage:  Accepted
Has patch:  0|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  0
Easy pickings:  0|UI/UX:  0
-+-
Comment (by Abhimanyu Singh Negi):

 I’d like to work on this issue.

 From what I understand the slowdown happens because duplicate submitted
 values are validated again and again. I’m thinking of validating only the
 unique values internally while keeping the original list unchanged so
 behaviour stays the same. I’ll add some tests too to make sure duplicates
 and invalid values are still handled correctly.

 If this sounds reasonable and no one else is working on it, i would like
 to assign the issue to myself.
-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To view this discussion visit 
https://groups.google.com/d/msgid/django-updates/0107019c4e25fec6-853b7690-ec62-4321-a658-bde2d8513433-00%40eu-central-1.amazonses.com.

Re: [Django] #36913: Optimise ChoiceField / MultipleChoiceField handling of duplicate submissions

[Django]

#36913: Optimise ChoiceField / MultipleChoiceField handling of duplicate
submissions
-+-
 Reporter:  Jake Howard  |Owner:  Sarah
 Type:   |  Boyce
  Cleanup/optimization   |   Status:  assigned
Component:  Forms|  Version:  6.0
 Severity:  Normal   |   Resolution:
 Keywords:   | Triage Stage:  Accepted
Has patch:  0|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  0
Easy pickings:  0|UI/UX:  0
-+-
Comment (by Abhimanyu Singh Negi):

 I’d like to work on this issue.

 From what I understand the slowdown happens because duplicate submitted
 values are validated again and again. I’m thinking of validating only the
 unique values internally while keeping the original list unchanged so
 behaviour stays the same. I’ll add some tests too to make sure duplicates
 and invalid values are still handled correctly.

 If this sounds reasonable, i would like to assign the issue to myself.
-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To view this discussion visit 
https://groups.google.com/d/msgid/django-updates/0107019c4df4fce9-8e845fb5-6930-475f-9eb6-daeed9f5d822-00%40eu-central-1.amazonses.com.

Re: [Django] #36913: Optimise ChoiceField / MultipleChoiceField handling of duplicate submissions

[Django]

#36913: Optimise ChoiceField / MultipleChoiceField handling of duplicate
submissions
-+-
 Reporter:  Jake Howard  |Owner:  Sarah
 Type:   |  Boyce
  Cleanup/optimization   |   Status:  assigned
Component:  Forms|  Version:  6.0
 Severity:  Normal   |   Resolution:
 Keywords:   | Triage Stage:  Accepted
Has patch:  0|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  0
Easy pickings:  0|UI/UX:  0
-+-
Changes (by Jacob Walls):

 * stage:  Unreviewed => Accepted

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To view this discussion visit 
https://groups.google.com/d/msgid/django-updates/0107019c499ccc60-a6f064c1-e3ad-4b24-b094-622cdb23d988-00%40eu-central-1.amazonses.com.

Re: [Django] #36913: Optimise ChoiceField / MultipleChoiceField handling of duplicate submissions

[Django]

#36913: Optimise ChoiceField / MultipleChoiceField handling of duplicate
submissions
-+-
 Reporter:  Jake Howard  |Owner:  Sarah
 Type:   |  Boyce
  Cleanup/optimization   |   Status:  assigned
Component:  Forms|  Version:  6.0
 Severity:  Normal   |   Resolution:
 Keywords:   | Triage Stage:
 |  Unreviewed
Has patch:  0|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  0
Easy pickings:  0|UI/UX:  0
-+-
Comment (by Jake Howard):

 Note that validation that submissions (and choices) are unique is being
 handled in a separate feature request: https://github.com/django/new-
 features/issues/121
-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To view this discussion visit 
https://groups.google.com/d/msgid/django-updates/0107019c46d98711-615dcfd2-b9b9-40af-86c1-6e460917e5ea-00%40eu-central-1.amazonses.com.