#29839: Make leading character in quote/unquote in contrib.admin.utils
configurable
-------------------------------------+-------------------------------------
Reporter: Chris Z. | Owner: nobody
Type: New feature | Status: new
Component: contrib.admin | Version: 2.1
Severity: Normal | Resolution:
Keywords: admin quote unquote | Triage Stage:
| Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Description changed by Chris Z.:
Old description:
> = **The problem:**
>
> The current leading character in the {{{contrib.admin.utils}}}
> {{{quote()}}} and {{{unquote()}}} functions (an underscore) causes some
> characters to be improperly escaped/unescaped if primary keys contain an
> underscore. This character could be configurable through a setting in
> {{{settings.py}}}, so that users who have underscores in text-based
> primary keys can change it. If there is a primary key with an underscore
> in it, for example, changing the leading character in the quote/unquote
> functions to a dot ({{{.}}}) will fix edit links in the django admin.
>
> = **The current (django 2.1.1) functions, with suggested changes:**
>
> {{{
> def quote(s):
> """
> Ensure that primary key values do not confuse the admin URLs by
> escaping
> any '/', '_' and ':' and similarly problematic characters.
> Similar to urllib.parse.quote(), except that the quoting is slightly
> different so that it doesn't get automatically unquoted by the Web
> browser.
> """
> if not isinstance(s, str):
> return s
> res = list(s)
> for i in range(len(res)):
> c = res[i]
> if c in """:/_#?;@&=+$,"[]<>%\n\\""":
> res[i] = '_%02X' % ord(c) # Make this underscore a
> configurable character
> return ''.join(res)
>
> def unquote(s):
> """Undo the effects of quote(). Based heavily on
> urllib.parse.unquote()."""
> mychr = chr
> myatoi = int
> list = s.split('_') # Make this underscore a configurable character
> res = [list[0]]
> myappend = res.append
> del list[0]
> for item in list:
> if item[1:2]:
> try:
> myappend(mychr(myatoi(item[:2], 16)) + item[2:])
> except ValueError:
> myappend('_' + item) # Make this underscore a
> configurable character
> else:
> myappend('_' + item) # Make this underscore a configurable
> character
> return "".join(res)
> }}}
>
> = **A couple example models:**
>
> {{{
> from django.db import models
> from django.contrib.auth.models import User
>
> # CUSTOMERS:
> class Customer(models.Model):
> user = models.OneToOneField(User, on_delete=models.CASCADE)
> customer_id = models.CharField(max_length=100, primary_key=True)
>
> def __str__(self):
> return self.customer_id
>
> # SUBSCRIPTIONS:
> class Subscription(models.Model):
>
> subscription_id = models.CharField(max_length=100, primary_key=True)
> customer_id = models.ForeignKey('Customer', db_column='customer_id',
> on_delete=models.CASCADE)
>
> def __str__(self):
> return self.subscription_id
> }}}
>
> = **Steps to reproduce the issue:**
>
> 1. Create the above models.
> 2. Create a {{{Customer}}} object with a primary key of
> {{{cus_C2testing}}}
> 3. Create a Subscription object with a primary key of {{{sub_C2testing}}}
> and a ForeignKey back to the object created in step 1
> 4. Navigate to the django admin and click the Subscription object you
> created. **Notice that the underscore in the primary key has been
> replaced with {{{_5F}}} in the URL**
> 5. Click the edit customer icon (the small pencil next to the dropdown
> containing your customer object) and notice that the popup displays
> {{{Customer with ID "cusÂtesting" doesn't exist. Perhaps it was
> deleted?}}}. The C2 has been replaced with its unicode codepoint, Â. I
> believe this is due to the fact that the unquote function splits the ID
> at the underscore, but regardless of the cause, **replacing the
> underscore in the quote/unquote functions at the locations in the code
> above with a dot ({{{.}}}) resolves the issue.**
>
> = **The suggestion:**
> My suggestion is that we make this configurable, so that the
> quote/unquote functions use an underscore by default, but take a second
> argument of a leading character if desired. It would ideally be
> configurable via settings.py in a variable like
> {{{ADMIN_QUOTE_UNQUOTE_LEADCHAR}}}. This will give users the flexibility
> to use primary keys with underscores, which are likely to be much more
> common than keys with dots.
New description:
= **Important Note:**
It's worth noting before reading all of this that apparently backporting
these two commits will actually eliminate the need for this feature:
https://github.com/django/django/commit/e9defb3f6e60b626e9ec40ff5df1322fceb52601
https://github.com/django/django/commit/e4df8e6dc021fa472fa77f9b835db74810184748
= **The problem:**
The current leading character in the {{{contrib.admin.utils}}}
{{{quote()}}} and {{{unquote()}}} functions (an underscore) causes some
characters to be improperly escaped/unescaped if primary keys contain an
underscore. This character could be configurable through a setting in
{{{settings.py}}}, so that users who have underscores in text-based
primary keys can change it. If there is a primary key with an underscore
in it, for example, changing the leading character in the quote/unquote
functions to a dot ({{{.}}}) will fix edit links in the django admin.
= **The current (django 2.1.1) functions, with suggested changes:**
{{{
def quote(s):
"""
Ensure that primary key values do not confuse the admin URLs by
escaping
any '/', '_' and ':' and similarly problematic characters.
Similar to urllib.parse.quote(), except that the quoting is slightly
different so that it doesn't get automatically unquoted by the Web
browser.
"""
if not isinstance(s, str):
return s
res = list(s)
for i in range(len(res)):
c = res[i]
if c in """:/_#?;@&=+$,"[]<>%\n\\""":
res[i] = '_%02X' % ord(c) # Make this underscore a
configurable character
return ''.join(res)
def unquote(s):
"""Undo the effects of quote(). Based heavily on
urllib.parse.unquote()."""
mychr = chr
myatoi = int
list = s.split('_') # Make this underscore a configurable character
res = [list[0]]
myappend = res.append
del list[0]
for item in list:
if item[1:2]:
try:
myappend(mychr(myatoi(item[:2], 16)) + item[2:])
except ValueError:
myappend('_' + item) # Make this underscore a configurable
character
else:
myappend('_' + item) # Make this underscore a configurable
character
return "".join(res)
}}}
= **A couple example models:**
{{{
from django.db import models
from django.contrib.auth.models import User
# CUSTOMERS:
class Customer(models.Model):
user = models.OneToOneField(User, on_delete=models.CASCADE)
customer_id = models.CharField(max_length=100, primary_key=True)
def __str__(self):
return self.customer_id
# SUBSCRIPTIONS:
class Subscription(models.Model):
subscription_id = models.CharField(max_length=100, primary_key=True)
customer_id = models.ForeignKey('Customer', db_column='customer_id',
on_delete=models.CASCADE)
def __str__(self):
return self.subscription_id
}}}
= **Steps to reproduce the issue:**
1. Create the above models.
2. Create a {{{Customer}}} object with a primary key of
{{{cus_C2testing}}}
3. Create a Subscription object with a primary key of {{{sub_C2testing}}}
and a ForeignKey back to the object created in step 1
4. Navigate to the django admin and click the Subscription object you
created. **Notice that the underscore in the primary key has been replaced
with {{{_5F}}} in the URL**
5. Click the edit customer icon (the small pencil next to the dropdown
containing your customer object) and notice that the popup displays
{{{Customer with ID "cusÂtesting" doesn't exist. Perhaps it was
deleted?}}}. The C2 has been replaced with its unicode codepoint, Â. I
believe this is due to the fact that the unquote function splits the ID at
the underscore, but regardless of the cause, **replacing the underscore in
the quote/unquote functions at the locations in the code above with a dot
({{{.}}}) resolves the issue.**
= **The suggestion:**
My suggestion is that we make this configurable, so that the quote/unquote
functions use an underscore by default, but take a second argument of a
leading character if desired. It would ideally be configurable via
settings.py in a variable like {{{ADMIN_QUOTE_UNQUOTE_LEADCHAR}}}. This
will give users the flexibility to use primary keys with underscores,
which are likely to be much more common than keys with dots.
--
--
Ticket URL: <https://code.djangoproject.com/ticket/29839#comment:1>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
--
You received this message because you are subscribed to the Google Groups
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-updates/065.9598062dab03c8f93e20ad9b413ad6b8%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.
