#23317: StaticFileStorage Encoding error on '\xf4'
--------------------------------------+--------------------------------
Reporter: achedeuzot | Owner: nobody
Type: Bug | Status: new
Component: File uploads/storage | Version: 1.7-rc-2
Severity: Normal | Keywords: UnicodeEncodeError
Triage Stage: Unreviewed | Has patch: 0
Easy pickings: 0 | UI/UX: 0
--------------------------------------+--------------------------------
While using the `collectstatic` command with `django-pipeline` and a i18n
js library (containing lots of foreign characters), I stumbled on an
encoding error:
{{{
Traceback (most recent call last):
File "./dev_manage.py", line 10, in <module>
execute_from_command_line(sys.argv)
File
"/Users/achedeuzot/Documents/Projects/project/www.project.net/lib/python3.4
/site-packages/django/core/management/__init__.py", line 385, in
execute_from_command_line
utility.execute()
File
"/Users/achedeuzot/Documents/Projects/project/www.project.net/lib/python3.4
/site-packages/django/core/management/__init__.py", line 377, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File
"/Users/achedeuzot/Documents/Projects/project/www.project.net/lib/python3.4
/site-packages/django/core/management/base.py", line 288, in run_from_argv
self.execute(*args, **options.__dict__)
File
"/Users/achedeuzot/Documents/Projects/project/www.project.net/lib/python3.4
/site-packages/django/core/management/base.py", line 337, in execute
output = self.handle(*args, **options)
File
"/Users/achedeuzot/Documents/Projects/project/www.project.net/lib/python3.4
/site-packages/django/core/management/base.py", line 532, in handle
return self.handle_noargs(**options)
File
"/Users/achedeuzot/Documents/Projects/project/www.project.net/lib/python3.4
/site-
packages/django/contrib/staticfiles/management/commands/collectstatic.py",
line 168, in handle_noargs
collected = self.collect()
File
"/Users/achedeuzot/Documents/Projects/project/www.project.net/lib/python3.4
/site-
packages/django/contrib/staticfiles/management/commands/collectstatic.py",
line 114, in collect
for original_path, processed_path, processed in processor:
File
"/Users/achedeuzot/Documents/Projects/project/www.project.net/lib/python3.4
/site-packages/pipeline/storage.py", line 32, in post_process
packager.pack_stylesheets(package)
File
"/Users/achedeuzot/Documents/Projects/project/www.project.net/lib/python3.4
/site-packages/pipeline/packager.py", line 94, in pack_stylesheets
variant=package.variant, **kwargs)
File
"/Users/achedeuzot/Documents/Projects/project/www.project.net/lib/python3.4
/site-packages/pipeline/packager.py", line 105, in pack
self.save_file(output_filename, content)
File
"/Users/achedeuzot/Documents/Projects/project/www.project.net/lib/python3.4
/site-packages/pipeline/packager.py", line 117, in save_file
return self.storage.save(path, ContentFile("\xf4"))
File
"/Users/achedeuzot/Documents/Projects/project/www.project.net/lib/python3.4
/site-packages/django/core/files/storage.py", line 51, in save
name = self._save(name, content)
File
"/Users/achedeuzot/Documents/Projects/project/www.project.net/lib/python3.4
/site-packages/django/core/files/storage.py", line 228, in _save
_file.write(chunk)
UnicodeEncodeError: 'ascii' codec can't encode character '\xf4' in
position 0: ordinal not in range(128)
}}}
After some enquiries, the same error could be reproduced using the
following lines:
{{{
from django.contrib.staticfiles.storage import StaticFilesStorage
from django.core.files.base import ContentFile
storage = StaticFilesStorage()
storage.save("test-output.txt", ContentFile("ô"))
}}}
I was able to "solve" the issue by using `bytearray`:
{{{
storage.save("test-output.txt", ContentFile(bytearray("ô", 'utf-8')))
}}}
Also, another solution is/was to add `LANG=en_US.utf-8` into the
environment.
The line that trows the error is the following:
{{{
mode = 'wb' if isinstance(chunk, bytes) else 'wt'
_file = os.fdopen(fd, mode)
_file.write(chunk) # <= this one
}}}
We can see that if I send bytes, the write() happens without errors while
if I send a string, the write() fails.
How come Django fails on a "standard" utf-8 string ? Is this a Django
issue or from something else (OS write(), ...)?
--
Ticket URL: <https://code.djangoproject.com/ticket/23317>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
--
You received this message because you are subscribed to the Google Groups
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-updates/053.dea2bc0d14b9e01b086eddb1a3fdd4b6%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.
