http://bugs.gpodder.org/show_bug.cgi?id=389
Summary: util.sanitize_* functions fail for non-ascii input
Product: gPodder
Version: unspecified
Platform: PC
OS/Version: Linux
Status: UNCONFIRMED
Severity: feature request
Priority: low
Component: Application
AssignedTo: [email protected]
ReportedBy: [email protected]
Created an attachment (id=266)
--> (http://bugs.gpodder.org/attachment.cgi?id=266)
Fix sanitize_filename function
In util.py there are two string sanitization functions:
sanitize_filename
sanitize_encoding
Both of these will throw an exception if the input string has a non-ascii
character in it:
python
>>> import util.py
>>> util.sanitize_filename("\x80")
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/andy/ubuntu/gpodder/src/gpodder/util.py", line 1029, in
sanitize_filename
return re.sub('[/|?*<>:+\[\]\"\\\]', '_', filename.strip().encode(e,
'ignore'))
UnicodeDecodeError: 'utf8' codec can't decode byte 0xbb in position 0:
unexpected code byte
I'm not quite sure what the best way of fixing this is, since I don't know what
the correct python-fu is to find the encoding used at a particular path. In any
case, we can't assume that the input string is even legally encoded, and I'm
not even sure how you are supposed to guess what the encoding of the input
string is at this point. My "solution" is to just strip out anything that isn't
ascii, but that is probably because I'm a mono-lingual English speaker.
Given that we don't have any idea what encoding the input is in, there is no
way we can do what the docstring for the function is claiming ("encode in the
native language"), I think the best solution is to just assume that the string
is already encoded with the correct encoding.
The problem with this solution is that the output might still be illegal (e.g.
if the encoding used in the filesystem is UTF-8 and the string is "\xbb"). But
fixing this is difficult (need to know the encoding of the input file and the
encoding of the filesystem at the particular path we are writing to). On the
other hand, the use_ascii flag will work.
What this patch does:
1) Remove the sanitize_encoding function (unused and buggy)
2) Change sanitize_filename function to not change the encoding, unless
use_ascii is true (in which case, change it to ascii encoding)
3) This patch also does the .strip() before truncating the filename (so if you
had " longfilename", we wouldn't chop off more than we needed to at the
end).
--
Configure bugmail: http://bugs.gpodder.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching all bug changes.
_______________________________________________
gPodder-Bugs mailing list
[email protected]
https://lists.berlios.de/mailman/listinfo/gpodder-bugs
