Alexander Belopolsky belopol...@users.sourceforge.net added the comment:
On Sun, Nov 7, 2010 at 4:24 AM, STINNER Victor rep...@bugs.python.org wrote:
..
Ok, the new patch (tokenize_open-2.patch) uses tokenize.open() name and adds a
test for BOM without coding cookie (test utf-8-sig encoding).
STINNER Victor victor.stin...@haypocalc.com added the comment:
what about open(.., encoding=fromcookie)?
Please don't do that. It remembers me the magical str.encode() method of
Python2. I prefer simple API with limited features: if you want to open a
Python script, use tokenize.open(), if
STINNER Victor victor.stin...@haypocalc.com added the comment:
On Saturday 06 November 2010 17:00:15 you wrote:
Note that it is useful for opening any text file with an encoding cookie,
not only python source code, so tokenize.open() sounds attractive.
Ok, the new patch
New submission from STINNER Victor victor.stin...@haypocalc.com:
In Python3, the following pattern becomes common:
with open(fullname, 'rb') as fp:
coding, line = tokenize.detect_encoding(fp.readline)
with open(fullname, 'r', encoding=coding) as fp:
...
Éric Araujo mer...@netwok.org added the comment:
Looks good.
In the test, do you have to remove the TESTFN file manually? Isn’t that
handled by unittest or regrtest?
I think the name of the function could be better but I don’t have a proposal.
--
nosy: +eric.araujo
STINNER Victor victor.stin...@haypocalc.com added the comment:
In the test, do you have to remove the TESTFN file manually? Isn’t that
handled by unittest or regrtest?
The test uses TESTFN+'.py', not TESTFN ;-)
I think the name of the function could be better but I don’t have a
proposal.
Éric Araujo mer...@netwok.org added the comment:
The test uses TESTFN+'.py', not TESTFN ;-)
Ah, right. You can use addCleanup then.
Do you prefer a name like tokenize.open()?
Hm, tokenize being “Tokenization help for Python programs”, I think this name
is rather good. Or
Alexander Belopolsky belopol...@users.sourceforge.net added the comment:
+1 on adding the function.
Note that it is useful for opening any text file with an encoding cookie, not
only python source code, so tokenize.open() sounds attractive.
Once we are at it, I would like to factor out and
Éric Araujo mer...@netwok.org added the comment:
Note that it is useful for opening any text file with an encoding
cookie, not only python source code,
This is cool. doctest could benefit from that, I think.
so tokenize.open() sounds attractive.
Agreed, even though the docstring of tokenize
Alexander Belopolsky belopol...@users.sourceforge.net added the comment:
On Sat, Nov 6, 2010 at 12:15 PM, Éric Araujo rep...@bugs.python.org wrote:
..
so tokenize.open() sounds attractive.
Agreed, even though the docstring of tokenize does talk about Python code.
I still like tokenize.open()
Éric Araujo mer...@netwok.org added the comment:
I don’t know if it would be okay to depend on tokenize in io. Anyway, I tend
to prefer tokenize.open over builtins.open(..., encoding='fromcookie').
--
nosy: +pitrou
___
Python tracker
Georg Brandl ge...@python.org added the comment:
+1 for the feature. However, tokenize.open() sounds a bit unspecific. But I
don't really have better suggestions; open_cookie() is wrong too, since it
doesn't open cookies :)
--
nosy: +georg.brandl
12 matches
Mail list logo
