[issue19518] Add new PyRun_xxx() functions to not encode the filename

Changes by STINNER Victor : -- resolution: -> out of date status: open -> closed ___ Python tracker ___

[issue19518] Add new PyRun_xxx() functions to not encode the filename

Adam Bartoš added the comment: I'm not sure this is the right issue. The support for Unicode filenames is not (at least on Windows) ideal. Let α.py be a Python script with invalid syntax. py α.py File encoding error, line 2 as as compile error ^ SyntaxError: invalid syntax On the

[issue19518] Add new PyRun_xxx() functions to not encode the filename

Changes by Nick Coghlan ncogh...@gmail.com: -- assignee: ncoghlan - ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19518 ___ ___ Python-bugs-list

[issue19518] Add new PyRun_xxx() functions to not encode the filename

Larry Hastings added the comment: So all the PyRun_*Object functions are new in 3.4, and none of them are documented yet? Option 4 is silly--I don't think we should ship them as public APIs in 3.4 if we're planning to rename them. I prefer the previous options. p.s. fwiw I hate ExName.

[issue19518] Add new PyRun_xxx() functions to not encode the filename

Serhiy Storchaka added the comment: So all the PyRun_*Object functions are new in 3.4, and none of them are documented yet? Not all. Only following functions are new in 3.4: Parser/parsetok.c:PyParser_ParseStringObject Parser/parsetok.c:PyParser_ParseFileObject

[issue19518] Add new PyRun_xxx() functions to not encode the filename

Larry Hastings added the comment: Are all the functions that use Object to indicate Unicode object instead of string new in 3.4? Of those, how many are undocumented? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19518

[issue19518] Add new PyRun_xxx() functions to not encode the filename

Serhiy Storchaka added the comment: Are all the functions that use Object to indicate Unicode object instead of string new in 3.4? Of those, how many are undocumented? Following 5 functions work with PyObject* filenames and have Object-less variants which works with char * filenames:

[issue19518] Add new PyRun_xxx() functions to not encode the filename

Larry Hastings added the comment: Are we proposing renaming any functions that are either a) not new in 3.4, or b) were documented as of 3.4 beta 1? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19518

[issue19518] Add new PyRun_xxx() functions to not encode the filename

Larry Hastings added the comment: Are those five functions new in 3.4 and undocumented? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19518 ___

[issue19518] Add new PyRun_xxx() functions to not encode the filename

Serhiy Storchaka added the comment: Are those five functions new in 3.4 and undocumented? PyErr_SetFromErrnoWithFilenameObject exists even in 2.7. Other 4 PyImport_*Object functions all added in 3.3 (see issue3080). All 5 functions are documented. 14 new functions were added in 3.4.

[issue19518] Add new PyRun_xxx() functions to not encode the filename

STINNER Victor added the comment: Sorry, but because of the bikeshedding, I'm not more interested to work on this issue. Don't hesitate to re-work my patch if you want to fix the bug (On Windows, these changes should allow to pass an unencodable filename on the command line). --

[issue19518] Add new PyRun_xxx() functions to not encode the filename

Changes by STINNER Victor victor.stin...@gmail.com: -- nosy: -haypo ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19518 ___ ___ Python-bugs-list

[issue19518] Add new PyRun_xxx() functions to not encode the filename

Nick Coghlan added the comment: Just getting this on Larry's radar and summarising the current position. The original problem: using char * to pass filenames around doesn't work properly on Windows, we need to use Unicode objects. The solution: parallel APIs that accept PyObject * rather than

[issue19518] Add new PyRun_xxx() functions to not encode the filename

Changes by Nick Coghlan ncogh...@gmail.com: -- priority: - normal ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19518 ___ ___ Python-bugs-list

[issue19518] Add new PyRun_xxx() functions to not encode the filename

STINNER Victor added the comment: So, which suffix should be used? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19518 ___ ___ Python-bugs-list

[issue19518] Add new PyRun_xxx() functions to not encode the filename

Serhiy Storchaka added the comment: *Unicode suffix in existing functions means Py_UNICODE* argument. May be *Ex2? It can't be misinterpreted but looks ugly. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19518

[issue19518] Add new PyRun_xxx() functions to not encode the filename

STINNER Victor added the comment: *Unicode suffix in existing functions means Py_UNICODE* argument. Yes, this is why I chose Object() suffix. Are you still opposed to Object suffix? (Yes, *Ex2 is really ugly.) -- ___ Python tracker

[issue19518] Add new PyRun_xxx() functions to not encode the filename

Nick Coghlan added the comment: How about ExName? This patch: PyRun_AnyFileExName PyRun_SimpleFileExName PyRun_InteractiveOneExName PyRun_InteractiveLoopExName PyRun_FileExName Previous patch: Py_CompileStringExName PyAST_FromNodeExName PyAST_CompileExName

[issue19518] Add new PyRun_xxx() functions to not encode the filename

STINNER Victor added the comment: PyRun_FileObject() looks misleading, because it works with FILE*, not with a file object. I simply replaced the current suffix with Object(). Only filename is converted from char* to PyObject*. Do you have a better suggestion for the new name? --

[issue19518] Add new PyRun_xxx() functions to not encode the filename

Serhiy Storchaka added the comment: No I have not a better suggestion. But I afraid that one day you will wanted to extend PyRun_File*() function to work with a general Python file object (perhaps there is such issue already) and then you will encountered a problem. --

[issue19518] Add new PyRun_xxx() functions to not encode the filename

Nick Coghlan added the comment: Perhaps we could we use the suffix Unicode rather than Object? These don't work with arbitrary objects, they expect a unicode string. PyRun_InteractiveOneObject would be updated to use the new suffix as well. That would both be clearer for the user, and address

[issue19518] Add new PyRun_xxx() functions to not encode the filename

STINNER Victor added the comment: FYI I already added a bunch of new functions with Object suffix when I replaced char* with PyObject*. Example: http://hg.python.org/cpython/rev/df2fdd42b375 http://bugs.python.org/issue11619 -- ___ Python tracker

[issue19518] Add new PyRun_xxx() functions to not encode the filename

Nick Coghlan added the comment: Hmm, reading more of those and I think Serhiy is definitely right - Object is the wrong suffix. Unicode isn't right either, since the main problem is that ambiguity around *which* parameter is a Python Unicode object. The API names that end in *StringObject or

[issue19518] Add new PyRun_xxx() functions to not encode the filename

Serhiy Storchaka added the comment: FYI I already added a bunch of new functions with Object suffix when I replaced char* with PyObject*. Most of them were added in 3.4. Unfortunately several functions were added earlier (e.g. PyImport_ExecCodeModuleObject,

[issue19518] Add new PyRun_xxx() functions to not encode the filename

STINNER Victor added the comment: Updated patch addressing some remarks of Serhiy and adding documentation. Anyone for a new review? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19518

[issue19518] Add new PyRun_xxx() functions to not encode the filename

Serhiy Storchaka added the comment: PyRun_FileObject() looks misleading, because it works with FILE*, not with a file object. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19518 ___

[issue19518] Add new PyRun_xxx() functions to not encode the filename

Nick Coghlan added the comment: PEP 432 doesn't really touch the PyRun_* APIs - it's all about refactoring Py_Initialize so you can use most of the C API during the latter parts of the configuration process (e.g. setting up the path for the import system). pythonrun.c is just a monstrous beast

[issue19518] Add new PyRun_xxx() functions to not encode the filename

New submission from STINNER Victor: The changeset af822a6c9faf of the issue #19512 added the function PyRun_InteractiveOneObject(). By the way, I forgot to document this function. This issue is also a reminder for that. The purpose of the new function is to avoid creation of temporary Unicode

[issue19518] Add new PyRun_xxx() functions to not encode the filename

Changes by Serhiy Storchaka storch...@gmail.com: -- components: +Interpreter Core stage: - test needed type: - enhancement ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19518 ___

[issue19518] Add new PyRun_xxx() functions to not encode the filename

Serhiy Storchaka added the comment: On Windows, these changes should allow to pass an unencodable filename on the command line (ex: japanese script name on an english setup). Doesn't the surrogateescape error handler solve this issue? -- ___

[issue19518] Add new PyRun_xxx() functions to not encode the filename

STINNER Victor added the comment: 2013/11/7 Serhiy Storchaka rep...@bugs.python.org: On Windows, these changes should allow to pass an unencodable filename on the command line (ex: japanese script name on an english setup). Doesn't the surrogateescape error handler solve this issue?

[issue19518] Add new PyRun_xxx() functions to not encode the filename

Changes by STINNER Victor victor.stin...@gmail.com: -- nosy: +georg.brandl ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19518 ___ ___

[issue19518] Add new PyRun_xxx() functions to not encode the filename

Serhiy Storchaka added the comment: I added some comments on Rietveld. Please do not commit without documentation and tests. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19518 ___

[issue19518] Add new PyRun_xxx() functions to not encode the filename

Changes by Arfrever Frehtes Taifersar Arahesis arfrever@gmail.com: -- nosy: +Arfrever ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19518 ___

[issue19518] Add new PyRun_xxx() functions to not encode the filename

STINNER Victor added the comment: Updated patch addressing some remarks of Serhiy and adding documentation. -- Added file: http://bugs.python.org/file32538/pyrun_object-2.patch ___ Python tracker rep...@bugs.python.org

[issue19518] Add new PyRun_xxx() functions to not encode the filename

STINNER Victor added the comment: Updated patch addressing some remarks of Serhiy and adding documentation. Oh, and it adds also an unit test. I didn't run the unit test on Windows yet. -- ___ Python tracker rep...@bugs.python.org

[issue19518] Add new PyRun_xxx() functions to not encode the filename

Eric Snow added the comment: PEP 432 relates pretty closely here. -- nosy: +eric.snow, ncoghlan ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19518 ___

[issue19518] Add new PyRun_xxx() functions to not encode the filename

STINNER Victor added the comment: PEP 432 relates pretty closely here. What is the relation between this issue and the PEP 432? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19518 ___

[issue19518] Add new PyRun_xxx() functions to not encode the filename

Eric Snow added the comment: PEP 432 is all about the PyRun_* API and especially relates to refactoring it with the goal of improving extensibility and maintainability. I'm sure Nick could expound, but the PEP is a response to the cruft that has accumulated over the years in