[issue21709] logging.__init__ assumes that __file__ is always set

[Gregory P. Smith]

Gregory P. Smith added the comment:

I applied the same _srcfile setting cleanup to 2.7 as we ran into this at work 
with an embedded python stdlib.  __file__ was set to something by the embedded 
importer, but to something different than the path seen on the actual code 
objects which appear to be based on the path seen during pyc compilation done 
at build time.

--
nosy: +gregory.p.smith

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21709] logging.__init__ assumes that __file__ is always set

[Roundup Robot]

Roundup Robot added the comment:

New changeset da971f467f6e by Gregory P. Smith in branch '2.7':
Issue #21709: Fix the logging module to not depend upon __file__ being set
https://hg.python.org/cpython/rev/da971f467f6e

New changeset bdb219a0219c by Gregory P. Smith in branch '2.7':
Issue21709: Call os.path.normcase when setting _srcfile for compatibility
https://hg.python.org/cpython/rev/bdb219a0219c

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21709] logging.__init__ assumes that __file__ is always set

[Vinay Sajip]

Vinay Sajip added the comment:

 Isn't __code__ implementation-specific?

Further data point: the contents of __code__ might be implementation-specific 
(as there are no other Python 3 implementations), but the 2.x equivalent, 
func_code, is to be found in CPython, Jython, IronPython and PyPy, and in each 
case it has a co_filename attribute pointing to a source file. Other internal 
details no doubt differ.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue21709
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21709] logging.__init__ assumes that __file__ is always set

[Eric Snow]

Eric Snow added the comment:

 addLevelName.__code__.co_filename

Isn't __code__ implementation-specific?

--
nosy: +eric.snow

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue21709
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21709] logging.__init__ assumes that __file__ is always set

[Vinay Sajip]

Vinay Sajip added the comment:

 Isn't __code__ implementation-specific?

It is, but ISTM it's worth getting a resolution for #21736 before another patch 
for this issue is developed.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue21709
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21709] logging.__init__ assumes that __file__ is always set

[Marc-Andre Lemburg]

Marc-Andre Lemburg added the comment:

While the current patch does not resolve the issue, I'm leaving the issue 
closed and have instead opened a new Issue21736 which tracks the idea to add a 
__file__ attribute to frozen modules per default.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue21709
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21709] logging.__init__ assumes that __file__ is always set

[Vinay Sajip]

Vinay Sajip added the comment:

 While the current patch does not resolve the issue, I'm leaving the issue 
 closed

That's fine - I will implement the changes we discussed in this issue, even if 
it's something of a stop-gap.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue21709
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21709] logging.__init__ assumes that __file__ is always set

[Roundup Robot]

Roundup Robot added the comment:

New changeset bec6f18dd636 by Vinay Sajip in branch '3.4':
Issue #21709: Improved implementation to cover the frozen module case.
http://hg.python.org/cpython/rev/bec6f18dd636

New changeset bd44ad77013a by Vinay Sajip in branch 'default':
Issue #21709: Merged update from 3.4.
http://hg.python.org/cpython/rev/bd44ad77013a

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue21709
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21709] logging.__init__ assumes that __file__ is always set

[Roundup Robot]

Roundup Robot added the comment:

New changeset 11a920a26f13 by Vinay Sajip in branch '3.4':
Issue #21709: Remove references to __file__ when part of a frozen application.
http://hg.python.org/cpython/rev/11a920a26f13

New changeset 149cc6364180 by Vinay Sajip in branch 'default':
Closes #21709: Merged fix from 3.4.
http://hg.python.org/cpython/rev/149cc6364180

--
nosy: +python-dev
resolution:  - fixed
stage:  - resolved
status: open - closed

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue21709
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21709] logging.__init__ assumes that __file__ is always set

[Marc-Andre Lemburg]

Marc-Andre Lemburg added the comment:

Hi Vinaj,

thanks for the patch, but it doesn't really help outside of py2exe. The 
sys.frozen flag is not an official Python API and it's unlikely to become one, 
since you can freeze the whole application or just parts of it, which 
sys.frozen would not be able to address.

Instead, the modules in the stdlib should be aware of the fact that __file__ is 
not always available and provide fallback solutions.

Could you please use a fix that works for Python tools in general ?

E.g. instead of doing an equal test inf .findCaller() it may be better to use a 
regular expression or you could patch the __init__ module's co_filename into 
the module as _srcfile (after it's fully initialized).

Thanks.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue21709
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21709] logging.__init__ assumes that __file__ is always set

[Vinay Sajip]

Vinay Sajip added the comment:

 Could you please use a fix that works for Python tools in general?

I suggested an alternative implementation altogether in Issue #16778, but it 
was suggested that we wait for frame annotations. I'm not sure what the 
schedule for that is.

 The sys.frozen flag is not an official Python API and it's unlikely to become 
 one

Would using imp.is_frozen('logging') rather than hasattr(sys, 'frozen') meet 
your requirement here? I'm not saying it's the ideal solution, but perhaps it 
will do until frame annotations arrive and we can avoid using filenames 
altogether?

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue21709
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21709] logging.__init__ assumes that __file__ is always set

[Marc-Andre Lemburg]

Marc-Andre Lemburg added the comment:

On 11.06.2014 11:25, Vinay Sajip wrote:
 
 Vinay Sajip added the comment:
 
 Could you please use a fix that works for Python tools in general?
 
 I suggested an alternative implementation altogether in Issue #16778, but it 
 was suggested that we wait for frame annotations. I'm not sure what the 
 schedule for that is.
 
 The sys.frozen flag is not an official Python API and it's unlikely to 
 become one
 
 Would using imp.is_frozen('logging') rather than hasattr(sys, 'frozen') meet 
 your requirement here? I'm not saying it's the ideal solution, but perhaps it 
 will do until frame annotations arrive and we can avoid using filenames 
 altogether?

I don't think any of this is needed here. _srcfile is only used to
identify the caller's stack frame and needs to be set to the co_filename
of the stack frame used by the logging.__init__ module.

Here's a sketch of what I had hinted at in my last reply:

def _get_module_filename():
return getLogger.func_code.co_filename

You simply use the .co_filename attribute of one of the module's functions
to get useable value for __file__.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue21709
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21709] logging.__init__ assumes that __file__ is always set

[Vinay Sajip]

Vinay Sajip added the comment:

 _srcfile is only used to identify the caller's stack frame

Not quite. It's also used to indicate whether findCaller() should be called at 
all: setting it to None avoids calling findCaller(), which might be desirable 
in some performance-sensitive scenarios.

So if you mean just call _get_module_filename() instead of accessing 
_srcFile, that won't do. If you mean set _srcFile to the return value of 
_get_module_filename(), that might work, if I e.g. move the _srcFile 
definition to after addLevelName (say) and do just

_srcFile = addLevelName.__code__.co_filename

How does that sound?

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue21709
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21709] logging.__init__ assumes that __file__ is always set

[Marc-Andre Lemburg]

Marc-Andre Lemburg added the comment:

On 11.06.2014 12:32, Vinay Sajip wrote:
 
 Vinay Sajip added the comment:
 
 _srcfile is only used to identify the caller's stack frame
 
 Not quite. It's also used to indicate whether findCaller() should be called 
 at all: setting it to None avoids calling findCaller(), which might be 
 desirable in some performance-sensitive scenarios.
 
 So if you mean just call _get_module_filename() instead of accessing 
 _srcFile, that won't do. If you mean set _srcFile to the return value of 
 _get_module_filename(), that might work, if I e.g. move the _srcFile 
 definition to after addLevelName (say) and do just
 
 _srcFile = addLevelName.__code__.co_filename
 
 How does that sound?

That's what I meant, yes. Please also add some comment explaining
why this is done in this way.

FWIW: Given that __file__ is not always set, it may be worthwhile
introducing some generic helper to the stdlib which uses the
.co_filename attribute to get the compile time filename as fallback
in case __file__ is not set.

Thanks,
-- 
Marc-Andre Lemburg
eGenix.com

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue21709
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21709] logging.__init__ assumes that __file__ is always set

[Vinay Sajip]

Vinay Sajip added the comment:

 Please also add some comment explaining why this is done in this way.

Natürlich :-)

 it may be worthwhile introducing some generic helper to the stdlib

Wouldn't you have to pass in a function (or code object) from a specific 
module, though? It seems more logical to have __file__ set, even for frozen 
modules (after all, if it's there in a code object's co_filename, is there some 
reason it shouldn't be exposed as a module attribute? (Even though it isn't at 
the moment.)

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue21709
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21709] logging.__init__ assumes that __file__ is always set

[Marc-Andre Lemburg]

Marc-Andre Lemburg added the comment:

On 11.06.2014 13:22, Vinay Sajip wrote:
 
 Vinay Sajip added the comment:
 
 Please also add some comment explaining why this is done in this way.
 
 Natürlich :-)

Prima :-)

 it may be worthwhile introducing some generic helper to the stdlib
 
 Wouldn't you have to pass in a function (or code object) from a specific 
 module, though? It seems more logical to have __file__ set, even for frozen 
 modules (after all, if it's there in a code object's co_filename, is there 
 some reason it shouldn't be exposed as a module attribute? (Even though it 
 isn't at the moment.)

Well, I guess passing in a reference to the module would suffice. The
function could then look around for functions, methods, etc. to find
a usable code object.

I agree that having a __file__ attribute in frozen modules would
be helpful, since it's obviously not widely known that this
attribute does not always exist (just grep the stdlib as example,
in particular the test suite).

The module object's PyModule_GetFilenameObject() even reports a missing
__file__ attribute as a SystemError.

Perhaps something to discuss on python-dev.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue21709
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21709] logging.__init__ assumes that __file__ is always set

[Marc-Andre Lemburg]

New submission from Marc-Andre Lemburg:

It is not when freezing the logging package, so any use of the logging package 
fails:

 import logging
Traceback (most recent call last):
  File console, line 1, in module
  File pyrun/importlib/_bootstrap.py, line 2237, in _find_and_load
  File pyrun/importlib/_bootstrap.py, line 2226, in _find_and_load_unlocked
  File pyrun/importlib/_bootstrap.py, line 1200, in _load_unlocked
  File pyrun/importlib/_bootstrap.py, line 1129, in _exec
  File pyrun/importlib/_bootstrap.py, line 1359, in exec_module
  File pyrun/logging/__init__.py, line 61, in module
NameError: name '__file__' is not defined

This is the code in question:

#
# _srcfile is used when walking the stack to check when we've got the first
# caller stack frame.
#
if hasattr(sys, 'frozen'): #support for py2exe
_srcfile = logging%s__init__%s % (os.sep, __file__[-4:])
else:
_srcfile = __file__
_srcfile = os.path.normcase(_srcfile)

Note the special case for py2exe. But this doesn't really help, since frozen 
modules don't have the __file__ attribute set - at least not when generated 
with Tools/freeze.

PS: This is with Python 3.4.1.

--
components: Interpreter Core, Library (Lib)
messages: 220196
nosy: lemburg
priority: normal
severity: normal
status: open
title: logging.__init__ assumes that __file__ is always set
versions: Python 3.4, Python 3.5

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue21709
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21709] logging.__init__ assumes that __file__ is always set

[Marc-Andre Lemburg]

Marc-Andre Lemburg added the comment:

The issue is similar to Issue20884.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue21709
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21709] logging.__init__ assumes that __file__ is always set

[Berker Peksag]

Changes by Berker Peksag berker.pek...@gmail.com:


--
nosy: +vinay.sajip

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue21709
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com