Re: [Python-Dev] backporting stdlib 2.7.x from pypy to cpython

2012-06-12 Thread Stephen J. Turnbull 
Brett Cannon writes:

 > Not necessarily. Just like discussions on SIGs can start and end
 > there, I see no requirement that discussions on the list end up on
 > python-dev.

You've missed my point, which is that for many people working on
CPython, python-dev will be the natural place to discuss those issues,
and the thread will end up on two different lists.

 > Discussions on the list would universally affect all VMs, so there is an
 > incentive to pay attention.

You've missed the point again, which is that the incentive only
motivates those who care about VMs.  It does not motivate those who
care about features that affect VMs.
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] backporting stdlib 2.7.x from pypy to cpython

2012-06-12 Thread Maciej Fijalkowski 
On Mon, Jun 11, 2012 at 8:58 AM, Nick Coghlan  wrote:

> On Mon, Jun 11, 2012 at 11:29 AM, Guido van Rossum 
> wrote:
> > But what guarantee do you have that (a) the right people sign up for
> > the new list, and (b) topics are correctly brought up there instead of
> > on python-dev? I agree that python-dev is turning into a firehose, but
> > I am reluctant to create backwaters where people might arrive at what
> > they think is a consensus only because the important opinions aren't
> > represented there.
>
> If that's a concern, I'd be happy to limit the use of the new list to
> "Input from other implementations needed on python-dev thread ".
>
> At the moment, it's a PITA to chase other implementations to get
> confirmation that they can cope with a change we're considering, so
> I'd like confirmation that either:
>

Hm.

Maybe we can do something to change the python-dev ethics in order to
alleviate the problem?

I'm skimming python-dev and read some of the discussion. The problem is
when a VM-related question shows up in the message 77 of a thread that does
not have anything in topic that would catch my attention. If all the
interesting questions that we have to answer are in their own topic, I
think it's fine to be subscribed to python-dev, provided I don't have to
read all the bikeshedding.

But maybe I'm too optimistic and this is not changeable.

Cheers,
fijal
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] issue #15038 - Optimize python Locks on Windows

2012-06-12 Thread Kristján Valur Jónsson 
Hi,
Could I get some feedback on this proposed patch?  It would be great to get it 
in before the beta.

Cheers,
Kristján
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] segfault - potential double free when using iterparse

2012-06-12 Thread Alon Horev 
Hi All,

First of all, I'm not opening a bug yet as I'm not certain whether this is
a CPython bug or lxml bug.

I'm getting a segfault within python's GC (garbage collector) module.
here's the stack trace:

#0  0x7fc7e9f6b76e in gc_list_remove (op=0x7fc79cef3d98) at
Modules/gcmodule.c:211
#1  PyObject_GC_Del (op=0x7fc79cef3d98) at Modules/gcmodule.c:1503
#2  0x7fc7e9f2ac0f in PyEval_EvalFrameEx (f=,
throwflag=) at Python/ceval.c:2894
#3  0x7fc7e9ea5b79 in gen_send_ex (arg=None, exc=,
gen=) at Objects/genobject.c:84
#4  0x7fc7e9ea6185 in gen_close (self=) at Objects/genobject.c:130
#5  gen_del (self=) at
Objects/genobject.c:165
#6  0x7fc7e9ea5a1b in gen_dealloc (gen=0x7fc7c1ba73c0) at
Objects/genobject.c:32

In order to see what object the gc is freeing i tried casting it to a
PyObject (we're freeing a lxml object):
(gdb) p (PyObject*) op
$17 = 

Similar bugs (http://osdir.com/ml/python.bugs/2000-12/msg00214.html) blame
the extension module for calling dealloc explicitly more than once or doing
forbidden things in __del__.

this is how i use lxml:

from lxml.etree import iterparse

def safe_iterparse(*args, **kwargs):
for event, element in iterparse(*args, **kwargs):
try:
yield (event, element)
finally:
element.clear()

I don't have the data that caused the crash, hopefully I'll get it after
the next crash.
anyone familiar with these kind of bugs in c extensions/cpython/lxml? could
you give pointers to what I should be looking for?

some version info:
CPython version: 2.7.2 on linux.
lxml: 2.3.3


  thanks, Alon Horev
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] backporting stdlib 2.7.x from pypy to cpython

2012-06-12 Thread [email protected] 
On Mon, Jun 11, 2012 at 10:46 PM, Nick Coghlan  wrote:
> To allow easier transition to a separate list (if that seems necessary
> at a later date), my preferred colour for the bikeshed is
> [compatibility-sig].
I for one approve of this bikeshed colour :)

-Frank
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] backporting stdlib 2.7.x from pypy to cpython

2012-06-12 Thread Brett Cannon 
On Tue, Jun 12, 2012 at 4:09 AM, Stephen J. Turnbull wrote:

> Brett Cannon writes:
>
>  > Not necessarily. Just like discussions on SIGs can start and end
>  > there, I see no requirement that discussions on the list end up on
>  > python-dev.
>
> You've missed my point, which is that for many people working on
> CPython, python-dev will be the natural place to discuss those issues,
> and the thread will end up on two different lists.
>

Ah, but you helped make my point! For the people working on CPython,
python-dev is a natural place. But what about PyPy, IronPython, or Jython
(toss in Cython or any future VMs and it just became an even larger spread
of teams)? Do they naturally think to discuss things on python-dev or their
own list? If anything this new list would act as a showing of good will
that python-dev does not view CPython as the centre of the world when it
comes to VMs (but obviously does for the language) and that the other VM
authors' time as just as important by not forcing them to wade through
python-dev (or have to set up a special filter just for python-dev to get
the occasional email thread).


>
>  > Discussions on the list would universally affect all VMs, so there is an
>  > incentive to pay attention.
>
> You've missed the point again, which is that the incentive only
> motivates those who care about VMs.  It does not motivate those who
> care about features that affect VMs.
>

But if you are doing something which will affect the VMs I hope you do care
about VMs period. We can also easily tell someone that they need to discuss
something on the other list if needed (we already do this when something
should be discussed in a SIG first as well).

Anyway, it looks like everyone else chiming in is capitulating to keeping
it on python-dev with a proper subject line, so we will start with there
and if it proves ineffective I will create the list.
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] backporting stdlib 2.7.x from pypy to cpython

2012-06-12 Thread Brett Cannon 
On Tue, Jun 12, 2012 at 9:49 AM, [email protected] <
[email protected]> wrote:

> On Mon, Jun 11, 2012 at 10:46 PM, Nick Coghlan  wrote:
> > To allow easier transition to a separate list (if that seems necessary
> > at a later date), my preferred colour for the bikeshed is
> > [compatibility-sig].
> I for one approve of this bikeshed colour :)
>

Fine by me. We will start with that then.
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] [compatibility-sig] making sure importlib.machinery.SourceLoader doesn't throw an exception if bytecode is not supported by a VM

2012-06-12 Thread Brett Cannon 
I would like to have importlib just work out of the box for all VMs in 3.3
instead of requiring a minor patch in order to not throw an exception when
loading from source and there is no bytecode. The relevant code for this
discussion can be seen at
http://hg.python.org/cpython/file/c2910971eb86/Lib/importlib/_bootstrap.py#l691
 .

First question is what are all the VMs doing for imp.cache_from_source()?
Are you implementing it like CPython, or are you returning None? And if you
implemented it, what does marshal.loads() do? Right now cache_from_source()
is implemented in importlib itself, but we can either provide a flag to
control what it does or in your setup code for import you can override the
function with ``lambda _, __=None: None``.

Second question, what do you set sys.dont_write_bytecode to?

The answers to those questions will dictate if there is anything to
actually discuss. =)
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Built-in sub modules

2012-06-12 Thread Brett Cannon 
On Mon, Jun 11, 2012 at 11:37 PM, Nick Coghlan  wrote:

> On Tue, Jun 12, 2012 at 1:17 PM, Albert Zeyer 
> wrote:
> > I also searched a bit around and I didn't directly found any easier
> > way to do this. Only a post from 2009
> > (http://mail.python.org/pipermail/cplusplus-sig/2009-January/014178.html
> )
> > which seems like a much more ugly hack.
>
> Right, it isn't currently supported.
> http://bugs.python.org/issue1644818 is a long standard feature request
> to add this functionality.
>
> For Python 3.3, the old import system (written in C) has been replaced
> with importlib (written in Python). This should make it easier to
> extend and experiment with builtin submodule support. An importlib
> based solution should also work in Python 3.2.
>
> Further discussions would be best directed to import-sig, until an
> importlib based solution is available for consideration.
>

I actually had code to make extensions modules work in packages as well,
but removed it when it broke "compatibility".
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] backporting stdlib 2.7.x from pypy to cpython

2012-06-12 Thread Mark Lawrence 

On 12/06/2012 15:40, Brett Cannon wrote:


Ah, but you helped make my point! For the people working on CPython,
python-dev is a natural place. But what about PyPy, IronPython, or Jython
(toss in Cython or any future VMs and it just became an even larger spread
of teams)? Do they naturally think to discuss things on python-dev or their
own list? If anything this new list would act as a showing of good will
that python-dev does not view CPython as the centre of the world when it
comes to VMs (but obviously does for the language) and that the other VM
authors' time as just as important by not forcing them to wade through
python-dev (or have to set up a special filter just for python-dev to get
the occasional email thread).


A bit late in the day and possibly the most stupid suggestion ever, but 
why not name python-dev cpython-dev?  At least everybody would know 
where they stand.


--
Cheers.

Mark Lawrence.

___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [compatibility-sig] making sure importlib.machinery.SourceLoader doesn't throw an exception if bytecode is not supported by a VM

2012-06-12 Thread Alex Gaynor 
For PyPy: I'm not an expert in our import, but from looking at the source

1) imp.cache_from_source is unimplemented, it's an AttributeError.

2) sys.dont_write_bytecode is always false, we don't respect that flag (we 
really
   should IMO, but it's not a high priority for me, or anyone else apparently)

Alex

___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] backporting stdlib 2.7.x from pypy to cpython

2012-06-12 Thread Brett Cannon 
On Tue, Jun 12, 2012 at 10:59 AM, Mark Lawrence wrote:

> On 12/06/2012 15:40, Brett Cannon wrote:
>
>>
>> Ah, but you helped make my point! For the people working on CPython,
>> python-dev is a natural place. But what about PyPy, IronPython, or Jython
>> (toss in Cython or any future VMs and it just became an even larger spread
>> of teams)? Do they naturally think to discuss things on python-dev or
>> their
>> own list? If anything this new list would act as a showing of good will
>> that python-dev does not view CPython as the centre of the world when it
>> comes to VMs (but obviously does for the language) and that the other VM
>> authors' time as just as important by not forcing them to wade through
>> python-dev (or have to set up a special filter just for python-dev to get
>> the occasional email thread).
>>
>
> A bit late in the day and possibly the most stupid suggestion ever, but
> why not name python-dev cpython-dev?  At least everybody would know where
> they stand.
>
>
Way too much stuff out there says python-dev over cpython-dev. Plus
python-dev covers the language of Python as well, which is not specific to
CPython.


> --
> Cheers.
>
> Mark Lawrence.
>
>
> __**_
> Python-Dev mailing list
> [email protected]
> http://mail.python.org/**mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/**mailman/options/python-dev/**
> brett%40python.org
>
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [compatibility-sig] making sure importlib.machinery.SourceLoader doesn't throw an exception if bytecode is not supported by a VM

2012-06-12 Thread Brett Cannon 
On Tue, Jun 12, 2012 at 12:38 PM, Alex Gaynor  wrote:

> For PyPy: I'm not an expert in our import, but from looking at the source
>
> 1) imp.cache_from_source is unimplemented, it's an AttributeError.
>

Well, you will have it come Python 3.3 one way or another. =)


>
> 2) sys.dont_write_bytecode is always false, we don't respect that flag (we
> really
>   should IMO, but it's not a high priority for me, or anyone else
> apparently)
>

But doesn't PyPy read and write .pyc files (
http://doc.pypy.org/en/latest/config/objspace.usepycfiles.html suggests you
do)? So I would assume you are not affected by this. Jython and IronPython,
though, would be (I think).
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [compatibility-sig] making sure importlib.machinery.SourceLoader doesn't throw an exception if bytecode is not supported by a VM

2012-06-12 Thread Brett Cannon 
On Tue, Jun 12, 2012 at 10:52 AM, Brett Cannon  wrote:

> I would like to have importlib just work out of the box for all VMs in 3.3
> instead of requiring a minor patch in order to not throw an exception when
> loading from source and there is no bytecode. The relevant code for this
> discussion can be seen at
> http://hg.python.org/cpython/file/c2910971eb86/Lib/importlib/_bootstrap.py#l691
>  .
>
> First question is what are all the VMs doing for imp.cache_from_source()?
> Are you implementing it like CPython, or are you returning None? And if you
> implemented it, what does marshal.loads() do? Right now cache_from_source()
> is implemented in importlib itself, but we can either provide a flag to
> control what it does or in your setup code for import you can override the
> function with ``lambda _, __=None: None``.
>

I should mention another option is to add sys.dont_read_bytecode (I think I
have discussed this with Frank at some point).

-Brett


>
> Second question, what do you set sys.dont_write_bytecode to?
>
> The answers to those questions will dictate if there is anything to
> actually discuss. =)
>
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [compatibility-sig] making sure importlib.machinery.SourceLoader doesn't throw an exception if bytecode is not supported by a VM

2012-06-12 Thread Alex Gaynor 
On Tue, Jun 12, 2012 at 11:47 AM, Brett Cannon  wrote:

>
>
> On Tue, Jun 12, 2012 at 12:38 PM, Alex Gaynor wrote:
>
>> For PyPy: I'm not an expert in our import, but from looking at the source
>>
>> 1) imp.cache_from_source is unimplemented, it's an AttributeError.
>>
>
> Well, you will have it come Python 3.3 one way or another. =)
>
>

Sure, I'm not totally up to speed on the py3k effort.


>
>> 2) sys.dont_write_bytecode is always false, we don't respect that flag
>> (we really
>>   should IMO, but it's not a high priority for me, or anyone else
>> apparently)
>>
>
> But doesn't PyPy read and write .pyc files (
> http://doc.pypy.org/en/latest/config/objspace.usepycfiles.html suggests
> you do)? So I would assume you are not affected by this. Jython and
> IronPython, though, would be (I think).
>

This is a compile time option, not a runtime option.  However, it looks
like I lied, someone did implement it correctly, so we have the same
behavior as CPython.

Alex


-- 
"I disapprove of what you say, but I will defend to the death your right to
say it." -- Evelyn Beatrice Hall (summarizing Voltaire)
"The people's good is the highest law." -- Cicero
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [compatibility-sig] making sure importlib.machinery.SourceLoader doesn't throw an exception if bytecode is not supported by a VM

2012-06-12 Thread Amaury Forgeot d'Arc 
2012/6/12 Alex Gaynor 

>
>
> On Tue, Jun 12, 2012 at 11:47 AM, Brett Cannon  wrote:
>
>>
>>
>> On Tue, Jun 12, 2012 at 12:38 PM, Alex Gaynor wrote:
>>
>>> For PyPy: I'm not an expert in our import, but from looking at the source
>>>
>>> 1) imp.cache_from_source is unimplemented, it's an AttributeError.
>>>
>>
>> Well, you will have it come Python 3.3 one way or another. =)
>>
>>
>
> Sure, I'm not totally up to speed on the py3k effort.
>

It's indeed implemented in pypy's py3k branch.


>
>
>>
>>> 2) sys.dont_write_bytecode is always false, we don't respect that flag
>>> (we really
>>>   should IMO, but it's not a high priority for me, or anyone else
>>> apparently)
>>>
>>
>> But doesn't PyPy read and write .pyc files (
>> http://doc.pypy.org/en/latest/config/objspace.usepycfiles.html suggests
>> you do)? So I would assume you are not affected by this. Jython and
>> IronPython, though, would be (I think).
>>
>
> This is a compile time option, not a runtime option.  However, it looks
> like I lied, someone did implement it correctly, so we have the same
> behavior as CPython.


Yes, PyPy seems to respect sys.dontwritebytecode.

-- 
Amaury Forgeot d'Arc
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [compatibility-sig] making sure importlib.machinery.SourceLoader doesn't throw an exception if bytecode is not supported by a VM

2012-06-12 Thread [email protected] 
On Tue, Jun 12, 2012 at 9:38 AM, Alex Gaynor  wrote:
> For PyPy: I'm not an expert in our import, but from looking at the source
>
> 1) imp.cache_from_source is unimplemented, it's an AttributeError.
Jython does not (yet) have a cache_from_source.

> 2) sys.dont_write_bytecode is always false, we don't respect that flag (we 
> really
>   should IMO, but it's not a high priority for me, or anyone else apparently)
Jython does support sys.dont_write_bytecode, but doesn't support
sys.dont_read_bytecode yet - do you happen to know when
dont_read_bytecode was added? It should be pretty straightforward, and
so I'll probably add it to our 2.7.

-Frank
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [compatibility-sig] making sure importlib.machinery.SourceLoader doesn't throw an exception if bytecode is not supported by a VM

2012-06-12 Thread [email protected] 
On Tue, Jun 12, 2012 at 10:04 AM, [email protected]
 wrote:
> Jython does support sys.dont_write_bytecode, but doesn't support
> sys.dont_read_bytecode yet - do you happen to know when
> dont_read_bytecode was added? It should be pretty straightforward, and
> so I'll probably add it to our 2.7.
Looking around it seems dont_read_bytecode came in sometime in 3.x so
I'll wait for 3 (and so I'll probably just use importlib? :)

-Frank
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [compatibility-sig] making sure importlib.machinery.SourceLoader doesn't throw an exception if bytecode is not supported by a VM

2012-06-12 Thread Brett Cannon 
On Tue, Jun 12, 2012 at 1:04 PM, [email protected] <
[email protected]> wrote:

> On Tue, Jun 12, 2012 at 9:38 AM, Alex Gaynor 
> wrote:
> > For PyPy: I'm not an expert in our import, but from looking at the source
> >
> > 1) imp.cache_from_source is unimplemented, it's an AttributeError.
> Jython does not (yet) have a cache_from_source.
>
> > 2) sys.dont_write_bytecode is always false, we don't respect that flag
> (we really
> >   should IMO, but it's not a high priority for me, or anyone else
> apparently)
> Jython does support sys.dont_write_bytecode, but doesn't support
> sys.dont_read_bytecode yet - do you happen to know when
> dont_read_bytecode was added?


It was never added since it doesn't currently exist; I said *add*
sys.dont_read_bytecode, not *use*. =)

Would the flag actually be beneficial? Have you had issues where people
assumed that Jython should be able to read bytecode since they just didn't
worry about a VM that can't read bytecode? I mean I'm open to suggestions
as ways to control bytecode reading to fail gracefully in case someone runs
Jython (or IronPython) in a directory where PyPy or CPython was run
previously and thus the bytecode exists from them. That's why I asked what
marshal.loads() does; if it returns None or raises some exception that can
be distinguished from a failure of badly formatted marshal data then I
could rely on that as another option.


> It should be pretty straightforward, and
> so I'll probably add it to our 2.7.
>

I would assume it would just be a flag for you. Importlib and other stdlib
code would be where all the work would be to start obeying the flag (if it
is added).
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] segfault - potential double free when using iterparse

2012-06-12 Thread Terry Reedy 

On 6/12/2012 9:40 AM, Alon Horev wrote:

Hi All,

First of all, I'm not opening a bug yet as I'm not certain whether this
is a CPython bug or lxml bug.


lxml is more likely, making this a topic for python-list and whatever 
lxml list.


...

from lxml.etree import iterparse

...


anyone familiar with these kind of bugs in c extensions/cpython/lxml?
could you give pointers to what I should be looking for?


I would ask the generic question on python-list and report the problem 
to lxml folks also.



some version info:
CPython version: 2.7.2 on linux.


2.7.3 has 6 months more of bugfixes, and if there is a problem with 
python, you would have to demonstrate a problem with that or a more 
recent build.


--
Terry Jan Reedy

___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] #12982: Should -O be required to *read* .pyo files?

2012-06-12 Thread Terry Reedy 

http://bugs.python.org/issue12982

Currently, cpython requires the -O flag to *read* .pyo files as well as 
the write them. This is a nuisance to people who receive them from 
others, without the source. The originator of the issue quotes the 
following from the doc (without giving the location).


"It is possible to have a file called spam.pyc (or spam.pyo when -O is 
used) without a file spam.py for the same module. This can be used to 
distribute a library of Python code in a form that is moderately hard to 
reverse engineer."


There is no warning that .pyo files are viral, in a sense. The user has 
to use -O, which is a) a nuisance to remember if he has multiple scripts 
and some need it and some not, and b) makes his own .py files used with 
.pyo imports cached as .pyo, without docstrings, like it or not.


Currently, the easiest workaround is to rename .pyo to .pyc and all 
seems to work fine, even with a mixture of true .pyc and renamed .pyo 
files. (The same is true with the -O flag and no renaming.) This 
suggests that there is no current reason for the restriction in that the 
*execution* of bytecode is not affected by the -O flag. (Another 
workaround might be a custom importer -- but this is not trivial, 
apparently.)


So is the import restriction either an accident or obsolete holdover? If 
so, can removing it be treated as a bugfix and put into current 
releases, or should it be treated as an enhancement only for a future 
release?


Or is the restriction an intentional reservation of the possibility of 
making *execution* depend on the flag? Which would mean that the 
restriction should be kept and only the doc changed?


--
Terry Jan Reedy

___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [compatibility-sig] making sure importlib.machinery.SourceLoader doesn't throw an exception if bytecode is not supported by a VM

2012-06-12 Thread Eric Snow 
On Tue, Jun 12, 2012 at 10:48 AM, Brett Cannon  wrote:
> I should mention another option is to add sys.dont_read_bytecode (I think I
> have discussed this with Frank at some point).

Or check for "sys.implementation.cache_tag is None"...

-eric
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] #12982: Should -O be required to *read* .pyo files?

2012-06-12 Thread Ethan Furman 

Terry Reedy wrote:

http://bugs.python.org/issue12982

Currently, cpython requires the -O flag to *read* .pyo files as well as 
the write them. This is a nuisance to people who receive them from 
others, without the source. The originator of the issue quotes the 
following from the doc (without giving the location).


"It is possible to have a file called spam.pyc (or spam.pyo when -O is 
used) without a file spam.py for the same module. This can be used to 
distribute a library of Python code in a form that is moderately hard to 
reverse engineer."


There is no warning that .pyo files are viral, in a sense. The user has 
to use -O, which is a) a nuisance to remember if he has multiple scripts 
and some need it and some not, and b) makes his own .py files used with 
.pyo imports cached as .pyo, without docstrings, like it or not.


Currently, the easiest workaround is to rename .pyo to .pyc and all 
seems to work fine, even with a mixture of true .pyc and renamed .pyo 
files. (The same is true with the -O flag and no renaming.) This 
suggests that there is no current reason for the restriction in that the 
*execution* of bytecode is not affected by the -O flag. (Another 
workaround might be a custom importer -- but this is not trivial, 
apparently.)


So is the import restriction either an accident or obsolete holdover? If 
so, can removing it be treated as a bugfix and put into current 
releases, or should it be treated as an enhancement only for a future 
release?


Or is the restriction an intentional reservation of the possibility of 
making *execution* depend on the flag? Which would mean that the 
restriction should be kept and only the doc changed?


I have no history so cannot say what was supposed to happen, but my 
$0.02 would be that if -O is *not* specified then we should try to read 
.pyc, then .pyo, and finally .py.  In other words, I vote for -O being a 
write flag, not a read flag.


~Ethan~
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] #12982: Should -O be required to *read* .pyo files?

2012-06-12 Thread Alexandre Zani 
On Tue, Jun 12, 2012 at 11:41 AM, Ethan Furman  wrote:
> Terry Reedy wrote:
>>
>> http://bugs.python.org/issue12982
>>
>> Currently, cpython requires the -O flag to *read* .pyo files as well as
>> the write them. This is a nuisance to people who receive them from others,
>> without the source. The originator of the issue quotes the following from
>> the doc (without giving the location).
>>
>> "It is possible to have a file called spam.pyc (or spam.pyo when -O is
>> used) without a file spam.py for the same module. This can be used to
>> distribute a library of Python code in a form that is moderately hard to
>> reverse engineer."
>>
>> There is no warning that .pyo files are viral, in a sense. The user has to
>> use -O, which is a) a nuisance to remember if he has multiple scripts and
>> some need it and some not, and b) makes his own .py files used with .pyo
>> imports cached as .pyo, without docstrings, like it or not.
>>
>> Currently, the easiest workaround is to rename .pyo to .pyc and all seems
>> to work fine, even with a mixture of true .pyc and renamed .pyo files. (The
>> same is true with the -O flag and no renaming.) This suggests that there is
>> no current reason for the restriction in that the *execution* of bytecode is
>> not affected by the -O flag. (Another workaround might be a custom importer
>> -- but this is not trivial, apparently.)
>>
>> So is the import restriction either an accident or obsolete holdover? If
>> so, can removing it be treated as a bugfix and put into current releases, or
>> should it be treated as an enhancement only for a future release?
>>
>> Or is the restriction an intentional reservation of the possibility of
>> making *execution* depend on the flag? Which would mean that the restriction
>> should be kept and only the doc changed?
>
>
> I have no history so cannot say what was supposed to happen, but my $0.02
> would be that if -O is *not* specified then we should try to read .pyc, then
> .pyo, and finally .py.  In other words, I vote for -O being a write flag,
> not a read flag.

What if I change .py?

>
> ~Ethan~
>
> ___
> Python-Dev mailing list
> [email protected]
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/alexandre.zani%40gmail.com
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [compatibility-sig] making sure importlib.machinery.SourceLoader doesn't throw an exception if bytecode is not supported by a VM

2012-06-12 Thread Brett Cannon 
On Tue, Jun 12, 2012 at 2:28 PM, Eric Snow wrote:

> On Tue, Jun 12, 2012 at 10:48 AM, Brett Cannon  wrote:
> > I should mention another option is to add sys.dont_read_bytecode (I
> think I
> > have discussed this with Frank at some point).
>
> Or check for "sys.implementation.cache_tag is None"...
>

Perfect! Will that work for Jython (Franke) and IronPython (Jeff)?

This does mean, though, that imp.cache_from_source() and
imp.source_from_cache() might need to be updated to raise a reasonable
exception when sys.implementation.cache_tag is set to None as I believe
right now it will raise a TypeError because None isn't a str. But what to
raise instead? TypeError? EnvironmentError?
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Using pdb with greenlet

2012-06-12 Thread Salman Malik 

Hi All,

I am sort of a newbie to Python ( have just started to use pdb). 
My problem is that I am debugging an application that uses greenlets and when I 
encounter something in code that spawns the coroutines or wait for an event, I 
lose control over the application (I mean that after that point I can no longer 
do 'n' or 's' on the code). Can anyone of you tell me how to tame greenlet with 
pdb, so that I can see step-by-step as to what event does a coroutine sees and 
how does it respond to it.

Any help would be highly appreciated.

Thanks,
Salman___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Using pdb with greenlet

2012-06-12 Thread Brian Curtin 
On Tue, Jun 12, 2012 at 2:31 PM, Salman Malik  wrote:
> Hi All,
>
> I am sort of a newbie to Python ( have just started to use pdb).
> My problem is that I am debugging an application that uses greenlets and
> when I encounter something in code that spawns the coroutines or wait for an
> event, I lose control over the application (I mean that after that point I
> can no longer do 'n' or 's' on the code). Can anyone of you tell me how to
> tame greenlet with pdb, so that I can see step-by-step as to what event does
> a coroutine sees and how does it respond to it.
>
> Any help would be highly appreciated.

Your question is better suited for python-list rather than python-dev.
This list is for the development *of* Python, not *with* Python.
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] #12982: Should -O be required to *read* .pyo files?

2012-06-12 Thread Ethan Furman 

Alexandre Zani wrote:

On Tue, Jun 12, 2012 at 11:41 AM, Ethan Furman  wrote:

Terry Reedy wrote:

http://bugs.python.org/issue12982

Currently, cpython requires the -O flag to *read* .pyo files as well as
the write them. This is a nuisance to people who receive them from others,
without the source. The originator of the issue quotes the following from
the doc (without giving the location).

"It is possible to have a file called spam.pyc (or spam.pyo when -O is
used) without a file spam.py for the same module. This can be used to
distribute a library of Python code in a form that is moderately hard to
reverse engineer."

There is no warning that .pyo files are viral, in a sense. The user has to
use -O, which is a) a nuisance to remember if he has multiple scripts and
some need it and some not, and b) makes his own .py files used with .pyo
imports cached as .pyo, without docstrings, like it or not.

Currently, the easiest workaround is to rename .pyo to .pyc and all seems
to work fine, even with a mixture of true .pyc and renamed .pyo files. (The
same is true with the -O flag and no renaming.) This suggests that there is
no current reason for the restriction in that the *execution* of bytecode is
not affected by the -O flag. (Another workaround might be a custom importer
-- but this is not trivial, apparently.)

So is the import restriction either an accident or obsolete holdover? If
so, can removing it be treated as a bugfix and put into current releases, or
should it be treated as an enhancement only for a future release?

Or is the restriction an intentional reservation of the possibility of
making *execution* depend on the flag? Which would mean that the restriction
should be kept and only the doc changed?


I have no history so cannot say what was supposed to happen, but my $0.02
would be that if -O is *not* specified then we should try to read .pyc, then
.pyo, and finally .py.  In other words, I vote for -O being a write flag,
not a read flag.


What if I change .py?


Well, the case in question is that there is no .py available.

But if it were available, and you changed it, then it would and should 
work just like it does now -- if .py is newer, compile it; if -O was 
specified, compile it optimized;  now run the compiled code.


~Ethan~
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Using pdb with greenlet

2012-06-12 Thread Salman Malik 

I am sorry for mailing it to this list.
Thanks for the correction.

Salman

> Date: Tue, 12 Jun 2012 14:34:04 -0500
> Subject: Re: [Python-Dev] Using pdb with greenlet
> From: [email protected]
> To: [email protected]
> CC: [email protected]
> 
> On Tue, Jun 12, 2012 at 2:31 PM, Salman Malik  wrote:
> > Hi All,
> >
> > I am sort of a newbie to Python ( have just started to use pdb).
> > My problem is that I am debugging an application that uses greenlets and
> > when I encounter something in code that spawns the coroutines or wait for an
> > event, I lose control over the application (I mean that after that point I
> > can no longer do 'n' or 's' on the code). Can anyone of you tell me how to
> > tame greenlet with pdb, so that I can see step-by-step as to what event does
> > a coroutine sees and how does it respond to it.
> >
> > Any help would be highly appreciated.
> 
> Your question is better suited for python-list rather than python-dev.
> This list is for the development *of* Python, not *with* Python.
  ___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [compatibility-sig] making sure importlib.machinery.SourceLoader doesn't throw an exception if bytecode is not supported by a VM

2012-06-12 Thread Jeff Hardy 
> 1) imp.cache_from_source is unimplemented, it's an AttributeError.

Same for IronPython.

>
> 2) sys.dont_write_bytecode is always false, we don't respect that flag (we 
> really
>   should IMO, but it's not a high priority for me, or anyone else apparently)

Always True for IronPython. You can change it, but it doesn't affect anything.

- Jeff
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [compatibility-sig] making sure importlib.machinery.SourceLoader doesn't throw an exception if bytecode is not supported by a VM

2012-06-12 Thread Jeff Hardy 
On Tue, Jun 12, 2012 at 12:01 PM, Brett Cannon  wrote:
> On Tue, Jun 12, 2012 at 2:28 PM, Eric Snow 
> wrote:
>>
>> On Tue, Jun 12, 2012 at 10:48 AM, Brett Cannon  wrote:
>> > I should mention another option is to add sys.dont_read_bytecode (I
>> > think I
>> > have discussed this with Frank at some point).
>>
>> Or check for "sys.implementation.cache_tag is None"...
>
>
> Perfect! Will that work for Jython (Franke) and IronPython (Jeff)?

IronPython will probably never *write* pyc files, but it might *read*
them at some point -- as I understand cache_tag, we'd set it to
whatever version of CPython's pyc files we could read (that seems to
violate the spirit of sys.implementation). The combination of that and
sys.dont_write_bytecode should cover all of the states; I'll just lock
down sys.dont_write_bytecode so that changes are completely ignored.

>
> This does mean, though, that imp.cache_from_source() and
> imp.source_from_cache() might need to be updated to raise a reasonable
> exception when sys.implementation.cache_tag is set to None as I believe
> right now it will raise a TypeError because None isn't a str. But what to
> raise instead? TypeError? EnvironmentError?

NotImplementedError?

- Jeff
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] #12982: Should -O be required to *read* .pyo files?

2012-06-12 Thread Ronan Lamy 
Le mardi 12 juin 2012 à 11:41 -0700, Ethan Furman a écrit :
> Terry Reedy wrote:
> > http://bugs.python.org/issue12982
> > 
> > Currently, cpython requires the -O flag to *read* .pyo files as well as 
> > the write them. This is a nuisance to people who receive them from 
> > others, without the source. The originator of the issue quotes the 
> > following from the doc (without giving the location).
> > 
> > "It is possible to have a file called spam.pyc (or spam.pyo when -O is 
> > used) without a file spam.py for the same module. This can be used to 
> > distribute a library of Python code in a form that is moderately hard to 
> > reverse engineer."
> > 
> > There is no warning that .pyo files are viral, in a sense. The user has 
> > to use -O, which is a) a nuisance to remember if he has multiple scripts 
> > and some need it and some not, and b) makes his own .py files used with 
> > .pyo imports cached as .pyo, without docstrings, like it or not.
> > 
> > Currently, the easiest workaround is to rename .pyo to .pyc and all 
> > seems to work fine, even with a mixture of true .pyc and renamed .pyo 
> > files. (The same is true with the -O flag and no renaming.) This 
> > suggests that there is no current reason for the restriction in that the 
> > *execution* of bytecode is not affected by the -O flag. (Another 
> > workaround might be a custom importer -- but this is not trivial, 
> > apparently.)
> > 
> > So is the import restriction either an accident or obsolete holdover? If 
> > so, can removing it be treated as a bugfix and put into current 
> > releases, or should it be treated as an enhancement only for a future 
> > release?
> > 
> > Or is the restriction an intentional reservation of the possibility of 
> > making *execution* depend on the flag? Which would mean that the 
> > restriction should be kept and only the doc changed?
> 
> I have no history so cannot say what was supposed to happen, but my 
> $0.02 would be that if -O is *not* specified then we should try to read 
> .pyc, then .pyo, and finally .py.  In other words, I vote for -O being a 
> write flag, not a read flag.

I don't know much about the history either, but under PEP 3147, there
are really two cases:
* .pyc and .pyo as compilation caches. These live in __pycache__/ and
have a cache_tag, their filename looks like
pkg/__pycache__/module.cpython-33.pyc and their only role is to speed up
imports.
* .pyc and .pyo as standalone, precompiled sources for modules. These
are found in the same place as .py files (e.g. pkg/module.pyc).

In the first case, I think that -O should dictate which of .pyc and .pyo
is used, while the other is completely ignored.

In the second case, both .pyc and .pyo should always be considered as
valid module sources, because -O is a compilation flag and loading a
bytecode file doesn't involve compilation. At most, -O could switch the
priority between .pyc and .pyo.

2.7 doesn't really differentiate between cached .pyc and
standalone .pyc, so I don't know if a consistent behaviour can be
achieved. Maybe the presence or absence of a matching .py can be used to
trigger the first or second case above.

___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [compatibility-sig] making sure importlib.machinery.SourceLoader doesn't throw an exception if bytecode is not supported by a VM

2012-06-12 Thread Brett Cannon 
On Tue, Jun 12, 2012 at 3:53 PM, Jeff Hardy  wrote:

> On Tue, Jun 12, 2012 at 12:01 PM, Brett Cannon  wrote:
> > On Tue, Jun 12, 2012 at 2:28 PM, Eric Snow 
> > wrote:
> >>
> >> On Tue, Jun 12, 2012 at 10:48 AM, Brett Cannon 
> wrote:
> >> > I should mention another option is to add sys.dont_read_bytecode (I
> >> > think I
> >> > have discussed this with Frank at some point).
> >>
> >> Or check for "sys.implementation.cache_tag is None"...
> >
> >
> > Perfect! Will that work for Jython (Franke) and IronPython (Jeff)?
>
> IronPython will probably never *write* pyc files, but it might *read*
> them at some point -- as I understand cache_tag, we'd set it to
> whatever version of CPython's pyc files we could read (that seems to
> violate the spirit of sys.implementation).


If you wanted to sneak around not writing your own bytecode but still
reading it, then yes, you could do that. And yes, that does somewhat
violate the concept of what sys.implementation.cache_tag was supposed to
do. =)


> The combination of that and

sys.dont_write_bytecode should cover all of the states; I'll just lock
> down sys.dont_write_bytecode so that changes are completely ignored.
>
>
Great!


> >
> > This does mean, though, that imp.cache_from_source() and
> > imp.source_from_cache() might need to be updated to raise a reasonable
> > exception when sys.implementation.cache_tag is set to None as I believe
> > right now it will raise a TypeError because None isn't a str. But what to
> > raise instead? TypeError? EnvironmentError?
>
> NotImplementedError?
>

That's also a possibility. I also realized that importlib checks for None
being returned by cache_from_source(), so that's another option (which ties
into how the rest of the PEP 302 methods work).
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] backporting stdlib 2.7.x from pypy to cpython

2012-06-12 Thread Antoine Pitrou 
On Mon, 11 Jun 2012 20:13:53 -0700
"[email protected]"  wrote:

> On Sun, Jun 10, 2012 at 11:58 PM, Nick Coghlan  wrote:
> > 1. Asking on python-dev is considered adequate. If an implementation
> > wants to be consulted on changes, one or more of their developers
> > *must* follow python-dev sufficiently closely that they don't miss
> > cross-VM compatibility questions. (My concern is that this isn't
> > reliable - we know from experience that other VMs can miss such
> > questions when they're mixed in with the rest of the python-dev
> > traffic)
> > 2. As 1, but we adopt a subject line convention to make it easier to
> > filter out general python-dev traffic for those that are just
> > interested in cross-vm questions
> > 3. Create a separate list for cross-VM discussions, *including*
> > discussions that aren't directly relevant to Python-the-language or
> > CPython-the-reference-interpreter (e.g. collaborating on a shared
> > standard library fork). python-dev threads may be advertised on the
> > new list if cross-VM feedback is considered particularly necessary.
> (2) and (3) work for me - I try to do (1) but often miss discussions
> until they have gone stale.

Either would be fine with me too.

cheers

Antoine.
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] time.clock_info() field names

2012-06-12 Thread Victor Stinner 
2012/5/4 Victor Stinner :
> Anyway, the implementation and/or the documentation is buggy and
> should be fixed (especially the Windows case).

Done, I renamed "adjusted" to "adjustable", fixed its value on Windows
(time.time) and Linux (time.monotonic), and updated the doc.
--
changeset:   77415:0e46e0cd368f
tag: tip
user:Victor Stinner 
date:Tue Jun 12 22:46:37 2012 +0200
files:   Doc/library/time.rst Include/pytime.h
Lib/test/test_time.py Misc/NEWS Modules/timemodule.c Python/pytime.c
description:
PEP 418: Rename adjusted attribute to adjustable in time.get_clock_info() result

Fix also its value on Windows and Linux according to its documentation:
"adjustable" indicates if the clock *can be* adjusted, not if it is or was
adjusted.

In most cases, it is not possible to indicate if a clock is or was adjusted.
--
Basically, time.get_clock_info().adjustable is only True for
time.time(). It can also be True for time.perf_counter() if
time.monotonic() is not available.

I prefer "adjustable" over "adjusted" because it is well defined and
its value is well known. For example, it is not easy to say if
time.monotonic() is "adjusted" or not on Linux, whereas I can say that
time.monotonic() is not *adjustable* on any OS.

I will update the PEP except if someone complains :-)

Sorry for being late, but I was exhausted by this PEP.

Victor
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [compatibility-sig] making sure importlib.machinery.SourceLoader doesn't throw an exception if bytecode is not supported by a VM

2012-06-12 Thread [email protected] 
On Tue, Jun 12, 2012 at 12:01 PM, Brett Cannon  wrote:
>
>
> On Tue, Jun 12, 2012 at 2:28 PM, Eric Snow 
> wrote:
>>
>> On Tue, Jun 12, 2012 at 10:48 AM, Brett Cannon  wrote:
>> > I should mention another option is to add sys.dont_read_bytecode (I
>> > think I
>> > have discussed this with Frank at some point).
>>
>> Or check for "sys.implementation.cache_tag is None"...
>
>
> Perfect! Will that work for Jython (Franke) and IronPython (Jeff)?
So Jython does actually emit bytecodes, but they are Java bytecodes
instead of Python bytecodes. Right now they end up next to the .py
files just like .pyc files. They have the possibly unfortunate naming
foo.py -> foo$py.class -- If I understand cache_tag (I may not) I
guess Python 3 is putting the .pyc files into hidden subdirectories
instead of putting them next to the .py files? If so we may do the
same with our $py.class files.

Incidentally we also have a mode for reading .pyc files -- though we
haven't implementing writing them yet (we probably will eventually)

I guess what I'm trying to say is that I don't know exactly how we
will handle these new flags, but chances are we will use them (Again
provided my guesses about what they do are anywhere near what they
really do).

>
> This does mean, though, that imp.cache_from_source() and
> imp.source_from_cache() might need to be updated to raise a reasonable
> exception when sys.implementation.cache_tag is set to None as I believe
> right now it will raise a TypeError because None isn't a str. But what to
> raise instead? TypeError? EnvironmentError?
NotImplementedError seems fine for me too if we don't end up using this flag.

-Frank
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] #12982: Should -O be required to *read* .pyo files?

2012-06-12 Thread Kristján Valur Jónsson 
[Vaguely related:
-B prevents the writing of .pyc and .pyo (don't know how it works for pep 3147)
However, it doesn't prevent the _reading_ of said files.  It's been discussed 
here before and considered useful, since rudiment .pyc files tend to stick 
around.  Maybe a -BB flag should be considered?]
K


Frá: [email protected] 
[[email protected]] fyrir hönd Ronan 
Lamy [[email protected]]
Sent: 12. júní 2012 19:57
To: Ethan Furman
Cc: [email protected]
Efni: Re: [Python-Dev] #12982: Should -O be required to *read* .pyo files?

Le mardi 12 juin 2012 à 11:41 -0700, Ethan Furman a écrit :
> Terry Reedy wrote:
> > http://bugs.python.org/issue12982
> >
> > Currently, cpython requires the -O flag to *read* .pyo files as well as
> > the write them. This is a nuisance to people who receive them from
> > others, without the source. The originator of the issue quotes the
> > following from the doc (without giving the location).
> >
> > "It is possible to have a file called spam.pyc (or spam.pyo when -O is
> > used) without a file spam.py for the same module. This can be used to
> > distribute a library of Python code in a form that is moderately hard to
> > reverse engineer."
> >
> > There is no warning that .pyo files are viral, in a sense. The user has
> > to use -O, which is a) a nuisance to remember if he has multiple scripts
> > and some need it and some not, and b) makes his own .py files used with
> > .pyo imports cached as .pyo, without docstrings, like it or not.
> >
> > Currently, the easiest workaround is to rename .pyo to .pyc and all
> > seems to work fine, even with a mixture of true .pyc and renamed .pyo
> > files. (The same is true with the -O flag and no renaming.) This
> > suggests that there is no current reason for the restriction in that the
> > *execution* of bytecode is not affected by the -O flag. (Another
> > workaround might be a custom importer -- but this is not trivial,
> > apparently.)
> >
> > So is the import restriction either an accident or obsolete holdover? If
> > so, can removing it be treated as a bugfix and put into current
> > releases, or should it be treated as an enhancement only for a future
> > release?
> >
> > Or is the restriction an intentional reservation of the possibility of
> > making *execution* depend on the flag? Which would mean that the
> > restriction should be kept and only the doc changed?
>
> I have no history so cannot say what was supposed to happen, but my
> $0.02 would be that if -O is *not* specified then we should try to read
> .pyc, then .pyo, and finally .py.  In other words, I vote for -O being a
> write flag, not a read flag.

I don't know much about the history either, but under PEP 3147, there
are really two cases:
* .pyc and .pyo as compilation caches. These live in __pycache__/ and
have a cache_tag, their filename looks like
pkg/__pycache__/module.cpython-33.pyc and their only role is to speed up
imports.
* .pyc and .pyo as standalone, precompiled sources for modules. These
are found in the same place as .py files (e.g. pkg/module.pyc).

In the first case, I think that -O should dictate which of .pyc and .pyo
is used, while the other is completely ignored.

In the second case, both .pyc and .pyo should always be considered as
valid module sources, because -O is a compilation flag and loading a
bytecode file doesn't involve compilation. At most, -O could switch the
priority between .pyc and .pyo.

2.7 doesn't really differentiate between cached .pyc and
standalone .pyc, so I don't know if a consistent behaviour can be
achieved. Maybe the presence or absence of a matching .py can be used to
trigger the first or second case above.

___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/kristjan%40ccpgames.com
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] issue #15038 - Optimize python Locks on Windows

2012-06-12 Thread Antoine Pitrou 
On Tue, 12 Jun 2012 11:23:28 +
Kristján Valur Jónsson  wrote:
> Hi,
> Could I get some feedback on this proposed patch?
> It would be great to get it in before the beta.

The review I made on your previous patch still applies.
You shouldn't ask for feedback if you aren't willing to take it into
account...

Regards

Antoine.


___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [compatibility-sig] making sure importlib.machinery.SourceLoader doesn't throw an exception if bytecode is not supported by a VM

2012-06-12 Thread Brett Cannon 
On Jun 12, 2012 6:24 PM, "[email protected]" 
wrote:
>
> On Tue, Jun 12, 2012 at 12:01 PM, Brett Cannon  wrote:
> >
> >
> > On Tue, Jun 12, 2012 at 2:28 PM, Eric Snow 
> > wrote:
> >>
> >> On Tue, Jun 12, 2012 at 10:48 AM, Brett Cannon 
wrote:
> >> > I should mention another option is to add sys.dont_read_bytecode (I
> >> > think I
> >> > have discussed this with Frank at some point).
> >>
> >> Or check for "sys.implementation.cache_tag is None"...
> >
> >
> > Perfect! Will that work for Jython (Franke) and IronPython (Jeff)?
> So Jython does actually emit bytecodes, but they are Java bytecodes
> instead of Python bytecodes. Right now they end up next to the .py
> files just like .pyc files. They have the possibly unfortunate naming
> foo.py -> foo$py.class -- If I understand cache_tag (I may not) I
> guess Python 3 is putting the .pyc files into hidden subdirectories
> instead of putting them next to the .py files?

Yes, __pycache__. The tag is to allow different versions of bytecode to
exist side-by-side (eg for CPython 3.3 it's cpython33 so the file ends up
being named foo-cpython33.pyc).

If so we may do the
> same with our $py.class files.

That was part of the hope when it was designed.

>
> Incidentally we also have a mode for reading .pyc files -- though we
> haven't implementing writing them yet (we probably will eventually)

If you can read .pyc files then you should be fine.

>
> I guess what I'm trying to say is that I don't know exactly how we
> will handle these new flags, but chances are we will use them (Again
> provided my guesses about what they do are anywhere near what they
> really do).

IOW it's too soon to be having this discussion. :) I mean regardless of
what happens you can always tweak the import lib code as necessary, I just
wanted to try to avoid it.

>
> >
> > This does mean, though, that imp.cache_from_source() and
> > imp.source_from_cache() might need to be updated to raise a reasonable
> > exception when sys.implementation.cache_tag is set to None as I believe
> > right now it will raise a TypeError because None isn't a str. But what
to
> > raise instead? TypeError? EnvironmentError?
> NotImplementedError seems fine for me too if we don't end up using this
flag.

OK, that's 2 votes for that exception.

>
> -Frank
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] backporting stdlib 2.7.x from pypy to cpython

2012-06-12 Thread Stephen J. Turnbull 
Brett Cannon writes:
 > On Tue, Jun 12, 2012 at 4:09 AM, Stephen J. Turnbull 
 > wrote:
 > > Brett Cannon writes:

 > Ah, but you helped make my point!

Not at all; your point has long since been made.  I certainly agree
that the current situation is unfortunate.  I think it's a bit rude of
you to assume that those who oppose a new discussion list don't
understand that.

The question is whether a new list will be a net positive for the
*whole* community, and whether it will significantly benefit the VM
developers (beyond giving them some leverage to say "you really should
have posted this to compatibility-sig, you know!")

 > If anything this new list would act as a showing of good will

"The road to Hell," as they say.

We tried this a couple of times at XEmacs; it didn't work.  In
practice, threads didn't move, they split, and the actual decisions
were taken on the main list, sometimes seriously offending the members
of the SIG list.  The analogy of topics is not exact, and Python is
more disciplined, so it might work better here.  But you should plan
for it, not merely appeal to "men of good will".

 > Anyway, it looks like everyone else chiming in is capitulating to keeping
 > it on python-dev with a proper subject line, so we will start with there
 > and if it proves ineffective I will create the list.

At that time, please consider an announce-only list that VM developers
can subscribe to in lieu of python-dev (maybe with reply-to directing
discussion to python-dev).
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [compatibility-sig] making sure importlib.machinery.SourceLoader doesn't throw an exception if bytecode is not supported by a VM

2012-06-12 Thread Nick Coghlan 
On Wed, Jun 13, 2012 at 11:10 AM, Brett Cannon  wrote:
>> > This does mean, though, that imp.cache_from_source() and
>> > imp.source_from_cache() might need to be updated to raise a reasonable
>> > exception when sys.implementation.cache_tag is set to None as I believe
>> > right now it will raise a TypeError because None isn't a str. But what
>> > to
>> > raise instead? TypeError? EnvironmentError?
>> NotImplementedError seems fine for me too if we don't end up using this
>> flag.
>
> OK, that's 2 votes for that exception.

+ 1 from me as well, both for skipping any implicit reading or writing
of the cache when cache_tag is None (IIRC, that's the use case we had
in mind when we allowed that field to be None in the PEP 421
discussion), and for *explicit* attempts to access the cache when the
tag is None triggering NotImplementedError.

That way people are free to use either LBYL (checking cache_tag) or
EAFP (catching NotImplementedError).

Cheers,
Nick.

-- 
Nick Coghlan   |   [email protected]   |   Brisbane, Australia
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] backporting stdlib 2.7.x from pypy to cpython

2012-06-12 Thread Nick Coghlan 
On Wed, Jun 13, 2012 at 2:30 PM, Stephen J. Turnbull  wrote:
> Brett Cannon writes:
>  > If anything this new list would act as a showing of good will
>
> "The road to Hell," as they say.
>
> We tried this a couple of times at XEmacs; it didn't work.  In
> practice, threads didn't move, they split, and the actual decisions
> were taken on the main list, sometimes seriously offending the members
> of the SIG list.  The analogy of topics is not exact, and Python is
> more disciplined, so it might work better here.  But you should plan
> for it, not merely appeal to "men of good will".

Aye, we already suffer the "split discussion" problem with import-sig
(and any other sig once conclusions are brought to python-dev for
ratification, although here every SIG already knows they will
eventually have to make their case on the main list for any standard
library changes).

I think the idea of using topic markers as a way to allow people to
set up their own filters that doesn't require spinning out a whole new
list is a good compromise. Adding a subject header is even less of a
burden than remembering to pass a question to a different list.

Cheers,
Nick.

-- 
Nick Coghlan   |   [email protected]   |   Brisbane, Australia
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com