Re: [Python-Dev] PEP 446: Open issues/questions

2013-07-29 Thread Antoine Pitrou
Le Sun, 28 Jul 2013 12:18:55 +0200,
Charles-François Natali  a écrit :
> 2013/7/28 Antoine Pitrou :
> >> (C) Should we handle standard streams (0: stdin, 1: stdout, 2:
> >> stderr) differently? For example, os.dup2(fd, 0) should make the
> >> file descriptor 0 (stdin) inheritable or non-inheritable? On
> >> Windows, os.set_inheritable(fd, False) fails (error 87, invalid
> >> argument) on standard streams (0, 1, 2) and copies of the standard
> >> streams (created by os.dup()).
> >
> > I have been advocating for that, but I now realize that
> > special-casing these three descriptors in a myriad of fd-creating
> > functions isn't very attractive.
> > (if a standard stream fd has been closed, any fd-creating function
> > can re-create that fd: including socket.socket(), etc.)
> >
> > So perhaps only the *original* standard streams should be left
> > inheritable?
> 
> Having stdin/stdout/stderr cloexec (e.g. after a dup() to redirect to
> a log file, a socket...) will likely break a lot of code, e.g. code
> using os.system(), or code calling exec manually (and I'm sure there's
> a bunch of it).

Hmm. os.exec*() could easily make standard streams non-CLOEXEC before
calling the underlying C library function. Things are more annoying for
os.system(), though.

> Also, it'll be puzzling to have syscalls automatically set the cloexec
> flag. I guess a lot of people doing system programming with Python
> will get bitten, but that's a discussion we already had months ago...

Perhaps this advocates for a global flag, e.g.
sys.set_default_fd_inheritance(), with False (non-inheritable) being
the default for sanity and security.

Regards

Antoine.


___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


[Python-Dev] MyOpenID.com no longer supported

2013-07-29 Thread Ethan Furman

Excerpt from http://meta.stackoverflow.com/q/190442/176681:

Janrain no longer actively supports MyOpenID, and announced on Twitter that 
their users should proceed with caution.

This decision was made by Janrain, [snip]

I know the Python bug tracker allows MyOpenID logins; if that is your only login method you should add another that 
doesn't depend on MyOpenID.


--
~Ethan~
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


[Python-Dev] PEP 446: cloexec flag on os.open

2013-07-29 Thread Ronald Oussoren
Victor,

PEP 446 mentions that a cloexec flag gets added to os.open. This API already 
has a way to specify this: the O_CLOEXEC bit in the flags argument. A new 
cloexec parameter is nicely consistent with the other APIs, but introcudes a 
second way to set that flag.

What will the following calls do?:

os.open(path, os.O_RDONLY|os.O_CLOEXEC, cloexec=False)
os.open(path, os.O_RDONLY, cloexec=True)

The PEP doesn't specify this, but the implementation for PEP 443 in issue 17036 
basicly ignores the cloexec argument in the first call and adds O_CLOEXEC in 
the second call. That can lead to confusing behavior when the flags argument to 
os.open is passed from elsewhere (e.g. a wrapper around os.open that passes in 
arguments and just overrides the cloexec argument).

It might be better to just drop the cloexec flag to os.open and make 
os.O_CLOEXEC an alias for os.O_NOINHERIT on Windows.

Ronald
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


[Python-Dev] PEP 447: add type.__locallookup__

2013-07-29 Thread Ronald Oussoren
Hi,

This PEP proposed to add a __locallookup__ slot to type objects,
which is used by _PyType_Lookup and super_getattro instead of peeking
in the tp_dict of classes.  The PEP text explains why this is needed.

Differences with the previous version:

* Better explanation of why this is a useful addition

* type.__locallookup__ is no longer optional.

* I've added benchmarking results using pybench.
  (using the patch attached to issue 18181)

Ronald




PEP: 447
Title: Add __locallookup__ method to metaclass
Version: $Revision$
Last-Modified: $Date$
Author: Ronald Oussoren 
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 12-Jun-2013
Post-History: 2-Jul-2013, 15-Jul-2013, 29-Jul-2013


Abstract


Currently ``object.__getattribute__`` and ``super.__getattribute__`` peek
in the ``__dict__`` of classes on the MRO for a class when looking for
an attribute. This PEP adds an optional ``__locallookup__`` method to
a metaclass that can be used to override this behavior.

Rationale
=

It is currently not possible to influence how the `super class`_ looks
up attributes (that is, ``super.__getattribute__`` unconditionally
peeks in the class ``__dict__``), and that can be problematic for
dynamic classes that can grow new methods on demand.

The ``__locallookup__`` method makes it possible to dynamicly add
attributes even when looking them up using the `super class`_.

The new method affects ``object.__getattribute__`` (and
`PyObject_GenericGetAttr`_) as well for consistency.

Background
--

The current behavior of ``super.__getattribute__`` causes problems for
classes that are dynamic proxies for other (non-Python) classes or types,
an example of which is `PyObjC`_. PyObjC creates a Python class for every
class in the Objective-C runtime, and looks up methods in the Objective-C
runtime when they are used. This works fine for normal access, but doesn't
work for access with ``super`` objects. Because of this PyObjC currently
includes a custom ``super`` that must be used with its classes.

The API in this PEP makes it possible to remove the custom ``super`` and
simplifies the implementation because the custom lookup behavior can be
added in a central location.


The superclass attribute lookup hook


Both ``super.__getattribute__`` and ``object.__getattribute__`` (or
`PyObject_GenericGetAttr`_ in C code) walk an object's MRO and peek in the
class' ``__dict__`` to look up attributes. A way to affect this lookup is
using a method on the meta class for the type, that by default looks up
the name in the class ``__dict__``.

In Python code
--

A meta type can define a method ``__locallookup__`` that is called during
attribute resolution by both ``super.__getattribute__`` and 
``object.__getattribute``::

class MetaType(type):
def __locallookup__(cls, name):
try:
return cls.__dict__[name]
except KeyError:
raise AttributeError(name) from None

The ``__locallookup__`` method has as its arguments a class and the name of the 
attribute
that is looked up. It should return the value of the attribute without invoking 
descriptors,
or raise `AttributeError`_ when the name cannot be found.

The `type`_ class provides a default implementation for ``__locallookup__``, 
that
looks up the name in the class dictionary.

Example usage
.

The code below implements a silly metaclass that redirects attribute lookup to 
uppercase
versions of names::

class UpperCaseAccess (type):
def __locallookup__(cls, name):
return cls.__dict__[name.upper()]

class SillyObject (metaclass=UpperCaseAccess):
def m(self):
return 42

def M(self):
return "fourtytwo"

obj = SillyObject()
assert obj.m() == "fortytwo"


In C code
-

A new slot ``tp_locallookup`` is added to the ``PyTypeObject`` struct, this slot
corresponds to the ``__locallookup__`` method on `type`_.

The slot has the following prototype::

PyObject* (*locallookupfunc)(PyTypeObject* cls, PyObject* name);

This method should lookup *name* in the namespace of *cls*, without looking at 
superclasses,
and should not invoke descriptors. The method returns ``NULL`` without setting 
an exception
when the *name* cannot be found, and returns a new reference otherwise (not a 
borrowed reference).

Use of this hook by the interpreter
---

The new method is required for metatypes and as such is defined on `type_`.  
Both
``super.__getattribute__`` and 
``object.__getattribute__``/`PyObject_GenericGetAttr`_
(through ``_PyType_Lookup``) use the this ``__locallookup__`` method when 
walking
the MRO.

Other changes to the implementation
---

The change for `PyObject_GenericGetAttr`_ will be done by changing the private 
function
``_PyType_Lookup``. This currently returns a borrowed reference, but

Re: [Python-Dev] PEP 447: add type.__locallookup__

2013-07-29 Thread Antoine Pitrou

Hi,

Le Mon, 29 Jul 2013 14:49:18 +0200,
Ronald Oussoren  a écrit :
> Hi,
> 
> This PEP proposed to add a __locallookup__ slot to type objects,
> which is used by _PyType_Lookup and super_getattro instead of peeking
> in the tp_dict of classes.  The PEP text explains why this is needed.
> 
> Differences with the previous version:
> 
> * Better explanation of why this is a useful addition
> 
> * type.__locallookup__ is no longer optional.
> 
> * I've added benchmarking results using pybench.
>   (using the patch attached to issue 18181)

Could you please run the whole benchmark suite?
http://hg.python.org/benchmarks/

Thanks

Antoine.


___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] PEP 447: add type.__locallookup__

2013-07-29 Thread Ronald Oussoren

On 29 Jul, 2013, at 14:58, Antoine Pitrou  wrote:

> 
> Hi,
> 
> Le Mon, 29 Jul 2013 14:49:18 +0200,
> Ronald Oussoren  a écrit :
>> Hi,
>> 
>> This PEP proposed to add a __locallookup__ slot to type objects,
>> which is used by _PyType_Lookup and super_getattro instead of peeking
>> in the tp_dict of classes.  The PEP text explains why this is needed.
>> 
>> Differences with the previous version:
>> 
>> * Better explanation of why this is a useful addition
>> 
>> * type.__locallookup__ is no longer optional.
>> 
>> * I've added benchmarking results using pybench.
>>  (using the patch attached to issue 18181)
> 
> Could you please run the whole benchmark suite?
> http://hg.python.org/benchmarks/

Sure.

Ronald

___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] PEP 447: add type.__locallookup__

2013-07-29 Thread Ronald Oussoren

On 29 Jul, 2013, at 15:07, Ronald Oussoren  wrote:

> 
> On 29 Jul, 2013, at 14:58, Antoine Pitrou  wrote:
> 
>> 
>> Hi,
>> 
>> Le Mon, 29 Jul 2013 14:49:18 +0200,
>> Ronald Oussoren  a écrit :
>>> Hi,
>>> 
>>> This PEP proposed to add a __locallookup__ slot to type objects,
>>> which is used by _PyType_Lookup and super_getattro instead of peeking
>>> in the tp_dict of classes.  The PEP text explains why this is needed.
>>> 
>>> Differences with the previous version:
>>> 
>>> * Better explanation of why this is a useful addition
>>> 
>>> * type.__locallookup__ is no longer optional.
>>> 
>>> * I've added benchmarking results using pybench.
>>> (using the patch attached to issue 18181)
>> 
>> Could you please run the whole benchmark suite?
>> http://hg.python.org/benchmarks/
> 
> Sure.

That's harder than I had expected, when I use the "make_perf3.sh" to create
a python 3 compatible version of the benchmark suite and then run the suite
it craps out because it cannnot find "spitfire", which isn't translated
(as are several other benchmarks).

I'll have to investigate why the suite doesn't work.

Ronald

> 
> Ronald
> 
> ___
> Python-Dev mailing list
> [email protected]
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> http://mail.python.org/mailman/options/python-dev/ronaldoussoren%40mac.com

___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] PEP 447: add type.__locallookup__

2013-07-29 Thread Antoine Pitrou
Le Mon, 29 Jul 2013 15:26:41 +0200,
Ronald Oussoren  a écrit :
> 
> On 29 Jul, 2013, at 15:07, Ronald Oussoren 
> wrote:
> 
> > 
> > On 29 Jul, 2013, at 14:58, Antoine Pitrou 
> > wrote:
> > 
> >> 
> >> Hi,
> >> 
> >> Le Mon, 29 Jul 2013 14:49:18 +0200,
> >> Ronald Oussoren  a écrit :
> >>> Hi,
> >>> 
> >>> This PEP proposed to add a __locallookup__ slot to type objects,
> >>> which is used by _PyType_Lookup and super_getattro instead of
> >>> peeking in the tp_dict of classes.  The PEP text explains why
> >>> this is needed.
> >>> 
> >>> Differences with the previous version:
> >>> 
> >>> * Better explanation of why this is a useful addition
> >>> 
> >>> * type.__locallookup__ is no longer optional.
> >>> 
> >>> * I've added benchmarking results using pybench.
> >>> (using the patch attached to issue 18181)
> >> 
> >> Could you please run the whole benchmark suite?
> >> http://hg.python.org/benchmarks/
> > 
> > Sure.
> 
> That's harder than I had expected, when I use the "make_perf3.sh" to
> create a python 3 compatible version of the benchmark suite and then
> run the suite it craps out because it cannnot find "spitfire", which
> isn't translated (as are several other benchmarks).

Ah, sorry. I think it's enough to run the "2n3" subsuite here, though.

Regards

Antoine.


___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] MyOpenID.com no longer supported

2013-07-29 Thread Guido van Rossum
On Mon, Jul 29, 2013 at 12:55 AM, Ethan Furman  wrote:
> Excerpt from http://meta.stackoverflow.com/q/190442/176681:
>
> Janrain no longer actively supports MyOpenID, and announced on Twitter that
> their users should proceed with caution.
>
> This decision was made by Janrain, [snip]
>
> I know the Python bug tracker allows MyOpenID logins; if that is your only
> login method you should add another that doesn't depend on MyOpenID.

I imagine plenty of people don't read either of the two lists you
posted to. Would it be possible to dig through the database and find
out which users use MyOpenID and email them individually?

-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] MyOpenID.com no longer supported

2013-07-29 Thread R. David Murray
On Mon, 29 Jul 2013 08:08:30 -0700, Guido van Rossum  wrote:
> On Mon, Jul 29, 2013 at 12:55 AM, Ethan Furman  wrote:
> > Excerpt from http://meta.stackoverflow.com/q/190442/176681:
> >
> > Janrain no longer actively supports MyOpenID, and announced on Twitter that
> > their users should proceed with caution.
> >
> > This decision was made by Janrain, [snip]
> >
> > I know the Python bug tracker allows MyOpenID logins; if that is your only
> > login method you should add another that doesn't depend on MyOpenID.
> 
> I imagine plenty of people don't read either of the two lists you
> posted to. Would it be possible to dig through the database and find
> out which users use MyOpenID and email them individually?

I don't know for sure that it is possible, but I presume that it is.
However, when I tried to do it, I couldn't figure out how to get from
the _openid_discovery table (which has 22 entries that have myopenid as
the endpoint server) to the corresponding user record.

Either Martin needs to clue me in, or I'll have to find time to read
his openid code :)

--David
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


[Python-Dev] C code coverage report with lcov

2013-07-29 Thread Christian Heimes
Hi,

I have done some experiments with GCC's gcov and lcov to get the C code
coverage of our unit test suite. You may find today's report at

  http://tiran.bitbucket.org/python-lcov/

I'm working on a patch for our Makefile to include all steps in one
simple make tag. http://bugs.python.org/issue18481

Christian

___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] C code coverage report with lcov

2013-07-29 Thread Brett Cannon
On Mon, Jul 29, 2013 at 1:15 PM, Christian Heimes wrote:

> Hi,
>
> I have done some experiments with GCC's gcov and lcov to get the C code
> coverage of our unit test suite. You may find today's report at
>
>   http://tiran.bitbucket.org/python-lcov/


Thanks!

I took a quick poke around and it seems some things are legitimately not
being executed, while others are error conditions that we wouldn't expect
to occur (e.g. memory exhaustion). If we ever decide to get serious about
code coverage (both C and Python code) we may need to have a discussion as
a group about our feelings related to pragmas dictating when code should be
left out of coverage reports.
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] C code coverage report with lcov

2013-07-29 Thread R. David Murray
On Mon, 29 Jul 2013 13:58:55 -0400, Brett Cannon  wrote:
> On Mon, Jul 29, 2013 at 1:15 PM, Christian Heimes wrote:
> 
> > Hi,
> >
> > I have done some experiments with GCC's gcov and lcov to get the C code
> > coverage of our unit test suite. You may find today's report at
> >
> >   http://tiran.bitbucket.org/python-lcov/
> 
> 
> Thanks!
> 
> I took a quick poke around and it seems some things are legitimately not
> being executed, while others are error conditions that we wouldn't expect
> to occur (e.g. memory exhaustion). If we ever decide to get serious about
> code coverage (both C and Python code) we may need to have a discussion as
> a group about our feelings related to pragmas dictating when code should be
> left out of coverage reports.

I suspect Victor will eventually have something for us for exercising the
memory exhaustion code :)

--David
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] C code coverage report with lcov

2013-07-29 Thread Christian Heimes
Am 29.07.2013 19:58, schrieb Brett Cannon:
> I took a quick poke around and it seems some things are legitimately not
> being executed, while others are error conditions that we wouldn't
> expect to occur (e.g. memory exhaustion). If we ever decide to get
> serious about code coverage (both C and Python code) we may need to have
> a discussion as a group about our feelings related to pragmas dictating
> when code should be left out of coverage reports.

Yeah, object allocation and creation checks as well as verification of
operating system API call are reason for lots branch and line misses.
I'm not sure what we can do about that. I don't want to plaster the code
with LCOV_EXCL_LINE comments.

As first action we should look into function coverage. We may not be
able to execute every branch but at least we should be able to call and
verify each function.

Christian
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] C code coverage report with lcov

2013-07-29 Thread Brett Cannon
On Mon, Jul 29, 2013 at 3:31 PM, Christian Heimes wrote:

> Am 29.07.2013 19:58, schrieb Brett Cannon:
> > I took a quick poke around and it seems some things are legitimately not
> > being executed, while others are error conditions that we wouldn't
> > expect to occur (e.g. memory exhaustion). If we ever decide to get
> > serious about code coverage (both C and Python code) we may need to have
> > a discussion as a group about our feelings related to pragmas dictating
> > when code should be left out of coverage reports.
>
> Yeah, object allocation and creation checks as well as verification of
> operating system API call are reason for lots branch and line misses.
> I'm not sure what we can do about that. I don't want to plaster the code
> with LCOV_EXCL_LINE comments.
>
> As first action we should look into function coverage. We may not be
> able to execute every branch but at least we should be able to call and
> verify each function.
>

If there's a way to report just function coverage then I think that's a
great place to start.
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] C code coverage report with lcov

2013-07-29 Thread Antoine Pitrou
On Mon, 29 Jul 2013 21:31:02 +0200
Christian Heimes  wrote:
> Am 29.07.2013 19:58, schrieb Brett Cannon:
> > I took a quick poke around and it seems some things are legitimately not
> > being executed, while others are error conditions that we wouldn't
> > expect to occur (e.g. memory exhaustion). If we ever decide to get
> > serious about code coverage (both C and Python code) we may need to have
> > a discussion as a group about our feelings related to pragmas dictating
> > when code should be left out of coverage reports.
> 
> Yeah, object allocation and creation checks as well as verification of
> operating system API call are reason for lots branch and line misses.
> I'm not sure what we can do about that. I don't want to plaster the code
> with LCOV_EXCL_LINE comments.

Ideally, we should run coverage runs on different systems (Unices,
Windows...) and merge the results together, so that we know which paths
are really uncovered.

Regards

Antoine.


___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] C code coverage report with lcov

2013-07-29 Thread Christian Heimes
Am 29.07.2013 21:38, schrieb Brett Cannon:
> If there's a way to report just function coverage then I think
> that's a great place to start.

lcov's genhtml command doesn't support just function coverage. But I
have removed branch coverage. It makes the report a little bit more
readable.

Christian
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] PEP 446: Open issues/questions

2013-07-29 Thread Victor Stinner
2013/7/28 Charles-François Natali :
> 2013/7/28 Antoine Pitrou :
>>> (C) Should we handle standard streams (0: stdin, 1: stdout, 2: stderr)
>>> differently? For example, os.dup2(fd, 0) should make the file
>>> descriptor 0 (stdin) inheritable or non-inheritable? On Windows,
>>> os.set_inheritable(fd, False) fails (error 87, invalid argument) on
>>> standard streams (0, 1, 2) and copies of the standard streams (created
>>> by os.dup()).
>>
>> I have been advocating for that, but I now realize that special-casing
>> these three descriptors in a myriad of fd-creating functions isn't very
>> attractive.
>> (if a standard stream fd has been closed, any fd-creating function can
>> re-create that fd: including socket.socket(), etc.)

>> So perhaps only the *original* standard streams should be left
>> inheritable?

I plan to only change functions *creating* (and replacing) new file
descriptors. Existing file descriptors (like 0, 1, 2) are unchanged.

> Having stdin/stdout/stderr cloexec (e.g. after a dup() to redirect to
> a log file, a socket...) will likely break a lot of code, e.g. code
> using os.system(), or code calling exec manually (and I'm sure there's
> a bunch of it).

I propose to add just one special case os.dup2(fd, fd2): if fd2 < 3,
fd2 is set to inheritable (otherwise, fd2 is set to non-inheritable).
os.dup2() is commonly used to replace stdin, stdout and stderr between
fork and exec. The http.server (cgi server) and pty modules use dup2()
for example

http://hg.python.org/features/pep-446/rev/f8a52518be4c

Victor
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] PEP 446: Open issues/questions

2013-07-29 Thread Antoine Pitrou
On Mon, 29 Jul 2013 23:42:36 +0200
Victor Stinner  wrote:
> 
> >> So perhaps only the *original* standard streams should be left
> >> inheritable?
> 
> I plan to only change functions *creating* (and replacing) new file
> descriptors. Existing file descriptors (like 0, 1, 2) are unchanged.

You can create fds 0, 1 and 2 if they were closed before.

> > Having stdin/stdout/stderr cloexec (e.g. after a dup() to redirect to
> > a log file, a socket...) will likely break a lot of code, e.g. code
> > using os.system(), or code calling exec manually (and I'm sure there's
> > a bunch of it).
> 
> I propose to add just one special case os.dup2(fd, fd2): if fd2 < 3,
> fd2 is set to inheritable (otherwise, fd2 is set to non-inheritable).
> os.dup2() is commonly used to replace stdin, stdout and stderr between
> fork and exec. The http.server (cgi server) and pty modules use dup2()
> for example

Mmh. Doing it only in dup2() sounds like too much special casing for me.
I would prefer adding a new explicit parameter:
  os.dup2(fd, fd2, inheritable=False)

(as a bonus, it can be implemented using dup3() under (GNU/)Linux)

Regards

Antoine.
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


[Python-Dev] importing modules

2013-07-29 Thread syed khalid
I am attempting to import modules from Shogun to python from a non-standard
python directory ie from my /home/xxx directory. is there a way on ubuntu
to selectively some modules, scripts, data  from one directory and others
modules, scripts from another directory. In other words, is there a file(s)
that provide pointers to where these modules are located.

regards

Sy
--
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] PEP 446: Open issues/questions

2013-07-29 Thread Victor Stinner
2013/7/28 Antoine Pitrou :
>> (B) Should subprocess make the file descriptors of pass_fds
>> inheritable? If yes, should it be done before or after the fork? If it
>> is done after the fork and before exec, it only affects the child
>> process, at least on Linux (the file descriptor is still
>> non-inheritable in the parent process).
>
> If it is done, it should definitely be done after the fork, IMO.
> subprocess shouldn't have any long-lasting effects on the process.

I modified the subprocess module to make fds of pass_fds inheritable.
http://hg.python.org/features/pep-446/rev/75e5d34898aa

If we don't do that, it will probably break all applications using
pass_fds (and so the backward compatibility).

Victor
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] importing modules

2013-07-29 Thread Steven D'Aprano

Hi Syed,

On 30/07/13 07:57, syed khalid wrote:

I am attempting to import modules from Shogun to python from a non-standard
python directory ie from my /home/xxx directory. is there a way on ubuntu
to selectively some modules, scripts, data  from one directory and others
modules, scripts from another directory. In other words, is there a file(s)
that provide pointers to where these modules are located.


This mailing list is for development OF Python, not development WITH Python. 
Questions like yours are best asked on the [email protected] mailing list, 
or if you prefer Usenet, comp.lang.python.

http://mail.python.org/mailman/listinfo/python-list

My (brief) answer to your question, is you probably should set sys.path 
appropriately. If that's not enough to point you in the right direction to 
solve the problem, please take any further discussion to python-list rather 
than ask here.



Regards,



--
Steven
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com