Re: [Python-Dev] py34 makes it harder to read all of a pty

2014-11-12 Thread Charles-François Natali
2014-11-12 22:16 GMT+00:00 Buck Golemon buck.2...@gmail.com: This is due to the fix for issue21090, which aimed to un-silence errors which previously went unheard. The fix is for me, as a user, to write a loop that uses os.read and interpretes EIO as EOF. This is what I had hoped file.read()

Re: [Python-Dev] RFC: PEP 475, Retry system calls failing with EINTR

2014-09-01 Thread Charles-François Natali
There's no return value, a KeywordInterrupt exception is raised. The PEP wouldn't change this behavior. As for the general behavior: all programming languages/platforms handle EINTR transparently. It's high time for Python to have a sensible behavior in this regard. 2014-09-01 8:38 GMT+01:00

Re: [Python-Dev] RFC: PEP 475, Retry system calls failing with EINTR

2014-09-01 Thread Charles-François Natali
2014-09-01 12:15 GMT+01:00 Marko Rauhamaa ma...@pacujo.net: Charles-François Natali cf.nat...@gmail.com: Which raises an interesting question: what happens to the os.read() return value if SIGINT is received? There's no return value, a KeywordInterrupt exception is raised. The PEP wouldn't

Re: [Python-Dev] Exposing the Android platform existence to Python modules

2014-08-01 Thread Charles-François Natali
2014-08-01 13:23 GMT+01:00 Shiz h...@shiz.me: Is your P.S. suggestive that you would not be willing to support your port for use by others? Of course, until it is somewhat complete, it is hard to know how complete and compatible it can be. Oh, no, nothing like that. It's just that I'm not

Re: [Python-Dev] PEP 471: scandir(fd) and pathlib.Path(name, dir_fd=None)

2014-07-02 Thread Charles-François Natali
2014-07-01 8:44 GMT+01:00 Victor Stinner victor.stin...@gmail.com: IMO we must decide if scandir() must support or not file descriptor. It's an important decision which has an important impact on the API. I don't think we should support it: it's way too complicated to use, error-prone, and

Re: [Python-Dev] PEP 471: scandir(fd) and pathlib.Path(name, dir_fd=None)

2014-07-02 Thread Charles-François Natali
2014-07-02 12:51 GMT+02:00 Charles-François Natali cf.nat...@gmail.com: I don't think we should support it: it's way too complicated to use, error-prone, and leads to messy APIs. Can you please elaborate? Which kind of issue do you see? Handling the lifetime of the directory file descriptor

Re: [Python-Dev] should tests be thread-safe?

2014-05-10 Thread Charles-François Natali
You might have forgotten to include Python-dev in the reply. Indeed, adding it back! Thank you for the reply. I might have expressed the question poorely. I meant: I have a script that I know is not thread-safe but it doesn't matter because the test itself doesn't run any threads and the

Re: [Python-Dev] API and process questions (sparked by Claudiu Popa on 16104

2014-04-29 Thread Charles-François Natali
2014-04-28 21:24 GMT+01:00 Claudiu Popa pcmantic...@gmail.com: [...] If anyone agrees with the above, then I'll modify the patch. This will be its last iteration, any other bikeshedding should be addressed by the core dev who'll apply it. I'm perfectly happy with those proposals.

Re: [Python-Dev] [issue6839] zipfile can't extract file

2014-04-29 Thread Charles-François Natali
2014-04-30 3:58 GMT+01:00 Steven D'Aprano st...@pearwood.info: On Tue, Apr 29, 2014 at 07:48:00PM -0700, Jessica McKellar wrote: Hi Adam, Gentlemen, Thanks for contributing to Python! But not everyone on this list is a guy. And not all of the guys are gentlemen :-) And I thought guys

Re: [Python-Dev] API and process questions (sparked by Claudiu Popa on 16104

2014-04-28 Thread Charles-François Natali
(2) The patch adds new functionality to use multiple processes in parallel. The normal parameter values are integers indicating how many processes to use. The parameter also needs two special values -- one to indicate use os.cpu_count, and the other to indicate don't use multiprocessing at

Re: [Python-Dev] API and process questions (sparked by Claudiu Popa on 16104

2014-04-28 Thread Charles-François Natali
And incidentally, I think that the argument *processes* should be renamed to *workers*, or *jobs* (like in make), and any mention of multiprocessing in the documentation should be removed (if any): multiprocessing is an implementation detail. When I type: make -jN I don't really care that make is

[Python-Dev] file objects guarantees

2014-04-28 Thread Charles-François Natali
Hi, What's meant exactly by a file object? Let me be more specific: for example, pickle.dump() accepts a file object. Looking at the code, it doesn't check the return value of its write() method. So it assumes that write() should always write the whole data (not partial write). Same thing for

Re: [Python-Dev] API and process questions (sparked by Claudiu Popa on 16104

2014-04-28 Thread Charles-François Natali
2014-04-28 18:29 GMT+01:00 Jim J. Jewett jimjjew...@gmail.com: On Mon, Apr 28, 2014 at 12:56 PM, Charles-François Natali cf.nat...@gmail.com wrote: Why would the user care if multiprocessing is used behind the scene? Err ... that was another set of questions that I forgot to ask. (A) Why

Re: [Python-Dev] [numpy wishlist] PyMem_*Calloc

2014-04-15 Thread Charles-François Natali
Indeed, that's very reasonable. Please open an issue on the tracker! ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe:

[Python-Dev] pickle self-delimiting

2014-04-01 Thread Charles-François Natali
Hi, Unless I'm mistaken, pickle's documentation doesn't mention that the pickle wire-format is self-delimiting. Is there any reason why it's not documented? The reason I'm asking is because I've seen some code out there doing its own ad-hoc length-prefix framing. Cheers, cf

Re: [Python-Dev] pickle self-delimiting

2014-04-01 Thread Charles-François Natali
No reason AFAIK. However, the fact that it is self-delimited is implicit in the fact that Bytes past the pickled object's representation are ignored: https://docs.python.org/dev/library/pickle.html#pickle.load I find this sentence worrying: it could lead one to think that load() could read

Re: [Python-Dev] Confirming status of new modules in 3.4

2014-03-16 Thread Charles-François Natali
2014-03-15 21:44 GMT+00:00 Nikolaus Rath nikol...@rath.org: Guido van Rossum gu...@python.org writes: This downside of using subclassing as an API should be well known by now and widely warned against. It wasn't known to me until now. Are these downsides described in some more detail

Re: [Python-Dev] Confirming status of new modules in 3.4

2014-03-15 Thread Charles-François Natali
2014-03-15 11:02 GMT+00:00 Giampaolo Rodola' g.rod...@gmail.com: One part which can be improved is that right now the selectors module doesn't take advance of e/poll()'s modify() method: instead it just unregister() and register() the fd every time, which is of course considerably slower

Re: [Python-Dev] PEP 428 - pathlib API questions

2013-11-25 Thread Charles-François Natali
2013/11/25 Greg Ewing greg.ew...@canterbury.ac.nz: Ben Hoyt wrote: However, it seems there was no further discussion about why not extension and extensions? I have never heard a filename extension being called a suffix. You can't have read many unix man pages, then! I just searched for

[Python-Dev] PEP 454 - tracemalloc - accepted

2013-11-21 Thread Charles-François Natali
Hi, I'm happy to officially accept PEP 454 aka tracemalloc. The API has substantially improved over the past weeks, and is now both easy to use and suitable as a fundation for high-level tools for memory-profiling. Thanks to Victor for his work! Charles-François

[Python-Dev] PEP 454 (tracemalloc) close to pronouncement

2013-11-10 Thread Charles-François Natali
Hi, After several exchanges with Victor, PEP 454 has reached a status which I consider ready for pronuncement [1]: so if you have any last minute comment, now is the time! Cheers, cf [1] http://www.python.org/dev/peps/pep-0454/ ___ Python-Dev

Re: [Python-Dev] Updated PEP 454 (tracemalloc): no more metrics!

2013-10-24 Thread Charles-François Natali
2013/10/24 Kristján Valur Jónsson krist...@ccpgames.com: Now, I would personally not truncate the stack, because I can afford the memory, but even if I would, for example, to hide a bunch of detail, I would want to throw away the _lower_ detals of the stack. It is unimportant to me to

Re: [Python-Dev] [Python-checkins] cpython: Switch subprocess stdin to a socketpair, attempting to fix issue #19293 (AIX

2013-10-23 Thread Charles-François Natali
For the record, pipe I/O seems a little faster than socket I/O under Linux: $ ./python -m timeit -s import os, socket; a,b = socket.socketpair(); r=a.fileno(); w=b.fileno(); x=b'x'*1000 os.write(w, x); os.read(r, 1000) 100 loops, best of 3: 1.1 usec per loop $ ./python -m timeit -s

[Python-Dev] pathlib (PEP 428) status

2013-10-23 Thread Charles-François Natali
Hi, What's the current status of pathlib? Is it targeted for 3.4? It would be a really nice addition, and AFAICT it has already been maturing a while on pypi, and discussed several times here. If I remember correctly, the only remaining issue was stat()'s result caching. cf

Re: [Python-Dev] PEP 454 (tracemalloc): new minimalist version

2013-10-19 Thread Charles-François Natali
``get_tracemalloc_memory()`` function: Get the memory usage in bytes of the ``tracemalloc`` module as a tuple: ``(size: int, free: int)``. * *size*: total size of bytes allocated by the module, including *free* bytes * *free*: number of free bytes available to store

Re: [Python-Dev] PEP 454 (tracemalloc): new minimalist version

2013-10-19 Thread Charles-François Natali
2013/10/19 Nick Coghlan ncogh...@gmail.com: Speaking of which... Charles-François, would you be willing to act as BDFL-Delegate for this PEP? This will be a very useful new analysis tool, and between yourself and Victor it looks like you'll be able to come up with a solid API. I just

Re: [Python-Dev] PEP 454 (tracemalloc): new minimalist version

2013-10-18 Thread Charles-François Natali
Hi, I'm happy to see this move forward! API === Main Functions -- ``clear_traces()`` function: Clear traces and statistics on Python memory allocations, and reset the ``get_traced_memory()`` counter. That's nitpicking, but how about just ``reset()`` (I'm probably

Re: [Python-Dev] cpython: Try doing a raw test of os.fork()/os.kill().

2013-10-17 Thread Charles-François Natali
2013/10/17 Antoine Pitrou solip...@pitrou.net: On Thu, 17 Oct 2013 15:33:02 +0200 (CEST) richard.oudkerk python-check...@python.org wrote: http://hg.python.org/cpython/rev/9558e9360afc changeset: 86401:9558e9360afc parent: 86399:9cd88b39ef62 user:Richard Oudkerk

Re: [Python-Dev] PEP 428: Pathlib

2013-09-16 Thread Charles-François Natali
2013/9/16 Antoine Pitrou solip...@pitrou.net: Le Sun, 15 Sep 2013 06:46:08 -0700, Ethan Furman et...@stoneleaf.us a écrit : I see PEP 428 is both targeted at 3.4 and still in draft status. What remains to be done to ask for pronouncement? I think I have a couple of items left to integrate

Re: [Python-Dev] DTRACE support

2013-09-06 Thread Charles-François Natali
As far as I know, Erlang, Ruby, PHP, Perl, etc., support Dtrace. Python is embarrasingly missing from this list. Some examples: http://crypt.codemancers.com/posts/2013-04-16-profile-ruby-apps-dtrace-part1/ http://www.phpdeveloper.org/news/18859

Re: [Python-Dev] DTRACE support

2013-09-06 Thread Charles-François Natali
The main value of DTrace is systemwide observability. You can see something strange at kernel level and trace it to a particular line of code in a random Python script. There is no other tool that can do that. You have complete transversal observability of ALL the code running in your

Re: [Python-Dev] Add a new tracemalloc module to trace memory allocations

2013-08-31 Thread Charles-François Natali
2013/8/29 Victor Stinner victor.stin...@gmail.com: Charles-François Natali and Serhiy Storchaka asked me to add this module somewhere in Python 3.4: how about adding pyfailmalloc to the main repo (maybe under Tools), with a script making it easy to run the tests suite with it enabled

[Python-Dev] EINTR handling...

2013-08-30 Thread Charles-François Natali
Hello, This has been bothering me for years: why don't we properly handle EINTR, by running registered signal handlers and restarting the interrupted syscall (or eventually returning early e.g. for sleep)? EINTR is really a nuisance, and exposing it to Python code is just pointless. Now some

Re: [Python-Dev] EINTR handling...

2013-08-30 Thread Charles-François Natali
2013/8/30 Amaury Forgeot d'Arc amaur...@gmail.com: I agree. Is there a way to see in C code where EINTR is not handled? EINTR can be returned on slow syscalls, so a good heuristic would be to start with code that releases the GIL. But I don't see a generic way apart from grepping for syscalls

[Python-Dev] hg.python.org is slow

2013-08-27 Thread Charles-François Natali
Hi, I'm trying to checkout a pristine clone from ssh://h...@hg.python.org/cpython, and it's taking forever: 07:45:35.605941 IP 192.168.0.23.43098 virt-7yvsjn.psf.osuosl.org.ssh: Flags [.], ack 22081460, win 14225, options [nop,nop,TS val 368519 ecr 2401783356], length 0 07:45:38.558348 IP

Re: [Python-Dev] hg.python.org is slow

2013-08-27 Thread Charles-François Natali
2013/8/27 Antoine Pitrou solip...@pitrou.net: Sounds a lot like a network problem, then? If I'm the only one, it's likely, although these pathological timeouts are transient, and I don't have any problem with other servers (my line sustains 8Mb/s without problem). Have you tried a traceroute?

Re: [Python-Dev] PEP 446 (make FD non inheritable) ready for a final review

2013-08-23 Thread Charles-François Natali
Hello, A couple remarks: The following functions are modified to make newly created file descriptors non-inheritable by default: [...] os.dup() then os.dup2() has a new optional inheritable parameter: os.dup2(fd, fd2, inheritable=True). fd2 is created inheritable by default, but

Re: [Python-Dev] PEP 446 (make FD non inheritable) ready for a final review

2013-08-23 Thread Charles-François Natali
About your example: I'm not sure that it is reliable/portable. I sa daemon libraries closing *all* file descriptors and then expecting new file descriptors to become 0, 1 and 2. Your example is different because w is still open. On Windows, I have seen cases with only fd 0, 1, 2 open, and the

Re: [Python-Dev] PEP 446: Open issues/questions

2013-08-02 Thread Charles-François Natali
2013/8/2 Victor Stinner victor.stin...@gmail.com: 2013/7/28 Antoine Pitrou solip...@pitrou.net: (A) How should we support support where os.set_inheritable() is not supported? Can we announce that os.set_inheritable() is always available or not? Does such platform exist? FD_CLOEXEC is POSIX:

Re: [Python-Dev] PEP 446: Open issues/questions

2013-08-02 Thread Charles-François Natali
2013/8/2 Victor Stinner victor.stin...@gmail.com: On Windows, inheritable handles (including open files) are still inherited when a standard stream is overriden in the subprocess module (default value of close_fds is set to False in this case). This issue cannot be solved (at least, I don't

Re: [Python-Dev] PEP 446: Open issues/questions

2013-07-30 Thread Charles-François Natali
Having stdin/stdout/stderr cloexec (e.g. after a dup() to redirect to a log file, a socket...) will likely break a lot of code, e.g. code using os.system(), or code calling exec manually (and I'm sure there's a bunch of it). Hmm. os.exec*() could easily make standard streams non-CLOEXEC

Re: [Python-Dev] PEP 446: Open issues/questions

2013-07-28 Thread Charles-François Natali
2013/7/28 Antoine Pitrou solip...@pitrou.net: (C) Should we handle standard streams (0: stdin, 1: stdout, 2: stderr) differently? For example, os.dup2(fd, 0) should make the file descriptor 0 (stdin) inheritable or non-inheritable? On Windows, os.set_inheritable(fd, False) fails (error 87,

Re: [Python-Dev] PEP 446: Add new parameters to configure the inherance of files and for non-blocking sockets

2013-07-07 Thread Charles-François Natali
2013/7/7 Cameron Simpson c...@zip.com.au: On 06Jul2013 11:23, Charles-François Natali cf.nat...@gmail.com wrote: | I've read your Rejected Alternatives more closely and Ulrich | Drepper's article, though I think the article also supports adding | a blocking (default True) parameter to open

Re: [Python-Dev] PEP 446: Add new parameters to configure the inherance of files and for non-blocking sockets

2013-07-06 Thread Charles-François Natali
I've read your Rejected Alternatives more closely and Ulrich Drepper's article, though I think the article also supports adding a blocking (default True) parameter to open() and os.open(). If you try to change that default on a platform where it doesn't work, an exception should be raised.

Re: [Python-Dev] PEP 446: Add new parameters to configure the inherance of files and for non-blocking sockets

2013-07-05 Thread Charles-François Natali
2013/7/4 Victor Stinner victor.stin...@gmail.com: Even if the PEP 433 was not explicitly rejected, no consensus could be reached. I didn't want to loose all my work on this PEP and so I'm proposing something new which should make everbody agrees :-) Thanks Victor, I think this one is perfectly

Re: [Python-Dev] stat module in C -- what to do with stat.py?

2013-06-20 Thread Charles-François Natali
2013/6/20 Thomas Wouters tho...@python.org: If the .py file is going to be wrong or incomplete, why would we want to keep it -- or use it as fallback -- at all? If we're dead set on having a .py file instead of requiring it to be part of the interpreter (whichever that is, however it was

Re: [Python-Dev] pyparallel and new memory API discussions...

2013-06-19 Thread Charles-François Natali
2013/6/19 Trent Nelson tr...@snakebite.org: The new memory API discussions (and PEP) warrant a quick pyparallel update: a couple of weeks after PyCon, I came up with a solution for the biggest show-stopper that has been plaguing pyparallel since its inception: being able to

Re: [Python-Dev] HAVE_FSTAT?

2013-05-19 Thread Charles-François Natali
2013/5/17 Antoine Pitrou solip...@pitrou.net: Hello, Some pieces of code are still guarded by: #ifdef HAVE_FSTAT ... #endif I would expect all systems to have fstat() these days. It's pretty basic POSIX, and even Windows has had it for ages. Shouldn't we simply make those code blocks

Re: [Python-Dev] [RELEASED] Python 3.2.5 and Python 3.3.2

2013-05-16 Thread Charles-François Natali
2013/5/16 Serhiy Storchaka storch...@gmail.com: 16.05.13 08:20, Georg Brandl написав(ла): On behalf of the Python development team, I am pleased to announce the releases of Python 3.2.5 and 3.3.2. The releases fix a few regressions in 3.2.4 and 3.3.1 in the zipfile, gzip and xml.sax

Re: [Python-Dev] Issue 11406: adding os.scandir(), a directory iterator returning stat-like info

2013-05-14 Thread Charles-François Natali
I wonder how sshfs compared to nfs. (I've modified your benchmark to also test the case where data isn't in the page cache). Local ext3: cached: os.walk took 0.096s, scandir.walk took 0.030s -- 3.2x as fast uncached: os.walk took 0.320s, scandir.walk took 0.130s -- 2.5x as fast NFSv3, 1Gb/s

Re: [Python-Dev] PEP 435 - requesting pronouncement

2013-05-05 Thread Charles-François Natali
I'm chiming in late, but am I the only one who's really bothered by the syntax? class Color(Enum): red = 1 green = 2 blue = 3 I really don't see why one has to provide values, since an enum constant *is* the value. In many cases, there's no natural mapping between an enum constant

Re: [Python-Dev] PEP 428: stat caching undesirable?

2013-05-02 Thread Charles-François Natali
Yes, definitely. This is exactly what my os.walk() replacement, Betterwalk, does: https://github.com/benhoyt/betterwalk#readme On Windows you get *all* stat information from iterating the directory entries (FindFirstFile etc). And on Linux most of the time you get enough for os.walk() not

Re: [Python-Dev] PEP 428: stat caching undesirable?

2013-05-01 Thread Charles-François Natali
3) Leave it up to performance critical code, such as the import machinery, or walkdirs that Nick mentioned, to do their own caching, and simplify the filepath API for the simple case. But one can still make life easier for code like that, by adding is_file() and friends on the stat result

Re: [Python-Dev] Slides from today's parallel/async Python talk

2013-04-05 Thread Charles-François Natali
Hello, async.submit_work(func, args, kwds, callback=None, errback=None) How do you implement arguments passing and return value? e.g. let's say I pass a list as argument: how do you iterate on the list from the worker thread without modifying the backing objects for refcounts (IIUC you use

Re: [Python-Dev] Slides from today's parallel/async Python talk

2013-04-04 Thread Charles-François Natali
Just a quick implementation question (didn't have time to read through all your emails :-) async.submit_work(func, args, kwds, callback=None, errback=None) How do you implement arguments passing and return value? e.g. let's say I pass a list as argument: how do you iterate on the list from the

Re: [Python-Dev] [Announcement] New mailing list for code quality tools including Flake8, Pyflakes and Pep8

2013-04-03 Thread Charles-François Natali
Are you planning to cover the code quality of the interpreter itself too? I've been recently reading through the cert.org secure coding practice recommendations and was wondering if there has is any ongoing effort to perform static analysis on the cpython codebase. AFAICT CPython already

Re: [Python-Dev] Release or not release the GIL

2013-02-01 Thread Charles-François Natali
dup2(oldfd, newfd) closes oldfd. No, it doesn't close oldfd. It may close newfd if it was already open. (I guess that's what he meant). Anyway, only dup2() should probably release the GIL. One reasonable heuristic is to check the man page: if the syscall can return EINTR, then the GIL

Re: [Python-Dev] PEP 433: Choose the default value of the new cloexec parameter

2013-01-28 Thread Charles-François Natali
Library code should not be relying on globals settings that can change. Library code should be explicit in its calls so that the current value of a global setting is irrelevant. That's one of the problems I've raised with this global flag since the beginning: it's useless for libraries,

[Python-Dev] usefulness of extension modules section in Misc/NEWS

2013-01-27 Thread Charles-François Natali
Hi, What's exactly the guideline for choosing between the Library and Extension modules section when updating Misc/NEWS? Is it just the fact that the modified files live under Lib/ or Modules/? I've frequently made a mistake when updating Misc/NEWS, and when looking at it, I'm not the only one.

Re: [Python-Dev] PEP 433: Choose the default value of the new cloexec parameter

2013-01-25 Thread Charles-François Natali
Hello, I tried to list in the PEP 433 advantages and drawbacks of each option. If I recorded correctly opinions, the different options have the following supporters: a) cloexec=False by default b) cloexec=True by default: Charles-François Natali c) configurable default value: Antoine

Re: [Python-Dev] PEP 433: Add cloexec argument to functions creating file descriptors

2013-01-13 Thread Charles-François Natali
Hello, PEP: 433 Title: Add cloexec argument to functions creating file descriptors I'm not a native English speaker, but it seems to me that the correct wording should be parameter (part of the function definition/prototype, whereas argument refers to the actual value supplied). This PEP

Re: [Python-Dev] fork or exec?

2013-01-11 Thread Charles-François Natali
*Lots* of applications make use of POSIX semantics for fork() / exec(). This doesn't mean much. We're talking about inheritance of FDs 2 upon exec, which is a very limited subset of POSIX semantics for fork() / exec(). I personally think that there's been enough feedback to show that we should

Re: [Python-Dev] Set close-on-exec flag by default in SocketServer

2013-01-10 Thread Charles-François Natali
The SocketServer class creates a socket to listen on clients, and a new socket per client (only for stream server like TCPServer, not for UDPServer). Until recently (2011-05-24, issue #5715), the listening socket was not closed after fork for the ForkingMixIn flavor. This caused two issues:

Re: [Python-Dev] fork or exec?

2013-01-10 Thread Charles-François Natali
So, I read your e-mail again and I'm wondering if you're making a logic error, or if I'm misunderstanding something: 1. first you're talking about duplicate file or socket objects after *fork()* (which is an issue I agree is quite annoying) 2. the solution you're proposing doesn't close the

Re: [Python-Dev] fork or exec?

2013-01-10 Thread Charles-François Natali
Network servers like inetd or apache MPM (prefork) uses a process listening on a socket, and then fork to execute a request in a child process. I don't know how it works exactly, but I guess that the child process need a socket from the parent to send the answer to the client. If the socket

Re: [Python-Dev] fork or exec?

2013-01-10 Thread Charles-François Natali
That could always be overcome by passing close_fds=False explicitly to subprocess from my code, though, right? I'm not doing that now, but then I'm not using the esoteric options in python-gnupg code, either. You could do that, or better explicitly support this option, and only specify this

Re: [Python-Dev] Set close-on-exec flag by default in SocketServer

2013-01-09 Thread Charles-François Natali
My question is: would you accept to break backward compatibility (in Python 3.4) to fix a potential security vulnerability? Although obvious, the security implications are not restricted to sockets (yes, it's a contrived example): # cat test_inherit.py import fcntl import os import pwd import

Re: [Python-Dev] Bumping autoconf from 2.68 to 2.69

2012-10-16 Thread Charles-François Natali
My understanding is that we use a specific version of autoconf. The reason is that otherwise we end up with useless churn in the repo as the generated file changes when different committers use different versions. In the past we have had issues with a new autoconf version actually breaking

Re: [Python-Dev] Bumping autoconf from 2.68 to 2.69

2012-10-16 Thread Charles-François Natali
It should be sufficient to install autoconf-x.y into /home/user/bin or something similar. Installing autoconf from source really takes about 3 minutes. Well, maybe, maybe not. autoconf depends on a least m4 and Perl, and you may very well have a compatibility issue here. That's why most

Re: [Python-Dev] Checking if unsigned int less then zero.

2012-06-23 Thread Charles-François Natali
Playing with cpython source, I found some strange strings in socketmodule.c: --- if (flowinfo 0 || flowinfo 0xf) { PyErr_SetString( PyExc_OverflowError, getsockaddrarg: flowinfo must be 0-1048575.); return 0;

[Python-Dev] [help wanted] - IrDA sockets support

2012-04-25 Thread Charles-François Natali
Hi, Issue #1522400 (http://bugs.python.org/issue1522400) has a patch adding IrDA socket support. It builds under Linux and Windows, however it cannot go any further because no developer involved in the issue has access to IrDA capable devices, which makes testing impossible. So, if you have

Re: [Python-Dev] cpython: Closes Issue #14661: posix module: add O_EXEC, O_SEARCH, O_TTY_INIT (I add some

2012-04-24 Thread Charles-François Natali
jesus.cea python-check...@python.org wrote: http://hg.python.org/cpython/rev/2023f48b32b6 changeset: 76537:2023f48b32b6 user:Jesus Cea j...@jcea.es date:Tue Apr 24 20:59:17 2012 +0200 summary: Closes Issue #14661: posix module: add O_EXEC, O_SEARCH, O_TTY_INIT (I add

Re: [Python-Dev] Experimenting with STM on CPython

2012-04-11 Thread Charles-François Natali
Yes, that's using STM on my regular laptop.  How HTM would help remains unclear at this point, because in this approach transactions are typically rather large --- likely much larger than what the first-generation HTM-capable processors will support next year. Ok. I guess once the code is

Re: [Python-Dev] PEP 418: Add monotonic clock

2012-03-28 Thread Charles-François Natali
What's wrong with time.time() again?  As documented in http://docs.python.org/py3k/library/time.html it makes no guarantees, and specifically there is *no* guarantee that it will ever behave *badly*wink/.  Of course, we'll have to guarantee that, if a badly-behaved clock is available, users

Re: [Python-Dev] folding cElementTree behind ElementTree in 3.3

2012-02-16 Thread Charles-François Natali
I personally don't see any reason to drop a module that isn't terminally broken or unmaintainable, apart from scaring users away by making them think that we don't care about backward compatibility. ___ Python-Dev mailing list Python-Dev@python.org

[Python-Dev] best place for an atomic file API

2012-02-15 Thread Charles-François Natali
Hi, Issue #8604 aims at adding an atomic file API to make it easier to create/update files atomically, using rename() on POSIX systems and MoveFileEx() on Windows (which are now available through os.replace()). It would also use fsync() on POSIX to make sure data is committed to disk. For

Re: [Python-Dev] PEP 394 request for pronouncement (python2 symlink in *nix systems)

2012-02-12 Thread Charles-François Natali
There actually *is* an easy way, in regular ls: look at the link count. It comes out of ls -l by default, and if it's 1, there will be an identical file. This doesn't tell me which file it is, which is practically useless if I have both python3.3 and python3.2 in that directory. You can use

Re: [Python-Dev] [Python-checkins] cpython: Backed out changeset 36f2e236c601: For some reason, rewinddir() doesn't work as

2012-01-09 Thread Charles-François Natali
Can rewinddir() end up touching the filesystem to retrieve data? I noticed that your previous change (the one this checkin reverted) moved it outside the GIL release macros. It just resets a position count. (in glibc). Actually, it also calls lseek() on the directory FD:

[Python-Dev] svn.python.org certificate expired

2012-01-09 Thread Charles-François Natali
Hi, All the buildbots are turning red because of test_ssl: == ERROR: test_connect (test.test_ssl.NetworkedTests) -- Traceback (most recent call last): File

Re: [Python-Dev] usefulness of Python version of threading.RLock

2012-01-08 Thread Charles-François Natali
The yes/no answer is No, we can't drop it. Thanks, that's a clear answer :-) I'm not convinced of the benefits of removing the pure Python RLock implementation Indeed. As noted, this issue with signal handlers is more general, so this wouldn't solve the problem at hand. I just wanted to know

Re: [Python-Dev] usefulness of Python version of threading.RLock

2012-01-06 Thread Charles-François Natali
Thanks for those precisions, but I must admit it doesn't help me much... Can we drop it? A yes/no answer will do it ;-) I'm pretty sure the Python version of RLock is in use in several alternative implementations that provide an alternative _thread.lock. I think gevent would fall into this

[Python-Dev] usefulness of Python version of threading.RLock

2012-01-05 Thread Charles-François Natali
Hi, Issue #13697 (http://bugs.python.org/issue13697) deals with a problem with the Python version of threading.RLock (a signal handler which tries to acquire the same RLock is called right at the wrong time) which doesn't affect the C version. Whether such a use case can be considered good

Re: [Python-Dev] Fwd: Anyone still using Python 2.5?

2011-12-21 Thread Charles-François Natali
Do people still have to use this in commercial environments or is everyone on 2.6+ nowadays? RHEL 5.7 ships with Python 2.4.3. So no, not everybody is on 2.6+ today, and this won't happen before a couple years. cf ___ Python-Dev mailing list

Re: [Python-Dev] STM and python

2011-11-30 Thread Charles-François Natali
However given advances in locking and garbage collection in the last decade, what attempts have been made recently to try these new ideas out? In particular, how unlikely is it that all the thread safe primitives, global contexts, and reference counting functions be made __transaction_atomic,

Re: [Python-Dev] Unexpected behaviour in compileall

2011-11-02 Thread Charles-François Natali
2011/11/2 Vinay Sajip vinay_sa...@yahoo.co.uk: I just started getting errors in my PEP 404 / pythonv branch, but they don't at first glance appear related to the functionality of this branch. What I'm seeing is that during installation, some of the .pyc/.pyo files written by compileall have

Re: [Python-Dev] socket module build failure

2011-10-07 Thread Charles-François Natali
Hello, 2011/10/7 Vinay Sajip vinay_sa...@yahoo.co.uk: I work on Ubuntu Jaunty for my cpython development work - an old version, I know, but still quite serviceable and has worked well for me over many months. With the latest default cpython repository, however, I can't run the regression

Re: [Python-Dev] cpython (3.2): Issue #11956: Skip test_import.test_unwritable_directory on FreeBSD when run as

2011-10-06 Thread Charles-François Natali
I'd have expect this test to fail on _any_ UNIX system if run as root. Root's allowed to write to stuff! Any stuff! About the only permission with any effect on root is the eXecute bit for the exec call, to prevent blindly running random data files. You're right, here's another test on Linux

Re: [Python-Dev] Using PEP384 Stable ABI for the lzma extension module

2011-10-06 Thread Charles-François Natali
That's not a given. Depending on the memory allocator, a copy can be avoided. That's why the str += str hack is much more efficient under Linux than Windows, AFAIK. Even Linux will have to copy a block on realloc in certain cases, no? Probably so. How often is totally unknown

Re: [Python-Dev] cpython (3.2): Issue #11956: Skip test_import.test_unwritable_directory on FreeBSD when run as

2011-10-04 Thread Charles-François Natali
summary: Issue #11956: Skip test_import.test_unwritable_directory on FreeBSD when run as root (directory permissions are ignored). The same directory permission semantics apply to other (all?) BSD-derived systems, not just FreeBSD. For example, the test still fails in the same way on

Re: [Python-Dev] [Python-checkins] cpython: Issue #12981: rewrite multiprocessing_{sendfd, recvfd} in Python.

2011-09-26 Thread Charles-François Natali
On Sun, Sep 25, 2011 at 4:04 AM, charles-francois.natali python-check...@python.org wrote: +if not(sys.platform == 'win32' or (hasattr(socket, 'CMSG_LEN') and +                                   hasattr(socket, 'SCM_RIGHTS'))):     raise ImportError('pickling of connections not supported')

Re: [Python-Dev] issue 6721 Locks in python standard library should be sanitized on fork

2011-08-29 Thread Charles-François Natali
+3 (agreed to Jesse, Antoine and Ask here).  The http://bugs.python.org/issue8713 described non-fork implementation that always uses subprocesses rather than plain forked processes is the right way forward for multiprocessing. I see two drawbacks: - it will be slower, since the interpreter

Re: [Python-Dev] Software Transactional Memory for Python

2011-08-27 Thread Charles-François Natali
Hi Armin, This is basically dangerous, because it corresponds to taking lock GIL and lock L, in that order, whereas the thread B takes lock L and plays around with lock GIL in the opposite order.  I think a reasonable solution to avoid deadlocks is simply not to use explicit locks inside

Re: [Python-Dev] sendmsg/recvmsg on Mac OS X

2011-08-24 Thread Charles-François Natali
The buildbots are complaining about some of tests for the new socket.sendmsg/recvmsg added by issue #6560 for *nix platforms that provide CMSG_LEN. Looks like kernel bugs: http://developer.apple.com/library/mac/#qa/qa1541/_index.html Yes. Mac OS X 10.5 fixes a number of kernel bugs related

Re: [Python-Dev] sendmsg/recvmsg on Mac OS X

2011-08-24 Thread Charles-François Natali
But Snow Leopard, where these failures occur, is OS X 10.6. *sighs* It still looks like a kernel/libc bug to me: AFAICT, both the code and the tests are correct. And apparently, there are still issues pertaining to FD passing on 10.5 (and maybe later, I couldn't find a public access to their bug

Re: [Python-Dev] issue 6721 Locks in python standard library should be sanitized on fork

2011-08-23 Thread Charles-François Natali
2011/8/23, Nir Aides n...@winpdb.org: Hi all, Hello Nir, Please consider this invitation to stick your head into an interesting problem: http://bugs.python.org/issue6721 Just for the record, I'm now in favor of the atfork mechanism. It won't solve the problem for I/O locks, but it'll at

Re: [Python-Dev] issue 6721 Locks in python standard library should be sanitized on fork

2011-08-23 Thread Charles-François Natali
2011/8/23 Antoine Pitrou solip...@pitrou.net: Well, I would consider the I/O locks the most glaring problem. Right now, your program can freeze if you happen to do a fork() while e.g. the stderr lock is taken by another thread (which is quite common when debugging). Indeed. To solve this, a

Re: [Python-Dev] Comments of the PEP 3151

2011-07-27 Thread Charles-François Natali
I assume that ESHUTDOWN is the errno in question?  (This is also already mentioned in the PEP.) Indeed, I mentioned it in the PEP, as it appears in asyncore.py. But I can't find it on www.opengroup.org, and no man page on my Linux system (except the errno man page) seems to mention it.

Re: [Python-Dev] cpython (2.7): - Issue #12603: Fix pydoc.synopsis() on files with non-negative st_mtime.

2011-07-27 Thread Charles-François Natali
+- Issue #12603: Fix pydoc.synopsis() on files with non-negative st_mtime. + Surely you mean non-positive? Non-negative st_mtime being the common case. Of course (st_mtime = 0). ___ Python-Dev mailing list Python-Dev@python.org

Re: [Python-Dev] [Python-checkins] cpython: Issue #11784: Improve multiprocessing.Process.join() documentation. Patch by

2011-07-26 Thread Charles-François Natali
There’s a dedicated file to thank doc contributors: Doc/ACKS.rst I didn't know about this file, thanks. In my defense, there's this comment at the top of Misc/ACKS: This list is not complete and not in any useful order, but I would like to thank everybody who contributed in any way, with code,

Re: [Python-Dev] cpython: Merge - Issue #12592: Make Python build on OpenBSD 5 (and future major

2011-07-23 Thread Charles-François Natali
Note that this commit wasn't actually a merge -- you'll have to use the hg merge command for that. You're right. I guess that's what happens when I try to work past my usual bedtime ;-) By the way, I'm still getting errors upon push, and it looks like when I push a patch, this doesn't trigger

  1   2   >