[issue16507] Patch selectmodule.c to support WSAPoll on Windows

2012-12-07 Thread Richard Oudkerk
Richard Oudkerk added the comment: It seems that the return code of WSAPoll() does not include the count of array items with revents == POLLNVAL. In the case where all of them are POLLNVAL, instead of returning 0 (which usually indicates a timeout) it returns -1 and WSAGetLastError

[issue16616] test_poll.PollTests.poll_unit_tests() is dead code

2012-12-10 Thread Richard Oudkerk
Changes by Richard Oudkerk shibt...@gmail.com: -- status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue16616 ___ ___ Python-bugs

[issue15526] test_startfile crash on Windows 7 AMD64

2012-12-10 Thread Richard Oudkerk
Changes by Richard Oudkerk shibt...@gmail.com: -- status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15526 ___ ___ Python-bugs

[issue16507] Patch selectmodule.c to support WSAPoll on Windows

2012-12-16 Thread Richard Oudkerk
Richard Oudkerk added the comment: Here is a new version with tests and docs. Note that the docs do not mention the bug mentioned in http://daniel.haxx.se/blog/2012/10/10/wsapoll-is-broken/ Maybe they should? Note that that bug makes it a bit difficult to use poll with tulip on Windows

[issue16718] Mysterious atexit fail

2012-12-18 Thread Richard Oudkerk
Richard Oudkerk added the comment: When you run wy.py the wow module gets partially imported, and then garbage collected because it fails to import successfully. The destructor for the module replaces values in the module's __dict__ with None. So when the cleanup function runs you get

[issue16718] Mysterious atexit fail

2012-12-19 Thread Richard Oudkerk
Richard Oudkerk added the comment: Things should be better in the future, when modules are cleared with true garbage collection. When is this future of which you speak? I am not sure whether it would affect performance, but a weakrefable subclass of dict could be used for module dicts

[issue16718] Mysterious atexit fail

2012-12-19 Thread Richard Oudkerk
Richard Oudkerk added the comment: See issue812369 for the shutdown procedure and modules cleanup. I am aware of that issue, but the original patch is 9 years old. Which is why I ask if/when it will actually happen. -- ___ Python tracker rep

[issue16736] select.poll() converts long to int without checking for overflow

2012-12-20 Thread Richard Oudkerk
New submission from Richard Oudkerk: Relevant code: int timeout = 0, poll_result, i, j; ... tout = PyNumber_Long(tout); if (!tout) return NULL; timeout = PyLong_AsLong(tout); -- implicit cast to int -- messages: 177811 nosy: sbt priority

[issue16736] select.poll() converts long to int without checking for overflow

2012-12-20 Thread Richard Oudkerk
Richard Oudkerk added the comment: Thanks. I will close. -- stage: - committed/rejected status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue16736

[issue16718] Mysterious atexit fail

2012-12-20 Thread Richard Oudkerk
Richard Oudkerk added the comment: Perhaps the simplest thing would be to stop doing anything special when a module is garbage collected: the garbage collector can take care of any orphaned ref-cycles involving the module dict. Then at shutdown the remaining modules in sys.modules could have

[issue16743] mmap accepts files 1 GB, but processes only 1 GB

2012-12-24 Thread Richard Oudkerk
Richard Oudkerk added the comment: I suspect that the size of the 5GB file is originally a 64 bit quantity, but gets cast unsafely to a 32 bit size_t to give 1GB. This is causing the miscalculations. There is no way to map all of a 5GB file in a 32 bit process -- 4GB is the maximum -- so

[issue16743] mmap accepts files 1 GB, but processes only 1 GB

2012-12-24 Thread Richard Oudkerk
Richard Oudkerk added the comment: This bit looks wrong to me: if (offset - size PY_SSIZE_T_MAX) /* Map area too large to fit in memory */ m_obj-size = (Py_ssize_t) -1; Should it not be size - offset instead of offset - size? (offset and size

[issue16743] mmap accepts files 1 GB, but processes only 1 GB

2012-12-26 Thread Richard Oudkerk
Richard Oudkerk added the comment: On 32 bit Unix mmap() will raise ValueError(mmap length is too large) in Marc's example. This is correct since Python's sequence protocol does not support indexes larger than sys.maxsize. But on 32 bit Windows, if length == 0 then the size check always

[issue16743] mmap accepts files 1 GB, but processes only 1 GB

2012-12-26 Thread Richard Oudkerk
Richard Oudkerk added the comment: This change is not backward compatible. Now user can mmap a larger file and safely access lower 2 GiB. With the patch it will fail. They should specify length=2GiB-1 if that is what they want. With length=0 you can only access the lower 2GiB if file_size

[issue16743] mmap accepts files 1 GB, but processes only 1 GB

2012-12-26 Thread Richard Oudkerk
Richard Oudkerk added the comment: New patch with same check for Unix. -- Added file: http://bugs.python.org/file28446/mmap.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue16743

[issue16743] mmap accepts files 1 GB, but processes only 1 GB

2012-12-26 Thread Richard Oudkerk
Richard Oudkerk added the comment: Isn't 2 GiB + 1 bytes mmap file enough for testing? Yes. But creating multigigabyte files is very slow on Windows. On Linux/FreeBSD test_mmap takes a fraction of a second, whereas on Windows it takes over 2 minutes. (Presumably Linux/FreeBSD

[issue16743] mmap on Windows can mishandle files larger than sys.maxsize

2012-12-26 Thread Richard Oudkerk
Changes by Richard Oudkerk shibt...@gmail.com: -- title: mmap accepts files 1 GB, but processes only 1 GB - mmap on Windows can mishandle files larger than sys.maxsize type: enhancement - behavior ___ Python tracker rep...@bugs.python.org http

[issue8713] multiprocessing needs option to eschew fork() under Linux

2012-12-27 Thread Richard Oudkerk
Richard Oudkerk added the comment: Richard, apart from performance, what's the advantage of this approach over the fork+exec version? It is really just performance. For context running the unittests in a 1 cpu linux VM gives me fork: real0m53.868s user0m1.496s sys 0m9.757s

[issue8713] multiprocessing needs option to eschew fork() under Linux

2012-12-27 Thread Richard Oudkerk
Richard Oudkerk added the comment: Numbers when running on Linux on a laptop with 2 cores + hyperthreading. RUNNING UNITTESTS: fork: real0m50.687s user0m9.213s sys 0m4.012s fork+exec: real1m9.062s user0m48.579s sys 0m6.648s forkserver

[issue8713] multiprocessing needs option to eschew fork() under Linux

2012-12-27 Thread Richard Oudkerk
Changes by Richard Oudkerk shibt...@gmail.com: Added file: http://bugs.python.org/file28461/8f08d83264a0.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue8713

[issue8713] multiprocessing needs option to eschew fork() under Linux

2012-12-27 Thread Richard Oudkerk
Richard Oudkerk added the comment: The safest default would be fork+exec though we need to implement the fork+exec code as a C extension module or have it use subprocess (as I noted in the mb_fork_exec.patch review). That was an old version of the patch. In the branch http

[issue16802] fileno argument to socket.socket() undocumented

2012-12-28 Thread Richard Oudkerk
New submission from Richard Oudkerk: The actual signature is socket.socket(family=AF_INET, type=SOCK_STREAM, proto=0, fileno=None) but the documented signature is socket.socket([family[, type[, proto]]]) Should the fileno argument be documented or is it considered an implementation

[issue16802] fileno argument to socket.socket() undocumented

2012-12-31 Thread Richard Oudkerk
Richard Oudkerk added the comment: The fileno argument looks like an implementation detail to me. It has at least one potential use. On Windows socket.detach() returns a socket handle but there is no documented way to close it -- os.close() will not work. The only way to close it that I

[issue16802] fileno argument to socket.socket() undocumented

2012-12-31 Thread Richard Oudkerk
Richard Oudkerk added the comment: There is an alternative (documented) interface: socket.fromfd(handle, socket.AF_INET, socket.SOCK_STREAM).close() socket.fromfd() duplicates the handle, so that does not close the original handle

[issue9586] warning: comparison between pointer and integer in multiprocessing build on Tiger

2013-01-01 Thread Richard Oudkerk
Changes by Richard Oudkerk shibt...@gmail.com: -- resolution: - fixed stage: - committed/rejected status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue9586

[issue12105] open() does not able to set flags, such as O_CLOEXEC

2013-01-03 Thread Richard Oudkerk
Richard Oudkerk added the comment: Note that on Windows there is an O_NOINHERIT flag which almost corresponds to O_CLOEXEC on Linux. I don't think there is a need to use the win32 api. -- nosy: +sbt ___ Python tracker rep...@bugs.python.org http

[issue12939] Add new io.FileIO using the native Windows API

2013-01-03 Thread Richard Oudkerk
Richard Oudkerk added the comment: A while ago I did write a PipeIO class which subclasses io.RawIOBase and works for overlapped pipe handles. (It was intended for multiprocessing and doing asynchronous IO with subprocess.) As it is it would not work with normal files because when you do

[issue12939] Add new io.FileIO using the native Windows API

2013-01-03 Thread Richard Oudkerk
Richard Oudkerk added the comment: Attached is a module for Python 3.3+ which subclasses io.RawIOBase. The constructor signature is WinFileIO(handle, mode=r, closehandle=True) where mode is r, w, r+ or w+. Handles can be created using _winapi.CreateFile(). Issues: - No support

[issue12939] Add new io.FileIO using the native Windows API

2013-01-03 Thread Richard Oudkerk
Changes by Richard Oudkerk shibt...@gmail.com: Added file: http://bugs.python.org/file28545/test_winfileio.py ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12939

[issue16873] increase epoll.poll() maxevents default value, and improve documentation

2013-01-06 Thread Richard Oudkerk
Richard Oudkerk added the comment: Is this actually a problem? If events are arranged in a queue and epoll_wait() just removes the oldest events (up to maxevents) from that queue then there would be no problem with using a small value for maxevents. I don't *know* if that is the case, but I

[issue12939] Add new io.FileIO using the native Windows API

2013-01-06 Thread Richard Oudkerk
Richard Oudkerk added the comment: Attached is a patch which adds a winio module which is a replacement for io, but uses windows handles instead of fds. It reimplements FileIO and open(), and provides openhandle() and closehandle() as replacements for os.open() and os.close(). test_io has

[issue16873] increase epoll.poll() maxevents default value, and improve documentation

2013-01-06 Thread Richard Oudkerk
Richard Oudkerk added the comment: I actually wrote a script to reproduce this issue: The program does *not* demonstrate starvation because you are servicing the resource represented by the starved duplicate fds before calling poll() again. You are creating thousands of duplicate handles

[issue16873] increase epoll.poll() maxevents default value, and improve documentation

2013-01-06 Thread Richard Oudkerk
Richard Oudkerk added the comment: The fact that that the FDs are duped shouldn't change anything to the events reported: it works while the number of FDs is less than FD_SETSIZE (epoll_wait() maxevents argument). That assumes that epoll_wait() is supposed to return *all* ready fds

[issue16873] increase epoll.poll() maxevents default value, and improve documentation

2013-01-06 Thread Richard Oudkerk
Richard Oudkerk added the comment: Here is a version which uses epoll to service a number of pipes which is larger than maxevents. (If NUM_WRITERS is too large then I get OSError: [Errno 24] Too many open files.) All pipes get serviced and the output is: Working with 20 FDs, 5 maxevents [5

[issue16873] increase epoll.poll() maxevents default value, and improve documentation

2013-01-06 Thread Richard Oudkerk
Richard Oudkerk added the comment: Yes, but the problem is that between two epoll_wait() calls, the readiness of the FDs can have changed: and if that happens, you'll get the same list over and over. If an fd *was* ready but isn't anymore then why would you want to know about it? Trying

[issue12939] Add new io.FileIO using the native Windows API

2013-01-06 Thread Richard Oudkerk
Richard Oudkerk added the comment: I don't like the idea of a specific I/O module for an OS. Is the public API different? It was partly to make integration with the existing tests easier: _io, _pyio and winio are tested in parallel. Can't you reuse the io module? In what sense? I don't

[issue16853] add a Selector to the select module

2013-01-08 Thread Richard Oudkerk
Richard Oudkerk added the comment: Richard, in Tulip's WSAPoll code, it reads: class WindowsPollPollster(PollPollster): Pollster implementation using WSAPoll. WSAPoll is only available on Windows Vista and later. Python does not currently support WSAPoll

[issue16261] Fix bare excepts in various places in std lib

2013-01-09 Thread Richard Oudkerk
Richard Oudkerk added the comment: try: _MAXFD = os.sysconf(SC_OPEN_MAX) -except: +except ValueError: _MAXFD = 256 os.sysconf() might raise OSError. I think ValueError is only raised if _SC_OPEN_MAX was undefined when the module was compiled. -- nosy: +sbt

[issue16920] multiprocessing.connection listener gets MemoryError on recv

2013-01-10 Thread Richard Oudkerk
Richard Oudkerk added the comment: Why are you connecting to a multiprocessing listener with a raw socket? You should be using multiprocessing.connection.Client to create a client connection. Connection.send(obj) writes a 32 bit unsigned int (in network order) to the socket representing

[issue12939] Add new io.FileIO using the native Windows API

2013-01-12 Thread Richard Oudkerk
Richard Oudkerk added the comment: Attached is a new patch which is implemented completely in C. It adds a WinFileIO class to the io module, which has the same API as FileIO except that: * It has a handle attribute instead of a fileno() method. * It has staticmethods openhandle

[issue12939] Add new io.FileIO using the native Windows API

2013-01-12 Thread Richard Oudkerk
Changes by Richard Oudkerk shibt...@gmail.com: Removed file: http://bugs.python.org/file28544/winfileio.c ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12939

[issue12939] Add new io.FileIO using the native Windows API

2013-01-12 Thread Richard Oudkerk
Changes by Richard Oudkerk shibt...@gmail.com: Removed file: http://bugs.python.org/file28545/test_winfileio.py ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12939

[issue12939] Add new io.FileIO using the native Windows API

2013-01-12 Thread Richard Oudkerk
Changes by Richard Oudkerk shibt...@gmail.com: Removed file: http://bugs.python.org/file28590/winfileio.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12939

[issue12939] Add new io.FileIO using the native Windows API

2013-01-12 Thread Richard Oudkerk
Richard Oudkerk added the comment: Forgot to mention, the handles are non-inheritable. You can use _winapi.DuplicateHandle() to create an inheritable duplicate handle if you really need to. -- ___ Python tracker rep...@bugs.python.org http

[issue16920] multiprocessing.connection listener gets MemoryError onrecv

2013-01-13 Thread Richard Oudkerk
Richard Oudkerk added the comment: If someone used regular sockets deliberately, they could crash multiprocessing server code deliberately. Any chance of doing a real message length check against the embedded message length check? You can do message = conn.recv_bytes(maxlength) if you

[issue16955] multiprocessing.connection poll() always returns false

2013-01-13 Thread Richard Oudkerk
Richard Oudkerk added the comment: Thanks for the report. It seems to only affect Windows, and only when using sockets rather than pipes. Till this is fixed you could use temp = bool(multiprocessing.connection.wait([cl], 1)) instead of temp = cl.poll(1) As I mentioned on the other

[issue10527] multiprocessing.Pipe problem: handle out of range in select()

2013-01-13 Thread Richard Oudkerk
Richard Oudkerk added the comment: The commits did not have the intended effect. They just define a _poll() function (and only on Windows) and it is not referenced anywhere else. I will look in to fixing this -- on 2.7 and 3.2 this will need to be done in the C code. -- resolution

[issue16955] multiprocessing.connection poll() always returns false

2013-01-13 Thread Richard Oudkerk
Richard Oudkerk added the comment: This should be fixed now. -- resolution: - fixed stage: - committed/rejected status: open - closed type: - behavior ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue16955

[issue10527] multiprocessing.Pipe problem: handle out of range in select()

2013-01-13 Thread Richard Oudkerk
Richard Oudkerk added the comment: What do you mean? The intent was to use poll() instead of select() anywhere available in order to avoid running out of fds. The change didn't affect Windows because as of right now select() is the only thing available. The change *only* effects Windows

[issue10527] multiprocessing.Pipe problem: handle out of range in select()

2013-01-13 Thread Richard Oudkerk
Richard Oudkerk added the comment: It looks like the change to multiprocessing/connection.py committed does not match the one uploaded as issue10527-3.patch changeset 81174:e971a70984b8 1.1 --- a/Lib/multiprocessing/connection.py 1.2 +++ b/Lib/multiprocessing/connection.py 1.3

[issue12939] Add new io.FileIO using the native Windows API

2013-01-14 Thread Richard Oudkerk
Richard Oudkerk added the comment: Added some comments on Rietveld. The .fileno() method is missing. Can this cause a problem when the file is passed to stdlib functions? subprocess for example? Thanks. An older version of the patch had a fileno() method which returned the handle

[issue12939] Add new io.FileIO using the native Windows API

2013-01-14 Thread Richard Oudkerk
Richard Oudkerk added the comment: What does this proposal bring exactly? Unless we are willing to completely replace fds with handles on Windows, perhaps not too much. (At one point I had assumed that that was the plan for py3k.) Although not advertised, openhandle() does have

[issue16920] multiprocessing.connection listener gets MemoryErroronrecv

2013-01-14 Thread Richard Oudkerk
Richard Oudkerk added the comment: If you want to communicate between processes of the same progam, you are best off calling multiprocessing.Pipe() or multiprocessing.Queue() in the main process. Queues or connections can then be inherited by the child processes. Usually all communication

[issue16966] Publishing multiprocessing listener code

2013-01-14 Thread Richard Oudkerk
Richard Oudkerk added the comment: For the reasons I wrote in the other issue, I don't think this an approach to encourage. There was no need to create a new issue: if you post to a closed issue then people on the nosy list will still see your message. So I will close this issue. (Maybe

[issue12939] Add new io.FileIO using the native Windows API

2013-01-15 Thread Richard Oudkerk
Richard Oudkerk added the comment: New patch reflecting Amaury's comments. -- Added file: http://bugs.python.org/file28737/winfileio.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12939

[issue12939] Add new io.FileIO using the native Windows API

2013-01-15 Thread Richard Oudkerk
Changes by Richard Oudkerk shibt...@gmail.com: Removed file: http://bugs.python.org/file28707/winfileio.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12939

[issue16998] Lost updates with multiprocessing.Value

2013-01-19 Thread Richard Oudkerk
Richard Oudkerk added the comment: I thought that access to the value field of Value instances was protected by locks to avoid lost updates. Loads and stores are both atomic. But += is made up of two operations, a load followed by a store, and the lock is dropped between the two. The same

[issue16998] Lost updates with multiprocessing.Value

2013-01-20 Thread Richard Oudkerk
Richard Oudkerk added the comment: I see. Then this is a documentation bug. The examples in the documentation use such non-thread-safe assignments (combined with the statement These shared objects will be process and thread safe.). Are you talking about this documentation: If lock

[issue16507] Patch selectmodule.c to support WSAPoll on Windows

2013-01-20 Thread Richard Oudkerk
Richard Oudkerk added the comment: That compiles (after hacking the line endings). One Tulip test fails, PollEventLooptests.testSockClientFail. But that's probably because the PollSelector class hasn't been adjusted for Windows yet (need to dig this out of the Pollster code

[issue16507] Patch selectmodule.c to support WSAPoll on Windows

2013-01-21 Thread Richard Oudkerk
Richard Oudkerk added the comment: On 21/01/2013 5:38pm, Guido van Rossum wrote: This is a very good question to which I have no good answer. If it weren't for this, we could probably do away with the distinction between add_writer and add_connector, and a lot of code could be simpler

[issue16507] Patch selectmodule.c to support WSAPoll on Windows

2013-01-21 Thread Richard Oudkerk
Richard Oudkerk added the comment: On 21/01/2013 7:00pm, Guido van Rossum wrote: Regarding your IOCP changes, that sounds pretty exciting. Richard, could you check those into the Tulip as a branch? (Maybe a new branch named 'iocp'.) OK. It may take me a while to rebase them

[issue16507] Patch selectmodule.c to support WSAPoll on Windows

2013-01-21 Thread Richard Oudkerk
Richard Oudkerk added the comment: I have created an iocp branch. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue16507 ___ ___ Python-bugs-list

[issue16507] Patch selectmodule.c to support WSAPoll on Windows

2013-01-22 Thread Richard Oudkerk
Richard Oudkerk added the comment: It appears that Linux's spurious readiness notifications are a deliberate deviation from the POSIX standard. (They are mentioned in the BUGS section of the man page for select.) Should I just apply the following patch to the default branch? diff -r

[issue16507] Patch selectmodule.c to support WSAPoll on Windows

2013-01-22 Thread Richard Oudkerk
Richard Oudkerk added the comment: According to Alan Cox It's a design decision and a huge performance win. It's one of the areas where POSIX read in its strictest form cripples your performance. See https://lkml.org/lkml/2011/6/18/103 (For write ready, you can obviously have

[issue16507] Patch selectmodule.c to support WSAPoll on Windows

2013-01-22 Thread Richard Oudkerk
Richard Oudkerk added the comment: For SOCK_STREAM, yes, not for SOCK_DGRAM I thought SOCK_DGRAM messages just got truncated at the receiving end. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue16507

[issue17018] Inconsistent behaviour of methods waiting for child process

2013-01-23 Thread Richard Oudkerk
Changes by Richard Oudkerk shibt...@gmail.com: -- nosy: +sbt ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17018 ___ ___ Python-bugs-list mailing

[issue16743] mmap on Windows can mishandle files larger than sys.maxsize

2013-01-27 Thread Richard Oudkerk
Richard Oudkerk added the comment: On 27/01/2013 8:27pm, Terry J. Reedy wrote: I agree we do not need to retain unpredictable 'dumb luck' -- in future versions. But in the absence of a clear discrepancy between doc and behavior (the definition of a bug) I believe breaking such code

[issue16743] mmap on Windows can mishandle files larger than sys.maxsize

2013-01-27 Thread Richard Oudkerk
Richard Oudkerk added the comment: On 27/01/2013 9:06pm, Serhiy Storchaka wrote: Every bugfix breaks some code. As a compromise I propose set m_obj-size = PY_SSIZE_T_MAX in case of overflow and emit a warning. Trying to allocate PY_SSIZE_T_MAX bytes always seems to fail

[issue17097] baseManager serve_client() not check EINTR when recv request

2013-02-01 Thread Richard Oudkerk
Changes by Richard Oudkerk shibt...@gmail.com: -- nosy: +sbt ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17097 ___ ___ Python-bugs-list mailing

[issue16743] mmap on Windows can mishandle files larger than sys.maxsize

2013-02-13 Thread Richard Oudkerk
Richard Oudkerk added the comment: Perhaps NEWS item needed for this change. Done. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue16743

[issue16743] mmap on Windows can mishandle files larger than sys.maxsize

2013-02-13 Thread Richard Oudkerk
Changes by Richard Oudkerk shibt...@gmail.com: -- resolution: - fixed stage: patch review - committed/rejected status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue16743

[issue15528] Better support for finalization with weakrefs

2013-02-14 Thread Richard Oudkerk
Richard Oudkerk added the comment: Richard, do you still want to push this forward? Otherwise I'd like to finalize the patch (in the other sense ;-). I started to worry a bit about daemon threads. I think they can still run while atexit functions are being run.* So if a daemon thread

[issue15528] Better support for finalization with weakrefs

2013-02-14 Thread Richard Oudkerk
Richard Oudkerk added the comment: On 14/02/2013 3:16pm, Antoine Pitrou wrote: Mmmh... thread switching is already blocked at shutdown: http://hg.python.org/cpython/file/0f827775f7b7/Python/ceval.c#l434 But in Py_Finalize(), call_py_exitfuncs() is called *before* _Py_Finalizing is set

[issue16246] Multiprocessing infinite loop on Windows

2013-02-14 Thread Richard Oudkerk
Changes by Richard Oudkerk shibt...@gmail.com: -- status: pending - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue16246 ___ ___ Python-bugs

[issue15528] Better support for finalization with weakrefs

2013-02-14 Thread Richard Oudkerk
Richard Oudkerk added the comment: In any case, I think it's just something we'll have to live with. Daemon threads are not a terrific idea in general. I agree. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15528

[issue17221] Resort Misc/NEWS

2013-02-18 Thread Richard Oudkerk
Richard Oudkerk added the comment: I did not realize there was a 'Extension Modules' section. I have been putting changes to C extensions in the 'Library' section instead. It looks like most people do the same as me. -- nosy: +sbt ___ Python

[issue17221] Resort Misc/NEWS

2013-02-18 Thread Richard Oudkerk
Richard Oudkerk added the comment: Was not it be yanked in 1fabff717ef4? Looks like it was reintroduced by this merge changeset: http://hg.python.org/cpython/rev/30fc620e240e -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org

[issue15004] add weakref support to types.SimpleNamespace

2013-02-20 Thread Richard Oudkerk
Richard Oudkerk added the comment: Good, except that you have to add a gc.collect() call for the non-refcounted implementations. Better to use test.support.gc_collect(). -- nosy: +sbt ___ Python tracker rep...@bugs.python.org http

[issue17258] multiprocessing.connection challenge implicitly uses MD5

2013-02-20 Thread Richard Oudkerk
Richard Oudkerk added the comment: Banning md5 as a matter of policy may be perfectly sensible. However, I think the way multiprocessing uses hmac authentication is *not* affected by the collision attacks the advisory talks about. These depend on the attacker being able to determine

[issue17261] multiprocessing.manager BaseManager cannot return proxies from proxies remotely (when listening on '')

2013-02-21 Thread Richard Oudkerk
Changes by Richard Oudkerk shibt...@gmail.com: -- nosy: +sbt ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17261 ___ ___ Python-bugs-list mailing

[issue17273] multiprocessing.pool.Pool task/worker handlers are not fork safe

2013-02-22 Thread Richard Oudkerk
Richard Oudkerk added the comment: A pool should only be used by the process that created it (unless you use a managed pool). If you are creating long lived processes then you could create a new pool on demand. For example (untested) pool_pid = (None, None) def get_pool

[issue17273] multiprocessing.pool.Pool task/worker handlers are not fork safe

2013-02-22 Thread Richard Oudkerk
Richard Oudkerk added the comment: Richard, are you suggesting that we close this, or do you see an actionable issue? (a plausible patch to the repository?) I skimmed the documentation and could not see that this restriction has been documented. So I think a documentation patch would

[issue10527] multiprocessing.Pipe problem: handle out of range in select()

2013-02-26 Thread Richard Oudkerk
Changes by Richard Oudkerk shibt...@gmail.com: -- resolution: - fixed stage: commit review - committed/rejected status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue10527

[issue17223] Initializing array.array with unicode type code and buffer segfaults

2013-02-26 Thread Richard Oudkerk
Richard Oudkerk added the comment: The new test seems to be reliably failing on Windows: == FAIL: test_issue17223 (__main__.UnicodeTest) -- Traceback (most

[issue17314] Stop using imp.find_module() in multiprocessing

2013-02-27 Thread Richard Oudkerk
Richard Oudkerk added the comment: I think this change will potentially make the main module get imported twice under different names when we transfer pickled data between processes. The current code (which is rather a mess) goes out of its way to avoid that. Basically the main process makes

[issue17025] reduce multiprocessing.Queue contention

2013-03-04 Thread Richard Oudkerk
Richard Oudkerk added the comment: It looks like queues_contention.diff has the line obj = pickle.dumps(obj) in both _feed() and put(). Might that be why the third set of benchmarks was slower than the second? -- ___ Python tracker rep

[issue17025] reduce multiprocessing.Queue contention

2013-03-04 Thread Richard Oudkerk
Richard Oudkerk added the comment: On 04/03/2013 8:01pm, Charles-Fran├žois Natali wrote: It looks like queues_contention.diff has the line obj = pickle.dumps(obj) in both _feed() and put(). Might that be why the third set of benchmarks was slower than the second? _feed() is a Queue

[issue17364] Multiprocessing documentation mentions function that doesn't exist

2013-03-05 Thread Richard Oudkerk
Changes by Richard Oudkerk shibt...@gmail.com: -- nosy: +sbt ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17364 ___ ___ Python-bugs-list mailing

[issue17367] subprocess deadlock when read() is interrupted

2013-03-06 Thread Richard Oudkerk
Richard Oudkerk added the comment: The change in your patch is in a Windows-only section -- a few lines before the chunk you can see _winapi.GetExitCodeProcess(). Since read() on Windows never fails with EINTR there is no need for _eintr_retry_call(). If you are using Linux then there must

[issue17367] subprocess deadlock when read() is interrupted

2013-03-06 Thread Richard Oudkerk
Richard Oudkerk added the comment: BTW, on threads are only used on Windows. On Unix select() or poll() is used. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17367

[issue17367] subprocess deadlock when read() is interrupted

2013-03-06 Thread Richard Oudkerk
Richard Oudkerk added the comment: I will close the issue then. If you track the problem down to a bug in Python then you can open a new one. -- resolution: - invalid stage: - committed/rejected status: open - closed ___ Python tracker rep

[issue16895] Batch file to mimic 'make' on Windows

2013-03-10 Thread Richard Oudkerk
Richard Oudkerk added the comment: +1 To use Tools/builbot/*.bat doesn't the current directory have to be the main directory of the repository? Then I see no point in the -C argument: just set the correct directory automatically. I think make.bat should also support creation of non-debug

[issue16895] Batch file to mimic 'make' on Windows

2013-03-10 Thread Richard Oudkerk
Richard Oudkerk added the comment: What does running 'kill-python before re-building python do? I have not seen it mentioned in the in the devguide or pcbuild/readme. It kills any currently running python(_d).exe processes. This is because Windows does not allow program or library files

[issue17395] Wait for live children in test_multiprocessing

2013-03-11 Thread Richard Oudkerk
Richard Oudkerk added the comment: LGTM (although the warning is actually harmless). -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17395

[issue16389] re._compiled_typed's lru_cache causes significant degradation of the mako_v2 bench

2013-03-11 Thread Richard Oudkerk
Richard Oudkerk added the comment: Which does give me a thought - perhaps lru_cache in 3.4 could accept a key argument that is called as key(*args, **kwds) to derive the cache key? (that would be a separate issue, of course) Agreed. I suggested the same in an earlier post

[issue17395] Wait for live children in test_multiprocessing

2013-03-11 Thread Richard Oudkerk
Richard Oudkerk added the comment: Why 1? This should be commented. The manager process will always be included in active_children(). -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17395

[issue17399] test_multiprocessing hang on Windows, non-sockets

2013-03-12 Thread Richard Oudkerk
Richard Oudkerk added the comment: Does this happen every time you run the tests? (I don't see these errors.) -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17399

[issue17399] test_multiprocessing hang on Windows, non-sockets

2013-03-12 Thread Richard Oudkerk
Richard Oudkerk added the comment: Could you try the following program: import socket import multiprocessing import multiprocessing.reduction import multiprocessing.connection def socketpair(): with socket.socket() as l: l.bind(('localhost', 0)) l.listen(1) s

[issue17399] test_multiprocessing hang on Windows, non-sockets

2013-03-12 Thread Richard Oudkerk
Richard Oudkerk added the comment: Now could you try the attached file? (It will not work on 2.7 because a missing socket.fromfd().) P.S. It looks like the error for 3.3 is associated with a file f:\python\mypy\traceback.py which presumably clashes with the one in the standard library

[issue17399] test_multiprocessing hang on Windows, non-sockets

2013-03-12 Thread Richard Oudkerk
Richard Oudkerk added the comment: Both 3.2 and 3.3 give essentially the same traceback as 3.2 did before, both with installed python and yesterdays debug builds. It looks like on your machine socket handles are not correctly inherited by child processes -- I had assumed that they always

  1   2   3   4   5   6   7   8   >