Changes by Richard Oudkerk :
--
assignee: -> docs@python
components: +Documentation -IO, Interpreter Core
nosy: +docs@python
resolution: -> fixed
stage: -> committed/rejected
status: open -> closed
type: behavior ->
___
Python
Richard Oudkerk added the comment:
> 1. "but should not cause any pratical difficulties" <-- you have a typo in
> 'pratical' there.
> 2. What exactly do you mean by "managed" queues in the new addition?
Woops. Fixed now see 860fc6a2bd21, 347647a1f
Richard Oudkerk added the comment:
I was thinking about the line
self.__dict__.update(state)
overwriting new data with stale data.
--
___
Python tracker
<http://bugs.python.org/issue17
Changes by Richard Oudkerk :
--
resolution: -> fixed
stage: -> committed/rejected
status: open -> closed
___
Python tracker
<http://bugs.python.or
Richard Oudkerk added the comment:
unfinished_tasks is simply used as a counter. It is only accessed while
holding self._cond. If you get this error then I think the error text is
correct -- your progam calls task_done() to many times.
The proposed patch silences the sanity check by making
Richard Oudkerk added the comment:
> I did this to use the same abstraction that was used extensively for
> other purposes, instead of recreating the same abstraction with a deque
> as its basis.
So you wanted a FIFO queue and preferred the API of Queue to that of deque?
Well i
Richard Oudkerk added the comment:
Shouldn't the import lock be held to make it threadsafe?
--
___
Python tracker
<http://bugs.python.org/issue17621>
___
___
Richard Oudkerk added the comment:
Apologies for being dense, but how would you actually use such a loader?
Would you need to install something in sys.meta_path/sys.path_hooks? Would it
make all imports lazy or only imports of specified modules?
--
nosy: +sbt
Richard Oudkerk added the comment:
Why would you use a multi-process queue to "pass messages from one part of the
program to another part, in the same process and thread"? Why not just use a
deque?
Is this something you actually did, or are you just trying to come up with a
Richard Oudkerk added the comment:
This is a very similar issue to #17985.
While it may seem counter-intuitive, I don't see how it makes any difference.
Another thread/process might remove the item before you can get it.
I find it very difficult to imagine a real program where you can s
Richard Milne added the comment:
I already see my name change will cause problems. The tarfile.py
(Lib/tarfile.py:277) module relies on the function being named 'filemode'.
As the stat.filemode function is relatively new, I don't know how many other
modules rely on it. I c
Richard Milne added the comment:
I've added some changes to the module which (I believe) not only fix this bug,
but add the reverse function, converting a permission string to a number.
The diff was made against Lib/stat.py @ 84234:f32dad3a243e, which was the most
recent version of the f
Changes by Richard Oudkerk :
--
resolution: -> works for me
stage: -> committed/rejected
status: open -> closed
___
Python tracker
<http://bugs.python.or
Richard Oudkerk added the comment:
See also #9573 and #15914.
--
___
Python tracker
<http://bugs.python.org/issue18122>
___
___
Python-bugs-list mailing list
Unsub
Changes by Richard Oudkerk :
--
resolution: -> fixed
stage: -> committed/rejected
status: open -> closed
___
Python tracker
<http://bugs.python.or
Richard Oudkerk added the comment:
I just tried freezing the program
from multiprocessing import freeze_support,Manager
if __name__ == '__main__':
freeze_support()
m=Manager()
l = m.list([1,2,3])
l.append(4)
print(l)
print(repr(l))
using cx_F
Richard Oudkerk added the comment:
We don't do non-security updates on Python 2.6 anymore.
As a workaround you might be able to do something like
import sys, multiprocessing
sys.frozen = True# or multiprocessing.forking.WINEXE = True
...
if __name__ ==
Changes by Richard Oudkerk :
--
keywords: +gsoc -patch
resolution: -> rejected
stage: -> committed/rejected
status: open -> closed
___
Python tracker
<http://bugs.python.or
Changes by Richard Oudkerk :
--
stage: -> committed/rejected
___
Python tracker
<http://bugs.python.org/issue18252>
___
___
Python-bugs-list mailing list
Un
Changes by Richard Oudkerk :
--
stage: committed/rejected ->
___
Python tracker
<http://bugs.python.org/issue18252>
___
___
Python-bugs-list mailing list
Un
Richard Oudkerk added the comment:
I think if you use timeit then the code is wrapped inside a function before it
is compiled. This means that your code can mostly use faster local lookups
rather than global lookups.
--
nosy: +sbt
___
Python
Richard Oudkerk added the comment:
On 15/06/2013 7:11pm, Antoine Pitrou wrote:
>> Usually garbage collection will end up clearing the module's dict anyway.
>
> This is not true, since global objects might have a __del__ and then hold
> the whole module dict alive throu
Changes by Richard Oudkerk :
--
nosy: +pitrou
___
Python tracker
<http://bugs.python.org/issue18214>
___
___
Python-bugs-list mailing list
Unsubscribe:
Richard Oudkerk added the comment:
Do you want something like
f.done() and not f.cancelled() and f.exception() is None
--
nosy: +sbt
___
Python tracker
<http://bugs.python.org/issue18
New submission from Richard Oudkerk:
Currently when a module is garbage collected its dict is purged by replacing
all values except __builtins__ by None. This helps clear things at shutdown.
But this can cause problems if it occurs *before* shutdown: if we use a
function defined in a module
Changes by Richard Oudkerk :
--
nosy: +sbt
___
Python tracker
<http://bugs.python.org/issue9122>
___
___
Python-bugs-list mailing list
Unsubscribe:
Richard Oudkerk added the comment:
Updated version which adds checks for handle leaks on Windows.
--
Added file: http://bugs.python.org/file30561/fdleak.patch
___
Python tracker
<http://bugs.python.org/issue18
Richard Oudkerk added the comment:
The test_shutil leak is caused by #17899. The others are fixed by
a7381fe515e8 and 46fe1bb0723c.
--
___
Python tracker
<http://bugs.python.org/issue18
Changes by Richard Oudkerk :
--
resolution: -> fixed
stage: -> committed/rejected
status: open -> closed
type: -> resource usage
versions: +Python 3.3, Python 3.4
___
Python tracker
<http://bugs.python
New submission from Richard Oudkerk:
Seems to be in error path of _PyImport_GetDynLoadWindows().
--
files: load_dynamic.patch
keywords: patch
messages: 190901
nosy: sbt
priority: normal
severity: normal
status: open
title: Refleak in test_imp on Windows
Added file: http
New submission from Richard Oudkerk:
If os.listdir() is used with an fd, but fdopendir() fails (e.g. if the the fd
is a normal file) then a duplicated fd is leaked.
This explains the leaks in test_shutil mentioned in #18174.
--
messages: 190875
nosy: sbt
priority: normal
severity
New submission from Richard Oudkerk:
regrtest already tests for refcount leaks and memory allocation leaks. It can
also be made to check for file descriptor leaks (and perhaps also handles on
Windows).
Running with the attached patch makes it look like test_openpty, test_shutil
Richard Oudkerk added the comment:
> PJE suggests importing atexit and registering finalize only when it's
> actually used. I guess this would be the easiest workaround.
Done.
--
status: open -> closed
___
Python tracker
<http:
Changes by Richard Oudkerk :
--
status: open -> closed
___
Python tracker
<http://bugs.python.org/issue17931>
___
___
Python-bugs-list mailing list
Unsubscri
Richard Oudkerk added the comment:
Attached is a patch that adds _Py_PARSE_INTPTR and _Py_PARSE_UINTPTR to
Include/longobject.h.
It also uses _Py_PARSE_INTPTR in Modules/posixmodule.c and PC/msvcrtmodule.c
and removes the definition for SIZEOF_PID.
--
Added file: http
Richard Oudkerk added the comment:
I see _Py_PARSE_PID already exists but no others ...
--
___
Python tracker
<http://bugs.python.org/issue17931>
___
___
Pytho
Richard Oudkerk added the comment:
> @sbt: Would you like to have a strict separation between UNIX-like pid
> (pid_t) and Windows process identifier (HANDLE)?
Yes. And would I certainly like SIZEOF_PID_T == sizeof(pid_t) ;-)
Note that _winapi takes the policy of treating HANDLE
Richard Oudkerk added the comment:
> pid_t is HANDLE on Windows, which is a pointer.
I think this is wrong.
The signature of getpid() is
int _getpid(void);
so pid_t should be equivalent to int.
The complication is that the return values of spawn*() etc are process handles
(cast
Richard Oudkerk added the comment:
> Furthermore, the complexity is rather bad: if T is the average number
> of waiting threads, an C the number of conditions being waited on, the
> wait is O(C) (appending to C wait queues) and wakeup is O(CT) (C
> removal from a T-length deque).
Richard Oudkerk added the comment:
Actually, you can use gc.get_referents(obj) which returns the direct children
of obj (and is presumably implemented using tp_traverse).
I will close.
--
resolution: -> rejected
stage: -> committed/rejected
status: open -&g
Richard Oudkerk added the comment:
On 03/06/2013 3:07pm, spresse1 wrote:
> I could reimplement the close_all_fds_except() call (in straight python, using
> os.closerange()). That seems like a reasonable solution, if a bit of a hack.
> However, given that pipes are exposed by multiproce
Richard Oudkerk added the comment:
Forking as a side effect of importing a module is evil. I think raising a
RuntimeError is preferable to trying to make it Just Work.
But maybe one could do
void
_PyImport_ReInitLock(void)
{
if (import_lock != NULL
Richard Oudkerk added the comment:
On 03/06/2013 1:02am, spresse1 wrote:
> Whats really bugging me is that it remains open and I can't fetch a reference.
> If I could do either of these, I'd be happy.
> ...
> Perhaps I really want to be implementing with os.fork(). Si
Richard Oudkerk added the comment:
> What I'm still trying to grasp is why Python explicitly leaves the
> parent processes info around in the child. It seems like there is
> no benefit (besides, perhaps, speed) and that this choice leads to
> non-intuitive behavior - like t
Richard Oudkerk added the comment:
Presumably this is caused by the fact that Popen.__del__() ressurects self by
appending self to _active if the process is still alive.
On Windows this is unnecessary. On Unix it would be more sensible to just
append the *pid* to _active.
--
nosy
Richard Oudkerk added the comment:
> So you're telling me that when I spawn a new child process, I have to
> deal with the entirety of my parent process's memory staying around
> forever?
With a copy-on-write implementation of fork() this quite likely to use less
memory th
Richard Oudkerk added the comment:
> The write end of that pipe goes out of scope and has no references in the
> child thread. Therefore, per my understanding, it should be garbage
> collected (in the child thread). Where am I wrong about this?
The function which starts the child p
Richard Oudkerk added the comment:
The way to deal with this is to pass the write end of the pipe to the child
process so that the child process can explicitly close it -- there is no reason
to expect garbage collection to make this happen automatically.
You don't explain the diffe
Richard Oudkerk added the comment:
I am not to familiar with the signal handling machinery. (I only did
some refactoring to expose the event handle already used by time.sleep().)
The change looks reasonable, but I am also not sure how necessary it is
Richard Oudkerk added the comment:
> Can't this just be a Python script?
That would cause bootstrap issues for people who do not already have
python installed.
--
___
Python tracker
<http://bugs.python.org
Richard Oudkerk added the comment:
I can't say I know enough about batch files to understand much of the code, but
a few notes:
Windows XP does not have the command "where" which you use -- Python 3.4 will
still support XP.
Except perhaps for looping I would prefer to get ri
Richard Oudkerk added the comment:
> You cannot associate several conditions to the *inner lock*, because you
> don't have access to them (otherwise I wouldn't open this issue).
Condition.wait_for_any() would create a single inner lock and add it to the
_waiters list for each
Richard Oudkerk added the comment:
> This could solve the waiting problem for the "thread", but also may
> keep the other Condition objs waiting -- and that may not be problem
> because I'm already using .notify_all()
I don't understand what you mean.
> Prob
Richard Oudkerk added the comment:
> BTW, I find .notify(N) not much useful, because the docs say it may wake
> more threads on a different implementation and also I can't never know
> whom am I waking.
The fact that more than N threads can wake up is not a problem if you are
Richard Oudkerk added the comment:
The unit tests pass with the patch already (if we don't delete the "import imp"
line).
What attributes will be set by init_module_attrs()?
--
___
Python tracker
<http://bugs.pyt
Richard Oudkerk added the comment:
Looks good to me.
(Any particular reason for ignoring AttributeError?)
--
___
Python tracker
<http://bugs.python.org/issue17
Richard Oudkerk added the comment:
I understand that a thread pool (in the general sense) might be used to
amortise the cost. But I think you would probably have to write this from
scratch rather than use the ThreadPool API.
The ThreadPool API does not really expose anything that the
Changes by Richard Oudkerk :
--
nosy: +sbt
___
Python tracker
<http://bugs.python.org/issue6461>
___
___
Python-bugs-list mailing list
Unsubscribe:
Richard Oudkerk added the comment:
> It's a very different architecture from that assumed by futures,
> so you need to drop down to the pool layer rather than using the
> executor model.
AIUI an ThreadPoolExecutor object (which must be explicitly created)
represents a thread/pro
Richard Oudkerk added the comment:
As far as I can see they are mostly equivalent. For instance, ApplyResult (the
type returned by Pool.apply_async()) is virtually the same as a Future.
When you say "explicit message passing", do you mean creating a queue and
making the worker
Richard Oudkerk added the comment:
I don't understand what you mean by "explicit message passing" and
"call-and-response model".
--
___
Python tracker
<http:
Richard Oudkerk added the comment:
Given that the change could only be made to 3.4, and we already have
concurrent.futures.ThreadPoolExecutor, I am not sure there is much point to
such a change now.
--
nosy: +sbt
___
Python tracker
<h
Changes by Richard Oudkerk :
--
resolution: -> fixed
stage: patch review -> committed/rejected
status: open -> closed
___
Python tracker
<http://bugs.python.or
Changes by Richard Oudkerk :
Removed file: http://bugs.python.org/file30293/readall.patch
___
Python tracker
<http://bugs.python.org/issue15758>
___
___
Python-bugs-list m
Changes by Richard Oudkerk :
Added file: http://bugs.python.org/file30295/readall.patch
___
Python tracker
<http://bugs.python.org/issue15758>
___
___
Python-bugs-list m
Richard Oudkerk added the comment:
New patch.
--
___
Python tracker
<http://bugs.python.org/issue15758>
___
___
Python-bugs-list mailing list
Unsubscribe:
Changes by Richard Oudkerk :
Removed file: http://bugs.python.org/file30291/readall.patch
___
Python tracker
<http://bugs.python.org/issue15758>
___
___
Python-bugs-list m
Changes by Richard Oudkerk :
Added file: http://bugs.python.org/file30293/readall.patch
___
Python tracker
<http://bugs.python.org/issue15758>
___
___
Python-bugs-list m
Changes by Richard Oudkerk :
Removed file: http://bugs.python.org/file30287/readall.patch
___
Python tracker
<http://bugs.python.org/issue15758>
___
___
Python-bugs-list m
Changes by Richard Oudkerk :
Added file: http://bugs.python.org/file26986/readall-benchmark.py
___
Python tracker
<http://bugs.python.org/issue15758>
___
___
Python-bug
Changes by Richard Oudkerk :
Removed file: http://bugs.python.org/file26985/readall-combined.patch
___
Python tracker
<http://bugs.python.org/issue15758>
___
___
Pytho
Richard Oudkerk added the comment:
Updated patch adressing Antoine's comments.
--
Added file: http://bugs.python.org/file30291/readall.patch
___
Python tracker
<http://bugs.python.org/is
Richard Oudkerk added the comment:
Rather than
self.sock = None
I would do
self.sock.close()
which should work better for non-refcounted Pythons.
Of course it would be better to do this immediately after forking (i.e. before
any more fds are created), otherwise you could still
Richard Oudkerk added the comment:
The line
sock = socket.socket(socket.AF_UNIX, socket.SOCK_DGRAM)
overwrites the old broken socket with a new one with the same fd. The old
socket's destructor closes the fd of the new socket.
--
nosy:
Changes by Richard Oudkerk :
Removed file: http://bugs.python.org/file26986/readall-benchmark.py
___
Python tracker
<http://bugs.python.org/issue15758>
___
___
Python-bug
Richard Oudkerk added the comment:
I have done an updated patch. It no longer special cases Windows, so realloc()
is always used for enlarging the buffer (except when fstat() is missing).
Antoine, do you think this is ready to commit?
--
Added file: http://bugs.python.org/file30287
Richard Oudkerk added the comment:
On 15/05/2013 11:33pm, Andre Dias wrote:
> But the example program has no races, no threads, nothing.
> empty() is returning TRUE even though qsize() is >0 (which actually is)
> And it happens almost every time I run that small example.
> I had
Richard Oudkerk added the comment:
On 15/05/2013 10:25pm, Andre Dias wrote:
> But qsize() is working. what is not working is empty()
empty() returns False when there is data in the underlying pipe. But
the data does not enter the pipe until a background thread has written
it to the p
Richard Oudkerk added the comment:
>From the docs:
qsize()
Return the approximate size of the queue. Because of
multithreading/multiprocessing semantics, this number
is not reliable.
Adding a short sleep before calling qsize() and empty() should make things
appear to work.
New submission from Richard Oudkerk:
In the attached file is an experimental implementation of an AsyncPopen
class. It should work for Python 3.3, 3.4 on Unix and Windows. Unlike
http://code.google.com/p/subprocdev
(see #1191964) this does not depend on using time.sleep() and polling.
It
Richard Oudkerk added the comment:
Kristjan, could you confirm whether joining the pool explicitly before shutdown
(in the way I suggested earlier) fixes the problem. I think it should -- at
shutdown you won't switch to a thread if it has already been j
Richard Oudkerk added the comment:
When pickling a class (or instance of a class) there is already a check
that the invariant
getattr(sys.modules[cls.__module__], cls.__name__) == cls
holds.
>>> import pickle
>>> class A: pass
...
>>> A.__module__ = '
Richard Oudkerk added the comment:
> We can consider two options then:
> 1) A multiprocessing specific fix. Removing this handle close gil
> release (which is superfluous, since these calls aren't blocking in any
> real sense) will certainly remove _this_ instance
Richard Oudkerk added the comment:
> In general, deallocators should have no side effects, I think.
> Releaseing the GIL is certainly a side effect.
Notice that socket and file objects also release the GIL when being
deallocated. At least for sockets close() can block (e.g. if you y
Changes by Richard Oudkerk :
--
nosy: +sbt
___
Python tracker
<http://bugs.python.org/issue17969>
___
___
Python-bugs-list mailing list
Unsubscribe:
Richard Oudkerk added the comment:
> If the name is a qualified dotted name, it will be split and the first
> part becomes the __module__.
That will not work correctly if the module name has a dot in it.
--
nosy: +sbt
___
Python tracker
Richard Oudkerk added the comment:
On Windows my encoding for stdout, stderr is "cp1252" which is implemented in
pure python.
By the time that _PyGC_DumpShutdownStats() runs the encoding.cp1252 module has
been purged so stdout and stderr are broken.
I am afraid I will have to lea
Richard Oudkerk added the comment:
I will try a fix.
--
___
Python tracker
<http://bugs.python.org/issue1545463>
___
___
Python-bugs-list mailing list
Unsubscribe:
Richard Oudkerk added the comment:
I think the problem is that the __del__ method fails on Windows, maybe because
sys.stdout and sys.__stderr__ have been replaced by None.
Consider the following program:
import os
class C:
def __del__(self, write=os.write):
write(1, b
Richard Oudkerk added the comment:
The test seems to be failing on Windows.
--
nosy: +sbt
status: closed -> open
___
Python tracker
<http://bugs.python.org/issue1
Richard Oudkerk added the comment:
The relevant changeset was c4f92b597074, but I wrote the wrong issue number in
the commit message and Misc/NEWS.
--
___
Python tracker
<http://bugs.python.org/issue13
Changes by Richard Oudkerk :
--
resolution: -> fixed
stage: -> committed/rejected
status: open -> closed
___
Python tracker
<http://bugs.python.or
Changes by Richard Oudkerk :
--
resolution: -> fixed
stage: -> committed/rejected
status: open -> closed
___
Python tracker
<http://bugs.python.or
Changes by Richard Oudkerk :
--
resolution: -> fixed
stage: patch review -> committed/rejected
status: open -> closed
___
Python tracker
<http://bugs.python.or
Richard Oudkerk added the comment:
I don't see any difference in meaning:
http://idioms.thefreedictionary.com/as+far+as+possible
--
nosy: +sbt
___
Python tracker
<http://bugs.python.org/is
Richard Oudkerk added the comment:
The only (non-doc, non-comment) changes were the two one-liners you suggested
in msg172077. So I will commit.
--
___
Python tracker
<http://bugs.python.org/issue15
Richard Oudkerk added the comment:
Here is an updated patch. It is only really the example in the docs which is
different, plus a note about daemon threads.
Antoine, do think this is ready to be committed?
--
Added file: http://bugs.python.org/file30140/finalize.patch
Richard Oudkerk added the comment:
Attached is a patch for 3.4 which uses the __cause__ hack to embed the remote
traceback in the local traceback. It will not work for 2.x though.
>>> import multiprocessing, subprocess
>>> with multiprocessing.Pool() as p: p.apply(su
Richard Oudkerk added the comment:
It might be simplest to make the implementation match the docs by making
AsyncResult an alias for ApplyResult.
--
___
Python tracker
<http://bugs.python.org/issue17
Richard Oudkerk added the comment:
> Ah. Then, a documentation error. The error message ("queue.Full"?) and
> the documentation are totally not clear about that.
Once something goes wrong you are likely to get a cascade of errors, and the
first one reported is not necessar
401 - 500 of 1064 matches
Mail list logo
