On May 12, 11:50 pm, Dennis Lee Bieber wrote:
> On Mon, 10 May 2010 10:40:51 +0800, "ÖÓ±þÓÂ"
> declaimed the following in gmane.comp.python.general:
>
> > I have a multi-thread program work with Queue.Queue(), sometimes put
> > request to the work queue,
I have a multi-thread program work with Queue.Queue(), sometimes put request to
the work queue, but throw an exception as below traceback information, it will
always throw the exception until restart program, cound please have any
experience, your help will be greatly appreciated!
File "
On Feb 8, 4:51 pm, Steven wrote:
Queue objects have support for this signaling baked in with
q.task_done and q.join.
After the server process has put all tasks into the queue, it can join
the queue itself, not the worker threads.
q.join()
This will block until all tasks have been gotten AND
On Feb 5, 7:45 am, "Frank Millman" wrote:
> Hi all
>
> Assume you have a server process running, a pool of worker threads to
> perform tasks, and aQueue.Queue() to pass the tasks to the workers.
>
> In order to shut down the server cleanly, you want to ensure that the
> workers have all finished t
On Feb 6, 7:59 am, "Gabriel Genellina" wrote:
En Fri, 05 Feb 2010 09:45:30 -0300, Frank Millman
escribió:
[...]
> However, the queue is not empty - it still has the final None in it.
Yes - but who cares? :)
That was my point. I didn't think I needed to care, but I wanted to be sure
I w
En Fri, 05 Feb 2010 09:45:30 -0300, Frank Millman
escribió:
Assume you have a server process running, a pool of worker threads to
perform tasks, and a Queue.Queue() to pass the tasks to the workers.
In order to shut down the server cleanly, you want to ensure that the
workers have all
Hi all
Assume you have a server process running, a pool of worker threads to
perform tasks, and a Queue.Queue() to pass the tasks to the workers.
In order to shut down the server cleanly, you want to ensure that the
workers have all finished their tasks. I like the technique of putting a
I would try something like this inside _process_outgoing:
while not self.stop_requested.isSet():
data_ok = False
while not self.stop_requested.isSet():
if not self.outgoing.empty():
try:
On 15 Gen, 10:22, Laszlo Nagy wrote:
> > then the speed goes up to 64 messages/sec on windows and 500
> > messages/sec on Linux.
>
> Finally I could reach 1500 messages/sec without using the queue. If I
> comment out one line (use the queue instead of direct write into socket)
> then speed decreas
then the speed goes up to 64 messages/sec on windows and 500
messages/sec on Linux.
Finally I could reach 1500 messages/sec without using the queue. If I
comment out one line (use the queue instead of direct write into socket)
then speed decreases to 40-60 messages/sec. I don't understand wh
I had this test program originally that sent message over TCP/IP.
Messages where buffered two a Queue.Queue instances, one for incoming
and one for outgoing.
#1. endpoint.send_message(msg) -> endpoint.outgoing.put(msg)
#2. endpoint._process_outgoing() is a thread, that d
i have the following custom extenstion of Queue.Queue() to save and load queue contents but I think there's a problem with it. Does anybody know qhether Queue.Queue() is pickle-able? if so, can I get sample code? If not, can anybody recommend a pickle-able Queue from another library t
[john peter]
> what happens behind the scenes when i create a Queue.Queue() without
> specifying a maxsize? does a block of space gets allocated initially then
> dynamically "expanded" as needed?
Yes.
> if so, what is the default size of the initial space?
It'
what happens behind the scene when i create a Queue.Queue() without specifyinga maxsize? does a block of space gets allocated initially then dynamically "expanded" as needed? if so, what is the default size of the initial space? is italways better to specify a big enou
what happens behind the scenes when i create a Queue.Queue() without specifying a maxsize? does a block of space gets allocated initially then dynamically "expanded" as needed? if so, what is the default size of the initial space? is it always better to specify a big enou
Op 2005-04-02, Paul Rubin schreef :
> Have you looked at this? A paper about adding asynchronous exceptions
> to Python.
>
> http://www.cs.williams.edu/~freund/papers/02-lwl2.ps
Looks interresting, but I doubt python will have it in the near
future. I'm very pessimitic about python development in
[EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote:
> Thinking about cross-platform issues. I found this, from the venerable
> Tim Peters to be enlightening for python's choice of design:
>
> "It's possible to build a better Queue implementation that runs only on
> POSIX systems, or only on Windows s
Paul Rubin wrote:
> Nick Craig-Wood <[EMAIL PROTECTED]> writes:
> > I believe futex is the thing you want for a modern linux. Not
> > very portable though.
>
> That's really cool, but I don't see how it can be a pure userspace
> operation if the futex has a timeout. The kernel must need to k
Nick Craig-Wood <[EMAIL PROTECTED]> writes:
> I believe futex is the thing you want for a modern linux. Not
> very portable though.
That's really cool, but I don't see how it can be a pure userspace
operation if the futex has a timeout. The kernel must need to keep
track of the timeouts. Howeve
Paul Rubin wrote:
> Antoon Pardon <[EMAIL PROTECTED]> writes:
> > I'm not sure that this would be an acceptable approach. I did the man
> > semop and it indicates this is part of system V IPC. This makes me
> > fear that semaphores will use file descriptors or other resources
> > that are only av
Have you looked at this? A paper about adding asynchronous exceptions
to Python.
http://www.cs.williams.edu/~freund/papers/02-lwl2.ps
--
http://mail.python.org/mailman/listinfo/python-list
"Paul L. Du Bois" <[EMAIL PROTECTED]> writes:
> Has anyone written a Queue.Queue replacement that avoids busy-waiting?
> It doesn't matter if it uses os-specific APIs (eg
> WaitForMultipleObjects). I did some googling around and haven't found
> anyt
Op 2005-03-31, [EMAIL PROTECTED] schreef <[EMAIL PROTECTED]>:
> Cool Code!
>
> One possible sticking point is that I think select only works on
> network sockets on windows. This would make the code not crossplatforn.
As far as I understand, what I did with pipes, can be done just as
fine with ne
Thinking about cross-platform issues. I found this, from the venerable
Tim Peters to be enlightening for python's choice of design:
"It's possible to build a better Queue implementation that runs only on
POSIX systems, or only on Windows systems, or only on one of a dozen
other less-popular target
Cool Code!
One possible sticking point is that I think select only works on
network sockets on windows. This would make the code not crossplatforn.
john
--
http://mail.python.org/mailman/listinfo/python-list
Antoon Pardon <[EMAIL PROTECTED]> writes:
> I'm not sure that this would be an acceptable approach. I did the man
> semop and it indicates this is part of system V IPC. This makes me
> fear that semaphores will use file descriptors or other resources
> that are only available in a limited amount. N
Op 2005-03-30, Paul Rubin schreef :
>> > I think the best bet for the short term is handling it at the C level,
>> > with sigalarm. Another way is to have chained sigalarm handlers in
>> > the main thread.
>>
>> Possible, but I don't have the time to investigate that possibility now.
>
> Actuall
Antoon Pardon <[EMAIL PROTECTED]> writes:
> > There needs to be a way to send signals to threads, or raise
> > asynchronous exceptions in them. There's been some discussion in
> > sourceforge about that, but the issues involved are complex.
>
> Well I have raised this issue before and as far as I
Op 2005-03-29, Paul Rubin schreef :
> Antoon Pardon <[EMAIL PROTECTED]> writes:
>> I'm not going to call my solution simple, but it wastes very few
>> cycles. if no thread is blocked on a lock, the select will just
>> block until that changes. No need for some kind of polling loop.
>
> I think I un
Antoon Pardon <[EMAIL PROTECTED]> writes:
> I'm not going to call my solution simple, but it wastes very few
> cycles. if no thread is blocked on a lock, the select will just
> block until that changes. No need for some kind of polling loop.
I think I understand. My original idea was to use a hea
Op 2005-03-29, Antoon Pardon schreef <[EMAIL PROTECTED]>:
> Op 2005-03-29, Paul Rubin schreef :
>> Antoon Pardon <[EMAIL PROTECTED]> writes:
>>> Well have a look at what I have written over the weekend. It uses
>>> a seperate thread with one pipe for a wakeup mechanisme.
>>
>> Thanks, I'll look at
Op 2005-03-29, Paul Rubin schreef :
> Antoon Pardon <[EMAIL PROTECTED]> writes:
>> Well have a look at what I have written over the weekend. It uses
>> a seperate thread with one pipe for a wakeup mechanisme.
>
> Thanks, I'll look at it. Why don't you use usleep instead of a pipe?
Because with th
Antoon Pardon <[EMAIL PROTECTED]> writes:
> Well have a look at what I have written over the weekend. It uses
> a seperate thread with one pipe for a wakeup mechanisme.
Thanks, I'll look at it. Why don't you use usleep instead of a pipe?
I decided over the weekend that using a separate thread wit
Op 2005-03-25, Paul Rubin schreef :
> Antoon Pardon <[EMAIL PROTECTED]> writes:
>> Well maybe you could use an os.pipe as a timeout lock then. When the lock is
>> instantiated you put one byte in it. Aquiring the lock is implemented by
>> reading one byte, releasing the lock is implemented by writi
Antoon Pardon <[EMAIL PROTECTED]> writes:
> Well maybe you could use an os.pipe as a timeout lock then. When the lock is
> instantiated you put one byte in it. Aquiring the lock is implemented by
> reading one byte, releasing the lock is implemented by writing a byte.
> Aquiring the lock with a tim
Op 2005-03-25, Paul Rubin schreef :
> Antoon Pardon <[EMAIL PROTECTED]> writes:
>> > I meant a semaphore to synchronize the queue when adding or removing
>> > objects.
>>
>> Last I looked there was a lock used for that.
>
> OK, that amounts to the same thing.
>
>> The loop is only for when you c
Antoon Pardon <[EMAIL PROTECTED]> writes:
> > I meant a semaphore to synchronize the queue when adding or removing
> > objects.
>
> Last I looked there was a lock used for that.
OK, that amounts to the same thing.
> The loop is only for when you cant remove or add an element immediatly
> and th
Op 2005-03-25, Paul Rubin schreef :
> Antoon Pardon <[EMAIL PROTECTED]> writes:
>> > I've never checked this code but it wouldn't have occurred to me that
>> > Queue uses any kind of timeout loop. Can't it work the obvious way
>> > with a semaphore?
>>
>> And how is this semaphore going to be rel
Antoon Pardon <[EMAIL PROTECTED]> writes:
> > I've never checked this code but it wouldn't have occurred to me that
> > Queue uses any kind of timeout loop. Can't it work the obvious way
> > with a semaphore?
>
> And how is this semaphore going to be released if the timeout is
> reached?
I meant
intermediate
>> cancel, as the loop that is used in Queue.Queue, so that was no
>> gain.
>
> I've never checked this code but it wouldn't have occurred to me that
> Queue uses any kind of timeout loop. Can't it work the obvious way
> with a semaphore?
And ho
Antoon Pardon <[EMAIL PROTECTED]> writes:
> I started once, using the Timer class in the Threading Module to
> break the lock. However the Timer class uses the same kind of
> sleep-polling loop, to delay the exection and allow an intermediate
> cancel, as the loop that is used
Op 2005-03-24, Paul L. Du Bois schreef <[EMAIL PROTECTED]>:
> Has anyone written a Queue.Queue replacement that avoids busy-waiting?
> It doesn't matter if it uses os-specific APIs (eg
> WaitForMultipleObjects). I did some googling around and haven't found
> anyt
Peter Hansen wrote:
> I don't believe the term "busy-wait" applies here.
> [Explanation]
Yes, well, you're right. I was thinking of calling it
"slacker-waiting" but didn't want to come off too cute.
p
--
http://mail.python.org/mailman/listinfo/python-list
Paul L. Du Bois wrote:
Has anyone written a Queue.Queue replacement that avoids busy-waiting?
It doesn't matter if it uses os-specific APIs (eg
WaitForMultipleObjects). I did some googling around and haven't found
anything so far.
Because I know someone will ask: no, the busy-waiting h
Has anyone written a Queue.Queue replacement that avoids busy-waiting?
It doesn't matter if it uses os-specific APIs (eg
WaitForMultipleObjects). I did some googling around and haven't found
anything so far.
Because I know someone will ask: no, the busy-waiting hasn't been a
pr
45 matches
Mail list logo
