Thanks, Yuval, for yor code. I will analyze that and adapt to my case. I
think want of the main lacks of python is the documentation. There are,
in general, few examples. Just a newbee opinion ;-)
Thanks,
Xan.
Al 05/01/13 21:43, En/na Yuval Greenfield ha escrit:
Googled a nice example at
http://www.blog.pythonlibrary.org/2012/08/01/python-concurrency-an-example-of-a-queue/ though
it's python 2.x
import os
import Queue
import threading
import urllib2
########################################################################
class Downloader(threading.Thread):
"""Threaded File Downloader"""
#----------------------------------------------------------------------
def __init__(self, queue):
threading.Thread.__init__(self)
self.queue = queue
#----------------------------------------------------------------------
def run(self):
while True:
# gets the url from the queue
url =self.queue.get()
# download the file
self.download_file(url)
# send a signal to the queue that the job is done
self.queue.task_done()
#----------------------------------------------------------------------
def download_file(self, url):
""""""
handle =urllib2.urlopen(url)
fname =os.path.basename(url)
withopen(fname,"wb") as f:
while True:
chunk = handle.read(1024)
if not chunk:break
f.write(chunk)
#----------------------------------------------------------------------
def main(urls):
"""
Run the program
"""
queue =Queue.Queue()
# create a thread pool and give them a queue
for iin range(5):
t = Downloader(queue)
t.setDaemon(True)
t.start()
# give the queue some data
for urlin urls:
queue.put(url)
# wait for the queue to finish
queue.join()
if __name__ =="__main__":
urls =["http://www.irs.gov/pub/irs-pdf/f1040.pdf";,
"http://www.irs.gov/pub/irs-pdf/f1040a.pdf";,
"http://www.irs.gov/pub/irs-pdf/f1040ez.pdf";,
"http://www.irs.gov/pub/irs-pdf/f1040es.pdf";,
"http://www.irs.gov/pub/irs-pdf/f1040sb.pdf";]
main(urls)
On Sat, Jan 5, 2013 at 7:37 PM, Guido van Rossum <[email protected]
<mailto:[email protected]>> wrote:
Yes. No (no time). You need to look for tutorials on threading in
Python.
On Sat, Jan 5, 2013 at 9:38 AM, xancorreu <[email protected]
<mailto:[email protected]>> wrote:
> Do you refer on this:
> http://docs.python.org/release/3.0.1/library/queue.html ?
> Can you give me an inspiring code, thanks?
>
> Xan.
>
> Al 05/01/13 17:34, En/na Guido van Rossum ha escrit:
>
>> I recommend that you look at the Queue module. Think in terms of
>> producers and consumers -- when the producer adds an item, it
pushes
>> it onto the queue. There may be multiple producers. The
consumers are
>> several threads that try to get items off the queue and process
them.
>> So your list becomes the queue.
>>
>> (BTW, this list is pretty inactive -- you're better off getting
help
>> on python-list.)
>>
>> On Sat, Jan 5, 2013 at 8:24 AM, xancorreu <[email protected]
<mailto:[email protected]>> wrote:
>>>
>>> Al 04/01/13 22:28, En/na Guido van Rossum ha escrit:
>>>
>>>> Hi Xan,
>>>>
>>>> You have simplified what you are trying to accomplish too
far. Surely
>>>> if it really was about popping from a list you shouldn't be
using a
>>>> thread pool. What is the real use case you are trying to
model here?
>>>>
>>>> --Guido
>>>
>>>
>>> Thanks, Guido, for answering. But why can't use threading for
poping a
>>> list?
>>> My real case is a list of downloaded files that dynamically
grows (as
>>> user
>>> add a file) and also it can lost elements (files which we
downloaded).
>>>
>>> I was thinking to program concurrently, with futures, but
before coding
>>> the
>>> biggest case, I select the essential (ans smaller) one.
>>>
>>> Any specific schema code for doing that?
>>>
>>> Thanks,
>>> Xan.
>>>
>>>
>>>> On Fri, Jan 4, 2013 at 11:59 AM, xancorreu
<[email protected] <mailto:[email protected]>> wrote:
>>>>>
>>>>> Hi,
>>>>>
>>>>> I just want to translate this classic code:
>>>>>
>>>>> ml = [i for i in range(100)]
>>>>>
>>>>> while ml:
>>>>> element = ml.pop()
>>>>> print(element)
>>>>>
>>>>> to futures for running pop asynchronously. I started, but I
don't see
>>>>> how
>>>>> to
>>>>> do that. This [https://gist.github.com/4455376] is the first
attempt,
>>>>> but
>>>>> it
>>>>> fails when I put more elements on ml (that is ml is growing).
>>>>>
>>>>> Any hints?
>>>>>
>>>>> Thanks in advance,
>>>>> Xan.
>>>>> _______________________________________________
>>>>> concurrency-sig mailing list
>>>>> [email protected] <mailto:[email protected]>
>>>>> http://mail.python.org/mailman/listinfo/concurrency-sig
>>>>
>>>>
>>>>
>>> _______________________________________________
>>> concurrency-sig mailing list
>>> [email protected] <mailto:[email protected]>
>>> http://mail.python.org/mailman/listinfo/concurrency-sig
>>
>>
>>
>
> _______________________________________________
> concurrency-sig mailing list
> [email protected] <mailto:[email protected]>
> http://mail.python.org/mailman/listinfo/concurrency-sig
--
--Guido van Rossum (python.org/~guido <http://python.org/%7Eguido>)
_______________________________________________
concurrency-sig mailing list
[email protected] <mailto:[email protected]>
http://mail.python.org/mailman/listinfo/concurrency-sig
_______________________________________________
concurrency-sig mailing list
[email protected]
http://mail.python.org/mailman/listinfo/concurrency-sig
_______________________________________________
concurrency-sig mailing list
[email protected]
http://mail.python.org/mailman/listinfo/concurrency-sig
