Re: is there a concurrent list for append in parallel programming in python? how to pass parameter in this parallel program with pool?

[Ian Kelly]

On Thu, Jun 16, 2016 at 2:45 AM, meInvent bbird  wrote:
> the name in c# is not called concurrent list, it is called
> blockingcollection
>
> dictionary called concurrent dictionary
>
> thread safe these kind of things
>
> https://msdn.microsoft.com/en-us/library/dd267312(v=vs.110).aspx
>
> https://msdn.microsoft.com/en-us/library/dd997369(v=vs.110).aspx
>
> https://msdn.microsoft.com/en-us/library/dd997305(v=vs.110).aspx

It sounds to me like you're looking for the Queue class:
https://docs.python.org/3/library/queue.html

Or possibly you want its multiprocessing equivalent, since it's not
clear to me which kind of concurrency you're interested in:
https://docs.python.org/3/library/multiprocessing.html#multiprocessing.Queue

Also, please don't top-post on this mailing list. Trimming quotations
and interleaving your replies is the accepted posting style here.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: is there a concurrent list for append in parallel programming in python? how to pass parameter in this parallel program with pool?

[meInvent bbird]

how can list be synchronized when multiprocessor working in it?

will one thread updating non-updated version, but another processor updating 
the version?

On Thursday, June 16, 2016 at 4:30:33 PM UTC+8, Steven D'Aprano wrote:
> On Thursday 16 June 2016 17:28, meInvent bbird wrote:
> 
> > is there like c# have concurrent list ?
> 
> What is a concurrent list?
> 
> Can you link to the C# documentation for this?
> 
> To me, "concurrent" describes a style of execution flow, and "list" describes 
> a 
> data structure. I am struggling to understand what "concurrent list" means.
> 
> > i find something these, but how can it pass an initlist list variable
> 
> initlist = ['a', 'b', 'c']
> result = comb(n, initlist)  # pass initlist
> 
> 
> > is it doing the same function as itertools.combinations ?
> 
> It is calling itertools.combinations. So, yes, it is doing the same function 
> as 
> itertools.combinations.
> 
> 
> > def comb(n, initlist): # the argument n is the number of items to select
> > res = list(itertools.combinations(initlist, n)) # create a list from the
> > # iterator 
> > return res
> 
> This does not generate the combinations in parallel. It generates them one at 
> a 
> time, and then creates a list of them.
> 
> This is an interesting question. Somebody could probably write a parallel 
> version of combinations. But I don't know that it would be very useful -- the 
> limiting factor on combinations is more likely to be memory, not time.
> 
> Suppose you generate combinations of 100 items, taken 10 at a time. If I 
> remember the formula for combinations correctly, the total number of 
> combinations is:
> 
> 100!/(10! * 90!) = 17310309456440
> 
> combinations in total. If each generated combination took just *one* byte of 
> memory, that would require over 17 TB of RAM.
> 
> 
> 
> -- 
> Steve

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: is there a concurrent list for append in parallel programming in python? how to pass parameter in this parallel program with pool?

[meInvent bbird]

the name in c# is not called concurrent list, it is called
blockingcollection

dictionary called concurrent dictionary

thread safe these kind of things

https://msdn.microsoft.com/en-us/library/dd267312(v=vs.110).aspx

https://msdn.microsoft.com/en-us/library/dd997369(v=vs.110).aspx

https://msdn.microsoft.com/en-us/library/dd997305(v=vs.110).aspx


On Thursday, June 16, 2016 at 4:30:33 PM UTC+8, Steven D'Aprano wrote:
> On Thursday 16 June 2016 17:28, meInvent bbird wrote:
> 
> > is there like c# have concurrent list ?
> 
> What is a concurrent list?
> 
> Can you link to the C# documentation for this?
> 
> To me, "concurrent" describes a style of execution flow, and "list" describes 
> a 
> data structure. I am struggling to understand what "concurrent list" means.
> 
> > i find something these, but how can it pass an initlist list variable
> 
> initlist = ['a', 'b', 'c']
> result = comb(n, initlist)  # pass initlist
> 
> 
> > is it doing the same function as itertools.combinations ?
> 
> It is calling itertools.combinations. So, yes, it is doing the same function 
> as 
> itertools.combinations.
> 
> 
> > def comb(n, initlist): # the argument n is the number of items to select
> > res = list(itertools.combinations(initlist, n)) # create a list from the
> > # iterator 
> > return res
> 
> This does not generate the combinations in parallel. It generates them one at 
> a 
> time, and then creates a list of them.
> 
> This is an interesting question. Somebody could probably write a parallel 
> version of combinations. But I don't know that it would be very useful -- the 
> limiting factor on combinations is more likely to be memory, not time.
> 
> Suppose you generate combinations of 100 items, taken 10 at a time. If I 
> remember the formula for combinations correctly, the total number of 
> combinations is:
> 
> 100!/(10! * 90!) = 17310309456440
> 
> combinations in total. If each generated combination took just *one* byte of 
> memory, that would require over 17 TB of RAM.
> 
> 
> 
> -- 
> Steve

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: is there a concurrent list for append in parallel programming in python? how to pass parameter in this parallel program with pool?

[Steven D'Aprano]

On Thursday 16 June 2016 17:28, meInvent bbird wrote:

> is there like c# have concurrent list ?

What is a concurrent list?

Can you link to the C# documentation for this?

To me, "concurrent" describes a style of execution flow, and "list" describes a 
data structure. I am struggling to understand what "concurrent list" means.

> i find something these, but how can it pass an initlist list variable

initlist = ['a', 'b', 'c']
result = comb(n, initlist)  # pass initlist


> is it doing the same function as itertools.combinations ?

It is calling itertools.combinations. So, yes, it is doing the same function as 
itertools.combinations.


> def comb(n, initlist): # the argument n is the number of items to select
> res = list(itertools.combinations(initlist, n)) # create a list from the
> # iterator 
> return res

This does not generate the combinations in parallel. It generates them one at a 
time, and then creates a list of them.

This is an interesting question. Somebody could probably write a parallel 
version of combinations. But I don't know that it would be very useful -- the 
limiting factor on combinations is more likely to be memory, not time.

Suppose you generate combinations of 100 items, taken 10 at a time. If I 
remember the formula for combinations correctly, the total number of 
combinations is:

100!/(10! * 90!) = 17310309456440

combinations in total. If each generated combination took just *one* byte of 
memory, that would require over 17 TB of RAM.



-- 
Steve

-- 
https://mail.python.org/mailman/listinfo/python-list

is there a concurrent list for append in parallel programming in python? how to pass parameter in this parallel program with pool?

[meInvent bbird]

is there like c# have concurrent list ? 

i find something these, but how can it pass an initlist list variable 

is it doing the same function as itertools.combinations ? 

def comb(n, initlist): # the argument n is the number of items to select 
res = list(itertools.combinations(initlist, n)) # create a list from the 
iterator 
return res 

#p = Pool(8) 
#times = range(0, len(initlist)+1)  
   
#values = p.map(comb, times) # pass the range as the sequence of arguments! 
#p.close() 
#p.join() 
-- 
https://mail.python.org/mailman/listinfo/python-list