Re: Nested loops is strangely slow, totally at a loss.

2014-12-10 Thread Peter Otten
Ian Kelly wrote: On Tue, Dec 9, 2014 at 11:30 PM, Chris Angelico ros...@gmail.com wrote: Are you sure it isn't? Your 'space' is an iterable cubic cross-product. Your first loop checks (0,0,0) which is the first element returned, and is thus fast... but it also *consumes* that first element.

Re: Nested loops is strangely slow, totally at a loss.

2014-12-10 Thread Shiyao Ma
Thanks guys. I was only aware of a limited iterables which themselves are iterators, e.g., the generator. Seems like its really a pitfall. Any glossary, list on the iterables that *might* exhaust themselves? Regards. -- https://mail.python.org/mailman/listinfo/python-list

Re: Nested loops is strangely slow, totally at a loss.

2014-12-10 Thread Peter Otten
Shiyao Ma wrote: Thanks guys. I was only aware of a limited iterables which themselves are iterators, e.g., the generator. Seems like its really a pitfall. Any glossary, list on the iterables that *might* exhaust themselves? Usually the test iterable is iter(iterable) returns True

Re: Nested loops is strangely slow, totally at a loss.

2014-12-10 Thread Steven D'Aprano
Shiyao Ma wrote: Thanks guys. I was only aware of a limited iterables which themselves are iterators, e.g., the generator. Seems like its really a pitfall. Any glossary, list on the iterables that *might* exhaust themselves? Iterables include: - iterators - sequences (e.g. lists,

Re: Nested loops is strangely slow, totally at a loss.

2014-12-10 Thread Ian Kelly
On Wed, Dec 10, 2014 at 1:21 AM, Peter Otten __pete...@web.de wrote: Ian Kelly wrote: Huh, I wasn't even aware that membership tests worked on iterables with no __contains__ method. Seems odd to me that 'x in y' should be supported but not 'len(y)'. To me def contains(iterable, value):

Re: Nested loops is strangely slow, totally at a loss.

2014-12-10 Thread Ian Kelly
On Wed, Dec 10, 2014 at 9:01 AM, Ian Kelly ian.g.ke...@gmail.com wrote: This also seems perfectly natural: def len(iterable): return sum(1 for item in iterable) My observation is that seems strange to me that one standard sequence operation should be supported for arbitrary iterators and

Nested loops is strangely slow, totally at a loss.

2014-12-09 Thread Shiyao Ma
When doing nested loop, the very first iteration of the innermost loop ends ultimately slow. Let's the code speak. The following code is quite contrived. Actually it's derived from my 3d-dct script. The actual difference is way more significant than this example. In case of any evil of gmail,

Re: Nested loops is strangely slow, totally at a loss.

2014-12-09 Thread Shiyao Ma
One thing to note, the logic of using in is not of concern here. This is a *contrived* example, the problem is the slowness of the first iteration. -- https://mail.python.org/mailman/listinfo/python-list

Re: Nested loops is strangely slow, totally at a loss.

2014-12-09 Thread Chris Angelico
On Wed, Dec 10, 2014 at 4:20 PM, Shiyao Ma i...@introo.me wrote: from itertools import product space_len = 580 space = product(xrange(space_len), xrange(space_len), xrange(space_len)) sparse_cloud = product(xrange(1000), xrange(1000), xrange(1000)) for i, j, k in sparse_cloud: ts =

Re: Nested loops is strangely slow, totally at a loss.

2014-12-09 Thread Steven D'Aprano
On Wed, 10 Dec 2014 13:20:25 +0800, Shiyao Ma wrote: When doing nested loop, the very first iteration of the innermost loop ends ultimately slow. Let's the code speak. The following code is quite contrived. Actually it's derived from my 3d-dct script. The actual difference is way more

Re: Nested loops is strangely slow, totally at a loss.

2014-12-09 Thread Chris Angelico
On Wed, Dec 10, 2014 at 5:44 PM, Steven D'Aprano st...@pearwood.info wrote: It would be nice if product iterators behaved like xrange() objects and could perform in tests without exhausting the iterator, but they don't. That's sad. It'd be very difficult to do that in the general sense. But it

Re: Nested loops is strangely slow, totally at a loss.

2014-12-09 Thread Ian Kelly
On Tue, Dec 9, 2014 at 11:30 PM, Chris Angelico ros...@gmail.com wrote: Are you sure it isn't? Your 'space' is an iterable cubic cross-product. Your first loop checks (0,0,0) which is the first element returned, and is thus fast... but it also *consumes* that first element. The next time you

Re: Nested loops is strangely slow, totally at a loss.

2014-12-09 Thread Terry Reedy
On 12/10/2014 1:53 AM, Chris Angelico wrote: On Wed, Dec 10, 2014 at 5:44 PM, Steven D'Aprano st...@pearwood.info wrote: It would be nice if product iterators behaved like xrange() objects and could perform in tests without exhausting the iterator, but they don't. That's sad. It'd be very

Re: Nested loops is strangely slow, totally at a loss.

2014-12-09 Thread Steven D'Aprano
On Wed, 10 Dec 2014 17:53:05 +1100, Chris Angelico wrote: On Wed, Dec 10, 2014 at 5:44 PM, Steven D'Aprano st...@pearwood.info wrote: It would be nice if product iterators behaved like xrange() objects and could perform in tests without exhausting the iterator, but they don't. That's sad.