Re: [Python-ideas] + operator on generators

[Serhiy Storchaka]

26.06.17 13:47, Joao S. O. Bueno пише:
On 25 June 2017 at 20:55, Danilo J. S. Bellini > wrote:


On Sun, Jun 25, 2017 at 3:06 PM, lucas via Python-ideas
>wrote:

I often use generators, and itertools.chain on them.
What about providing something like the following:

 a = (n for n in range(2))
 b = (n for n in range(2, 4))
 tuple(a + b)  # -> 0 1 2 3


You know you can do `tuple(*a, *b)` , right?

The problem with the "*" notation is that it will actually render the 
iterable

contents eagerly - unlike something that would just chain them.
But for creating tuples, it just works.


Even the tuple constructor is not needed.

>>> *a, *b
(0, 1, 2, 3)

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] A suggestion for a do...while loop

[Todd]

On Jun 26, 2017 2:15 AM, "Chris Angelico"  wrote:

On Mon, Jun 26, 2017 at 4:08 PM, Terry Reedy  wrote:
>> If we wanted to allow that to be expressed literally, we could
>> probably special case the "while not break" keyword sequence as a do
>> loop:
>>
>>  while not break:
>>  # Setup
>>  if condition: break
>>  # Loop continuation
>
>
> We would then also need 'while not return:'

And for completeness, "while not throw:"


All these situations could be handled by making a "while:" with no
condition act as "while True:"

But they could also be handled by updating pep8 to make "while True:" the
recommended infinite loop syntax and make linters smarter about this (if
they aren't already).
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] + operator on generators

[Sven R. Kunze]

Personally, I find syntactic sugar for concating interators would come 
in handy.


The purpose of iterators and generators is performance and efficiency.  
So, lowering the bar of using them is a good idea IMO. Also hoping back 
and forth a generator/iterator-based solution and a, say, 
list-based/materialized solution would become a lot easier.



On 25.06.2017 16:04, Stephan Houben wrote:

I would like to add that for example numpy ndarrays are iterables, but
they have an __add__ with completely different semantics, namely 
element-wise ( numerical) addition.


So this proposal would conflict with existing libraries with iterable 
objects.


I don't see a conflict.



Op 25 jun. 2017 2:51 p.m. schreef "Serhiy Storchaka" 
>:


It would be weird if the addition is only supported for instances
of the generator class, but not for other iterators. Why (n for n
in range(2)) + (n for n in range(2, 4)) works, but iter(range(2))
+ iter(range(2, 4)) and iter([0, 1]) + iter((2, 3)) don't?
itertools.chain() supports arbitrary iterators. Therefore you will
need to implement the __add__ method for *all* iterators in the world.



I don't think it's necessary to start with *all* iterators in the world.

So, adding iterators and/or generators, should be possible without any 
problems. It's a start and could already help a lot if I have my 
use-cases correctly.



However itertools.chain() accepts not just *iterators*. It works
with *iterables*. Therefore you will need to implement the __add__
method also for all iterables in the world. But __add__ already is
implemented for list and tuple, and many other sequences, and your
definition conflicts with this.



As above, I don't see a conflict.


Regards,
Sven
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Reducing collisions in small dicts/sets

[MRAB]

On 2017-06-26 19:21, Tim Peters wrote:

Some explanations and cautions.  An advantage of sticking with pure
Python instead of C is that spare moments are devoted to investigating
the problem instead of thrashing with micro-optimizations ;-)

Why does the current scheme suffer for small tables?  With hindsight
it's pretty obvious:  it can visit table slots more than once (the
other schemes cannot), and while the odds of that happening are tiny
in large tables, they're high for small tables.


[snip]


So where does that leave us?  I don't know.  All schemes have good &
bad points ;-)  I haven't yet thought of a cheap way to compute an
`inc` for double-hashing that isn't vulnerable to bad behavior for
_some_ easily constructed set of int keys.  If you forget "cheap",
it's easy; e.g.,

 random.seed(h)
 inc = random.choice(range(1, mask + 1, 2))

Regardless, I'll attach the current version of the code.

If the current scheme suffers only for small tables, couldn't you use an 
alternative scheme only for small tables?

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Reducing collisions in small dicts/sets

[Tim Peters]

[MRAB ]
> If the current scheme suffers only for small tables, couldn't you use an
> alternative scheme only for small tables?

Sure.  But whether that's desirable partly depends on timing actual C
code.  Try it ;-)  For maintenance sanity, it's obviously better to
have only one scheme to wrestle with.

Note that "may visit a slot more than once" isn't the only thing in
play, just one of the seemingly important things.  For example, the
current scheme requires 3 adds, 2 shifts, and a mask on each loop
iteration (or 1 add and 1 shift of those could be replaced by 1
multiplication).  The double-hashing schemes only require 1 add and 1
mask per iteration.

In cases of collision, that difference is probably swamped by waiting
for cache misses.  But, as I said in the first msg:

"""
This is a brain dump for someone who's willing to endure the
interminable pain of arguing about
benchmarks ;-)
"""
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] + operator on generators

[Koos Zevenhoven]

On Mon, Jun 26, 2017 at 4:53 PM, Serhiy Storchaka 
wrote:

> 26.06.17 13:47, Joao S. O. Bueno пише:
>
>> On 25 June 2017 at 20:55, Danilo J. S. Bellini > > wrote:
>>
>> On Sun, Jun 25, 2017 at 3:06 PM, lucas via Python-ideas
>> > >wrote:
>>
>> I often use generators, and itertools.chain on them.
>> What about providing something like the following:
>>
>>  a = (n for n in range(2))
>>  b = (n for n in range(2, 4))
>>  tuple(a + b)  # -> 0 1 2 3
>>
>>
>> You know you can do `tuple(*a, *b)` , right?
>>
>> The problem with the "*" notation is that it will actually render the
>> iterable
>> contents eagerly - unlike something that would just chain them.
>> But for creating tuples, it just works.
>>
>
> Even the tuple constructor is not needed.
>
> >>> *a, *b
> (0, 1, 2, 3)
>

​And you can also do

def a_and_b():
yield from a
yield from b

c = a_and_b() # iterable that yields 0, 1, 2, 3


I sometimes wish there was something like

c from:
yield from a
yield from b

​...or to get a list:

c as list from:
yield from a
yield from b

...or a sum:

c as sum from:
yield from a
yield from b

These would be great for avoiding crazy oneliner generator expressions.
They would also be equivalent to things like:

@list
@from
def c():
yield from a
yield from b

@sum
@from
def c():
yield from a
yield from b

the above, given:
def from(genfunc):
return genfunc()

Except of course `from` is a keyword and it should probably just be `call`.
​
But this still doesn't naturally extend to allow indexing and slicing, like
c[2] and c[1:3], for the case where the concatenated iterables are
Sequences.

-- Koos


-- 
+ Koos Zevenhoven + http://twitter.com/k7hoven +
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] A suggestion for a do...while loop

[Ethan Furman]

On 06/26/2017 01:20 PM, Mikhail V wrote:


I dont't like "while True:"  simply because it does not make enough
visual distinction with the "while condition:" statement.


My "while True:" loops look something like:

  while "":

--
~Ethan~
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] A suggestion for a do...while loop

[Cameron Simpson]

On 26Jun2017 13:30, Ethan Furman  wrote:

On 06/26/2017 01:20 PM, Mikhail V wrote:

I dont't like "while True:"  simply because it does not make enough
visual distinction with the "while condition:" statement.


My "while True:" loops look something like:

 while "":


O_o

Nice!

Cheers,
Cameron Simpson 
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Reducing collisions in small dicts/sets

[Tim Peters]

[Tim]
>...  I haven't yet thought of a cheap way to compute an
> `inc` for double-hashing that isn't vulnerable to bad behavior for
> _some_ easily constructed set of int keys.  If you forget "cheap",
> it's easy; e.g.,
>
> random.seed(h)
> inc = random.choice(range(1, mask + 1, 2))

Heh.  I always tell people not to believe what they think:  run code
to make sure!  So I did ;-)

def duni(h, nbits):
from random import seed, choice
mask = (1 << nbits) - 1
i = h & mask
yield i
seed(h)
inc = choice(range(1, mask + 1, 2))
while True:
i = (i + inc) & mask
yield i

On the terrible-for-good-reasons-for-failing-`double`-1023*i-searches
case, it turned out `duni` did just as bad as `dfib`:

"""
bits 20 nslots 1,048,576 dlen 699,050 alpha 0.67 # built 1
theoretical avg probes for uniform hashing when found 1.65 not found 3.00
...
 crisp ... skipping (slow!)
prober current
found min 1:100.00% max 1 mean 1.00
fail  min 1:33.33% max 34 mean 3.04
prober double
found min 1:100.00% max 1 mean 1.00
fail  min 1:33.33% max 699049 mean 1867.51
prober dfib
found min 1:100.00% max 1 mean 1.00
fail  min 1:33.33% max 427625 mean 8.09
prober duni
found min 1:100.00% max 1 mean 1.00
fail  min 1:33.33% max 433846 mean 9.84
prober uniform
found min 1:66.65% max 24 mean 1.65
fail  min 1:33.35% max 35 mean 3.00

Yikes!  That really surprised me.  So - are integers of the form
1023*i so magical they also manage to provoke random.seed() into
amazingly regular behavior?  Or is there something about the whole
idea of "double hashing" that leaves it vulnerable no matter how
"random" an odd increment it picks?

It's not the former.  More code showed that a great number of distinct
`inc` values are in fact used.

And double hashing is widely & successfully used in other contexts. so
it's not a problem with the idea _in general_.

What does appear to be the case:  it doesn't always play well with
taking the last `nbits` bits as the initial table index.  In other
double-hashing contexts, they strive to pick a pseudo-random initial
table index too.

Why this matters:  for any odd integer N,

N*i = N*j (mod 2**k)

if and only if

i = j (mod 2**k)

So, when integers are their own hash codes, for any `i` all the values in

range(i*N, i*N + N*2**k, N)

hash to unique slots in a table with 2**k slots (actually any table
with 2**m slots for any m >= k).  In particular, mod 2**k they map to
the slots

i*N + 0*N
i*N + 1*N
i*N + 2*N
i*N + 3*N
   ...

So, on a failing search for an integer of the form j*N, whenever
double-hashing picks an increment that happens to be a multiple of N,
it can bump into a huge chain of collisions, because double-hashing
also uses a fixed increment between probes.  And this is so for any
odd N.  For example, using a "yield i * 11" generator:

"""
bits 20 nslots 1,048,576 dlen 699,050 alpha 0.67 # built 1
theoretical avg probes for uniform hashing when found 1.65 not found 3.00
...
prober current
found min 1:100.00% max 1 mean 1.00
fail  min 1:33.33% max 34 mean 3.00
prober double
found min 1:100.00% max 1 mean 1.00
fail  min 1:33.33% max 667274 mean 34.10
prober dfib
found min 1:100.00% max 1 mean 1.00
fail  min 1:33.33% max 539865 mean 9.30
prober duni
found min 1:100.00% max 1 mean 1.00
fail  min 1:33.33% max 418562 mean 8.65
prober uniform
found min 1:66.63% max 25 mean 1.65
fail  min 1:33.31% max 32 mean 3.00
"""

All three double-hashing variants have horrible max collision chains,
for the reasons just explained.  In "normal" double-hashing contexts,
the initial table indices are scrambled; it's my fault they follow a
(mod 2**k) arithmetic progression in Python.

Which fault I gladly accept ;-)  It's valuable in practice.

But, so long as that stays, it kills the double-hashing idea for me:
while it's great for random-ish hash codes, the worst cases are
horribly bad and very easily provoked (even by accident).
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] A suggestion for a do...while loop

[Mikhail V]

>All these situations could be handled by making a "while:" with no
>condition act as "while True:"

>But they could also be handled by updating pep8 to make "while True:" the
>recommended infinite loop syntax and make linters smarter about this (if
>they aren't already).

There was a big related discussion on Python-list in April (subject "Looping" ).

IMHO the cleanest way to denote an infinite loop would be
the statement "loop:"
Without introducing new keyword I think the optimal would be just "while:"

I dont't like "while True:"  simply because it does not make enough
visual distinction with the "while condition:" statement.
E.g. I can have

while True:
...
while Blue:


Which adds some extra brain load. So if there was explicit "while:" or "loop:"
I would update for it globally in my projects.



Mikhail
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] + operator on generators

[Wes Turner]

On Sunday, June 25, 2017, Wes Turner  wrote:

>
>
> On Sunday, June 25, 2017, Danilo J. S. Bellini  > wrote:
>
>> On Sun, Jun 25, 2017 at 3:06 PM, lucas via Python-ideas <
>> python-ideas@python.org> wrote:
>>
>>> I often use generators, and itertools.chain on them.
>>> What about providing something like the following:
>>>
>>> a = (n for n in range(2))
>>> b = (n for n in range(2, 4))
>>> tuple(a + b)  # -> 0 1 2 3
>>
>>
>> AudioLazy does that: https://github.com/danilobellini/audiolazy
>>
>
> - http://toolz.readthedocs.io/en/latest/api.html#toolz.itertoolz.concat
> and concatv
>
> - https://github.com/kachayev/fn.py#streams-and-infinite-
> sequences-declaration
>  - Stream() << obj
>

<< is __lshift__()
<<= is __ilshift__()

https://docs.python.org/2/library/operator.html

Do Stream() and __lshift__() from fn.py not solve here?


>
>
>>
>>
>> --
>> Danilo J. S. Bellini
>> ---
>> "*It is not our business to set up prohibitions, but to arrive at
>> conventions.*" (R. Carnap)
>>
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] A suggestion for a do...while loop

[Joao S. O. Bueno]

and "while not except"  :-/   maybe we just stick with "while True' and put
forward a documenting
PEP advising linter packages to look for ways of getting out of the loop.

On 26 June 2017 at 08:08, Terry Reedy  wrote:

> On 6/26/2017 12:41 AM, Nick Coghlan wrote:
>
>> On 26 June 2017 at 10:25, Rob Cliffe  wrote:
>>
>>>
>>>
>>> On 25/06/2017 12:58, Markus Meskanen wrote:
>>>

 I'm a huge fan of the do...while loop in other languages, and it would
 often be useful in Python too, when doing stuff like:

 while True:
  password = input()
  if password == ...:
  break

 [...]I suggest [...]

 do:
  password = input('Password: ')
  until password == secret_password

  # This line only gets printed if until failed
  print('Invalid password, try again!')


 I don't see any significant advantage in providing an extra Way To Do
>>> It.
>>> Granted, the "while True" idiom is an idiosyncrasy, but it is frequently
>>> used and IMHO intuitive and easy to get used to.  Your suggestion doesn't
>>> even save a line of code, given that you can write:
>>>
>>>  while True:
>>>  password = input('Password:')
>>>  if password == secret_password: break
>>>  print('Invalid password, try again!')
>>>
>>
>> Right, this is the key challenge for do-while loops in Python: can you
>> come up with something that's significantly clearer than the current
>> "while True/if/break" pattern?
>>
>> The biggest weakness of that idiom is that it isn't really explicit in
>> the header line - there's nothing about "while True:" that directly
>> tells the reader "This loop is expected to exit via a break
>> statement".
>>
>> If we wanted to allow that to be expressed literally, we could
>> probably special case the "while not break" keyword sequence as a do
>> loop:
>>
>>  while not break:
>>  # Setup
>>  if condition: break
>>  # Loop continuation
>>
>
> We would then also need 'while not return:'
>
> That more explicit declaration of intent ("The code in the loop body
>> will conditionally break out of this loop") would allow a couple of
>> things:
>>
>> - the compiler could warn that an else clause attached to such a loop
>> will never execute (technically it could do that for any expression
>> that resolves to `True` as a constant)
>> - code linters could check the loop body for a break statement and
>> complain if they didn't see one
>>
>> Logically, it's exactly the same as writing "while True:", but whereas
>> that spelling suggests "infinite loop", the "while not break:"
>> spelling would more directly suggest "terminated inside the loop body
>> via a break statement"
>>
>
>
>
>
> --
> Terry Jan Reedy
>
>
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] A suggestion for a do...while loop

[Terry Reedy]

On 6/26/2017 12:41 AM, Nick Coghlan wrote:

On 26 June 2017 at 10:25, Rob Cliffe  wrote:



On 25/06/2017 12:58, Markus Meskanen wrote:


I'm a huge fan of the do...while loop in other languages, and it would
often be useful in Python too, when doing stuff like:

while True:
 password = input()
 if password == ...:
 break

[...]I suggest [...]

do:
 password = input('Password: ')
 until password == secret_password

 # This line only gets printed if until failed
 print('Invalid password, try again!')



I don't see any significant advantage in providing an extra Way To Do It.
Granted, the "while True" idiom is an idiosyncrasy, but it is frequently
used and IMHO intuitive and easy to get used to.  Your suggestion doesn't
even save a line of code, given that you can write:

 while True:
 password = input('Password:')
 if password == secret_password: break
 print('Invalid password, try again!')


Right, this is the key challenge for do-while loops in Python: can you
come up with something that's significantly clearer than the current
"while True/if/break" pattern?

The biggest weakness of that idiom is that it isn't really explicit in
the header line - there's nothing about "while True:" that directly
tells the reader "This loop is expected to exit via a break
statement".

If we wanted to allow that to be expressed literally, we could
probably special case the "while not break" keyword sequence as a do
loop:

 while not break:
 # Setup
 if condition: break
 # Loop continuation


We would then also need 'while not return:'


That more explicit declaration of intent ("The code in the loop body
will conditionally break out of this loop") would allow a couple of
things:

- the compiler could warn that an else clause attached to such a loop
will never execute (technically it could do that for any expression
that resolves to `True` as a constant)
- code linters could check the loop body for a break statement and
complain if they didn't see one

Logically, it's exactly the same as writing "while True:", but whereas
that spelling suggests "infinite loop", the "while not break:"
spelling would more directly suggest "terminated inside the loop body
via a break statement"





--
Terry Jan Reedy

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] A suggestion for a do...while loop

[Chris Angelico]

On Mon, Jun 26, 2017 at 2:41 PM, Nick Coghlan  wrote:
> If we wanted to allow that to be expressed literally, we could
> probably special case the "while not break" keyword sequence as a do
> loop:
>
> while not break:
> # Setup
> if condition: break
> # Loop continuation
>
> That more explicit declaration of intent ("The code in the loop body
> will conditionally break out of this loop") would allow a couple of
> things:
>
> - the compiler could warn that an else clause attached to such a loop
> will never execute (technically it could do that for any expression
> that resolves to `True` as a constant)
> - code linters could check the loop body for a break statement and
> complain if they didn't see one
>
> Logically, it's exactly the same as writing "while True:", but whereas
> that spelling suggests "infinite loop", the "while not break:"
> spelling would more directly suggest "terminated inside the loop body
> via a break statement"
>

What I like doing is writing these loops with a string literal as the
"condition". It compiles to the same bytecode as 'while True' does,
and then you can say what you like in the string. (An empty string
would be like 'while False', but there's no point doing that anyway.)
So, for example:

while "password not correct":
password = input('Password:')
if password == secret_password: break
print('Invalid password, try again!')

ChrisA
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] A suggestion for a do...while loop

[Chris Angelico]

On Mon, Jun 26, 2017 at 4:08 PM, Terry Reedy  wrote:
>> If we wanted to allow that to be expressed literally, we could
>> probably special case the "while not break" keyword sequence as a do
>> loop:
>>
>>  while not break:
>>  # Setup
>>  if condition: break
>>  # Loop continuation
>
>
> We would then also need 'while not return:'

And for completeness, "while not throw:".

ChrisA
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] A suggestion for a do...while loop

[Greg Ewing]

Chris Angelico wrote:

And for completeness, "while not throw:".


And just to completely confuse everyone, "while not pass". :-)

--
Greg
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] A suggestion for a do...while loop

[Chris Angelico]

On Mon, Jun 26, 2017 at 7:22 PM, Greg Ewing  wrote:
> And just to completely confuse everyone, "while not pass". :-)

while "gandalf":

ChrisA
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] + operator on generators

[Joao S. O. Bueno]

On 25 June 2017 at 20:55, Danilo J. S. Bellini 
wrote:

> On Sun, Jun 25, 2017 at 3:06 PM, lucas via Python-ideas <
> python-ideas@python.org> wrote:
>
>> I often use generators, and itertools.chain on them.
>> What about providing something like the following:
>>
>> a = (n for n in range(2))
>> b = (n for n in range(2, 4))
>> tuple(a + b)  # -> 0 1 2 3
>
>
> You know you can do `tuple(*a, *b)` , right?

The problem with the "*" notation is that it will actually render the
iterable
contents eagerly - unlike something that would just chain them.
But for creating tuples, it just works.


> AudioLazy does that: https://github.com/danilobellini/audiolazy
>
> --
> Danilo J. S. Bellini
> ---
> "*It is not our business to set up prohibitions, but to arrive at
> conventions.*" (R. Carnap)
>
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/