Re: [Python-ideas] unittest: 0 tests pass means failure of the testsuite

[Matěj Cepl]

Nathaniel Smith píše v St 06. 03. 2019 v 19:21 -0800:
> You probably want to file a bug on the setuptools tracker:
> https://github.com/pypa/setuptools
> 
> It's maintained by different people than Python itself, and is
> responsible for defining 'setup.py test'.

I think we have to bugs (or deficiencies) here:

1. setup.py tries too hard and it pretends to collect zero tests suite even 
when it failed

2. unittest claims everything is OK, when zero tests passed (and zero were 
skipped, that's a good point by Terry Reedy, completely skipped test suite is 
legitimate in some situations). And no https://bugs.python.org/issue34279 is 
not exactly it, it is about regrtest, not unittest, but it could be probably 
adapted.

Matěj

-- 
https://matej.ceplovi.cz/blog/, Jabber: mc...@ceplovi.cz
GPG Finger: 3C76 A027 CA45 AD70 98B5  BC1D 7920 5802 880B C9D8
 
Give your heartache to him. (1Pt 5,7; Mt 11:28-30)


signature.asc
Description: This is a digitally signed message part
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] unittest: 0 tests pass means failure of the testsuite

[Serhiy Storchaka]

06.03.19 22:12, Matěj Cepl пише:

I am a lead maintainer of Python packages in OpenSUSE and I can
see the pattern of many packagers adding blindly

 python setup.py test

to %check section of our SPEC file. The problem is that if the
package doesn't use unittest (it actually uses nose, pytest or
something), it could lead to zero found tests, which pass and
Python returns exit code 0 (success) even though nothing has been
tested. It seems from the outside that everything is all right,
package is being tested on every build, but actually it is lie.

Would it be possible to change unittest runner, so that when 0
tests pass, whole test suite would end up failing?


There was a related issue: https://bugs.python.org/issue34279.

It may be worth to make that warning more visible. Or make just setup.py 
more pedantic.


___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] unittest: 0 tests pass means failure of the testsuite

[Nathaniel Smith]

On Wed, Mar 6, 2019 at 12:13 PM Matěj Cepl  wrote:
>
> Hi,
>
> I am a lead maintainer of Python packages in OpenSUSE and I can
> see the pattern of many packagers adding blindly
>
> python setup.py test
>
> to %check section of our SPEC file. The problem is that if the
> package doesn't use unittest (it actually uses nose, pytest or
> something), it could lead to zero found tests, which pass and
> Python returns exit code 0 (success) even though nothing has been
> tested. It seems from the outside that everything is all right,
> package is being tested on every build, but actually it is lie.
>
> Would it be possible to change unittest runner, so that when 0
> tests pass, whole test suite would end up failing?

You probably want to file a bug on the setuptools tracker:
https://github.com/pypa/setuptools

It's maintained by different people than Python itself, and is
responsible for defining 'setup.py test'.

-n

-- 
Nathaniel J. Smith -- https://vorpus.org
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Allow creation of polymorph function (async function executable syncronously)

[Wes Turner]

Here's syncer/syncer.py:
https://github.com/miyakogi/syncer/blob/master/syncer.py

I think the singledispatch is pretty cool.

```python
#!/usr/bin/env python3
# -*- coding: utf-8 -*-

import sys
from functools import singledispatch, wraps
import asyncio
import inspect
import types
from typing import Any, Callable, Generator


PY35 = sys.version_info >= (3, 5)


def _is_awaitable(co: Generator[Any, None, Any]) -> bool:
if PY35:
return inspect.isawaitable(co)
else:
return (isinstance(co, types.GeneratorType) or
isinstance(co, asyncio.Future))


@singledispatch
def sync(co: Any):
raise TypeError('Called with unsupported argument: {}'.format(co))


@sync.register(asyncio.Future)
@sync.register(types.GeneratorType)
def sync_co(co: Generator[Any, None, Any]) -> Any:
if not _is_awaitable(co):
raise TypeError('Called with unsupported argument: {}'.format(co))
return asyncio.get_event_loop().run_until_complete(co)


@sync.register(types.FunctionType)
@sync.register(types.MethodType)
def sync_fu(f: Callable[..., Any]) -> Callable[..., Any]:
if not asyncio.iscoroutinefunction(f):
raise TypeError('Called with unsupported argument: {}'.format(f))

@wraps(f)
def run(*args, **kwargs):
return asyncio.get_event_loop().run_until_complete(f(*args,
**kwargs))
return run


if PY35:
sync.register(types.CoroutineType)(sync_co)
```

On Wed, Mar 6, 2019 at 9:20 PM Nathaniel Smith  wrote:

> On Wed, Mar 6, 2019 at 4:37 PM pylang  wrote:
> >> def maybe_async(fn):
> >> @functools.wraps(fn)
> >> def wrapper(*args, **kwargs):
> >> coro = fn(*args, **kwargs)
> >> if asyncio.get_running_loop() is not None:
> >> return coro
> >> else:
> >> return await coro
> >
> > I was unable to run his example as-is (in Python 3.6 at least) since the
> `await` keyword is only permitted inside an `async def` function.
>
> Oh yeah, that was a brain fart. I meant to write:
>
> def maybe_async(fn):
> @functools.wraps(fn)
> def wrapper(*args, **kwargs):
> coro = fn(*args, **kwargs)
> if asyncio.get_running_loop() is not None:
> return coro
> else:
> return asyncio.run(coro)
>
> -n
>
> --
> Nathaniel J. Smith -- https://vorpus.org
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] unittest: 0 tests pass means failure of the testsuite

[Terry Reedy]

On 3/6/2019 3:12 PM, Matěj Cepl wrote:

Hi,

I am a lead maintainer of Python packages in OpenSUSE and I can
see the pattern of many packagers adding blindly

 python setup.py test

to %check section of our SPEC file.


I am not familiar with setup.py, so I don't know how this affects the 
presence and contents of any particular files.



The problem is that if the
package doesn't use unittest (it actually uses nose, pytest or
something), it could lead to zero found tests,


Hence I don't know how unittest might be invoked in the situation you 
describe nor what output you see and whether you mean 0 test file(s) 
found or 0 test methods found or 0 lines of test code executed.



which pass and
Python returns exit code 0 (success) even though nothing has been
tested.


0 test methods does not mean 0 code executed in the tested module.  Here 
is a possible minimal test file test_mod that is better than nothing.


import mod
import unittest

class MinTest(unittest.TestCase):
def setUp(self):
self.instance = mod.MainClass()


It seems from the outside that everything is all right,
package is being tested on every build, but actually it is lie.


Unless a test covers 100% of both lines *and* logic, 'success' never 
means 'everything is all right'.



Would it be possible to change unittest runner, so that when 0
tests pass, whole test suite would end up failing?


Yes, but unless a change were very narrow, and only affected the 
particular situation presented, it would be a bad idea.  The unittest 
system is premised on 'success' rather than 'failure' being the default.


1. A test file may do better-that-nothing testing without running a test 
method.  See above. Calling a mininal pass a 'fail' would be wrong.


2. A test file should skip everything when running on a system that 
cannot runs the tests.  Several stdlib modules are OS-specific; their 
test modules skip all tests on some OS.  There is no OS that can run 
every file in the Python test suite.  Skipped test modules must not fail 
the test suite.


IDLE and tkinter require graphics hardware and are then optional.  IDLE 
depends on idlelib and tkinter.  Tkinter depends on _tkinter and tcl/tk. 
 Tk depends on having a graphic system, which servers and, in 
particular, *nix buildbots, generally lack.  Again, skipped IDLE and 
tkinter test.test_x files must not fail a test suite.


I agree that labeling the result of running a single test file can be 
problematical.  The following could be either a 'SUCCESS' or 'FAIL', 
depending on what one wanted and expected.  So one should read the 
detail and judge for oneself.


0:00:00 [1/1] test_idle
test_idle skipped -- No module named 'idlelib'  # or tkinter or ...
test_idle skipped

== Tests result: SUCCESS ==
1 test skipped:
test_idle

Total duration: 109 ms
Tests result: SUCCESS

Unittest effectively assumes the context 'test file in test suite'.

--
Terry Jan Reedy


___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Allow creation of polymorph function (async function executable syncronously)

[Nathaniel Smith]

On Wed, Mar 6, 2019 at 4:37 PM pylang  wrote:
>> def maybe_async(fn):
>> @functools.wraps(fn)
>> def wrapper(*args, **kwargs):
>> coro = fn(*args, **kwargs)
>> if asyncio.get_running_loop() is not None:
>> return coro
>> else:
>> return await coro
>
> I was unable to run his example as-is (in Python 3.6 at least) since the 
> `await` keyword is only permitted inside an `async def` function.

Oh yeah, that was a brain fart. I meant to write:

def maybe_async(fn):
@functools.wraps(fn)
def wrapper(*args, **kwargs):
coro = fn(*args, **kwargs)
if asyncio.get_running_loop() is not None:
return coro
else:
return asyncio.run(coro)

-n

-- 
Nathaniel J. Smith -- https://vorpus.org
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Allow creation of polymorph function (async function executable syncronously)

[pylang]

Jorropo states:

Polymorph function work exacly like async function BUT they assure of the
> ability to execute syncronously. - sic
>

Async functions can call sync functions, but not vice versa.  Consider a
third party solution - trio , that
allows sync functions to call async functions.

>From the docs
:

async def async_double(x):
return 2 * x


trio.run(async_double, 3)# returns 6

---

If you want a stdlib solution, let's revisit Nathaniel Smith's example:


> def maybe_async(fn):
> @functools.wraps(fn)
> def wrapper(*args, **kwargs):
> coro = fn(*args, **kwargs)
> if asyncio.get_running_loop() is not None:
> return coro
> else:
> return await coro
>

I was unable to run his example as-is (in Python 3.6 at least) since the
`await` keyword is only permitted inside an `async def` function.  However,
the idea is intriguing and can be adapted.  See the example below.

Code:

def maybe_async(fn):
async def _process(fn, *args, **kwargs):
coro_fn = fn(*args, **kwargs)
if asyncio.iscoroutinefunction(fn):
return await coro_fn
else:
return coro_fn

@functools.wraps(fn)
def wrapper(*args, **kwarg):
loop = asyncio.get_event_loop()
res = loop.run_until_complete(_process(fn, *args, **kwarg))
return res

return wrapper

Demo:

@maybe_async
async def agreet(delay):
print("hello")
await asyncio.sleep(delay)
print("world")


@maybe_async
def greet(delay):
print("hello")
time.sleep(delay)
print("world")


agreet(2)# prints hello world after 2 seconds
greet(1)# print hello world after 1 second

Now you can call either sync or async functions like regular functions.
Hope this helps.

---


On Wed, Mar 6, 2019 at 12:54 AM Nathaniel Smith  wrote:

> Defining a single polymorphic function is easy at the library level.
> For example, with asyncio:
>
> 
>
> def maybe_async(fn):
> @functools.wraps(fn)
> def wrapper(*args, **kwargs):
> coro = fn(*args, **kwargs)
> if asyncio.get_running_loop() is not None:
> return coro
> else:
> return await coro
>
> @maybe_async
> async def my_func(...):
> ... use asyncio freely in here ...
>
> 
>
> You can't do it at the language level though (e.g. with your proposed
> 'polymorph' keyword), because the language doesn't know whether an
> event loop is running or not.
>
> Extending this from a single function to a whole library API is
> substantially more complex, because you have to wrap every function
> and method, deal with __iter__ versus __aiter__, etc.
>
> -n
>
> On Tue, Mar 5, 2019 at 8:02 PM Jorropo .  wrote:
> >
> > I was doing some async networking and I wondered, why I have to use 2
> different api for making the same things in async or sync regime.
> > Even if we make 2 perfectly identical api (except function will be sync
> and async), it will still 2 different code).
> >
> > So I first thinked to allow await in syncronous function but that create
> some problems (ex: an async function calling async.create_task) so if we
> allow that we have to asume to allways be in async regime (like js).
> >
> > Or we can differentiate async function wich can be awaited in syncronous
> regime, maybe with a new keyword (here I will use polymorph due to a lack
> of imagination but I find that one too long) ?
> >
> > So a polymorph function can be awaited in a syncronous function, and a
> polymorph function can only await polymorph functions.
> >
> > Polymorph function work exacly like async function BUT they assure of
> the ability to execute syncronously.
> > And in a syncronous regime if an await need to wait (like async.sleep or
> network operation), just wait (like the equivalent of this function in
> syncronous way).
> >
> > So why made that ?
> > To provide the same api for async and sync regime when its possible,
> example http api.
> > This allow to code less librairy.
> > Syncronous users can just use the librairy like any other sync lib (with
> the keyword await for executing but, personally, I think that is worth).
> > And asyncronous users can run multiples tasks using the same lib.
> > Moving from a regime to an other is simpler, source code size is reduced
> (doesn't need to create 2 api for the same lib), gain of time for the same
> reason.
> >
> > Also why it need to await in syncronous function, why not just execute
> polymorph function like any sync function while called in a sync function ?
> > Because we need to create runnable objects for async.run, ...
> >
> > So I would have your though about that, what can be improved, a better
> name for polymorph ?
> > ___
> > Python-ideas mailing list
> > Python-ideas@python.org
> > https://mail.python.org/mailman/listinfo/python-ideas
> > 

Re: [Python-ideas] PEP: Dict addition and subtraction

[Chris Angelico]

On Thu, Mar 7, 2019 at 10:52 AM Josh Rosenberg
 wrote:
> The closest I can come to a thorough definition of what + does in Python (and 
> most languages) right now is that:
>
> 1. Returns a new thing of the same type (or a shared coerced type for number 
> weirdness)
> 2. That combines the information of the input operands
> 3. Is associative ((a + b) + c produces the same thing as a + (b + c)) 
> (modulo floating point weirdness)
> 4. Is "reversible": Knowing the end result and *one* of the inputs is 
> sufficient to determine the value of the other input; that is, for c = a + b, 
> knowing any two of a, b and c allows you to determine a single unambiguous 
> value for the remaining value (numeric coercion and floating point weirdness 
> make this not 100%, but you can at least know a value equal to other value; 
> e.g. for c = a + b, knowing c is 5.0 and a is 1.0 is sufficient to say that b 
> is equal to 4, even if it's not necessarily an int or float). For numbers, 
> reversal is done with -; for sequences, it's done by slicing c using the 
> length of a or b to "subtract" the elements that came from a/b.
> 5. (Actual addition only) Is commutative (modulo floating point weirdness); a 
> + b == b + a
> 6. (Concatenation only) Is order preserving (really a natural consequence of 
> #4, but a property that people expect)
>
> Allowing dicts to get involved in + means:
>
> 1. Fewer consistent rules apply to +;
> 2. The particular idiosyncrasies of Python dict ordering and "which value 
> wins" rules are now tied to +. for concatenation, there is only one set of 
> possible rules AFAICT so every language naturally agrees on behavior, but 
> dict merging obviously has many possible rules that would be unlikely to 
> match the exact rules of any other language except by coincidence). a winning 
> on order and b winning on value is a historical artifact of how Python's dict 
> developed; I doubt any other language would intentionally choose to split 
> responsibility like that if they weren't handcuffed by history.
>
> Again, there's nothing wrong with making dict merges easier. But it shouldn't 
> be done by (further) abusing +.

Lots of words that basically say: Stuff wouldn't be perfectly pure.

But adding dictionaries is fundamentally *useful*. It is expressive.
It will, in pretty much all situations, do exactly what someone would
expect, based on knowledge of how Python works in other areas. The
semantics for edge cases have to be clearly defined, but they'll only
come into play on rare occasions; most of the time, for instance, we
don't have to worry about identity vs equality in dictionary keys. If
you tell people "adding two dictionaries combines them, with the right
operand winning collisions", it won't matter that this isn't how lists
or floats work; it'll be incredibly useful as it is.

Practicality. Let's have some.

ChrisA
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP: Dict addition and subtraction

[Josh Rosenberg]

On Wed, Mar 6, 2019 at 10:31 PM Greg Ewing 
wrote:

>
> You might as well say that using the + operator on vectors is
> nonsense, because len(v1 + v2) is not in general equal to
> len(v1) + len(v2).
>
> Yet mathematicians are quite happy to talk about "addition"
> of vectors.
>
>
Vectors addition is *actual* addition, not concatenation. You're so busy
loosening the definition of + as relates to , to make it make sense for
dicts that you've forgotten that + is, first and foremost, about addition
in the mathematical sense, where vector addition is just one type of
addition. Concatenation is already a minor abuse of +, but one commonly
accepted by programmers, thanks to it having some similarities to addition
and a single, unambiguous set of semantics to avoid confusion.

You're defending + on dicts because vector addition isn't concatenation
already, which only shows how muddled things get when you try to use + to
mean multiple concepts that are at best loosely related.

The closest I can come to a thorough definition of what + does in Python
(and most languages) right now is that:

1. Returns a new thing of the same type (or a shared coerced type for
number weirdness)
2. That combines the information of the input operands
3. Is associative ((a + b) + c produces the same thing as a + (b + c))
(modulo floating point weirdness)
4. Is "reversible": Knowing the end result and *one* of the inputs is
sufficient to determine the value of the other input; that is, for c = a +
b, knowing any two of a, b and c allows you to determine a single
unambiguous value for the remaining value (numeric coercion and floating
point weirdness make this not 100%, but you can at least know a value equal
to other value; e.g. for c = a + b, knowing c is 5.0 and a is 1.0 is
sufficient to say that b is equal to 4, even if it's not necessarily an int
or float). For numbers, reversal is done with -; for sequences, it's done
by slicing c using the length of a or b to "subtract" the elements that
came from a/b.
5. (Actual addition only) Is commutative (modulo floating point weirdness);
a + b == b + a
6. (Concatenation only) Is order preserving (really a natural consequence
of #4, but a property that people expect)

Note that these rules are consistent across most major languages that allow
+ to mean combine collections (the few that disagree, like Pascal, don't
support | as a union operator).

Concatenation is missing element #5, but otherwise aligns with actual
addition. dict merges (and set unions for that matter) violate #4 and #6;
for c = a + b, knowing c and either a or b still leaves a literally
infinite set of possible inputs for the other input (it's not infinite for
sets, where the options would be a subset of the result, but for dicts,
there would be no such limitation; keys from b could exist with any
possible value in a). dicts order preserving aspect *almost* satisfies #6,
but not quite (if 'x' comes after 'y' in b, there is no guarantee that it
will do so in c, because a gets first say on ordering, and b gets the final
word on value).

Allowing dicts to get involved in + means:

1. Fewer consistent rules apply to +;
2. The particular idiosyncrasies of Python dict ordering and "which value
wins" rules are now tied to +. for concatenation, there is only one set of
possible rules AFAICT so every language naturally agrees on behavior, but
dict merging obviously has many possible rules that would be unlikely to
match the exact rules of any other language except by coincidence). a
winning on order and b winning on value is a historical artifact of how
Python's dict developed; I doubt any other language would intentionally
choose to split responsibility like that if they weren't handcuffed by
history.

Again, there's nothing wrong with making dict merges easier. But it
shouldn't be done by (further) abusing +.

-Josh Rosenberg
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP: Dict addition and subtraction

[Greg Ewing]

Ka-Ping Yee wrote:
len(dict1 + dict2) does not equal len(dict1) + len(dict2), so using the 
+ operator is nonsense.


You might as well say that using the + operator on vectors is
nonsense, because len(v1 + v2) is not in general equal to
len(v1) + len(v2).

Yet mathematicians are quite happy to talk about "addition"
of vectors.

--
Greg
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] unittest: 0 tests pass means failure of the testsuite

[Guido van Rossum]

I would just file a bug and add a PR.

On Wed, Mar 6, 2019 at 12:14 PM Matěj Cepl  wrote:

> Hi,
>
> I am a lead maintainer of Python packages in OpenSUSE and I can
> see the pattern of many packagers adding blindly
>
> python setup.py test
>
> to %check section of our SPEC file. The problem is that if the
> package doesn't use unittest (it actually uses nose, pytest or
> something), it could lead to zero found tests, which pass and
> Python returns exit code 0 (success) even though nothing has been
> tested. It seems from the outside that everything is all right,
> package is being tested on every build, but actually it is lie.
>
> Would it be possible to change unittest runner, so that when 0
> tests pass, whole test suite would end up failing?
>
> Thank you for considering this,
>
> Matěj
>
> --
> https://matej.ceplovi.cz/blog/, Jabber: mc...@ceplovi.cz
> GPG Finger: 3C76 A027 CA45 AD70 98B5  BC1D 7920 5802 880B C9D8
>
> Never ascribe to malice that which is adequately explained by
> stupidity.
> -- Napoleon Bonaparte (or many other people to whom this
>quote is ascribed)
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>


-- 
--Guido van Rossum (python.org/~guido)
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] dict literal allows duplicate keys

[Jonathan Fine]

Hi Guido

You wrote:

> Would it shut down this particular subthread if (as the language's designer, 
> if not its BDFL) I declared that this was an explicit design decision that I 
> made nearly 30 years ago? I should perhaps blog about the background of this 
> decision, but it was quite a conscious one. There really is no point in 
> thinking that this is an accident of implementation or could be changed.

Thank you for sharing this with us.

I'd be fascinated to hear about the background to this conscious
decision, and I think it would help me and others understand better
what makes Python what it is. And it might help persuade me that my
surprise at {'a': 0, 'a': 1} is misplaced, or at least exaggerated and
one-sided.

Do you want menial help writing the blog?  Perhaps if you share your
recollections, others will find the traces in the source code. For
example, I've found the first dictobject.c, dating back to 1994.
https://github.com/python/cpython/blob/956640880da20c20d5320477a0dcaf2026bd9426/Objects/dictobject.c

I'm a great fan of your Python conversation (with Biancuzzi and Warden) in
http://shop.oreilly.com/product/9780596515171.do # Masterminds of Programming

I've read this article several times, and have wished that it was more
widely available. My personal view is that putting a copy of this
article in docs.python.org would provide more benefit to the community
than you blogging on why dict literals allow duplicate keys. However,
it need not be either/or. Perhaps someone could ask the PSF to talk
with O'Reilly about getting copyright clearance to do this.

Finally, some personal remarks. I've got a long training as a pure
mathematician. For me consistency and application of simple basic
principles is important to me. And also the discovery of basic
principles.

In your interview, you say (paraphrased and without context) that most
Python code is written simply to get a job done. And that pragmatism,
rather then being hung up about theoretical concept, is the
fundamental quality in being proficient in developing with Python.

Thank you for inventing Python, and designing the language. It's a
language popular both with pure mathematicians, and also pragmatic
people who want to get things done. That's quite an achievement, which
has drawn people like me into your community.

with best regards

Jonathan
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] unittest: 0 tests pass means failure of the testsuite

[MRAB]

On 2019-03-06 20:12, Matěj Cepl wrote:

Hi,

I am a lead maintainer of Python packages in OpenSUSE and I can
see the pattern of many packagers adding blindly

 python setup.py test

to %check section of our SPEC file. The problem is that if the
package doesn't use unittest (it actually uses nose, pytest or
something), it could lead to zero found tests, which pass and
Python returns exit code 0 (success) even though nothing has been
tested. It seems from the outside that everything is all right,
package is being tested on every build, but actually it is lie.

Would it be possible to change unittest runner, so that when 0
tests pass, whole test suite would end up failing?

Thank you for considering this,


Strictly speaking, it's not a lie because none of the tests have failed.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] unittest: 0 tests pass means failure of the testsuite

[Matěj Cepl]

Hi,

I am a lead maintainer of Python packages in OpenSUSE and I can
see the pattern of many packagers adding blindly

python setup.py test

to %check section of our SPEC file. The problem is that if the
package doesn't use unittest (it actually uses nose, pytest or
something), it could lead to zero found tests, which pass and 
Python returns exit code 0 (success) even though nothing has been
tested. It seems from the outside that everything is all right,
package is being tested on every build, but actually it is lie.

Would it be possible to change unittest runner, so that when 0
tests pass, whole test suite would end up failing?

Thank you for considering this,

Matěj

-- 
https://matej.ceplovi.cz/blog/, Jabber: mc...@ceplovi.cz
GPG Finger: 3C76 A027 CA45 AD70 98B5  BC1D 7920 5802 880B C9D8
 
Never ascribe to malice that which is adequately explained by
stupidity.
-- Napoleon Bonaparte (or many other people to whom this
   quote is ascribed)


signature.asc
Description: This is a digitally signed message part
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] OT: about hasty posts from phones

[Abdur-Rahmaan Janhangeer]

Sorry about adding a few words here, i know you are all more 'advanced'
programmers than me.

I just wanted to ask the list to keep threads informative. Just today i
decided to take the bulls by the horn and read the add dictionaries by
using the + operator. Midway, i asked myself if i was getting value from
reading each mail one by one.

I don't think i can use the lists as a hands-on reference (was not meant to
but could've been), it fits more as an nlp data sci project with data
cleaning and all.

I don't want an exact SO line though in the sense of absolutely no crap for
beginners would be scared away. The talking of it's members sometimes do
keep things lively. But for technical posts i think we can keep it to
technical points and avoid the like of

>> if we do this some trajic things might happen

> what do you mean by "trajic"?

and so on. Those create diversions in the road to understanding a topic.
For discussions posts like CoC update or whatever things off code, i think
there can be a little leniency.

I have only neen using python since 3.4, i have a lot to catch up and learn
from you all.

yours,

Abdur-Rahmaan Janhangeer
http://www.pythonmembers.club | https://github.com/Abdur-rahmaanJ
Mauritius
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] dict literal allows duplicate keys

[Guido van Rossum]

Would it shut down this particular subthread if (as the language's
designer, if not its BDFL) I declared that this was an explicit design
decision that I made nearly 30 years ago? I should perhaps blog about the
background of this decision, but it was quite a conscious one. There really
is no point in thinking that this is an accident of implementation or could
be changed.

On Wed, Mar 6, 2019 at 9:10 AM Jonathan Fine  wrote:

> SUMMARY: The outcome of a search for: python dict literal duplicate
> keys. No conclusions (so far).
>
> BACKGROUND
> In the thread  "PEP: Dict addition and subtraction" I wrote
>
> > >>>  {'a': 0, 'a': 1}
> > {'a': 1}
>
> > I wonder, is this behaviour of {'a': 0, 'a': 1} documented (or tested)
> > anywhere? I didn't find it in these URLs:
> > https://docs.python.org/3/library/stdtypes.html#mapping-types-dict
> > https://docs.python.org/3/tutorial/datastructures.html#dictionaries
>
> LINKS
> I've since found some relevant URLs.
>
> [1]
> https://stackoverflow.com/questions/34539772/is-a-dict-literal-containing-repeated-keys-well-defined
> [2]
> https://help.semmle.com/wiki/display/PYTHON/Duplicate+key+in+dict+literal
> [3] https://bugs.python.org/issue26910
> [4] https://bugs.python.org/issue16385
> [5] https://realpython.com/python-dicts/
>
> ANALYSIS
> [1] gives a reference to [6], which correctly states the behaviour of
> {'a':0, 'a':1}, although without giving an example. (Aside: Sometimes
> one example is worth 50 or more words.)
>
> [2] is from Semmle, who provide an automated code review tool, called
> LGTM. The page [2] appears to be part of the documentation for LGTM.
> This page provides a useful link to [7].
>
> [3] is a re-opening of [4]. It was rapidly closed by David Murray, who
> recommended reopening the discussion on python-ideas.
> [4] was raised by Albert Ferras, based on his real-world experience.
> In particular, a configuration file that contains a long dict literal.
> This was closed by Benjamin Peterson, who said that raising an error
> was "out of the question for compatibility isssues". Given few use
> case and little support on python-ideas,Terry Ready supported the
> closure. Raymond Hettinger supported the closure.
>
> [5] is from RealPython, who provide online tutorials. This page
> contains the statement "a given key can appear in a dictionary only
> once. Duplicate keys are not allowed." Note that
> {'a': 0, 'a': 1}
> can reasonably be thought of as a dictionary with duplicate keys.
>
> NOTE
> As I recall SGML (this shows my age) allows multiple entity declarations,
> as in
> 
> 
>
> And as I recall, in SGML the first value "original" is the one that is
> in effect. This is what happens with the LaTeX command
> \providecommand.
>
> FURTHER LINKS
> [6]
> https://docs.python.org/3/reference/expressions.html#dictionary-displays
> [7] https://cwe.mitre.org/data/definitions/561.html # CWE-561: Dead Code
>
> --
> Jonathan
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>


-- 
--Guido van Rossum (python.org/~guido)
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] OT: about hasty posts from phones

[Guido van Rossum]

On Wed, Mar 6, 2019 at 9:12 AM Christopher Barker 
wrote:

> [...]



> Sorry — on a phone, kinda hard to check now.
>

A point of order: if you're away from a real keyboard/screen, maybe it's
better to wait. The conversation isn't real-time, and you don't win points
by answering first.

We could all be reminded of the goal for StackOverflow: the intent there is
to create a useful artifact. While it's not quite the same for
python-ideas, in the end we're trying to get to insights and (tentative)
decisions, which should be long-lasting. Fewer posts may be better.

NOTE: I'm not picking on you specifically Chris! I see this a lot, and I do
it myself too (and regularly regret it).

-- 
--Guido van Rossum (python.org/~guido)
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP: Dict addition and subtraction

[Michael Lee]

>
> If we use this "literally concat" metaphor, I still think set should have
> `+` as alias to `|` for consistency.
>

I agree.

I think "|" keeps commutativity only because it's minor than `+`.
>

I suppose that's true, fair point.

I guess I would be ok with | no longer always implying commutativity if we
were repurposing it for some radically different purpose. But dicts and
sets are similar enough that I think having them both use similar but
ultimately different definitions of "|" is going to have non-zero cost,
especially when reading or modifying future code that makes heavy use of
both data structures.

Maybe that cost is worth it. I'm personally not convinced, but I do think
it should be taken into account..

Hmm.  The PEP proposed dict - dict, which is similar to set - set
> (difference).
>

Now that you point it out, I think I also dislike `d1 - d2` for the same
reasons I listed earlier: it's not consistent with set semantics. One other
objection I overlooked is that the PEP currently requires both operands to
be dicts when doing "d1 - d2" . So doing {"a": 1, "b": 2, "c": 3} - ["a",
"b"] is currently disallowed (though doing d1 -= ["a", "b"] is apparently
ok).

I can sympathize: allowing "d1 - some_iter" feels a little too magical to
me. But it's unfortunately restrictive -- I suspect removing keys stored
within a list or something would be just as common of a use-case if not
more so then removing keys stored in another dict.

I propose that we instead add methods like "d1.without_keys(...)" and
"d1.remove_keys(...)" that can accept any iterable of keys. These two
methods would replace "d1.__sub__(...)" and "d1.__isub__(...)"
respectively. The exact method names and semantics could probably do with a
little more bikeshedding, but I think this idea would remove a false
symmetry between "d1 + d2" and "d1 - d2" that doesn't actually really exist
while being more broadly useful.

Or I guess we could just remove that restriction: "it feels too magical"
isn't a great objection on my part. Either way, that part of the PEP could
use some more refinement, I think.

-- Michael







On Wed, Mar 6, 2019 at 8:29 AM Inada Naoki  wrote:

> On Wed, Mar 6, 2019 at 10:59 PM Michael Lee 
> wrote:
>
> >
> > I think the behavior proposed in the PEP makes sense whether you think
> of "+" as meaning "concatenation" or "merging".
> >
> > If your instinct is to assume "+" means "concatenation", then it would
> be natural to assume that {"a": 1, "b": 2} + {"c": 3, "b": 4} would be
> identical to {"a": 1, "b": 2, "c": 3, "b": 4} -- literally concat the
> key-value pairs into a new dict.
> >
>
> Nice explanation.  You reduced my opposite to `+` by "literally concat".
> Better example, {"a": 1, "b": 2} + {"c": 4, "b": 3} == {"a": 1, "b":
> 2, "c": 4, "b": 3} == {"a": 1, "b": 3, "c": 4}
>
> On the other hand, union of set is also "literally concat".  If we use
> this "literally concat" metaphor,
> I still think set should have `+` as alias to `|` for consistency.
>
> >
> > Using "|" would also violate an important existing property of unions:
> the invariant "d1 | d2 == d2 | d1" is no longer true. As far as I'm aware,
> the union operation is always taken to be commutative in math, and so I
> think it's important that we preserve that property in Python. At the very
> least, I think it's far more important to preserve commutativity of unions
> then it is to preserve some of the invariants I've seen proposed above,
> like "len(d1 + d2) == len(d1) + len(d2)".
> >
>
> I think both rule are "rather a coincidence than a conscious decision".
>
> I think "|" keeps commutativity only because it's minor than `+`.  Easy
> operator
> is abused easily more than minor operator.
>
> And I think every "coincidence" rules are important.  They makes
> understanding Python easy.
> Every people "discover" rules and consistency while learning language.
>
> This is a matter of balance.  There are no right answer.  Someone
> *feel* rule A is important than B.
> Someone feel opposite.
>
>
> > But I do know that I'm a strong -1 on adding set operations to dicts:
> it's not possible to preserve the existing semantics of union (and
> intersection) with dict and  think expressions like "d1 | d2" and "d1 & d2"
> would just be confusing and misleading to encounter in the wild.
>
> Hmm.  The PEP proposed dict - dict, which is similar to set - set
> (difference).
> To me, {"a": 1, "b": 2} - {"b": 3} = {"a": 1} is confusing than {"a":
> 1, "b": 2} - {"b"} = {"a": 1}.
>
> So I think borrow some semantics from set is good idea.
> Both of `dict - set` and `dict & set` makes sense to me.
>
> * `dict - set` can be used to remove private keys by "blacklist".
> * `dict & set` can be used to choose public keys by "whiltelist".
>
> --
> Inada Naoki  
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] OT: Dictionary display documentation

[Rhodri James]

On 06/03/2019 18:12, Rhodri James wrote:

On 06/03/2019 17:43, Jonathan Fine wrote:

Indeed. Although off-topic, I think


{'a': 0, 'a': 1} == {'a': 1}

True

is much better than "This means that you can specify the same key
multiple times in the key/datum list, and the final dictionary’s value
for that key will be the last one given."


I disagree.  An example is an excellent thing, but the words are 
definitive and must be there.


Sigh.  I hit SEND before I finished changing the title.  Sorry, folks.

--
Rhodri James *-* Kynesim Ltd
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] OT:

[Rhodri James]

On 06/03/2019 17:43, Jonathan Fine wrote:

Indeed. Although off-topic, I think


{'a': 0, 'a': 1} == {'a': 1}

True

is much better than "This means that you can specify the same key
multiple times in the key/datum list, and the final dictionary’s value
for that key will be the last one given."


I disagree.  An example is an excellent thing, but the words are 
definitive and must be there.


--
Rhodri James *-* Kynesim Ltd
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] dict literal allows duplicate keys

[Jonathan Fine]

SUMMARY: Off-thread-topic comment on examples and words in documentation.

Inada Naoki quoted (from doc.python ref [6] in my original post):

> > If a comma-separated sequence of key/datum pairs is given, they are 
> > evaluated from left to right to define the entries of the dictionary: each 
> > key object is used as a key into the dictionary to store the corresponding 
> > datum. This means that you can specify the same key multiple times in the 
> > key/datum list, and the final dictionary’s value for that key will be the 
> > last one given.

Indeed. Although off-topic, I think

>>> {'a': 0, 'a': 1} == {'a': 1}
True

is much better than "This means that you can specify the same key
multiple times in the key/datum list, and the final dictionary’s value
for that key will be the last one given."

By the way, today I think we'd say key/value pairs. And I've read

https://www.theguardian.com/guardian-observer-style-guide-d
data takes a singular verb (like agenda), though strictly a plural;
you come across datum, the singular of data, about as often as you
hear about an agendum

Oh, and "the final dictionary's value" should I think be "the
dictionary's final value" or perhaps just "the dictionary's value"

But now we're far from the thread topic. I'm happy to join in on a
thread on improving documentation (by using simpler language and good
examples).

-- 
Jonathan
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] dict literal allows duplicate keys

[Inada Naoki]

https://docs.python.org/3/reference/expressions.html#dictionary-displays

> If a comma-separated sequence of key/datum pairs is given, they are evaluated 
> from left to right to define the entries of the dictionary: each key object 
> is used as a key into the dictionary to store the corresponding datum. This 
> means that you can specify the same key multiple times in the key/datum list, 
> and the final dictionary’s value for that key will be the last one given.

On Thu, Mar 7, 2019 at 2:09 AM Jonathan Fine  wrote:
>
> SUMMARY: The outcome of a search for: python dict literal duplicate
> keys. No conclusions (so far).
>
> BACKGROUND
> In the thread  "PEP: Dict addition and subtraction" I wrote
>
> > >>>  {'a': 0, 'a': 1}
> > {'a': 1}
>
> > I wonder, is this behaviour of {'a': 0, 'a': 1} documented (or tested)
> > anywhere? I didn't find it in these URLs:
> > https://docs.python.org/3/library/stdtypes.html#mapping-types-dict
> > https://docs.python.org/3/tutorial/datastructures.html#dictionaries
>
> LINKS
> I've since found some relevant URLs.
>
> [1] 
> https://stackoverflow.com/questions/34539772/is-a-dict-literal-containing-repeated-keys-well-defined
> [2] https://help.semmle.com/wiki/display/PYTHON/Duplicate+key+in+dict+literal
> [3] https://bugs.python.org/issue26910
> [4] https://bugs.python.org/issue16385
> [5] https://realpython.com/python-dicts/
>
> ANALYSIS
> [1] gives a reference to [6], which correctly states the behaviour of
> {'a':0, 'a':1}, although without giving an example. (Aside: Sometimes
> one example is worth 50 or more words.)
>
> [2] is from Semmle, who provide an automated code review tool, called
> LGTM. The page [2] appears to be part of the documentation for LGTM.
> This page provides a useful link to [7].
>
> [3] is a re-opening of [4]. It was rapidly closed by David Murray, who
> recommended reopening the discussion on python-ideas.
> [4] was raised by Albert Ferras, based on his real-world experience.
> In particular, a configuration file that contains a long dict literal.
> This was closed by Benjamin Peterson, who said that raising an error
> was "out of the question for compatibility isssues". Given few use
> case and little support on python-ideas,Terry Ready supported the
> closure. Raymond Hettinger supported the closure.
>
> [5] is from RealPython, who provide online tutorials. This page
> contains the statement "a given key can appear in a dictionary only
> once. Duplicate keys are not allowed." Note that
> {'a': 0, 'a': 1}
> can reasonably be thought of as a dictionary with duplicate keys.
>
> NOTE
> As I recall SGML (this shows my age) allows multiple entity declarations, as 
> in
> 
> 
>
> And as I recall, in SGML the first value "original" is the one that is
> in effect. This is what happens with the LaTeX command
> \providecommand.
>
> FURTHER LINKS
> [6] https://docs.python.org/3/reference/expressions.html#dictionary-displays
> [7] https://cwe.mitre.org/data/definitions/561.html # CWE-561: Dead Code
>
> --
> Jonathan
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/



-- 
Inada Naoki  
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP: Dict addition and subtraction

[Jonathan Fine]

I wrote:

> I wonder, is this behaviour of {'a': 0, 'a': 1} documented (or tested)
> anywhere?

I've answered my own question here:
[Python-ideas] dict literal allows duplicate keys
https://mail.python.org/pipermail/python-ideas/2019-March/055717.html

Finally, Christopher Barker wrote:
> Yes, and had already been brought up in this thread ( I think by Guido). 
> (Maybe not well documented, but certainly well understood and deliberate)

Thank you for this, Christopher.

-- 
Jonathan
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Suggestions: dict.flow_update and dict.__add__

[Christopher Barker]

Do go read the recent thread about this - there is a lot there!

Titled something like “fluent programming”

Sorry — on a phone, kinda hard to check now.

-CHB

-- 
Christopher Barker, PhD

Python Language Consulting
  - Teaching
  - Scientific Software Development
  - Desktop GUI and Web Development
  - wxPython, numpy, scipy, Cython
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP: Dict addition and subtraction

[Inada Naoki]

On Wed, Mar 6, 2019 at 10:59 PM Michael Lee  wrote:

>
> I think the behavior proposed in the PEP makes sense whether you think of "+" 
> as meaning "concatenation" or "merging".
>
> If your instinct is to assume "+" means "concatenation", then it would be 
> natural to assume that {"a": 1, "b": 2} + {"c": 3, "b": 4} would be identical 
> to {"a": 1, "b": 2, "c": 3, "b": 4} -- literally concat the key-value pairs 
> into a new dict.
>

Nice explanation.  You reduced my opposite to `+` by "literally concat".
Better example, {"a": 1, "b": 2} + {"c": 4, "b": 3} == {"a": 1, "b":
2, "c": 4, "b": 3} == {"a": 1, "b": 3, "c": 4}

On the other hand, union of set is also "literally concat".  If we use
this "literally concat" metaphor,
I still think set should have `+` as alias to `|` for consistency.

>
> Using "|" would also violate an important existing property of unions: the 
> invariant "d1 | d2 == d2 | d1" is no longer true. As far as I'm aware, the 
> union operation is always taken to be commutative in math, and so I think 
> it's important that we preserve that property in Python. At the very least, I 
> think it's far more important to preserve commutativity of unions then it is 
> to preserve some of the invariants I've seen proposed above, like "len(d1 + 
> d2) == len(d1) + len(d2)".
>

I think both rule are "rather a coincidence than a conscious decision".

I think "|" keeps commutativity only because it's minor than `+`.  Easy operator
is abused easily more than minor operator.

And I think every "coincidence" rules are important.  They makes
understanding Python easy.
Every people "discover" rules and consistency while learning language.

This is a matter of balance.  There are no right answer.  Someone
*feel* rule A is important than B.
Someone feel opposite.


> But I do know that I'm a strong -1 on adding set operations to dicts: it's 
> not possible to preserve the existing semantics of union (and intersection) 
> with dict and  think expressions like "d1 | d2" and "d1 & d2" would just be 
> confusing and misleading to encounter in the wild.

Hmm.  The PEP proposed dict - dict, which is similar to set - set (difference).
To me, {"a": 1, "b": 2} - {"b": 3} = {"a": 1} is confusing than {"a":
1, "b": 2} - {"b"} = {"a": 1}.

So I think borrow some semantics from set is good idea.
Both of `dict - set` and `dict & set` makes sense to me.

* `dict - set` can be used to remove private keys by "blacklist".
* `dict & set` can be used to choose public keys by "whiltelist".

-- 
Inada Naoki  
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Suggestions: dict.flow_update and dict.__add__

[Brice Parent]

Why not simply propose an external lib with FluentDict and other 
Fluent[Anything] already packaged? I don't know if I'd use it 
personnally, but it definitely could have some users.



Le 05/03/2019 à 09:48, Jonathan Fine a écrit :

SUMMARY
Instead of using dict + dict, perhaps use dict.flow_update. Here,
flow_update is just like update, except that it returns self.

BACKGROUND
There's a difference between a sorted copy of a list, and sorting the
list in place.

 >>> items = [2, 0, 1, 9]
 >>> sorted(items), items
 ([0, 1, 2, 9], [2, 0, 1, 9])
 >>> items.sort(), items
(None, [0, 1, 2, 9])

In Python, mutating methods generally return None. Here, this prevents
beginners thinking their code has produced a sorted copy of a list,
when in fact it has done an in-place sort on the list. If they write
 >>> aaa = my_list.sort()
they'll get a None error when they use aaa.

The same goes for dict.update. This is a useful feature, particularly
for beginners. It helps them think clearly, and express themselves
clearly.

THE PROBLEM
This returning None can be a nuisance, sometimes. Suppose we have a
dictionary of default values, and a dictionary of use supplied
options. We wish to combine the two dictionaries, say into a new
combined dictionary.

One way to do this is:

combined = defaults.copy()
combined.update(options)

But this is awkward when you're in the middle of calling a function:

   call_big_method(
   # lots of arguments, one to a line, with comments
   arg = combined, # Look up to see what combined is.
  # more arguments
 )

USING +
There's a suggestion, that instead one extends Python so that this works:
 arg = defaults + options # What does '+' mean here?

USING flow_update
Here's another suggestion. Instead write:
 dict_arg = defaults.copy().flow_update(options) # Is this clearer?

IMPLEMENTATION
Here's an implementation, as a subclass of dict.

 class mydict(dict):

 def flow_update(self, *argv, **kwargs):
 self.update(*argv, **kwargs)
 return self

 def copy(self):
 return self.__class__(self)

A DIRTY HACK
Not tested, using an assignment expression.
dict_arg = (tmp := defaults.copy(), tmp.update(options))[0]
Not recommend.



___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP: Dict addition and subtraction

[Jonathan Fine]

Michael Lee wrote:

> If your instinct is to assume "+" means "concatenation", then it would be 
> natural to assume that {"a": 1, "b": 2} + {"c": 3, "b": 4} would be identical 
> to {"a": 1, "b": 2, "c": 3, "b": 4} -- literally concat the key-value pairs 
> into a new dict.

> But of course, you can't have duplicate keys in Python. So, you would either 
> recall or look up how duplicate keys are handled when constructing a dict and 
> learn that the rule is that the right-most key wins. So the natural 
> conclusion is that "+" would follow this existing rule -- and you end up with 
> exactly the behavior described in the PEP.

This is a nice argument. And well presented. And it gave me surprise,
that taught me something. Here goes:

>>> {'a': 0}
{'a': 0}
>>> {'a': 0, 'a': 0}
{'a': 0}
>>> {'a': 0, 'a': 1}
{'a': 1}
>>> {'a': 1, 'a': 0}
{'a': 0}

This surprised me quite a bit. I was expecting to get an exception. However

>>> dict(a=0)
{'a': 0}
>>> dict(a=0, a=0)
SyntaxError: keyword argument repeated

does give an exception.

I wonder, is this behaviour of {'a': 0, 'a': 1} documented (or tested)
anywhere? I didn't find it in these URLs:
https://docs.python.org/3/library/stdtypes.html#mapping-types-dict
https://docs.python.org/3/tutorial/datastructures.html#dictionaries

I think this behaviour might give rise to gotchas. For example, if we
define inverse_f by
>>> inverse_f = { f(a): a, f(b): b }
then is the next statement always true (assuming a <> b)?
>>> inverse_f[ f(a) ] == a

Well, it's not true with these values
>>> a, b = 1, 2
>>> def f(n): pass # There's a bug here, f(n) should be a bijection.

A quick check that len(inverse) == 2 would provide a sanity check. Or
perhaps better, len(inverse_f) == len(set(a, b)). (I don't have an
example of this bug appearing 'in the wild'.)

Once again, I thank Michael for his nice, instructive and
well-presented example.
-- 
Jonathan
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP: Dict addition and subtraction

[Chris Angelico]

On Thu, Mar 7, 2019 at 12:59 AM Michael Lee  wrote:
> If your instinct is to assume "+" means "concatenation", then it would be 
> natural to assume that {"a": 1, "b": 2} + {"c": 3, "b": 4} would be identical 
> to {"a": 1, "b": 2, "c": 3, "b": 4} -- literally concat the key-value pairs 
> into a new dict.
>
> But of course, you can't have duplicate keys in Python. So, you would either 
> recall or look up how duplicate keys are handled when constructing a dict and 
> learn that the rule is that the right-most key wins. So the natural 
> conclusion is that "+" would follow this existing rule -- and you end up with 
> exactly the behavior described in the PEP.
>

Which, by the way, is also consistent with assignment:

d = {}; d["a"] = 1; d["b"] = 2; d["c"] = 3; d["b"] = 4

Rightmost one wins. It's the most logical behaviour.

ChrisA
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP: Dict addition and subtraction

[Michael Lee]

>
> I strongly agree with Ka-Ping. '+' is intuitively concatenation not
> merging. The behavior is overwhelmingly more similar to the '|' operator in
> sets (whether or not a user happens to know the historical implementation
> overlap).


I think the behavior proposed in the PEP makes sense whether you think of
"+" as meaning "concatenation" or "merging".

If your instinct is to assume "+" means "concatenation", then it would be
natural to assume that {"a": 1, "b": 2} + {"c": 3, "b": 4} would be
identical to {"a": 1, "b": 2, "c": 3, "b": 4} -- literally concat the
key-value pairs into a new dict.

But of course, you can't have duplicate keys in Python. So, you would
either recall or look up how duplicate keys are handled when constructing a
dict and learn that the rule is that the right-most key wins. So the
natural conclusion is that "+" would follow this existing rule -- and you
end up with exactly the behavior described in the PEP.

This also makes explaining the behavior of "d1 + d2" slightly easier than
explaining "d1 | d2". For the former, you can just say "d1 + d2 means we
concat the two dicts together" and stop there. You almost don't need to
explain the merging/right-most key wins behavior at all, since that
behavior is the only one consistent with the existing language rules.

In contrast, you *would* need to explain this with "d1 | d2": I would
mentally translate this expression to mean "take the union of these two
dicts" and there's no real way to deduce which key-value pair ends up in
the final dict given that framing. Why is it that key-value pairs in d2 win
over pairs in d1 here? That choice seems pretty arbitrary when you think of
this operation in terms of unions, rather than either concat or merge.

Using "|" would also violate an important existing property of unions: the
invariant "d1 | d2 == d2 | d1" is no longer true. As far as I'm aware, the
union operation is always taken to be commutative in math, and so I think
it's important that we preserve that property in Python. At the very least,
I think it's far more important to preserve commutativity of unions then it
is to preserve some of the invariants I've seen proposed above, like
"len(d1 + d2) == len(d1) + len(d2)".

Personally, I don't really have a strong opinion on this PEP, or the other
one I've seen proposed where we add a "d1.merge(d2, d3, ...)". But I do
know that I'm a strong -1 on adding set operations to dicts: it's not
possible to preserve the existing semantics of union (and intersection)
with dict and  think expressions like "d1 | d2" and "d1 & d2" would just be
confusing and misleading to encounter in the wild.

-- Michael



On Wed, Mar 6, 2019 at 4:53 AM David Mertz  wrote:

> I strongly agree with Ka-Ping. '+' is intuitively concatenation not
> merging. The behavior is overwhelmingly more similar to the '|' operator in
> sets (whether or not a user happens to know the historical implementation
> overlap).
>
> I think growing the full collection of set operations world be a pleasant
> addition to dicts. I think shoe-horning in plus would always be jarring to
> me.
>
> On Wed, Mar 6, 2019, 5:30 AM Ka-Ping Yee  wrote:
>
>> len(dict1 + dict2) does not equal len(dict1) + len(dict2), so using the +
>> operator is nonsense.
>>
>> len(dict1 + dict2) cannot even be computed by any expression
>> involving +.  Using len() to test the semantics of the operation is not
>> arbitrary; the fact that the sizes do not add is a defining quality of a
>> merge.  This is a merge, not an addition.  The proper analogy is to sets,
>> not lists.
>>
>> The operators should be |, &, and -, exactly as for sets, and the
>> behaviour defined with just three rules:
>>
>> 1. The keys of dict1 [op] dict2 are the elements of dict1.keys() [op]
>> dict2.keys().
>>
>> 2. The values of dict2 take priority over the values of dict1.
>>
>> 3. When either operand is a set, it is treated as a dict whose values are
>> None.
>>
>> This yields many useful operations and, most importantly, is simple to
>> explain.  "sets and dicts can |, &, -" takes up less space in your brain
>> than "sets can |, &, - but dicts can only + and -, where dict + is like set
>> |".
>>
>> merge and update some items:
>>
>> {'a': 1, 'b': 2} | {'b': 3, 'c': 4} => {'a': 1, 'b': 3, 'c': 4}
>>
>> pick some items:
>>
>> {'a': 1, 'b': 2} & {'b': 3, 'c': 4} => {'b': 3}
>>
>> remove some items:
>>
>> {'a': 1, 'b': 2} - {'b': 3, 'c': 4} => {'a': 1}
>>
>> reset values of some keys:
>>
>> {'a': 1, 'b': 2} | {'b', 'c'} => {'a': 1, 'b': None, 'c': None}
>>
>> ensure certain keys are present:
>>
>> {'b', 'c'} | {'a': 1, 'b': 2} => {'a': 1, 'b': 2, 'c': None}
>>
>> pick some items:
>>
>> {'b', 'c'} | {'a': 1, 'b': 2} => {'b': 2}
>>
>> remove some items:
>>
>> {'a': 1, 'b': 2} - {'b', 'c'} => {'a': 1}
>>
>> On Wed, Mar 6, 2019 at 1:51 AM Rémi Lapeyre 
>> wrote:
>>
>>> Le 6 mars 2019 à 10:26:15, Brice Parent
>>> (cont...@brice.xyz(mailto:cont...@brice.xyz)) a écrit:
>>>

Re: [Python-ideas] PEP: Dict addition and subtraction

[Brice Parent]

Le 06/03/2019 à 13:53, Chris Angelico a écrit :

On Wed, Mar 6, 2019 at 11:18 PM Brice Parent  wrote:

The major implication to such a
modification of the Dict.update method, is that when you're using it
with keyword arguments (by opposition to passing another dict/iterable
as positional), you're making a small non-backward compatible change in
that if in some code, someone was already using the keyword that would
be chosing (here "on_collision"), their code would be broken by the new
feature.
Anyway, if
the keyword is slected wisely, the collision case will almost never
happen, and be quite easy to correct if it ever happened.

You can make it unlikely, yes, but I'd dispute "easy to correct".
Let's suppose that someone had indeed used the chosen keyword (and
remember, the more descriptive the argument name, the more likely that
it'll be useful elsewhere and therefore have a collision). How would
they discover this? If they're really lucky, there MIGHT be an
exception (if on_collision accepts only a handful of keywords, and the
collision isn't one of them), but if your new feature is sufficiently
flexible, that might not happen. There'll just be incorrect behaviour.

As APIs go, using specific keyword args at the same time as **kw is a
bit odd. Consider:

button_options.update(button_info, on_click=frobnicate, style="KDE",
on_collision="replace")

It's definitely not obvious which of those will end up in the
dictionary and which won't. Big -1 from me on that change.
That's indeed a good point. Even if the correction is quite easy to make 
in most cases. With keyword only changes:


button_options.update(dict(on_click=frobnicate, style="KDE", on_collision="replace"))  # or 
button_options.update(dict(on_collision="replace"), on_click=frobnicate, style="KDE")

In the exact case you proposed, it could become a 2-liners:

button_options.update(button_info)
button_options.update(dict(on_click=frobnicate, style="KDE", 
on_collision="replace"))

In my code, I would probably make it into 2 lines, to make clear that we 
have 2 levels of data merging, one that is general (the first), and one 
that is specific to this use-case (as it's hard written in the code), 
but not everyone doesn't care about the number of lines.


But for the other part of your message, I 100% agree with you. The main 
problem with such a change is not (to me) that it can break some edge 
cases, but that it would potentially break them silently. And that, I 
agree, is worth a big -1 I guess.

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP: Dict addition and subtraction

[Chris Angelico]

On Wed, Mar 6, 2019 at 11:18 PM Brice Parent  wrote:
> The major implication to such a
> modification of the Dict.update method, is that when you're using it
> with keyword arguments (by opposition to passing another dict/iterable
> as positional), you're making a small non-backward compatible change in
> that if in some code, someone was already using the keyword that would
> be chosing (here "on_collision"), their code would be broken by the new
> feature.
> Anyway, if
> the keyword is slected wisely, the collision case will almost never
> happen, and be quite easy to correct if it ever happened.

You can make it unlikely, yes, but I'd dispute "easy to correct".
Let's suppose that someone had indeed used the chosen keyword (and
remember, the more descriptive the argument name, the more likely that
it'll be useful elsewhere and therefore have a collision). How would
they discover this? If they're really lucky, there MIGHT be an
exception (if on_collision accepts only a handful of keywords, and the
collision isn't one of them), but if your new feature is sufficiently
flexible, that might not happen. There'll just be incorrect behaviour.

As APIs go, using specific keyword args at the same time as **kw is a
bit odd. Consider:

button_options.update(button_info, on_click=frobnicate, style="KDE",
on_collision="replace")

It's definitely not obvious which of those will end up in the
dictionary and which won't. Big -1 from me on that change.

ChrisA
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP: Dict addition and subtraction

[Jonathan Fine]

Rhodri James wrote:

> Making assumptions about length where any dictionary
> manipulations are concerned seems unwise to me

I think you're a bit hasty here. Some assumptions are sensible. Suppose

a = len(d1)
b = len(d2)
c = len(d1 + d2) # Using the suggested syntax.

Then we know
   max(a, b) <= c <= a + b

And this is, in broad terms, characteristic of merge operations.
-- 
Jonathan
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP: Dict addition and subtraction

[Jonathan Fine]

Ka-Ping Yee wrote:
>
> len(dict1 + dict2) does not equal len(dict1) + len(dict2), so using the + 
> operator is nonsense.
>
> len(dict1 + dict2) cannot even be computed by any expression involving +.  
> Using len() to test the semantics of the operation is not arbitrary; the fact 
> that the sizes do not add is a defining quality of a merge.  This is a merge, 
> not an addition.  The proper analogy is to sets, not lists.

For me, this comment is excellent. It neatly expresses the central
concern about this proposal. I think most us will agree that the
proposal is to use '+' to express a merge operation, namely update.
(There are other merge operations, when there are two values to
combine, such as taking the min or max of the two values.)

Certainly, many of the posts quite naturally use the word merge.
Indeed PEP 584 writes "This PEP suggests adding merge '+' and
difference '-' operators to the built-in dict class."

We would all agree that it would be obviously wrong to suggest adding
merge '-' and difference '+' operators. (Note: I've swapped '+' and
'-'.) And why? Because it is obviously wrong to use '-' to denote
merge, etc.

Some of us are also upset by the use of '+' to denote merge. By the
way, there is already a widespread symbol for merge. It appears on
many road signs. It looks like an upside down 'Y'. It even has merge
left and merge right versions.

Python already has operator symbols '+', '-', '*', '/' and so on. See
https://docs.python.org/3/reference/lexical_analysis.html#operators

Perhaps we should add a merge or update symbol to this list, so that
we don't overload to breaking point the humble '+' operator. Although
that would make Python a bit more like APL.

By the way, Pandas already has a merge operation, called merge, that
takes many parameters. I've only glanced at it.
https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.merge.html

-- 
Jonathan
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP: Dict addition and subtraction

[Josh Rosenberg]

On Wed, Mar 6, 2019 at 11:52 AM Rhodri James  wrote:

> On 06/03/2019 10:29, Ka-Ping Yee wrote:
> > len(dict1 + dict2) does not equal len(dict1) + len(dict2), so using the +
> > operator is nonsense.
>
> I'm sorry, but you're going to have to justify why this identity is
> important.  Making assumptions about length where any dictionary
> manipulations are concerned seems unwise to me, which makes a nonsense
> of your claim that this is nonsense :-)
>

It's not "nonsense" per se. If we were inventing programming languages in a
vacuum, you could say + can mean "arbitrary combination operator" and it
would be fine. But we're not in a vacuum; every major language that uses +
with general purpose containers uses it to mean element-wise addition or
concatenation, not just "merge". Concatenation is what imposes that
identity (and all the others people are defending, like no loss of input
values); you're taking a sequence of things, and shoving another sequence
of things on the end of it, preserving order and all values.

The argument here isn't that you *can't* make + do arbitrary merges that
don't adhere to these semantics. It's that adding yet a third meaning to +
(and it is a third meaning; it has no precedent in any existing type in
Python, nor in any other major language; even in the minor languages that
allow it, they use + for sets as well, so Python using + is making Python
itself internally inconsistent with the operators used for set), for
limited benefit.

- Josh Rosenberg
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP: Dict addition and subtraction

[Brice Parent]

Le 06/03/2019 à 10:50, Rémi Lapeyre a écrit :

Le 05/03/2019 à 23:40, Greg Ewing a écrit :

Steven D'Aprano wrote:

The question is, is [recursive merge] behaviour useful enough and
common enough to be built into dict itself?

I think not. It seems like just one possible way of merging
values out of many. I think it would be better to provide
a merge function or method that lets you specify a function
for merging values.


That's what this conversation led me to. I'm not against the addition
for the most general usage (and current PEP's describes the behaviour I
would expect before reading the doc), but for all other more specific
usages, where we intend any special or not-so-common behaviour, I'd go
with modifying Dict.update like this:

foo.update(bar, on_collision=updator) # Although I'm not a fan of the
keyword I used

This won’t be possible update() already takes keyword arguments:


foo = {}
bar = {'a': 1}
foo.update(bar, on_collision=lambda e: e)
foo

{'a': 1, 'on_collision':  at 0x10b8df598>}

I don't see that as a problem at all.
Having a function's signature containing a **kwargs doesn't disable to 
have explicit keyword arguments at the same time:
`def foo(bar="baz", **kwargs):` is perfectly valid, as well as `def 
spam(ham: Dict, eggs="blah", **kwargs):`, so `update(other, 
on_collision=None, **added) is too, no? The major implication to such a 
modification of the Dict.update method, is that when you're using it 
with keyword arguments (by opposition to passing another dict/iterable 
as positional), you're making a small non-backward compatible change in 
that if in some code, someone was already using the keyword that would 
be chosing (here "on_collision"), their code would be broken by the new 
feature.
I had never tried to pass a dict and kw arguments together, as it seemed 
to me that it wasn't supported (I would even have expected an exception 
to be raised), but it's probably my level of English that isn't high 
enough to get it right, or this part of the doc that doesn't describe 
well the full possible usage of the method (see here: 
https://docs.python.org/3/library/stdtypes.html#dict.update). Anyway, if 
the keyword is slected wisely, the collision case will almost never 
happen, and be quite easy to correct if it ever happened.


___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP: Dict addition and subtraction

[Rhodri James]

On 06/03/2019 10:29, Ka-Ping Yee wrote:

len(dict1 + dict2) does not equal len(dict1) + len(dict2), so using the +
operator is nonsense.


I'm sorry, but you're going to have to justify why this identity is 
important.  Making assumptions about length where any dictionary 
manipulations are concerned seems unwise to me, which makes a nonsense 
of your claim that this is nonsense :-)


--
Rhodri James *-* Kynesim Ltd
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP: Dict addition and subtraction

[Ka-Ping Yee]

len(dict1 + dict2) does not equal len(dict1) + len(dict2), so using the +
operator is nonsense.

len(dict1 + dict2) cannot even be computed by any expression involving +.
Using len() to test the semantics of the operation is not arbitrary; the
fact that the sizes do not add is a defining quality of a merge.  This is a
merge, not an addition.  The proper analogy is to sets, not lists.

The operators should be |, &, and -, exactly as for sets, and the behaviour
defined with just three rules:

1. The keys of dict1 [op] dict2 are the elements of dict1.keys() [op]
dict2.keys().

2. The values of dict2 take priority over the values of dict1.

3. When either operand is a set, it is treated as a dict whose values are
None.

This yields many useful operations and, most importantly, is simple to
explain.  "sets and dicts can |, &, -" takes up less space in your brain
than "sets can |, &, - but dicts can only + and -, where dict + is like set
|".

merge and update some items:

{'a': 1, 'b': 2} | {'b': 3, 'c': 4} => {'a': 1, 'b': 3, 'c': 4}

pick some items:

{'a': 1, 'b': 2} & {'b': 3, 'c': 4} => {'b': 3}

remove some items:

{'a': 1, 'b': 2} - {'b': 3, 'c': 4} => {'a': 1}

reset values of some keys:

{'a': 1, 'b': 2} | {'b', 'c'} => {'a': 1, 'b': None, 'c': None}

ensure certain keys are present:

{'b', 'c'} | {'a': 1, 'b': 2} => {'a': 1, 'b': 2, 'c': None}

pick some items:

{'b', 'c'} | {'a': 1, 'b': 2} => {'b': 2}

remove some items:

{'a': 1, 'b': 2} - {'b', 'c'} => {'a': 1}

On Wed, Mar 6, 2019 at 1:51 AM Rémi Lapeyre  wrote:

> Le 6 mars 2019 à 10:26:15, Brice Parent
> (cont...@brice.xyz(mailto:cont...@brice.xyz)) a écrit:
>
> >
> > Le 05/03/2019 à 23:40, Greg Ewing a écrit :
> > > Steven D'Aprano wrote:
> > >> The question is, is [recursive merge] behaviour useful enough and
> > > > common enough to be built into dict itself?
> > >
> > > I think not. It seems like just one possible way of merging
> > > values out of many. I think it would be better to provide
> > > a merge function or method that lets you specify a function
> > > for merging values.
> > >
> > That's what this conversation led me to. I'm not against the addition
> > for the most general usage (and current PEP's describes the behaviour I
> > would expect before reading the doc), but for all other more specific
> > usages, where we intend any special or not-so-common behaviour, I'd go
> > with modifying Dict.update like this:
> >
> > foo.update(bar, on_collision=updator) # Although I'm not a fan of the
> > keyword I used
>
> Le 6 mars 2019 à 10:26:15, Brice Parent
> (cont...@brice.xyz(mailto:cont...@brice.xyz)) a écrit:
>
> >
> > Le 05/03/2019 à 23:40, Greg Ewing a écrit :
> > > Steven D'Aprano wrote:
> > >> The question is, is [recursive merge] behaviour useful enough and
> > > > common enough to be built into dict itself?
> > >
> > > I think not. It seems like just one possible way of merging
> > > values out of many. I think it would be better to provide
> > > a merge function or method that lets you specify a function
> > > for merging values.
> > >
> > That's what this conversation led me to. I'm not against the addition
> > for the most general usage (and current PEP's describes the behaviour I
> > would expect before reading the doc), but for all other more specific
> > usages, where we intend any special or not-so-common behaviour, I'd go
> > with modifying Dict.update like this:
> >
> > foo.update(bar, on_collision=updator) # Although I'm not a fan of the
> > keyword I used
>
> This won’t be possible update() already takes keyword arguments:
>
> >>> foo = {}
> >>> bar = {'a': 1}
> >>> foo.update(bar, on_collision=lambda e: e)
> >>> foo
> {'a': 1, 'on_collision':  at 0x10b8df598>}
>
> > `updator` being a simple function like this one:
> >
> > def updator(updated, updator, key) -> Any:
> > if key == "related":
> > return updated[key].update(updator[key])
> >
> > if key == "tags":
> > return updated[key] + updator[key]
> >
> > if key in ["a", "b", "c"]: # Those
> > return updated[key]
> >
> > return updator[key]
> >
> > There's nothing here that couldn't be made today by using a custom
> > update function, but leaving the burden of checking for values that are
> > in both and actually inserting the new values to Python's language, and
> > keeping on our side only the parts that are specific to our use case,
> > makes in my opinion the code more readable, with fewer possible bugs and
> > possibly better optimization.
> >
> >
> > ___
> > Python-ideas mailing list
> > Python-ideas@python.org
> > https://mail.python.org/mailman/listinfo/python-ideas
> > Code of Conduct: http://python.org/psf/codeofconduct/
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___

Re: [Python-ideas] PEP: Dict addition and subtraction

[Rémi Lapeyre]

Le 6 mars 2019 à 10:26:15, Brice Parent
(cont...@brice.xyz(mailto:cont...@brice.xyz)) a écrit:

>
> Le 05/03/2019 à 23:40, Greg Ewing a écrit :
> > Steven D'Aprano wrote:
> >> The question is, is [recursive merge] behaviour useful enough and
> > > common enough to be built into dict itself?
> >
> > I think not. It seems like just one possible way of merging
> > values out of many. I think it would be better to provide
> > a merge function or method that lets you specify a function
> > for merging values.
> >
> That's what this conversation led me to. I'm not against the addition
> for the most general usage (and current PEP's describes the behaviour I
> would expect before reading the doc), but for all other more specific
> usages, where we intend any special or not-so-common behaviour, I'd go
> with modifying Dict.update like this:
>
> foo.update(bar, on_collision=updator) # Although I'm not a fan of the
> keyword I used

Le 6 mars 2019 à 10:26:15, Brice Parent
(cont...@brice.xyz(mailto:cont...@brice.xyz)) a écrit:

>
> Le 05/03/2019 à 23:40, Greg Ewing a écrit :
> > Steven D'Aprano wrote:
> >> The question is, is [recursive merge] behaviour useful enough and
> > > common enough to be built into dict itself?
> >
> > I think not. It seems like just one possible way of merging
> > values out of many. I think it would be better to provide
> > a merge function or method that lets you specify a function
> > for merging values.
> >
> That's what this conversation led me to. I'm not against the addition
> for the most general usage (and current PEP's describes the behaviour I
> would expect before reading the doc), but for all other more specific
> usages, where we intend any special or not-so-common behaviour, I'd go
> with modifying Dict.update like this:
>
> foo.update(bar, on_collision=updator) # Although I'm not a fan of the
> keyword I used

This won’t be possible update() already takes keyword arguments:

>>> foo = {}
>>> bar = {'a': 1}
>>> foo.update(bar, on_collision=lambda e: e)
>>> foo
{'a': 1, 'on_collision':  at 0x10b8df598>}

> `updator` being a simple function like this one:
>
> def updator(updated, updator, key) -> Any:
> if key == "related":
> return updated[key].update(updator[key])
>
> if key == "tags":
> return updated[key] + updator[key]
>
> if key in ["a", "b", "c"]: # Those
> return updated[key]
>
> return updator[key]
>
> There's nothing here that couldn't be made today by using a custom
> update function, but leaving the burden of checking for values that are
> in both and actually inserting the new values to Python's language, and
> keeping on our side only the parts that are specific to our use case,
> makes in my opinion the code more readable, with fewer possible bugs and
> possibly better optimization.
>
>
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP: Dict addition and subtraction

[Brice Parent]

Le 05/03/2019 à 23:40, Greg Ewing a écrit :

Steven D'Aprano wrote:

The question is, is [recursive merge] behaviour useful enough and

> common enough to be built into dict itself?

I think not. It seems like just one possible way of merging
values out of many. I think it would be better to provide
a merge function or method that lets you specify a function
for merging values.

That's what this conversation led me to. I'm not against the addition 
for the most general usage (and current PEP's describes the behaviour I 
would expect before reading the doc), but for all other more specific 
usages, where we intend any special or not-so-common behaviour, I'd go 
with modifying Dict.update like this:


foo.update(bar, on_collision=updator)  # Although I'm not a fan of the 
keyword I used


`updator` being a simple function like this one:

def updator(updated, updator, key) -> Any:
    if key == "related":
    return updated[key].update(updator[key])

    if key == "tags":
    return updated[key] + updator[key]

    if key in ["a", "b", "c"]:  # Those
    return updated[key]

    return updator[key]

There's nothing here that couldn't be made today by using a custom 
update function, but leaving the burden of checking for values that are 
in both and actually inserting the new values to Python's language, and 
keeping on our side only the parts that are specific to our use case, 
makes in my opinion the code more readable, with fewer possible bugs and 
possibly better optimization.



___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP: Dict addition and subtraction

[Stefan Behnel]

INADA Naoki schrieb am 05.03.19 um 08:03:> On Tue, Mar 5, 2019 at 12:02 AM
Stefan Behnel wrote:
>> INADA Naoki schrieb am 04.03.19 um 11:15:
>>> Why statement is not enough?
>> 
>> I'm not sure I understand why you're asking this, but a statement is 
>> "not enough" because it's a statement and not an expression. It does 
>> not replace the convenience of an expression.
> 
> It seems tautology and say nothing.

That's close to what I thought when I read your question. :)


> What is "convenience of an expression"?

It's the convenience of being able to write an expression that generates
the thing you need, rather than having to split code into statements that
create it step by step before you can use it.

Think of comprehensions versus for-loops. Comprehensions are expressions
that don't add anything to the language that a for-loop cannot achieve.
Still, everyone uses them because they are extremely convenient.


> Is it needed to make Python more readable language?

No, just like comprehensions, it's not "needed". It's just convenient.


> Anyway, If "there is expression" is the main reason for this proposal, 
> symbolic operator is not necessary.

As said, "needed" is not the right word. Being able to use a decorator
closes a gap in the language. Just like list comprehensions fit generator
expressions and vice versa. There is no "need" for being able to write

[x**2 for x in seq]
{x**2 for x in seq}

when you can equally well write

list(x**2 for x in seq)
set(x**2 for x in seq)

But I certainly wouldn't complain about that redundancy in the language.


> `new = d1.updated(d2)` or `new = dict.merge(d1, d2)` are enough. Python 
> preferred name over symbol in general. Symbols are readable and 
> understandable only when it has good math metaphor.
> 
> Sets has symbol operator because it is well known in set in math, not 
> because set is frequently used.
> 
> In case of dict, there is no simple metaphor in math.

So then, if "list+list" and "tuple+tuple" wasn't available through an
operator, would you also reject the idea of adding it, argueing that we
could use this:

L = L1.extended(L2)

I honestly do not see the math relation in concatenation via "+".

But, given that "+" and "|" already have the meaning of "merging two
containers into one" in Python, I think it makes sense to allow that also
for dicts.


> It just cryptic and hard to Google.

I honestly doubt that it's something people would have to search for any
more than they have to search for the list "+" operation. My guess is that
it's pretty much what most people would try first when they have the need
to merge two dicts, and only failing that, they would start a web search.

In comparison, very few users would be able to come up with "{**d1, **d2}"
on their own, or even "d1.updated(d2)".

My point is, given the current language, "dict+dict" is a gap that is worth
closing.

Stefan

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/