[Python-ideas] Re: Awaiting until a condition is met

[Kyle Lahnakoski]

On 2022-05-14 10:10 p.m., Aaron Fink wrote:

start_moving_to(dest)
await lambda: distance_to(dest) <= tolerance



Instead of evaluating an expression you could build a notification chain 
that you await upon.


x = Variable(value=2)
y = Variable(value=2)

# we create a Future that watches x and y
condition = sqrt(x * x + y * y) <= 1

# some other task is launched to do work

await condition
print("done")



Here is an example implementation,

import asyncio
import math
import operator
import random
from asyncio import Future


class Variable(object):
def __init__(self, value=None):
self._value = value
self.listeners = []

@property
def value(self):
return self._value

@value.setter
def value(self, value):
self._value = value
for l in self.listeners:
l.update()

def __mul__(self, other):
return Function(operator.mul, self, other)

def __add__(self, other):
return Function(operator.add, self, other)

def __le__(self, other):
return Boolean(operator.le, self, other)

def __lt__(self, other):
return Boolean(operator.lt, self, other)


class Function(Variable):
def __init__(self, func, *params):
Variable.__init__(self)
self.func = func
self.params = []
for p in params:
if not isinstance(p, Variable):
p = Variable(p)
self.params.append(p)
p.listeners.append(self)
self.update()

def update(self):
self.value = self.func(*[p.value for p in self.params])


class Boolean(Future, Function):
def __init__(self, func, *params):
Function.__init__(self, func, *params)
Future.__init__(self)

def update(self):
Function.update(self)
if self.value and not self.done():
self.set_result(self.value)

def __bool__(self):
return self._value


def sqrt(value):
return Function(math.sqrt, value)


x = Variable(2)
y = Variable(2)

condition = sqrt(x * x + y * y) < 1


async def worker():
while not condition:
await asyncio.sleep(1)
if random.Random().randrange(2):
x.value = max(x.value-1, 0)
else:
y.value = max(y.value-1, 0)
print(f"{x.value}, {y.value}")


async def watch():
await condition
print("watch done")


async def main():
loop = asyncio.get_event_loop()
t1 = loop.create_task(watch())
t2 = loop.create_task(worker())
await asyncio.wait([t1, t2])

loop = asyncio.get_event_loop()
loop.run_until_complete(main())
loop.close()
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/EP73EWIAGAXDLCMVFYVPPCNVUZMX7NAM/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 671: Syntax for late-bound function argument defaults

[Kyle Lahnakoski]

On 2021-10-25 3:31 p.m., Mike Miller wrote:
> "defer" please.
>
> This construct did not happen in the past, and it's shorter of course.
>
> -Mike
>


I also like `defer`:

> def range(a, min=0, max = defer len(a)):
> return a[min:max]

`default` is also nice:

> def range(a, min=0, max default len(a)):
> return a[min:max]


I was concerned with this proposal at first because an inner function 
definition may be ambiguous:


> def do_work():
> a = ['this', 'is', 'a', 'list']
> def range(a, min=0, max = defer len(a)):
> return a[min:max]

which `a` does `len(a)` refer to?

Looking through my code, it seems this is not a problem; outer method 
variables rarely conflict with inner method parameters in practice.


Can deferred defaults refer to variables in scope?  Can I use this to 
evaluate arguments lazily?



> def coalesce(a, b):
> def _coalesce(x = defer a(), y = defer b()):
> if x is None:
> return x
> return y
> _coalesce()
>
>
> def expensive_method():
> return 84
>
>
> print(coalesce(lambda: 42, expensive_method))


Thank you






___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/OM7QIRQBGIGQX5RVN675NVESMYLGEXHJ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: What about having a .get(index, default) method for arrays like we have for dicts?

[Kyle Lahnakoski]

On 2020-06-27 09:34, Daniel. wrote:

When I need to traverse nested dicts, is a common pattern to do

somedict.get('foo', {}).get('bar', {})

But there is no such equivalent for arrays, wouldn't be nice if we can 
follow


somedict.get('foo', {}).get('bar', []).get(10) ... ?

What do you think?



I would use this. I do something similar already, albeit as a set of 
classes that wrap around Python `dict` and `list` to provide the 
null-safe access.


to_data(somedict).foo.bar[10]

Specifically, I wrap `list` in `FlatList`, which will return `Null` 
(null-safe version of `None`) instead of raising and IndexError.  This 
allows math on indexes without concern for corner cases, and makes 
window functions easier to write:


|    my_list = FlatList(my_list)
|    deltas = []
|    for i, this_week in enumerate(my_list):
|    last_week = my_list[i-7]
|    deltas.append(this_week - last_week)

by avoiding exception handling, code is simplified, and procedures 
simplify to functions.  Instead of code that constructs a list; the code 
reads as a list definition:


|    deltas = [
|    this_week - last_week
|    for my_list in [FlatList(my_list)]
|    for i, this_week in enumerate(my_list)
|    for last_week in [my_list[i-7]]
|    ]

> please forgive me:  where `for x in [y]` can be read as `x=y`  (I am 
still hesitant to use `:=`)


There are detriments to using a set of null-safe objects:

1. data structures can be polluted with a combination of null-safe 
objects and regular Python structures


2. with null-safe code, there are more nulls and it seems EVERYTHING 
must be null-safe, including arithmetic (notice null-safe subtraction above)


3. runs slower; all the null-safe checks dominate my profiler time.


I would be nice if Python had a series of null-safe operators. Until 
then, `list.get` would eliminate my extra object creation:


|    deltas = [
|    this_week - (last_week or 0)
|    for i, this_week in enumerate(my_list)
|    for last_week in [my_list.get(i-7)]
|    ]

Although we see additional null-checks required (as #2 states).

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/UKU2PN43EQZ4ZWVDFXR6XO4ST3WBCNVE/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Keyword arguments self-assignment

[Kyle Lahnakoski]

On 2020-04-19 07:23, Richard Damon wrote:

There is also the issue that if we are building a function that might be
used with another function, we will have an incentive to name our
keyword parameters that there is a reasonable chance would also be
passed to that other function with the same keyword name, even if that
might not be the most descriptive name.


Many of the function keyword parameters I deal with are data property 
names; so it makes sense that the data has the same name throughout the 
codebase.  The incentive to align our variable names would be a good 
thing.  Consider pymysql, and the connect parameters


>    connect(
>    host=host,
>        port=port,
>        user=username,
>        passwd=password
>    )

With the proposal, right, or wrong, there would be an incentive for me 
to write the caller to use pymysql property names, and the callers of 
that caller to also use the same property names. This will spread until 
the application has a standard name for username and password: There is 
less guessing about the property names.  I have done this in ES6 code, 
and it looks nice.


Maybe aligning variable names with function keyword parameteres is an 
anti-pattern, but I have not seen it.


I reviewed my code:  of 20,360 keyword arguments, 804 (4%) are have the 
x=x format.  I do not know if this is enough to justify such a proposal, 
but I would suggest that is a minimum:  Currently there is no incentive 
to have identical names for identical things through a call chain;  an 
incentive will only increase the use of this pattern.


___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/UQGVKME6PO3RYJE4S2BTRB7L66UB4B6B/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Copying a multiple values of a dict to another with a lot easier syntax

[Kyle Lahnakoski]

On 2019-10-21 10:44, gedizgu...@gmail.com wrote:

m and n are lists or dicts or enumerates or classes or anything it can be 
assigned like following:

instead of :

m.a=n.a;
m.b=n.b;
m.c=n.c;
...

I suggest:

a,b,c of m to n ;


Interesting.  I also saw this type of redundancy in my code. Instead of 
making a function, like `copy_attrs()` I changed my APIs to accept 
over-specified attributes.  Instead, I would write


m = n

... or simply use `n` where `m` was expected.  Yes, `n` would have more 
attributes than needed, but that is solved by ensuring the receiver can 
gracefully reject the extra attributes; I decorate pedantic functions 
that complain about extra parameters.


In your other example, I would write:

adiv=document.createElement("div")
adiv.style = comp

There is the problem of unfortunate namespace overlap; where the 
destination expects some attribute X and the source has an attribute of 
the same name, X,  but the value is wrong.  But this is not a large 
problem: You can plan your attribute names on a program-wide scale so 
that the same attribute name means the same thing across your whole 
program, and different attribute names mean different things across your 
whole program.  This does not require too much forward planning, rather 
more refactoring as you rename parameters to match, or differ, as 
needed.  This is what the ES6 structuring/destructuring is  promoting. 
Consider a function that uses `adiv`:


drawBox = (adiv) => {
    const {x, y, height, width} = adiv.style;
}

the extra attributes of `comp` are ignored; specific attribute copying 
is not required.



___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/FY24FAGNQAGTA5CRZCI2WWEGF5RQPPPD/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Allow kwargs in __{get|set|del|}item__

[Kyle Lahnakoski]

On 2019-10-07 20:35, Steven D'Aprano wrote:


As per Caleb's initial post, this is how Pandas currently does it:

 db[db['x'] == 1]

Replacing that with db[x=1] seems like a HUGE win to me.

Even db[{'x': 1}] is pretty clunky.


For Pandas, db['x'] which creates an expression involving `x`, and can 
be used in more than one place.  It is small matter to start your method 
(or class, or module) with some declarations:


x = db['x']

which is boilerplate, but also easily ignored.  Then you can write

db[x==1]

Which is just as clean, and more specific, than whatever `db[x=1]` 
means.  I do not believe this syntax will help Pandas because equality 
is just one-of-many useful operators.  This does not help me if I want 
to say


db[x>1]

or say

db[x==1 || t > 2]

I am not sure how many expressions are actually found in realworld 
code.  Examples are written out, but most code is not examples.  Most 
code, that I have seen, manipulates expressions that come from 
elsewhere.  Here is an Elasticsearch expression:


e = {"term": {"x": 1}}

which is not seen in the code; assignment to `e` is done elsewhere, 
maybe from a config file, maybe from another application.  The code only 
sees `e`.  Therefore an example like


db[db['x'] == 1]

looks like

db[e]

in realworld code.




___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/36SKB65DI64CUTS277ICCUDDKXZSZJTG/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Set operations with Lists

[Kyle Lahnakoski]

On 2019-09-20 9:52 p.m., Richard Higginbotham wrote:

Andrew Barnert wrote:

set(b).intersection(a) or set(a) & set(b) or sb =
set(b) then [x for x in a if x in sb] and you’re done. They can easily
understand why it works. If they want to know why it’s faster, you can easily 
explain it,
and they’ve learned something widely useful.

This isn't technically correct. It's not faster. It all depends on the use case which when it 
contradicts your expectations you just deride as "artifical micro benchmarks". Python 
isn't just used as a toy scripting language. 50 million elements in a collection is not even large 
by a long shot at least from where I sit. You can make a case that it's not a good language for 
that type of problem, say HPC clusters. Or you can tell people to go copy some C code "like 
people have been doing for decades". That you ask if that is a proper response to your users 
is very concerning to me.



Richard,

I can identify with the need to intersect large sets; and I often faced 
with poor choices when it comes to dealing with them.  I love Python, 
and I would love to use it for my data munging, but the data is too big 
to fit in memory and Python is too slow.  I find it disappointing to 
convert an elegant expression declaring *what* I want, like "a-b", into 
code that declares *how* to do it, like "set(b).intersection(a)".  This 
disappointment is magnified by the fact someone, somewhere already 
written code to do set subtraction faster, and I can not use it; my 
choices are quick ugly code, or a time consuming search for an elegant 
solution** on the internet.


Maybe we are looking for a different type of solution.  You seem to be 
asking for the elegance of Python with the expression optimization (aka 
query planning) of a database.  A database may be able to deliver the 
speeds you require; it can pack data tighter; it has access to more 
processors; it may already leverage SIMD; it can optimize the operation 
according to how big the data is.  I am suggesting a Python container 
implementation, like sortedcontainers but using a database (maybe 
Sqlite), may be a solution.


Of course, there is the problem of moving data in and out of the 
database, but maybe that can be amortized over other operations, and 
made relatively insignificant.  There is the delay when translating 
__sub_() call into SQL, but maybe that is  relatively small compared to 
size of work we are requesting.


May you link to your repo where these tests are run?  I seem to have 
lost the link in the long chain of emails on this list.  I am 
considering adding a Sqlite example, if only to prove to myself that it 
is the slowest option of all.


Thank you.


** The sortedcontainers mentioned is an interesting demonstration of how 
fast Python can get when you are aware of L1 cache effects.




___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/2TBY7XJAWWMPWJ7NUBMRYLA7KIS5HOP6/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Exceptions with Message Templates

[Kyle Lahnakoski]

On 2019-08-08 11:52, Ryan Fox wrote:
My proposal is a new exception class as the preferred base for 
user-defined exceptions:


>>> class MyException(ExceptionTemplate):
...    message = 'Bad thing happened during {action} in {context}'
>>> raise MyException(action=current_action, context=current_context)


The `message` string and the `MyException` class are 1-1; maybe you can 
remove more boilerplate and do it in one step:


> raise ExceptionUsingTemplate(
>     'Bad thing happened during {action} in {context}',
>     action=current_action,
>     context=current_context
> )

Plus, give it a shorter name:

> error(
>     'Bad thing happened during {action} in {context}',
>     action=current_action,
>     context=current_context
> )

You still have the "[templates] copied from place to place" problem; in 
those cases you raise the same type of error in many different 
locations, you can define a constant, and that constant represents the 
exception class:


> BAD_THING_HAPPENED = 'Bad thing happened during {action} in {context}'
> ...
> error(
>     BAD_THING_HAPPENED,
>     action=current_action,
>     context=current_context
> )

When catching exceptions, my code rarely cares about the specific type 
of exception, rather it only cares if an exception was raised.  But in a 
few rare cases the exception handler is discerning:


> except Exception as e:
> if BAD_THING_HAPPENED in e:
>     # special case
> else:
> # all the other ways this failed
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/GN3ZCYWPRYGZSBXM67ZGVQGVM4I2XVMD/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Operator as first class citizens -- like in scala -- or yet another new operator?

[Kyle Lahnakoski]

Correct me if I am wrong: Yanghao would like an elegant way to build
graphs. Simply using << to declare the connections in the graph [1] is
not an option because << is already needed for legitimate left-shift
operation.  The problem is not assignment; rather, Yanghao's HDL
requires more operators than Python has in total (or at least the
remaining operators look terrible).

Here is a half baked idea:

Instead of solving the problem of operator-needed-for-use-case-X, can we
solve the bigger problem of adding user-defined operators?

"User-defined operators" are a limited set of operators, with no
preconceived definition, and can be defined by writing the appropriate
dunder method.  User-defined operators match /[!@$%^-+=&*:<>?/|]{2,4}/
and necessarily exclude all the existing python-defined operator
combinations.  Maybe 50K operators is easier to add to Python than a few
pre-defined operators?

Example:

> class A:
>     def assign(self, other):
>     # whatever this means

To add a user defined operator to a class, you add it as a method,
exactly the name of the operator:

> setattr(A, "<==", A.assign)

I admit this is a bit ugly, but some decorator can make this nicer:

> class A:
> @operator("<==")
>     def assign(self, other):
>     # whatever this means

Maybe the biggest problem would be with linting; not being able to
distinguish between a typo and a user-defined operator until runtime,
when no implementation is found.

A ++= B  # is this typo?

Abusive use of operators will likely happen; but the codebases that do
will not be popular because of the learning curve. Instead, experiments
will be done, some will succeed, and the best operators for each domain
will become popular.  Then the python devs can pick the winners to
include in the core library, if any.

[1] Here is toy example that will build a graph using << and >>:
https://github.com/klahnakoski/graph-builder/blob/master/graph_builder.py


On 2019-06-04 08:28, Steven D'Aprano wrote:
> On Fri, May 31, 2019 at 02:48:24PM +0100, Rhodri James wrote:
>> On 29/05/2019 08:31, Yanghao Hua wrote:
>>> Python does not need to know this ... just hand it over to end user
>>> who knows how to implement such a thing. Python need to provide the
>>> mechanism.
>> It really doesn't.  If the end user is going to implement the logic of 
>> this anyway, implementing signal linkage as a method call or class all 
>> of its own is not a significant extra burden.
> That's not really fair: syntax matters, and for people working in a 
> domain where certain syntax is expected, asking them to use something 
> different is a significant cognitive burden. We don't ask people doing 
> arithmetic to write code like this:
>
> x.mul(y.sub(1))  # a * (y - 1)
>
> There have been at least two times that Python has added syntax to the 
> language to allow a third-party library to write more-idiomatic code in 
> their domain. Both extended slicing seq[a:b:c] and the matrix- 
> multiplication operator were added for numpy.
>
> So *in principle* we certainly could add a new arrow operator for 
> Yanghao Hua so his libraries and code will be more expressive and 
> idiomatic in his domain.
>
> But *in practice*, the hard truth is this:
>
> - Yanghao Hua is one developer interested in HDL in Python;
>
> - numpy was, and still is, one of the most important "killer apps"
>   responsible for Python's success.
>
> Adding syntax for the benefit of numpy helps millions of users; adding 
> syntax for HDL-like helps... how many people? Twenty? A hundred?
>
> Millions of users will have to learn the syntax. Unless they get some 
> benefit, why consider this?
>
> (But having said that... I reckon that if we had left and right arrow 
> operators, <== and ==>, I have some DSLs where they would work for me 
> too. Maybe.)
>
>
>> I'm pretty much done with this conversation too.  You have repeatedly 
>> been asked what problem you are trying to solve and repeatedly respond 
>> by restating your solution, which appears to be to impose HDL sematics 
>> onto a non-HDL language.  That's never going to be a good idea.
> That's totally unfair to Yanghao Hua on two levels.
>
> (1) He has never asked for the interpreter to support the semantics he 
> wants. He's not asking for Python to understand and implement HDL 
> semantics in the language: he can do that himself, in the class.
>
> (2) He has explained the problem he is trying to solve: he wants to 
> write a DSL using syntax which doesn't look like crap from the 
> perspective of people in that domain.
>
> He just wants an operator that looks kinda like assignment that calls a 
> dunder. He can't use assignment because the semantics are baked into the 
> language (and there's no chance of that changing). He can't use existing 
> operators because they're already in use.
>
> This is not an unreasonable request. Python has bowed to similar 
> requests at least twice before, and honestly, it's not like a <== 
> operator would be weirder than 

Re: [Python-ideas] Trigonometry in degrees

[Kyle Lahnakoski]

Maybe a library of trig functions that include an Angle type? This is
related to the timespan units discussion we just had


def sin(angle):


On 2018-06-08 01:44, Yuval Greenfield wrote:
> On Thu, Jun 7, 2018 at 10:38 PM Stephen J. Turnbull
>  > wrote:
>
>
> 6.123233995736766e-17
> >>>
>
> is good enough for government work, including at the local public high
> school.
>
>
> There probably is room for a library like "fractions" that represents
> multiples of pi or degrees precisely. I'm not sure how complicated or
> valuable of an endeavor that would be. But while I agree that floating
> point is good enough, we probably can do better.
>  
>
>
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Type hints for functions with side-effects and for functions raising exceptions

[Kyle Lahnakoski]

On 2019-02-22 17:20, Chris Angelico wrote:
> On Sat, Feb 23, 2019 at 9:14 AM Kyle Lahnakoski  
> wrote:
>> Can Python provide better support for the CNCR pattern? If it is
>> lightweight enough, maybe people will use it, and then we can say
>> something useful about the (restricted) range of exceptions coming from
>> a method:
> The CNCR pattern, if used repeatedly, will quickly create a long chain
> of exceptions, where each exception represents one function call.
> Python already has very good support for seeing the function call
> history that led to the exception - it's called a traceback. You even
> get the full function locals as part of the exception object... and it
> requires no code whatsoever! Simply allowing exceptions to bubble up
> will have practically the same benefit.

I like your point that the exception chain is long, and similar to the
stack: I do find that in practice. This may indicate there is an
optimization in how CNCR can be handled:  Maybe instantiation of CNCR
exceptions can be deferred to the point where an exception handler
actually does something beyond CNCR, if ever.

I also agree, with a debugger at hand, we can inspect the stack trace.
We can also write code that reflects on the method names in the stack
trace to figure out how to handle an exception. But, I am concerned that
stack trace inspection is brittle because methods can be renamed and
refactored.  Also, the CNCR pattern is not 1-1 with methods, there can
be more than one type of CNCR exception emitted from a method.  Methods
could be split, so each only throws one type of exception; and then the
stack trace would suffice; but that brings us back to brittle: A split
method may be accidentally merged for clarity at a later time.

We should also consider what happens in the case that an exception chain
is not handled: It may be printed to the log: We can not reasonably
print all the locals; there are many, some do not serialize, and some
are sensitive.  CNCR is being explicit about what locals(), if any, are
important.



___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Type hints for functions with side-effects and for functions raising exceptions

[Kyle Lahnakoski]

On 2019-02-21 03:09, Christopher Barker wrote:
>
> But yes, there is no (easy) way to distinguish an Exception raised by
the function you called, and one raised somewhere deeper that.
>
> And I have been bitten by that more than once. It makes "Easier to ask
forgiveness than permission" kind of tricky.

> And Exception handling is messy -- the point made by the OP, I'm not
sure there's a better way to do it.


It seems to me that exception *classes* are hardly ever required. If two
methods raise the same exception type, then I contend that they are
actually throwing two different types of errors (as evidenced by the
difference in the message). Instead, I suggest every method emits its
own exception type.

By demanding support for a plethora of exception types, at least as many
as there are methods, then we may have a different way of looking at
exceptions: The first conclusion is, it is too expensive to make classes
for every exception type, rather, exception type should be defined by
the message.


Exception handling is messy. I find most of my Python methods look like:

    | def my_method():
    | try:
    | # do something
    | except Exception as e:
    | error("some description", cause=e)

I am not using Python3 everywhere yet, so maybe it should read like:

    | def my_method():
    | try:
    | # do something
    | except Exception as e:
    | raise Exception("some description") from e


Let me call this pattern the Catch-It-Name-It-Chain-It-Raise-It (CNCR)
pattern  

There are a few reasons for this.

1. I can add runtime values to the exception so I get a better sense of
the program state without going to the debugger:  `error("some
description", {"url": url}, cause=e)`
2. I prevent exception leakage; I have no idea the diversity of
exceptions my code can raise, so I CNCR.
3. Every exception is it's own unique type; I can switch on the message
if I want (I rarely do this, but it happens, see below)

Can Python provide better support for the CNCR pattern? If it is
lightweight enough, maybe people will use it, and then we can say
something useful about the (restricted) range of exceptions coming from
a method:

A context manager, a `with` block, could do this:  

    | def my_method():
    | with Explanation("some description"):
    | # do something

I propose a simple line, which effectively defines a try block to the
end of the current code block. Call it the `on-raises` syntax:

    | def my_method():
    | on Exception raises "some description"
    | # do something

It is better than the `with` block because:

* it does not put code in the happy path
* it has less indentation
* we can conclude "some description" is the only exception raised by
this method  

The `on-raises` need not start at the beginning of a code block, but
then we can say less about what exceptions come out of `my_method`:

    | def my_method():
    | # no exception checks here
    | on Exception raises "some description"
    | # do something

Since `on-raises` can be used in any code block, we can save some
indentation. Instead of

    | def my_method():
    | with some_file:
    | try:
    | # do something
    | except Exception as e:
    | raise Exception("some description") from e

we have

    | def my_method():
    | with some_file:
    | on Exception raises "some description"
    | # do something

of course we can have nested `on-raises`,

    | def my_method():
    | on Exception raises "bigger problem"
    | with some_file:
    | on Exception raises "some description"
    | # do something

Plus we know only "bigger problem" exceptions can be raised.

The above is the same as:
    | def my_method():
    | try:
    | with some_file:
    | try:
    | # do something
    | except Exception as e:
    | raise Exception("some description") from e
    | except Exception as e:
    | raise Exception("bigger problem") from e

in the rare case we actually want to deal with an exception, we revert
back to trusty old try/except:

    | def my_other_method():
    | on Exception raises "some other description"
    | try:
    | my_method()
    | except "some description" as e:
    | # I know how to handle this case
    | return

which is the same as:

    | def my_other_method():
    | try:
    | my_method()
    | except Exception as e:
    | if "some description" in e:
    | # I know how to handle this case
    | return
    | error("some other description", cause=e)
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] add fluent operator to everything

[Kyle Lahnakoski]

On 2019-02-20 10:10, Steven D'Aprano wrote:
> Or if you're worried about the line length:
> result = function(mystr.strip()
>.expandtabs()
>.lower()
>.replace('ham', 'spam')
>   I think 


It seems that this fluency discussion, and the vector discussion is
similar; I made a toy class [1] to demonstrate. It is much like
DavidMertz's vector [2], but focused on chained methods .

The `vector()` method lets us enter "vector mode". There are methods
that act on elements (eg. map), methods that act on the whole (eg.
sort), and methods that exit vector mode (eg. list).

output = vector([3, 2, 1]).append(4).sort().limit(10).list()

Fluency  can be had by entering vector mode on a singleton list:

output = (
    vector([mystr])
    .strip()
    .expandtabs()
    .lower()
    .replace("ham", "spam")
    .map(function)
    .first()
)

Given vector() is quite succinct, and Numpy and Pandas do vector
operations elegantly already, I do not think there is need for
vectorized operators or fluency operators in Python.

[1] My toy class -
https://github.com/klahnakoski/mo-vector/blob/master/mo_vector/__init__.py

[2] DavidMertz vector 0-
https://github.com/DavidMertz/stringpy/blob/master/vector.py


___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Vectorization [was Re: Add list.join() please]

[Kyle Lahnakoski]

CHB,

Thank you! I had forgotten that discussion at the beginning of July [1]. 

Googling the list [2] also shows mention of PythonQL [3], which may
point to use cases that can guide a Vectorization idea.


[1] groupby discussion -
https://mail.python.org/pipermail/python-ideas/2018-July/051786.html

[2] google search -
https://www.google.ca/search?q=group+by+site%3Ahttps%3A%2F%2Fmail.python.org%2Fpipermail%2Fpython-ideas%2F=group+by+site%3Ahttps%3A%2F%2Fmail.python.org%2Fpipermail%2Fpython-ideas%2F

[3] PythonQL - https://github.com/pythonql/pythonql



On 2019-02-11 10:43, Christopher Barker wrote:
> Do take a look in the fairly recent archives of this list for a big
> discussion of groupby -- it kind of petered out but there were a
> couple options on the table.
>
> -CHB
>

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Vectorization [was Re: Add list.join() please]

[Kyle Lahnakoski]

On 2019-02-10 18:30, Steven D'Aprano wrote:
>
> Can you post a simplified example of how you would do it in SQL, 
> compared to what you would have to do in standard Python?

Can I do the same in standard Python? If I did, then I would use Pandas:
it has groupby, and some primitive joining, and window functions may
come naturally because of its imperative nature, but I have not tried
it.  If I can not use Pandas, then I would write the groupby and window
functions and call them in sequence. This is similar to what you see in
my code now: a number of properties who's values get dispatched to
Python functions.  My code is more complicated only because those
structures can be dispatched to translators for databases too.

I am certain there are many variations of groupby out in the wild, and
it would be nice to have the concept standardized when/if Python has
vector operations. Join would be nice to have too, but I do not use it
much; dictionary lookup seems to fill that need.  Window functions
(which are like mini queries) are powerful, but like Pandas, may end up
end up being free because Python is imperative.

My code I pointed to has two parts. Here is the first part in SQL (well,
an approximation of SQL since I did not test this, and now I am rusty). 
A detailed description is below

|   WITH time_range AS (
|   SELECT
|   num
|   FROM
|   all_integers
|   WHERE
|   num % 60 =0 AND
|   num >= floor(<>/60/60)*60*60-<> AND
|   num < floor(<>/60/60) + 60*60
|   )
|   SELECT 
|   availability_zone,
|   instance_type,
|   time_range.num AS time
|   MAX(price) as PRICE,
|   COUNT(1) AS `COUNT`,
|   LAST(current_price) OVER (
|   PARTITION BY
|   availability_zone,
|   instance_type
|   ORDER BY
|   timestamp
|   ) AS current_price
|   FROM
|   (
|   SELECT
|   *,
|   COALESCE(LAG(timestampvalue, 1), <>) OVER (
|   PARTITION BY
|   availability_zone,
|   instance_type
|   ORDER BY
|   timestamp
|   ) AS expire,
|   timestamp-<> AS effective
|   FROM
|   prices
|   ) temp
|   RIGHT JOIN
|   time_range ON time_range.num BETWEEN temp.effective AND temp.expire
|   GROUP BY
|   availability_zone,
|   instance_type,
|   time_range.num AS time
|   WHERE
|   expire > floor(<>/60/60)*60*60 - <>


Now, for the same, with description:

This WITH clause is not real SQL; it is meant to stand in for a
temporary table that contains all hours of the time range I am
interested. Definitely easier to do in Python. All time is assumed to be
in seconds since epoch.

|   WITH time_range AS (
|   SELECT
|   num
|   FROM
|   all_integers
|   WHERE
|   num % 60 =0 AND
|   num >= floor(<>/60/60)*60*60-<> AND
|   num < floor(<>/60/60) + 60*60
|   )

We will select the three dimensions we are interested in (see GROUP BY
below), along with the MAX price we have seen in the given hour, and the
current_price for any (availability_zone, instance_type) pair.

|   SELECT 
|   availability_zone,
|   instance_type,
|   time_range.num AS time
|   MAX(price) as PRICE,
|   COUNT(1) AS `COUNT`,
|   LAST(current_price) OVER (
|   PARTITION BY
|   availability_zone,
|   instance_type
|   ORDER BY
|   timestamp
|   ) AS current_price
|   FROM

The prices coming from Amazon only have a timestamp for when that price
is effective; so this sub-query adds an `effective` start time, and an
`expire` time so the rest of the query need only deal with ranges. The 
timestamp-<> is putting the start time back further
into the past so the past can "see" future pricing. 

|   (
|   SELECT
|   *,
|   COALESCE(LAG(timestamp, 1), <>) OVER (
|   PARTITION BY
|   availability_zone,
|   instance_type
|   ORDER BY
|   timestamp
|   ) AS expire,
|   timestamp-<> AS effective
|   FROM
|   prices
|   ) temp

This is the point where we use the time_range from above and find every
hour a price is effective.  This could have been a sub-query, but I am
rusty at SQL

|   RIGHT JOIN
|   time_range ON time_range.num BETWEEN temp.effective AND temp.expire

These are the three dimensions we are interested in

|   GROUP BY
|   availability_zone,
|   instance_type,
|   time_range.num AS time

and we are only interested in calculating back to a certain point

|   WHERE
|   expire > floor(<>/60/60)*60*60 - <>






___
Python-ideas mailing list
Python-ideas@python.org

Re: [Python-ideas] Vectorization [was Re: Add list.join() please]

[Kyle Lahnakoski]

On 2019-02-02 18:11, Steven D'Aprano wrote:
> We can improve that comprehension a tiny bit by splitting it into
> multiple steps:
>
>  temp1 = [d+e for d, e in zip(vector, sequence)]
>  temp2 = [process(c) for x in temp1]
>  result = [a*b for a, b in zip(temp2, items)]
>
> but none of these are as elegant or readable as the vectorized syntax
>
>  result = process.(vector .+ sequence) .* items

The following reads a little better:

| result = [
|     process(v+s)*i
| for v, s, i in zip(vector, sequence, items)
| ]

Vector operations will promote the use of data formats that work well
with vector operations. So, I would expect data to appear like rows in a
table, rather than in the columnar form shown above. Even if columnar
form must be dealt with, we can extend our Vector class (or whatever
abstraction you are using to enter vector space) to naturally zip() columns.

| Vector(zip(vector, sequence, items))
|     .map(lambda v, s, i: process(v+s)*i)    

If we let Vector represent a list of tuples instead of a list of values,
we can make construction simpler:

| Vector(vector, sequence, items)
|     .map(lambda v, s, i: process(v+s)*i)    

If we have zip() to extend the tuples in the Vector, then we can be
verbose to demonstrate how to use columnar data:

| Vector(vector)
| .zip(sequence)
| .map(operator.add)
| .map(process)
| .zip(items)
| .map(operator.mul)

This looks verbose, but it is not too far from the vectorized syntax:

the Vector() brings us to vector mode, and the two zip()s convert from
columnar form. This verbose form may be *better* than the vectorized
syntax because the operations are in order, rather than the mixing infix
and functional forms seen in the vectorized syntax form.

I suggest this discussion include vector operations on (frozen)
dicts/objects and (frozen) lists/tuples.  Then we can have an
interesting discussion about the meaning of group_by, join, and window
functions, plus other operations we find in database query languages.

I am interested in vector operations.  I have situations where I want to
perform some conceptually simple operations on a series of
not-defined-by-me objects to make a series of conclusions.  The
calculations can be done succinctly in SQL, but Python makes them
difficult. Right now, my solution is to describe the transformations in
JSON, and have an interpreter do the processing:

https://github.com/klahnakoski/SpotManager/blob/65f2c5743f3a9cfd1363cafec258c0a663e194c3/spot/spot_manager.py#L611



___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Implementing a set of operation (+, /, - *) on dict consistent with linearAlgebrae

[Kyle Lahnakoski]

Julien,

Personally, I would be able to use the module you are proposing to
accumulate arbitrarily-named measures. I can not think of a use case for
division, but it would be nice for completion.  I have made my own
library that implements a small part of what you propose [1].

I was looking through the pstats.py [2] source code; and I thought it
could benefit from vector operations.  I have seen other code that
collect measures have the same redundant pattern.  Maybe some fancy
regex can identify other += code sequences that would benefit.  If you
make a module, and show how it can simplify pstats.py, maybe you have a
winner? 

[1] "vector" addition? -
https://github.com/klahnakoski/mo-dots/blob/dev/tests/test_dot.py#L610

[2] pstats.py source code -
https://github.com/python/cpython/blob/3.7/Lib/pstats.py#L156


On 2018-10-30 11:31, julien tayon wrote:
> Hello :)
>
> the idea is described here:
> http://jul.github.io/cv/pres.html#printable
>
> Summary of the idea :
>
> Take a linear algebrae book, and implements all the rules as a TDD.
> https://github.com/jul/archery/blob/master/consistent_algebrae.py
>
> make it works based on abstract base class and sets of Mixins.
> https://archery.readthedocs.io/en/latest/
>
> And see if we can make cos/__abs__/dot and if it gives naively the intended
> results ? (spoiler: yes)
>
> Making it work with dict, and "other" dictionary like counter by using
> ineritance
> https://archery.readthedocs.io/en/latest/#advanced-usage
>
> My idea is : wouldn't it be nice if we introduced geometries as sets of
> mixins for objects ?
> (Hilbertian algebrae could be nice too, and we could make MutableMapping
> behave like bra/kets).
>
> So I was proposing a soft discussion on : could we agree that it would be
> nice to consider operation overloading as a whole set of behaviours that
> could profit from being consistent in a categorized way ? (like the + of []
> could be the + of "RecordAlgebrae")
> Meaning we could define sets of "expected behaviour consistent interaction
> between operators" as we defined the abc and call them algebrae?
>
> I offer the LinearAlgebrae Mixins as a POC, and was thinking of creating a
> unittest to qualify if an object is following the rules of linear algebrae.
>
> What are your opinions ?
> I don't actually see a lot of use case except it was funny to build. But
> maybe it can be of use.
>
>
> Cordialement
>
> -- 
> Julien
>
>
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] TypeHinting: From variable name to type

[Kyle Lahnakoski]

On 2018-10-21 10:44, David Mertz wrote:
> On Fri, Oct 19, 2018 at 3:18 AM Thomas Güttler
> mailto:guettl...@thomas-guettler.de>>
> wrote:
>
> Now my idea: Per module and/or per file type hinting from variable
> name.
> Maybe a magic docstring in the __init__.py file:
> variable-name-mapping:
>   {
>     request: django.http.HttpRequest,
> }
>
>
> In general, I like this idea; but really only on a per-file basis.  A
> short header at the top would be easy enough for an IDE or linter to
> scan.  But imposing the conventions project-wide feels too broad.
>

Maybe in a .editorconfig file?  This would allow type hinting at a
number of directory levels.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Suggestion: Extend integers to include iNaN

[Kyle Lahnakoski]

On 2018-09-30 10:45, Anders Hovmöller wrote:
>
>>> I am roughing out such a class and some test cases which will hopefully 
>>> include some cases where the hoped for advantages can be realised.
>>>
>>> My thinking on bitwise operations is to do the same as arithmetic 
>>> operations, i.e. (anything op iNaN) = iNaN and likewise for shift 
>>> operations.
>> Steve,
>>
>> While you are extending a number system, can every int be truthy, while
>> only iNan be falsey?  I found that behaviour more useful because
>> checking if there is a value is more common than checking if it is a
>> zero value.
> I’m not saying you’re wrong in principle but such a change to Python seems 
> extremely disruptive. And if we’re talking about robustness of code then 
> truthiness would be better like in Java (!) imo, where only true is true and 
> false is false and everything else is an error. If we’re actually talking 
> about changing the truth table of Python for basic types then this is the 
> logical next step.
>
> But making any change to the basic types truth table is a big -1 from me. 
> This seems like a Python 2-3 transition to me. 

Sorry, I can see I was unclear:  I was only asking that the new number
system (and the class that implements it) have truthy defined differently. 

My imagination never considered extending ints with iNaN. I would
imagine the iNaN checks on every int operation to be noticeably slower,
so out of the question.

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Suggestion: Extend integers to include iNaN

[Kyle Lahnakoski]

On 2018-09-30 10:15, David Mertz wrote:
> For similar reasons, I'd like an iInf too, FWIW.  It's good for an
> overflow value, although it's hard to get there in Python ints (would
> 'NaNAwareInt(1)/0' be an exception or iInf?).  Bonus points for anyone
> who knows the actual maximum size of Python ints :-).
>
> However, the main use I'd have for iInf is simply as a starting value
> in a minimization loop.  E.g.
>
> minimum = NaNAwareInt('inf')
> for i in the_data:
>     minimum = min(i, minimum)
>
>     other_stuff(i, minimum, a, b, c)
>
>
> I've written that code a fair number of times; usually I just pick a
> placeholder value that is "absurdly large relative to my domain", but
> a clean infinity would be slightly better.  E.g. 'minimum = 10**100'.

If we conceptualize iNan as "not an integer", then we can define
operators in two manners:

Let |●| be any operator:

1) "conservative" - operators that define a|● |b==iNaN if either a or b
is iNan
2) "decisive" - operators that never return iNan

With a decisive min(a, b), you can write the code you want without
needing iINF

 

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Suggestion: Extend integers to include iNaN

[Kyle Lahnakoski]

On 2018-09-30 09:41, Steve Barnes wrote:
> I am roughing out such a class and some test cases which will hopefully 
> include some cases where the hoped for advantages can be realised.
>
> My thinking on bitwise operations is to do the same as arithmetic 
> operations, i.e. (anything op iNaN) = iNaN and likewise for shift 
> operations.

Steve,

While you are extending a number system, can every int be truthy, while
only iNan be falsey?  I found that behaviour more useful because
checking if there is a value is more common than checking if it is a
zero value.

Thank you
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Suggestion: Extend integers to include iNaN

[Kyle Lahnakoski]

On 2018-09-30 09:41, Steve Barnes wrote:
> I am roughing out such a class and some test cases which will hopefully 
> include some cases where the hoped for advantages can be realised.
>
> My thinking on bitwise operations is to do the same as arithmetic 
> operations, i.e. (anything op iNaN) = iNaN and likewise for shift 
> operations.

Steve,

While you are extending a number system, can every int be truthy, while
only iNan be falsey?  I found that behaviour more useful because
checking if there is a value is more common than checking if it is a
zero value.

Thank you
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Why is design-by-contracts not widely adopted?

[Kyle Lahnakoski]

I use DbC occasionally to clarify my thoughts during a refactoring, and
then only in the places that continue to make mistakes. In general, I am
not in a domain that benefits from DbC.

Contracts are code: More code means more bugs. Declarative contracts are
succinct, but difficult to debug when wrong; I believe this because the
debugger support for contracts is poor; There is no way to step through
the logic and see the intermediate reasoning in complex contracts.  A
contract is an incomplete duplication of what the code already does: at
some level of complexity I prefer to use a duplicate independent
implementation and compare inputs/outputs.

Writing contracts cost time and money; and that cost should be weighed
against the number and flexibility of the customers that use the code. 
A one-time script, a webapp for you team, an Android app for your
startup, fraud software, and Facebook make different accounting
decisions.  I contend most code projects can not justify DbC.


On 2018-09-24 03:46, Marko Ristin-Kaufmann wrote:
> When you are documenting a method you have the following options:
> 1) Write preconditions and postconditions formally and include them
> automatically in the documentation (/e.g., /by using icontract library).
> 2) Write precondtions and postconditions in docstring of the method as
> human text.
> 3) Write doctests in the docstring of the method.
> 4) Expect the user to read the actual implementation.
> 5) Expect the user to read the testing code.
>

There are other ways to communicate how a method works.

6) The name of the method
7) How the method is called throughout the codebase
8) observing input and output values during debugging
9) observing input and output values in production
10) relying on convention inside, and outside, the application
11) Don't communicate - Sometimes / is too
high; code is not repaired, only replaced.


> This is again something that eludes me and I would be really thankful
> if you could clarify. Please consider for an example, pypackagery
> (https://pypackagery.readthedocs.io/en/latest/packagery.html) and the
> documentation of its function resolve_initial_paths:
>
> |packagery.||resolve_initial_paths|(/initial_paths/)
>
> Resolve the initial paths of the dependency graph by recursively
> adding |*.py| files beneath given directories.
>
> Parameters:   
>
> *initial_paths* (|List|[|Path|]) – initial paths as absolute paths
>
> Return type:  
>
> |List|[|Path|]
>
> Returns:  
>
> list of initial files (/i.e./ no directories)
>
> Requires: 
>
>   * |all(pth.is_absolute() for pth in initial_paths)|
>
> Ensures:  
>
>   * |len(result) >= len(initial_paths) if initial_paths else
> result == []|
>   * |all(pth.is_absolute() for pth in result)|
>   * |all(pth.is_file() for pth in result)|
>
>
> How is this difficult to read,[...]?

This contract does not help me: 

Does it work on Windows?
What is_absolute()?  is "file:///" absolute?
How does this code fail? 
What does a permission access problem look like? 
Can initial_paths can be None?
Can initial_paths be files? directories? 
What are the side effects?

resolve_initial_path() is a piece code is better understood by looking
at the callers (#7), or not exposing it publicly (#11).  You can also
use a different set of abstractions, to make the code easier to read: 
  
UNION(file for p in initial_paths for file in p.leaves() if
file.extension=="py")

At a high level, I can see the allure of DbC:  Programming can be a
craft, and a person can derive deep personal satisfaction from
perfecting the code they work on. DbC provides you with more decoration,
more elaboration, more ornamentation, more control.  This is not bad,
but I see all your arguments as personal ascetic sense.  DbC is only
appealing under certain accounting rules.  Please consider the
possibility that "the best code" is: low $$$, buggy, full of tangles,
and mostly gets the job done.   :)
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Asynchronous exception handling around with/try statement borders

[Kyle Lahnakoski]

On 2017-06-28 07:40, Erik Bray wrote:
> Hi folks,

Since the java.lang.Thread.stop() "debacle", it has been obvious that
stopping code to run other code has been dangerous.  KeyboardInterrupt
(any interrupt really) is dangerous. Now, we can probably code a
solution, but how about we remove the danger:

I suggest we remove interrupts from Python, and make them act more like
java.lang.Thread.interrupt(); setting a thread local bit to indicate an
interrupt has occurred.  Then we can write explicit code to check for
that bit, and raise an exception in a safe place if we wish.  This can
be done with Python code, or convenient places in Python's C source
itself.  I imagine it would be easier to whitelist where interrupts can
raise exceptions, rather than blacklisting where they should not.

In the meantime, my solution is to spawn new threads to do the work,
while the main thread has the sole purpose to sleep, and set the "please
stop" flag upon interrupt.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Keyword only argument on function call

[Kyle Lahnakoski]

I agree that this is a familiar pattern, but I long since forgot the
specifics of the domain it happens in.  I borrowed your code, and added
filename tracking to see what source files had high
`could_have_been_a_matched_kwarg`.  Here is the top one:

https://github.com/django/django/blob/master/tests/migrations/test_autodetector.py

The argument-name-matches-the-local-variable-name pattern does appear to
happen in many test files. I assume programmers are more agnostic about
variable names in a test because they have limited impact on the rest of
the program; matching the argument names makes sense.

There are plenty of non-test files that can use this pattern, here are
two intense ones:

https://github.com/django/django/blob/master/django/contrib/admin/options.py
(212 call parameters match)
https://github.com/django/django/blob/master/django/db/backends/base/schema.py
(69 call parameters match)

Opening these in an IDE, and looking at the function definitions, there
is a good chance you find a call where the local variable and argument
names match.  It is interesting to see this match, but I not sure how I
feel about it.  For example, the options.py has a lot of small methods
that deal with (request, obj) pairs: eg 
`has_view_or_change_permission(self, request, obj=None)`  Does that mean
there should be a namedtuple("request_on_object", ["request", "obj"]) to
"simplify" all these calls?  There are also many methods that accept a
single `request` argument; but I doubt they would benefit from the new
syntax.



On 2018-09-06 06:15, Anders Hovmöller wrote:
> I have a working implementation for a new syntax which would make using 
> keyword arguments a lot nicer. Wouldn't it be awesome if instead of:
>
>   foo(a=a, b=b, c=c, d=3, e=e)
>
> we could just write:
>
>   foo(*, a, b, c, d=3, e)
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Is this PEP-able? "with" statement inside genexps / list comprehensions

[Kyle Lahnakoski]

Rudy,

I think your proposal may be very specific to iterable context managers;
in which case, make a method that makes that assumption:

> def iter_with(obj):
> with obj as context:
> yield from context

and use it

> g = (
> f.read()
> for fn in filenames
> for f in iter_with(open(fn))
> )

On 2018-07-30 15:15, Rudy Matela wrote:
> Hello,
>
> Do you think it would be nice to allow with statements inside genexps or
> list comprehensions?  The functions __enter__ and __exit__ would be
> automatically called as iterables are traversed.  I am thinking of
> drafting a PEP about this.  Examples:
>
>
> This 
>
>   g = (f.read() for fn in filenames with open(fn) as f)
>
> would be equivalent to the following use of a generator function:
>
>   def __gen():
>   for fn in filenames:
>   with open(fn) as f:
>   yield f.read()
>   g = __gen()
>
>
> This
>
>   list = [f.read() for fn in filenames with open(fn) as f]
>
> would be equivalent to the following:
>
>   list = []
>   for fn in filenames:
>   with open(fn) as f:
>   list.append(f.read())
>
> --
> Rudy
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Redefining method

[Kyle Lahnakoski]

I think C# calls methods that are added to a class "extension methods".
You can make a decorator that will do that for you:

> def extend(cls):
>     """
>     DECORATOR TO ADD METHODS TO CLASSES
>     :param cls: THE CLASS TO ADD THE METHOD TO
>     :return:
>     """
>     def extender(func):
>     setattr(cls, get_function_name(func), func)
>     return func
>     return extender

and use it

> C = MyClass
> @extend(C)
> def foo(self, schesome_param):
>     some_code()



On 2018-07-30 14:12, Chris Barker via Python-ideas wrote:
> On Mon, Jul 30, 2018 at 9:10 AM, Nick Coghlan  > wrote:
>
> If you need to replace them for some reason, it will preferably be
> within a temporary bounded scope, using a tool like
> unittest.mock.patch, rather than as a permanent change that affects
> every other use of the class within the process.
>
>
>  yup -- but you can still do a raw monkey-patch anyway. You asked for:
>
> > Thank you for your question. You asked why not
> >> c = MyClass
> >> o = c()
> >>
> >> def c.foo(cls): ...
>
>
> do you mean this? you want a classmethod? or is this what you mean? --
> which you can do:
>
> C = MyClass
>
> def foo(self, some_param):
>     some_code()
>
> C.foo = foo
>
> (note that I used a capital C -- capitalized names are traditional for
> classes -- see PEP 8)
>
> In that case, any existing, and new instances of the C class will now
> have the foo method.
>
> Also -- that line: C = MyClass -- binds the name "C" to the very same
> class object as MyClass -- so you will have just monkey=patched
> MyClass -- I"m guessing you didn't want that. If not, and you wanted C
> t be a copy of MyClass that you could then change/add/remove methods
> from, then you want subclassing -- that is exactly what it is for.
>
> Then there is this:
>  
>
> >> def o.bar(self): ...
>
>
> which is monkey patching an instance object, which you can also do,
> but you don't get a bound method -- i.e. it doesn't get self passed in
> automatically.
>
> -CHB
>
>
>
>
>
>
>
>
>
>  
>
> > I've the same same query, but never had the courage to ask. So that
> > you for asking. And also giving me a chance to share my thoughts.
>
> It's essentially due to the fact that while we deliberately allow
> runtime monkeypatching (as it's sometimes the best available answer),
> we also strongly encourage the idea of treating it as a last resort
> option (outside specific use cases like testing).
>
> So if you want to define methods on a class, the strongly preferred
> place to define them is inside the class definition, where they're
> easy for future maintainers of that class to find.
>
>
> Cheers,
> Nick.
>
> P.S. While it's *not* explicitly part of Python's design rationale,
> http://connascence.io/locality.html
>  and the rest of that site
> provide
> some good info on the kinds of problems that "action at a distance"
> effects, like monkeypatching class definitions, can cause in a code
> base.
>
> -- 
> Nick Coghlan   |   ncogh...@gmail.com  
>  |   Brisbane, Australia
> ___
> Python-ideas mailing list
> Python-ideas@python.org 
> https://mail.python.org/mailman/listinfo/python-ideas
> 
> Code of Conduct: http://python.org/psf/codeofconduct/
> 
>
>
>
>
> -- 
>
> Christopher Barker, Ph.D.
> Oceanographer
>
> Emergency Response Division
> NOAA/NOS/OR            (206) 526-6959   voice
> 7600 Sand Point Way NE   (206) 526-6329   fax
> Seattle, WA  98115       (206) 526-6317   main reception
>
> chris.bar...@noaa.gov 
>
>
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP 505: None-aware operators

[Kyle Lahnakoski]

On 2018-07-25 23:53, David Mertz wrote:
> On Wed, Jul 25, 2018, 11:27 PM Chris Angelico  > wrote:
>
>
> means that you cannot do this:
>
> >>> NullCoalesce(spam).nil is None
>
> This IS fundamentally unfixable in a library.
>
>
> Right now, you can still always call .unbox() at the end to get the
> underlying value. I agree that's a little ugly, hence why I added the
> wrapt proxy stuff. Most operations trigger unboxing, but indeed not
> simply echoing to the shell.
>

Chris is correct to point out this problem with comparing to None. I
have that problem with my own version of the proxy library, similar to
what David is building:  I use the proxy  heavily; to the point where
almost any line may be touching a proxy rather than a real value. To
avoid bugs, I disallow "is None" comparisons, mandating "== None" instead. 

Using unbox() is an alternative,  but it is uglier than than swapping
"as" for "==".


___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Idea: Deferred Default Arguments?

[Kyle Lahnakoski]

I agree this is a problem, which I have seen solved by removing the
method signature, which is unfortunate:

> def flexible_method(**kwargs):
>     # Read the code to find out the expected parameters   

I have an @override decorator to handle this type of pattern. It will
perform the null-coalescing with properties found in a special "kwargs"
parameter. "kwargs" is assigned a dict that has a copy of the method
arguments. The value of a callee's argument is, in order,

* a not None value provided by the caller or
* a not None value found in the kwargs dict or
* the default value provided by the method declaration or
* None

I was not clear on where you wanted to define your defaults.  Either
like this:

> @override
>     def subfunction_1(a=None, b=None, c=None, kwargs=None):
>         return a+b*c
>
> @override
>     def subfunction_2(d=None, e=None, f=None, kwargs=None):
>         return d*e+f
>
> @orverride
>     def main_function(a=2, b=3, c=4, d=5, e=6, f=7, kwargs=None):
>         return subfunction_1(a, b, c) + subfunction_2(d, e, f)
>         return subfunction_1(kwargs) + subfunction_2(kwargs)  # IF YOU
WANT TO BE LAZY

or like this:

> @override
>     def subfunction_1(a=2, b=3, c=4, kwargs=None):
>         return a+b*c
>
> @override
>     def subfunction_2(d=5, e=6, f=7, kwargs=None):
>         return d*e+f
>
> @orverride
>     def main_function(a=None, b=None, c=None, d=None, e=None, f=None,
kwargs=None):
>         return subfunction_1(a, b, c) + subfunction_2(d, e, f)
>         return subfunction_1(kwargs) + subfunction_2(kwargs)  # IF YOU
WANT TO BE LAZY

both are identical except for where you declare the default values.


https://github.com/klahnakoski/mo-kwargs

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] A better (simpler) approach to PEP 505

[Kyle Lahnakoski]

I agree a class can provide a very good alternative to PEP505.  I built
one a while ago, and still use it https://github.com/klahnakoski/mo-dots
for most my data transformation code.

The library only boxes dicts and lists, which removes the need for
.unbox() in many of my use cases. My point is, if a class is chosen
instead of PEP505, I doubt the unboxing will be a big issue.


On 2018-07-23 11:12, David Mertz wrote:
> Here is a way of solving the "deep attribute access to messy data"
> problem that is:
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Secure string disposal (maybe other inmutable seq types too?)

[Kyle Lahnakoski]

Ezequiel (Ezekiel) Brizuela,

How is the secret "password" getting into a Python variable?  It is
coming from disk, or network? Do the buffers of those systems have a
copy?  How about methods that operate on the secrets?  Do they
internally decrypt secrets to perform the necessary operations?

I had this problem, and the only solution was a hardware security module
(HSM): Private keys do not leave the module;
encryption/decryption/verification are all done on the module. 
Passwords enter the secure system via hardware keypads; which encrypt
the password before transmitting bytes to the local computer.

I do not think you can trust a network connected machine to have private
keys; all private keys end their life stolen, lost or expired.


On 2018-06-22 20:31, Ezequiel Brizuela [aka EHB or qlixed] wrote:
> As all the string in python are immutable, is impossible to overwrite
> the value or to make a "secure disposal" (overwrite-then-free) of a
> string using something like:
>
> >>> a = "something to hide"
> >>> a =  "x"*len(a)
>
> This will lead on the process memory "something to hide" and "x"
> repeated len(a) times.
>
> - Who cares? Why is this relevant?
>   Well if you handle some sensitive information like CC numbers,
> Passwords, PINs, or other kind of information you wanna minimize the
> chance of leaking any of it.
>
> - How this "leak" can happen?
>   If you get a core/memory dump of an app handling sensitive
> information you will get all the information on that core exposed!
>
> - Well, so what we can do about this?
>   I propose to make the required changes on the string objects to add
> an option to overwrite the underlying buffer. To do so:
>
>   * Add a wiped as an attribute that is read-only to be set when the
> string is overwrited.
>   * Add a wipe() method that overwrite the internal string buffer.
>
> So this will work like this:
>
> >>> pwd =getpass.getpass('Set your password:') # could be other
> sensitive data.
> >>> encrypted_pwd = crypt.crypt(pwd)  # crypt() just as example.
> >>> pwd.wiped  # Check if pwd was wiped.
> False
> >>> pwd.wipe()  # Overwrite the underlying buffer
> >>> pwd.wiped  # Check if pwd was wiped.
> True
> >>> print(pwd)  # Print noise (or empty str?)
> >>> del pwd  # Now is in hands of the GC.
>
> The wipe method immediately overwrite the underlying string buffer,
> setting wiped as True for reference so if the string is further used
> this can be checked to confirm that the change was made by a wipe and
> not by another procedure. Also initially the idea is to use unicode
> NULL datapoint to overwrite the string, but this could be change to
> let the user parametrize it over wipe() method.
> An alternative to this is to add a new exception "WipedError" that
> could be throw where the string is accessed again, but I found this
> method too disruptive to implement for a normal/standard string
> workflow usage.
>  
> Quick & Dirty FAQ:
>
> - You do it wrong!, the correct code to do that in a secure way is:
> >>> pwd = crypt.crypt(getpass.getpass('Set your password'))
> Don't you know that fool?
>
>   Well no, the code still generate a temporary string in memory to
> pass to crypt. But now this string is lying there and can't be
> accessed for an overwrite with wipe()
>
>
> - Why not create a new type like in C# or Java?
>
>   I see that this tend to disrupt the usual workflow of string usage.
> Also the idea here is not to offer secure storage of string in memory
> because there is already a few mechanism to achieve with the current
> Python base. I just want to have the hability to overwrite the buffer.
>
>
> - Why don't use one of the standard algorithms to overwrite like
> DoD5220 or MIL-STD-414?
>
>   This kind of standard usually are oriented for usage on persistent
> storage, specially on magnetic media for where the data could be
> "easily" recoverd. But this could ve an option that could be
> implemented adding the option to plug a function that do the overwrite
> work inside the wipe method.
>
>
> - This is far beyond of the almost implementation-agnostic definition
> of the python lang. How about to you make a module with this
> functionality and left the lang as is?
>
>   Well I already do it:
>
> https://github.com/qlixed/python-memwiper/
> 
>
>   But i hit a lot of problems in the road, I was working on me free
> time over the last year on this and make it "almost" work, but that is
> not relevant to the proposal.
>   I think that this kind of security things needs to be tackled from
> within the language itself specially when the lang have GC. I firmly
> believe that the security and protections needs to be part of the
> "with batteries" offer of Python. And I think that this is one little
> thing that could help a lot to secure our apps.
>   Let me know what do you think!
>
> ~ Ezequiel (Ezekiel) Brizuela [ aka Qlixed ] ~
>
>
>
> ___
> Python-ideas mailing list

Re: [Python-ideas] Trigonometry in degrees

[Kyle Lahnakoski]

On 2018-06-08 01:44, Yuval Greenfield wrote:
> On Thu, Jun 7, 2018 at 10:38 PM Stephen J. Turnbull
>  > wrote:
>
>
> 6.123233995736766e-17
> >>>
>
> is good enough for government work, including at the local public high
> school.
>
>
> There probably is room for a library like "fractions" that represents
> multiples of pi or degrees precisely. I'm not sure how complicated or
> valuable of an endeavor that would be. But while I agree that floating
> point is good enough, we probably can do better.
>  
>

Yes, I agree with making a module (called `rational_trig`?), that
defines some Angle constants, and defines trig functions that accept
Angle objects. Using angle objects will prevent the explosion of
unit-specific variations on the trig functions (sin, sindeg, singrad,
etc).  Like mentioned above, the Angle object is probably best
implemented as a Rational of 2*pi, which will allow our favorite angles
to be represented without floating point error.  We can define `degrees`
and `radians` constants which can be used as units; then trig looks
something like:

from rational_trig import cos

if cos(90*degrees) == 0:
    print("yay!")

It is probably slow as molasses, but maybe good enough for a teaching
environment?


___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Runtime assertion with no overhead when not active

[Kyle Lahnakoski]

I currently use the form

     and log_function(  )

where  is some module variable, usually "DEBUG".  I do
this because it is one line, and it ensures the log_function parameters
are not evaluated.

*IF* runtime assertions had a switch so they have no overhead when not
active, how much faster can it get?  How expensive is the 
check?


On 2018-05-10 03:55, Barry Scott wrote:
>
> My logging example would be
>
> log( control_flag, msg_expr )
>
> expanding to:
>
> if :
> log_function(  )
>
> Barry
>
> This idea requires the same sort of machinery in python that I was
> hoping for to implement the short circuit logging.



___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] datetime.timedelta literals

[Kyle Lahnakoski]

Pål Grønås Drange,

I do like the idea of literals typed with scientific units, but I often
get short variable names mixed up, so I am not sure if I could use them
without a cheat sheet. Formatting datetime is a good example of how
confusing a collection of short names can get: Is month %m or %M?  Is
minute %m or %i?
https://docs.python.org/2/library/datetime.html#strftime-and-strptime-behavior

In case you are thinking, "Kyle, how can you even *think* "%i" means
minutes?!", please see
https://dev.mysql.com/doc/refman/5.7/en/date-and-time-functions.html#function_date-format
 
:)

I made a class, with magic methods, to get close to what you want:

(2.5*HOUR - 14*MINUTE + 9300*MILLISECOND).total_seconds()

I used full names for less confusion, but you can do the same with
shorter names:

(2.5*h- 14*min + 9300*ms).total_seconds()


Maybe the Python parser can be made to add an implied multiplication
between a-number-followed-directly-by-a-variable-name. If so, then I
could write:

(2.5HOUR - 14MINUTE + 9300MILLISECOND).total_seconds()

You can define your short, domain specific, suffixes:

(2.5h - 14m + 9300ms).total_seconds()




On 2018-06-02 08:21, Pål Grønås Drange wrote:
> Elevator pitch:
>
> (2.5h - 14min + 9300ms).total_seconds()
> # 8169.3
>
> from datetime import datetime as dt
> start = dt.now()
> end = dt.now()
> (end-start) < 5s
> # True
>
>
> chrono::duration:
>
> In C++ 14 the std::chrono::duration was introduced which corresponds
> somewhat to
> datetime.timedelta.
>
> C++ 14 introduced so-called chrono literals[1], which are literals
> specified as
> [number][h|min|s|ms|us|ns], e.g.
>
> * 2.5h
> * 14min
> * 9300ms
>
> These literals should correspond to
>
> * datetime.timedelta(0, 9000)  # 2.5h = 2.5*3600 = 9000 seconds
> * datetime.timedelta(0, 840)   # 14min = 14*60 = 840 seconds
> * datetime.timedelta(0, 9, 30)  # 9300ms = 9 seconds + 3*10^5
> microseconds
>
>
> If a literal was interpreted as a datetime.timedelta, the following
> would work
> out of the box:
>
> 2.5h - 14min + 9300ms * 2
>
> which would correspond to
>
> from datetime import timedelta as D
>
> D(hours=2.5) - D(minutes=14) + D(milliseconds=9300) * 2
> # datetime.timedelta(0, 8178, 60)  # (*2 precedes, so that's to be
> expected)
>
>
> (D(hours=2.5) - D(minutes=14) + D(milliseconds=9300)) * 2
> # datetime.timedelta(0, 16338, 60)
>
>
>
> Notes:
>
> * C++ uses `min` instead of `m`.  `min` is a keyword in Python.
> * In C++, `1d` means the first day of a month [2].
> * In C++, `1990y` means the year 1990 (in the Proleptic Gregorian
> calendar) [3].
> * C++ have the types signed integers and not floats, so 2.5h would not
> be valid.
> * My apologies if this has been discussed before; my search-fu gave me
> nothing.
>
>
>
> References:
>
>
> [1] std::literals::chrono_literals::operator""min
> http://en.cppreference.com/w/cpp/chrono/operator%22%22min
>
> [2] http://en.cppreference.com/w/cpp/chrono/day
>
> [3] http://en.cppreference.com/w/cpp/chrono/year
>
>
> Best regards,
> Pål Grønås Drange
>
>
>
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Proposal: A Reduce-Map Comprehension and a "last" builtin

[Kyle Lahnakoski]

On 2018-04-05 21:18, Steven D'Aprano wrote:
> (I don't understand why so many people have such an aversion to writing 
> functions and seek to eliminate them from their code.)
>

I think I am one of those people that have an aversion to writing functions!

I hope you do not mind that I attempt to explain my aversion here. I
want to clarify my thoughts on this, and maybe others will find
something useful in this explanation, maybe someone has wise words for
me. I think this is relevant to python-ideas because someone with this
aversion will make different language suggestions than those that don't.

Here is why I have an aversion to writing functions: Every unread
function represents multiple unknowns in the code. Every function adds
to code complexity by mapping an inaccurate name to specific
functionality. 

When I read code, this is what I see:

>    x = you_will_never_guess_how_corner_cases_are_handled(a, b, c)
>    y =
you_dont_know_I_throw_a_BaseException_when_I_do_not_like_your_arguments(j,
k, l)

Not everyone sees code this way: I see people read method calls, make a
number of wild assumptions about how those methods work, AND THEY ARE
CORRECT!  How do they do it!?  It is as if there are some unspoken
convention about how code should work that's opaque to me.

For example before I read the docs on
itertools.accumulate(list_of_length_N, func), here are the unknowns I see:

* Does it return N, or N-1 values?
* How are initial conditions handled?
* Must `func` perform the initialization by accepting just one
parameter, and accumulate with more-than-one parameter?
* If `func` is a binary function, and `accumulate` returns N values,
what's the Nth value?
* if `func` is a non-cummutative binary function, what order are the
arguments passed? 
* Maybe accumulate expects func(*args)?
* Is there a window size? Is it equal to the number of arguments of `func`?

These are not all answered by reading the docs, they are answered by
reading the code. The code tells me the first value is a special case;
the first parameter of `func` is the accumulated `total`; `func` is
applied in order; and an iterator is returned.  Despite all my
questions, notice I missed asking what `accumulate` returns? It is the
unknown unknowns that get me most.

So, `itertools.accumulate` is a kinda-inaccurate name given to a
specific functionality: Not a problem on its own, and even delightfully
useful if I need it often. 

What if I am in a domain where I see `accumulate` only a few times a
year? Or how about a program that uses `accumulate` in only one place?
For me, I must (re)read the `accumulate` source (or run the caller
through the debugger) before I know what the code is doing. In these
cases I advocate for in-lining the function code to remove these
unknowns. Instead of an inaccurate name, there is explicit code. If we
are lucky, that explicit code follows idioms that make the increased
verbosity easier to read.

Consider Serhiy Storchaka's elegant solution, which I reformatted for
readability

> smooth_signal = [
> average
>     for average in [0]
>     for x in signal
> for average in [(1-decay)*average + decay*x]
> ]

We see the initial conditions, we see the primary function, we see how
the accumulation happens, we see the number of returned values, and we
see it's a list. It is a compact, easy read, from top to bottom. Yes, we
must know `for x in [y]` is an idiom for assignment, but we can reuse
that knowledge in all our other list comprehensions.  So, in the
specific case of this Reduce-Map thread, I would advocate using the list
comprehension. 

In general, all functions introduce non-trivial code debt: This debt is
worth it if the function is used enough; but, in single-use or rare-use
cases, functions can obfuscate.



Thank you for your time.









___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP 572: Statement-Local Name Bindings, take three!

[Kyle Lahnakoski]

On 2018-03-23 06:01, Chris Angelico wrote:
> https://www.python.org/dev/peps/pep-0572/
>

A suggestion:

Under the rejected "Special-casing comprehensions", you show
"prefix-local-name-bindings": Name bindings that appear before the loop,
like:

> stuff = [(y, x/y) where y = f(x) for x in range(5)]


Please add mention of rejecting "postfix-local-name-bindings": Name
bindings that happen after the loop.  For example:

> stuff = [(y, x/y) for x in range(5) where y = f(x)]


Since all the same reasoning applies to both prefix and postfix
variations, maybe distinguishing between prefix and postfix can be done
in the last paragraph of "Special-casing comprehensions".

Thank you.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Descouraging the implicit string concatenation

[Kyle Lahnakoski]

On 2018-03-14 08:18, Facundo Batista wrote:
> Hello!
>
> What would you think about formally descouraging the following idiom?
>
> long_string = (
> "some part of the string "
> "with more words, actually is the same "
> "string that the compiler puts together")
>

Thank you for bring this up!  A regex through one of my programs
revealed three bugs caused by implicit concatenation in lists.  Here is
one of them:
https://github.com/klahnakoski/ActiveData-ETL/blob/etl/activedata_etl/transforms/pulse_block_to_task=_cluster.py#L897
 
The missing commas were not caught until now because of the lists are
long, and deal with rare cases.

I did use implicit concatenation for a long SQL statement, and a few
long error messages, but the byte savings is not worth the increased bug
density.

>    self.db.execute(
>    "CREATE TABLE files (" +
>    "   bucket TEXT," +
>    "   key TEXT," +
>    "   name TEXT," +
>    "   last_modified REAL," +
>    "   size INTEGER," +
>    "   annotate TEXT, " +
>    "   CONSTRAINT pk PRIMARY KEY (bucket, name)" +
>    ")"
>    )

is almost identical to

>    self.db.execute(
>    "CREATE TABLE files ("
>    "   bucket TEXT,"
>    "   key TEXT,"
>    "   name TEXT,"
>    "   last_modified REAL,"
>    "   size INTEGER,"
>    "   annotate TEXT, "
>    "   CONSTRAINT pk PRIMARY KEY (bucket, name)"
>    ")"
>    )
   
It looks like I am in the market for a linter that prevents implicit
string concatenation!
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP 572: Statement-Local Name Bindings

[Kyle Lahnakoski]

On 2018-02-28 02:46, Matt Arcidy wrote:
> From readability, the examples put forth have been to explain the
> advantage, with which I agree.  However, i do not believe this scales
> well.
>
> [(foo(x,y) as g)*(bar(y) as i) + g*foo(x,a) +baz(g,i) for x... for y...]
>
> That's 3 functions, 2 iterators, 3 calls saved ('a' is some constant
> just to trigger a new call on foo).  I'm not trying to show ugly
> statements can be constructed, but show how quickly in _n iterators
> and _m functions readability declines.
>

You could put it on multiple lines

> [
> (g * i) + g * foo(x, a) + baz(g, i)
> for x in X
> for y in Y
> for g in [foo(x,y)]
> for i in [bar(y)]
> ]

and then notice a common factor!  :)

> [
> g * (i + foo(x, a) + baz(g, i))
> for x in X
> for y in Y
> for g in [foo(x,y)]
> for i in [bar(y)]
> ]




___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Temporary variables in comprehensions

[Kyle Lahnakoski]

I believe list comprehensions are difficult to read because they are not
formatted properly. For me, list comprehension clauses are an
expression, followed by clauses executed in the order. Any list
comprehension with more than one clause should be one-line-per clause.

Examples inline:

On 2018-02-15 19:57, Steven D'Aprano wrote:
> Where should the assignment go? [(y, y**2) let y = x+1 for x in (1, 2,
> 3, 4)] [(y, y**2) for x in (1, 2, 3, 4) let y = x+1] 

Since y is a function of x, it must follow the for clause:

> [
>     (y, y ** 2)
>     for x in (1, 2, 3, 4)
>     let y = x + 1
> ]

> How do they interact when you have multiple loops and if-clauses? [(w,
> w**2) for x in (1, 2, 3, 4) let y = x+1 for a in range(y) let z = a+1
> if z > 2 for b in range(z) let w = z+1] 

They are applied in order:

> [
> (w, w**2)
> for x in (1, 2, 3, 4)
> let y = x+1
> for a in range(y)
> let z = a+1
> if z > 2
> for b in range(z)
> let w = z+1
> ]

which is a short form for:

> def stuff():
> for x in (1, 2, 3, 4):
> y = x+1
> for a in range(y):
> z = a+1
> if z > 2:
> for b in range(z):
> w = z+1
> yield (w, w**2)
>
> list(stuff())


___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Temporary variables in comprehensions

[Kyle Lahnakoski]

On 2018-02-17 11:23, fhsxfhsx wrote:
> [
>   {
>     'id': goods.id,
>     'name': goods.name,
>     'category': gc.name,
>     'category_type': gc.type,
>   }
>   for goods_id in goods_id_list
>   for goods is Goods.get_by_id(goods_id)
>   for gc is GoodsCategory.get_by_id(goods.category_id)
> ]

in the short term, it seems for...in [...] is good enough:

> [
>     {
>     'id': goods.id,
>     'name': goods.name,
>     'category': gc.name,
>     'category_type': gc.type,
>     }
>     for goods_id in goods_id_list
>     for goods in [Goods.get_by_id(goods_id)]
>     for gc in [GoodsCategory.get_by_id(goods.category_id)]
> ]

I personally would like to see with...as... syntax allowed in list
comprehensions, despite `with` being limited to context managers to date.

> [
>     {
>     'id': goods.id,
>     'name': goods.name,
>     'category': gc.name,
>     'category_type': gc.type,
>     }
>     for goods_id in goods_id_list
>     with Goods.get_by_id(goods_id) as goods
>     with GoodsCategory.get_by_id(goods.category_id) as gc
> ]

..,or maybe `let` reads easier?

> [
>     {
>     'id': goods.id,
>     'name': goods.name,
>     'category': gc.name,
>     'category_type': gc.type,
>     }
>     for goods_id in goods_id_list
>     let goods = Goods.get_by_id(goods_id)
>     let gc = GoodsCategory.get_by_id(goods.category_id)
> ]




___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Arguments to exceptions

[Kyle Lahnakoski]

I agree with Ken for the need to make rich exceptions easy to write, but
I do not know enough to say if changing BaseException to support this is
a good idea; I made my own error reporting library to do this: For
example, to define a new exception type with a `name` attribute:

raise Log.error("name {{name|quote}} is not defined.", name=name)

My point is that the first parameter, the template string, acts as the
class definition.

Now, there are some extra "features": The mustaches demand named
parameters, and use pipe for some simple formatting hints; I hope you
can overlook that and maybe use f-strings, or similar. The template
string is not expanded unless the text log is required; the errors
serialize nicely to JSON for a structured logging system; querying for
instances of this error is the same as looking for the template string.

My library does not solve the problem of extracting parameters out of
the standard lib errors; but catching them early, chaining, and properly
classing them, is good enough.

>>> try:
... a={}  # imagine this is something complicated
... c = a.b
... except Exception as e:
... Log.error("can not do something complicated", cause=e)  # just like 
raise from, plus we are defining a new exception type


On 2017-07-05 00:21, Terry Reedy wrote:
> On 7/4/2017 6:31 PM, Greg Ewing wrote:
>> Terry Reedy wrote:
>>> Attaching a *constant* string is very fast, to the consternation of
>>> people who would like the index reported.
>
> Actually, the constant string should be attached to the class, so
> there is no time needed.
>
>> Seems to me that storing the index as an attribute would help
>> with this. It shouldn't be much slower than storing a constant
>> string,
>
> Given that the offending int is available as a Python int, then
> storing a reference should be quick, though slower than 0  (see above
> ;-).
>
>> and formatting the message would be deferred until
>> it's needed, if at all.
>
> I agree that this would be the way to do it.  I will let an advocate
> of this enhancement lookup the rejected issue (there may be more than
> one) proposing to make the bad index available and see if this is the
> actual proposal rejected and if so, why (better than I may remember).
>
> It occurs to me that if the exception object has no reference to any
> python object, then all would be identical and only one cached
> instance should be needed.  I checked and neither IndexError nor
> ZeroDivisionError do this.
>


___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Way to repeat other than "for _ in range(x)"

[Kyle Lahnakoski]

On 2017-03-30 05:18, Markus Meskanen wrote:
> Hi Pythonistas,
>
> yet again today I ended up writing:
>
> d = [[0] * 5 for _ in range(10)]
>
>
> Thoughts?

It looks like you are initializing matrices.  Can you make a helper
function?

d = matrix(shape=(5, 10), default=0)

or maybe use NumPy?
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Proposal: Query language extension to Python (PythonQL)

[Kyle Lahnakoski]

Pavel,

I like PythonQL. I perform a lot of data transformation, and often find
Python's list comprehensions too limiting; leaving me wishing for
LINQ-like language features.

As an alternative to extending Python with PythonQL, Terry Reedy
suggested interpreting a DSL string, and Pavel Velikhov alluded to using
magic method tricks found in ORM libraries. I can see how both these are
not satisfactory. 

A third alternative could be to encode the query clauses as JSON
objects. For example: 

result = [ select (x, sum_y)
   for x in range(1,8), 
   y in range(1,7)
   where x % 2 == 0 and y % 2 != 0 and x > y
   group by x
   let sum_y = sum(y)
   where sum_y % 2 != 0
   ]

result = pq([
{"select":["x", "sum_y"]},
{"for":{"x": range(1,8), "y": range(1,7)}},
{"where": lambda x,y: x % 2 == 0 and y % 2 != 0 and x > y},
{"groupby": "x"},
{"with":{"sum_y":{"SUM":"y"}},
{"where": {"neq":[{"mod":["sum_y", 2]}, 0]}}
 ])

This representation does look a little lispy, and it may resemble
PythonQL's parse tree. I think the benefits are:

1) no python language change
2) easier to parse
3) better than string-based DSL for catching syntax errors
4) {"clause": parameters} format is flexible for handling common query
patterns **
5) works in javascript too
6) easy to compose with automation (my favorite)

It is probably easy for you to see the drawbacks.


** The `where` clause can accept a native lambda function, or an
expression tree





"If you are writing a loop, you are doing it wrong!" :)


On 2017-03-24 11:10, Pavel Velikhov wrote:
> Hi folks!
>
>   We started a project to extend Python with a full-blown query
> language about a year ago. The project is call PythonQL, the links are
> given below in the references section. We have implemented what is
> kind of an alpha version now, and gained some experience and insights
> about why and where this is really useful. So I’d like to share those
> with you and gather some opinions whether you think we should try to
> include these extensions in the Python core.
>
> *Intro*
>
>   What we have done is (mostly) extended Python’s comprehensions with
> group by, order by, let and window clauses, which can come in any
> order, thus comprehensions become a query language a bit cleaner and
> more powerful than SQL. And we added a couple small convenience
> extensions, like a  We have identified three top motivations for folks
> to use these extensions:
>
> *Our Motivations*
>
> 1. This can become a standard for running queries against database
> systems. Instead of learning a large number of different SQL dialects
> (the pain point here are libraries of functions and operators that are
> different for each vendor), the Python developer needs only to learn
> PythonQL and he can query any SQL and NoSQL database.
>
> 2. A single PythonQL expression can integrate a number of
> databases/files/memory structures seamlessly, with the PythonQL
> optimizer figuring out which pieces of plans to ship to which
> databases. This is a cool virtual database integration story that can
> be very convenient, especially now, when a lot of data scientists use
> Python to wrangle the data all day long.
>
> 3. Querying data structures inside Python with the full power of SQL
> (and a bit more) is also really convenient on its own. Usually folks
> that are well-versed in SQL have to resort to completely different
> means when they need to run a query in Python on top of some data
> structures.
>
> *Current Status*
>
> We have PythonQL running, its installed via pip and an encoding hack,
> that runs our preprocessor. We currently compile PythonQL into Python
> using our executor functions and execute Python subexpressions via
> eval. We don’t do any optimization / rewriting of queries into
> languages of underlying systems. And the query processor is basic too,
> with naive implementations of operators. But we’ve build DBMS systems
> before, so if there is a good amount of support for this project,
> we’ll be able to build a real system here.
>
> *Your take on this*
>
> Extending Python’s grammar is surely a painful thing for the
> community. We’re now convinced that it is well worth it, because of
> all the wonderful functionality and convenience this extension offers.
> We’d like to get your feedback on this and maybe you’ll suggest some
> next steps for us.
>
> *References*
>
> PythonQL GitHub page: https://github.com/pythonql/pythonql
> PythonQL Intro and Tutorial (this is all User Documentation we have
> right
> now): https://github.com/pythonql/pythonql/wiki/PythonQL-Intro-and-Tutorial
> A use-case of querying Event Logs and doing Process Mining with
> PythonQL: 
> https://github.com/pythonql/pythonql/wiki/Event-Log-Querying-and-Process-Mining-with-PythonQL
> PythonQL demo site: www.pythonql.org 
>
> Best regards,
> PythonQL Team
>
>
>
>
>
>
> 

Re: [Python-ideas] get() method for list and tuples

[Kyle Lahnakoski]

I must mention a get() method for lists and tuples would be very useful
for me too. It is so useful, that I spent too much time making my own
module to handle this case, plus many of the other dealing-with-None
subjects found on this list.  Michel is correct to point out that this
is domain specific problem; a domain where you deal with many varied,
and schema-free, data formats. I deal with JSON emitted from multiple
systems of often-changing schemas.  In my experience, this is not a
result of bad or buggy programming, rather, it is about representing
facts and annotating them with a multitude of optional properties and
descriptive structures.

Now, in the specific case of list.get(), I would be disappointed that it
is used to extract parameters from an arg list: Parameters should be
named; packing them into an ordered list looses that important
information, but it happens[1], and list.get() would help. For the args
scenario, I do like Ed's solution: dict(enumerate(args)).

In conclusion, I may have talked myself out of liking list.get(): Python
has a fundamentally different philosophy about None that conflicts with
what I need for my domain [2] where I am transforming and interpreting
data. Using a set of classes that make a different set of assumptions
about None is not arduous, it keeps the definitions separate, and I
still get all wonderfulness of Python.

[1] also happens when reading csv files: Missing values indicate
default, or variable number of columns indicate that the missing
rightmost columns are all null.
[2] For Python, None is a missing value, or a special case. For data
transformation, None means "the operation you performed does not apply
to this datatype" which avoids exceptions, which gives you an algebra
over data (with [], dot and slice as operators), which allows you to
build complex list comprehensions (data transformation queries) without
the exception catching logic. Databases query languages do this.

On 2017-02-28 21:02, Michel Desmoulin wrote:
>
> Le 01/03/2017 à 02:23, Ethan Furman a écrit :
>> On 02/28/2017 05:18 PM, Michel Desmoulin wrote:
>>
>>> I love this proposal but Guido rejected it. Fighting for it right now
>>> would probably be detrimental to the current proposed feature which
>>> could potentially be more easily accepted.
>> PEP 463 has a better chance of being accepted than this one does, for
>> reasons that D'Aprano succinctly summarized.
>>
>> -- 
>> ~Ethan~
> The debate is not even over and you are already declaring a winner.
> That's not really fair. Give the idea a chance and read until the end.
>
> D'Aprano's argument is mostly "I don't encounter IndexError really often
> and when I do I have this twisted one liner to get away it".
>
> Well, that's not really a good reason to reject things for Python
> because it's a language with a very diverse user base. Some bankers,
> some web dev, some geographers, some mathematicians, some students, some
> 3D graphists, etc. And the language value obvious, readable, predictable
> code for all.
>
> Most people on this list have a specialty, because their speciality
> don't see a use for the feature doesn't mean there is not one.
>
> So I provided on my last answer an explanation of what I would use it for.
>
>
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Efficient debug logging

[Kyle Lahnakoski]

Can you wrap the expensive functions in lambdas? And have your logger
evaluate it, only if required?

> debugLog( ‘info is %r’ % (lambda: expensiveFunction(),) )


On 2017-02-14 10:51, Barry Scott wrote:
> A common pattern I use is to have logging calls for debug and information 
> with my applications.
> The logging calls can be separately enabled and disabled.
>
> For example:
>
> debug_log_enabled = False
> def debugLog( msg ):
>   If debug_log_enabled:
> print( ‘Debug: %s’ % (msg,) )
>
> Then the caller can simple write:
>
> def main():
>   debugLog( ‘Start of main’ )
>
> This is fine until the evaluation of the msg becomes expensive.
>
>   debugLog( ‘info is %r’ % (expensiveFunction(),) )
>
> What would be nice is to be able to avoid evaluation the tuple of arguments 
> if debug is
> disabled as this can be expensive. I can write this:
>
>   if debug_log_enabled:  debugLog( ‘info is %r’ % (expensiveFunction(),) )
>
> But that is a more code then I would like to write. And if the debug code is 
> a performance problem cannot
> be left in the production code.
>
> I could combine the boolean and the log function by using a class to tidy up 
> the implementation.
>
> class DebugLog:
>   def __init__( self, enabled = False ):
>   self.enabled = enabled
>
>   def __bool__( self ):
>   return self.enabled
>
>   def __call__( self, msg ):
>   if self.enabled: print( ‘Debug: %s’ % (msg,) )
>
> And call like this:
>
>   dbg_log = DebugLog()
>
>If dbg_log: dbg_log( ‘a debug message’ )
>
> But I’d like to only write:
>
>   dbg_log( ‘a debug message’ )
>
> And have the evaluation of the argument skipped unless its dbg_log is enabled.
>
> I cannot see how to do this with python as it stands.
>
> Something would have to be added to allow python to short circuit the 
> argument tuple evaluation.
>
> Maybe python can check for a special dunder on the class that know how to do 
> this idiom, __if_true_call__?
>
> Thoughts?
>
> Barry
>
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Fwd: Define a method or function attributeoutsideof a class with the dot operator

[Kyle Lahnakoski]

On 2017-02-12 14:01, Joao S. O. Bueno wrote:
> On 12 February 2017 at 14:51, Markus Meskanen  
> wrote:
>> 1. Allowing the class to be used in the method's header, f.e. for typing and
>> decorators:
>>
>>   @decorate(MyClass)
>>   def MyClass.method(self, other: MyClass) -> List[MyClass]:
>>   ...
>>
>> This is useful since you can't refer the class itself inside of its body. At
>> the moment the way to use typing is to write the class's name as a string...
>> It feels awful.
> You realize now that if we accept this change, and given your example,
> any "well behaved" Python code with markup will in a  couple months
> required to be like
>
> class MyClass:
>   """Docstring."""
>
> def MyClass.__init__(self: MyClass, ...) -> None:
>  ...
>
> # add other methods here.

I am for method-outside-of-class form: If it is allowed, I will use it
extensively:

* instance methods and extension methods have the same form
* less lines between the line you are looking at and the name of the class
* explicit class name helps with searching for methods
* reduces indentation

thanks





___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Define a method or function attribute outside of a class with the dot operator

[Kyle Lahnakoski]

On 2017-02-10 05:44, Markus Meskanen wrote:
>
>
> On Fri, Feb 10, 2017 at 12:29 PM, Stephan Houben  > wrote:
>
> What about using a simple decorator instead?
>
> def monkey_patch(cls):
> return lambda func: setattr(cls, func.__name__, func)
>

I suggest naming the decorator with something less generic than
"monkey_patch", like "extension_method".





___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP 532: A circuit breaking operator and protocol

[Kyle Lahnakoski]

It would be nice if

coalesce(EXPR1, EXPR2, EXPR3)

Evaluated the N+1 argument only if the Nth evaluated to None.  Of course 
this may break the general patterns in the Python language. Maybe we can 
fake it by wrapping the expressions in lambdas:


coalesce(lambda: EXPR1(), lambda EXPR2(), lambda EXPR3())

and defining a `coalesce` as

def coalesce(*args):
for a in args:
a_val=a()
if a_val is not None:
return a_val
return None

Making a coalesce call looks painful, but allowing the called function 
to control the evaluation of its parameters may be useful. Suppose we 
can pass methods to functions; using generators to do so: Let & refer to 
lazy-evaluated parameters:


def coalesce(, ):
if a is None:
return b
else:
return a

c = coalesce(expr1(), expr2())

Would be converted to :

def coalesce(a, b):
a = yield
a_val = a()
if a_val is None:
b = yield
b_val = b()
return b_val
else:
return a_val

exprs = [expr1, expr2]
temp = coalesce()
for e in exprs:
try:
c = temp.next(e)
except StopIteration:
break

Or, even better...

def coalesce(*):
for a_val in args:
if a_val is not None:
return a_val
return None

c = coalesce(expr1(), expr2())

Gets converted to

def coalesce(*args):
for a in args:
a_val = a()
if a_val is not None:
return a_val
return None

exprs = [expr1, expr2]
temp = coalesce()
for e in exprs:
try:
c = temp.next(e)
except StopIteration:
break

...or something like that.  I can not think of other reasons for this 
type of expansion; maybe logical `and` can be given a magic method: 
"__logand__":


def __logand__(self, ):
if self:
return True
return o

c = my_object and some_other()

which has a combination of immediately-evaluated parameters, and 
lazy-evaluated parameters:


class MyClass(object):
def __logand__(self):
if self:
yield True
return
other = yield
return other()

exprs = [some_other]
temp = MyClass.__logand__(my_object)
for e in exprs:
try:
c = temp.next(e)
except StopIteration:
break

I hope that the acrobatics shown here might be easier to implement at a 
lower level; where in-line generator code collapses to simple branched 
logic.




On 11/13/2016 1:51 AM, Nick Coghlan wrote:


At that point, if we did decide to offer a builtin instead of 
dedicated syntax, the option I'd argue for is actually SQL's 
"coalesce": coalesce(EXPR1) else coalesce(EXPR2) else EXPR3 Yes, it's 
computer science jargon, but the operation itself is an odd one that 
doesn't really have an established mathematical precedent or 
grammatical English equivalent. Cheers, Nick.


___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Null coalescing operator

[Kyle Lahnakoski]

On 11/2/2016 2:30 PM, Zero Piraeus wrote:


If I write something like obj.attr, the failure mode I care about is that
obj has no attribute attr, rather than that obj is specifically None (or
one of a defined group of somewhat Nonelike objects).



I agree with this understanding.  The problem with None-coalescing is it 
doesn't consider where None came from.  I suspect enumerating the source 
of None values will reveal the majority of them are a result of 
`getattr(obj, attr)` returning None (or obj.get(attr) returning None).


If your code deals with annotated objects, rather than strictly typed 
objects, you will have many instances of attributes resolving to None. 
Making specific classes for each of the annotations, or combinations of 
annotations, is prohibitive, so you don't.  Rather, you use a few 
general types and set some attributes to None to indicate a property is 
not-relevant-for-this-object.


None checks are type checks. They are late type checks .




___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/