Re: [Python-ideas] Delayed Execution via Keyword

Abe Dillon <abedil...@gmail.com> >>>>> wrote: >>>>> >>>>>> On Fri, Feb 17, 2017, Steven D'Aprano wrote: >>>>>> >>>>>> JIT compilation delays *compiling* the code to run-time. This is a >>>>>> prop

Re: [Python-ideas] Delayed Execution via Keyword

;>>>> My thought was that if a compiler is capable of determining what needs >>>>> to be compiled just in time, then an interpreter might be able to >>>>> determine >>>>> what expressions need to be evaluated just when their results are actually &g

Re: [Python-ideas] Delayed Execution via Keyword

%s", expensive()) >>>> >>>> The interpreter could skip evaluating the expensive function if the >>>> result is never used. It would only evaluate it "just in time". This would >>>> almost certainly require just in time compilation as well, othe

Re: [Python-ideas] Delayed Execution via Keyword

This would >>> almost certainly require just in time compilation as well, otherwise the >>> byte code that calls the "log.debug" function would be unaware of the byte >>> code that implements the function. >>> >>> This is probably a pipe-dream, though

Re: [Python-ideas] Delayed Execution via Keyword

t; function would be unaware of the byte >>> code that implements the function. >>> >>> This is probably a pipe-dream, though; because the interpreter would >>> have to be aware of side effects. >>> >>> >>> >>> On Mon,

Re: [Python-ideas] Delayed Execution via Keyword

to be aware of side effects. > > > > On Mon, Feb 20, 2017 at 5:18 AM, <tritium-l...@sdamon.com> wrote: > > > > > -Original Message- > > From: Python-ideas [mailto:python-ideas-bounces+tritium- > > list=sdamon....@python.org] On Behalf Of Miche

Re: [Python-ideas] Delayed Execution via Keyword

s+tritium- > > list=sdamon@python.org] On Behalf Of Michel Desmoulin > > Sent: Monday, February 20, 2017 3:30 AM > > To: python-ideas@python.org > > Subject: Re: [Python-ideas] Delayed Execution via Keyword > > > > I wrote a blog post about this, and s

Re: [Python-ideas] Delayed Execution via Keyword

> -Original Message- > From: Python-ideas [mailto:python-ideas-bounces+tritium- > list=sdamon@python.org] On Behalf Of Michel Desmoulin > Sent: Monday, February 20, 2017 3:30 AM > To: python-ideas@python.org > Subject: Re: [Python-ideas] Delayed Execution via Ke

Re: [Python-ideas] Delayed Execution via Keyword

I wrote a blog post about this, and someone asked me if it meant allowing lazy imports to make optional imports easier. Someting like: lazy import foo lazy from foo import bar So now if I don't use the imports, the module is not loaded, which could also significantly speed up applications

Re: [Python-ideas] Delayed Execution via Keyword

On 2/19/17, Michel Desmoulin wrote: > Evnetually we also may need to allow this: > > a = lazy stuff > if a is not lazy: > print(a) > > But then lazy can't be used a var name to help with the transition. What about this? if not inspect.islazy(a): print(a)

Re: [Python-ideas] Delayed Execution via Keyword

On Sun, Feb 19, 2017 at 10:47 AM, Joseph Hackman wrote: > Your argument has convinced me, and I now take (what i believe to be) your >> position: >> > > def stuff(arg = lazy f()): > > should result in a function where the default value of arg is not > evaluated until

Re: [Python-ideas] Delayed Execution via Keyword

> > This doesn't make sense. Function definition time is very different than > function execution time. Changing that distinction is a WAY bigger change > than I think we should contemplate. > Moreover, there is a completely obvious way to spell the behavior you want: > def stuff(): > arg =

Re: [Python-ideas] Delayed Execution via Keyword

On Sun, Feb 19, 2017 at 10:13 AM, Joseph Hackman wrote: > > My honest preference would be that the [] is evaluated fresh each time the > function is called. > def stuff(arg=delayed f()): > would result in f() being called every time stuff() is. This seems more > valuable

Re: [Python-ideas] Delayed Execution via Keyword

> > One last thing: my vote is not dropping the ":" in front of they > keyword. > > > I think the colon has parser problems, as I showed in some examples. > Plus I don't like how it looks. But I'd much rather have `a = lazy: > stuff` than not have the construct at all, nonetheless. >

Re: [Python-ideas] Delayed Execution via Keyword

On Sun, Feb 19, 2017 at 8:24 AM, Michel Desmoulin wrote: > A great proposal, although now I would have to explain to my students > the subtle difference between: > > res = (print(i * i) for i in range(x)) > res = delayed [print(i * i) for i in range(x)] > They

Re: [Python-ideas] Delayed Execution via Keyword

A great proposal, although now I would have to explain to my students the subtle difference between: res = (print(i * i) for i in range(x)) print('foo') print(res) And res = delayed [print(i * i) for i in range(x)] print('foo') all(res) They seems doing something similar, but they really

Re: [Python-ideas] Delayed Execution via Keyword

On Fri, Feb 17, 2017 at 9:34 PM, David Mertz wrote: > On Fri, Feb 17, 2017 at 6:20 PM, Nathaniel Smith wrote: >> >> value = delayed: some_dict.get("whatever") >> if value is None: >> ... >> >> I.e., the question is, how does 'is' work on delayed objects? I

Re: [Python-ideas] Delayed Execution via Keyword

On Fri, Feb 17, 2017 at 9:45 PM, David Mertz wrote: > That will make it pretty much impossible to tell whether something is a >> > delayed "thunk" or not, since *any* attempt to inspect it in any way >> will cause it to reify. Maybe that's what we want. > > > This feels like a

Re: [Python-ideas] Delayed Execution via Keyword

On Fri, Feb 17, 2017 at 5:23 PM, Steven D'Aprano wrote: > > try: > Aardvark > except NameError: > from backport import Aardvark > > No such thing is possible for new syntax. So that counts as a > disadvantage of new syntax. Are we positive that there *must* be new >

Re: [Python-ideas] Delayed Execution via Keyword

On Fri, Feb 17, 2017 at 6:20 PM, Nathaniel Smith wrote: > > value = delayed: some_dict.get("whatever") > if value is None: > ... > > I.e., the question is, how does 'is' work on delayed objects? I guess > it has to force the promise and walk the proxy chain in each input and >

Re: [Python-ideas] Delayed Execution via Keyword

On Fri, Feb 17, 2017 at 2:35 PM, Joseph Hackman wrote: > I think we should use the colon to make the delayed word (or whatever word > is selected), unambiguously used in this way (and to prevent any existing > code from breaking). > > On 17 February 2017 at 17:09, David

Re: [Python-ideas] Delayed Execution via Keyword

On Thu, Feb 16, 2017 at 9:24 PM, Joseph Hackman wrote: > Howdy All! > > This suggestion is inspired by the question on "Efficient debug logging". > > I propose a keyword to mark an expression for delayed/lazy execution, for > the purposes of standardizing such behavior

Re: [Python-ideas] Delayed Execution via Keyword

On 17 February 2017 at 20:23, Steven D'Aprano wrote: > > > I think it would be key, like async/await, to narrowly define the scope > in > > which the word delayed functions as a keyword. > > The PEP makes it clear that's just a transition phase: they will be > turned into

Re: [Python-ideas] Delayed Execution via Keyword

On Fri, Feb 17, 2017 at 04:02:01PM -0600, Abe Dillon wrote: > I'm fairly novice, so I could be way off base here, but it seems like the > inevitable conclusion to this problem is something like JIT compilation, > right? (admittedly, I know very little about JIT compilation) No. JIT compilation

Re: [Python-ideas] Delayed Execution via Keyword

On Fri, 17 Feb 2017 at 23:14 Joshua Morton wrote: > @ Ed > > Its my understanding that d[k] is always d[k], even if d or k or both are > delayed. On the other hand, `delayed d[k]` would not be, but you would need > to explicitly state that. I think its worth expanding

Re: [Python-ideas] Delayed Execution via Keyword

On 17 February 2017 at 18:13, Joshua Morton wrote: > @ Joseph > > Function annotations can be arbitrary python expressions, it is completely > legal to have something like > > >>> def foo(bar: lambda x: x + 1): > ... pass > > Why you would want that I can't

Re: [Python-ideas] Delayed Execution via Keyword

@ Joseph Function annotations can be arbitrary python expressions, it is completely legal to have something like >>> def foo(bar: lambda x: x + 1): ... pass Why you would want that I can't say, but it is legal. In the same way, `def foo(bar: delayed 1 + 1)` should probably be legal

Re: [Python-ideas] Delayed Execution via Keyword

On 17 February 2017 at 06:10, Steven D'Aprano wrote: > On Fri, Feb 17, 2017 at 12:24:53AM -0500, Joseph Hackman wrote: > > > I propose a keyword to mark an expression for delayed/lazy execution, for > > the purposes of standardizing such behavior across the language. > > > >

Re: [Python-ideas] Delayed Execution via Keyword

I think we should use the colon to make the delayed word (or whatever word is selected), unambiguously used in this way (and to prevent any existing code from breaking). On 17 February 2017 at 17:09, David Mertz wrote: > That was a problem with the colon that occurred to me. I

Re: [Python-ideas] Delayed Execution via Keyword

That was a problem with the colon that occurred to me. I think it can't be tokenized in function annotations. Plus I still think the no-colon looks better. But that's bikeshedding. Also other words are plausible. I like lazy even more than delayed, I think. Still, I'd love the construct whatever

Re: [Python-ideas] Delayed Execution via Keyword

On Fri, 17 Feb 2017 at 21:58 Joshua Morton wrote: > Ed, I'm not seeing this perceived problem either. > > if we have > > >>> d = delayed {'a': 1, 'b': 2} # I'm not sure how this is delayed > exactly, but sure > >>> k = delayed string.ascii_lowercase[0] >

Re: [Python-ideas] Delayed Execution via Keyword

On Fri, 17 Feb 2017 at 21:57 Joseph Jevnik wrote: > > You should be able to pass the result to *any* existing code that > expects a function and sometimes calls it, and the function should be > called when that happens, rather than evaluated to a delayed object and > then

Re: [Python-ideas] Delayed Execution via Keyword

Ed, I'm not seeing this perceived problem either. if we have >>> d = delayed {'a': 1, 'b': 2} # I'm not sure how this is delayed exactly, but sure >>> k = delayed string.ascii_lowercase[0] >>> d[k] 1 I'm not sure how the delayedness of any of the subexpressions matter, since

Re: [Python-ideas] Delayed Execution via Keyword

On Fri, 17 Feb 2017 at 21:21 Joseph Jevnik wrote: > About the "whatever is d[k]" in five minutes comment: If I created an > explict closure like: `thunk = lambda: d[k]` and then mutated `d` before > evaluating the closure you would have the same issue. I don't think it is >

Re: [Python-ideas] Delayed Execution via Keyword

On Fri, 17 Feb 2017 at 21:18 Joseph Jevnik wrote: > There is no existing code that uses delayed execution so we don't need to > worry about breaking it. > I think you're missing the point here. This thing is transparent—that's sort of the entire point—so you can pass delayed

Re: [Python-ideas] Delayed Execution via Keyword

There is no existing code that uses delayed execution so we don't need to worry about breaking it. I think it would be much easier to reason about if forcing an expression was always explicit. I am not sure what you mean with the second case; why are you delaying a function if you care about the

Re: [Python-ideas] Delayed Execution via Keyword

On Fri, 17 Feb 2017 at 19:38 Joseph Jevnik wrote: > Delayed execution and respecting mutable semantics seems like a nightmare. > For most indexers we assume hashability which implies immutability, why > can't we also do that here? Also, why do we need to evaluate callables >

Re: [Python-ideas] Delayed Execution via Keyword

> > Couldn't the same thing be true of delayed if it is always followed by a > colon? No. Because there are other reasons you'd follow the variable `delayed` with a colon: >>> delayed = 1 >>> d = {delayed: "oops!"} My earlier proposal (using unpacking syntax) doesn't work for the same reason.

Re: [Python-ideas] Delayed Execution via Keyword

I think it could even be true without, but the colon may cause ambiguity problems with function annotations. def foo(delayed: delayed: 1 + 2) is a bit odd, especially if `delayed` is chainable. --Josh On Fri, Feb 17, 2017 at 3:32 PM Joseph Hackman wrote: >

Re: [Python-ideas] Delayed Execution via Keyword

Couldn't the same thing be true of delayed if it is always followed by a colon? I.e. delayed=1 x= delayed: slow_function() print(delayed) # prints 1 -Joseph > On Feb 17, 2017, at 2:39 PM, Mark E. Haase wrote: > >> On Fri, Feb 17, 2017 at 1:55 PM, Joshua Morton

Re: [Python-ideas] Delayed Execution via Keyword

On Fri, Feb 17, 2017 at 1:55 PM, Joshua Morton wrote: > but I'm wondering how common async and await were when that was proposed > and accepted? Actually, "async" and "await" are backwards compatible due to a clever tokenizer hack. The "async" keyword may only appear

Re: [Python-ideas] Delayed Execution via Keyword

Delayed execution and respecting mutable semantics seems like a nightmare. For most indexers we assume hashability which implies immutability, why can't we also do that here? Also, why do we need to evaluate callables eagerly? re the thunk replacing itself with the result instead of memoizing the

Re: [Python-ideas] Delayed Execution via Keyword

I think trying to eager-ify subexpressions is absurdly difficult to do right, and also a problem that occurs in other places in Python already, so solving it only for this new thing that might very well go no further is a bit odd. I don't think versions that aren't transparent are much use. >

Re: [Python-ideas] Delayed Execution via Keyword

I did some quick thinking and a bit of research about some aspects of this proposal: There are a number of keyword options (delay, defer, lazy, delayed, deferred, etc.), a quick look through github says that of these, "deferred" seems to be the least used, but it still comes up quite a lot (350K

Re: [Python-ideas] Delayed Execution via Keyword

I'd like to suggest a shorter keyword: `lazy` This isn't an endorsement. I haven't had time to digest how big this change would be. If this is implemented, I'd also like to suggest that perhaps packing and unpacking should be delayed by default and not evaluated until the contents are used. It

Re: [Python-ideas] Delayed Execution via Keyword

Agreed. I think this may require some TLC to get right, but posting here for feedback on the idea overall seemed like a good start. As far as I know, the basic list and dict do not inspect what they contain. I.e. d = {} d['a']= delayed: stuff() b=d['a'] b would end up as still the thunk, and

Re: [Python-ideas] Delayed Execution via Keyword

On Sat, Feb 18, 2017 at 3:29 AM, Joseph Hackman wrote: > ChrisA: I am not sure about collections. I think it may be fine to not > special case it: if the act of putting it in the collection reads anything, > then it is evaluated, and if it doesn't it isn't. The ideal

Re: [Python-ideas] Delayed Execution via Keyword

Pavol: I think that some sort of magic string that is not a string and is actually containing Python code could function, but is less elegant. ChrisA: I am not sure about collections. I think it may be fine to not special case it: if the act of putting it in the collection reads anything, then

Re: [Python-ideas] Delayed Execution via Keyword

On Sat, Feb 18, 2017 at 2:12 AM, Joseph Hackman wrote: > As for what triggers execution? I think everything except being on the right > side of an assignment. Even identity. So if a delayed expression would > evaluate to None, then code that checks is None should return

Re: [Python-ideas] Delayed Execution via Keyword

On Fri, Feb 17, 2017 at 10:10 PM, Steven D'Aprano wrote: >> the expression is executed and the delayed >> expression is replaced with the result. (Thus, the delayed expression is >> only every evaluated once). > > That's easily done by having the "delayed" keyword cache each

Re: [Python-ideas] Delayed Execution via Keyword

On Fri, Feb 17, 2017 at 12:24:53AM -0500, Joseph Hackman wrote: > I propose a keyword to mark an expression for delayed/lazy execution, for > the purposes of standardizing such behavior across the language. > > The proposed format is: > delayed: > i.e. log.info("info is %s", delayed:

Re: [Python-ideas] Delayed Execution via Keyword

Hi all, If we want this it might be interesting to investigate what the Scheme community has been doing, since they have had this (under the name "promises") for many years. Basically: Scheme: (delay expr) <=> proposed Python: delayed: expr The Scheme community has experimented with what

Re: [Python-ideas] Delayed Execution via Keyword

Even with the new syntax I would highly discourage delaying a function with observable side effects. It would make reasoning about the behavior of the program very difficult and debugging becomes much harder. On Fri, Feb 17, 2017 at 3:31 AM, David Mertz wrote: > I had forgotten

Re: [Python-ideas] Delayed Execution via Keyword

I had forgotten about Daisy! It's an interesting project too. The behavior of 'autodask()' is closer to what I'd want in new syntax than is plain dask.delayed(). I'm not sure of all the corners. But is definitely love to have it for expressions generally, not only pure functions. On Feb 17, 2017

Re: [Python-ideas] Delayed Execution via Keyword

You can let dask "see" into the function by entering it and wrapping all of the operations in `delayed`; this is how daisy[0] builds up large compute graphs. In this case, you could "inline" the identity function and the delayed object would flow through the function and the call to identity never

Re: [Python-ideas] Delayed Execution via Keyword

On Thu, Feb 16, 2017 at 11:15 PM, David Mertz wrote: > > This also means that a 'delayed' object needs to be idempotent. So > > x = delayed 2+2 > > y = delayed x > > z = delayed delayed delayed y > > > Wrapping more delays around an existing delayed object should probably > just

Re: [Python-ideas] Delayed Execution via Keyword

I think maybe the idiomatic pattern should be assignment rather than just bare name. E.g. f = delayed 1 + 2 # We want to evaluate f before next op for some reason *f = f* # f is already a concrete value now, before calculating g g = f * 7 I think if we follow my rule that "everything lazy

Re: [Python-ideas] Delayed Execution via Keyword

On Thu, Feb 16, 2017 at 10:33 PM, Joshua Morton wrote: > David, can you elaborate on your example? > > if we replaced line four with > > >>> x = my_lazy_func(b, delayed c) > > what would the value of `x` be, and how would this differ from either > The value of the

Re: [Python-ideas] Delayed Execution via Keyword

David, can you elaborate on your example? if we replaced line four with >>> x = my_lazy_func(b, delayed c) what would the value of `x` be, and how would this differ from either >>> x = delayed my_lazy_func(b, delayed c) or >>> x = delayed my_lazy_func(b, c) To put it another

Re: [Python-ideas] Delayed Execution via Keyword

Dask also has a function delayed() that may be used as an decorator and in other ways like: >>> from dask import delayed >>> from operator import add, mul >>> a = delayed(add)(1, 2) >>> b = delayed(mul)(a, 3) >>> b Delayed('mul-1907f29b-60a4-48af-ba2a-938556555f9b') >>> c = b.compute() >>> c

Re: [Python-ideas] Delayed Execution via Keyword

I rather like this at first brush! On Feb 16, 2017 9:25 PM, "Joseph Hackman" wrote: > Howdy All! > > This suggestion is inspired by the question on "Efficient debug logging". > > I propose a keyword to mark an expression for delayed/lazy execution, for > the purposes of

[Python-ideas] Delayed Execution via Keyword

Howdy All! This suggestion is inspired by the question on "Efficient debug logging". I propose a keyword to mark an expression for delayed/lazy execution, for the purposes of standardizing such behavior across the language. The proposed format is: delayed: i.e. log.info("info is %s", delayed: