[Python-ideas] Re: Generalized deferred computation in Python

2022-06-22 Thread Martin Di Paola
Hi David, I read the PEP and I think it would be useful to expand the Motivation and Examples sections. While indeed Dask uses lazy evaluation to build a complex computation without executing it, I don't think that it is the whole story. Dask takes this deferred complex computation and *plans* h

[Python-ideas] Re: Generalized deferred computation in Python

2022-06-22 Thread Steve Jorgensen
I think I have an idea how to do something like what you're asking with less magic, and I think an example implementation of this could actually be done in pure Python code (though a more performant implementation would need support at the C level). What if a deferred object has 1 magic method

[Python-ideas] Re: Generalized deferred computation in Python

2022-06-22 Thread David Mertz, Ph.D.
Hi Martin, Short answer: yes, I agree. Slightly longer: I would be eternally grateful if you wish to contribute to the PEP with any such expansion of the Motivation and Expansion. On Wed, Jun 22, 2022 at 10:47 AM Martin Di Paola wrote: > Hi David, I read the PEP and I think it would be useful t

[Python-ideas] Re: Generalized deferred computation in Python

2022-06-22 Thread Paul Moore
On Wed, 22 Jun 2022 at 18:35, David Mertz, Ph.D. wrote: > > Hi Martin, > > Short answer: yes, I agree. > Slightly longer: I would be eternally grateful if you wish to contribute to > the PEP with any such expansion of the Motivation and Expansion. One concern I have, triggered by Martin's Dask,

[Python-ideas] Re: Generalized deferred computation in Python

2022-06-22 Thread Eric V. Smith via Python-ideas
> On Jun 22, 2022, at 2:12 PM, Paul Moore wrote: > > On Wed, 22 Jun 2022 at 18:35, David Mertz, Ph.D. > wrote: >> >> Hi Martin, >> >> Short answer: yes, I agree. >> Slightly longer: I would be eternally grateful if you wish to contribute to >> the PEP with any such expansion of the Motivat

[Python-ideas] Re: Generalized deferred computation in Python

2022-06-22 Thread David Mertz, Ph.D.
On Wed, Jun 22, 2022 at 10:47 AM Martin Di Paola wrote: > Hi David, I read the PEP and I think it would be useful to expand the > Motivation and Examples sections. > While indeed Dask uses lazy evaluation to build a complex computation > without executing it, I don't think that it is the whole st

[Python-ideas] Re: Generalized deferred computation in Python

2022-06-22 Thread David Mertz, Ph.D.
On Wed, Jun 22, 2022 at 2:17 PM Eric V. Smith wrote: > Every time I’ve looked at this, I come back to: other than the clunky > syntax, how is explicit evaluation different from a zero-argument lambda? The difference is in composition of operations. I can write a dozen zero-argument lambdas eas

[Python-ideas] Re: Generalized deferred computation in Python

2022-06-22 Thread Neil Girdhar
Could this idea be used to specify the default factory of a dataclass field? For example, @dataclass class X: x: list[int] = deferred [] instead of @dataclass class X: x: list[int] = field(default_factory=list) If so, it might be worth adding to your proposal as another common motivating

[Python-ideas] Re: Generalized deferred computation in Python

2022-06-22 Thread Barry Scott
> On 22 Jun 2022, at 19:09, Paul Moore wrote: > > I suspect that you consider evaluation-on-reference as an important > feature of your proposal, but could you consider explicit evaluation > as an alternative? Or at the very least address in the PEP the fact > that this would close the door on

[Python-ideas] Re: Generalized deferred computation in Python

2022-06-22 Thread Martin Di Paola
On Wed, Jun 22, 2022 at 02:30:14PM -0400, David Mertz, Ph.D. wrote: The difference is in composition of operations. I can write a dozen zero-argument lambdas easily enough. But those are all isolated. But basically, think about `x = (later expensive1() + later expensive2()) / later expensive3(

[Python-ideas] Re: Generalized deferred computation in Python

2022-06-22 Thread Paul Moore
On Wed, 22 Jun 2022 at 22:52, Martin Di Paola wrote: > Perhaps this is the real focus to analyze. The PEP suggests that > x.compute() does something *fundamentally* different from just executing > the intermediate expressions. Hang on, did the PEP change? The version I saw didn't have a compute()

[Python-ideas] Ellipsis (...) to be roughly synonymous with * in destructuring but without capture.

2022-06-22 Thread Steve Jorgensen
This is based on previous discussions of possible ways of matching all remaining items during destructuring but without iterating of remaining final items. This is not exactly a direct replacement for that idea though, and skipping iteration of final items might or might not be part of the goal.

[Python-ideas] Re: Ellipsis (...) to be roughly synonymous with * in destructuring but without capture.

2022-06-22 Thread Steve Jorgensen
Steve Jorgensen wrote: > This is based on previous discussions of possible ways of matching all > remaining items during destructuring but without iterating of remaining final > items. This is not exactly a direct replacement for that idea though, and > skipping iteration of final items might or

[Python-ideas] Re: Ellipsis (...) to be roughly synonymous with * in destructuring but without capture.

2022-06-22 Thread Chris Angelico
On Thu, 23 Jun 2022 at 08:56, Steve Jorgensen wrote: > > This is based on previous discussions of possible ways of matching all > remaining items during destructuring but without iterating of remaining final > items. This is not exactly a direct replacement for that idea though, and > skipping

[Python-ideas] Re: Generalized deferred computation in Python

2022-06-22 Thread Rob Cliffe via Python-ideas
Thank you for your proposal David.  At last we have a counter-proposal to talk about.  A few points: (1) (As I pointed out in an earlier post) There is a flaw in using the syntax of an expression PRECEDED by a SOFT keyword:     x     =    later    -y With your proposal, x is assigned a deferre

[Python-ideas] Re: Generalized deferred computation in Python

2022-06-22 Thread Chris Angelico
On Thu, 23 Jun 2022 at 10:44, Rob Cliffe via Python-ideas wrote: > > Thank you for your proposal David. At last we have a counter-proposal to > talk about. A few points: > > (1) (As I pointed out in an earlier post) There is a flaw in using the syntax > of an expression PRECEDED by a SOFT keyw

[Python-ideas] Re: Generalized deferred computation in Python

2022-06-22 Thread Joao S. O. Bueno
Martin Di Paola wrote: > Three cases: Dask/PySpark, Django's ORM and selectq. All of them > implement deferred expressions but all of them "compute" them in very > specific ways (aka, they plan and execute the computation differently). So - I've been hit with the "transparency execution of deferr

[Python-ideas] Re: Generalized deferred computation in Python

2022-06-22 Thread Martin Di Paola
On Wed, Jun 22, 2022 at 11:22:05PM +0100, Paul Moore wrote: Hang on, did the PEP change? The version I saw didn't have a compute() method, deferred objects were just evaluated when they were referenced. You are right, the PEP does not mention a compute() method but uses the that term. I just

[Python-ideas] Re: Generalized deferred computation in Python

2022-06-22 Thread David Mertz, Ph.D.
Thanks Rob, I recognize that I have so-far skirted the order-of-precedence concern. I believe I have used parens in my example everywhere there might be a question... But that's not a general description or rule. I have a bunch of issues that I know I need to flesh out, many coming as suggestions

[Python-ideas] Re: Ellipsis (...) to be roughly synonymous with * in destructuring but without capture.

2022-06-22 Thread Steve Jorgensen
> No need to have an object there - you could just define it as a syntactic > construct instead. Assignment targets aren't themselves objects (although the > same syntax can often be used on the RHS, when it would resolve to one). Right. Thanks. That _should_ have been obvious. :) > Having a wa

[Python-ideas] dataclass field argument to allow converting value on init

2022-06-22 Thread Dexter Hill
The idea is to have a `default_factory` like argument (either in the `field` function, or a new function entirely) that takes a function as an argument, and that function, with the value provided by `__init__`, is called and the return value is used as the value for the respective field. For exa

[Python-ideas] Re: Generalized deferred computation in Python

2022-06-22 Thread Ben Rudiak-Gould
> > On Wed, Jun 22, 2022 at 02:30:14PM -0400, David Mertz, Ph.D. wrote: > >But basically, think about `x = (later expensive1() + later expensive2()) > / > >later expensive3()`. How can we make `x` itself be a zero argument > >lambda? [... see below ...] > x = lambda: (expensive1() + expensive2

[Python-ideas] Re: Generalized deferred computation in Python

2022-06-22 Thread Ben Rudiak-Gould
On Wed, Jun 22, 2022 at 6:36 PM Joao S. O. Bueno wrote: > implement "all possible" > dunder methods, and proxy those to the underlying object, for a "future > type" that was > calculated off-process, and did not need any ".value()" or ".result()" > methods to be called. > Here's a package on PyP

[Python-ideas] Re: Generalized deferred computation in Python

2022-06-22 Thread Chris Angelico
On Thu, 23 Jun 2022 at 11:35, Joao S. O. Bueno wrote: > > Martin Di Paola wrote: > > Three cases: Dask/PySpark, Django's ORM and selectq. All of them > > implement deferred expressions but all of them "compute" them in very > > specific ways (aka, they plan and execute the computation differently)