Could this idea be used to specify the default factory of a dataclass
field? For example,
@dataclass
class X:
x: list[int] = deferred []
instead of
@dataclass
class X:
x: list[int] = field(default_factory=list)
If so, it might be worth adding to your proposal as another common
motivating example?
Best,
Neil
On Wednesday, June 22, 2022 at 2:23:13 PM UTC-4 David Mertz, Ph.D. wrote:
> On Wed, Jun 22, 2022 at 10:47 AM Martin Di Paola <martinp...@gmail.com>
> wrote:
>
>> Hi David, I read the PEP and I think it would be useful to expand the
>> Motivation and Examples sections.
>> While indeed Dask uses lazy evaluation to build a complex computation
>> without executing it, I don't think that it is the whole story.
>> Dask takes this deferred complex computation and *plans* how to execute it
>> and then it *executes* it in non-obvious/direct ways.
>>
>
> Dask is very clever about execution planning. Ray is possibly even more
> clever in that regard.
>
> However, I think that that should be an explicit non-goal of the PEP.
> DeferredObjects should create a DAG, yes. But I think Python itself should
> not think about being *clever* in evaluating that DAG, nor in itself think
> about parallelism. If my PEP were adopted, that would be something other
> libraries like Dask or Django could build on top of with more elaborate
> evaluation plans.
>
> But just the DAG itself gets you more than just "wait until needed to do
> the final computation." It allows for intermediate computation of nodes of
> the DAG lower down than the final result. For example, imagine
> dependencies like this (where all the computation steps are expensive):
>
> A -> B
> B -> Z
> B -> Y
> B -> X
> A -> C
> C -> X
> X -> F
> X -> G
> C -> W
> C -> V
> A -> D
> D -> V
> D -> U
> D -> T
>
> Hopefully you either see my ASCII art in fixed font, or it's at least
> intelligible. If I want to force evaluation of A, I need to do
> everything. But if it turns out all I need within my program is C, then I
> have to do computations C, X, F, G, W, V. Which is maybe still expensive,
> but at least I don't worry about B, Z, Y, U, T, or A.
>
> Yes, I should add something like this to the PEP.
>
>
>
_______________________________________________
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at
https://mail.python.org/archives/list/python-ideas@python.org/message/US5MCILKVRPUQBUERK352JKTH7RNJMTJ/
Code of Conduct: http://python.org/psf/codeofconduct/
