On 08/12/2021 23:09, David Mertz, Ph.D. wrote:
On Wed, Dec 8, 2021, 5:55 PM Rob Cliffe via Python-ideas
But AIUI (i.e. practically not at all) Dask is about parallel
computing, which is not the same thing as deferred evaluation,
though doubtless they overlap. Again AIUI, parallel computing is
mainly useful when you have multiple cores or multiple computers.
Much of Dask is about parallelism. But Dask Delayed really isn't. I
mean, yes it's a good adjunct to actual parallelism, but much of the
benefit is independent.
In particular, in Dask delayed—much as in a thoroughly lazy language
like Haskell—you can express a graph of interrelated computations that
you might POTENTIALLY perform.
There are many times when expressing those dependencies is useful,
even before you know which, if any, of them will actually need to be
performed. The site I linked as many more fleshed out examples, but
suppose I have this dataflow relationship:
A -> B -> C -> D -> E
Each of those letters name some expensive computation (or maybe
expensive I/O, or both).
In a particular run of our program, we might determine that we need
the data created by B. But in that particular run, we never wind up
using C, D or E. Of course, a different run, based on different
conditions, will actually need E.
In this simplest possible DAG, I've deliberately avoided any possible
parallelism. Every step entirely depends on the one before it. But
delayed compution can still be useful. Of course, when the DAG has
branches, often operating on branches can often be usefully
parallelized (but that's still not required for laziness to remain useful.
This is all abstract. You give no clue to what your application is or
what it is meant to do. Please, may I refer you to my previous post:
"/Can anyone give examples (in Python pseudo-code perhaps) showing
how *deferred evaluation* would be useful for a concrete task? (Solving
an equation. Drawing a graph. Analysing a document. Manufacturing a
widget. Planning a journey. Firing a missile. *Anything!* *You* name
it./)"
David? Anybody??
Best wishes
Rob Cliffe
_______________________________________________
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at
https://mail.python.org/archives/list/python-ideas@python.org/message/BKLACJG4ELMSWP73T76IHN2J4RTK66W4/
Code of Conduct: http://python.org/psf/codeofconduct/
