cisaacstern commented on code in PR #27618:
URL: https://github.com/apache/beam/pull/27618#discussion_r1362857088
##########
sdks/python/apache_beam/runners/dask/dask_runner.py:
##########
@@ -177,6 +183,6 @@ def run_pipeline(self, pipeline, options):
dask_visitor = self.to_dask_bag_visitor()
pipeline.visit(dask_visitor)
-
- futures = client.compute(list(dask_visitor.bags.values()))
+ opt_graph = dask.optimize(list(dask_visitor.bags.values()))
Review Comment:
Reverted this in
https://github.com/apache/beam/pull/27618/commits/9557e3aba4b67ca85d770bcfd9cc677affce2178
... I wish I could explain why I did in the first place but looks like my test
env was just deeply messed up in some unknown way 🤦 . Ok, onward!
##########
sdks/python/apache_beam/runners/dask/dask_runner.py:
##########
@@ -177,6 +183,6 @@ def run_pipeline(self, pipeline, options):
dask_visitor = self.to_dask_bag_visitor()
pipeline.visit(dask_visitor)
-
- futures = client.compute(list(dask_visitor.bags.values()))
+ opt_graph = dask.optimize(list(dask_visitor.bags.values()))
Review Comment:
Reverted this in
https://github.com/apache/beam/pull/27618/commits/9557e3aba4b67ca85d770bcfd9cc677affce2178
... I wish I could explain why I did in the first place but looks like my test
env was just deeply messed up in some unknown way 🤦 . Ok, onward!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
