aa1371 commented on a change in pull request #16829:
URL: https://github.com/apache/airflow/pull/16829#discussion_r673831944
##########
File path: airflow/executors/dask_executor.py
##########
@@ -81,7 +81,22 @@ def airflow_run():
if not self.client:
raise AirflowException(NOT_STARTED_MESSAGE)
- future = self.client.submit(airflow_run, pure=False)
+ resources = None
+ if queue is not None:
+ avail_resources = self.client.run_on_scheduler(
+ lambda dask_scheduler: dask_scheduler.resources
+ )
Review comment:
Cool thanks! That helped me get it working with a couple extra tweaks.
The `get_resources` method of the plugin class needed to a argument (looks
like a comm.tcp.TCP objet being passed in to it). Also needed to call
client.sync like so: `client.sync(client.scheduler.get_resources)`, without the
invocation of get_resources.
That said, how do you feel about client.get_worker_resources for the reason
I listed above? I think that would be the better approach, since I think that's
something users should easily be able to lookup.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
