oneturkmen commented on issue #27358: URL: https://github.com/apache/airflow/issues/27358#issuecomment-1300724007
Thanks Jed. That makes much more sense now. On Wed, Nov 2, 2022, 11:31 Jed Cunningham ***@***.***> wrote: > I think you are misunderstanding what KubernetesExecutor is actually > doing. KE spins up a Airflow worker pod for every task. In your case, it'll > spin up a pod and say "Airflow, run task 'foo_task' for dag 'foo_dag' > run_id 'manual__...'" (which matches the args KE sets). That worker then > will run your (in this case) bash_command (or do whatever else you've asked > it to do). > > KPO is a different situation. The conceptual "kubectl create pod" is > replacing the bash_command, but it still runs from an Airflow worker. > > Short version: You want to put all your task specific logic in > bash_command when doing a BashSensor. Bonus, this keeps it portable between > executors! > > I actually gave a talk that covered this at Airflow Summit this year, it's > short so might be worth a watch: https://youtu.be/H8JjhiVGOlg > > — > Reply to this email directly, view it on GitHub > <https://github.com/apache/airflow/issues/27358#issuecomment-1300686637>, > or unsubscribe > <https://github.com/notifications/unsubscribe-auth/AEJDMLAUFQXP4OLIXJA5RXTWGKCLXANCNFSM6AAAAAARRRH2TI> > . > You are receiving this because you were mentioned.Message ID: > ***@***.***> > -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
