[
https://issues.apache.org/jira/browse/AIRFLOW-5567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17039991#comment-17039991
]
ASF GitHub Bot commented on AIRFLOW-5567:
-----------------------------------------
stale[bot] commented on pull request #6210: [AIRFLOW-5567]
BaseReschedulePokeOperator
URL: https://github.com/apache/airflow/pull/6210
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
> Improved primitive for building Operators that benefit from reschedule mode
> ---------------------------------------------------------------------------
>
> Key: AIRFLOW-5567
> URL: https://issues.apache.org/jira/browse/AIRFLOW-5567
> Project: Apache Airflow
> Issue Type: Improvement
> Components: models, operators
> Affects Versions: 1.10.5
> Reporter: Jacob Ferriero
> Assignee: Jacob Ferriero
> Priority: Trivial
>
> Often times airflow operators (derived from BaseOperator) kick-off a long
> running tasks and then waits / polls, blocking a worker slot until the long
> running task completes. This can be problematic in environments with many
> long running tasks.
> BaseSensorOperator was improved by implementing `reschedule` mode to solve
> the similar issue with long running sensors blocking a worker to poll for a
> long time.
> This issue is to track how we could provide a primitive that would make it
> easy to develop operators for long running tasks that reschedule a `poll`
> operation rather than blocking in their `execute` method.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
