We're not using LocalExecutor. It's CeleryExecutor. We're running on
virtual hardware and perhaps the signal module is simply slow. (It
shouldn't be this slow.) We *do* get a timeout eventually. Just not at
the 10 minute mark. It doesn't trip until 5 hours.
On 8 Jun 2016, at 8:22, Maxime Beauchemin wrote:
I created this JIRA to document the issue.
https://issues.apache.org/jira/browse/AIRFLOW-221
Max
On Wed, Jun 8, 2016 at 8:16 AM, Maxime Beauchemin <
[email protected]> wrote:
Are you using the LocalExecutor? there's a limitation around using
the
task timeout in LocalExecutor under some circumstances. If that's the
case
you should see this somewhere in your logs:
https://github.com/apache/incubator-airflow/blob/master/airflow/utils/timeout.py#L45
We may want to move the logic enforcing the timeout from the context
manager/signal approach to BaseJob.
Max
On Tue, Jun 7, 2016 at 5:47 PM, Alyssa Kwan
<[email protected]>
wrote:
Hi,
We're having trouble with a `PythonOperator`'s `execution_timeout`.
It's
being set directly in the constructor as
`datetime.timedelta(minutes=10)`.
However, the task doesn't timeout until almost 5 hours is passed.
This
behavior is pretty consistent. It's merely a SQL query issued by the
Python
callable that can get deadlocked.
Any thoughts?
Thanks,
Alyssa
