[
https://issues.apache.org/jira/browse/AIRFLOW-656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Riccomini updated AIRFLOW-656:
------------------------------------
Description:
We are seeing task failures with this periodically:
{noformat}
(_mysql_exceptions.OperationalError) (1205, 'Lock wait timeout
exceeded; try restarting transaction') [SQL: u'DELETE FROM xcom WHERE
xcom.`key` = %s AND xcom.execution_date = %s AND xcom.task_id = %s AND
xcom.dag_id = %s'] [parameters: (u'return_value',
datetime.datetime(2016, 11, 28, 15, 47),
'gcs2bq_fraud.fraud_rules_executions',
'db_monolith_fraud_fraud_rules_executions_partition_15m')]
{noformat}
According to [[email protected]]:
{quote}
I just looked quickly and it looks like no index is defined on the XCom
model beyond the PK. We should add a composite btree index on dag_id,
task_id and execution_date. I don't think MySQL will let you include `key`
in there as it's too long (512). The 3 other fields are plenty selective so
that will work just as well.
Creating the index straight in the db should fix the problem right away in
your environment, but it'd be nice to add it to the model definition and
generate the db migration script for it.
{quote}
was:
We are seeing task failures with this periodically:
(_mysql_exceptions.OperationalError) (1205, 'Lock wait timeout
exceeded; try restarting transaction') [SQL: u'DELETE FROM xcom WHERE
xcom.`key` = %s AND xcom.execution_date = %s AND xcom.task_id = %s AND
xcom.dag_id = %s'] [parameters: (u'return_value',
datetime.datetime(2016, 11, 28, 15, 47),
'gcs2bq_fraud.fraud_rules_executions',
'db_monolith_fraud_fraud_rules_executions_partition_15m')]
According to [[email protected]]:
{quote}
I just looked quickly and it looks like no index is defined on the XCom
model beyond the PK. We should add a composite btree index on dag_id,
task_id and execution_date. I don't think MySQL will let you include `key`
in there as it's too long (512). The 3 other fields are plenty selective so
that will work just as well.
Creating the index straight in the db should fix the problem right away in
your environment, but it'd be nice to add it to the model definition and
generate the db migration script for it.
{quote}
> Lock wait timeout exceeded on XCOM table
> ----------------------------------------
>
> Key: AIRFLOW-656
> URL: https://issues.apache.org/jira/browse/AIRFLOW-656
> Project: Apache Airflow
> Issue Type: Bug
> Components: xcom
> Affects Versions: Airflow 1.7.1.2
> Reporter: Chris Riccomini
> Assignee: Chris Riccomini
>
> We are seeing task failures with this periodically:
> {noformat}
> (_mysql_exceptions.OperationalError) (1205, 'Lock wait timeout
> exceeded; try restarting transaction') [SQL: u'DELETE FROM xcom WHERE
> xcom.`key` = %s AND xcom.execution_date = %s AND xcom.task_id = %s AND
> xcom.dag_id = %s'] [parameters: (u'return_value',
> datetime.datetime(2016, 11, 28, 15, 47),
> 'gcs2bq_fraud.fraud_rules_executions',
> 'db_monolith_fraud_fraud_rules_executions_partition_15m')]
> {noformat}
> According to [[email protected]]:
> {quote}
> I just looked quickly and it looks like no index is defined on the XCom
> model beyond the PK. We should add a composite btree index on dag_id,
> task_id and execution_date. I don't think MySQL will let you include `key`
> in there as it's too long (512). The 3 other fields are plenty selective so
> that will work just as well.
> Creating the index straight in the db should fix the problem right away in
> your environment, but it'd be nice to add it to the model definition and
> generate the db migration script for it.
> {quote}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
