[jira] [Commented] (AIRFLOW-5544) GCP datastore operation pull should have some retry to avoid false alarm

[Kamil Bregula (Jira)]

    [ 
https://issues.apache.org/jira/browse/AIRFLOW-5544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16982961#comment-16982961
 ] 

Kamil Bregula commented on AIRFLOW-5544:
----------------------------------------

Some body from community make change:

[https://github.com/apache/airflow/commit/16e7e6175ae16d73d84c86163686a6c0b9b707ff#diff-d67992de738848a0b4278915ffd9b797]

> GCP datastore operation pull should have some retry to avoid false alarm
> ------------------------------------------------------------------------
>
>                 Key: AIRFLOW-5544
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-5544
>             Project: Apache Airflow
>          Issue Type: Bug
>          Components: gcp
>    Affects Versions: 1.10.5
>            Reporter: Yang Hu
>            Priority: Minor
>
> Hi Airflow,
> We noticed that sometime, google status url will return us 500 and airflow 
> will mark the whole task failed, but is actually runs fine.
>  
> I think we need some retry before failing the task, at least at this 500 case
> Sample request:
> > [2019-09-18 21:45:19,543] \{base_task_runner.py:98} INFO - Subtask: 
> googleapiclient.errors.HttpError: &lt;HttpError 500 when requesting <a 
> href="https://datastore.googleapis.com/v1/projects/MASKED/operations/AS_MASKED_RI?alt=json";
>  
> target="_blank">https://datastore.googleapis.com/v1/projects/MASKED/operations/AS_MASKED_RI?alt=json</a>
>  returned &quot;Unknown Error.&quot;&gt;
> Ref code:
> [https://github.com/apache/airflow/blob/7e4330cce0b4f333b1658cdc315b06505cf9dd76/airflow/gcp/hooks/datastore.py#L269]



--
This message was sent by Atlassian Jira
(v8.3.4#803005)