dabla commented on issue #44618: URL: https://github.com/apache/airflow/issues/44618#issuecomment-2568104948
> Hello! Thank you very much for taking the time to look at the code and making this suggestion. I tried to reduce the `check_interval` to 5, unfortunately it did not seem to fix the issue on our side. > > Also, I thought that the default timeout was 1 week because of this line: > > https://github.com/apache/airflow/blob/608de6b97ef04307559d79b2044899adfb9b2a93/providers/src/airflow/providers/microsoft/azure/triggers/powerbi.py#L57 > > But maybe I got this wrong. Am I mistaken? > > Regarding the bug, I think the issue may come from the very first request to get the refresh status, at this line: > > https://github.com/apache/airflow/blob/f7da5e48a2740a2bfb1c3dd57bb463ae29599a26/providers/src/airflow/providers/microsoft/azure/triggers/powerbi.py#L116 > > This request is made by the PowerBi hook (using the method `get_refresh_details_by_refresh_id`) and I think that, sometimes, the API is not yet "up-to-date" with the latest refresh IDs and can't find the status of our refresh ID. So, we fall in this case: > > https://github.com/apache/airflow/blob/f7da5e48a2740a2bfb1c3dd57bb463ae29599a26/providers/src/airflow/providers/microsoft/azure/hooks/powerbi.py#L159 > > and the hook raises the error "Unable to fetch the details of dataset refresh with Request Id" (which is also what I find in my Airflow logs). Do you think it makes sense? > > I did try to add a retry on this request (and it seemed to fix the bug), but I understand that you don't want to introduce this type of custom retry mechanism. Maybe another option could be to wait "check_interval" seconds BEFORE sending the first request. What do you think? @Ohashiro from what I can see from my smartphone (no laptop atm) your explanation does totally make sense. Waiting before doing the actual invocation could make sense, but I would suggest checking the error message and or exception type whenever possible and reinvoke the method after the interval when this error occurs within the code. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
