baruch-shteken commented on PR #34927: URL: https://github.com/apache/airflow/pull/34927#issuecomment-1825922057
I tested my changes locally (I didn't manage to pull my changes from GitHub. However, we use an Airflow docker so I changed the file `airflow/providers/tableau/hooks/tableau.py` in the docker to my new code). **Tl;dr** the code change did solve some of the problem but not all of it. **Explanation**: The set up is 6 data source refresh tasks. Each task triggers the data source refresh and every 3 minutes sends a request to Tableau to check if the refresh finished. When I triggered the DAG, all tasks excepts 1 failed. I remembered that there is a problem of concurrency because if all tasks try to get a new session token at the same time, only 1 will succeed. So I implemented a 2 seconds delay between the tasks in the DAG file. After this change, all the tasks finished successfully. This means that as long as only 1 task requests a session token at any moment, the process will work. @Michae94 Do you have any recommendation for me on how to improve the process? Can I configure somewhere in the Airflow code that only 1 Tableau hook object can be active at any moment? @eladkal I can ask Tableau their recommendation about authenticating from two separated processes that try to refresh two different dashboards at the same time but they need to use the same PAT. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
