sabiroid opened a new issue #16209: URL: https://github.com/apache/superset/issues/16209
## [SIP-71] Proposal for API to pass metadata over to Celery workers ### Motivation There are several types of metadata required to run analytical queries. Among them are user-impersonating authentication tokens and granular source information (e.g. dashboard or chart IDs). Unfortunately, this "global" information is not available on the Celery workers since they run remotely in a different Flask context. This proposal suggests to add Superset config API to pass required "global" data over so user code on Celery worker is able to utilize it. ### Proposed Change Add `CELERY_EXTRACT_METADATA` config function allowing to save all necessary data from `flask.g`, `flask.request` and `flask.context` on the request side and `CELERY_INIT_METADATA` config function restoring the data on the Celery worker side. ### New or Changed Public Interfaces Superset config will just have new `CELERY_EXTRACT_METADATA` and `CELERY_INIT_METADATA` config functions. ### New dependencies None ### Migration Plan and Compatibility The API is completely backward compatible ### Rejected Alternatives * I thought of just having `CELERY_PASSOVER_METADATA` config tuple with names of fields of `flask.g`, but it would not allow to populate `flask.g` with custom fields (mb derived from `flask.request` or `flask.session`), and adding a new functionality to populate `flask.g` in the first place would become as complex as the proposal while being way less intuitive. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
