Owen-CH-Leung commented on PR #74337: URL: https://github.com/apache/airflow/pull/74337#issuecomment-6073135019
> Yes but this is more than just new provider. We need to figure out how we distribute opensearch/elasticsearch but keep code and docs in one place. For reference I propose this: 1. One distribution, `apache-airflow-providers-search`, contains all the code and all the documentation. It supports Elasticsearch and OpenSearch. 2. The `apache-airflow[elasticsearch]` and `apache-airflow[opensearch]` extras also install the new distribution. When the old providers are removed, the extras install only the new distribution. 3. The official Docker image also installs the new distribution. 4. The `elasticsearch` and `opensearch` sections of the Helm chart write the `[search]` configuration. 5. The current Elasticsearch and OpenSearch providers get one more release. This release shows a deprecation warning and links to a migration guide. 6. After a minimum of six months, the PMC votes to move the two providers to the attic. Changes for users: - Helm chart users: no change. - Task logs: set `[logging] remote_base_log_folder = search://`. The provider reads the `[elasticsearch]` and `[opensearch]` sections if `[search]` is not set. - Logs that are already in the cluster: no change. The documents use the same fields. - Dags: change the imports to the new hooks and operators. Dags that use the vendor client from `hook.client` must create that client themselves. With this plan, there is one codebase and one set of documentation. Users can still install the provider with the name of their engine. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
