nelson-lark opened a new issue #16723: URL: https://github.com/apache/superset/issues/16723
**Is your feature request related to a problem? Please describe.** As of a couple months ago, the Elasticsearch organization has made the official python elasticsearch plugin incompatible with Amazon supported OpenSearch. If you fire up Superset using the current helm chart and attempt to connect to a recently deployed AWS "Elasticsearch" - which is now an Apache 2.0 licensed OpenSearch - you will receive the ambiguous error from Superset ```` superset 2021-09-16 17:34:09,348:WARNING:superset.views.base:[SupersetError(message='(builtins.NoneType) None\n(Background on this error at: http://sqlalche.me/e/13/dbapi)', error_type=<SupersetErrorType.GENERIC_DB_ENGINE_ERROR: 'GENERIC_DB_ENGINE_ERROR'>, level=<ErrorLevel.ERROR: 'error'>, extra={'engine_name': 'ElasticSearch (SQL API)', 'issue_codes': [{'code': 1002, ' message': 'Issue 1002 - The database returned an unexpected error.'}]})] ``` If I run python directly on the Superset pod and connect to AWS "OpenSearch/Elasticsearch" cluster, i get the more detailed error from the elasticsearch python module directly: ```elasticsearch.exceptions.UnsupportedProductError: The client noticed that the server is not a supported distribution of Elasticsearch ``` It would be nice if Superset could support OpenSearch natively. **Describe the solution you'd like** - At the least, highlight in the docs that the elasticsearch plugin needs downgrading to be compatible with AWS Elasticsearch/OpenSearch clusters for the time being. - Find a way to support OpenSearch natively. Quite annoyingly, I think AWS only published a java library for OpenSearch. An official python library that could be used by Sqlalchemy is needed I believe. **Describe alternatives you've considered** I am currently using the following `boostrapScript` which downgrades the elasticsearch python plugin to the last known compatible version (before Elasticsearch became hostile to AWS, rightfully or not). ``` bootstrapScript: | #!/bin/bash rm -rf /var/lib/apt/lists/* && \\ pip uninstall elasticsearch && \\ pip install \\ elasticsearch==7.13.4 \\ elasticsearch-dbapi==0.2.5 \\ elasticsearch-dbapi[opendistro] \\ psycopg2==2.8.5 \\ redis==3.2.1 && if [ ! -f ~/bootstrap ]; then echo "Running Superset with uid {{ .Values.runAsUser }}" > ~/bootstrap; fi ``` **Additional context** Ultimately, the fix may need to be made in SQLALCHEMY since that is the middle man between Superset and the databases, but I thought it was worth requesting here in case anyone else is trying to connect to an Amazon "Elasticsearch"/Opensearch cluster and was having difficulty. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@superset.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: notifications-unsubscr...@superset.apache.org For additional commands, e-mail: notifications-h...@superset.apache.org
