aminghadersohi opened a new pull request, #44703: URL: https://github.com/apache/superset/pull/44703
### SUMMARY `OpenDistroEngineSpec.fetch_data_with_cursor` (used for drill-to-detail sample pages after the first, since the engine has `supports_offset = False`) reuses the Elasticsearch request and response handling in `_fetch_page_via_cursor`. Against OpenSearch that fails in two ways: 1. It sends `Content-Type: application/json` explicitly. The Elasticsearch client needs that, but opensearch-py already sets the header, so OpenSearch receives two and rejects the request with HTTP 400 `content_type_header_exception: only one Content-Type header should be provided`. 2. The OpenSearch SQL plugin answers in its JDBC format (`schema` / `datarows`, and `datarows` only on follow-up pages), not Elasticsearch's `columns` / `rows`. So even a successful request returned no columns and no rows. `_fetch_page_via_cursor` now takes optional `headers`, `rows_key` and `columns_key`. `ElasticSearchEngineSpec` passes the JSON header and keeps the Elasticsearch keys, so its behaviour is unchanged. `OpenDistroEngineSpec` sends no extra header and reads `schema` / `datarows`. A column's `alias` is used as its name when present, which is how the SQL plugin reports `COUNT(*) AS c`. ### TESTING INSTRUCTIONS - `pytest tests/unit_tests/db_engine_specs/test_elasticsearch.py`: 27 passed. The two new tests (`test_opendistro_fetch_data_with_cursor_reads_jdbc_format_pages`, `test_opendistro_fetch_data_with_cursor_sends_no_extra_content_type`) and the updated `test_opendistro_fetch_data_with_cursor_uses_opendistro_endpoints` fail without the change (3 failed, 24 passed) and pass with it. - Reproduced live before the change against OpenSearch 2.19.1 and 3.2.0 containers via elasticsearch-dbapi: page 2 of a 25-row index raised the HTTP 400 above. With the same approach (no extra header, JDBC keys) it returns rows 11–20. ### ADDITIONAL INFORMATION - [ ] Has associated issue: - [ ] Required feature flags: - [ ] Changes UI - [ ] Includes DB Migration - [ ] Introduces new feature or API - [ ] Removes existing feature or API -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
