dependabot[bot] opened a new pull request, #45318: URL: https://github.com/apache/superset/pull/45318
Bumps [elasticsearch-dbapi](https://github.com/preset-io/elasticsearch-dbapi) from 0.2.13 to 0.3.0. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/preset-io/elasticsearch-dbapi/releases";>elasticsearch-dbapi's releases</a>.</em></p> <blockquote> <h2>0.3.0</h2> <ul> <li>feat: OpenSearch 3 support (<a href="https://redirect.github.com/preset-io/elasticsearch-dbapi/issues/124";>#124</a>) [Amin Ghadersohi]. The OpenSearch dialect uses <code>_plugins/_sql</code> by default (OpenSearch 3 removed <code>_opendistro/_sql</code>) and falls back once to <code>_opendistro/_sql</code> for Open Distro / Elasticsearch 7.10; single-table columns compile unqualified; aliases are listed as views in v2 mode.</li> <li>fix: SQLAlchemy 2 and DB-API correctness fixes found by live testing (<a href="https://redirect.github.com/preset-io/elasticsearch-dbapi/issues/124";>#124</a>) [Amin Ghadersohi] <ul> <li>follow the SQL cursor across pages on Elasticsearch and OpenSearch, so results larger than <code>fetch_size</code> are no longer cut to the first page; Elasticsearch cursor pagination by Evan Rusackas (from <a href="https://redirect.github.com/preset-io/elasticsearch-dbapi/issues/123";>#123</a>)</li> <li>never render the dummy <code>default</code> schema, including on projected columns</li> <li>reflect byte/short as <code>SmallInteger</code>, unsigned_long as <code>BigInteger</code> and date_nanos as <code>DateTime</code>; unknown result types no longer raise <code>KeyError</code></li> <li>table/view listing and the OpenSearch <code>SELECT 1</code> ping work without cluster privileges; <code>has_table</code> is true for aliases</li> <li><code>server_version_info</code> from the cluster; SQL compilation cache enabled</li> <li>boolean URL arguments (<code>v2</code>, <code>verify_certs</code>, ...) accept <code>true/1/yes/on</code> and <code>false/0/no/off</code></li> </ul> </li> <li>fix: reflected columns report the field's type (<code>DOUBLE</code>, <code>FLOAT</code>, <code>INTEGER</code>, <code>LONG</code>, <code>BOOLEAN</code>) instead of <code>LONG</code>/<code>FLOAT</code> for every numeric and boolean field (<a href="https://redirect.github.com/preset-io/elasticsearch-dbapi/issues/126";>#126</a>) [Amin Ghadersohi]. Reflected types retain their generic SQLAlchemy compilation on other dialects, including when copying a table to another database.</li> </ul> <h4>Behaviour changes / upgrade notes</h4> <ul> <li> <p><strong><code>time_zone</code> on the OpenSearch dialect (<code>odelasticsearch</code>)</strong> is logged as ignored and dropped (UTC, <code>Z</code> and <code>+00:00</code> silently). The SQL plugin always ignored it and returns UTC. <em>Migration:</em> none required; remove <code>time_zone</code> from the URL to silence the warning and convert to local time in the application.</p> </li> <li> <p><strong>Supported SQL temporal values are decoded to <code>datetime.datetime</code> / <code>datetime.date</code> / <code>datetime.time</code></strong> instead of strings. Timezone-aware Elasticsearch values are normalised to UTC to avoid applying one row's DST offset to an entire result column; decoded OpenSearch timestamps remain naive UTC. This is not a guarantee that every date-mapped column returns an object: conversion depends on the SQL engine's result type and value, and nanosecond or unrecognised representations remain strings. In particular, legacy OpenSearch results can label a full timestamp as <code>date</code>; previously even <code>v2=true</code> queries with LIMIT could take that legacy path. LIMIT queries on a v2 connection now stay on v2, preserving its functions and timestamp result types. <em>Migration:</em> accept both temporal objects and preserved strings; use objects directly (or <code>str()</code>/<code>isoformat()</code> them), and do not discard precis ion when parsing preserved strings. Convert UTC-aware values to the desired display timezone explicitly.</p> </li> <li> <p><strong><code>double</code>, <code>float</code>, <code>half_float</code> and <code>scaled_float</code> columns are reflected as <code>Float</code></strong> (Python <code>float</code>) instead of <code>Numeric</code> (<code>Decimal</code> rounded to 10 places). <em>Migration:</em> code expecting <code>Decimal</code> should convert explicitly.</p> </li> <li> <p><strong>Client exceptions are raised as DB-API exceptions</strong> from <code>es.exceptions</code>: connection and TLS errors, authentication/authorization errors as <code>OperationalError</code>; <code>RequestError</code> and <code>NotFoundError</code> as <code>ProgrammingError</code>; every other <code>TransportError</code> as <code>DatabaseError</code>. The original <code>elasticsearch.*</code> / <code>opensearchpy.*</code> exception is chained as <code>__cause__</code>. <em>Migration:</em> catch <code>es.exceptions.*</code> (or the DB-API classes) instead of <code>elasticsearch.exceptions.*</code> / <code>opensearchpy.exceptions.*</code>, or inspect <code>__cause__</code>.</p> </li> <li> <p><strong>The default OpenSearch <code>sql_path</code> is <code>_plugins/_sql</code></strong>, with one fallback to <code>_opendistro/_sql</code> when the cluster has no <code>_plugins/_sql</code> endpoint (<code>no handler found</code>, or Elasticsearch 7.10's <code>invalid_index_name_exception</code> for <code>_plugins</code>, which is <code>index_not_found_exception</code> with <code>action.auto_create_index=false</code>). A proxy or IAM policy that only allows <code>_opendistro/_sql</code> answers 403, which is not a missing endpoint, so there is no fallback. <em>Migration:</em> add <code>sql_path=_opendistro/_sql</code> to the URL in that case (an explicit <code>sql_path</code> is never replaced).</p> </li> <li> <p><strong>An unpaged OpenSearch result that may have been cut at <code>plugins.query.size_limit</code> raises <code>DataError</code></strong> instead of returning partial rows. <em>Migration:</em> add a <code>LIMIT</code> of at most the size limit, or narrow the query.</p> </li> <li> <p><strong>Full result sets are held in memory.</strong> A DB-API <code>SELECT</code> without <code>LIMIT</code> used to return at most <code>fetch_size</code> (10 000) rows; every page is now fetched. <em>Migration:</em> add a <code>LIMIT</code> to queries over large indices.</p> </li> <li> <p><strong>Single-table columns compile unqualified on OpenSearch</strong> (<code>SELECT a FROM t ORDER BY a</code>), which OpenSearch 3 requires. A column keeps its table qualifier when a label of another expression in the same select has its name. <em>Migration:</em> none expected; raw SQL is not affected.</p> </li> <li> <p><strong>In v2 mode, GROUP BY, DISTINCT, aggregates, joins and other statements the SQL plugin cannot page are sent without <code>fetch_size</code></strong>, avoiding the legacy engine's default top-200 result. This does <strong>not</strong> mean the v2 engine returns every bucket: unpaged aggregations stop at a backend bucket ceiling and cannot be cursor-paged. That ceiling is 1000 buckets on Open Distro and OpenSearch 1.x; OpenSearch 2.x/3.x also stop at <code>plugins.query.size_limit</code> when it is below 1000, which is the default on 2.11–2.15 (200), on either <code>sql_path</code>. A result at the applicable ceiling without an explicit LIMIT of at most that ceiling now raises <code>DataError</code> instead of silently returning a partial answer, so exactly 200 groups raise on OpenSearch 2.11–2.15 at the default size limit (or when the server version cannot be read), and are accepted where 200 is not a ceiling (Open Distro, OpenSearch 1.x, and 2.19/3.x at their default size limit). Plain SELECTs without LIMIT are still paged on OpenSearch (<code>_plugins/_sql</code>); on Open Distro (<code>_opendistro/_sql</code>), whose v2 engine cannot page, v2 statements are sent unpaged as in 0.2.13, and if OpenSearch refuses a paged request for lack of privileges it is retried once unpaged. <code>v2=false</code> from a URL now means false (it used to switch v2 mode on) and keeps the legacy engine; an aggregation that reaches the legacy engine's 200-bucket cap is asked of the v2 engine, rather than assuming those 200 groups are complete. <em>Migration:</em> narrow high-cardinality aggregations or use an explicit bounded LIMIT; do not assume an unpaged aggregate is complete merely because v2 is enabled.</p> </li> <li> <p><strong>Statements over subqueries return correct results</strong> (e.g. Superset virtual datasets). A subquery without a LIMIT used to stop at the size limit (200 rows on Open Distro) and silently drop rows from the outer result; the legacy engine (v2 off) ignored the outer WHERE; and Open Distro dropped a row when an outer WHERE and LIMIT met without ORDER BY. Subqueries now get an explicit LIMIT of one search window, run on the v2 engine, and on Open Distro the outer LIMIT is applied to the rows.</p> </li> <li> <p><strong>A Superset virtual dataset (or any statement over a subquery) whose subquery returns more than 10000 rows raises <code>DataError</code></strong> on OpenSearch and Open Distro, e.g. a virtual dataset of <code>SELECT * FROM idx</code> over an index with 10k+ documents. 0.2.13 answered these with silently wrong results (a filtered <code>COUNT</code> of 0 where the true count was 9). The SQL plugin cannot page a subquery, so no complete answer exists. On Open Distro, which refuses to count past the window, a subquery of exactly 10000 rows raises too. <em>Migration:</em> narrow the subquery (the virtual dataset's SQL) with a WHERE, an aggregation or a LIMIT of at most 10000, or use a physical dataset.</p> </li> <li> <p><strong>Reflected column types render differently.</strong> <code>double</code>, <code>float</code>, <code>integer</code>, <code>long</code> and <code>boolean</code> fields report <code>DOUBLE</code>, <code>FLOAT</code>, <code>INTEGER</code>, <code>LONG</code> and <code>BOOLEAN</code>; 0.2.13 reported <code>LONG</code> for <code>double</code>, <code>integer</code>, <code>long</code> and <code>boolean</code>. <code>byte</code> and <code>short</code> report <code>INTEGER</code>, <code>half_float</code> <code>FLOAT</code>, <code>scaled_float</code> <code>DOUBLE</code> and <code>unsigned_long</code> <code>LONG</code> (0.2.13 reported <code>STRING</code> for all but <code>half_float</code>): the SQL plugins cannot <code>CAST</code> to their own names, and Superset's default <code>column_type_mappings</code> do not know them. Superset recognises every rendered name, so these numeric fields are now numeric and boolean fields are boolean rather than numeric. <em>Migration:</em> none fo r Superset; code comparing the rendered type strings should expect the new names.</p> </li> <li> <p><strong>An explicit <code>LIMIT</code> on a v2 plain SELECT bypasses <code>plugins.query.size_limit</code></strong>, so only the 10000-row search window is checked (an exactly-200-row answer to <code>LIMIT 1001</code> is complete, not truncated). A plain SELECT whose LIMIT is above 10000 is replayed without the LIMIT through SQL cursors and cut at its LIMIT locally; if cursors are unavailable (including a limited cursor whose next page is empty) it raises <code>DataError</code>. <em>Migration:</em> none; results above 10000 rows need SQL cursors on the cluster.</p> </li> <li> <p><strong>Grouped top-N on the v2 engine is validated.</strong> The v2 engine sorts <code>GROUP BY ... ORDER BY COUNT(*) DESC LIMIT 5</code> only after its capped composite aggregation, so it can return the wrong five groups. The driver probes the unsorted group listing (one additional SQL request) and raises <code>DataError</code> when that listing reaches its bucket ceiling. Ordering only by grouped fields (<code>GROUP BY k ORDER BY k DESC LIMIT 3</code>) is sorted inside the aggregation, so it is not probed. <em>Migration:</em> narrow the grouping, or use <code>v2=false</code> for legacy top-N.</p> </li> <li> <p><strong>OFFSET is an exception to the grouped-field ordering shortcut:</strong> <code>SELECT k, COUNT(*) FROM t GROUP BY k ORDER BY k LIMIT 3 OFFSET 999</code> is probed and raises <code>DataError</code> at the bucket ceiling instead of silently returning one row from 5000 groups.</p> </li> <li> <p><strong>OpenSearch 1.x v2 plain SELECTs stay on the v2 engine</strong> (paging them would hand them to the legacy engine), preserving floating values and timestamp objects. The server version is discovered once per connection, best effort; when it cannot be read, unpaged v2 semantics are preferred to legacy paging.</p> </li> <li> <p><strong>The legacy engine no longer confuses a table-qualified column with a select-list alias of the same name</strong> (<code>ORDER BY grp.k</code> next to <code>v AS k</code> sorted by <code>v</code>): such aliases are renamed in the statement sent and restored in the result.</p> </li> <li> <p><strong>Open Distro returns every row of a plain SELECT.</strong> Open Distro 1.13 stops a SELECT without its own LIMIT at <code>opendistro.query.size_limit</code> (200) and has SQL cursors off by default; 0.2.13 returned those 200 rows silently. A capped answer is now asked for again with an explicit LIMIT of the 10000-row search window, and beyond that paged with the SQL cursor. A plain SELECT whose own LIMIT is past that window (e.g. SQL Lab's <code>LIMIT 100001</code>), which Open Distro refuses, is answered the same way and cut at its LIMIT. <em>Migration:</em> results above 10000 rows need <code>opendistro.sql.cursor.enabled: true</code> on the cluster, otherwise the query raises <code>DataError</code> saying so.</p> </li> <li> <p>OpenSearch 2.11/2.15 v2 aggregations can stop at <code>plugins.query.size_limit</code> (200 by default), on either SQL endpoint. Ambiguous full buckets now raise <code>DataError</code>, rather than silently returning incomplete groups. Exactly 200 remains accepted on Open Distro and OpenSearch 1.x, and on newer servers at their default size limit. A custom size limit below 1000 is also checked on OpenSearch 2.x/3.x.</p> </li> <li> <p>An explicit v2 SELECT <code>LIMIT</code> bypasses the query size limit; only the search window is checked. Plain SELECTs with a LIMIT above 10000 are replayed without LIMIT using SQL cursors, then capped locally; unavailable cursors raise <code>DataError</code>, including a limited cursor whose next page is empty.</p> </li> <li> <p>The v2 engine sorts grouped top-N results <strong>after</strong> its capped composite aggregation, so <code>GROUP BY ... ORDER BY COUNT(*) DESC LIMIT 5</code> can select the wrong groups even though only five rows are returned. The driver now probes the underlying group listing and raises <code>DataError</code> at its bucket ceiling. Narrow the grouping or use <code>v2=false</code> for legacy top-N. Safe small listings incur one additional SQL request. Ordering only by grouped fields (e.g. <code>GROUP BY k ORDER BY k DESC LIMIT 3</code>) is sorted inside the aggregation and is not probed.</p> </li> </ul> <!-- raw HTML omitted --> </blockquote> <p>... (truncated)</p> </details> <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/preset-io/elasticsearch-dbapi/blob/master/CHANGELOG.md";>elasticsearch-dbapi's changelog</a>.</em></p> <blockquote> <h3>0.3.0</h3> <ul> <li>feat: OpenSearch 3 support (<a href="https://redirect.github.com/preset-io/elasticsearch-dbapi/issues/124";>#124</a>) [Amin Ghadersohi]. The OpenSearch dialect uses <code>_plugins/_sql</code> by default (OpenSearch 3 removed <code>_opendistro/_sql</code>) and falls back once to <code>_opendistro/_sql</code> for Open Distro / Elasticsearch 7.10; single-table columns compile unqualified; aliases are listed as views in v2 mode.</li> <li>fix: SQLAlchemy 2 and DB-API correctness fixes found by live testing (<a href="https://redirect.github.com/preset-io/elasticsearch-dbapi/issues/124";>#124</a>) [Amin Ghadersohi] <ul> <li>follow the SQL cursor across pages on Elasticsearch and OpenSearch, so results larger than <code>fetch_size</code> are no longer cut to the first page; Elasticsearch cursor pagination by Evan Rusackas (from <a href="https://redirect.github.com/preset-io/elasticsearch-dbapi/issues/123";>#123</a>)</li> <li>never render the dummy <code>default</code> schema, including on projected columns</li> <li>reflect byte/short as <code>SmallInteger</code>, unsigned_long as <code>BigInteger</code> and date_nanos as <code>DateTime</code>; unknown result types no longer raise <code>KeyError</code></li> <li>table/view listing and the OpenSearch <code>SELECT 1</code> ping work without cluster privileges; <code>has_table</code> is true for aliases</li> <li><code>server_version_info</code> from the cluster; SQL compilation cache enabled</li> <li>boolean URL arguments (<code>v2</code>, <code>verify_certs</code>, ...) accept <code>true/1/yes/on</code> and <code>false/0/no/off</code></li> </ul> </li> <li>fix: reflected columns report the field's type (<code>DOUBLE</code>, <code>FLOAT</code>, <code>INTEGER</code>, <code>LONG</code>, <code>BOOLEAN</code>) instead of <code>LONG</code>/<code>FLOAT</code> for every numeric and boolean field (<a href="https://redirect.github.com/preset-io/elasticsearch-dbapi/issues/126";>#126</a>) [Amin Ghadersohi]. Reflected types retain their generic SQLAlchemy compilation on other dialects, including when copying a table to another database.</li> </ul> <h4>Behaviour changes / upgrade notes</h4> <ul> <li> <p><strong><code>time_zone</code> on the OpenSearch dialect (<code>odelasticsearch</code>)</strong> is logged as ignored and dropped (UTC, <code>Z</code> and <code>+00:00</code> silently). The SQL plugin always ignored it and returns UTC. <em>Migration:</em> none required; remove <code>time_zone</code> from the URL to silence the warning and convert to local time in the application.</p> </li> <li> <p><strong>Supported SQL temporal values are decoded to <code>datetime.datetime</code> / <code>datetime.date</code> / <code>datetime.time</code></strong> instead of strings. Timezone-aware Elasticsearch values are normalised to UTC to avoid applying one row's DST offset to an entire result column; decoded OpenSearch timestamps remain naive UTC. This is not a guarantee that every date-mapped column returns an object: conversion depends on the SQL engine's result type and value, and nanosecond or unrecognised representations remain strings. In particular, legacy OpenSearch results can label a full timestamp as <code>date</code>; previously even <code>v2=true</code> queries with LIMIT could take that legacy path. LIMIT queries on a v2 connection now stay on v2, preserving its functions and timestamp result types. <em>Migration:</em> accept both temporal objects and preserved strings; use objects directly (or <code>str()</code>/<code>isoformat()</code> them), and do not discard precis ion when parsing preserved strings. Convert UTC-aware values to the desired display timezone explicitly.</p> </li> <li> <p><strong><code>double</code>, <code>float</code>, <code>half_float</code> and <code>scaled_float</code> columns are reflected as <code>Float</code></strong> (Python <code>float</code>) instead of <code>Numeric</code> (<code>Decimal</code> rounded to 10 places). <em>Migration:</em> code expecting <code>Decimal</code> should convert explicitly.</p> </li> <li> <p><strong>Client exceptions are raised as DB-API exceptions</strong> from <code>es.exceptions</code>: connection and TLS errors, authentication/authorization errors as <code>OperationalError</code>; <code>RequestError</code> and <code>NotFoundError</code> as <code>ProgrammingError</code>; every other <code>TransportError</code> as <code>DatabaseError</code>. The original <code>elasticsearch.*</code> / <code>opensearchpy.*</code> exception is chained as <code>__cause__</code>. <em>Migration:</em> catch <code>es.exceptions.*</code> (or the DB-API classes) instead of <code>elasticsearch.exceptions.*</code> / <code>opensearchpy.exceptions.*</code>, or inspect <code>__cause__</code>.</p> </li> <li> <p><strong>The default OpenSearch <code>sql_path</code> is <code>_plugins/_sql</code></strong>, with one fallback to <code>_opendistro/_sql</code> when the cluster has no <code>_plugins/_sql</code> endpoint (<code>no handler found</code>, or Elasticsearch 7.10's <code>invalid_index_name_exception</code> for <code>_plugins</code>, which is <code>index_not_found_exception</code> with <code>action.auto_create_index=false</code>). A proxy or IAM policy that only allows <code>_opendistro/_sql</code> answers 403, which is not a missing endpoint, so there is no fallback. <em>Migration:</em> add <code>sql_path=_opendistro/_sql</code> to the URL in that case (an explicit <code>sql_path</code> is never replaced).</p> </li> <li> <p><strong>An unpaged OpenSearch result that may have been cut at <code>plugins.query.size_limit</code> raises <code>DataError</code></strong> instead of returning partial rows. <em>Migration:</em> add a <code>LIMIT</code> of at most the size limit, or narrow the query.</p> </li> <li> <p><strong>Full result sets are held in memory.</strong> A DB-API <code>SELECT</code> without <code>LIMIT</code> used to return at most <code>fetch_size</code> (10 000) rows; every page is now fetched. <em>Migration:</em> add a <code>LIMIT</code> to queries over large indices.</p> </li> <li> <p><strong>Single-table columns compile unqualified on OpenSearch</strong> (<code>SELECT a FROM t ORDER BY a</code>), which OpenSearch 3 requires. A column keeps its table qualifier when a label of another expression in the same select has its name. <em>Migration:</em> none expected; raw SQL is not affected.</p> </li> <li> <p><strong>In v2 mode, GROUP BY, DISTINCT, aggregates, joins and other statements the SQL plugin cannot page are sent without <code>fetch_size</code></strong>, avoiding the legacy engine's default top-200 result. This does <strong>not</strong> mean the v2 engine returns every bucket: unpaged aggregations stop at a backend bucket ceiling and cannot be cursor-paged. That ceiling is 1000 buckets on Open Distro and OpenSearch 1.x; OpenSearch 2.x/3.x also stop at <code>plugins.query.size_limit</code> when it is below 1000, which is the default on 2.11–2.15 (200), on either <code>sql_path</code>. A result at the applicable ceiling without an explicit LIMIT of at most that ceiling now raises <code>DataError</code> instead of silently returning a partial answer, so exactly 200 groups raise on OpenSearch 2.11–2.15 at the default size limit (or when the server version cannot be read), and are accepted where 200 is not a ceiling (Open Distro, OpenSearch 1.x, and 2.19/3.x at their default size limit). Plain SELECTs without LIMIT are still paged on OpenSearch (<code>_plugins/_sql</code>); on Open Distro (<code>_opendistro/_sql</code>), whose v2 engine cannot page, v2 statements are sent unpaged as in 0.2.13, and if OpenSearch refuses a paged request for lack of privileges it is retried once unpaged. <code>v2=false</code> from a URL now means false (it used to switch v2 mode on) and keeps the legacy engine; an aggregation that reaches the legacy engine's 200-bucket cap is asked of the v2 engine, rather than assuming those 200 groups are complete. <em>Migration:</em> narrow high-cardinality aggregations or use an explicit bounded LIMIT; do not assume an unpaged aggregate is complete merely because v2 is enabled.</p> </li> <li> <p><strong>Statements over subqueries return correct results</strong> (e.g. Superset virtual datasets). A subquery without a LIMIT used to stop at the size limit (200 rows on Open Distro) and silently drop rows from the outer result; the legacy engine (v2 off) ignored the outer WHERE; and Open Distro dropped a row when an outer WHERE and LIMIT met without ORDER BY. Subqueries now get an explicit LIMIT of one search window, run on the v2 engine, and on Open Distro the outer LIMIT is applied to the rows.</p> </li> <li> <p><strong>A Superset virtual dataset (or any statement over a subquery) whose subquery returns more than 10000 rows raises <code>DataError</code></strong> on OpenSearch and Open Distro, e.g. a virtual dataset of <code>SELECT * FROM idx</code> over an index with 10k+ documents. 0.2.13 answered these with silently wrong results (a filtered <code>COUNT</code> of 0 where the true count was 9). The SQL plugin cannot page a subquery, so no complete answer exists. On Open Distro, which refuses to count past the window, a subquery of exactly 10000 rows raises too. <em>Migration:</em> narrow the subquery (the virtual dataset's SQL) with a WHERE, an aggregation or a LIMIT of at most 10000, or use a physical dataset.</p> </li> <li> <p><strong>Reflected column types render differently.</strong> <code>double</code>, <code>float</code>, <code>integer</code>, <code>long</code> and <code>boolean</code> fields report <code>DOUBLE</code>, <code>FLOAT</code>, <code>INTEGER</code>, <code>LONG</code> and <code>BOOLEAN</code>; 0.2.13 reported <code>LONG</code> for <code>double</code>, <code>integer</code>, <code>long</code> and <code>boolean</code>. <code>byte</code> and <code>short</code> report <code>INTEGER</code>, <code>half_float</code> <code>FLOAT</code>, <code>scaled_float</code> <code>DOUBLE</code> and <code>unsigned_long</code> <code>LONG</code> (0.2.13 reported <code>STRING</code> for all but <code>half_float</code>): the SQL plugins cannot <code>CAST</code> to their own names, and Superset's default <code>column_type_mappings</code> do not know them. Superset recognises every rendered name, so these numeric fields are now numeric and boolean fields are boolean rather than numeric. <em>Migration:</em> none fo r Superset; code comparing the rendered type strings should expect the new names.</p> </li> <li> <p><strong>An explicit <code>LIMIT</code> on a v2 plain SELECT bypasses <code>plugins.query.size_limit</code></strong>, so only the 10000-row search window is checked (an exactly-200-row answer to <code>LIMIT 1001</code> is complete, not truncated). A plain SELECT whose LIMIT is above 10000 is replayed without the LIMIT through SQL cursors and cut at its LIMIT locally; if cursors are unavailable (including a limited cursor whose next page is empty) it raises <code>DataError</code>. <em>Migration:</em> none; results above 10000 rows need SQL cursors on the cluster.</p> </li> <li> <p><strong>Grouped top-N on the v2 engine is validated.</strong> The v2 engine sorts <code>GROUP BY ... ORDER BY COUNT(*) DESC LIMIT 5</code> only after its capped composite aggregation, so it can return the wrong five groups. The driver probes the unsorted group listing (one additional SQL request) and raises <code>DataError</code> when that listing reaches its bucket ceiling. Ordering only by grouped fields (<code>GROUP BY k ORDER BY k DESC LIMIT 3</code>) is sorted inside the aggregation, so it is not probed. <em>Migration:</em> narrow the grouping, or use <code>v2=false</code> for legacy top-N.</p> </li> <li> <p><strong>OFFSET is an exception to the grouped-field ordering shortcut:</strong> <code>SELECT k, COUNT(*) FROM t GROUP BY k ORDER BY k LIMIT 3 OFFSET 999</code> is probed and raises <code>DataError</code> at the bucket ceiling instead of silently returning one row from 5000 groups.</p> </li> <li> <p><strong>OpenSearch 1.x v2 plain SELECTs stay on the v2 engine</strong> (paging them would hand them to the legacy engine), preserving floating values and timestamp objects. The server version is discovered once per connection, best effort; when it cannot be read, unpaged v2 semantics are preferred to legacy paging.</p> </li> <li> <p><strong>The legacy engine no longer confuses a table-qualified column with a select-list alias of the same name</strong> (<code>ORDER BY grp.k</code> next to <code>v AS k</code> sorted by <code>v</code>): such aliases are renamed in the statement sent and restored in the result.</p> </li> <li> <p><strong>Open Distro returns every row of a plain SELECT.</strong> Open Distro 1.13 stops a SELECT without its own LIMIT at <code>opendistro.query.size_limit</code> (200) and has SQL cursors off by default; 0.2.13 returned those 200 rows silently. A capped answer is now asked for again with an explicit LIMIT of the 10000-row search window, and beyond that paged with the SQL cursor. A plain SELECT whose own LIMIT is past that window (e.g. SQL Lab's <code>LIMIT 100001</code>), which Open Distro refuses, is answered the same way and cut at its LIMIT. <em>Migration:</em> results above 10000 rows need <code>opendistro.sql.cursor.enabled: true</code> on the cluster, otherwise the query raises <code>DataError</code> saying so.</p> </li> <li> <p>OpenSearch 2.11/2.15 v2 aggregations can stop at <code>plugins.query.size_limit</code> (200 by default), on either SQL endpoint. Ambiguous full buckets now raise <code>DataError</code>, rather than silently returning incomplete groups. Exactly 200 remains accepted on Open Distro and OpenSearch 1.x, and on newer servers at their default size limit. A custom size limit below 1000 is also checked on OpenSearch 2.x/3.x.</p> </li> <li> <p>An explicit v2 SELECT <code>LIMIT</code> bypasses the query size limit; only the search window is checked. Plain SELECTs with a LIMIT above 10000 are replayed without LIMIT using SQL cursors, then capped locally; unavailable cursors raise <code>DataError</code>, including a limited cursor whose next page is empty.</p> </li> <li> <p>The v2 engine sorts grouped top-N results <strong>after</strong> its capped composite aggregation, so <code>GROUP BY ... ORDER BY COUNT(*) DESC LIMIT 5</code> can select the wrong groups even though only five rows are returned. The driver now probes the underlying group listing and raises <code>DataError</code> at its bucket ceiling. Narrow the grouping or use <code>v2=false</code> for legacy top-N. Safe small listings incur one additional SQL request. Ordering only by grouped fields (e.g. <code>GROUP BY k ORDER BY k DESC LIMIT 3</code>) is sorted inside the aggregation and is</p> </li> </ul> <!-- raw HTML omitted --> </blockquote> <p>... (truncated)</p> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/preset-io/elasticsearch-dbapi/commit/83fd3c392ebb5c1fbb2a569b2251c6ffb6d4d053";><code>83fd3c3</code></a> release: 0.3.0 (<a href="https://redirect.github.com/preset-io/elasticsearch-dbapi/issues/125";>#125</a>)</li> <li><a href="https://github.com/preset-io/elasticsearch-dbapi/commit/2303279b514c0855f38104ce2e7bf03b4f37b623";><code>2303279</code></a> fix: SQLAlchemy 2 and DB-API correctness fixes found by live testing (<a href="https://redirect.github.com/preset-io/elasticsearch-dbapi/issues/124";>#124</a>)</li> <li>See full diff in <a href="https://github.com/preset-io/elasticsearch-dbapi/compare/0.2.13...0.3.0";>compare view</a></li> </ul> </details> <br /> [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) </details> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
