dependabot[bot] opened a new pull request, #45916: URL: https://github.com/apache/superset/pull/45916
Bumps [duckdb](https://github.com/duckdb/duckdb-python) from 1.5.6 to 2.0.0. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/duckdb/duckdb-python/releases";>duckdb's releases</a>.</em></p> <blockquote> <h2>DuckDB v2.0.0</h2> <p>duckdb-python 2.0.0 is the Python package for DuckDB 2.0. It supports Python >= 3.11 and drops Python 3.10. From Python 3.12 on, one wheel per platform serves every Python version. The binding layer was rewritten on <a href="https://github.com/wjakob/nanobind";>nanobind</a>, query results stream instead of being collected up front, the relational API was rebuilt on SQL text, and the way DataFrames, Arrow data and polars frames are read was replaced. Many long-standing bugs are fixed, and moving data in and out of Python is much faster. The engine is <a href="https://github.com/duckdb/duckdb/releases/tag/v2.0.0";>DuckDB 2.0.0</a>; see its release notes for what changed in SQL and storage.</p> <h2>Highlights</h2> <ul> <li><strong>Moving data is much faster.</strong> Numeric exports to numpy, pandas, Arrow and polars take a quarter to two fifths of the 1.5 time; in-memory Arrow, polars and pandas 3 inputs are scanned on all threads, 8x to 20x faster; Arrow UDFs run about 20x faster; draining an Arrow reader on 16 threads takes an eighth of the time. (<a href="https://redirect.github.com/duckdb/duckdb-python/issues/693";>#693</a>, <a href="https://redirect.github.com/duckdb/duckdb-python/issues/697";>#697</a>, <a href="https://redirect.github.com/duckdb/duckdb-python/issues/700";>#700</a>)</li> <li><strong>Results stream.</strong> <code>con.execute()</code> starts the query and you read rows as you fetch them; <code>con.complete()</code> and <code>con.abort()</code> finish or cancel an open result, and an open result or Arrow reader owns the connection until it is done.</li> <li><strong>The relational API is rebuilt on SQL text.</strong> A relation is a lazy query, bound at every step so its schema is known; <code>rel.sql_query()</code> shows exactly what will run. Many new methods and expression helpers (<code>col</code>, <code>lit</code>, <code>when</code>, window functions). One semantic change: a plain string in an expression is a value, not a column name. (<a href="https://redirect.github.com/duckdb/duckdb-python/issues/697";>#697</a>)</li> <li><strong>A new Appender for fast bulk inserts</strong>: <code>with con.appender("t") as a: a.append_row(...)</code>, under a microsecond per row. (<a href="https://redirect.github.com/duckdb/duckdb-python/issues/661";>#661</a>)</li> <li><strong>One stable ABI wheel per platform</strong> for Python 3.12 and later, so a new Python release works on day one. Python 3.15 is supported. (<a href="https://redirect.github.com/duckdb/duckdb-python/issues/672";>#672</a>, <a href="https://redirect.github.com/duckdb/duckdb-python/issues/673";>#673</a>, <a href="https://redirect.github.com/duckdb/duckdb-python/issues/684";>#684</a>)</li> <li><strong>Arrow export on <code>con.execute()</code> results changed</strong>: <code>fetch_arrow_table</code>, <code>fetch_record_batch</code> and <code>arrow(n)</code> are gone; use <code>con.sql(query).to_arrow_table()</code> or <code>.to_arrow_reader()</code>. (<a href="https://redirect.github.com/duckdb/duckdb-python/issues/702";>#702</a>)</li> <li><strong>The experimental Spark API is brought up to date</strong>: it is now compatible with <a href="https://spark.apache.org/docs/latest/api/python/";>PySpark</a> 4.2.0 and 3.5.9 (<a href="https://redirect.github.com/duckdb/duckdb-python/issues/686";>#686</a>). We would love your input on whether to continue support or to drop it in future releases.</li> </ul> <h2>Upgrading from 1.x: what to check</h2> <ol> <li>Make sure you run Python 3.11 or later.</li> <li>If you call <code>fetch_arrow_table</code>, <code>fetch_record_batch</code>, <code>arrow(n)</code> or <code>pl()</code> on the result of <code>con.execute()</code>, switch to <code>con.sql(query).to_arrow_table()</code>, <code>to_arrow_reader()</code> or <code>pl()</code>. Libraries that call the removed names on an execute result need an update. (<a href="https://redirect.github.com/duckdb/duckdb-python/issues/702";>#702</a>)</li> <li>If you call <code>fetchone()</code>, <code>fetchmany()</code>, <code>fetch_df_chunk()</code> or <code>execute()</code> on a relation, use <code>rel.limit(n).fetchall()</code>, <code>rel.to_arrow_reader()</code>, <code>rel.materialize()</code>, or run the query through <code>con.execute()</code>. (<a href="https://redirect.github.com/duckdb/duckdb-python/issues/697";>#697</a>)</li> <li>If you compare a relation column with a plain string, <code>col("s") == "x"</code> now compares with the text <code>x</code>. Wrap column names in <code>col()</code>. (<a href="https://redirect.github.com/duckdb/duckdb-python/issues/697";>#697</a>)</li> <li>If you rely on <code>import duckdb</code> having opened the default database, note that it happens on the first <code>duckdb.sql()</code> or <code>duckdb.connect()</code> call instead. (<a href="https://redirect.github.com/duckdb/duckdb-python/issues/680";>#680</a>)</li> <li>If you pass <code>Decimal</code> values with a positive exponent, such as <code>Decimal("1E+2")</code>, they now arrive as the number they denote, 100, where 1.5 produced 1.00. (<a href="https://redirect.github.com/duckdb/duckdb-python/issues/654";>#654</a>)</li> <li>If you write files with <code>overwrite=False</code>, an existing file is now refused instead of replaced. (<a href="https://redirect.github.com/duckdb/duckdb-python/issues/668";>#668</a>)</li> <li>If a test or script depends on <code>str(rel)</code> being free, note that it runs the query each time now.</li> </ol> <h2>Breaking changes</h2> <h3>Arrow export on DB-API results (<a href="https://redirect.github.com/duckdb/duckdb-python/issues/702";>#702</a>)</h3> <p>Results of <code>con.execute()</code> are now read row by row and are not kept in memory, so they cannot hand their data to Arrow for free. <code>fetch_arrow_table</code>, <code>fetch_record_batch</code>, <code>arrow(rows_per_batch)</code> and <code>pl()</code> on a DB-API result are removed; they had been deprecated. <code>to_arrow_table()</code> and <code>to_arrow_reader()</code> remain, also as <code>duckdb.to_arrow_table()</code> and <code>duckdb.to_arrow_reader()</code>, but they materialize the rows twice, once as the result and once as Arrow, and emit a <code>UserWarning</code> pointing at the relational path, <code>con.sql(query).to_arrow_table()</code>, which we recommend instead. The reader no longer streams: its first batch arrives after the whole query ran. After a partial <code>fetchone</code> or <code>fetchmany</code> the Arrow call returns the remaining rows instead of raising. With <code>preserve_insertion_order</code> off, the row order can differ between calls . <code>duckdb.arrow(n)</code> with an integer is gone; <code>duckdb.arrow(arrow_object)</code> still builds a relation.</p> <pre lang="python"><code># 1.5 table = con.execute("SELECT * FROM t").fetch_arrow_table() reader = con.execute("SELECT * FROM t").fetch_record_batch(10_000) <h1>2.0</h1> <p>table = con.sql("SELECT * FROM t").to_arrow_table()<br /> reader = con.sql("SELECT * FROM t").to_arrow_reader(10_000)<br /> </code></pre></p> <h3>Relations are SQL text (<a href="https://redirect.github.com/duckdb/duckdb-python/issues/697";>#697</a>)</h3> <p>The relational API is a query building API, not an execution API. A relation is a piece of SQL text with a schema bound once when it is built, so every step knows its columns and types. It runs nothing until you ask for data, and it runs again on every read; <code>rel.sql_query()</code> returns exactly the text that runs, as one SELECT per group of steps and a common table expression where the order of clauses breaks. The engine's binder decides every column name and type; nothing is renamed by the client.</p> <p>What you are most likely to notice:</p> <ul> <li>A bare string operand of an expression is a value, not a column. For one release a warning is raised when the string matches a column name; the warning goes in 2.1. A dotted string is not split into parts; pass the parts separately.</li> </ul> <!-- raw HTML omitted --> </blockquote> <p>... (truncated)</p> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/duckdb/duckdb-python/commit/83c719961098611e769dc4bab0bda2bf149a2d1e";><code>83c7199</code></a> Build the dictionary test's indices with a 64 bit arange (<a href="https://redirect.github.com/duckdb/duckdb-python/issues/705";>#705</a>)</li> <li><a href="https://github.com/duckdb/duckdb-python/commit/b12fd55f6b5c72700533d91210b8e3c7eb9a1807";><code>b12fd55</code></a> Build the dictionary test's indices with a 64 bit arange</li> <li><a href="https://github.com/duckdb/duckdb-python/commit/6aa2ce30e28b30c0a9b5d0a2d27b55549f73dc39";><code>6aa2ce3</code></a> Build the Linux and macOS wheels with clang and ThinLTO (<a href="https://redirect.github.com/duckdb/duckdb-python/issues/703";>#703</a>)</li> <li><a href="https://github.com/duckdb/duckdb-python/commit/2f0ac7942cc1b477a4e760c2f8dfc4c677946550";><code>2f0ac79</code></a> Pin submodule to v2.0.0 hash</li> <li><a href="https://github.com/duckdb/duckdb-python/commit/f7826ad6b996500af32ee128478905298e1aeef0";><code>f7826ad</code></a> Fix wrong values and lost parallelism in the pandas, numpy and Arrow input sc...</li> <li><a href="https://github.com/duckdb/duckdb-python/commit/2f2a0dd1dc9482b81328930ad96968f54e8df392";><code>2f2a0dd</code></a> Bring back to_arrow_table and to_arrow_reader on DB-API results over a re-fee...</li> <li><a href="https://github.com/duckdb/duckdb-python/commit/6086a15466f36f85dd0685a05c407e59eb7247c8";><code>6086a15</code></a> Keep the macOS wheels native, like the engine's release</li> <li><a href="https://github.com/duckdb/duckdb-python/commit/07ee3e2eb98b0b64dd66a58a5bbf1e8001d1030d";><code>07ee3e2</code></a> Build the Linux and macOS wheels with clang and ThinLTO</li> <li><a href="https://github.com/duckdb/duckdb-python/commit/a2c151f4bbdb9015aaa2814eb0dc4d5b5813b0c8";><code>a2c151f</code></a> Register the linked extensions when the module loads, not only on import (<a href="https://redirect.github.com/duckdb/duckdb-python/issues/701";>#701</a>)</li> <li><a href="https://github.com/duckdb/duckdb-python/commit/dd2a3e9251d6f6a385f41c18ac88628522172031";><code>dd2a3e9</code></a> Read numpy masks a word at a time</li> <li>Additional commits viewable in <a href="https://github.com/duckdb/duckdb-python/compare/v1.5.6...v2.0.0";>compare view</a></li> </ul> </details> <br /> [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) </details> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
