dependabot[bot] opened a new pull request, #4249: URL: https://github.com/apache/iceberg-python/pull/4249
Bumps [polars](https://github.com/pola-rs/polars) from 1.44.2 to 2.1.0. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/pola-rs/polars/releases";>polars's releases</a>.</em></p> <blockquote> <h2>Python Polars 2.1.0</h2> <h2>💥 Breaking changes</h2> <ul> <li>Disallow mixed string <=> numeric arithmetic in eager (<a href="https://redirect.github.com/pola-rs/polars/issues/30071";>#30071</a>)</li> </ul> <h2>🚀 Performance improvements</h2> <ul> <li>Limit the memory of buckets sorted ahead in the sort flush (<a href="https://redirect.github.com/pola-rs/polars/issues/30297";>#30297</a>)</li> <li>Make file readers wait for spills while over the memory budget (<a href="https://redirect.github.com/pola-rs/polars/issues/30296";>#30296</a>)</li> <li>Count unspilled frames in the memory estimate right away (<a href="https://redirect.github.com/pola-rs/polars/issues/30295";>#30295</a>)</li> <li>Simplify more boolean expressions and drop redundant null checks (<a href="https://redirect.github.com/pola-rs/polars/issues/30293";>#30293</a>)</li> <li>Wait for spills while over the out-of-core memory budget (<a href="https://redirect.github.com/pola-rs/polars/issues/30294";>#30294</a>)</li> <li>Lower the default out-of-core prefetch fraction to 0.5 (<a href="https://redirect.github.com/pola-rs/polars/issues/30292";>#30292</a>)</li> <li>Aggregate unique and n_unique in the streaming group-by itself (<a href="https://redirect.github.com/pola-rs/polars/issues/30281";>#30281</a>)</li> <li>Return freed memory to the OS right away while out-of-core data is spilled (<a href="https://redirect.github.com/pola-rs/polars/issues/30280";>#30280</a>)</li> <li>Gather <code>List(Array)</code> and <code>Array</code> columns without per-row slices (<a href="https://redirect.github.com/pola-rs/polars/issues/30278";>#30278</a>)</li> <li>Evaluate the streaming window node out of core (<a href="https://redirect.github.com/pola-rs/polars/issues/30282";>#30282</a>)</li> <li>Avoid deep copies when building a DataFrame from flat dataclasses (<a href="https://redirect.github.com/pola-rs/polars/issues/30275";>#30275</a>)</li> <li>Sort the window partitions by each order key instead of their row encoding (<a href="https://redirect.github.com/pola-rs/polars/issues/30277";>#30277</a>)</li> <li>Scan the haystack for a single needle in <code>is_in</code> (<a href="https://redirect.github.com/pola-rs/polars/issues/30276";>#30276</a>)</li> <li>Prefer fixed-width columns when only the row count is needed (<a href="https://redirect.github.com/pola-rs/polars/issues/30274";>#30274</a>)</li> <li>Native gather for List arrays (<a href="https://redirect.github.com/pola-rs/polars/issues/30019";>#30019</a>)</li> <li>Evaluate windows in filter predicates in the streaming window node (<a href="https://redirect.github.com/pola-rs/polars/issues/30272";>#30272</a>)</li> <li>Don't busy-wait in RAYON.block_on (<a href="https://redirect.github.com/pola-rs/polars/issues/30259";>#30259</a>)</li> <li>Encode nested Parquet levels directly from the level buffers (<a href="https://redirect.github.com/pola-rs/polars/issues/30251";>#30251</a>)</li> <li>Reduce memory of CSV scans on wide rows (<a href="https://redirect.github.com/pola-rs/polars/issues/30165";>#30165</a>)</li> <li>Avoid quadratic JSON serialization of nested lists and maps (<a href="https://redirect.github.com/pola-rs/polars/issues/30203";>#30203</a>)</li> <li>Skip the gather when exploding lists without nulls (<a href="https://redirect.github.com/pola-rs/polars/issues/30234";>#30234</a>)</li> <li>Speed up writing nested Parquet columns (<a href="https://redirect.github.com/pola-rs/polars/issues/30245";>#30245</a>)</li> <li>Don't keep whole parquet string dictionaries alive for a few selected rows (<a href="https://redirect.github.com/pola-rs/polars/issues/30248";>#30248</a>)</li> <li>Set default spill compression level to 1 (<a href="https://redirect.github.com/pola-rs/polars/issues/30237";>#30237</a>)</li> <li>Reuse zstd contexts in compression and avoid memcpys (<a href="https://redirect.github.com/pola-rs/polars/issues/30233";>#30233</a>)</li> <li>Remove repeated and scalar ordering keys before the sort rewrites (<a href="https://redirect.github.com/pola-rs/polars/issues/30231";>#30231</a>)</li> <li>Remove repeated keys from by's (<a href="https://redirect.github.com/pola-rs/polars/issues/30095";>#30095</a>)</li> <li>Keep only one guessed unique key per join order leaf (<a href="https://redirect.github.com/pola-rs/polars/issues/30191";>#30191</a>)</li> <li>Don't let extra join keys shrink an estimate once one key is unique (<a href="https://redirect.github.com/pola-rs/polars/issues/30190";>#30190</a>)</li> <li>Reduce binary size of asof join with <code>by</code> keys (<a href="https://redirect.github.com/pola-rs/polars/issues/30192";>#30192</a>)</li> <li>Reduce binary size of SQL visitors and parallel group-by collect (<a href="https://redirect.github.com/pola-rs/polars/issues/30171";>#30171</a>)</li> <li>Push as-of join key filters to the right input (<a href="https://redirect.github.com/pola-rs/polars/issues/30123";>#30123</a>)</li> <li>Drop null as-of right keys with a parallel filter before the join node (<a href="https://redirect.github.com/pola-rs/polars/issues/30125";>#30125</a>)</li> <li>Keep as-of nearest join state small (<a href="https://redirect.github.com/pola-rs/polars/issues/30126";>#30126</a>)</li> <li>Explode lists in group_by aggregations without a per-group apply (<a href="https://redirect.github.com/pola-rs/polars/issues/30020";>#30020</a>)</li> <li>Streaming OOC group-by with shuffles (<a href="https://redirect.github.com/pola-rs/polars/issues/29911";>#29911</a>)</li> <li>Drop scalar ordering keys in the optimizer (<a href="https://redirect.github.com/pola-rs/polars/issues/30063";>#30063</a>)</li> <li>Merge union inputs that scan files of the same kind into one multi-file scan (<a href="https://redirect.github.com/pola-rs/polars/issues/30056";>#30056</a>)</li> <li>Use the streaming scalar window for std, var and combined reductions in <code>over</code> (<a href="https://redirect.github.com/pola-rs/polars/issues/30066";>#30066</a>)</li> <li>Limit the bytes of local CSV and NDJSON files read at once in multi-file scans (<a href="https://redirect.github.com/pola-rs/polars/issues/30055";>#30055</a>)</li> <li>Check the matched flag of join keys in the table entry before the bitset (<a href="https://redirect.github.com/pola-rs/polars/issues/30067";>#30067</a>)</li> <li>Track matched build rows of all join tables in a bitset (<a href="https://redirect.github.com/pola-rs/polars/issues/30050";>#30050</a>)</li> <li>Add dedicated list gather kernels (<a href="https://redirect.github.com/pola-rs/polars/issues/30049";>#30049</a>)</li> </ul> <!-- raw HTML omitted --> </blockquote> <p>... (truncated)</p> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/pola-rs/polars/commit/7f902bd3ffd27ff82d14278a09a295ff2f7aed92";><code>7f902bd</code></a> Python Polars 2.1.0 (<a href="https://redirect.github.com/pola-rs/polars/issues/30124";>#30124</a>)</li> <li><a href="https://github.com/pola-rs/polars/commit/dfb63c72307a5dd11159dd397ebf4144efe9f86f";><code>dfb63c7</code></a> fix(rust): Include rolling index columns in expression metadata (<a href="https://redirect.github.com/pola-rs/polars/issues/29245";>#29245</a>)</li> <li><a href="https://github.com/pola-rs/polars/commit/a26073c8b2128110002f781fe6907ba2f72625d5";><code>a26073c</code></a> perf: Limit the memory of buckets sorted ahead in the sort flush (<a href="https://redirect.github.com/pola-rs/polars/issues/30297";>#30297</a>)</li> <li><a href="https://github.com/pola-rs/polars/commit/8d110869029150618f1a315b4d23c5adf2de44de";><code>8d11086</code></a> test: Fix allocator-dependent failure in masked zero-copy <code>to_numpy</code> test (<a href="https://redirect.github.com/pola-rs/polars/issues/3";>#3</a>...</li> <li><a href="https://github.com/pola-rs/polars/commit/a9a998b91be5adb4c8987274e3f0928cd0573ddf";><code>a9a998b</code></a> perf: Make file readers wait for spills while over the memory budget (<a href="https://redirect.github.com/pola-rs/polars/issues/30296";>#30296</a>)</li> <li><a href="https://github.com/pola-rs/polars/commit/8b57b775d5edf57318025b903b870e25e92f9cd6";><code>8b57b77</code></a> perf: Count unspilled frames in the memory estimate right away (<a href="https://redirect.github.com/pola-rs/polars/issues/30295";>#30295</a>)</li> <li><a href="https://github.com/pola-rs/polars/commit/720d744d8359651dc577f19e97b94488e88611c6";><code>720d744</code></a> refactor: Simplify the out-of-core decay state and tests (<a href="https://redirect.github.com/pola-rs/polars/issues/30283";>#30283</a>)</li> <li><a href="https://github.com/pola-rs/polars/commit/f7a32274ebc8c2c6f6f6191f9ab1f5c60502fc75";><code>f7a3227</code></a> perf: Simplify more boolean expressions and drop redundant null checks (<a href="https://redirect.github.com/pola-rs/polars/issues/30293";>#30293</a>)</li> <li><a href="https://github.com/pola-rs/polars/commit/48de7aeb5fa93eaa0de1854dbe44302de1eff2c9";><code>48de7ae</code></a> perf: Wait for spills while over the out-of-core memory budget (<a href="https://redirect.github.com/pola-rs/polars/issues/30294";>#30294</a>)</li> <li><a href="https://github.com/pola-rs/polars/commit/28cfd43573759a60bbe39ec4456c0145421ed339";><code>28cfd43</code></a> fix: Compare <code>Array(Struct)</code> rows field by field (<a href="https://redirect.github.com/pola-rs/polars/issues/30290";>#30290</a>)</li> <li>Additional commits viewable in <a href="https://github.com/pola-rs/polars/compare/py-1.44.2...py-2.1.0";>compare view</a></li> </ul> </details> <br /> [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) </details> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
