dependabot[bot] opened a new pull request, #2851: URL: https://github.com/apache/iceberg-python/pull/2851
Bumps [ray](https://github.com/ray-project/ray) from 2.44.0 to 2.53.0. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/ray-project/ray/releases";>ray's releases</a>.</em></p> <blockquote> <h2>Ray-2.53.0</h2> <h2>Highlights</h2> <ul> <li>Ray plans to drop support for Pydantic V1 starting version 2.56.0. Please see <a href="https://redirect.github.com/ray-project/ray/issues/58876";>this RFC</a> for details.</li> <li>Ray Data now has support for bounded reading from Kafka and improved Iceberg support.</li> </ul> <h2>Ray Data</h2> <p>🎉 New Features</p> <ul> <li><strong>Autoscaling</strong>: New utilization-based cluster autoscaler for Ray Data workloads (<a href="https://redirect.github.com/ray-project/ray/issues/59353";>#59353</a>, <a href="https://redirect.github.com/ray-project/ray/issues/59362";>#59362</a>, <a href="https://redirect.github.com/ray-project/ray/issues/59366";>#59366</a>). To use this new autoscaler set RAY_DATA_CLUSTER_AUTOSCALER=V2.</li> <li><strong>Kafka Datasource</strong>: Add Kafka as a native datasource for data ingestion (<a href="https://redirect.github.com/ray-project/ray/issues/58592";>#58592</a>)</li> <li><strong>Dataset summary API</strong>: Add <code>Dataset.summary()</code> API for quick dataset inspection (<a href="https://redirect.github.com/ray-project/ray/issues/58862";>#58862</a>)</li> <li><strong>Iceberg support</strong>: Add Iceberg schema evolution, upsert, and overwrite support (<a href="https://redirect.github.com/ray-project/ray/issues/59210";>#59210</a>, <a href="https://redirect.github.com/ray-project/ray/issues/59335";>#59335</a>)</li> <li><strong>Graceful error handling</strong>: Add <code>should_continue_on_error</code> for graceful error handling in batch inference (<a href="https://redirect.github.com/ray-project/ray/issues/59212";>#59212</a>)</li> <li><strong>Datetime compute expressions</strong>: Add datetime compute expressions support (<a href="https://redirect.github.com/ray-project/ray/issues/58740";>#58740</a>)</li> <li><strong>Grouped <code>with_column</code> expressions</strong>: Enable expressions for grouped <code>with_column</code> in Ray Data (<a href="https://redirect.github.com/ray-project/ray/issues/58231";>#58231</a>)</li> <li><strong>Parallelized collation</strong>: Parallelize <code>DefaultCollateFn</code>, <code>arrow_batch_to_tensors</code> (<a href="https://redirect.github.com/ray-project/ray/issues/58821";>#58821</a>)</li> </ul> <p>💫 Enhancements</p> <ul> <li><strong>Optimized Autoscaler Step Size</strong>: Optimize autoscaler to support configurable step size for actor pool scaling (<a href="https://redirect.github.com/ray-project/ray/issues/58726";>#58726</a>)</li> <li><strong>Improved Streaming Repartition</strong>: Improve streaming repartition performance (<a href="https://redirect.github.com/ray-project/ray/issues/58728";>#58728</a>)</li> <li><strong>Actor init retry</strong>: Add actor retry if there's a failure in <code>__init__</code> (<a href="https://redirect.github.com/ray-project/ray/issues/59105";>#59105</a>)</li> <li><strong>Fused Repartition + MapBatches</strong>: Fuse StreamingRepartition with MapBatches operators to scale collate (<a href="https://redirect.github.com/ray-project/ray/issues/59108";>#59108</a>)</li> <li><strong>Combined repartitions</strong>: Combine consecutive repartitions for efficiency (<a href="https://redirect.github.com/ray-project/ray/issues/59145";>#59145</a>)</li> <li><strong>Prefetch buffering</strong>: Handle prefetch buffering in <code>iter_batches</code> (<a href="https://redirect.github.com/ray-project/ray/issues/58657";>#58657</a>)</li> <li><strong>HashShuffle block breakdown</strong>: <code>HashShuffleAggregator</code> breaks down blocks on finalize (<a href="https://redirect.github.com/ray-project/ray/issues/58603";>#58603</a>)</li> <li><strong>Backpressure tuning</strong>: Tune concurrency cap backpressure object store budget ratio (<a href="https://redirect.github.com/ray-project/ray/issues/58813";>#58813</a>)</li> <li><strong>Non-string ApproximateTopK</strong>: Support non-string items for <code>ApproximateTopK</code> aggregator (<a href="https://redirect.github.com/ray-project/ray/issues/58659";>#58659</a>)</li> <li><strong>Lance version support</strong>: Add version support to <code>read_lance()</code> (<a href="https://redirect.github.com/ray-project/ray/issues/58895";>#58895</a>)</li> <li><strong>Dashboard metrics</strong>: Add <code>time_to_first_batch</code> and <code>get_ref_bundles</code> metrics to data dashboard (<a href="https://redirect.github.com/ray-project/ray/issues/58912";>#58912</a>)</li> <li><strong>Iter prefetched bytes stats</strong>: Add <code>iter_prefetched_bytes</code> statistics tracking (<a href="https://redirect.github.com/ray-project/ray/issues/58900";>#58900</a>)</li> <li><strong>Configurable batching for <code>iter_batches</code></strong>: Add configurable batching for <code>resolve_block_refs</code> to speed up <code>iter_batches</code> (<a href="https://redirect.github.com/ray-project/ray/issues/58467";>#58467</a>)</li> <li><strong>Improved dashboard metrics</strong>: Improve Ray Data dashboard metrics display (<a href="https://redirect.github.com/ray-project/ray/issues/58667";>#58667</a>)</li> <li><strong>Histogram percentiles</strong>: Update Ray Data histograms to show percentiles in data dashboard (<a href="https://redirect.github.com/ray-project/ray/issues/58650";>#58650</a>)</li> <li><strong>Deprecated API removal</strong>: Remove deprecated <code>read_parquet_bulk</code> API (<a href="https://redirect.github.com/ray-project/ray/issues/58970";>#58970</a>)</li> <li><strong>Block shaping option</strong>: Add disable block shaping option to BlockOutputBuffer (<a href="https://redirect.github.com/ray-project/ray/issues/58757";>#58757</a>)</li> <li><strong>Removed concurrency lock</strong>: Remove concurrency lock for better performance (<a href="https://redirect.github.com/ray-project/ray/issues/56798";>#56798</a>)</li> </ul> <p>🔨 Fixes</p> <ul> <li><strong>Fixes to Unique</strong>: Fix support of list types for Unique aggregator (<a href="https://redirect.github.com/ray-project/ray/issues/58916";>#58916</a>)</li> <li><strong>Parquet NaN fix</strong>: Fix reading from written parquet for numpy with NaNs (<a href="https://redirect.github.com/ray-project/ray/issues/59172";>#59172</a>)</li> <li><strong>Hash Shuffle empty block</strong>: Fix empty block sort in hash shuffle operator (<a href="https://redirect.github.com/ray-project/ray/issues/58836";>#58836</a>)</li> <li><strong>Hive partitioning pushdown</strong>: Fix pushdown optimizations with Hive partitioning (<a href="https://redirect.github.com/ray-project/ray/issues/58723";>#58723</a>)</li> <li><strong>Object Store usage reporting</strong>: Fix <code>obj_store_mem_max_pending_output_per_task</code> reporting (<a href="https://redirect.github.com/ray-project/ray/issues/58864";>#58864</a>)</li> <li><strong>Pyarrow FileSystem serialization fix</strong>: Handle filesystem serialization issue in <code>get_parquet_dataset</code> (<a href="https://redirect.github.com/ray-project/ray/issues/57047";>#57047</a>)</li> <li><strong>Azure UC SAS</strong>: Handle Azure UC user delegation SAS (<a href="https://redirect.github.com/ray-project/ray/issues/59393";>#59393</a>)</li> <li><strong>Async UDF Thread Cleanup</strong>: Close threads from async UDF after actor died (<a href="https://redirect.github.com/ray-project/ray/issues/59261";>#59261</a>)</li> </ul> <!-- raw HTML omitted --> </blockquote> <p>... (truncated)</p> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/ray-project/ray/commit/0de211850589aea71f842873bc32574c702ab492";><code>0de2118</code></a> [Data] Concurrency cap backpressure with tuning (Disabled) (<a href="https://redirect.github.com/ray-project/ray/issues/59519";>#59519</a>)</li> <li><a href="https://github.com/ray-project/ray/commit/1736619b36ca2e48a9075daf65a7c66d281c66a7";><code>1736619</code></a> [Data][Cherry-pick] Fix bug where <code>AutoscalingCoordinator</code> crashes if you req...</li> <li><a href="https://github.com/ray-project/ray/commit/3332314b258b7bf679a502ac94711f8e770ed8ea";><code>3332314</code></a> add missing cuda 12.9 ray-extra (<a href="https://redirect.github.com/ray-project/ray/issues/59495";>#59495</a>)</li> <li><a href="https://github.com/ray-project/ray/commit/b7d0c5b543415db693354778ca370095f256c185";><code>b7d0c5b</code></a> change version to 2.53.0 (<a href="https://redirect.github.com/ray-project/ray/issues/59487";>#59487</a>)</li> <li><a href="https://github.com/ray-project/ray/commit/0ddb7eea6e84cad4b51dbdb5745164535e0a5e97";><code>0ddb7ee</code></a> [core][rdt] Enable nixl for RDT Microbenchmarks (<a href="https://redirect.github.com/ray-project/ray/issues/59291";>#59291</a>)</li> <li><a href="https://github.com/ray-project/ray/commit/31a0e1eb74cec8e5f0541cd70ec60ed9057eeb22";><code>31a0e1e</code></a> [Serve][3/n] Add router queue latency (<a href="https://redirect.github.com/ray-project/ray/issues/59233";>#59233</a>)</li> <li><a href="https://github.com/ray-project/ray/commit/22cda946375db60517c7da7b57b6fd16a029160f";><code>22cda94</code></a> [train] removes single python ml build (<a href="https://redirect.github.com/ray-project/ray/issues/59465";>#59465</a>)</li> <li><a href="https://github.com/ray-project/ray/commit/8d9a15f9397e9dce622710ce9156d6306b77abac";><code>8d9a15f</code></a> [core] Revert server CreateCall creation change <a href="https://redirect.github.com/ray-project/ray/issues/55904";>#55904</a> (<a href="https://redirect.github.com/ray-project/ray/issues/59448";>#59448</a>)</li> <li><a href="https://github.com/ray-project/ray/commit/694e6fd68c4d2c4558c91cd278b379b77098a5a9";><code>694e6fd</code></a> Revert "[Data] Fix DataContext deserialization issue with StatsActor" (<a href="https://redirect.github.com/ray-project/ray/issues/59458";>#59458</a>)</li> <li><a href="https://github.com/ray-project/ray/commit/e0049dcee72cd2865ba1405a85382d284a3a010c";><code>e0049dc</code></a> [docs/data] Add <code>download</code> to key user journeys in documentation (<a href="https://redirect.github.com/ray-project/ray/issues/59417";>#59417</a>)</li> <li>Additional commits viewable in <a href="https://github.com/ray-project/ray/compare/ray-2.44.0...ray-2.53.0";>compare view</a></li> </ul> </details> <br /> [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) </details> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
