dependabot[bot] opened a new pull request, #4230: URL: https://github.com/apache/iceberg-python/pull/4230
Bumps [gcsfs](https://github.com/fsspec/gcsfs) from 2026.8.1 to 2026.10.1. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/fsspec/gcsfs/releases";>gcsfs's releases</a>.</em></p> <blockquote> <h2>2026.10.1</h2> <h3>Bug Fix</h3> <p><strong>Fixed event-loop mismatch in</strong> <code>AdaptiveReadaheadCache</code> <strong>after</strong> <code>fsspec.asyn.reset_lock()</code> <strong>or with a custom</strong> <code>loop</code>. When <code>fsspec</code> initializes <code>BackgroundPrefetcher</code> on a loop that differs from <code>gcsfs.loop</code>, <code>GCSFile</code> and <code>ZonalFile</code> now rebind the prefetcher to <code>gcsfs.loop</code>.</p> <h2>2026.10.0</h2> <p><strong>New Features</strong></p> <p><strong>Mutual TLS (mTLS) for HTTP requests when a client certificate is configured.</strong> When <code>google-auth</code> is configured to use a client certificate (<code>GOOGLE_API_USE_CLIENT_CERTIFICATE=true</code>, or a certificate config with a <code>workload</code> section) and a default client certificate source exists, GCSFS now presents that certificate on its HTTP session and, for the default <code>googleapis.com</code> universe, defaults to <code>https://storage.mtls.googleapis.com</code>, as <code>google-cloud-storage</code> does. This is needed for access tokens bound to a client certificate (e.g. Agent Identity), which GCS accepts only over mTLS.</p> <ul> <li><code>endpoint_url</code> and <code>STORAGE_EMULATOR_HOST</code> still take precedence over the mTLS endpoint. If <code>session_kwargs</code> provides a <code>connector</code>, it is used as-is and the certificate is not presented.</li> <li>Set <code>GOOGLE_API_USE_MTLS_ENDPOINT=never</code> to keep the regular endpoint (the certificate is still presented).</li> <li>No behavior change when no client certificate is configured.</li> <li><strong>Known limitations:</strong> gRPC connections (used for Rapid bucket reads/writes, bucket-type detection, and hierarchical namespace folder operations) do not present the client certificate and stay on the regular endpoint. The certificate is loaded when the HTTP session is created and is not reloaded if it rotates.</li> </ul> <p>(<a href="https://redirect.github.com/fsspec/gcsfs/issues/1080";>#1080</a>)</p> <p><strong>Migrate adaptive prefetcher to native fsspec</strong> <code>AdaptiveReadaheadCache</code>: The internal <code>gcsfs.prefetcher</code> module has been removed from GCSFS and upstreamed into <code>fsspec>=2026.9.0</code>, and GCSFS now uses <code>cache_type="adaptive"</code> as its default streaming read caching policy. The prefetcher is still enabled by default when <code>cache_type</code> is not set, and <code>max_prefetch_size</code> and <code>concurrency</code> passed to <code>open()</code> still apply. To use a different cache or disable prefetching, pass <code>cache_type</code> explicitly (e.g. <code>cache_type="none"</code>). For configuration options, architecture, and benchmarks, see <a href="https://github.com/fsspec/gcsfs/blob/main/docs/source/prefetcher.rst";>https://github.com/fsspec/gcsfs/blob/main/docs/source/prefetcher.rst</a></p> <ul> <li>Removed experimental configuration flags <code>USE_EXPERIMENTAL_ADAPTIVE_PREFETCHING</code> and <code>use_experimental_adaptive_prefetching</code>. <strong>(Warning)</strong> They are now silently ignored: if you used them to turn prefetching off, set <code>cache_type</code> explicitly instead.</li> <li>Bumped minimum <code>fsspec</code> dependency to <code>>=2026.9.0</code>.</li> </ul> <p>(<a href="https://redirect.github.com/fsspec/gcsfs/issues/993";>#993</a>)</p> <p><strong>Bug Fixes & Improvements</strong></p> <ul> <li><strong>401 Invalid Credentials responses are now retried in single-request</strong> <code>get_file()</code> <strong>downloads.</strong> The error body was previously not read before validation, so the retry check never saw the "Invalid Credentials" message. Additionally, 401 responses whose message contains lowercase "invalid" (e.g. <code>invalid_token</code>) now raise <code>HttpError</code> instead of <code>ValueError</code>, so requests that GCSFS retries are retried on them when the message contains "Invalid Credentials". (<a href="https://redirect.github.com/fsspec/gcsfs/issues/1084";>#1084</a>)</li> <li><strong>Fixed errors and deadlocks when files are closed from the event loop thread or by the garbage collector.</strong> Previously this could raise <code>NotImplementedError</code> or deadlock the fsspec I/O event loop. Files closed in those situations are now torn down on a dedicated background worker thread; errors during these deferred closes are logged rather than raised. <code>ZonalFile</code> read-pool and write-stream teardown now run independently, each bounded by a timeout (default 60 s), so a read-side failure no longer skips write finalization. Files should still be closed explicitly (e.g. <code>with fs.open(...)</code>): a file still open at interpreter shutdown is marked closed without being flushed. (<a href="https://redirect.github.com/fsspec/gcsfs/issues/1009";>#1009</a>, <a href="https://redirect.github.com/fsspec/gcsfs/issues/1025";>#1025</a>, <a href="https://redirect.github.com/fsspec/gcsfs/issues/1031";>#1031</a>, <a href="https://redirect.github.com/fsspec /gcsfs/issues/1040">#1040</a>)</li> <li><strong>Fixed paths for objects whose names start with</strong> <code>/</code>. Listing and <code>info()</code> results previously dropped the bucket name for such objects (returning <code>/name</code>); they now return <code>bucket//name</code>. (<a href="https://redirect.github.com/fsspec/gcsfs/issues/1044";>#1044</a>)</li> <li><code>rm()</code> <strong>now raises the original error when a whole delete batch fails</strong>, instead of an unrelated <code>TypeError</code>. Aggregation of batch results is also linear instead of quadratic. (<a href="https://redirect.github.com/fsspec/gcsfs/issues/1060";>#1060</a>)</li> <li><strong>Lower CPU overhead in recursive</strong> <code>find()</code> (also used by <code>glob()</code>, <code>du()</code>, and recursive <code>rm()</code>/<code>get()</code>/<code>copy()</code> path expansion): directory entries are now built once per unique directory. In an in-memory benchmark that excludes network time, this step was up to 14x faster on deep hierarchies. (<a href="https://redirect.github.com/fsspec/gcsfs/issues/1046";>#1046</a>)</li> <li><strong>Opening an existing object for reading no longer calls</strong> <code>mimetypes.guess_type()</code> when GCS metadata already provides <code>contentType</code>. (<a href="https://redirect.github.com/fsspec/gcsfs/issues/1059";>#1059</a>)</li> <li><strong>Rapid (zonal) bucket gRPC reads now report the cache type</strong> in request metadata (<code>x-goog-api-client</code>), as HTTP reads already do in the <code>User-Agent</code>. (<a href="https://redirect.github.com/fsspec/gcsfs/issues/1019";>#1019</a>)</li> <li><strong>Bumped minimum</strong> <code>google-cloud-storage</code> <strong>dependency to</strong> <code>>=3.14.1</code>. (<a href="https://redirect.github.com/fsspec/gcsfs/issues/1039";>#1039</a>, <a href="https://redirect.github.com/fsspec/gcsfs/issues/1050";>#1050</a>)</li> </ul> <p><strong>Contributor Tooling: Agent Skills</strong></p> <p>The repository now includes AI coding-agent skills under <code>_agents/skills/</code> (<a href="https://github.com/fsspec/gcsfs/tree/main/_agents/skills";>https://github.com/fsspec/gcsfs/tree/main/_agents/skills</a>) for performance work on GCSFS:</p> <ul> <li><code>create-microbenchmark</code>: instructions for scaffolding a new microbenchmark for a given method under <code>gcsfs/tests/perf/microbenchmarks</code>. The agent first checks that no benchmark for that method exists yet, and asks for confirmation if a method it calls is already benchmarked. (<a href="https://redirect.github.com/fsspec/gcsfs/issues/1012";>#1012</a>)</li> <li><code>autoresearch-profiling</code>: instructions for profile-guided performance optimization. The agent picks a profiler (py-spy, cProfile, or memory_profiler) to find bottlenecks, then runs modify → verify → keep/discard iterations against microbenchmark or subsystem-benchmark metrics. The iteration loop requires the separate <code>autoresearch</code> skill, which is not included in this repository. (<a href="https://redirect.github.com/fsspec/gcsfs/issues/1071";>#1071</a>)</li> </ul> <h2>2026.8.2</h2> <p>Fix <code>NotImplementedError</code> and deadlocks when a file is closed from the event loop thread or by the garbage collector (<a href="https://redirect.github.com/fsspec/gcsfs/issues/1009";>#1009</a>, <a href="https://redirect.github.com/fsspec/gcsfs/issues/1025";>#1025</a>, <a href="https://redirect.github.com/fsspec/gcsfs/issues/1031";>#1031</a>, <a href="https://redirect.github.com/fsspec/gcsfs/issues/1040";>#1040</a>). Such closes now run on a background thread, and prefetcher/zonal teardown is bounded by a timeout.</p> </blockquote> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/fsspec/gcsfs/commit/22cb947018737ba62eb96ceae2bf5dad7886d49e";><code>22cb947</code></a> chore: release 2026.10.1</li> <li><a href="https://github.com/fsspec/gcsfs/commit/e3646f5fc452064c5f6056b8fd3dc6cfb232f73a";><code>e3646f5</code></a> fix: bind AdaptiveReadaheadCache prefetcher to gcsfs.loop after reset_lock (#...</li> <li><a href="https://github.com/fsspec/gcsfs/commit/07427a256c150a7b796cd1017b2c425453747bcc";><code>07427a2</code></a> chore: release 2026.10.0 (<a href="https://redirect.github.com/fsspec/gcsfs/issues/1041";>#1041</a>)</li> <li><a href="https://github.com/fsspec/gcsfs/commit/6089cd65a9c05f6d5d663b9cbb729e935627aa75";><code>6089cd6</code></a> docs(prefetcher): document max_prefetch_size default and how to set it (<a href="https://redirect.github.com/fsspec/gcsfs/issues/1092";>#1092</a>)</li> <li><a href="https://github.com/fsspec/gcsfs/commit/a0bbf46582ca7a36965d022f22a82cf3134e00d3";><code>a0bbf46</code></a> fix(subsystembenchmarks): skip checkpoint_write rows in amplification scrape ...</li> <li><a href="https://github.com/fsspec/gcsfs/commit/8b1f7f11c608b3464a3afa22ad6cdbcbebc7da71";><code>8b1f7f1</code></a> refactor(macrobench): rename hf-pytorch-lightning-cpu workload to hf-datasets...</li> <li><a href="https://github.com/fsspec/gcsfs/commit/4cbb2e29427a4638bc6a0164e45533bc6359b7f7";><code>4cbb2e2</code></a> feat: use fsspec adaptive-readahead cache and remove internal prefetcher (<a href="https://redirect.github.com/fsspec/gcsfs/issues/993";>#993</a>)</li> <li><a href="https://github.com/fsspec/gcsfs/commit/04f9945dc36a5c173e08292ad46922af367c362f";><code>04f9945</code></a> feat(subsystembenchmarks): deliver data-loading benchmark batches to GPU on C...</li> <li><a href="https://github.com/fsspec/gcsfs/commit/c406e1dca0fa720385846493742aeb3d2837cf9f";><code>c406e1d</code></a> fix: read error response body in _get_file_request to enable 401 retry (<a href="https://redirect.github.com/fsspec/gcsfs/issues/1084";>#1084</a>)</li> <li><a href="https://github.com/fsspec/gcsfs/commit/c7fbdb3028d39a8a7da34442e4b99e6706af36af";><code>c7fbdb3</code></a> perf(subsystembenchmarks): trim default WebDataset sweep to 11 cases (<a href="https://redirect.github.com/fsspec/gcsfs/issues/1079";>#1079</a>)</li> <li>Additional commits viewable in <a href="https://github.com/fsspec/gcsfs/compare/2026.8.1...2026.10.1";>compare view</a></li> </ul> </details> <br /> [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) </details> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
