This is an automated email from the ASF dual-hosted git repository.
CTTY pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/iceberg-rust.git
The following commit(s) were added to refs/heads/main by this push:
new 7c4565114 feat(encryption) [6/N] support `ArrowReaderOptions` in
`get_metadata` (#2402)
7c4565114 is described below
commit 7c4565114bef054a363712b15ae515c8d5332315
Author: Xander <[email protected]>
AuthorDate: Mon May 18 23:19:17 2026 +0100
feat(encryption) [6/N] support `ArrowReaderOptions` in `get_metadata`
(#2402)
## Which issue does this PR close?
Working towards https://github.com/apache/iceberg-rust/issues/2034
<!--
We generally require a GitHub issue to be filed for all bug fixes and
enhancements and this helps us generate change logs for our releases.
You can link an issue to this PR using the GitHub syntax. For example
`Closes #123` indicates that this PR will close issue #123.
-->
More detail in the description of the issue but previously we weren't
able to pass down `ArrowReaderOptions` to the `ParquetMetaDataReader`
but since arrow v55.1 we can. At the same time I also pass in
`file_decryption_properties` which is required for supporting
[encryption](https://github.com/apache/iceberg-rust/issues/2034)
- Closes #1934
## What changes are included in this PR?
<!--
Provide a summary of the modifications in this PR. List the main changes
such as new features, bug fixes, refactoring, or any other updates.
-->
## Are these changes tested?
<!--
Specify what test covers (unit test, integration test, etc.).
If tests are not included in your PR, please explain why (for example,
are they covered by existing tests)?
-->
---
crates/iceberg/Cargo.toml | 2 +-
crates/iceberg/src/arrow/reader/file_reader.rs | 14 ++++++++++----
2 files changed, 11 insertions(+), 5 deletions(-)
diff --git a/crates/iceberg/Cargo.toml b/crates/iceberg/Cargo.toml
index de34d365e..b3f082e0d 100644
--- a/crates/iceberg/Cargo.toml
+++ b/crates/iceberg/Cargo.toml
@@ -62,7 +62,7 @@ moka = { version = "0.12.10", features = ["future"] }
murmur3 = { workspace = true }
once_cell = { workspace = true }
ordered-float = { workspace = true }
-parquet = { workspace = true, features = ["async"] }
+parquet = { workspace = true, features = ["async", "encryption"] }
rand = { workspace = true }
reqwest = { workspace = true }
roaring = { workspace = true }
diff --git a/crates/iceberg/src/arrow/reader/file_reader.rs
b/crates/iceberg/src/arrow/reader/file_reader.rs
index 79fbcc796..63b7f0675 100644
--- a/crates/iceberg/src/arrow/reader/file_reader.rs
+++ b/crates/iceberg/src/arrow/reader/file_reader.rs
@@ -106,12 +106,16 @@ impl AsyncFileReader for ArrowFileReader {
.boxed()
}
- // TODO: currently we don't respect `ArrowReaderOptions` cause it don't
expose any method to access the option field
- // we will fix it after `v55.1.0` is released in
https://github.com/apache/arrow-rs/issues/7393
fn get_metadata(
&mut self,
- _options: Option<&'_ ArrowReaderOptions>,
+ options: Option<&'_ ArrowReaderOptions>,
) -> BoxFuture<'_, parquet::errors::Result<Arc<ParquetMetaData>>> {
+ let decryption_properties = options
+ .and_then(|opts| opts.file_decryption_properties())
+ .cloned();
+
+ let metadata_options = options.map(|opts|
opts.metadata_options().clone());
+
async move {
let reader = ParquetMetaDataReader::new()
.with_prefetch_hint(self.parquet_read_options.metadata_size_hint())
@@ -124,7 +128,9 @@ impl AsyncFileReader for ArrowFileReader {
))
.with_offset_index_policy(PageIndexPolicy::from(
self.parquet_read_options.preload_offset_index(),
- ));
+ ))
+ .with_metadata_options(metadata_options)
+ .with_decryption_properties(decryption_properties);
let size = self.meta.size;
let meta = reader.load_and_finish(self, size).await?;
