xanderbailey commented on code in PR #2701:
URL: https://github.com/apache/iceberg-rust/pull/2701#discussion_r3668229870
##########
crates/iceberg/src/writer/file_writer/parquet_writer.rs:
##########
@@ -85,19 +90,34 @@ impl ParquetWriterBuilder {
/// Currently translates the content-defined-chunking keys
/// (`write.parquet.content-defined-chunking.*`); other keys fall back to
/// parquet-rs defaults.
- pub fn from_table_properties(table_props: &TableProperties, schema:
SchemaRef) -> Self {
+ ///
+ /// When `encryption.key-id` is set, records the DEK length.
+ pub fn from_table_properties(table_props: &TableProperties, schema:
SchemaRef) -> Result<Self> {
let cdc = table_props.cdc_enabled.then_some(CdcOptions {
min_chunk_size: table_props.cdc_min_chunk_size,
max_chunk_size: table_props.cdc_max_chunk_size,
norm_level: table_props.cdc_norm_level,
});
+
// TODO: translate the remaining write.parquet.* keys (e.g.
compression-codec,
// row-group-size-bytes, page-size-bytes).
// This constructor is intended to be the single place that maps them.
let props = WriterProperties::builder()
.set_content_defined_chunking(cdc)
.build();
- Self::new_with_match_mode(props, schema, FieldMatchMode::Id)
+
+ let data_encryption_key_size = table_props
+ .encryption_key_id
+ .is_some()
+ .then(||
AesKeySize::from_key_length(table_props.encryption_data_key_length))
+ .transpose()?;
Review Comment:
Yeah I thought about this earlier this week. Fortunately in the case of
`encryption.key-id` the spec notes that it should not be changed or removed for
the lifetime of the table. So it's existence is immutable per the spec.
https://iceberg.apache.org/docs/nightly/encryption/#catalog-security-requirements
.
I need to think a little more deeply about the case where you have
encryption keys added in memory in the manager and then you refresh the table
before the commit which risks loosing those keys but I think we're safe from
that today.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
