c-thiel commented on code in PR #2188:
URL: https://github.com/apache/iceberg-rust/pull/2188#discussion_r3330061392
##########
crates/iceberg/src/writer/file_writer/parquet_writer.rs:
##########
@@ -191,20 +207,11 @@ impl SchemaVisitor for IndexByParquetPathName {
}
fn primitive(&mut self, _p: &PrimitiveType) -> Result<Self::T> {
- let full_name = self.field_names.iter().map(String::as_str).join(".");
- let field_id = self.field_id;
- if let Some(existing_field_id) =
self.name_to_id.get(full_name.as_str()) {
- return Err(Error::new(
- ErrorKind::DataInvalid,
- format!(
- "Invalid schema: multiple fields for name {full_name}:
{field_id} and {existing_field_id}"
- ),
- ));
- } else {
- self.name_to_id.insert(full_name, field_id);
- }
+ self.insert_current_path()
+ }
- Ok(())
+ fn variant(&mut self, _v: &VariantType) -> Result<Self::T> {
Review Comment:
this PR is unshredded variants only, with two follow-ups:
1. Write annotation (your comment): we emit a plain Struct(Binary,Binary)
without variantType(...) since `variant_experimental` is off. Doesn't break
Java read-back — it resolves variant by field-id, not the annotation — but I'll
track adding it.
2. Shredded reads: a `typed_value` sub-field was being silently dropped
(corrupt data). Added a guard that returns `FeatureUnsupported` + a test; full
shredding reconstruction is a follow-up.
Guard added here:
https://github.com/apache/iceberg-rust/pull/2188/commits/b702f5ae07bd24b69fa722b4d31c0bce19b45a14
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
