sdf-jkl commented on code in PR #11511:
URL: https://github.com/apache/arrow-rs/pull/11511#discussion_r4595936130
##########
parquet-variant/src/builder/object.rs:
##########
@@ -151,6 +151,26 @@ impl<'a, S: BuilderSpecificState> ObjectBuilder<'a, S> {
Ok(())
}
+ /// Adds the field named by entry `field_id` of the metadata dictionary,
copying the encoded
+ /// `value` as is.
+ ///
+ /// The caller must ensure that `field_id`, and every field id inside
`value`, belong to the
+ /// metadata dictionary that this builder writes against. Copying a field
out of an object that
+ /// uses that same dictionary, as shredding does, satisfies this.
+ #[doc(hidden)]
+ pub fn insert_encoded(&mut self, field_id: u32, value: &[u8]) ->
Result<(), ArrowError> {
+ let state = &mut self.parent_state;
+ let field_start = state.value_builder.offset() -
state.saved_value_builder_offset;
+ if self.fields.insert(field_id, field_start).is_some() &&
self.validate_unique_fields {
Review Comment:
🤖 💬 : Could we check for a duplicate before changing `self.fields`? `insert`
replaces the original field's offset even when this method returns `Err`. If
the caller continues using the builder, a later field can silently become the
value of the rejected duplicate.
Minimal reproducer (save as `parquet-variant/tests/pr11511_duplicate.rs`):
```rust
use parquet_variant::{Variant, VariantBuilder};
#[test]
fn encoded_duplicate_preserves_original_value() {
let mut builder = VariantBuilder::new()
.with_field_names(["a", "b"])
.with_validate_unique_fields(true);
let mut object = builder.new_object();
object.insert_encoded(0, &[12, 1]).unwrap();
assert!(object.insert_encoded(0, &[12, 2]).is_err());
object.insert_encoded(1, &[12, 3]).unwrap();
object.finish();
let (metadata, value) = builder.finish();
let variant = Variant::try_new(&metadata, &value).unwrap();
assert_eq!(variant.get_object_field("a"), Some(Variant::Int8(1)));
assert_eq!(variant.get_object_field("b"), Some(Variant::Int8(3)));
}
```
Run:
```sh
cargo test -p parquet-variant --test pr11511_duplicate
```
Expected: the original `a=1` survives the rejected insertion, and `b=3` is
appended. Actual: the first assertion fails with `left: Some(Int8(3)), right:
Some(Int8(1))`; the finished object contains `a=3, b=3` and passes
`Variant::try_new` validation.
Reproduced on this PR and with its patch applied to upstream `7a1e79d0c2`.
The equivalent sequence using `try_insert_bytes` passes. An entry check that
rejects the duplicate before mutation would preserve the existing insertion
behavior.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]