[
https://issues.apache.org/jira/browse/AVRO-3974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
mknaw updated AVRO-3974:
------------------------
Description:
A schema with a field that is a ref to a different schema is incompatible with
itself, per `SchemaCompatibility`:
{code:java}
use apache_avro::{schema_compatibility::SchemaCompatibility, Schema};fn main() {
let schema_strs = vec![
r#"{
"type": "record",
"name": "Child",
"namespace": "avro",
"fields": [
{
"name": "val",
"type": "int"
}
]
}
"#,
r#"{
"type": "record",
"name": "Parent",
"namespace": "avro",
"fields": [
{
"name": "child",
"type": "avro.Child"
}
]
}
"#,
];
let schemas = Schema::parse_list(&schema_strs).unwrap();
if let Err(e) = SchemaCompatibility::can_read(
&schemas[1],
&schemas[1]
) {
eprintln!("{}", e);
}
} {code}
Here it is somewhat ambiguous to me how the fix should look. of course the
simplest is just to verify the names are the same, and maybe if doing compat
checks across a whole batch of schemas then check for incompatibilities when
checking the referenced schemas themselves. But I'm not sure this is quite
satisfactory - imagine having a complex field that was previously defined
inline, but now want to reuse it across multiple schemas. In this case, it does
not seem like replacing the inline definition with a ref should be
incompatible. So, it seems that some sort of "recursive flattening" of the
schema prior to compat check would be the ideal fix.
was:
A schema with a logicalType uuid field is incompatible with itself, per
`SchemaCompatibility`:
{code:java}
use apache_avro::{Schema, schema_compatibility::SchemaCompatibility};
fn main() {
let schema_str = r#"\{"type": "string", "logicalType": "uuid"}"#;
let writers_schema = Schema::parse_str(schema_str).unwrap();
let readers_schema = Schema::parse_str(schema_str).unwrap();
if let Err(err) = SchemaCompatibility::can_read(&writers_schema,
&readers_schema) {
eprintln!("Error: {}", err);
}
}
{code}
this is the behavior on `main` as of writing (`876eae32`) and on the latest
version on crates.rs (`v0.16.0`)
I have a proposed fix here:
[https://github.com/apache/avro/compare/main...mknaw:avro:logical-type-compat-fix]
Perhaps there are other valid logicalTypes with this behavior. I have not
checked every logicalType yet.
> [Rust] incorrect compatibility checks with ref fields
> -----------------------------------------------------
>
> Key: AVRO-3974
> URL: https://issues.apache.org/jira/browse/AVRO-3974
> Project: Apache Avro
> Issue Type: Bug
> Components: rust
> Reporter: mknaw
> Priority: Minor
> Labels: pull-request-available
>
> A schema with a field that is a ref to a different schema is incompatible
> with itself, per `SchemaCompatibility`:
> {code:java}
> use apache_avro::{schema_compatibility::SchemaCompatibility, Schema};fn
> main() {
> let schema_strs = vec![
> r#"{
> "type": "record",
> "name": "Child",
> "namespace": "avro",
> "fields": [
> {
> "name": "val",
> "type": "int"
> }
> ]
> }
> "#,
> r#"{
> "type": "record",
> "name": "Parent",
> "namespace": "avro",
> "fields": [
> {
> "name": "child",
> "type": "avro.Child"
> }
> ]
> }
> "#,
> ];
>
> let schemas = Schema::parse_list(&schema_strs).unwrap();
> if let Err(e) = SchemaCompatibility::can_read(
> &schemas[1],
> &schemas[1]
> ) {
> eprintln!("{}", e);
> }
> } {code}
>
> Here it is somewhat ambiguous to me how the fix should look. of course the
> simplest is just to verify the names are the same, and maybe if doing compat
> checks across a whole batch of schemas then check for incompatibilities when
> checking the referenced schemas themselves. But I'm not sure this is quite
> satisfactory - imagine having a complex field that was previously defined
> inline, but now want to reuse it across multiple schemas. In this case, it
> does not seem like replacing the inline definition with a ref should be
> incompatible. So, it seems that some sort of "recursive flattening" of the
> schema prior to compat check would be the ideal fix.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
