This may be related to another issue I am currently facing while working on
avro_derive. Right now, schema parsing will replace Schema::Ref 's inside
the json, with the fully qualified definitions, and when printing the
canonical form, it uses the fully qualifies definition:
Parsing this schema:
{
"type":"record",
"name":"TestStruct",
"fields": [
{
"name":"a",
"type":{
"type":"record",
"name": "Inner",
"fields": [ {
"name":"z",
"type":"int"
}]
}
},
{
"name":"b",
"type":"Inner"
}
]
}
Results in this internal Schema enum representation:
Record {
name: Name { name: "TestStruct", namespace: None, aliases: None },
doc: None,
fields: [
RecordField {
name: "a",
doc: None,
default: None,
schema: Record {
name: Name { name: "Inner", namespace: None, aliases: None },
doc: None,
fields: [
RecordField {
name: "z",
doc: None,
default: None,
schema: Int,
order: Ascending,
position: 0
}
],
lookup: {"z": 0}
},
order: Ascending, position: 0 },
RecordField {
name: "b",
doc: None,
default: None,
schema: Record {
name: Name { name: "Inner", namespace: None, aliases: None },
doc: None,
fields: [
RecordField {
name: "z",
doc: None, default: None,
schema: Int,
order: Ascending,
position: 0
}
],
lookup: {"z": 0}
},
order: Ascending, position: 1 }
],
lookup: {"a": 0, "b": 1} }
which results in this cannotical form:
{
"name": "TestStruct",
"type": "record",
"fields": [
{
"name": "a",
"type": {
"name": "Inner",
"type": "record",
"fields": [
{
"name": "z",
"type": "int"
}
]
}
},
{
"name": "b",
"type": {
"name": "Inner",
"type": "record",
"fields": [
{
"name": "z",
"type": "int"
}
]
}
}
]
}
TLDR:
Parsing and printing a schema with Shema::Ref's in the schema definition
results in a printed definition that is logically different than the input.
On Sun, Mar 6, 2022 at 3:14 PM ASF subversion and git services (Jira) <
[email protected]> wrote:
>
> [
> https://issues.apache.org/jira/browse/AVRO-3433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17502017#comment-17502017
> ]
>
> ASF subversion and git services commented on AVRO-3433:
> -------------------------------------------------------
>
> Commit e64fe245707647251e462d126cabaccce0aa9191 in avro's branch
> refs/heads/avro-3433-preserve-schema-ref-in-json from Martin Tzvetanov
> Grigorov
> [ https://gitbox.apache.org/repos/asf?p=avro.git;h=e64fe24 ]
>
> AVRO-3433: Use Name as a key for parsed/resolving/input schemata
>
> This is needed to have the fullname (namespace + '.' + name) for lookups
>
> Signed-off-by: Martin Tzvetanov Grigorov <[email protected]>
>
>
> > Rust: The canonical form should preserve schema references
> > ----------------------------------------------------------
> >
> > Key: AVRO-3433
> > URL: https://issues.apache.org/jira/browse/AVRO-3433
> > Project: Apache Avro
> > Issue Type: Bug
> > Components: rust
> > Affects Versions: 1.12.0
> > Reporter: Martin Tzvetanov Grigorov
> > Assignee: Martin Tzvetanov Grigorov
> > Priority: Major
> > Labels: pull-request-available
> > Time Spent: 10m
> > Remaining Estimate: 0h
> >
> > Reported at [
> https://github.com/flavray/avro-rs/issues/182#issuecomment-1059762821]
> > =================================
> > There still seems to be an issue with {{can't refine}} errors, at least
> in some non-recursive cases. Take the following example:
> > {code:java}
> > fn main() {
> > let schema = r#"
> > {
> > "name": "test.test",
> > "type": "record",
> > "fields": [
> > {
> > "name": "bar",
> > "type": { "name": "test.foo", "type": "record", "fields": [{
> "name": "id", "type": "long" }] }
> > },
> > { "name": "baz", "type": "test.foo" }
> > ]
> > }
> > "#;
> > let schema =
> apache_avro::schema::Schema::parse_str(&schema).unwrap();
> > println!("{}", serde_json::to_string(&schema).unwrap());
> > } {code}
> > This prints the following (the same thing happens if the {{test.foo}}
> definition is in a separate file):
> > {code:java}
> > $ target/release/avro-test | jq
> > {
> > "type": "record",
> > "name": "test.test",
> > "fields": [
> > {
> > "name": "bar",
> > "type": {
> > "type": "record",
> > "name": "test.foo",
> > "fields": [
> > {
> > "name": "id",
> > "type": "long"
> > }
> > ]
> > }
> > },
> > {
> > "name": "baz",
> > "type": {
> > "type": "record",
> > "name": "test.foo",
> > "fields": [
> > {
> > "name": "id",
> > "type": "long"
> > }
> > ]
> > }
> > }
> > ]
> > } {code}
> > Which will cause the Java tooling to fail with the
> {{org.apache.avro.SchemaParseException: Can't redefine: test}} error above.
>
>
>
> --
> This message was sent by Atlassian Jira
> (v8.20.1#820001)
>
