[
https://issues.apache.org/jira/browse/AVRO-3240?focusedWorklogId=668632&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-668632
]
ASF GitHub Bot logged work on AVRO-3240:
----------------------------------------
Author: ASF GitHub Bot
Created on: 21/Oct/21 20:37
Start Date: 21/Oct/21 20:37
Worklog Time Spent: 10m
Work Description: ultrabug opened a new pull request #1379:
URL: https://github.com/apache/avro/pull/1379
When providing your own schema to `from_avro_datum`,
the deserialization is not backward compatible with messages
containing a previous schema even if the schemas are created
to be backward compatible.
This is due to the `decode_variable` function in `utils` returning
`Error::ReadVariableIntegerBytes` when the reader object is
smaller than expected and thus cannot fill the read_exact buffer.
Reading a message generated with an older schema version than the
one we are statically using, the payload buffer is by essence
smaller than the expected payload from a message serialized with
a newer schema (backward compatible schemas are basically append
only).
The proposed fix takes that into account and just returns empty
Ok() which will be interpreted as null. This is in line with the
fact that a backward compatible schema has to allow for null
values to the newly added fields.
Make sure you have checked _all_ steps below.
### Jira
- [x] [AVRO-3240](https://issues.apache.org/jira/browse/AVRO-3240)
### Tests
- [ ] I'd like comments before working on tests
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 668632)
Remaining Estimate: 0h
Time Spent: 10m
> Schema deserialization is not backwards compatible
> --------------------------------------------------
>
> Key: AVRO-3240
> URL: https://issues.apache.org/jira/browse/AVRO-3240
> Project: Apache Avro
> Issue Type: Bug
> Components: rust
> Reporter: Ultrabug
> Priority: Critical
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Hello,
>
> When providing your own schema to `from_avro_datum`, the deserialization is
> not backward compatible with messages containing a previous schema even if
> the schemas are created to be backward compatible.
> This is due to the `decode_variable` function in `utils` returning
> `Error::ReadVariableIntegerBytes` when the reader object is smaller than
> expected when reading a message with previous schema version (which is indeed
> smaller) and trying to decode it with a backward compatible schema.
> I have fixed the issue locally with the following patch which I will submit
> as a PR
>
> ```
> diff --git a/lang/rust/src/util.rs b/lang/rust/src/util.rs
> index f9daf285..3538399e 100644
> --- a/lang/rust/src/util.rs
> +++ b/lang/rust/src/util.rs
> @@ -96,7 +96,7 @@ fn decode_variable<R: Read>(reader: &mut R) ->
> AvroResult<u64> {
> }
> reader
> .read_exact(&mut buf[..])
> - .map_err(Error::ReadVariableIntegerBytes)?;
> + .or(Ok(()))?; // return nullable to support avro schemas backward
> compatibility
> i |= (u64::from(buf[0] & 0x7F)) << (j * 7);
> if (buf[0] >> 7) == 0 {
> break;
> ```
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
