Kriskras99 commented on issue #424:
URL: https://github.com/apache/avro-rs/issues/424#issuecomment-4826697954
Draft release notes:
The Apache Avro Rust SDK 0.22.0 release is a large release with lots of
features, fixes and breaking changes. It is strongly recommended to read the
[Breaking changes](#breaking-changes) and [New features](#new-features)
sections of the changelog.
# What's changed
## Breaking changes
- Rework schema compatibility by @Kriskras99 in #342
- `SchemaCompatiblity::can_read` and `SchemaCompatiblity::mutual_read` now
return a `Compatiblity` type
- Bump MSRV from 1.85 to 1.88 by @Kriskras99 in #342, #348
- `Schema::Duration` now has a `FixedSchema` allowing access to attributes
and `Serialize` and `Deserialize` is implemented for `apache_avro::Duration` by
@jdarais in #382
- `AvroSchema` and `AvroSchemaComponent` are no longer behind the `derive`
feature and have moved from `apache_avro::schema::derive` to
`apache_avro::serde` by @martin-g in #394 and @Kriskras99 in #433
- Change the implementation of `AvroSchemaComponent` for `Uuid` from
`Schema::Uuid(String)` to `Schema::Uuid(Fixed(name:
"org.apache.avro.rust.Uuid", size: 16))` and allow overwriting a field's schema
with the `with` attribute when using `#[derive(AvroSchema)]` by @Kriskras99 in
#397, #558
- When `#[avro(with)]` is specified without an argument, it will call the
`get_schema_in_ctxt(&mut HashSet<Name>, NamespaceRef<'_>) -> Schema` in the
same module as specified by `#[serde(with)]`
- When `#[avro(with = path::to::some_fn)]` is specified with a path, the
function it's pointing to will be called with two parameters `&mut
HashSet<Name>, NamespaceRef<'_>`
- When `#[avro(with = || {})]` is specified with a closure, that closure
is invoked
- The name of the schema for `uuid::Uuid` is `org.apache.avro.rust.uuid`
- The schema for `Uuid` can be changed back to a `Schema::Uuid(String)`
using `#[avro(with = || Schema::Uuid(UuidSchema::String))]` on a field with a
`Uuid` and setting `human_readable` to `true`
- Replace the `From<&str>` implementations for `Name` and `Alias` with
`TryFrom<&str>`, `TryFrom<String>`, and `FromStr` as the conversion is not
fallible by @Kriskras99 in #423
- Rework `SpecificSingleObjectWriter` removing `::with_capacity` and
changing `write_avro_datum_ref` to also take a `&NamesRef` parameter by
@Kriskras99 in #445
- Remove the `default` field from `FixedSchema` as it is not in the
specification by @Kriskras99 in #460
- `AvroSchemaComponent::get_schema_in_ctxt` now takes a `&mut HashSet<Name>`
parameter for the `named_schemas` parameter by @Kriskras99 in #471
- Replace the `Schema::map{,_with_attributes}` and
`Schema::array{,_with_attributes}` with builders allowing the user to set more
fields by @martin-g in #472
- This also adds `Schema::r#enum`, `Schema::fixed` and `Schema::record`
- Stricter schema parsing by @Kriskras99 in #479
- This now rejects schemas that were previously accepted by this library,
they were already rejected by parsers in different languages
- `Alias::name` and `Alias::namespace` now return references by @Kriskras99
in #486
- Remove unused `order` and `position` fields in `RecordField` by
@Kriskras99 in #491
- Rework `Name` to be more performant which requires changing the
`SchemaNameValidator` trait by @Kriskras99 in #493
- The regex returned by `SchemaNameValidator::regex` must have a capture
group named `name` which captures the unqualified name when not overriding the
`SchemaNameValidator::validate` function
- The `SchemaNameValidator::validate` must now return the start byte of
the unqualified name
- Support enums and tuples in `SchemaAwareSerializer`, implement
`SchemaAwareDeserializer` and document the mapping between the Serde and Avro
data models by @Kriskras99 in #512, #558
- The `SchemaAware*` now have specific requirements for what input they
accept, so type definitions might need to be modified to make them work with
the new code (especially for enums and tuples)
- This also makes `human_readable` configurable per reader and writer
instance
- Array and Map serialisation will not write the block size in bytes
unless configured to do so, allowing unbuffered serialisation
- The `AvroSchemaComponent` implementation for `std::time::Duration` has
changed to match the (de)serialisation implementation as defined by Serde
- It is now a `Schema::Record(name: "org.apache.avro.rust.Duration",
fields: ["secs": "org.apache.avro.rust.u64", "nanos": "long"])`
- The `AvroSchemaComponent` implementation for `BigDecimal` is now a
`Schema::String` matching the (de)serialisation implementation as defined by
bigdecimal
- It can be changed back to a `Schema::String` using `#[serde(with =
"apache_avro::serde::bigdecimal"), avro(with)]` on a field with a `BigDecimal`
- Resolve schema(ta) in the builder of `Writer` by @Kriskras99 in #328
- The builder now returns an `AvroResult`
## New features
- Make `DeflateSettings::compression_level` public by @EmilyMatt in #335
- Support `fixed` as a type for `uuid` by @Kriskras99 in #339
- Don't depend on Serde to provide fields in the right order by @Kriskras99
in #351
- It is still recommended to match the field order between the type and
the schema as out-of-order serialisation incurs a performance penalty
- Support the `#[serde(flatten)]` attribute by @Kriskras99 in #359, #448
- This attribute is not supported when using `apache_avro::serde::to_value`
- Use the Serde attributes and check for conflict with the Avro attributes
by @Kriskras99 in #377
- This deprecates the Avro attributes for which the value must always
match the Serde attribute:
- Container attributes:
- `#[avro(name = "..")]` -> `#[serde(rename = "..")]`
- `#[avro(rename_all = "..")]` -> `#[serde(rename_all = "..")]`
- Variant attributes:
- `#[avro(rename = "..")]` -> `#[serde(rename = "..")]`
- Field attributes:
- `#[avro(alias = "..")]` -> `#[serde(alias = "..")]`
- `#[avro(rename = "..")]` -> `#[serde(rename = "..")]`
- `#[avro(skip)]` -> `#[serde(skip)]`
- On nightly compilers, using deprecated attributes will throw a compiler
warning
- Support the `#[serde(transparent)]` attribute on structs by @Kriskras99 in
#398
- Implement `AvroSchemaComponent` for `char`, `u64`, `u128`, `i128`, `()` by
@Kriskras99 in #414, #486, #558
- `char`: `Schema::String`
- `u64`: `Schema::Fixed(name: "org.apache.avro.rust.u64", size: 8)`
- `u128`: `Schema::Fixed(name: "org.apache.avro.rust.u128", size: 16)`
- `i128`: `Schema::Fixed(name: "org.apache.avro.rust.i128", size: 16)`
- `()`: `Schema::Null`
- Add `RecordSchemaBuilder::try_name` which accepts any `TryInto<Name>` and
make `RecordFieldBuilder::name` take any `Into<String>` by @martin-g in #429
- Move and rename the `serde_avro_*` modules to `apache_avro::serde::*`, the
old modules are still available but deprecated and will be removed in a future
release by @Kriskras99 in #430
- `apache_avro::serde_avro_bytes` -> `apache_avro::serde::bytes`
- `apache_avro::serde_avro_bytes_opt` -> `apache_avro::serde::bytes_opt`
- `apache_avro::serde_avro_fixed` -> `apache_avro::serde::fixed`
- `apache_avro::serde_avro_fixed_opt` -> `apache_avro::serde::fixed_opt`
- `apache_avro::serde_avro_slice` -> `apache_avro::serde::slice`
- `apache_avro::serde_avro_slice_opt` -> `apache_avro::serde::slice_opt`
- Add posibility to append values to writer without validating by
@Kriskras99 in #447
- This renames `Writer::append` to `Writer::append_value` although the old
function currently still exists and is deprecated
- This adds `Writer::unvalidated_append_value` and
`Writer::unvalidated_append_value_ref` which skip the validation step
- Rework the documentation by @Kriskras99 in #451
- The introduction to Avro itself was moved to the `documentation::primer`
module
- Using the crate the "Avro" way was moved to the `documentation::generic`
module
- Using the crate the "Serde" way was moved to the `serde` module
- Calculating schema fingerprints was moved to the `Schema::fingerprint`
function
- Custom name validators was moved to the `validator` module
- Custom schema equality was moved to the `schema_equality` module
- Document `#[derive(AvroSchema)]` in the `serde` module
- Suggest the `boolean` type when a schema contains a type named `bool` that
does not resolve by @Kriskras99 in #459
- Add support for resetting a `Writer` backed by a clearable buffer by
@Kriskras99 in #469
- Make `Schema::denormalize` public by @Kriskras99 in #470
- Allow types implementing `AvroSchemaComponent` to provide a default value
by @Kriskras99 in #477
- Custom `Debug` implementations for `*Schema` types hiding empty fields for
more readability by @Kriskras99 in #488
- Add a builder for unions: `UnionSchemaBuilder` by @Kriskras99 in #486
- Replace the `from_avro_datum*` functions with `GenericDatumReader` by
@Kriskras99 in #496
- The functions are still available, but are deprecated and will be
removed in a future version
- Replace the `to_datum*` functions with `GenericDatumWriter` by @Kriskras99
in #499
- The functions are still available, but are deprecated and will be
removed in a future version
- Support `#[serde(remote = "..")]` by @Kriskras99 in #564
- Support deriving `AvroSchema` on enums and tuples by @Kriskras99 in #569
## Fixes
- Choose correct schema when serializing an `Option<Enum>` by @PookieBuns in
#337
- Support deserialization of `BigDecimal` fields when using "the Serde way"
by @eft-prima in #338
- Don't write the `avro.codec` attribute if the codec is Null by @Kriskras99
in #356
- Resolving a `Value::Long` into a `Value::Int` no longer truncates and
returns an error instead by @Kriskras99 in #392
- `logicalType` is no longer included in custom attributes by @jdarais in
#395
- Do not match types by name in the derive code by @Kriskras99 in #401
- Calculate the `RecordSchema` lookup table on
`RecordSchemaBuilder::build()` by @martin-g in #411
- `UnionSchema::new` no longer accepts multiple schemas with the same name
by @marting-g in #413
- Support recursive types for `Schema::independent_canonical_form` by
@Kriskras99 in #420
- Select the first union variant with the correct byte length when
serializing by @Kriskras99 in #421
- Use RAII for setting thread locals used during (de)serialisation so
failure doesn't impact the next (de)serialisation by @martin-g in #432
- Ensure `SpecificSingleObjectWriter` writes header on every call by @flxo
in #438
- Don't allow serializing `bytes` as a `Uuid::String` by @Kriskras99 in #441
- Don't allow resolving a schema that is already known by @martin-g in #444
- Don't silently truncate numbers larger than `i64::MAX` when converting
from JSON by @Kriskras99 in #450
- Fix links in rustdoc by @martin-g in #505
- Remove unused `CHANGELOG.md` by @martin-g in #541
- Support `BigDecimal` in `Deserializer` when
`apache_avro::serde::bigdecimal` is used by @jdarais in #543
- Return an error instead of panicking on invalid name, alias, or type
reference by @youichi-uda in #547
## Internal changes
- Simplify reader tests by @Kriskras99 in #355
- Fix the `test_overflow` test to actually test for overflow by @Kriskras99
in #357
- Uncomment old test by @Kriskras99 in #362
- Enable the `std` feature for Serde by @Kriskras99 in #364
- This feature was already transitively enabled by dependencies on the
`avro` crate but not when building only the `avro_derive` crate
- Configure Clippy in the workspace instead of the CI by @Kriskras99 in #369
- Remove unnecessary mod in `avro_derive/tests/derive.rs` by @Kriskras99 in
#375
- Add CI check for license headers in all non-generated files by @martin-g
in #387
- Debug print the error in `TestResult` by @martin-g in #393
- Split `schema.rs` in various modules by @martin-g in #405, #409, #410,
#412, #415, #417
- Simplify `Deserializer` by @Kriskras99 in #416
- Disable the test logger helper on wasm32 by @martin-g in #426
- Remove anyhow as a dev-dependency by @martin-g in #427
- Simplify `Serializer` by @Kriskras99 in #428
- Restructure resolving logic by @Kriskras99 in #442
- Remove a benchmark for serde_json by @martin-g in #446
- Split `reader.rs` into various modules by @martin-g in #474, #475, #478
- Small code improvements by @Kriskras99 in #336, #486, #513, #559
- Refactor derive code by @Kriskras99 in #487, #560, #561
- Split `writer.rs` into various modules by @Kriskras99 in #497, #498
- Use macrotest to test the `AvroSchema` derive generated code by
@Kriskras99 in #501
- Run tests on fewer targets by @Kriskras99 in #503
- Add CI step for `cargo doc` by @martin-g in #507
- Apply some suggestion from `-Dclippy::pedantic` by @martin-g in #514
- Do not use Dependabot to update Github Actions by @martin-g in #525
- Enabling updating PR branches from Github UI by @martin-g in #527
- Optimize `Alias::new()` and `Parser::fix_aliases_namespace` by @Kriskras99
in #557
- Fix clippy lints from Rust 1.90.1 by @martin-g in #330
## Dependency bumps
- Bump actions/checkout from 5 to 6
- Bump actions/setup-java from 5.0.0 to 5.2.0
- Bump bigdecimal from 0.4.9 to 0.4.10
- Bump bon from 3.8.1 to 3.9.3
- Bump criterion from 0.7.0 to 0.8.2
- Bump ctor from 0.6.0 to 0.11.1
- Bump darling from 0.21.3 to 0.23.0
- Bump digest from 0.11.2 to 0.11.3
- Bump env_logger from 0.11.8 to 0.11.11
- Bump hex-literal from 1.0.0 to 1.1.0
- Bump liblzma from 0.4.5 to 0.4.6
- Bump log from 0.4.28 to 0.4.33
- Bump miniz_oxide from 0.8.9 to 0.9.1
- Bump proc-macro2 from 1.0.103 to 1.0.106
- Bump proptest from 1.9.0 to 1.11.0
- Bump quote from 1.0.41 to 1.0.46
- Bump rand from 0.9.2 to 0.10.1
- Bump regex-lite from 0.1.8 to 0.1.9
- Bump serde_json from from 1.0.145 to 1.0.150
- Bump sha2 from 0.10.9 to 0.11.0
- Bump snap from 1.1.0 to 1.1.1, darling from 0.21.3 to 0.23.0, wasm-bindgen
from 0.2.97 to 0.2.114 by @martin-g in #495
- Bump strum from 0.27.2 to 0.28.0 and replace strum-macros with `features =
["derive"]` on strum by @Kriskras99 in #483
- Bump syn from 2.0.108 to 2.0.118
- Bump thiserror from 2.0.17 to 2.0.18
- Bump trybuild from 1.0.114 to 1.0.117
- Bump uuid from 1.18.1 to 1.23.4
- Bump wasm-bindgen-test from 0.3.55 to 0.3.64
- Pin dtolnay/rust-toolchain to 29eef336d9b2848a0b548edc03f92a220660cdb8
(stable tag at that time) by @martin-g in #526
- This is because Github Actions dependencies need to be approved by the
Apache team
- Pin cargo-rdme to 1.4.2 by @martin-g in #567
- This is because cargo-rdme 2 depends on specific nightly versions and
therefore easily breaks in CI
# New contributors
- @emilymatt made their first contribution in #335
- @PookieBuns made their first contribution in #337
- @eft-prima made their first contribution in #338
- @flxo made their first contribution in #438
- @youichi-uda made their first contribution in #548
**Full Changelog**:
[rel/release-0.21.0..rel/release-0.22.0](https://github.com/apache/avro-rs/compare/rel/release-0.21.0...rel/release-0.22.0)
Note: this includes #569 which is not yet merged
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
