alamb commented on code in PR #17861:
URL: https://github.com/apache/datafusion/pull/17861#discussion_r2984351794
##########
docs/source/library-user-guide/upgrading/53.0.0.md:
##########
@@ -472,3 +472,42 @@ Now:
+-------+
0 row(s) fetched.
```
+
+### Avro API, feature, and timestamp decoding changes
+
+As part of the Avro reader migration (see [#17861]), DataFusion now delegates
+Avro-to-Arrow type decoding to `arrow-avro`, and several Avro-related
+APIs/feature wiring changed:
+
+- `DataFusionError::AvroError` has been removed.
+- `From<apache_avro::Error> for DataFusionError` has been removed.
+- Avro crate re-export changed:
+ - Before: `datafusion::apache_avro`
+ - After: `datafusion::arrow_avro`
+- Cargo feature wiring changed:
+ - `datafusion` crate `avro` feature no longer enables
`datafusion-common/avro`
+ - `datafusion-proto` crate `avro` feature no longer enables
`datafusion-common/avro`
+- **Avro timestamp logical type interpretation changed.** Notable effects:
+ - Avro `timestamp-*` logical types are read as UTC timezone-aware Arrow
timestamps
+ (`Timestamp(..., Some("+00:00"))`)
+ - Avro `local-timestamp-*` logical types remain timezone-naive
+ (`Timestamp(..., None)`)
+
+**Who is affected:**
+
+- Users matching on `DataFusionError::AvroError`
+- Users importing `datafusion::apache_avro`
+- Users depending on the old `datafusion-common/avro` feature wiring
+- Users relying on previous Avro timestamp logical type behavior
+
+**Migration guide:**
+
+- Replace `datafusion::apache_avro` imports with `datafusion::arrow_avro`.
+- Update error handling code that matches on `DataFusionError::AvroError` to
use
+ the current error surface.
+- If you depend on Avro feature propagation, update Cargo feature expectations:
+ `datafusion`/`datafusion-proto` `avro` no longer enables
`datafusion-common/avro`.
Review Comment:
As above, I am not sure this is important
##########
Cargo.lock:
##########
@@ -1170,31 +1164,6 @@ dependencies = [
"time",
]
-[[package]]
-name = "bon"
Review Comment:
it is nice to remove some more random dependencies
##########
docs/source/library-user-guide/upgrading/53.0.0.md:
##########
@@ -472,3 +472,42 @@ Now:
+-------+
0 row(s) fetched.
```
+
+### Avro API, feature, and timestamp decoding changes
+
+As part of the Avro reader migration (see [#17861]), DataFusion now delegates
+Avro-to-Arrow type decoding to `arrow-avro`, and several Avro-related
+APIs/feature wiring changed:
Review Comment:
```suggestion
DataFusion has switched to use `arrow-avro` (see [#17861]) when reading avro
files
which results in a few changes:
```
##########
docs/source/library-user-guide/upgrading/53.0.0.md:
##########
@@ -472,3 +472,42 @@ Now:
+-------+
0 row(s) fetched.
```
+
+### Avro API, feature, and timestamp decoding changes
+
+As part of the Avro reader migration (see [#17861]), DataFusion now delegates
+Avro-to-Arrow type decoding to `arrow-avro`, and several Avro-related
+APIs/feature wiring changed:
+
+- `DataFusionError::AvroError` has been removed.
+- `From<apache_avro::Error> for DataFusionError` has been removed.
+- Avro crate re-export changed:
+ - Before: `datafusion::apache_avro`
+ - After: `datafusion::arrow_avro`
+- Cargo feature wiring changed:
+ - `datafusion` crate `avro` feature no longer enables
`datafusion-common/avro`
+ - `datafusion-proto` crate `avro` feature no longer enables
`datafusion-common/avro`
Review Comment:
I don't understand why the feature wiring was called out. I don't think
downstream users of DataFusion need to worry about this during upgrade (it
seems more like an internal implementation detail)
##########
docs/source/library-user-guide/upgrading/53.0.0.md:
##########
@@ -472,3 +472,42 @@ Now:
+-------+
0 row(s) fetched.
```
+
+### Avro API, feature, and timestamp decoding changes
+
+As part of the Avro reader migration (see [#17861]), DataFusion now delegates
+Avro-to-Arrow type decoding to `arrow-avro`, and several Avro-related
+APIs/feature wiring changed:
+
+- `DataFusionError::AvroError` has been removed.
+- `From<apache_avro::Error> for DataFusionError` has been removed.
+- Avro crate re-export changed:
+ - Before: `datafusion::apache_avro`
+ - After: `datafusion::arrow_avro`
+- Cargo feature wiring changed:
+ - `datafusion` crate `avro` feature no longer enables
`datafusion-common/avro`
+ - `datafusion-proto` crate `avro` feature no longer enables
`datafusion-common/avro`
+- **Avro timestamp logical type interpretation changed.** Notable effects:
+ - Avro `timestamp-*` logical types are read as UTC timezone-aware Arrow
timestamps
+ (`Timestamp(..., Some("+00:00"))`)
+ - Avro `local-timestamp-*` logical types remain timezone-naive
+ (`Timestamp(..., None)`)
+
+**Who is affected:**
+
+- Users matching on `DataFusionError::AvroError`
+- Users importing `datafusion::apache_avro`
+- Users depending on the old `datafusion-common/avro` feature wiring
Review Comment:
likewise this is likely to be small
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
