Vancior opened a new pull request, #20040:
URL: https://github.com/apache/flink/pull/20040
## What is the purpose of the change
This PR add Avro GenericRecord serialization at Java side and
deserialization at Python side, which is useful when reading parquet files with
a Avro schema into python native object records.
## Brief change log
- add LengthPrefixWrapperSerializer to wrap Avro records into length-prefix
bytes
- add avro serializer creation logic using dynamic class loading without
introducing new dependency
- add avro type info in flink-fn-api
- add FlinkAvroDecoder and FlinkDatumReader at Python side to match
customized serialization of Avro records at Java sdie
## Verifying this change
Tests will be added in another PR, which supports parquet-avro format for
PyFlink and solves shading issue.
## Does this pull request potentially affect one of the following parts:
- Dependencies (does it add or upgrade a dependency): (no)
- The public API, i.e., is any changed class annotated with
`@Public(Evolving)`: (no)
- The serializers: (no)
- The runtime per-record code paths (performance sensitive): (no)
- Anything that affects deployment or recovery: JobManager (and its
components), Checkpointing, Kubernetes/Yarn, ZooKeeper: (no)
- The S3 file system connector: (no)
## Documentation
- Does this pull request introduce a new feature? (yes)
- If yes, how is the feature documented? (not applicable)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
