Hi Kevin, I'm not as familiar with initiatives around the new sources, but it seems that the BulkFormat for Avro [1] has been added recently and will be released with the Flink 1.15.x.
[1] https://issues.apache.org/jira/browse/FLINK-24565 Best, D. On Fri, Jan 7, 2022 at 7:23 PM Kevin Lam <kevin....@shopify.com> wrote: > Hi all, > > We're looking into using the new FileSource > <https://nightlies.apache.org/flink/flink-docs-release-1.14/api/java/org/apache/flink/connector/file/src/FileSource.html> > API, we see that there is a BulkFormat > <https://nightlies.apache.org/flink/flink-docs-release-1.14/api/java/org/apache/flink/connector/file/src/reader/BulkFormat.html> > for Parquet, via ParquetColumnarRowFormat > <https://nightlies.apache.org/flink/flink-docs-release-1.14/api/java/org/apache/flink/formats/parquet/ParquetColumnarRowInputFormat.html>. > Is there similar BulkFormat available or in the works for Avro files? > > I imagined it may be a common use-case in the community so wanted to check > here before we invest time implementing our own. > > Thanks in advance! >
