typo in my first sentence: I actually also noticed the reference of filesystem connector in the avro-confluent format doc which is confusing.
On Thu, Oct 6, 2022 at 1:24 PM liuxiangcao <xiangcaohe...@gmail.com> wrote: > Hi Martin, > > Thank you for the reply. I actually also noticed the reference of > avro-confluent.subject in the avro-confluent format doc which is confusing. > > I tried using 'connector' = 'filesystem' and 'format' = 'avro-confluent' > and it would fail with options validation when I tried to insert data into > the table. > > If I do not specify 'url' in the WITH block, it gives > "org.apache.flink.table.api.ValidationException: One or more required > options are missing. > Missing required options are: > url > " > > if I specify 'url', it gives > "org.apache.flink.table.api.ValidationException: Unsupported options found > for 'filesystem'. > Unsupported options: > url" > > I also tried to specify 'value.avro-confluent.url' and fails with > "org.apache.flink.table.api.ValidationException: Unsupported options found > for 'filesystem'. > Unsupported options: > value.avro-confluent.url" > > On Tue, Oct 4, 2022 at 1:30 PM Martijn Visser <martijnvis...@apache.org> > wrote: > >> Hi, >> >> I'm wondering if the documentation is correct, because the documentation >> also mentions the option to use 'avro-confluent.subject' which refers to >> using it with filesystem [1]. >> >> Have you tried to use this already? Did you get an error message? >> >> Best regards, >> >> Martijn >> >> [1] >> >> https://nightlies.apache.org/flink/flink-docs-release-1.15/docs/connectors/table/formats/avro-confluent/#avro-confluent-subject >> >> On Tue, Oct 4, 2022 at 3:18 PM liuxiangcao <xiangcaohe...@gmail.com> >> wrote: >> >> > Hi Flink developer community, >> > >> > According to flink doc, avro-confluent([1]) is only supported for kafka >> sql >> > connector and upsert kafka sql connector. >> > >> > I'm wondering if there is any reason this format is not supported for >> > Filesystem sql connector ([2]) ? >> > >> > We are looking to use FileSystem sink to write to s3 in avro format and >> > would like to keep the sink to be in sync with avro schema registry. Are >> > there any gotchas that we should be aware of in implementing >> avro-confluent >> > format for file system sink? >> > >> > [1] >> > >> > >> https://nightlies.apache.org/flink/flink-docs-release-1.15/docs/connectors/table/formats/avro-confluent/ >> > [2] >> > >> > >> https://nightlies.apache.org/flink/flink-docs-release-1.15/docs/connectors/table/filesystem/ >> > >> > >> > -- >> > Best Wishes & Regards >> > Shawn Xiangcao Liu >> > >> > > > -- > Best Wishes & Regards > Shawn Xiangcao Liu > -- Best Wishes & Regards Shawn Xiangcao Liu
