Hi Andreas, here is the PR for adding the support to Iceberg https://github.com/apache/iceberg/pull/6557
On Fri, Apr 1, 2022 at 10:23 AM Hailu, Andreas <andreas.ha...@gs.com> wrote: > I found it – it was part of the versioned iceberg-flink modules, > iceberg-flink-1.13. > > > > ah > > > > *From:* Hailu, Andreas [Engineering] > *Sent:* Friday, April 1, 2022 10:37 AM > *To:* dev@iceberg.apache.org > *Subject:* RE: Flink GenericRecord Iceberg Usage > > > > Thanks, Steven, I’ll try this out! > > > > I’m having a spot of trouble locating FlinkSchemaUtil – which Iceberg > module is that from? I thought it belonged to iceberg-flink, but seems > 0.13.1 doesn’t contain it. > > > > ah > > > > *From:* Steven Wu <stevenz...@gmail.com> > *Sent:* Thursday, March 31, 2022 10:42 PM > *To:* dev@iceberg.apache.org > *Subject:* Re: Flink GenericRecord Iceberg Usage > > > > funny that I was just working on this recently. You can plugin this mapper > to the FlinkSink builder > > https://gist.github.com/stevenzwu/4b824556973b47178824852083ab7a50 > <https://urldefense.proofpoint.com/v2/url?u=https-3A__gist.github.com_stevenzwu_4b824556973b47178824852083ab7a50&d=DwMFaQ&c=7563p3e2zaQw0AB1wrFVgyagb2IE5rTZOYPxLxfZlX4&r=hRr4SA7BtUvKoMBP6VDhfisy2OJ1ZAzai-pcCC6TFXM&m=hg3GJeOj4I2ktkldMegIhMKlQw6J0ZIchFOXRjWJjXw&s=HUI4x5nvdxCJG7gTwvUyn6hwFsNHEbvg6Wr4hAvXcws&e=> > > > > RowType rowType = FlinkSchemaUtil.*convert*(icebergSchema); > FlinkSink.*builderFor*( > dataStream, > AvroGenericRecordToRowDataMapper.*forAvroSchema*(avroSchema), > FlinkCompatibilityUtil.*toTypeInfo*(rowType)) > .table(table) > .tableLoader(tableLoader) > .writeParallelism(parallelism) > .append(); > > > > On Thu, Mar 31, 2022 at 11:32 AM Hailu, Andreas <andreas.ha...@gs.com> > wrote: > > Hello, > > > > I’m looking to write a proof of concept incorporating Iceberg into our ETL > application. We use Avro as our schema management solution, and the Iceberg > Flink documentation examples only show writing DataStream<Row> and > DataStream<RowData> types. > > > > How can this be applied to for DataStream<GenericRecord>? Is there a > translation step/utility? I found a discussion on an issue from 2020 [2] > regarding the same thing, but it’s unclear of a solution was reached. > > > > [1] https://iceberg.apache.org/docs/latest/flink/ > <https://urldefense.proofpoint.com/v2/url?u=https-3A__iceberg.apache.org_docs_latest_flink_&d=DwMFaQ&c=7563p3e2zaQw0AB1wrFVgyagb2IE5rTZOYPxLxfZlX4&r=hRr4SA7BtUvKoMBP6VDhfisy2OJ1ZAzai-pcCC6TFXM&m=hg3GJeOj4I2ktkldMegIhMKlQw6J0ZIchFOXRjWJjXw&s=bCLuKffWSJGedPeqtEfLZe1UJZWw9BhQITqttcQdhFc&e=> > > [2] https://github.com/apache/iceberg/issues/1885 > <https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_apache_iceberg_issues_1885&d=DwMFaQ&c=7563p3e2zaQw0AB1wrFVgyagb2IE5rTZOYPxLxfZlX4&r=hRr4SA7BtUvKoMBP6VDhfisy2OJ1ZAzai-pcCC6TFXM&m=hg3GJeOj4I2ktkldMegIhMKlQw6J0ZIchFOXRjWJjXw&s=v3mVfawvHkDP645gX2lgLv9JmuNB4fGso-j3Wm91nRA&e=> > > > > > > best, > > ah > > > > > ------------------------------ > > > Your Personal Data: We may collect and process information about you that > may be subject to data protection laws. For more information about how we > use and disclose your personal data, how we protect your information, our > legal basis to use your information, your rights and who you can contact, > please refer to: www.gs.com/privacy-notices > > > ------------------------------ > > Your Personal Data: We may collect and process information about you that > may be subject to data protection laws. For more information about how we > use and disclose your personal data, how we protect your information, our > legal basis to use your information, your rights and who you can contact, > please refer to: www.gs.com/privacy-notices >
