swapna267 opened a new pull request, #16450: URL: https://github.com/apache/iceberg/pull/16450
Resurrecting the closed PR, https://github.com/apache/iceberg/pull/15471 (Was Stale). Re-adding the context here, Added, **VariantAvroDynamicTableRecordGenerator**: Custom Dynamic table record generator designed for deserializing the Variant with the schema provided and writing to Iceberg Tables matching with avro schema passed in SQL. **VariantRowDataWrapper**: Wrapper class for variant row data that provides type-safe conversion and handling of variant data structures as RowData. Changes allow to ingest data in Flink SQL using dynamic iceberg sink. ``` CREATE TABLE test_dynamic ( data VARIANT, -- JSON payload `catalog-database` STRING, -- Target database `catalog-table` STRING, -- Target table avro_schema STRING, -- Schema definition avro_schema_id STRING, -- Schema versioning ,mainly used for caching instead of relying on full schema comparison. branch STRING, -- Iceberg branch `write-parallelism` INT -- Performance tuning ) WITH ('use-dynamic-iceberg-sink'='true', ...) ``` ``` INSERT INTO test_dynamic VALUES (PARSE_JSON('{"id": 1, "name": "AAA"}'), 'db', 'table', '<avro-schema>', 'TestSchema:1', 'main', 1) ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
