It should be. I briefly looked at it and it seems possible to use SchemaRegistry for this, although it will need some additional wiring for named schemas.
Regards, Anton On Mon, Aug 13, 2018 at 4:14 PM Reuven Lax <[email protected]> wrote: > Is it possible to extend Beam's SchemaRegistry to do this? > > On Mon, Aug 13, 2018 at 4:06 PM Anton Kedin <[email protected]> wrote: > >> Hi, >> >> I am planning to work on implementing a support for external schema >> providers for Beam SQL and wanted to share a high level idea how I think >> this can work. >> >> *Short Version* >> Implement CREATE FOREIGN SCHEMA statement: >> >> CREATE FOREIGN SCHEMA >> >> TYPE 'bigquery' >> >> LOCATION 'dataset_example' >> >> AS bq; >> >> CREATE FOREIGN SCHEMA >> >> TYPE 'hcatalog' >> >> LOCATION 'hive-server:2341' >> >> AS hive; >> >> SELECT * >> >> FROM >> >> bq.table_example_bq AS bq_table1 >> >> JOIN >> >> hive.table_example_hive AS hive_table1 >> >> ON >> bq_table1.some_field = hive_table1.some_other_field; >> >> *A Bit Longer Version: * >> https://docs.google.com/document/d/1Ilk3OpDxrp3bHNlcnYDoj29tt9bd1E0EXt8i0WytNmQ >> >> Thoughts, ideas? >> >> Regards, >> Anton >> >
