awesome proposal to integrate with existing external schemas, add some comments in doc.
On Mon, Aug 13, 2018 at 4:13 PM, Reuven Lax <[email protected]> wrote: > Is it possible to extend Beam's SchemaRegistry to do this? > > On Mon, Aug 13, 2018 at 4:06 PM Anton Kedin <[email protected]> wrote: > >> Hi, >> >> I am planning to work on implementing a support for external schema >> providers for Beam SQL and wanted to share a high level idea how I think >> this can work. >> >> *Short Version* >> Implement CREATE FOREIGN SCHEMA statement: >> >> CREATE FOREIGN SCHEMA >> >> TYPE 'bigquery' >> >> LOCATION 'dataset_example' >> >> AS bq; >> >> CREATE FOREIGN SCHEMA >> >> TYPE 'hcatalog' >> >> LOCATION 'hive-server:2341' >> >> AS hive; >> >> SELECT * >> >> FROM >> >> bq.table_example_bq AS bq_table1 >> >> JOIN >> >> hive.table_example_hive AS hive_table1 >> >> ON >> bq_table1.some_field = hive_table1.some_other_field; >> >> *A Bit Longer Version: *https://docs.google.com/document/d/ >> 1Ilk3OpDxrp3bHNlcnYDoj29tt9bd1E0EXt8i0WytNmQ >> >> Thoughts, ideas? >> >> Regards, >> Anton >> > -- ---- Mingmin
