Hi,
I am planning to work on implementing a support for external schema
providers for Beam SQL and wanted to share a high level idea how I think
this can work.
*Short Version*
Implement CREATE FOREIGN SCHEMA statement:
CREATE FOREIGN SCHEMA
TYPE 'bigquery'
LOCATION 'dataset_example'
AS bq;
CREATE FOREIGN SCHEMA
TYPE 'hcatalog'
LOCATION 'hive-server:2341'
AS hive;
SELECT *
FROM
bq.table_example_bq AS bq_table1
JOIN
hive.table_example_hive AS hive_table1
ON
bq_table1.some_field = hive_table1.some_other_field;
*A Bit Longer Version: *
https://docs.google.com/document/d/1Ilk3OpDxrp3bHNlcnYDoj29tt9bd1E0EXt8i0WytNmQ
Thoughts, ideas?
Regards,
Anton