Re: How do you branch your code with BigQuery?

2018-10-15 Thread Anthony Brown
I do intend to create a PR (when I get the chance) to get this into main airflow repo If anybody has any comments about this before I do, please let me know On Mon, 15 Oct 2018 at 10:33, airflowuser wrote: > Awesome! > I think this would be a fine addition to the BigQuery operators if you >

Re: How do you branch your code with BigQuery?

2018-10-15 Thread airflowuser
Awesome! I think this would be a fine addition to the BigQuery operators if you ever think about PR this to airflow master cheers Sent with ProtonMail Secure Email. ‐‐‐ Original Message ‐‐‐ On Monday, October 15, 2018 10:02 AM, Anthony Brown wrote: > Hi > I have created a custom

Re: How do you branch your code with BigQuery?

2018-10-15 Thread Anthony Brown
Hi I have created a custom plugin that allows you to branch on the results of a BigQuery query. The code for it is at https://github.com/JohnLewisandPartners/custom-airflow-plugins/blob/master/bq_branch/plugins/bq_branch.py. The version in master only works on airflow 1.10, but there is a

How do you branch your code with BigQuery?

2018-10-14 Thread airflowuser
I believe this is quite common case when working with data. If something : do A else: do B With coding PythonBranchOperator is the solution. But when working on Google Cloud there is no way to do this. All existed operators are designed to continue or fail on comparison of specific value: