GitHub user Pablo-aps added a comment to the discussion: Pre-flight SQL query 
syntax validation and partition guard in Common SQL provider

I'd start with a `SQLExecuteQueryOperator` subclass. At the start of 
`execute()` you have the rendered SQL, and you can reject it before calling 
`super().execute(context)`.

Here's the pair of cases I'd test first on a table partitioned by `event_date`:

```sql
WHERE event_date = DATE '2026-11-01' AND status = 'retry'
WHERE event_date = DATE '2026-11-01' OR  status = 'retry'
```

Both parse. Both mention the partition column. The second one still needs 
retries from every date. BigQuery rejects that form when 
`require_partition_filter` is enabled. [BigQuery's partition filter 
examples](https://docs.cloud.google.com/bigquery/docs/querying-partitioned-tables#require_a_partition_filter_in_queries).

That makes the local check useful for catching a missing filter or a typo. For 
an actual spending limit on BigQuery on-demand, I'd set `maximumBytesBilled` on 
the job too. Even the first query can scan a lot if one day's partition is 
large. [Query cost 
limits](https://docs.cloud.google.com/bigquery/docs/best-practices-costs).

GitHub link: 
https://github.com/apache/airflow/discussions/76373#discussioncomment-19632267

----
This is an automatically sent email for [email protected].
To unsubscribe, please send an email to: [email protected]

Reply via email to