Hi all, I'd like to discuss a SQL Lab-first implementation of SIP-180, building on the existing estimator and Beto's earlier draft.
SIP: https://github.com/apache/superset/issues/34384 Earlier draft: https://github.com/apache/superset/pull/34385 The proposal is an optional estimate check before SQL Lab runs a query. Above a configured threshold, Superset would show the estimate and offer Edit, Cancel or Run anyway. These are advisory warnings; ordinary permissions and database-native limits still apply. I'd add a separate warning feature flag, disabled by default. An administrator would also need to enable warnings and set a threshold for each supported connection. Existing connections would not automatically receive warnings. Warnings would require estimation to be available. If the existing estimation flag is off, the warning settings would be inactive with an explanation. The connection controls would sit under Edit connection > Advanced > SQL Lab, below "Enable query cost estimation". I'd store one validated warning rule in Database.extra. The Run workflow would: - Capture the selected query and its execution settings. - Estimate it and compare the result against the saved threshold before display rounding. - Proceed automatically when a usable estimate is within the threshold. - Above the threshold, wait for the user's decision. - Discard the decision if relevant inputs change. Cancel would submit no execution request. For a recognized timeout or unusable estimate, I'd explain that usage is unknown and offer an explicit Run without estimate choice. Identified input, permission and native-limit errors would remain errors. Unexpected failures would show an error rather than silently execute. I'd initially cover deliberate Run, shortcuts and rerun for single read-only queries, including synchronous and asynchronous execution. Dashboard checks, scheduled execution, automatic retries, templates and multi-statement estimation would be deferred. I'd start implementation with BigQuery processed bytes and test the shared design with PostgreSQL planner cost early. Each engine would retain its own units and limitations. Whether PostgreSQL thresholds provide useful warnings would be part of that validation. I'd keep the manual estimate button and API backward compatible. Validation would cover matching query settings, threshold boundaries, cancellation, changed inputs, failures and added latency. I'd appreciate feedback on three choices: 1. Is this SQL Lab-first scope appropriate for a small core integration? 2. Does a separate off-by-default flag plus connection-level opt-in make sense? 3. Should a recognized unavailable estimate allow explicit execution without an estimate? Thanks, Alankrit
