mikebridge opened a new pull request, #45395: URL: https://github.com/apache/superset/pull/45395
### SUMMARY **TL;DR:** Provider authors get an opt-in pytest conformance suite that runs without a Superset application and checks observable results and failure categories. **Background:** The semantic-layer SDK defines common query and metadata contracts, while each provider owns construction, transport and backend fixtures. **Problem:** A provider can implement the right methods yet return the wrong grain, filter result, ordering or error category. Shared behavioral checks make those regressions easier to detect before integration. **What this changes:** - Add `SemanticProviderConformance`, with provider-owned fixtures for deterministic queries, controlled errors and catalog changes. - Check selection identity, grains, NULL/OR filtering, explicit ordering before limits, advertised capabilities, refusal/fault/completeness boundaries, metadata generations and unbound discovery. - Check provider imports and static configuration schemas in a fresh interpreter, rejecting host imports and common schema I/O operations. - Run the suite against an SDK-only SQLite example and document how providers supply independent expected results. Depends on https://github.com/apache/superset/pull/45393. The conformance layer diff is available in the [fork comparison](https://github.com/mikebridge/superset/compare/sc-127358-sdk-provider-obligations...sc-127359-sdk-conformance-kit). The suite is optional test support; importing the SDK normally does not load pytest. Factory signatures, context types, production transports, host execution, and count/value success conventions are unchanged. Default chronological ordering remains a separate change in #45389. Provider integration lanes must use their pinned SDK and retain host and live-provider checks. ### BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF Not applicable: SDK test support and documentation only. ### TESTING INSTRUCTIONS With the SDK dependencies and pytest installed, run from the repository root: ```bash PYTEST_DISABLE_PLUGIN_AUTOLOAD=1 PYTHONPATH=$PWD/superset-core/src python -m pytest -q -o addopts='' superset-core/tests/semantic_layers/test_conformance.py superset-core/tests/semantic_layers/test_registration.py superset-core/tests/semantic_layers/test_view.py ``` The example uses in-process SQLite, no server or credentials. Expected Arrow tables are independently specified. Controlled mutations were detected for NULL/OR behavior, grain handling, ordering, partial success, fault classification, stale catalogs and token relabeling. Negative import/schema tests also verify that catching a tripwire exception cannot make the check pass. The import/schema tripwires are not a sandbox or exhaustive proof against native-driver I/O. Provider lanes must also poison their own transport and credential entry points. Passing these checks does not establish deployed host or live-warehouse conformance. ### ADDITIONAL INFORMATION - [ ] Has associated issue: - [ ] Required feature flags: - [ ] Changes UI - [ ] Includes DB Migration (follow approval process in [SIP-59](https://github.com/apache/superset/issues/13351)) - [ ] Migration is atomic, supports rollback & is backwards-compatible - [ ] Confirm DB migration upgrade and downgrade tested - [ ] Runtime estimates and downtime expectations provided - [x] Introduces new feature or API - [ ] Removes existing feature or API -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
