dhruv-techdev opened a new pull request, #45789: URL: https://github.com/apache/superset/pull/45789
### SUMMARY This adds a CI test that runs the Hive Week time grain expression on a real Hive 2.3.2, so a regression of #45434 now fails the build. It is a follow-up to #45435 and implements option 3 from @sadpandajoe's review there. **Why it's needed** - The existing coverage for the fix evaluates the expression on SQLite with Python stand-ins for the Hive functions, so it can't catch a difference in how Hive itself behaves. - The `test-postgres-hive` job only selects `chart_data_flow` and `sql_json_flow`, and the chart-data tests skip on Hive. Nothing in CI ran a time grain expression on Hive. **What it does** - `test_week_time_grain_buckets_to_monday_on_hive` in `tests/integration_tests/db_engine_specs/hive_tests.py` runs `HiveEngineSpec`'s actual `WEEK` expression on the Hive examples database over 14 literal days (2025-01-06 to 2025-01-19). It checks that every day maps to the Monday of its week. - The test is marked `hive_engine_spec`, and the `test-postgres-hive` job now selects that marker. It runs against Hive 2.3.2 from `scripts/databases/hive/docker-compose.yml`. **Design decisions** - The test reads the expression from `HiveEngineSpec._time_grain_expressions`, so it always checks the shipped SQL, never a copy. - The dates come from `explode(array(...))`, so no table is created and nothing is written to HDFS. - It skips when the examples database isn't Hive, because the MySQL, Postgres and SQLite jobs run all integration tests without a marker filter. - The marker isn't added to `pytest.ini`, following the existing `chart_data_flow` and `sql_json_flow` markers. ### BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF N/A. This change only touches a test and the CI workflow. ### TESTING INSTRUCTIONS **In CI:** the `test-postgres-hive` job log should show: ``` tests/integration_tests/db_engine_specs/hive_tests.py::test_week_time_grain_buckets_to_monday_on_hive PASSED ``` In the MySQL, Postgres and SQLite integration jobs, the test should show as `SKIPPED`. **Locally:** ```bash mkdir -p /tmp/superset_uploads export UPLOAD_FOLDER=/tmp/superset_uploads/ docker compose -f scripts/databases/hive/docker-compose.yml up -d export SUPERSET_CONFIG=tests.integration_tests.superset_test_config export SUPERSET_TESTENV=true export SUPERSET__SQLALCHEMY_EXAMPLES_URI=hive://localhost:10000/default superset db upgrade && superset init && superset load-test-users pytest tests/integration_tests/db_engine_specs/hive_tests.py -m hive_engine_spec -v ``` Results on Hive 2.3.2 (2.3.2 r857a9fd8ad725a53bd95c1b2d6612f9b1155f44d): | Setup | Result | |---|---| | Expression on `master` | `1 passed` | | Previous expression (`7-u`) swapped in | `1 failed`, e.g. `2025-01-07 → 2025-01-02` and `2025-01-15 → 2025-01-11` | | Examples DB not Hive | `1 skipped` (`Requires a Hive examples database`) | `pre-commit run` passes on both changed files (ruff, mypy, check-yaml, zizmor). **Out of scope:** CI runs Hive 2.3.2 only. How newer Hive versions handle `'u'`, the caveat raised in #45435, will be checked separately. ### ADDITIONAL INFORMATION - [x] Has associated issue: Related to #45434, follow-up to #45435 - [ ] Required feature flags: - [ ] Changes UI - [ ] Includes DB Migration (follow approval process in [SIP-59](https://github.com/apache/superset/issues/13351)) - [ ] Migration is atomic, supports rollback & is backwards-compatible - [ ] Confirm DB migration upgrade and downgrade tested - [ ] Runtime estimates and downtime expectations provided - [ ] Introduces new feature or API - [ ] Removes existing feature or API -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
