sadpandajoe opened a new pull request, #43173:
URL: https://github.com/apache/superset/pull/43173
## SUMMARY
The dataset "Hours offset" setting had two independent bugs:
1. **Filter bounds could admit or drop a day.** For a DATE-typed column, the
filter bound was shifted by the raw Hours Offset before being rendered as
a
date-only literal. Any sub-24h offset could move which calendar day a
bound
truncated to -- silently admitting an out-of-range day on one end while
dropping the last requested day on the other. TIMESTAMP-typed columns were
unaffected (they keep exact-hour precision).
2. **Displayed labels could disagree with the filter bounds.** The offset was
applied only in pandas, after the database had already truncated a
temporal
axis to its time grain using the raw (unshifted) value. A row near a grain
boundary could bucket to the wrong day, so the label shown to the user
could
disagree with which rows the filter actually selected.
## FIX
1. The offset is now quantized to whole days (truncating toward zero) before
being applied to a DATE column's filter bounds, so a sub-day offset never
moves the selected day window. This part is engine-agnostic.
2. The offset is now applied in SQL, before grain truncation, at the
axis-construction call sites -- gated behind a new `apply_dataset_offset`
opt-in parameter so filter construction and other callers are unaffected.
A new `sql_shifted_temporal_labels` signal tells dataframe normalization
which columns were already shifted in SQL, so the offset isn't re-applied
on top in pandas.
Since the SQL-side shift depends on dialect-specific interval syntax, it's
gated behind a new `BaseEngineSpec.supports_temporal_column_shift`
capability flag (default `False`), enabled explicitly on the two engines
it's verified against (PostgreSQL, SQLite). Engines that don't opt in keep
the prior pandas-only behavior for bug (2) -- no new regression -- while
unconditionally gaining the bound-quantization fix for bug (1), which is
engine-agnostic.
## TESTING
Added `tests/unit_tests/models/test_hours_offset_bound_truncation.py`,
covering
both bugs across an offset x grain x column-type matrix: DATE-column filter
bounds across offsets 0/+1/+24/+25/-1/-25 (with a TIMESTAMP control), grained
physical and adhoc BASE_AXIS temporal axes applying the offset exactly once
before truncation, negative sub-day DATE offsets that must not move the
bucket, byte-identical SQL for the explicitly out-of-scope grained-filter
paths, and the new per-engine capability gate (including its pandas-fallback
path for unsupported engines).
## ADDITIONAL SCOPE
Grained temporal *filters* combined with a shifted grained *axis* are out of
scope for this change (the filter path is unchanged) and may need a
follow-up.
DST handling and the `extra.timezone` override path are also unchanged.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
