sadpandajoe commented on code in PR #43694:
URL: https://github.com/apache/superset/pull/43694#discussion_r3982621108
##########
superset/utils/pandas_postprocessing/pivot.py:
##########
@@ -190,6 +190,33 @@ def _restore_dropped_metric_columns(
return df
+def _fill_dimension_column(df: DataFrame, col: str, fill_value: str) -> None:
+ """Fill missing values in a groupby dimension column before pivoting.
+
+ Handles categorical dtypes (adding fill_value to categories) and datetime
+ dtypes (converting to string representation with fill_value for NaT) to
prevent
+ dtype errors and preserve NULL/NaN/NaT keys through pivot_table().
+ """
+ s = df[col]
+ if isinstance(s.dtype, pd.CategoricalDtype) and fill_value not in
s.cat.categories:
+ df[col] = s.cat.add_categories([fill_value]).fillna(value=fill_value)
+ elif (
+ pd.api.types.is_datetime64_any_dtype(s.dtype)
+ or getattr(s.dtype, "kind", None) == "M"
+ ):
+ if s.isna().any():
+ df[col] = s.astype(str).replace(
+ {
+ "NaT": fill_value,
+ "<NA>": fill_value,
+ "nan": fill_value,
+ "None": fill_value,
+ }
+ )
+ else:
+ df[col] = s.fillna(value=fill_value)
Review Comment:
Nullable integer, float, and boolean dimensions still reject `<NULL>` here,
so a pivot over an `Int64` grouping column containing `pd.NA` aborts with
`TypeError` instead of preserving the null group. Could this upcast extension
dtypes that cannot accept the sentinel and add an `Int64` regression case?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
