[
https://issues.apache.org/jira/browse/BEAM-12495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Beam JIRA Bot updated BEAM-12495:
---------------------------------
Labels: dataframe-api (was: dataframe-api stale-P2)
> DataFrame API: groupby(dropna=False) still drops NAs when grouping on
> multiple columns or indexes
> -------------------------------------------------------------------------------------------------
>
> Key: BEAM-12495
> URL: https://issues.apache.org/jira/browse/BEAM-12495
> Project: Beam
> Issue Type: Bug
> Components: dsl-dataframe, sdk-py-core
> Reporter: Brian Hulette
> Priority: P3
> Labels: dataframe-api
> Time Spent: 2h 10m
> Remaining Estimate: 0h
>
> {code}
> df.groupby(['foo', 'bar'], dropna=False).sum()
> {code}
> This will still drop NAs in the output.
> This is due to pandas bug
> [36470|https://github.com/pandas-dev/pandas/issues/36470] "BUG: groupby(...,
> dropna=False) excludes NA values when grouping on MultiIndex levels".
> We implement groupby by moving all grouped data into the index and requiring
> Index() partitioning, so we will always run into this issue, even when the
> user is grouping on columns, not indexes.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
