[
https://issues.apache.org/jira/browse/BEAM-11777?focusedWorklogId=585470&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-585470
]
ASF GitHub Bot logged work on BEAM-11777:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 19/Apr/21 23:46
Start Date: 19/Apr/21 23:46
Worklog Time Spent: 10m
Work Description: rohdesamuel commented on a change in pull request
#14438:
URL: https://github.com/apache/beam/pull/14438#discussion_r616247521
##########
File path: sdks/python/apache_beam/dataframe/frames.py
##########
@@ -922,33 +923,79 @@ def dropna(self, **kwargs):
to_string = frame_base.wont_implement_method(
pd.Series, 'to_string', reason="non-deferred-result")
- def aggregate(self, func, axis=0, *args, **kwargs):
+ @frame_base.args_to_kwargs(pd.Series)
+ @frame_base.populate_defaults(pd.Series)
+ def aggregate(self, func, axis, *args, **kwargs):
+ if kwargs.get('skipna', False):
+ # Eagerly generate a proxy to make sure skipna is a valid argument
+ # for this aggregation method
+ _ = self._expr.proxy().aggregate(func, axis, *args, **kwargs)
Review comment:
If I understand this correctly, this forwards the arguments to an empty
DataFrame to validate the arguments with a given aggregation function (e.g.
sum, proc, etc). Since the code here seems to be duplicated in other functions,
does it make sense to automate or at least break it out into a helper function?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 585470)
Time Spent: 11.5h (was: 11h 20m)
> Support correct kwargs in aggregation methods on DataFrame, Series
> ------------------------------------------------------------------
>
> Key: BEAM-11777
> URL: https://issues.apache.org/jira/browse/BEAM-11777
> Project: Beam
> Issue Type: Improvement
> Components: sdk-py-core
> Reporter: Brian Hulette
> Priority: P2
> Labels: dataframe-api
> Time Spent: 11.5h
> Remaining Estimate: 0h
>
> {DataFrame,Series}.{all, any, max, min, prod, mean, median, sum} are all
> implemented via frame_base._agg_method, which just re-uses
> {DataFrame,Series}.agg}. However the pandas operations have some different
> kwargs that are not supported by agg. Some are universal (level=, skip_na=),
> others are unique to each operation (numeric_only= or bool_only=).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
