[
https://issues.apache.org/jira/browse/BEAM-12181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17499247#comment-17499247
]
Brian Hulette commented on BEAM-12181:
--------------------------------------
> Could you elaborate on why it may be necessary to create our own
> implementation of kde()? Is the scipy impl incompatible with the series
> format?
Sorry seems like I wasn't clear here. What I meant was we might consider adding
a DeferredSeries.kde() method, even though pandas.Series does not have a kde
method. Under the hood it could certainly defer to scipy.gaussian_kde for
computing partial results on the workers. The only reason to do it would be for
separation of concerns - I thought it might be more straightforward to first
add a distributed kde implementation that gets the same results as
(non-distributed) scipy.gaussian_kde. Then we could add an approximate mode
option on top of that.
> Implement parallelized (approximate) mode
> -----------------------------------------
>
> Key: BEAM-12181
> URL: https://issues.apache.org/jira/browse/BEAM-12181
> Project: Beam
> Issue Type: Improvement
> Components: dsl-dataframe, sdk-py-core
> Reporter: Brian Hulette
> Priority: P3
> Labels: dataframe-api
>
> Currently we require Singleton partitioning to compute mode(). We should
> provide an option to compute approximate mode() which can be parallelized.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
