[
https://issues.apache.org/jira/browse/BEAM-10855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17199836#comment-17199836
]
Monica Song commented on BEAM-10855:
------------------------------------
the algorithm is KMV (keeping k largest instead of k smallest) with correction
for hash collisions akin to cardinality estimation using bloom filters
> Add reference for appoximate unique algorithms in java and python SDK
> ---------------------------------------------------------------------
>
> Key: BEAM-10855
> URL: https://issues.apache.org/jira/browse/BEAM-10855
> Project: Beam
> Issue Type: Improvement
> Components: sdk-java-core, sdk-py-core
> Reporter: Monica Song
> Assignee: Monica Song
> Priority: P3
> Labels: doc_cleanup
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> The java [1] and python [2] classes for ApproximateUnique should reference a
> paper or some other form of online documentation as a source for their
> algorithm.
> [1]
> [https://github.com/apache/beam/blob/master/sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/ApproximateUnique.java]
> [2]
> [https://github.com/apache/beam/blob/master/sdks/python/apache_beam/transforms/stats.py]
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
