[
https://issues.apache.org/jira/browse/BEAM-10900?focusedWorklogId=484705&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-484705
]
ASF GitHub Bot logged work on BEAM-10900:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 15/Sep/20 18:32
Start Date: 15/Sep/20 18:32
Worklog Time Spent: 10m
Work Description: tvalentyn commented on a change in pull request #12851:
URL: https://github.com/apache/beam/pull/12851#discussion_r488878863
##########
File path: sdks/python/apache_beam/transforms/stats_test.py
##########
@@ -89,6 +90,12 @@ def setUp(self):
None,
0.1,
'assert:global_by_error_with_large_population'),
+ (
+ 'numpy_input_data',
+ np.array(range(10)),
Review comment:
I see. At this point I am curious about how users use
ApproximateUnique.
The use-case you are addressing is when users pass a PCollection of
elements, where each element is a single value stored in a numpy datatype.
Since it's a single value, we convert it to a scalar. Is that right?
I am wondering if there is also a use-case when users pass a PCollection of
numpy arrays (perhaps erroneously). In which case the current combiner will
pick the first element of the array, so approximation may not be very precise.
I wonder if a more precise implementation makes sense or this use-case is not
common.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 484705)
Time Spent: 1h 40m (was: 1.5h)
> stats.ApproximateUniqueCombineFn should handle numpy array
> ----------------------------------------------------------
>
> Key: BEAM-10900
> URL: https://issues.apache.org/jira/browse/BEAM-10900
> Project: Beam
> Issue Type: Bug
> Components: sdk-py-core
> Reporter: Monica Song
> Assignee: Monica Song
> Priority: P1
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
> stats.ApproximateUniqueCombineFn uses DeterministicFastPrimitivesCoder to
> encode input data. DeterministicFastPrimitivesCoder only accepts python data
> types and iterables, so numpy data types throw an error.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
