[
https://issues.apache.org/jira/browse/BEAM-11740?focusedWorklogId=558380&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-558380
]
ASF GitHub Bot logged work on BEAM-11740:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 26/Feb/21 03:44
Start Date: 26/Feb/21 03:44
Worklog Time Spent: 10m
Work Description: boyuanzz commented on a change in pull request #13924:
URL: https://github.com/apache/beam/pull/13924#discussion_r583363939
##########
File path:
sdks/java/harness/src/main/java/org/apache/beam/fn/harness/data/PCollectionConsumerRegistry.java
##########
@@ -210,8 +233,8 @@ public void accept(WindowedValue<T> input) throws Exception
{
try (Closeable close =
MetricsEnvironment.scopedMetricsContainer(this.unboundMetricContainer)) {
// Increment the counter for each window the element occurs in.
- this.counter.inc(input.getWindows().size());
-
+ this.elementCountCounter.inc(input.getWindows().size());
+ this.sampledByteSizeDistribution.tryUpdate(input.getValue(),
this.coder);
Review comment:
It's not only a mean value but a distribution value(min, max, sum
count), right? Can we put a TODO here to take care the case when we have
multiple windows optimization some day? Besides, I was wrong in my comment.
When considering multiple windows, the size should not be multiplied but the
counter should be updated per window with the same element size.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 558380)
Time Spent: 2h (was: 1h 50m)
> Add PCollection size estimate to Java SDK harness
> -------------------------------------------------
>
> Key: BEAM-11740
> URL: https://issues.apache.org/jira/browse/BEAM-11740
> Project: Beam
> Issue Type: New Feature
> Components: sdk-java-harness
> Reporter: Kiley Sok
> Assignee: Kiley Sok
> Priority: P2
> Time Spent: 2h
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
