[
https://issues.apache.org/jira/browse/BEAM-9488?focusedWorklogId=427087&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-427087
]
ASF GitHub Bot logged work on BEAM-9488:
----------------------------------------
Author: ASF GitHub Bot
Created on: 24/Apr/20 20:38
Start Date: 24/Apr/20 20:38
Worklog Time Spent: 10m
Work Description: lukecwik commented on pull request #11514:
URL: https://github.com/apache/beam/pull/11514#issuecomment-619228095
I looked through the implementation and it seems as though adding the
pcollection id to the ConsumerSet doesn't work out since operations don't have
that level of visibility in pipeline proto and consumers works off of a index
-> receiver map and expects tags to get mapped to indices so we would need to
go through all three layers. I suggest that we stick with this brittle approach
until we can delete the non-portable Python worker implementation which would
make a lot of the layers simpler.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 427087)
Time Spent: 1.5h (was: 1h 20m)
> Python SDK sending unexpected MonitoringInfo
> --------------------------------------------
>
> Key: BEAM-9488
> URL: https://issues.apache.org/jira/browse/BEAM-9488
> Project: Beam
> Issue Type: Bug
> Components: sdk-py-core
> Reporter: Ruoyun Huang
> Assignee: Luke Cwik
> Priority: Minor
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> element_count metrics is supposed to be tied with pcollection ids, but by
> inspecting what is sent over by python sdk, we see there are monitoringInfo
> sent wit ptransforms in it.
> [Double checked the job graph, these seem to be redundant. i.e. the
> corresponding pcollection does have its own MonitoringInfo reported.]
> Likely a bug.
> Proof:
> urn: "beam:metric:element_count:v1"
> type: "beam:metrics:sum_int_64"
> metric {
> counter_data {
> int64_value: 1
> }
> }
> labels {
> key: "PTRANSFORM"
> value: "start/MaybeReshuffle/Reshuffle/RemoveRandomKeys-ptransform-85"
> }
> labels {
> key: "TAG"
> value: "None"
> }
> timestamp {
> seconds: 1583949073
> nanos: 842402935
> }
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
