[
https://issues.apache.org/jira/browse/BEAM-13130?focusedWorklogId=671038&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-671038
]
ASF GitHub Bot logged work on BEAM-13130:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 27/Oct/21 21:10
Start Date: 27/Oct/21 21:10
Worklog Time Spent: 10m
Work Description: jrmccluskey commented on a change in pull request
#15815:
URL: https://github.com/apache/beam/pull/15815#discussion_r737849490
##########
File path: sdks/go/pkg/beam/core/runtime/exec/pardo.go
##########
@@ -324,6 +324,11 @@ func (n *ParDo) postInvoke() error {
func (n *ParDo) fail(err error) error {
Review comment:
Adding the comment here for the record since we already talked offline:
the defer 100% will be added to the InvokeProcessFn() and InvokeDafaFn() cases
since we know we'll need to do that. I'll likely re-write the postInvoke() loop
to check if the cache has been instantiated too so we aren't sweating a
potential nil pointer exception if we got a panic beforehand.
The question remains as to how we handle potential errors being raised by
the deferred postInvoke() calls. It's a minor detail but I don't want to
accidentally hide errors.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 671038)
Time Spent: 1h 10m (was: 1h)
> [Go SDK] Side Input Memory Leak
> -------------------------------
>
> Key: BEAM-13130
> URL: https://issues.apache.org/jira/browse/BEAM-13130
> Project: Beam
> Issue Type: Bug
> Components: sdk-go
> Reporter: Jack McCluskey
> Assignee: Jack McCluskey
> Priority: P2
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> Keeping a list of the open stateKeyReader structs in the ScopedStateReader is
> holding on to memory for the life of bundles, leading to significant memory
> usage in large batch jobs. This list needs to be removed to the garbage
> collector can clean up stateKeyReader structs sooner.
>
> stateKeyReader:
> [https://github.com/apache/beam/blob/d916c1f55e57a61b54135d0922ad8660735bd287/sdks/go/pkg/beam/core/runtime/harness/statemgr.go#L105]
> List kept by ScopedStateReader:
> https://github.com/apache/beam/blob/d916c1f55e57a61b54135d0922ad8660735bd287/sdks/go/pkg/beam/core/runtime/harness/statemgr.go#L39
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
