[
https://issues.apache.org/jira/browse/BEAM-13212?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jack McCluskey updated BEAM-13212:
----------------------------------
Status: Open (was: Triage Needed)
> [Go SDK] Add support for setting upper memory limit for side input cache
> ------------------------------------------------------------------------
>
> Key: BEAM-13212
> URL: https://issues.apache.org/jira/browse/BEAM-13212
> Project: Beam
> Issue Type: Improvement
> Components: sdk-go
> Reporter: Jack McCluskey
> Priority: P3
>
> This is implementing https://issues.apache.org/jira/browse/BEAM-8297 for the
> Go SDK's side input cache. The current implementation reads the entire side
> input into memory and keeps track of how many distinct inputs to hold;
> however, each side input is not guaranteed to occupy the same amount of
> memory. An extra parameter should be created to be passed in to the
> SideInputCache
> ([https://github.com/apache/beam/blob/master/sdks/go/pkg/beam/core/runtime/harness/statecache/statecache.go)]
> to set a maximum memory limit in place of a maximum element capacity, then
> add a hook (see
> https://github.com/apache/beam/blob/master/sdks/go/pkg/beam/core/runtime/harness/cache_hooks.go)
> and corresponding function to the harnessopts package
> ([https://github.com/apache/beam/blob/master/sdks/go/pkg/beam/util/harnessopts/cache.go)]
> so users can set this value themselves.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
