[
https://issues.apache.org/jira/browse/BEAM-10212?focusedWorklogId=643326&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-643326
]
ASF GitHub Bot logged work on BEAM-10212:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 30/Aug/21 08:04
Start Date: 30/Aug/21 08:04
Worklog Time Spent: 10m
Work Description: scwhittle commented on pull request #15300:
URL: https://github.com/apache/beam/pull/15300#issuecomment-908132864
Actually I see now you are saying that the amount of memory available for
processing on a machine with a fixed amount of cpu but increased ram will be
greater. That is true, however if the cache itself is the cause of OOMs (not
user processing) it doesn't help.
With the windmill cache, the objects were cached using the serialized size
as the weight but the cached objects were deserialized (as that added a large
benefit to the cache). However this meant that the memory used by the cache
could be much more than the configured limit depending on the objects cached.
It seems here we might be caching the protos so perhaps the size estimate is
reliable. However if we change to caching user objects in the future we might
need to be more careful.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 643326)
Time Spent: 14h 40m (was: 14.5h)
> Add support for Java SDK harness state caching
> ----------------------------------------------
>
> Key: BEAM-10212
> URL: https://issues.apache.org/jira/browse/BEAM-10212
> Project: Beam
> Issue Type: Sub-task
> Components: sdk-java-harness
> Reporter: Luke Cwik
> Assignee: Anthony Zhu
> Priority: P3
> Labels: Clarified, portability
> Time Spent: 14h 40m
> Remaining Estimate: 0h
>
> Tech spec:
> [https://docs.google.com/document/d/1BOozW0bzBuz4oHJEuZNDOHdzaV5Y56ix58Ozrqm2jFg/edit#heading=h.7ghoih5aig5m]
> Relevant document:
> [https://docs.google.com/document/d/1ltVqIW0XxUXI6grp17TgeyIybk3-nDF8a0-Nqw-s9mY/edit#|https://docs.google.com/document/d/1ltVqIW0XxUXI6grp17TgeyIybk3-nDF8a0-Nqw-s9mY/edit]
> Mailing list link:
> [https://lists.apache.org/thread.html/caa8d9bc6ca871d13de2c5e6ba07fdc76f85d26497d95d90893aa1f6@%3Cdev.beam.apache.org%3E]
>
> See https://issues.apache.org/jira/browse/BEAM-5428 and
> https://issues.apache.org/jira/browse/BEAM-8298 for Python implementation
> details.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
