[
https://issues.apache.org/jira/browse/BEAM-10212?focusedWorklogId=636962&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-636962
]
ASF GitHub Bot logged work on BEAM-10212:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 11/Aug/21 15:59
Start Date: 11/Aug/21 15:59
Worklog Time Spent: 10m
Work Description: anthonyqzhu commented on a change in pull request
#15300:
URL: https://github.com/apache/beam/pull/15300#discussion_r686968345
##########
File path:
sdks/java/harness/src/main/java/org/apache/beam/fn/harness/state/CachingBeamFnStateClient.java
##########
@@ -175,4 +179,71 @@ static StateCacheKey create(ByteString cacheToken,
ByteString continuationToken)
return new AutoValue_CachingBeamFnStateClient_StateCacheKey(cacheToken,
continuationToken);
}
}
+
+ /**
+ * Entry for a key in the state cache that adds a weight counter onto the
map of cache keys to
+ * state responses.
+ */
+ public static class StateCacheEntry {
+ // Initial size of hash tables per entry.
+ private static final int INITIAL_HASH_MAP_CAPACITY = 4;
+ // Overhead of each hash map entry.
+ private static final int HASH_MAP_ENTRY_OVERHEAD = 16;
+ // Overhead of each StateCacheEntry. One long, plus a concurrent hash map
+ private static final int PER_CACHE_ENTRY_OVERHEAD =
+ 8 + HASH_MAP_ENTRY_OVERHEAD * INITIAL_HASH_MAP_CAPACITY;
Review comment:
These are based off of WindmillStateCache
(https://github.com/apache/beam/blob/master/runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/WindmillStateCache.java)
- can someone let me know if these numbers are reasonable?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 636962)
Time Spent: 12h (was: 11h 50m)
> Add support for Java SDK harness state caching
> ----------------------------------------------
>
> Key: BEAM-10212
> URL: https://issues.apache.org/jira/browse/BEAM-10212
> Project: Beam
> Issue Type: Sub-task
> Components: sdk-java-harness
> Reporter: Luke Cwik
> Assignee: Anthony Zhu
> Priority: P3
> Labels: Clarified, portability
> Time Spent: 12h
> Remaining Estimate: 0h
>
> Tech spec:
> [https://docs.google.com/document/d/1BOozW0bzBuz4oHJEuZNDOHdzaV5Y56ix58Ozrqm2jFg/edit#heading=h.7ghoih5aig5m]
> Relevant document:
> [https://docs.google.com/document/d/1ltVqIW0XxUXI6grp17TgeyIybk3-nDF8a0-Nqw-s9mY/edit#|https://docs.google.com/document/d/1ltVqIW0XxUXI6grp17TgeyIybk3-nDF8a0-Nqw-s9mY/edit]
> Mailing list link:
> [https://lists.apache.org/thread.html/caa8d9bc6ca871d13de2c5e6ba07fdc76f85d26497d95d90893aa1f6@%3Cdev.beam.apache.org%3E]
>
> See https://issues.apache.org/jira/browse/BEAM-5428 and
> https://issues.apache.org/jira/browse/BEAM-8298 for Python implementation
> details.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
