StefanRRichter commented on a change in pull request #8611:
[FLINK-12693][state] Store state per key-group in CopyOnWriteStateTable
URL: https://github.com/apache/flink/pull/8611#discussion_r293558549
##########
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/state/heap/StateTable.java
##########
@@ -156,10 +203,23 @@ public boolean isEmpty() {
* @param transformation the transformation function.
* @throws Exception if some exception happens in the transformation
function.
*/
- public abstract <T> void transform(
+ public <T> void transform(
N namespace,
T value,
- StateTransformationFunction<S, T> transformation)
throws Exception;
+ StateTransformationFunction<S, T> transformation)
throws Exception {
+ K key = keyContext.getCurrentKey();
+ checkKeyNamespacePreconditions(key, namespace);
+
+ int keyGroup = keyContext.getCurrentKeyGroupIndex();
+ StateMap<K, N, S> stateMap = getMapForKeyGroup(keyGroup);
+
+ if (stateMap == null) {
+ stateMap = createStateMap();
Review comment:
I think you make a trade-off here between the attempt to save some space,
and some code complexity and checking overhead for `null` in the array. Given
that the keys are typically uniformly distributed across key-groups, I would
assume that no map will stay `null` for long. What I would suggest is to
initialize all array positions with maps at the time of array allocation and
keep the invariant that all places in the array are strictly non-null. This
allows you to drop all null checks on around this array.
What I can forsee is that maybe the reason to have the checks is for
followup work that implements the spilling and for spilled key-groups you can
have some maps removed from the array? If that is the case, then we could
either keep the check or (maybe better) have a special implementation for an
empty/spilled map?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
