scwhittle commented on code in PR #41303:
URL: https://github.com/apache/beam/pull/41303#discussion_r4584197843


##########
runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/windmill/state/WindmillMap.java:
##########


Review Comment:
   can this be removed? could be followup



##########
runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/windmill/state/WindmillMultimap.java:
##########
@@ -115,15 +115,31 @@ public void put(K key, V value) {
   // Initiates a backend state read to fetch all entries if necessary.
   private Future<Iterable<Map.Entry<ByteString, Iterable<V>>>> 
necessaryEntriesFromStorageFuture(
       boolean omitValues) {
-    if (complete) {
-      // Since we're complete, even if there are entries in storage we don't 
need to read them.
+    if (complete || (omitValues && allKeysKnown)) {
+      // Since we're complete (or all keys are known when only keys are 
requested), even if there
+      // are entries in storage we don't need to read them.
       return Futures.immediateFuture(Collections.emptyList());
     } else {
       return reader.multimapFetchAllFuture(
           omitValues, stateKey.byteString(), stateFamily, valueCoder);
     }
   }
 
+  private boolean isKeyValuesCached(Object structuralKey) {
+    if (complete) {
+      return true;
+    }
+    KeyState keyState = keyStateMap.get(structuralKey);
+    if (allKeysKnown
+        && (keyState == null || keyState.existence == 
KeyExistence.UNKNOWN_EXISTENCE)) {

Review Comment:
   seems odd that if we have UKNOWN_EXISTENCE that we would return true



##########
runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/windmill/state/WindmillMap.java:
##########
@@ -253,6 +252,13 @@ public void clear() {
     complete = true;
   }
 
+  private boolean isKeyCached(K key) {
+    return localRemovals.contains(key)
+        || negativeCache.contains(key)
+        || cachedValues.get(key) != null
+        || complete;

Review Comment:
   check complete first, it's cheap. maybe cachedValues second if that is 
perhaps more common that others



##########
runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/windmill/state/WindmillMultimap.java:
##########
@@ -719,6 +741,11 @@ public Boolean read() {
 
     @Override
     public ReadableState<Boolean> readLater() {
+      for (KeyState keyState : keyStateMap.values()) {
+        if (keyState.existence == KeyExistence.KNOWN_EXIST) {
+          return this;

Review Comment:
   ditto



##########
runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/windmill/state/WindmillMultimap.java:
##########
@@ -693,6 +711,10 @@ public Boolean read() {
 
     @Override
     public ReadableState<Boolean> readLater() {
+      KeyState keyState = keyStateMap.get(structuralKey);
+      if (keyState != null && keyState.existence != 
KeyExistence.UNKNOWN_EXISTENCE) {
+        return this;

Review Comment:
   nit: how about a boolean readRequired that you set here and then check that 
below so there is only the single return statement



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to