Re: [PR] Preserve pane index through reshuffle. [beam]

[via GitHub]

claudevdm commented on code in PR #34348:
URL: https://github.com/apache/beam/pull/34348#discussion_r2014620674


##########
sdks/go/pkg/beam/runners/prism/internal/urns/urns.go:
##########
@@ -124,6 +124,7 @@ var (
        CoderTimer              = cdrUrn(pipepb.StandardCoders_TIMER)
 
        CoderKV                  = cdrUrn(pipepb.StandardCoders_KV)
+       CoderTuple               = "beam:coder:tuple:v1"

Review Comment:
   I placed it here because python switches between tuple coder depending on 
number of elements which cauuses python tests to fail with prism runner 
https://github.com/apache/beam/blob/4fe34db0f42633aac1045591b14fdaabd664a43c/sdks/python/apache_beam/coders/coders.py#L1232
   
   For example, with the pane index change there ends up being a tuple coder 
with 3 elements (because pane info is now included)
   ```
   coders {
       key: "ref_Coder_TupleCoder_6"
       value {
         spec {
           urn: "beam:coder:tuple:v1"
         }
         component_coder_ids: "ref_Coder_BytesCoder_1"
         component_coder_ids: "ref_Coder_NullableCoder_7"
         component_coder_ids: "ref_Coder_FastPrimitivesCoder_8"
       }
     }
   ```
   
   But without pane info it falls back to using KV coder which is supported by 
prism
   ```
   coders {
       key: "ref_Coder_TupleCoder_6"
       value {
         spec {
           urn: "beam:coder:kv:v1"
         }
         component_coder_ids: "ref_Coder_BytesCoder_1"
         component_coder_ids: "ref_Coder_NullableCoder_7"
       }
     }
   ```
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscr...@beam.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org