Runners can never know about every coder that a user may want to write which is why we need to have a mechanism for Runners to be able to convert any unknown coder to one it can handle. This is done via WireCoders.instantiateRunnerWireCoder but this modifies the original coder which is why WireCoders.addSdkWireCoder creates the proto definition that the SDK should be told to use. In your case, your correct in that KV<Void, T> becomes KVCoder<LengthPrefixCoder<ByteArrayCoder>, LengthPrefixCoder<ByteArrayCoder>> on the runner side and on the SDK side it should be KVCoder<LengthPrefixCoder<VoidCoder>, LengthPrefixCoder<CoderForT>>. More details in [1].
1: http://doc/1IGduUqmhWDi_69l9nG8kw73HZ5WI5wOps9Tshl5wpQA#heading=h.sh4d5klmtfis On Fri, Sep 28, 2018 at 11:02 AM Shen Li <cs.she...@gmail.com> wrote: > Hi, > > I noticed that ModelCoderRegistrar only includes 9 out of ~40 coders. May > I know the rationale behind this decision? > > > https://github.com/apache/beam/blob/release-2.7.0/runners/core-construction-java/src/main/java/org/apache/beam/runners/core/construction/ModelCoderRegistrar.java > > I think one consequence of the above configuration is > that WireCoders.instantiateRunnerWireCoder cannot instantiate KV coders > correctly, where VoidCoder (key coder) becomes > LengthPrefixCoder(ByteArrayCoder). What is the appropriate way to get > KvCoder<Void, T> from RunnerApi.Pipeline? > > Thanks, > Shen >
