I had worked on coders implementation and started testing java pipelines on
the runner. Noticed something interesting, Typically Pipeline object
contain map of coder id and the Coder object like:
```
"ByteArrayCoder": Coder {
spec: Some(
FunctionSpec {
urn: "beam:coder:bytes:v1",
payload: [],
},
),
component_coder_ids: [],
},
``
Which makes perfect sense for the runner to use the coder_id of pcollection
to get the right coder implementation based on urn and encode/decode when
that pcollection arrives. But my pipeline had a dofn that gets the string
input and prints it, it's the end of pipeline and dofn returns void.
DoFn<String,Void>. Coder for this looked like :
```
"VoidCoder": Coder {
spec: Some(
FunctionSpec {
urn: "beam:coders:javasdk:0.1",
payload: [
```
I understand VoidCoder is for handling empty values, basically do nothing
when we receive the coder? But why doesn't it have its own urn like
others, what's the purpose of beam:coders:javasdk:0.1. Since it's a dofn,if
worker executes it and output pcol is void, worker sends no elements to
runner in data channel?
On Wed, May 20, 2026 at 5:26 PM Ganesh Sivakumar <
[email protected]> wrote:
> Thanks, This is a really valuable reference. I'm working on coders Rust
> implementation, will reach out if there are any questions.
>
> On Tue, May 19, 2026 at 7:11 PM Robert Burke <[email protected]> wrote:
>
>> Just a quick reply
>>
>> You're right that you use the Beam coders to interpret the bytes. You
>> just need an implementation of them in rust (and in every language building
>> Beam components).
>>
>> For the Prism runner (written in Go, default for Python and Go) we use
>> the Go SDK coder implementations, because they were already present. But
>> not every thing makes sense to use directly from the SDK within a runner
>> context.
>>
>>
>> https://github.com/apache/beam/blob/master/sdks/go/pkg/beam/runners/prism/internal/coders.go
>>
>> For example, for timers, it made more sense to reimplement certain
>> portions within the engine portion of prism, than to route towards SDK
>> constructs.
>>
>>
>> https://github.com/apache/beam/blob/master/sdks/go/pkg/beam/runners/prism/internal/engine/timers.go#L135
>>
>>
>> I wrote up the flow that Prism uses for managing bundles here. There are
>> flowcharts that provide the broad strokes, and I hope they are useful to
>> someone building their own runner.
>>
>>
>> https://github.com/apache/beam/blob/master/sdks/go/pkg/beam/runners/prism/internal/README.md
>>
>>
>> Finally, while the Beam Pipeline Protos and runner APIs dictate how
>> things communicate between runners and SDKs. The rest is up to you.
>> I have a languishing "hobby" Go SDK that uses a different approach to
>> coders to make them easier to deal with and manipulate, vs just the
>> bytestream approach.
>>
>> https://github.com/lostluck/beam-go/blob/main/coders/coders.go
>>
>> It mostly wraps a byte buffer, and then allows callers to pop or push
>> values to it. But this ends up playing very well for the garbage collector
>> in Go.
>>
>> Rust has different constraints and problems to solve, so don't feel
>> constrained by how the other languages do it.
>>
>> Hope this helps, and let me know if you have questions.
>> Robert Burke
>>
>> On Tue, May 19, 2026, 5:29 AM Ganesh Sivakumar <
>> [email protected]> wrote:
>>
>>> Hey Everyone,
>>>
>>> I am working on a new Rust based portable Beam Runner and I'm at
>>> pipeline execution phase where the Rust runner side needs to
>>> communicate with the worker sdk harness(Java) and execute the stages(
>>> stages are nothing but a set of fused transforms, formed using the greedy
>>> fusion approach from Beam Java utils, rewritten in Rust for the runner)
>>>
>>> For the stage to run on a worker, the runner needs to register the stage
>>> information with the worker and then send a run request via grpc channels.
>>> The worker will execute the transforms in the stage and send the output
>>> back to runner via data grpc channel in the form of Elements [1] Elements
>>> contain the output data as raw bytes which the runner needs to decode to
>>> get the actual data like String, Int or POJO. Typically other runners do it
>>> with Beam's defined coders for encoding and decoding. But for Rust there
>>> isn't Beam coders implementation. Curious if anyone previously worked on
>>> coders for cross languages, or similar things, and how did you implement in
>>> general.
>>>
>>> Thanks,
>>> Ganesh.
>>>
>>> [1] -
>>> https://github.com/apache/beam/blob/55eb624e5cd00e546ab19fc411281a0e5f596142/model/fn-execution/src/main/proto/org/apache/beam/model/fn_execution/v1/beam_fn_api.proto#L724
>>>
>>
