Hello guys
I have a question, is it possible to use KV along with AutoValueSchema
objects? I'm having troubles when I try to use it together.
I have an object like the following
@AutoValue
@DefaultSchema(AutoValueSchema.class)
public abstract class SharedCoreEvent {
@JsonProperty("subscriptionId")
public abstract String getSubscription();
<other properties>
}
Then I have a pipeline like the following:
input.apply(ParDo.of(new ProcessEvents()))
.apply(Reshuffle.of()).apply(Values.create()).apply(output);
My input is a single object and my ProcessEvents will produce tons of
events, in a fan-out fashion. that is why I used Reshuffle here
But when I run this pipeline it throws the following error:
lang.IllegalStateException: Unable to return a default Coder for
MCEvents/ParDo(ProcessEvents)/ParMultiDo(ProcessEvents).output
[PCollection@2131266396]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide
coder for parameterized type
org.apache.beam.sdk.values.KV<java.lang.Long, events.SharedCoreEvent>:
Unable to provide a Coder for events.SharedCoreEvent
Building a Coder using a registered CoderProvider failed.
Something similar happens with my source when I use KafkaIO and the
source produces a KV<String,Session> PCollection.
Is there any reason for this? Maybe I'm misusing the schemas?
Really appreciate your help
Thanks
Ruben
