Hi!
I have a parameterized a DoFn using generics to return different Classes
depending on where I use it. The generics type is required to be extending
a superclass I have specified myself.

The ParDo seem to function as expected and I get the right output using it
but when I try to test this the PAssert containsAnyOrder only compares the
superclass of the objects and not the parameterized type.

I might be missing something Java specific here but the PAssert is really
hard to get what it is actually doing looking at the code.

So is there any way to force it to compare the correct class or can I in
some way extract the objects of the pcollection in the test and do the
checks manually?

The method signature looks like this:

static class MaterializeFn<T extends Entity> extends DoFn<KV<String,
Log>, KV<String, T>>

And if the superclass Entity does not have a Default coder I get the
following error:

java.lang.IllegalStateException: Unable to return a default Coder for
Materialize/ParMultiDo(Materialize).out0 [PCollection]. Correct one of
the following root causes:
  No Coder has been manually specified;  you may do so using .setCoder().
  Inferring a Coder from the CoderRegistry failed: Cannot provide
coder for parameterized type
org.apache.beam.sdk.values.KV<java.lang.String, T>: Unable to provide
a Coder for T.
  Building a Coder using a registered CoderProvider failed.
  See suppressed exceptions for detailed failures.
  Using the default output Coder from the producing PTransform failed:
PTransform.getOutputCoder called.

Which kind of makes sense i guess as it needs to make sure there is a coder
for all possible classes in runtime. The Sub classes do also have a coder
specified (all use AvroCoder) and seem to work fine when running the
pipeline as I get the data I am expecting as output (in Kafka).

Any ideas on what do do? I tried to look at the PAssert to figure out why
this could be but it was a bit over my head.

Thanks,
Vilhelm von Ehrenheim
​

Reply via email to