Hi Lukasz, I have solved this issue by setting ServicesResourceTransformer on my maven-shade transformets list. And there are another way to solve this question by adding HBaseMutationCoder in code manually.
Thanks. On Fri, Nov 9, 2018 at 3:16 AM Lukasz Cwik <[email protected]> wrote: > Posted my answer on SO, reposting here for visibility: > > Note that the HBaseCoderProviderRegistrar[1] already registers the > HBaseMutationCoder for the Mutation type automatically already. > > Using the maven-shade plugin without handling service files contained in > META-INF/ inside your output jar is a common pitfall for users. You want to > add the ServicesResourceTransformer to your maven-shade transformers list. > > See Apache documentation[2] and ServiceLoader javadoc[3] for more details. > There are many other questions like this on StackOverflow. > > > [1]: > https://github.com/apache/beam/blob/95f1a5aeba85cd4a6e0eb1ce6eee141e3667d86d/sdks/java/io/hbase/src/main/java/org/apache/beam/sdk/io/hbase/HBaseCoderProviderRegistrar.java#L30 > [2]: > https://maven.apache.org/plugins/maven-shade-plugin/examples/resource-transformers.html#ServicesResourceTransformer > [3]: > https://docs.oracle.com/javase/8/docs/api/java/util/ServiceLoader.html > > > On Thu, Nov 8, 2018 at 10:13 AM K Fred <[email protected]> wrote: > >> Hi， >> >> I am running into an issue using HBaseIO with FlinkRunner. >> The exception is below: >> Exception in thread "main" java.lang.IllegalStateException: Unable to >> return a default Coder for >> ParDo(HBaseProfile)/ParMultiDo(HBaseProfile).output [PCollection]. Correct >> one of the following root causes: >> No Coder has been manually specified; you may do so using .setCoder(). >> Inferring a Coder from the CoderRegistry failed: Unable to provide a >> Coder for org.apache.hadoop.hbase.client.Mutation. >> Building a Coder using a registered CoderProvider failed. >> See suppressed exceptions for detailed failures. >> Using the default output Coder from the producing PTransform failed: >> PTransform.getOutputCoder called. >> at >> org.apache.beam.repackaged.beam_sdks_java_core.com.google.common.base.Preconditions.checkState(Preconditions.java:444) >> at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278) >> at >> org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:115) >> at >> org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:190) >> at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:536) >> at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:488) >> at org.apache.beam.sdk.values.PCollection.apply(PCollection.java:370) >> at ac.cn.iie.process.ProfileProcess.process(ProfileProcess.java:91) >> at ac.cn.iie.Bootstrap.main(Bootstrap.java:25) >> >> I have no idea about this question. Just guess that may be related to >> Coder register. So i regist Coder provider manually. The detail is : >> https://stackoverflow.com/questions/53200867/unable-to-provide-a-coder-for-org-apache-hadoop-hbase-client-mutation-using-hbas/53200894#53200894 >> >> Does anyone have better solution? I think it shouldn't register by user. >> It camn register automatically by pipeline. >> -- >> >> Fred >> > -- Fred
