Tianzi Cai created BEAM-12085:
---------------------------------
Summary: PubsubLiteIO.read() fails in DataflowRunner
Key: BEAM-12085
URL: https://issues.apache.org/jira/browse/BEAM-12085
Project: Beam
Issue Type: Bug
Components: extensions-java-gcp
Affects Versions: 2.28.0
Reporter: Tianzi Cai
Reading from a Pub/Sub Lite subscription using PubsubLiteIO fails with
DataflowRunner. It works in DirectRunner.
{code:java}
import org.apache.beam.sdk.io.gcp.pubsublite.PubsubLiteIO;
//..
pipeline
.apply("Read From Lite", PubsubLiteIO.read(subscriberOpitons))
.apply("Convert and print",
MapElements.into(TypeDescriptors.strings()).via(
(SequencedMessage sequencedMessage) -> {
String data =
sequencedMessage.getMessage().getData().toStringUtf8();
LOG.info("Received: " + data);
return data;
}
));
{code}
{noformat}
java.lang.UnsupportedOperationException: BundleFinalizer unsupported by
non-portable Dataflow.
at
org.apache.beam.runners.dataflow.worker.SplittableProcessFnFactory$SplittableDoFnRunnerFactory.lambda$createRunner$2(SplittableProcessFnFactory.java:170)
at
org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.OutputAndTimeBoundedSplittableProcessElementInvoker$1.bundleFinalizer(OutputAndTimeBoundedSplittableProcessElementInvoker.java:195)
at
org.apache.beam.sdk.io.gcp.pubsublite.PerSubscriptionPartitionSdf$DoFnInvoker.invokeProcessElement(Unknown
Source)
at
org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.OutputAndTimeBoundedSplittableProcessElementInvoker.invokeProcessElement(OutputAndTimeBoundedSplittableProcessElementInvoker.java:123)
at
org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SplittableParDoViaKeyedWorkItems$ProcessFn.processElement(SplittableParDoViaKeyedWorkItems.java:523)
{noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
