[
https://issues.apache.org/jira/browse/BEAM-12867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kenneth Knowles updated BEAM-12867:
-----------------------------------
Description:
The following pipeline fails to print out all numbers 1 to 100 when run on
DirectRunner in streaming mode.
This was identified implementing
org.apache.beam.sdk.io.gcp.pubsublite.ReadWriteIT, which uses a workaround for
the bug in either Create or DirectRunner:
{code}
private static final int COUNT = 100;
private static AtomicInteger CREATED_COUNT = new AtomicInteger();
public static void run(Pipeline pipeline) {
PCollection<Integer> indexes = pipeline.apply( "createIndexes",
Create.of(IntStream.range(0,
COUNT).boxed().collect(Collectors.toList())));
indexes.apply(
"createMessages",
MapElements.via(
new SimpleFunction<Integer, Integer>(
index -> {
System.err.println("Created message index " +
createdCount.incrementAndGet());
return index;
}) {}));
pipeline.run().waitUntilFinish(); // Never terminates
}
{code}
was:
The following pipeline fails to print out all numbers 1 to 100 when run on
DirectRunner in streaming mode.
This was identified implementing
org.apache.beam.sdk.io.gcp.pubsublite.ReadWriteIT, which uses a workaround for
the bug in either Create or DirectRunner:
{code}
private static final int COUNT = 100;
private static AtomicInteger CREATED_COUNT = new AtomicInteger();
public static void run(Pipeline pipeline) {
PCollection<Integer> indexes =
pipeline.apply(
"createIndexes",
Create.of(IntStream.range(0, COUNT).boxed().collect(Collectors.toList())));
indexes.apply(
"createMessages",
MapElements.via(
new SimpleFunction<Integer, Integer>(
index -> {
System.err.println("Created message index " + createdCount.incrementAndGet());
return index; }) {}));
pipeline.run().waitUntilFinish(); // Never terminates
}
{code}
> Either Create or DirectRunner fails to produce all elements to the following
> transform
> --------------------------------------------------------------------------------------
>
> Key: BEAM-12867
> URL: https://issues.apache.org/jira/browse/BEAM-12867
> Project: Beam
> Issue Type: Bug
> Components: runner-direct
> Reporter: Daniel Collins
> Priority: P1
>
> The following pipeline fails to print out all numbers 1 to 100 when run on
> DirectRunner in streaming mode.
> This was identified implementing
> org.apache.beam.sdk.io.gcp.pubsublite.ReadWriteIT, which uses a workaround
> for the bug in either Create or DirectRunner:
> {code}
> private static final int COUNT = 100;
> private static AtomicInteger CREATED_COUNT = new AtomicInteger();
> public static void run(Pipeline pipeline) {
> PCollection<Integer> indexes = pipeline.apply( "createIndexes",
> Create.of(IntStream.range(0,
> COUNT).boxed().collect(Collectors.toList())));
> indexes.apply(
> "createMessages",
> MapElements.via(
> new SimpleFunction<Integer, Integer>(
> index -> {
> System.err.println("Created message index " +
> createdCount.incrementAndGet());
> return index;
> }) {}));
> pipeline.run().waitUntilFinish(); // Never terminates
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
