Maximilian Michels created BEAM-5999:
----------------------------------------
Summary: Proto error when running test_pardo_timers of Python
PortableValidatesRunner
Key: BEAM-5999
URL: https://issues.apache.org/jira/browse/BEAM-5999
Project: Beam
Issue Type: Bug
Components: sdk-py-core, testing
Reporter: Maximilian Michels
Fix For: 2.9.0
When I run {{FlinkRunnerTest.test_pardo_timers}}, I get the following error
during fusing the Proto pipeline. Note, the Java ValidatesPortableRunner timer
tests run fine. Seems to be an issue with the Python translation.
{noformat}
[flink-runner-job-server] ERROR
org.apache.beam.runners.flink.FlinkJobInvocation - Error during job invocation
test_pardo_timers_1541532770.41_45967eac-8089-4a5f-9d2b-7a4a9d01c7f1.
java.lang.IllegalArgumentException
at
org.apache.beam.model.pipeline.v1.RunnerApi$PTransform.getInputsOrThrow(RunnerApi.java:7418)
at
org.apache.beam.runners.core.construction.graph.QueryablePipeline.lambda$getTimers$11(QueryablePipeline.java:385)
at
java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
at java.util.Iterator.forEachRemaining(Iterator.java:116)
at
java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
at
java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
at
java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at
java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499)
at
org.apache.beam.runners.core.construction.graph.QueryablePipeline.getTimers(QueryablePipeline.java:392)
at
org.apache.beam.runners.core.construction.graph.GreedyStageFuser.forGrpcPortRead(GreedyStageFuser.java:93)
at
org.apache.beam.runners.core.construction.graph.GreedyPipelineFuser.fuseSiblings(GreedyPipelineFuser.java:340)
at
org.apache.beam.runners.core.construction.graph.GreedyPipelineFuser.fusePipeline(GreedyPipelineFuser.java:142)
at
org.apache.beam.runners.core.construction.graph.GreedyPipelineFuser.<init>(GreedyPipelineFuser.java:75)
at
org.apache.beam.runners.core.construction.graph.GreedyPipelineFuser.fuse(GreedyPipelineFuser.java:88)
at
org.apache.beam.runners.flink.FlinkJobInvocation.runPipeline(FlinkJobInvocation.java:96)
at
org.apache.beam.repackaged.beam_runners_flink_2.11.com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:111)
at
org.apache.beam.repackaged.beam_runners_flink_2.11.com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:58)
at
org.apache.beam.repackaged.beam_runners_flink_2.11.com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:75)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
{noformat}
To reproduce, enable the test in {{FlinkRunnerTest}}, then run the following
from {{sdks/python}}:
{noformat}
python -m apache_beam.runners.portability.flink_runner_test
--flink_job_server_jar=/path/to//beam/runners/flink/job-server/build/libs/beam-runners-flink_2.11-job-server-2.9.0-SNAPSHOT.jar
--streaming FlinkRunnerTest.test_pardo_timers
{noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
