shunping commented on issue #30513: URL: https://github.com/apache/beam/issues/30513#issuecomment-2469074574
Ok, take another look at this. The test started to fail at 11/06 6:32PM (https://github.com/apache/beam/actions/runs/11713650994), the last successful run was at 11/06 12:33PM (https://github.com/apache/beam/actions/runs/11708854671). There are two commits during this time internal: - Distro Python SDK: https://github.com/apache/beam/commit/81f35ab62298a2ec9fadeded82461b363b6401db, which causes the previously mentioned error during docker image building (:sdks:python:container:py39:docker) - Kafka: https://github.com/apache/beam/commit/eeebae1bda6b211463e53a4e4ca469bfa9763399, which seems to be the reason for failure (:sdks:python:test-suites:portable:py39:postCommitPy39IT). The Kafka error message is shown below: ``` FAILED apache_beam/io/external/xlang_kafkaio_it_test.py::CrossLanguageKafkaIOTest::test_local_kafkaio_populated_key - RuntimeError: Pipeline BeamApp-runner-1111115329-514dd26a_03822608-80d0-4037-bc13-11d632204f46 failed in state FAILED: java.lang.RuntimeException: Error received from SDK harness for instruction 3: org.apache.beam.sdk.util.UserCodeException: java.io.IOException: KafkaWriter : failed to send 1 records (since last report) at org.apache.beam.sdk.util.UserCodeException.wrap(UserCodeException.java:39) at org.apache.beam.sdk.io.kafka.KafkaWriter$DoFnInvoker.invokeProcessElement(Unknown Source) at org.apache.beam.fn.harness.FnApiDoFnRunner.processElementForParDo(FnApiDoFnRunner.java:810) at org.apache.beam.fn.harness.data.PCollectionConsumerRegistry$MetricTrackingFnDataReceiver.accept(PCollectionConsumerRegistry.java:348) at org.apache.beam.fn.harness.data.PCollectionConsumerRegistry$MetricTrackingFnDataReceiver.accept(PCollectionConsumerRegistry.java:275) at org.apache.beam.fn.harness.FnApiDoFnRunner.outputTo(FnApiDoFnRunner.java:1837) at org.apache.beam.fn.harness.FnApiDoFnRunner.access$3100(FnApiDoFnRunner.java:145) at org.apache.beam.fn.harness.FnApiDoFnRunner$NonWindowObservingProcessBundleContext.output(FnApiDoFnRunner.java:2695) at org.apache.beam.sdk.transforms.MapElements$2.processElement(MapElements.java:151) at org.apache.beam.sdk.transforms.MapElements$2$DoFnInvoker.invokeProcessElement(Unknown Source) at org.apache.beam.fn.harness.FnApiDoFnRunner.processElementForParDo(FnApiDoFnRunner.java:810) at org.apache.beam.fn.harness.data.PCollectionConsumerRegistry$MetricTrackingFnDataReceiver.accept(PCollectionConsumerRegistry.java:348) at org.apache.beam.fn.harness.data.PCollectionConsumerRegistry$MetricTrackingFnDataReceiver.accept(PCollectionConsumerRegistry.java:275) at org.apache.beam.fn.harness.BeamFnDataReadRunner.forwardElementToConsumer(BeamFnDataReadRunner.java:213) at org.apache.beam.sdk.fn.data.BeamFnDataInboundObserver.multiplexElements(BeamFnDataInboundObserver.java:172) at org.apache.beam.sdk.fn.data.BeamFnDataInboundObserver.awaitCompletion(BeamFnDataInboundObserver.java:136) at org.apache.beam.fn.harness.control.ProcessBundleHandler.processBundle(ProcessBundleHandler.java:550) at org.apache.beam.fn.harness.control.BeamFnControlClient.delegateOnInstructionRequestType(BeamFnControlClient.java:150) at org.apache.beam.fn.harness.control.BeamFnControlClient$InboundObserver.lambda$onNext$0(BeamFnControlClient.java:115) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) at org.apache.beam.sdk.util.UnboundedScheduledExecutorService$ScheduledFutureTask.run(UnboundedScheduledExecutorService.java:163) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base/java.lang.Thread.run(Thread.java:829) Caused by: java.io.IOException: KafkaWriter : failed to send 1 records (since last report) at org.apache.beam.sdk.io.kafka.KafkaWriter.checkForFailures(KafkaWriter.java:183) at org.apache.beam.sdk.io.kafka.KafkaWriter.processElement(KafkaWriter.java:66) Caused by: org.apache.kafka.common.errors.TimeoutException: Topic xlang_kafkaio_test_populated_key_e9df3a07-037f-45a1-afde-7cea599f9570 not present in metadata after 60000 ms. ``` @Abacn , could you check this and see if we need to roll it back? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
