[
https://issues.apache.org/jira/browse/BEAM-9482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17383092#comment-17383092
]
Ankur Goenka commented on BEAM-9482:
------------------------------------
I tried [https://github.com/apache/beam/pull/15187] but it is failing with
{code:java}
// java.lang.RuntimeException: java.lang.IllegalStateException: Could not
find any partitions info. Please check Kafka configuration and make sure that
provided topics exist.
at
org.apache.beam.runners.dataflow.ReadTranslator.translateReadHelper(ReadTranslator.java:55)
at
org.apache.beam.runners.dataflow.DataflowRunner$StreamingUnboundedRead$ReadWithIdsTranslator.translate(DataflowRunner.java:1886)
at
org.apache.beam.runners.dataflow.DataflowRunner$StreamingUnboundedRead$ReadWithIdsTranslator.translate(DataflowRunner.java:1882)
at
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator.visitPrimitiveTransform(DataflowPipelineTranslator.java:515)
at
org.apache.beam.sdk.runners.TransformHierarchy$Node.visit(TransformHierarchy.java:593)
at
org.apache.beam.sdk.runners.TransformHierarchy$Node.visit(TransformHierarchy.java:585)
at
org.apache.beam.sdk.runners.TransformHierarchy$Node.visit(TransformHierarchy.java:585)
at
org.apache.beam.sdk.runners.TransformHierarchy$Node.visit(TransformHierarchy.java:585)
at
org.apache.beam.sdk.runners.TransformHierarchy$Node.visit(TransformHierarchy.java:585)
at
org.apache.beam.sdk.runners.TransformHierarchy$Node.access$500(TransformHierarchy.java:240)
at
org.apache.beam.sdk.runners.TransformHierarchy.visit(TransformHierarchy.java:214)
at org.apache.beam.sdk.Pipeline.traverseTopologically(Pipeline.java:469)
at
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator.translate(DataflowPipelineTranslator.java:448)
at
org.apache.beam.runners.dataflow.DataflowPipelineTranslator.translate(DataflowPipelineTranslator.java:190)
at
org.apache.beam.runners.dataflow.DataflowRunner.run(DataflowRunner.java:991)
at
org.apache.beam.runners.dataflow.DataflowRunner.run(DataflowRunner.java:191)
at org.apache.beam.sdk.Pipeline.run(Pipeline.java:323)
at org.apache.beam.sdk.testing.TestPipeline.run(TestPipeline.java:398)
at org.apache.beam.sdk.testing.TestPipeline.run(TestPipeline.java:334)
at
org.apache.beam.sdk.io.kafka.KafkaIOIT.testKafkaIOReadsAndWritesCorrectlyInStreaming(KafkaIOIT.java:152)
Caused by:
java.lang.IllegalStateException: Could not find any partitions info.
Please check Kafka configuration and make sure that provided topics exist.
at
org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at
org.apache.beam.sdk.io.kafka.KafkaUnboundedSource.split(KafkaUnboundedSource.java:70)
at
org.apache.beam.runners.dataflow.internal.CustomSources.serializeToCloudSource(CustomSources.java:87)
at
org.apache.beam.runners.dataflow.ReadTranslator.translateReadHelper(ReadTranslator.java:52)
... 19 more
{code}
> beam_PerformanceTests_Kafka_IO failing due to " provided port is already
> allocated"
> -----------------------------------------------------------------------------------
>
> Key: BEAM-9482
> URL: https://issues.apache.org/jira/browse/BEAM-9482
> Project: Beam
> Issue Type: Bug
> Components: test-failures
> Reporter: Chamikara Madhusanka Jayalath
> Priority: P1
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> For example,
> [https://builds.apache.org/view/A-D/view/Beam/view/PerformanceTests/job/beam_PerformanceTests_Kafka_IO/514/console]
>
> 18:55:33 Error from server (Invalid): error when creating
> "/home/jenkins/jenkins-slave/workspace/beam_PerformanceTests_Kafka_IO/src/.test-infra/kubernetes/kafka-cluster/04-outside-services/outside-0.yml":
> Service "outside-0" is invalid: spec.ports[0].nodePort: Invalid value:
> 32400: provided port is already allocated
> 18:55:33 Error from server (Invalid): error when creating
> "/home/jenkins/jenkins-slave/workspace/beam_PerformanceTests_Kafka_IO/src/.test-infra/kubernetes/kafka-cluster/04-outside-services/outside-1.yml":
> Service "outside-1" is invalid: spec.ports[0].nodePort: Invalid value:
> 32401: provided port is already allocated
> 18:55:33 Error from server (Invalid): error when creating
> "/home/jenkins/jenkins-slave/workspace/beam_PerformanceTests_Kafka_IO/src/.test-infra/kubernetes/kafka-cluster/04-outside-services/outside-2.yml":
> Service "outside-2" is invalid: spec.ports[0].nodePort: Invalid value:
> 32402: provided port is already allocated
> 1
>
> Seems like we tried three ports but they were being used. Probably we should
> update code to find an unused port dynamically.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
