Yubiao Feng created FLINK-37436:
-----------------------------------
Summary: flink-connector-pulsar used a incorrect API when
dynamically creating topics by DynamicTopicRouter
Key: FLINK-37436
URL: https://issues.apache.org/jira/browse/FLINK-37436
Project: Flink
Issue Type: Bug
Components: Connectors / Pulsar
Reporter: Yubiao Feng
Background 1: dynamically create a Pulsar Topic by Flink connector-pulsar
Flink connector-pulsar provided a way to dynamically create a Pulsar Topic when
DynamicTopicRouter returns a non-existing one. see also:
[flink-connector-pulsar/flink-connector-pulsar/src/main/java/org/apache/flink/connector/pulsar/sink/writer/topic/ProducerRegister.java
at main ·
apache/flink-connector-pulsar|https://github.com/apache/flink-connector-pulsar/blob/main/flink-connector-pulsar/src/main/java/org/apache/flink/connector/pulsar/sink/writer/topic/ProducerRegister.java#L221].
* `pulsarClient.getPartitionsForTopic(topic)` will create a topic
automatically if it does not exist.
---
Background 2: how dynamically created topics in Pulsar Server
- There is a config named `allowAutoTopicCreationType`, which can be set to `
partitioned` or `non-partitioned`
- If it was set `partitioned`, Pulsar will create a partitioned topic with
`{defaultNumPartitions}` partitions. For example, Pulsar will create topics
named `\{tenant}/\{namespace}/\{topic name}-partition-0` and
`\{tenant}/\{namespace}/\{topic name}-partition-1`, and create a relationship
between them, which indicates they are in a same partitioned topic.
- If it was set `non-partitioned`, Pulsar will create a non-partitioned topic.
Pulsar will create topics named `\{tenant}/\{namespace}/\{topic name}`, which
does not include a suffix `partition-\{num}`
---
Issue:
- if `pulsarClient.getPartitionsForTopic(topic)` get a param
`\{tenant}/\{namespace}/\{topic name}-partition-0`, which includes the suffix
`partition-0`, Pulsar will create a non-partitioned topic named
`\{tenant}/\{namespace}/\{topic name}-partition-0`
- After you call `pulsarClient.getPartitionsForTopic(topic)` with a param
`\{tenant}/\{namespace}/\{topic name}-partition-1`, you will get two partitions
named `\{tenant}/\{namespace}/\{topic name}-partition-0` and
`\{tenant}/\{namespace}/\{topic name}-partition-1`, but there is no
relationship record between them.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
