[
https://issues.apache.org/jira/browse/FLINK-37436?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yubiao Feng updated FLINK-37436:
--------------------------------
Description:
h3. Background 1: dynamically create a Pulsar Topic by Flink connector-pulsar
Flink connector-pulsar provided a way to dynamically create a Pulsar Topic when
DynamicTopicRouter returns a non-existing one. see also:
[flink-connector-pulsar/flink-connector-pulsar/src/main/java/org/apache/flink/connector/pulsar/sink/writer/topic/ProducerRegister.java
at main ·
apache/flink-connector-pulsar|https://github.com/apache/flink-connector-pulsar/blob/main/flink-connector-pulsar/src/main/java/org/apache/flink/connector/pulsar/sink/writer/topic/ProducerRegister.java#L221].
* `pulsarClient.getPartitionsForTopic(topic)` will create a topic
automatically if it does not exist.
—
h3. Background 2: how dynamically created topics in Pulsar Server
- There is a config named `allowAutoTopicCreationType`, which can be set to `
partitioned` or `non-partitioned`
- If it was set `partitioned`, Pulsar will create a partitioned topic with `
\{defaultNumPartitions}` partitions. For example, Pulsar will create topics
named `\{tenant}/\{namespace}/\{topic name}-partition-0` and
`\{tenant}/\{namespace}/\{topic name}-partition-1`, and create a relationship
between them, which indicates they are in a same partitioned topic.
- If it was set `non-partitioned`, Pulsar will create a non-partitioned topic.
Pulsar will create topics named `\{tenant}/\{namespace}/\{topic name}`, which
does not include a suffix `partition-\{num}`
—
h3. Issue:
- if `pulsarClient.getPartitionsForTopic(topic)` get a param
`\{tenant}/\{namespace}/\{topic name}-partition-0`, which includes the suffix
`partition-0`, Pulsar will create a non-partitioned topic named
`\{tenant}/\{namespace}/\{topic name}-partition-0`
- After you call `pulsarClient.getPartitionsForTopic(topic)` with a param
`\{tenant}/\{namespace}/\{topic name}-partition-1`, you will get two partitions
named `\{tenant}/\{namespace}/\{topic name}-partition-0` and
`\{tenant}/\{namespace}/\{topic name}-partition-1`, but there is no
relationship record between them.
was:
Background 1: dynamically create a Pulsar Topic by Flink connector-pulsar
Flink connector-pulsar provided a way to dynamically create a Pulsar Topic when
DynamicTopicRouter returns a non-existing one. see also:
[flink-connector-pulsar/flink-connector-pulsar/src/main/java/org/apache/flink/connector/pulsar/sink/writer/topic/ProducerRegister.java
at main ·
apache/flink-connector-pulsar|https://github.com/apache/flink-connector-pulsar/blob/main/flink-connector-pulsar/src/main/java/org/apache/flink/connector/pulsar/sink/writer/topic/ProducerRegister.java#L221].
* `pulsarClient.getPartitionsForTopic(topic)` will create a topic
automatically if it does not exist.
---
Background 2: how dynamically created topics in Pulsar Server
- There is a config named `allowAutoTopicCreationType`, which can be set to `
partitioned` or `non-partitioned`
- If it was set `partitioned`, Pulsar will create a partitioned topic with
`{defaultNumPartitions}` partitions. For example, Pulsar will create topics
named `\{tenant}/\{namespace}/\{topic name}-partition-0` and
`\{tenant}/\{namespace}/\{topic name}-partition-1`, and create a relationship
between them, which indicates they are in a same partitioned topic.
- If it was set `non-partitioned`, Pulsar will create a non-partitioned topic.
Pulsar will create topics named `\{tenant}/\{namespace}/\{topic name}`, which
does not include a suffix `partition-\{num}`
---
Issue:
- if `pulsarClient.getPartitionsForTopic(topic)` get a param
`\{tenant}/\{namespace}/\{topic name}-partition-0`, which includes the suffix
`partition-0`, Pulsar will create a non-partitioned topic named
`\{tenant}/\{namespace}/\{topic name}-partition-0`
- After you call `pulsarClient.getPartitionsForTopic(topic)` with a param
`\{tenant}/\{namespace}/\{topic name}-partition-1`, you will get two partitions
named `\{tenant}/\{namespace}/\{topic name}-partition-0` and
`\{tenant}/\{namespace}/\{topic name}-partition-1`, but there is no
relationship record between them.
> flink-connector-pulsar used a incorrect API when dynamically creating topics
> by DynamicTopicRouter
> --------------------------------------------------------------------------------------------------
>
> Key: FLINK-37436
> URL: https://issues.apache.org/jira/browse/FLINK-37436
> Project: Flink
> Issue Type: Bug
> Components: Connectors / Pulsar
> Reporter: Yubiao Feng
> Priority: Major
>
> h3. Background 1: dynamically create a Pulsar Topic by Flink connector-pulsar
>
> Flink connector-pulsar provided a way to dynamically create a Pulsar Topic
> when DynamicTopicRouter returns a non-existing one. see also:
> [flink-connector-pulsar/flink-connector-pulsar/src/main/java/org/apache/flink/connector/pulsar/sink/writer/topic/ProducerRegister.java
> at main ·
> apache/flink-connector-pulsar|https://github.com/apache/flink-connector-pulsar/blob/main/flink-connector-pulsar/src/main/java/org/apache/flink/connector/pulsar/sink/writer/topic/ProducerRegister.java#L221].
>
> * `pulsarClient.getPartitionsForTopic(topic)` will create a topic
> automatically if it does not exist.
> —
>
> h3. Background 2: how dynamically created topics in Pulsar Server
> - There is a config named `allowAutoTopicCreationType`, which can be set to `
> partitioned` or `non-partitioned`
> - If it was set `partitioned`, Pulsar will create a partitioned topic with `
> \{defaultNumPartitions}` partitions. For example, Pulsar will create topics
> named `\{tenant}/\{namespace}/\{topic name}-partition-0` and
> `\{tenant}/\{namespace}/\{topic name}-partition-1`, and create a relationship
> between them, which indicates they are in a same partitioned topic.
> - If it was set `non-partitioned`, Pulsar will create a non-partitioned
> topic. Pulsar will create topics named `\{tenant}/\{namespace}/\{topic
> name}`, which does not include a suffix `partition-\{num}`
>
> —
> h3. Issue:
> - if `pulsarClient.getPartitionsForTopic(topic)` get a param
> `\{tenant}/\{namespace}/\{topic name}-partition-0`, which includes the suffix
> `partition-0`, Pulsar will create a non-partitioned topic named
> `\{tenant}/\{namespace}/\{topic name}-partition-0`
> - After you call `pulsarClient.getPartitionsForTopic(topic)` with a param
> `\{tenant}/\{namespace}/\{topic name}-partition-1`, you will get two
> partitions named `\{tenant}/\{namespace}/\{topic name}-partition-0` and
> `\{tenant}/\{namespace}/\{topic name}-partition-1`, but there is no
> relationship record between them.
>
>
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
