[
https://issues.apache.org/jira/browse/KAFKA-14750?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergei Morozov updated KAFKA-14750:
-----------------------------------
Description:
Steps to reproduce:
# In {{{}config/connect-standalone.properties{}}}, set:
{code:bash}
plugin.path=libs/connect-file-3.3.1.jar
{code}
# In {{{}config/connect-file-sink.properties{}}}, remove the {{topics=}} line
and add this one:
{code:bash}
topics.regex=connect-test-.*
{code}
# Start zookeeper:
{code:bash}
bin/zookeeper-server-start.sh config/zookeeper.properties
{code}
# Start the brokers:
{code:bash}
bin/kafka-server-start.sh config/server.properties
{code}
# Start the file sink connector:
{code:bash}
bin/connect-standalone.sh config/connect-standalone.properties
config/connect-file-sink.properties
{code}
# Create topics for the sink connector to subscribe to:
{code:bash}
for i in {0..2}; do
for j in $(seq $(($i * 100)) $(( ($i + 1) * 100 - 1 ))); do
bin/kafka-topics.sh \
--bootstrap-server localhost:9092 \
--create \
--topic connect-test-$j
done &
done
wait
{code}
# Wait until all the created topics are assigned to the connector. Check the
number of partitions to be > 0 in the output of:
{code:bash}
bin/kafka-consumer-groups.sh \
--bootstrap-server localhost:9092 \
--group connect-local-file-sink \
--describe --members
{code}
# Delete the created topics:
{code:bash}
for i in {0..2}; do
for j in $(seq $(($i * 100)) $(( ($i + 1) * 100 - 1 ))); do
bin/kafka-topics.sh \
--bootstrap-server localhost:9092 \
--delete \
--topic connect-test-$j
echo Deleted topic connect-test-$j.
done &
done
wait
{code}
# Observe the connector fail with the following error:
{quote}org.apache.kafka.common.errors.TimeoutException: Timeout of 60000ms
expired before the position for partition connect-test-211-0 could be determined
{quote}
was:
Steps to reproduce:
# In {{{}config/connect-standalone.properties{}}}, set:
{code:java}
plugin.path=libs/connect-file-3.3.1.jar
{code}
# In {{{}config/connect-file-sink.properties{}}}, remove the {{topics=}} line
and add this one:
{code:java}
topics.regex=connect-test-.*
{code}
# Start zookeeper:
{code:java}
bin/zookeeper-server-start.sh config/zookeeper.properties
{code}
# Start the brokers:
{code:java}
bin/kafka-server-start.sh config/server.properties
{code}
# Start the file sink connector:
{code:java}
bin/connect-standalone.sh config/connect-standalone.properties
config/connect-file-sink.properties
{code}
# Create topics for the sink connector to subscribe to:
{code:java}
for i in \{0..2}; do
for j in {$(($i * 100))..$(( ($i + 1) * 100 - 1 ))}; do
bin/kafka-topics.sh \
--bootstrap-server localhost:9092 \
--create \
--topic connect-test-$j
done &
done
wait
{code}
# Wait until all the created topics are assigned to the connector. Check the
number of partitions to be > 0 in the output of:
{code:java}
bin/kafka-consumer-groups.sh \
--bootstrap-server localhost:9092 \
--group connect-local-file-sink \
--describe --members
{code}
# Delete the created topics:
{code:java}
for i in \{0..2}; do
for j in {$(($i * 100))..$(( ($i + 1) * 100 - 1 ))}; do
bin/kafka-topics.sh \
--bootstrap-server localhost:9092 \
--delete \
--topic connect-test-$j
echo Deleted topic connect-test-$j.
done &
done
wait
{code}
# Observe the connector fail with the following error:
{quote}org.apache.kafka.common.errors.TimeoutException: Timeout of 60000ms
expired before the position for partition connect-test-211-0 could be determined
{quote}
> Sink connector fails if a topic matching its topics.regex gets deleted
> ----------------------------------------------------------------------
>
> Key: KAFKA-14750
> URL: https://issues.apache.org/jira/browse/KAFKA-14750
> Project: Kafka
> Issue Type: Bug
> Components: KafkaConnect
> Affects Versions: 3.3.1
> Reporter: Sergei Morozov
> Priority: Major
>
> Steps to reproduce:
> # In {{{}config/connect-standalone.properties{}}}, set:
> {code:bash}
> plugin.path=libs/connect-file-3.3.1.jar
> {code}
> # In {{{}config/connect-file-sink.properties{}}}, remove the {{topics=}} line
> and add this one:
> {code:bash}
> topics.regex=connect-test-.*
> {code}
> # Start zookeeper:
> {code:bash}
> bin/zookeeper-server-start.sh config/zookeeper.properties
> {code}
> # Start the brokers:
> {code:bash}
> bin/kafka-server-start.sh config/server.properties
> {code}
> # Start the file sink connector:
> {code:bash}
> bin/connect-standalone.sh config/connect-standalone.properties
> config/connect-file-sink.properties
> {code}
> # Create topics for the sink connector to subscribe to:
> {code:bash}
> for i in {0..2}; do
> for j in $(seq $(($i * 100)) $(( ($i + 1) * 100 - 1 ))); do
> bin/kafka-topics.sh \
> --bootstrap-server localhost:9092 \
> --create \
> --topic connect-test-$j
> done &
> done
> wait
> {code}
> # Wait until all the created topics are assigned to the connector. Check the
> number of partitions to be > 0 in the output of:
> {code:bash}
> bin/kafka-consumer-groups.sh \
> --bootstrap-server localhost:9092 \
> --group connect-local-file-sink \
> --describe --members
> {code}
> # Delete the created topics:
> {code:bash}
> for i in {0..2}; do
> for j in $(seq $(($i * 100)) $(( ($i + 1) * 100 - 1 ))); do
> bin/kafka-topics.sh \
> --bootstrap-server localhost:9092 \
> --delete \
> --topic connect-test-$j
> echo Deleted topic connect-test-$j.
> done &
> done
> wait
> {code}
> # Observe the connector fail with the following error:
> {quote}org.apache.kafka.common.errors.TimeoutException: Timeout of 60000ms
> expired before the position for partition connect-test-211-0 could be
> determined
> {quote}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
