[
https://issues.apache.org/jira/browse/KAFKA-3213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gwen Shapira updated KAFKA-3213:
--------------------------------
Description:
Looking at logs of attempt to reconfigure connector while leader is restarting,
I see:
[code]
[2016-01-29 20:31:01,799] ERROR IO error forwarding REST request:
(org.apache.kafka.connect.runtime.rest.RestServer)
java.net.ConnectException: Connection refused
[2016-01-29 20:31:01,802] ERROR Request to leader to reconfigure connector
tasks failed (org.apache.kafka.connect.runtime.distributed.DistributedHerder)
org.apache.kafka.connect.runtime.rest.errors.ConnectRestException: IO Error
trying to forward REST request: Connection refused
[2016-01-29 20:31:01,802] ERROR Failed to reconfigure connector's tasks,
retrying after backoff:
(org.apache.kafka.connect.runtime.distributed.DistributedHerder)
org.apache.kafka.connect.runtime.rest.errors.ConnectRestException: IO Error
trying to forward REST request: Connection refused
[2016-01-29 20:31:01,803] DEBUG Sending POST with input
[{"tables":"bar","table.poll.interval.ms":"1000","incrementing.column.name":"id","connection.url":"jdbc:mysql://worker1:3306/testdb?user=root","name":"test-mysql-jdbc","tasks.max":"3","task.class":"io.confluent.connect.jdbc.JdbcSourceTask","poll.interval.ms":"1000","topic.prefix":"test-","connector.class":"io.confluent.connect.jdbc.JdbcSourceConnector","mode":"incrementing"},{"tables":"foo","table.poll.interval.ms":"1000","incrementing.column.name":"id","connection.url":"jdbc:mysql://worker1:3306/testdb?user=root","name":"test-mysql-jdbc","tasks.max":"3","task.class":"io.confluent.connect.jdbc.JdbcSourceTask","poll.interval.ms":"1000","topic.prefix":"test-","connector.class":"io.confluent.connect.jdbc.JdbcSourceConnector","mode":"incrementing"}]
to http://worker2:8083/connectors/test-mysql-jdbc/tasks
(org.apache.kafka.connect.runtime.rest.RestServer)
[2016-01-29 20:31:01,803] ERROR IO error forwarding REST request:
(org.apache.kafka.connect.runtime.rest.RestServer)
java.net.ConnectException: Connection refused
[2016-01-29 20:31:01,804] ERROR Request to leader to reconfigure connector
tasks failed (org.apache.kafka.connect.runtime.distributed.DistributedHerder)
org.apache.kafka.connect.runtime.rest.errors.ConnectRestException: IO Error
trying to forward REST request: Connection refused
[code]
Note that it looks like we are retrying every 1ms, while I'd expect a retry
every 250ms.
was:
Looking at logs of attempt to reconfigure connector while leader is restarting,
I see:
````
[2016-01-29 20:31:01,799] ERROR IO error forwarding REST request:
(org.apache.kafka.connect.runtime.rest.RestServer)
java.net.ConnectException: Connection refused
[2016-01-29 20:31:01,802] ERROR Request to leader to reconfigure connector
tasks failed (org.apache.kafka.connect.runtime.distributed.DistributedHerder)
org.apache.kafka.connect.runtime.rest.errors.ConnectRestException: IO Error
trying to forward REST request: Connection refused
[2016-01-29 20:31:01,802] ERROR Failed to reconfigure connector's tasks,
retrying after backoff:
(org.apache.kafka.connect.runtime.distributed.DistributedHerder)
org.apache.kafka.connect.runtime.rest.errors.ConnectRestException: IO Error
trying to forward REST request: Connection refused
[2016-01-29 20:31:01,803] DEBUG Sending POST with input
[{"tables":"bar","table.poll.interval.ms":"1000","incrementing.column.name":"id","connection.url":"jdbc:mysql://worker1:3306/testdb?user=root","name":"test-mysql-jdbc","tasks.max":"3","task.class":"io.confluent.connect.jdbc.JdbcSourceTask","poll.interval.ms":"1000","topic.prefix":"test-","connector.class":"io.confluent.connect.jdbc.JdbcSourceConnector","mode":"incrementing"},{"tables":"foo","table.poll.interval.ms":"1000","incrementing.column.name":"id","connection.url":"jdbc:mysql://worker1:3306/testdb?user=root","name":"test-mysql-jdbc","tasks.max":"3","task.class":"io.confluent.connect.jdbc.JdbcSourceTask","poll.interval.ms":"1000","topic.prefix":"test-","connector.class":"io.confluent.connect.jdbc.JdbcSourceConnector","mode":"incrementing"}]
to http://worker2:8083/connectors/test-mysql-jdbc/tasks
(org.apache.kafka.connect.runtime.rest.RestServer)
[2016-01-29 20:31:01,803] ERROR IO error forwarding REST request:
(org.apache.kafka.connect.runtime.rest.RestServer)
java.net.ConnectException: Connection refused
[2016-01-29 20:31:01,804] ERROR Request to leader to reconfigure connector
tasks failed (org.apache.kafka.connect.runtime.distributed.DistributedHerder)
org.apache.kafka.connect.runtime.rest.errors.ConnectRestException: IO Error
trying to forward REST request: Connection refused
````
Note that it looks like we are retrying every 1ms, while I'd expect a retry
every 250ms.
> [CONNECT] It looks like we are not backing off properly when reconfiguring
> tasks
> --------------------------------------------------------------------------------
>
> Key: KAFKA-3213
> URL: https://issues.apache.org/jira/browse/KAFKA-3213
> Project: Kafka
> Issue Type: Bug
> Components: copycat
> Reporter: Gwen Shapira
> Assignee: Ewen Cheslack-Postava
>
> Looking at logs of attempt to reconfigure connector while leader is
> restarting, I see:
> [code]
> [2016-01-29 20:31:01,799] ERROR IO error forwarding REST request:
> (org.apache.kafka.connect.runtime.rest.RestServer)
> java.net.ConnectException: Connection refused
> [2016-01-29 20:31:01,802] ERROR Request to leader to reconfigure connector
> tasks failed (org.apache.kafka.connect.runtime.distributed.DistributedHerder)
> org.apache.kafka.connect.runtime.rest.errors.ConnectRestException: IO Error
> trying to forward REST request: Connection refused
> [2016-01-29 20:31:01,802] ERROR Failed to reconfigure connector's tasks,
> retrying after backoff:
> (org.apache.kafka.connect.runtime.distributed.DistributedHerder)
> org.apache.kafka.connect.runtime.rest.errors.ConnectRestException: IO Error
> trying to forward REST request: Connection refused
> [2016-01-29 20:31:01,803] DEBUG Sending POST with input
> [{"tables":"bar","table.poll.interval.ms":"1000","incrementing.column.name":"id","connection.url":"jdbc:mysql://worker1:3306/testdb?user=root","name":"test-mysql-jdbc","tasks.max":"3","task.class":"io.confluent.connect.jdbc.JdbcSourceTask","poll.interval.ms":"1000","topic.prefix":"test-","connector.class":"io.confluent.connect.jdbc.JdbcSourceConnector","mode":"incrementing"},{"tables":"foo","table.poll.interval.ms":"1000","incrementing.column.name":"id","connection.url":"jdbc:mysql://worker1:3306/testdb?user=root","name":"test-mysql-jdbc","tasks.max":"3","task.class":"io.confluent.connect.jdbc.JdbcSourceTask","poll.interval.ms":"1000","topic.prefix":"test-","connector.class":"io.confluent.connect.jdbc.JdbcSourceConnector","mode":"incrementing"}]
> to http://worker2:8083/connectors/test-mysql-jdbc/tasks
> (org.apache.kafka.connect.runtime.rest.RestServer)
> [2016-01-29 20:31:01,803] ERROR IO error forwarding REST request:
> (org.apache.kafka.connect.runtime.rest.RestServer)
> java.net.ConnectException: Connection refused
> [2016-01-29 20:31:01,804] ERROR Request to leader to reconfigure connector
> tasks failed (org.apache.kafka.connect.runtime.distributed.DistributedHerder)
> org.apache.kafka.connect.runtime.rest.errors.ConnectRestException: IO Error
> trying to forward REST request: Connection refused
> [code]
> Note that it looks like we are retrying every 1ms, while I'd expect a retry
> every 250ms.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
