[jira] [Updated] (KAFKA-3213) [CONNECT] It looks like we are not backing off properly when reconfiguring tasks
[ https://issues.apache.org/jira/browse/KAFKA-3213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewen Cheslack-Postava updated KAFKA-3213: - Priority: Major (was: Critical) > [CONNECT] It looks like we are not backing off properly when reconfiguring > tasks > > > Key: KAFKA-3213 > URL: https://issues.apache.org/jira/browse/KAFKA-3213 > Project: Kafka > Issue Type: Bug > Components: KafkaConnect >Reporter: Gwen Shapira >Assignee: Liquan Pei > Fix For: 0.10.2.0 > > > Looking at logs of attempt to reconfigure connector while leader is > restarting, I see: > {code} > [2016-01-29 20:31:01,799] ERROR IO error forwarding REST request: > (org.apache.kafka.connect.runtime.rest.RestServer) > java.net.ConnectException: Connection refused > [2016-01-29 20:31:01,802] ERROR Request to leader to reconfigure connector > tasks failed (org.apache.kafka.connect.runtime.distributed.DistributedHerder) > org.apache.kafka.connect.runtime.rest.errors.ConnectRestException: IO Error > trying to forward REST request: Connection refused > [2016-01-29 20:31:01,802] ERROR Failed to reconfigure connector's tasks, > retrying after backoff: > (org.apache.kafka.connect.runtime.distributed.DistributedHerder) > org.apache.kafka.connect.runtime.rest.errors.ConnectRestException: IO Error > trying to forward REST request: Connection refused > [2016-01-29 20:31:01,803] DEBUG Sending POST with input > [{"tables":"bar","table.poll.interval.ms":"1000","incrementing.column.name":"id","connection.url":"jdbc:mysql://worker1:3306/testdb?user=root","name":"test-mysql-jdbc","tasks.max":"3","task.class":"io.confluent.connect.jdbc.JdbcSourceTask","poll.interval.ms":"1000","topic.prefix":"test-","connector.class":"io.confluent.connect.jdbc.JdbcSourceConnector","mode":"incrementing"},{"tables":"foo","table.poll.interval.ms":"1000","incrementing.column.name":"id","connection.url":"jdbc:mysql://worker1:3306/testdb?user=root","name":"test-mysql-jdbc","tasks.max":"3","task.class":"io.confluent.connect.jdbc.JdbcSourceTask","poll.interval.ms":"1000","topic.prefix":"test-","connector.class":"io.confluent.connect.jdbc.JdbcSourceConnector","mode":"incrementing"}] > to http://worker2:8083/connectors/test-mysql-jdbc/tasks > (org.apache.kafka.connect.runtime.rest.RestServer) > [2016-01-29 20:31:01,803] ERROR IO error forwarding REST request: > (org.apache.kafka.connect.runtime.rest.RestServer) > java.net.ConnectException: Connection refused > [2016-01-29 20:31:01,804] ERROR Request to leader to reconfigure connector > tasks failed (org.apache.kafka.connect.runtime.distributed.DistributedHerder) > org.apache.kafka.connect.runtime.rest.errors.ConnectRestException: IO Error > trying to forward REST request: Connection refused > {code} > Note that it looks like we are retrying every 1ms, while I'd expect a retry > every 250ms. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KAFKA-3213) [CONNECT] It looks like we are not backing off properly when reconfiguring tasks
[ https://issues.apache.org/jira/browse/KAFKA-3213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-3213: - Fix Version/s: (was: 0.10.1.1) 0.10.2.0 > [CONNECT] It looks like we are not backing off properly when reconfiguring > tasks > > > Key: KAFKA-3213 > URL: https://issues.apache.org/jira/browse/KAFKA-3213 > Project: Kafka > Issue Type: Bug > Components: KafkaConnect >Reporter: Gwen Shapira >Assignee: Liquan Pei >Priority: Critical > Fix For: 0.10.2.0 > > > Looking at logs of attempt to reconfigure connector while leader is > restarting, I see: > {code} > [2016-01-29 20:31:01,799] ERROR IO error forwarding REST request: > (org.apache.kafka.connect.runtime.rest.RestServer) > java.net.ConnectException: Connection refused > [2016-01-29 20:31:01,802] ERROR Request to leader to reconfigure connector > tasks failed (org.apache.kafka.connect.runtime.distributed.DistributedHerder) > org.apache.kafka.connect.runtime.rest.errors.ConnectRestException: IO Error > trying to forward REST request: Connection refused > [2016-01-29 20:31:01,802] ERROR Failed to reconfigure connector's tasks, > retrying after backoff: > (org.apache.kafka.connect.runtime.distributed.DistributedHerder) > org.apache.kafka.connect.runtime.rest.errors.ConnectRestException: IO Error > trying to forward REST request: Connection refused > [2016-01-29 20:31:01,803] DEBUG Sending POST with input > [{"tables":"bar","table.poll.interval.ms":"1000","incrementing.column.name":"id","connection.url":"jdbc:mysql://worker1:3306/testdb?user=root","name":"test-mysql-jdbc","tasks.max":"3","task.class":"io.confluent.connect.jdbc.JdbcSourceTask","poll.interval.ms":"1000","topic.prefix":"test-","connector.class":"io.confluent.connect.jdbc.JdbcSourceConnector","mode":"incrementing"},{"tables":"foo","table.poll.interval.ms":"1000","incrementing.column.name":"id","connection.url":"jdbc:mysql://worker1:3306/testdb?user=root","name":"test-mysql-jdbc","tasks.max":"3","task.class":"io.confluent.connect.jdbc.JdbcSourceTask","poll.interval.ms":"1000","topic.prefix":"test-","connector.class":"io.confluent.connect.jdbc.JdbcSourceConnector","mode":"incrementing"}] > to http://worker2:8083/connectors/test-mysql-jdbc/tasks > (org.apache.kafka.connect.runtime.rest.RestServer) > [2016-01-29 20:31:01,803] ERROR IO error forwarding REST request: > (org.apache.kafka.connect.runtime.rest.RestServer) > java.net.ConnectException: Connection refused > [2016-01-29 20:31:01,804] ERROR Request to leader to reconfigure connector > tasks failed (org.apache.kafka.connect.runtime.distributed.DistributedHerder) > org.apache.kafka.connect.runtime.rest.errors.ConnectRestException: IO Error > trying to forward REST request: Connection refused > {code} > Note that it looks like we are retrying every 1ms, while I'd expect a retry > every 250ms. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KAFKA-3213) [CONNECT] It looks like we are not backing off properly when reconfiguring tasks
[ https://issues.apache.org/jira/browse/KAFKA-3213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson updated KAFKA-3213: --- Fix Version/s: (was: 0.10.1.0) 0.10.1.1 > [CONNECT] It looks like we are not backing off properly when reconfiguring > tasks > > > Key: KAFKA-3213 > URL: https://issues.apache.org/jira/browse/KAFKA-3213 > Project: Kafka > Issue Type: Bug > Components: KafkaConnect >Reporter: Gwen Shapira >Assignee: Liquan Pei >Priority: Critical > Fix For: 0.10.1.1 > > > Looking at logs of attempt to reconfigure connector while leader is > restarting, I see: > {code} > [2016-01-29 20:31:01,799] ERROR IO error forwarding REST request: > (org.apache.kafka.connect.runtime.rest.RestServer) > java.net.ConnectException: Connection refused > [2016-01-29 20:31:01,802] ERROR Request to leader to reconfigure connector > tasks failed (org.apache.kafka.connect.runtime.distributed.DistributedHerder) > org.apache.kafka.connect.runtime.rest.errors.ConnectRestException: IO Error > trying to forward REST request: Connection refused > [2016-01-29 20:31:01,802] ERROR Failed to reconfigure connector's tasks, > retrying after backoff: > (org.apache.kafka.connect.runtime.distributed.DistributedHerder) > org.apache.kafka.connect.runtime.rest.errors.ConnectRestException: IO Error > trying to forward REST request: Connection refused > [2016-01-29 20:31:01,803] DEBUG Sending POST with input > [{"tables":"bar","table.poll.interval.ms":"1000","incrementing.column.name":"id","connection.url":"jdbc:mysql://worker1:3306/testdb?user=root","name":"test-mysql-jdbc","tasks.max":"3","task.class":"io.confluent.connect.jdbc.JdbcSourceTask","poll.interval.ms":"1000","topic.prefix":"test-","connector.class":"io.confluent.connect.jdbc.JdbcSourceConnector","mode":"incrementing"},{"tables":"foo","table.poll.interval.ms":"1000","incrementing.column.name":"id","connection.url":"jdbc:mysql://worker1:3306/testdb?user=root","name":"test-mysql-jdbc","tasks.max":"3","task.class":"io.confluent.connect.jdbc.JdbcSourceTask","poll.interval.ms":"1000","topic.prefix":"test-","connector.class":"io.confluent.connect.jdbc.JdbcSourceConnector","mode":"incrementing"}] > to http://worker2:8083/connectors/test-mysql-jdbc/tasks > (org.apache.kafka.connect.runtime.rest.RestServer) > [2016-01-29 20:31:01,803] ERROR IO error forwarding REST request: > (org.apache.kafka.connect.runtime.rest.RestServer) > java.net.ConnectException: Connection refused > [2016-01-29 20:31:01,804] ERROR Request to leader to reconfigure connector > tasks failed (org.apache.kafka.connect.runtime.distributed.DistributedHerder) > org.apache.kafka.connect.runtime.rest.errors.ConnectRestException: IO Error > trying to forward REST request: Connection refused > {code} > Note that it looks like we are retrying every 1ms, while I'd expect a retry > every 250ms. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KAFKA-3213) [CONNECT] It looks like we are not backing off properly when reconfiguring tasks
[ https://issues.apache.org/jira/browse/KAFKA-3213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewen Cheslack-Postava updated KAFKA-3213: - Fix Version/s: 0.10.1.0 > [CONNECT] It looks like we are not backing off properly when reconfiguring > tasks > > > Key: KAFKA-3213 > URL: https://issues.apache.org/jira/browse/KAFKA-3213 > Project: Kafka > Issue Type: Bug > Components: KafkaConnect >Reporter: Gwen Shapira >Assignee: Liquan Pei >Priority: Critical > Fix For: 0.10.1.0 > > > Looking at logs of attempt to reconfigure connector while leader is > restarting, I see: > {code} > [2016-01-29 20:31:01,799] ERROR IO error forwarding REST request: > (org.apache.kafka.connect.runtime.rest.RestServer) > java.net.ConnectException: Connection refused > [2016-01-29 20:31:01,802] ERROR Request to leader to reconfigure connector > tasks failed (org.apache.kafka.connect.runtime.distributed.DistributedHerder) > org.apache.kafka.connect.runtime.rest.errors.ConnectRestException: IO Error > trying to forward REST request: Connection refused > [2016-01-29 20:31:01,802] ERROR Failed to reconfigure connector's tasks, > retrying after backoff: > (org.apache.kafka.connect.runtime.distributed.DistributedHerder) > org.apache.kafka.connect.runtime.rest.errors.ConnectRestException: IO Error > trying to forward REST request: Connection refused > [2016-01-29 20:31:01,803] DEBUG Sending POST with input > [{"tables":"bar","table.poll.interval.ms":"1000","incrementing.column.name":"id","connection.url":"jdbc:mysql://worker1:3306/testdb?user=root","name":"test-mysql-jdbc","tasks.max":"3","task.class":"io.confluent.connect.jdbc.JdbcSourceTask","poll.interval.ms":"1000","topic.prefix":"test-","connector.class":"io.confluent.connect.jdbc.JdbcSourceConnector","mode":"incrementing"},{"tables":"foo","table.poll.interval.ms":"1000","incrementing.column.name":"id","connection.url":"jdbc:mysql://worker1:3306/testdb?user=root","name":"test-mysql-jdbc","tasks.max":"3","task.class":"io.confluent.connect.jdbc.JdbcSourceTask","poll.interval.ms":"1000","topic.prefix":"test-","connector.class":"io.confluent.connect.jdbc.JdbcSourceConnector","mode":"incrementing"}] > to http://worker2:8083/connectors/test-mysql-jdbc/tasks > (org.apache.kafka.connect.runtime.rest.RestServer) > [2016-01-29 20:31:01,803] ERROR IO error forwarding REST request: > (org.apache.kafka.connect.runtime.rest.RestServer) > java.net.ConnectException: Connection refused > [2016-01-29 20:31:01,804] ERROR Request to leader to reconfigure connector > tasks failed (org.apache.kafka.connect.runtime.distributed.DistributedHerder) > org.apache.kafka.connect.runtime.rest.errors.ConnectRestException: IO Error > trying to forward REST request: Connection refused > {code} > Note that it looks like we are retrying every 1ms, while I'd expect a retry > every 250ms. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KAFKA-3213) [CONNECT] It looks like we are not backing off properly when reconfiguring tasks
[ https://issues.apache.org/jira/browse/KAFKA-3213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewen Cheslack-Postava updated KAFKA-3213: - Priority: Critical (was: Major) > [CONNECT] It looks like we are not backing off properly when reconfiguring > tasks > > > Key: KAFKA-3213 > URL: https://issues.apache.org/jira/browse/KAFKA-3213 > Project: Kafka > Issue Type: Bug > Components: KafkaConnect >Reporter: Gwen Shapira >Assignee: Liquan Pei >Priority: Critical > > Looking at logs of attempt to reconfigure connector while leader is > restarting, I see: > {code} > [2016-01-29 20:31:01,799] ERROR IO error forwarding REST request: > (org.apache.kafka.connect.runtime.rest.RestServer) > java.net.ConnectException: Connection refused > [2016-01-29 20:31:01,802] ERROR Request to leader to reconfigure connector > tasks failed (org.apache.kafka.connect.runtime.distributed.DistributedHerder) > org.apache.kafka.connect.runtime.rest.errors.ConnectRestException: IO Error > trying to forward REST request: Connection refused > [2016-01-29 20:31:01,802] ERROR Failed to reconfigure connector's tasks, > retrying after backoff: > (org.apache.kafka.connect.runtime.distributed.DistributedHerder) > org.apache.kafka.connect.runtime.rest.errors.ConnectRestException: IO Error > trying to forward REST request: Connection refused > [2016-01-29 20:31:01,803] DEBUG Sending POST with input > [{"tables":"bar","table.poll.interval.ms":"1000","incrementing.column.name":"id","connection.url":"jdbc:mysql://worker1:3306/testdb?user=root","name":"test-mysql-jdbc","tasks.max":"3","task.class":"io.confluent.connect.jdbc.JdbcSourceTask","poll.interval.ms":"1000","topic.prefix":"test-","connector.class":"io.confluent.connect.jdbc.JdbcSourceConnector","mode":"incrementing"},{"tables":"foo","table.poll.interval.ms":"1000","incrementing.column.name":"id","connection.url":"jdbc:mysql://worker1:3306/testdb?user=root","name":"test-mysql-jdbc","tasks.max":"3","task.class":"io.confluent.connect.jdbc.JdbcSourceTask","poll.interval.ms":"1000","topic.prefix":"test-","connector.class":"io.confluent.connect.jdbc.JdbcSourceConnector","mode":"incrementing"}] > to http://worker2:8083/connectors/test-mysql-jdbc/tasks > (org.apache.kafka.connect.runtime.rest.RestServer) > [2016-01-29 20:31:01,803] ERROR IO error forwarding REST request: > (org.apache.kafka.connect.runtime.rest.RestServer) > java.net.ConnectException: Connection refused > [2016-01-29 20:31:01,804] ERROR Request to leader to reconfigure connector > tasks failed (org.apache.kafka.connect.runtime.distributed.DistributedHerder) > org.apache.kafka.connect.runtime.rest.errors.ConnectRestException: IO Error > trying to forward REST request: Connection refused > {code} > Note that it looks like we are retrying every 1ms, while I'd expect a retry > every 250ms. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KAFKA-3213) [CONNECT] It looks like we are not backing off properly when reconfiguring tasks
[ https://issues.apache.org/jira/browse/KAFKA-3213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gwen Shapira updated KAFKA-3213: Description: Looking at logs of attempt to reconfigure connector while leader is restarting, I see: {code} [2016-01-29 20:31:01,799] ERROR IO error forwarding REST request: (org.apache.kafka.connect.runtime.rest.RestServer) java.net.ConnectException: Connection refused [2016-01-29 20:31:01,802] ERROR Request to leader to reconfigure connector tasks failed (org.apache.kafka.connect.runtime.distributed.DistributedHerder) org.apache.kafka.connect.runtime.rest.errors.ConnectRestException: IO Error trying to forward REST request: Connection refused [2016-01-29 20:31:01,802] ERROR Failed to reconfigure connector's tasks, retrying after backoff: (org.apache.kafka.connect.runtime.distributed.DistributedHerder) org.apache.kafka.connect.runtime.rest.errors.ConnectRestException: IO Error trying to forward REST request: Connection refused [2016-01-29 20:31:01,803] DEBUG Sending POST with input [{"tables":"bar","table.poll.interval.ms":"1000","incrementing.column.name":"id","connection.url":"jdbc:mysql://worker1:3306/testdb?user=root","name":"test-mysql-jdbc","tasks.max":"3","task.class":"io.confluent.connect.jdbc.JdbcSourceTask","poll.interval.ms":"1000","topic.prefix":"test-","connector.class":"io.confluent.connect.jdbc.JdbcSourceConnector","mode":"incrementing"},{"tables":"foo","table.poll.interval.ms":"1000","incrementing.column.name":"id","connection.url":"jdbc:mysql://worker1:3306/testdb?user=root","name":"test-mysql-jdbc","tasks.max":"3","task.class":"io.confluent.connect.jdbc.JdbcSourceTask","poll.interval.ms":"1000","topic.prefix":"test-","connector.class":"io.confluent.connect.jdbc.JdbcSourceConnector","mode":"incrementing"}] to http://worker2:8083/connectors/test-mysql-jdbc/tasks (org.apache.kafka.connect.runtime.rest.RestServer) [2016-01-29 20:31:01,803] ERROR IO error forwarding REST request: (org.apache.kafka.connect.runtime.rest.RestServer) java.net.ConnectException: Connection refused [2016-01-29 20:31:01,804] ERROR Request to leader to reconfigure connector tasks failed (org.apache.kafka.connect.runtime.distributed.DistributedHerder) org.apache.kafka.connect.runtime.rest.errors.ConnectRestException: IO Error trying to forward REST request: Connection refused {code} Note that it looks like we are retrying every 1ms, while I'd expect a retry every 250ms. was: Looking at logs of attempt to reconfigure connector while leader is restarting, I see: [code] [2016-01-29 20:31:01,799] ERROR IO error forwarding REST request: (org.apache.kafka.connect.runtime.rest.RestServer) java.net.ConnectException: Connection refused [2016-01-29 20:31:01,802] ERROR Request to leader to reconfigure connector tasks failed (org.apache.kafka.connect.runtime.distributed.DistributedHerder) org.apache.kafka.connect.runtime.rest.errors.ConnectRestException: IO Error trying to forward REST request: Connection refused [2016-01-29 20:31:01,802] ERROR Failed to reconfigure connector's tasks, retrying after backoff: (org.apache.kafka.connect.runtime.distributed.DistributedHerder) org.apache.kafka.connect.runtime.rest.errors.ConnectRestException: IO Error trying to forward REST request: Connection refused [2016-01-29 20:31:01,803] DEBUG Sending POST with input [{"tables":"bar","table.poll.interval.ms":"1000","incrementing.column.name":"id","connection.url":"jdbc:mysql://worker1:3306/testdb?user=root","name":"test-mysql-jdbc","tasks.max":"3","task.class":"io.confluent.connect.jdbc.JdbcSourceTask","poll.interval.ms":"1000","topic.prefix":"test-","connector.class":"io.confluent.connect.jdbc.JdbcSourceConnector","mode":"incrementing"},{"tables":"foo","table.poll.interval.ms":"1000","incrementing.column.name":"id","connection.url":"jdbc:mysql://worker1:3306/testdb?user=root","name":"test-mysql-jdbc","tasks.max":"3","task.class":"io.confluent.connect.jdbc.JdbcSourceTask","poll.interval.ms":"1000","topic.prefix":"test-","connector.class":"io.confluent.connect.jdbc.JdbcSourceConnector","mode":"incrementing"}] to http://worker2:8083/connectors/test-mysql-jdbc/tasks (org.apache.kafka.connect.runtime.rest.RestServer) [2016-01-29 20:31:01,803] ERROR IO error forwarding REST request: (org.apache.kafka.connect.runtime.rest.RestServer) java.net.ConnectException: Connection refused [2016-01-29 20:31:01,804] ERROR Request to leader to reconfigure connector tasks failed (org.apache.kafka.connect.runtime.distributed.DistributedHerder) org.apache.kafka.connect.runtime.rest.errors.ConnectRestException: IO Error trying to forward REST request: Connection refused [code] Note that it looks like we are retrying every 1ms, while I'd expect a retry every 250ms. > [CONNECT] It looks like we are not backing off properly when reconfiguring > tasks > >
[jira] [Updated] (KAFKA-3213) [CONNECT] It looks like we are not backing off properly when reconfiguring tasks
[ https://issues.apache.org/jira/browse/KAFKA-3213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gwen Shapira updated KAFKA-3213: Description: Looking at logs of attempt to reconfigure connector while leader is restarting, I see: [code] [2016-01-29 20:31:01,799] ERROR IO error forwarding REST request: (org.apache.kafka.connect.runtime.rest.RestServer) java.net.ConnectException: Connection refused [2016-01-29 20:31:01,802] ERROR Request to leader to reconfigure connector tasks failed (org.apache.kafka.connect.runtime.distributed.DistributedHerder) org.apache.kafka.connect.runtime.rest.errors.ConnectRestException: IO Error trying to forward REST request: Connection refused [2016-01-29 20:31:01,802] ERROR Failed to reconfigure connector's tasks, retrying after backoff: (org.apache.kafka.connect.runtime.distributed.DistributedHerder) org.apache.kafka.connect.runtime.rest.errors.ConnectRestException: IO Error trying to forward REST request: Connection refused [2016-01-29 20:31:01,803] DEBUG Sending POST with input [{"tables":"bar","table.poll.interval.ms":"1000","incrementing.column.name":"id","connection.url":"jdbc:mysql://worker1:3306/testdb?user=root","name":"test-mysql-jdbc","tasks.max":"3","task.class":"io.confluent.connect.jdbc.JdbcSourceTask","poll.interval.ms":"1000","topic.prefix":"test-","connector.class":"io.confluent.connect.jdbc.JdbcSourceConnector","mode":"incrementing"},{"tables":"foo","table.poll.interval.ms":"1000","incrementing.column.name":"id","connection.url":"jdbc:mysql://worker1:3306/testdb?user=root","name":"test-mysql-jdbc","tasks.max":"3","task.class":"io.confluent.connect.jdbc.JdbcSourceTask","poll.interval.ms":"1000","topic.prefix":"test-","connector.class":"io.confluent.connect.jdbc.JdbcSourceConnector","mode":"incrementing"}] to http://worker2:8083/connectors/test-mysql-jdbc/tasks (org.apache.kafka.connect.runtime.rest.RestServer) [2016-01-29 20:31:01,803] ERROR IO error forwarding REST request: (org.apache.kafka.connect.runtime.rest.RestServer) java.net.ConnectException: Connection refused [2016-01-29 20:31:01,804] ERROR Request to leader to reconfigure connector tasks failed (org.apache.kafka.connect.runtime.distributed.DistributedHerder) org.apache.kafka.connect.runtime.rest.errors.ConnectRestException: IO Error trying to forward REST request: Connection refused [code] Note that it looks like we are retrying every 1ms, while I'd expect a retry every 250ms. was: Looking at logs of attempt to reconfigure connector while leader is restarting, I see: [2016-01-29 20:31:01,799] ERROR IO error forwarding REST request: (org.apache.kafka.connect.runtime.rest.RestServer) java.net.ConnectException: Connection refused [2016-01-29 20:31:01,802] ERROR Request to leader to reconfigure connector tasks failed (org.apache.kafka.connect.runtime.distributed.DistributedHerder) org.apache.kafka.connect.runtime.rest.errors.ConnectRestException: IO Error trying to forward REST request: Connection refused [2016-01-29 20:31:01,802] ERROR Failed to reconfigure connector's tasks, retrying after backoff: (org.apache.kafka.connect.runtime.distributed.DistributedHerder) org.apache.kafka.connect.runtime.rest.errors.ConnectRestException: IO Error trying to forward REST request: Connection refused [2016-01-29 20:31:01,803] DEBUG Sending POST with input [{"tables":"bar","table.poll.interval.ms":"1000","incrementing.column.name":"id","connection.url":"jdbc:mysql://worker1:3306/testdb?user=root","name":"test-mysql-jdbc","tasks.max":"3","task.class":"io.confluent.connect.jdbc.JdbcSourceTask","poll.interval.ms":"1000","topic.prefix":"test-","connector.class":"io.confluent.connect.jdbc.JdbcSourceConnector","mode":"incrementing"},{"tables":"foo","table.poll.interval.ms":"1000","incrementing.column.name":"id","connection.url":"jdbc:mysql://worker1:3306/testdb?user=root","name":"test-mysql-jdbc","tasks.max":"3","task.class":"io.confluent.connect.jdbc.JdbcSourceTask","poll.interval.ms":"1000","topic.prefix":"test-","connector.class":"io.confluent.connect.jdbc.JdbcSourceConnector","mode":"incrementing"}] to http://worker2:8083/connectors/test-mysql-jdbc/tasks (org.apache.kafka.connect.runtime.rest.RestServer) [2016-01-29 20:31:01,803] ERROR IO error forwarding REST request: (org.apache.kafka.connect.runtime.rest.RestServer) java.net.ConnectException: Connection refused [2016-01-29 20:31:01,804] ERROR Request to leader to reconfigure connector tasks failed (org.apache.kafka.connect.runtime.distributed.DistributedHerder) org.apache.kafka.connect.runtime.rest.errors.ConnectRestException: IO Error trying to forward REST request: Connection refused Note that it looks like we are retrying every 1ms, while I'd expect a retry every 250ms. > [CONNECT] It looks like we are not backing off properly when reconfiguring > tasks > > >