On 12/17/2012 11:34 AM, Jakub Hrozek wrote:
On Sun, Dec 16, 2012 at 09:49:06PM +0100, Pavel Březina wrote:
The problem is when we are about to reset the server status, we don't
get through the timeout (30 seconds) because the "switch to primary
server" task is scheduled 30 seconds after fall back to a backup
server. Thus the server status is resetted after another 30 seconds.
It can be nicely seen in the logs:
[be_primary_server_timeout] (0x0400): Looking for primary server!
[fo_resolve_service_send] (0x0100): Trying to resolve service 'LDAP'
[get_server_status] (0x1000): Status of server 'backup.pb' is 'working'
[get_port_status] (0x1000): Port status of port 389 for server
'backup.pb' is 'working'
[get_server_status] (0x1000): Status of server 'ipa-server.ipa.pb'
is 'working'
[get_port_status] (0x1000): Port status of port 389 for server
'ipa-server.ipa.pb' is 'not working'
[get_port_status] (0x0010): ===== DIFF = 30 > 30
[get_server_status] (0x1000): Status of server 'backup.pb' is 'working'
[get_port_status] (0x1000): Port status of port 389 for server
'backup.pb' is 'working'
[fo_resolve_service_activate_timeout] (0x2000): Resolve timeout set
to 10 seconds
[get_server_status] (0x1000): Status of server 'backup.pb' is 'working'
[be_resolve_server_process] (0x1000): Saving the first resolved server
[be_resolve_server_process] (0x0200): Found address for server
backup.pb: [10.16.78.43] TTL 60
[be_primary_server_timeout_activate] (0x2000): Primary server
reactivation timeout set to 30 seconds
The question is how we should deal with it? The easiest solution (in
the attached patch) is just switching > with >=. But I'm not quite sure
whether it is the best solution. It doesn't really fit into my
understanding of timeout.
My other thoughts were:
1. schedule "switch to primary server" task to 31 seconds
2. reset server status of all primary servers when this task is
triggered
We talked about the problem with Pavel off-list and we agreed it would
be better to schedule the switch to backup server after retry_timeout+1
seconds to be sure the retry will be performed rather than changing the
condition. That way, even if we change the retry_timeout in the future,
everything will keep working.
Patch is attached.
From 534b2c91c530564c001a0cc6a011aeff74d14c15 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Pavel=20B=C5=99ezina?= <[email protected]>
Date: Mon, 17 Dec 2012 12:04:54 +0100
Subject: [PATCH] try primary server after retry_timeout + 1 seconds when
switching to backup
https://fedorahosted.org/sssd/ticket/1679
The problem is when we are about to reset the server status, we don't
get through the timeout (30 seconds) because the "switch to primary
server" task is scheduled 30 seconds after fall back to a backup
server. Thus the server status remains "not working" and is resetted
after another 30 seconds.
We need to make sure that the server status is tried after the
timeout period. retry_timeout is currently hardcoded to 30, thus
the change in man page.
---
src/man/include/failover.xml | 2 +-
src/providers/data_provider_fo.c | 3 ++-
src/providers/fail_over.c | 9 +++++++++
src/providers/fail_over.h | 2 ++
4 files changed, 14 insertions(+), 2 deletions(-)
diff --git a/src/man/include/failover.xml b/src/man/include/failover.xml
index dbf9d5464f66726431b6f263bd2167493a0dee26..4865ce2be988b58dbd1c0736b6512f378be80d55 100644
--- a/src/man/include/failover.xml
+++ b/src/man/include/failover.xml
@@ -17,7 +17,7 @@
<emphasis>primary</emphasis> and <emphasis>backup</emphasis>.
The idea is that servers in the primary list are preferred and
backup servers are only searched if no primary servers can be
- reached. If a backup server is selected, a timeout of 30 seconds
+ reached. If a backup server is selected, a timeout of 31 seconds
is set. After this timeout SSSD will periodically try to reconnect
to one of the primary servers. If it succeeds, it will replace
the current active (backup) server.
diff --git a/src/providers/data_provider_fo.c b/src/providers/data_provider_fo.c
index a1b96be35d36ea07e39631109ca63ada9d692bb5..69ade47b932406f05d5e566daec0ae80582dd77d 100644
--- a/src/providers/data_provider_fo.c
+++ b/src/providers/data_provider_fo.c
@@ -531,6 +531,7 @@ static void be_resolve_server_done(struct tevent_req *subreq)
struct tevent_req);
struct be_resolve_server_state *state = tevent_req_data(req,
struct be_resolve_server_state);
+ time_t timeout = fo_get_service_retry_timeout(state->svc->fo_service) + 1;
int ret;
ret = be_resolve_server_process(subreq, state, &new_subreq);
@@ -546,7 +547,7 @@ static void be_resolve_server_done(struct tevent_req *subreq)
/* FIXME: make the timeout configurable */
ret = be_primary_server_timeout_activate(state->ctx, state->ev,
state->ctx, state->svc,
- 30);
+ timeout);
if (ret != EOK) {
goto fail;
}
diff --git a/src/providers/fail_over.c b/src/providers/fail_over.c
index f3bba37f7cf1edccb779041bedc12dd48965f3f0..e7c44174ded773a8e3bb99dc436c45d4e8ca277d 100644
--- a/src/providers/fail_over.c
+++ b/src/providers/fail_over.c
@@ -1555,6 +1555,15 @@ fo_get_server_hostname_last_change(struct fo_server *server)
return server->common->last_status_change.tv_sec;
}
+time_t fo_get_service_retry_timeout(struct fo_service *svc)
+{
+ if (svc == NULL || svc->ctx == NULL || svc->ctx->opts == NULL) {
+ return 0;
+ }
+
+ return svc->ctx->opts->retry_timeout;
+}
+
void fo_reset_services(struct fo_ctx *fo_ctx)
{
struct fo_service *service;
diff --git a/src/providers/fail_over.h b/src/providers/fail_over.h
index 0eb212584aa3621087c53a73088b0406aa505938..1ad081e78c866390a1a345feacc5c0899adf91a4 100644
--- a/src/providers/fail_over.h
+++ b/src/providers/fail_over.h
@@ -192,6 +192,8 @@ time_t fo_get_server_hostname_last_change(struct fo_server *server);
int fo_is_srv_lookup(struct fo_server *s);
+time_t fo_get_service_retry_timeout(struct fo_service *svc);
+
void fo_reset_services(struct fo_ctx *fo_ctx);
bool fo_svc_has_server(struct fo_service *service, struct fo_server *server);
--
1.7.11.7
_______________________________________________
sssd-devel mailing list
[email protected]
https://lists.fedorahosted.org/mailman/listinfo/sssd-devel
