- **status**: review --> fixed
- **Comment**:

commit 9518b6eb21e32b3486d9fcbd63cf0f9f0bdb5a39 (HEAD -> develop, 
origin/develop, ticket-3407)
Author: Thang Duc Nguyen <[email protected]>
Date:   Fri Oct 16 01:17:11 2026 +0000

    amf: reboot node when CLM initialization hangs indefinitely [#3407]

    avnd_clm_init() retries saClmInitialize_4() indefinitely on
    SA_AIS_ERR_TRY_AGAIN, SA_AIS_ERR_TIMEOUT, and SA_AIS_ERR_UNAVAILABLE
    with only a 100ms sleep between retries and no timeout. This causes
    osafamfnd to hang permanently, leaving the node disabled in AMF with
    no automatic recovery.

    Fix by adding a 60-second wall-clock deadline to the retry loop. If
    saClmInitialize_4() has not succeeded within 60 seconds, call
    opensaf_reboot() with reason "CLM init timeout" so the node re-joins
    the cluster cleanly rather than staying permanently disabled.

    The retry interval remains 100ms; the wall-clock deadline is the
    primary bound on total wait time. A pure retry count is insufficient
    because each call can block up to ~10 seconds when CLMS is registered
    but unresponsive, making elapsed time per retry unpredictable. The
    wall-clock deadline bounds the total wait regardless of how long each
    individual call takes.




---

**[tickets:#3407] amf: amfnd hangs indefinitely when CLMS is unresponsive 
during initialization**

**Status:** fixed
**Milestone:** 5.27.01
**Created:** Thu Oct 15, 2026 10:40 AM UTC by Thang Duc Nguyen
**Last Updated:** Fri Oct 16, 2026 04:09 AM UTC
**Owner:** Thang Duc Nguyen


avnd_clm_init() retries saClmInitialize_4() indefinitely on 
SA_AIS_ERR_TRY_AGAIN, SA_AIS_ERR_TIMEOUT, and SA_AIS_ERR_UNAVAILABLE with only 
a 100ms sleep between retries, but no timeout or reboot trigger. This causes 
osafamfnd to stay permanently stuck, leaving the node disabled in AMF with no 
automatic recovery.

Root Cause:

saClmInitialize_4() can fail persistently in two distinct ways:

1. CLMS not yet registered in MDS (clms_up == false): returns 
SA_AIS_ERR_UNAVAILABLE immediately. This occurs when CLMD has not yet started 
or has not yet registered with MDS. Each retry returns instantly, so the 100ms 
sleep paces the loop at ~10 retries/second — but since there is no retry limit, 
the loop runs forever if CLMS never comes up.

2. CLMS registered but unresponsive (clms_up == true): the agent proceeds to 
send an MDS SNDRSP message to CLMS and waits for a response. If CLMS does not 
respond (e.g. CLMD has started and registered in MDS but has not yet been 
elected active coordinator by AMF), the MDS sync send blocks for the full 
CLMS_WAIT_TIME timeout (1000 × 10ms = 10 seconds) before returning 
SA_AIS_ERR_TRY_AGAIN. In this case each retry costs ~10 seconds, making the 
100ms sleep irrelevant and the stuck state effectively permanent.


---

Sent from sourceforge.net because [email protected] is 
subscribed to https://sourceforge.net/p/opensaf/tickets/

To unsubscribe from further messages, a project admin can change settings at 
https://sourceforge.net/p/opensaf/admin/tickets/options.  Or, if this is a 
mailing list, you can unsubscribe from the mailing list.
_______________________________________________
Opensaf-tickets mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/opensaf-tickets

Reply via email to