Summary: msgd: handle abrupt restart of remote node [#2840]
Review request for Ticket(s): 2840
Peer Reviewer(s): Srinivas
Pull request to:
Affected branch(es): develop
Development branch: ticket-2840
Base revision: dd6a9bfe9d897fe9cc3a70e21d7e066b7a727d44
Personal repository: git://git.code.sf.net/u/trguitar/review
--------------------------------
Impacted area Impact y/n
--------------------------------
Docs n
Build system n
RPM/packaging n
Configuration files n
Startup scripts n
SAF services y
OpenSAF services n
Core libraries n
Samples n
Tests n
Other n
NOTE: Patch(es) contain lines longer than 80 characers
Comments (indicate scope for each "y" above):
---------------------------------------------
revision e04d343ab46a7409772001c61624eb39c2eb50aa
Author: Alex Jones <ajo...@rbbn.com>
Date: Wed, 25 Apr 2018 10:27:13 -0400
msgd: handle abrupt restart of remote node [#2840]
Sometimes when a remote node restarts abruptly, queues which were created on
that node, are unable to be opened again when that node comes up.
There is a race condition when the remote node goes down between msgd getting
the CLM and MDS events indicating node down, and immd removing the implementer
for that remote node. When msgd gets the CLM and MDS events indicating node
down it temporarily becomes the implementer for any queues on that node so that
it can remove the entries in IMM. If IMM has not yet removed the implementer,
msgd will fail to remove the IMM entries. When the remote node comes back up,
and the queues are opened, they will fail because the IMM entries are still
there.
When msgd recevies ERR_EXIST from implementer set in this case, it should
treat it as TRY_AGAIN.
Complete diffstat:
------------------
src/msg/msgd/mqd_clm.c | 60 +++++++++++++++++++++++++++++++++++++------------
src/msg/msgd/mqd_db.h | 2 +-
src/msg/msgd/mqd_evt.c | 12 ++++++++--
src/msg/msgd/mqd_util.c | 2 +-
4 files changed, 58 insertions(+), 18 deletions(-)
Testing Commands:
-----------------
1) create 10 or so queues on node 2
2) reboot -f of node 2 (you may need to do this 10x to exhibit the problem)
3) when node comes back up it should reopen the queues
Testing, Expected Results:
--------------------------
1) when node comes back up after abrupt reboot, it should successfully reopen
the queues
Conditions of Submission:
-------------------------
May 1, or ack from developer
Arch Built Started Linux distro
-------------------------------------------
mips n n
mips64 n n
x86 n n
x86_64 y y
powerpc n n
powerpc64 n n
Reviewer Checklist:
-------------------
[Submitters: make sure that your review doesn't trigger any checkmarks!]
Your checkin has not passed review because (see checked entries):
___ Your RR template is generally incomplete; it has too many blank entries
that need proper data filled in.
___ You have failed to nominate the proper persons for review and push.
___ Your patches do not have proper short+long header
___ You have grammar/spelling in your header that is unacceptable.
___ You have exceeded a sensible line length in your headers/comments/text.
___ You have failed to put in a proper Trac Ticket # into your commits.
___ You have incorrectly put/left internal data in your comments/files
(i.e. internal bug tracking tool IDs, product names etc)
___ You have not given any evidence of testing beyond basic build tests.
Demonstrate some level of runtime or other sanity testing.
___ You have ^M present in some of your files. These have to be removed.
___ You have needlessly changed whitespace or added whitespace crimes
like trailing spaces, or spaces before tabs.
___ You have mixed real technical changes with whitespace and other
cosmetic code cleanup changes. These have to be separate commits.
___ You need to refactor your submission into logical chunks; there is
too much content into a single commit.
___ You have extraneous garbage in your review (merge commits etc)
___ You have giant attachments which should never have been sent;
Instead you should place your content in a public tree to be pulled.
___ You have too many commits attached to an e-mail; resend as threaded
commits, or place in a public tree for a pull.
___ You have resent this content multiple times without a clear indication
of what has changed between each re-send.
___ You have failed to adequately and individually address all of the
comments and change requests that were proposed in the initial review.
___ You have a misconfigured ~/.gitconfig file (i.e. user.name, user.email etc)
___ Your computer have a badly configured date and time; confusing the
the threaded patch review.
___ Your changes affect IPC mechanism, and you don't present any results
for in-service upgradability test.
___ Your changes affect user manual and documentation, your patch series
do not contain the patch that updates the Doxygen manual.
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Opensaf-devel mailing list
Opensaf-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/opensaf-devel
