Summary: msgd: put node down handling on thread [#2852]
Review request for Ticket(s): 2852
Peer Reviewer(s): Srinivas
Pull request to:
Affected branch(es): develop
Development branch: ticket-2852
Base revision: 93e2808fb0bd3143a77e31dd2f0115a6596479ed
Personal repository: git://git.code.sf.net/u/trguitar/review
--------------------------------
Impacted area Impact y/n
--------------------------------
Docs n
Build system n
RPM/packaging n
Configuration files n
Startup scripts n
SAF services y
OpenSAF services n
Core libraries n
Samples n
Tests n
Other n
Comments (indicate scope for each "y" above):
---------------------------------------------
revision 9bb598f8390aaf41c1e0dcd458ee0d82fae58999
Author: Alex Jones <ajo...@rbbn.com>
Date: Fri, 11 May 2018 11:04:34 -0400
msgd: put node down handling on thread [#2852]
If multiple nodes go down simultaneously which are hosting msg queues (e.g.
multiple VMs on a host, and the host goes down), msgd can take a long time to
process the node downs which blocks the main thread, and therefore the
healthcheck doesn't get processed, so msgd dies, which restarts the controller.
msgd needs to sit in a loop waiting for imm to release the implementers for
each of the down nodes. For many nodes which went down simultaneously this can
take up to 20 seconds when done serially.
Node down logic needs to be put on a thread, so that we can continue to process
other messages like healthcheck. This also allows us to parallelize the node
down handling.
Complete diffstat:
------------------
src/msg/msgd/mqd_asapi.c | 17 +++--
src/msg/msgd/mqd_clm.c | 183 +++++++++++++++++++++++++++++++----------------
src/msg/msgd/mqd_evt.c | 5 ++
src/msg/msgd/mqd_mbcsv.c | 16 +++--
src/msg/msgd/mqd_ntf.cc | 4 ++
5 files changed, 151 insertions(+), 74 deletions(-)
Testing Commands:
-----------------
1) have multiple nodes (in our test we have 17) which hold msg queues
2) take them all down at the same time, and bring them back up reopening the msg
queues
3) do this repeatedly
Testing, Expected Results:
--------------------------
1) msgd should not fail healthcheck
2) msg queues should be successfully reopened on the nodes
Conditions of Submission:
-------------------------
May 17, or ack from developer
Arch Built Started Linux distro
-------------------------------------------
mips n n
mips64 n n
x86 n n
x86_64 y y
powerpc n n
powerpc64 n n
Reviewer Checklist:
-------------------
[Submitters: make sure that your review doesn't trigger any checkmarks!]
Your checkin has not passed review because (see checked entries):
___ Your RR template is generally incomplete; it has too many blank entries
that need proper data filled in.
___ You have failed to nominate the proper persons for review and push.
___ Your patches do not have proper short+long header
___ You have grammar/spelling in your header that is unacceptable.
___ You have exceeded a sensible line length in your headers/comments/text.
___ You have failed to put in a proper Trac Ticket # into your commits.
___ You have incorrectly put/left internal data in your comments/files
(i.e. internal bug tracking tool IDs, product names etc)
___ You have not given any evidence of testing beyond basic build tests.
Demonstrate some level of runtime or other sanity testing.
___ You have ^M present in some of your files. These have to be removed.
___ You have needlessly changed whitespace or added whitespace crimes
like trailing spaces, or spaces before tabs.
___ You have mixed real technical changes with whitespace and other
cosmetic code cleanup changes. These have to be separate commits.
___ You need to refactor your submission into logical chunks; there is
too much content into a single commit.
___ You have extraneous garbage in your review (merge commits etc)
___ You have giant attachments which should never have been sent;
Instead you should place your content in a public tree to be pulled.
___ You have too many commits attached to an e-mail; resend as threaded
commits, or place in a public tree for a pull.
___ You have resent this content multiple times without a clear indication
of what has changed between each re-send.
___ You have failed to adequately and individually address all of the
comments and change requests that were proposed in the initial review.
___ You have a misconfigured ~/.gitconfig file (i.e. user.name, user.email etc)
___ Your computer have a badly configured date and time; confusing the
the threaded patch review.
___ Your changes affect IPC mechanism, and you don't present any results
for in-service upgradability test.
___ Your changes affect user manual and documentation, your patch series
do not contain the patch that updates the Doxygen manual.
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Opensaf-devel mailing list
Opensaf-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/opensaf-devel
