Summary: MDS: performance improvement [#654]
Review request for Trac Ticket(s): #654
Peer Reviewer(s): hans/anders BJ/mathi
Pull request to: <<LIST THE PERSON WITH PUSH ACCESS HERE>>
Affected branch(es): default
Development branch: default
--------------------------------
Impacted area Impact y/n
--------------------------------
Docs n
Build system n
RPM/packaging n
Configuration files n
Startup scripts n
SAF services n
OpenSAF services n
Core libraries y
Samples n
Tests n
Other n
Comments (indicate scope for each "y" above):
---------------------------------------------
<<EXPLAIN/COMMENT THE PATCH SERIES HERE>>
changeset b312b9139a0512d48699936e30825399ec601f24
Author: A V Mahesh <mahesh.va...@oracle.com>
Date: Wed, 29 Jan 2014 16:44:31 +0530
MDS: TIPC performance improvement [#654]
1) MDS fragmentation value changed from 1400 to MDS_DIRECT_BUF_MAXSIZE
(65535 maximum packet size)-(56 MDS header) , #defined
MDTM_NORMAL_MSG_FRAG_SIZE_MAX MDS_DIRECT_BUF_MAXSIZE .
2) MDS MDS_DIRECT_BUF_MAXSIZE value changed from 8000 to (65535 maximum
packet size)-(56 MDS header) , #defined MDTM_NORMAL_MSG_FRAG_SIZE_MAX
(65535
maximum packet size)-(56 MDS header)
3) This PAYLOAD_BUF_SIZE value is suppose to be equal to
MDS_DIRECT_BUF_MAXSIZE (65535 maximum packet size)-(56 MDS header) ,but
in
the previous releases of Opensaf the value of TIPC inbuf size of
MDTM_RECV_BUFFER_SIZE (mds_dt_tipc.c) TIPC was limited to(8000+MDS
header )
, we have in-service Upgrade issue, so it is not possible to send the
new
MDS_DIRECT_BUF_MAXSIZE (65535 maximum packet size)-(56 MDS header) value
previous Opensaf version nodes , so for the current release it is
limited to
8000, possibly adjust this in the future Opensaf releases to
MDS_DIRECT_BUF_MAXSIZE.
4) To support in-service MDS TIPC Upgrade issue
MDTM_NORMAL_MSG_FRAG_SIZE_MIN is provide, and this fragment size is
used to
send to previous version of Opensaf Node while in-service Upgrade.
In the previous releases of Opensaf the value of TIPC inbuf size of
MDTM_RECV_BUFFER_SIZE (mds_dt_tipc.c) TIPC was limited to(8000+MDS
header) ,
we have in-service Upgrade issue, so it is not possible to send the new
MDS_DIRECT_BUF_MAXSIZE (65535 maximum packet size)-(56 MDS header) value
previous Opensaf version nodes , so for the current release it is
limited to
8000, possibly this will be removed in the future Opensaf releases .
5) To handle in-service Upgrade of fragmentation changes , New node do
install & subscribe its Node Mds version. the
MDTM_NORMAL_MSG_FRAG_SIZE_MIN
fragment size is used to send to previous version of Opensaf Node while
in-
service Upgrade ,and the old nodes will not publishes so we need to
fragment
to MDTM_NORMAL_MSG_FRAG_SIZE_MIN size.mds_mdtm_node_info_install_tipc()
&
mds_mdtm_node_info_subscribe_tipc()
changeset 4768c54be6086c3d1328387bc8c53601545b70ea
Author: A V Mahesh <mahesh.va...@oracle.com>
Date: Wed, 29 Jan 2014 16:45:56 +0530
MDS: TCP performance improvement [#654] 1) Their is NO issue with MDS
TCP
in-service Upgrade ,The value of SO_RCVBUF & SO_SNDBUF of TCP sockets
was
set to 64000 (MDS_SND_RCV_SIZE) using setsockopt() ,the kernel doubles
this
value ,and this doubled value is returned by getsockopt(). So
MDTM_NORMAL_MSG_FRAG_SIZE_MAX size is used to send for for both
previous &
current versions of Opensaf nodes which are using MDS TCP as transport
2) Made DTM-TCP inter-node & intra-node socket SO_SNDBUF and SO_RCVBUF
buffer in bytes configurable . The kernel doubles this value (to allow
space
for bookkeeping over-head) when it is set using setsockopt(),and this
doubled value is returned by getsockopt(). The default value is set to
DTM_SOCK_SND_RCV_BUF_SIZE=126976
3) Made TCP_NODELAY configurable to specifi whether MDS TCP transport
should
follow the Nagle algorithm for deciding when to send data.By default,
TCP
will follow the Nagle algorithm is disable this behavior , can be
configured
to TRUE(1) or FALSE(0)
changeset d7112ad10800b2b5c1b488127bb3045e95e0dde8
Author: A V Mahesh <mahesh.va...@oracle.com>
Date: Wed, 29 Jan 2014 16:47:01 +0530
MDS: IMMSV macros tune against MDS performance [#654] 1) Adjusted
IMMSV_DEFAULT_MAX_SYNC_BATCH_SIZE to MDS_DIRECT_BUF_MAXSIZE (65535
maximum
packet size)-(56 MDS header)
2) Adjusted IMMND_SEARCH_BUNDLE_SIZE to MDS_DIRECT_BUF_MAXSIZE (65535
maximum packet size)-(56 MDS header)
Complete diffstat:
------------------
osaf/libs/common/immsv/include/immsv_api.h | 6 +-
osaf/libs/core/include/mds_papi.h | 4 +-
osaf/libs/core/include/ncsusrbuf.h | 17 ++++++-
osaf/libs/core/mds/include/mds_core.h | 25 ++++++++++
osaf/libs/core/mds/include/mds_dt.h | 25 +++++++++-
osaf/libs/core/mds/include/mds_dt2c.h | 1 +
osaf/libs/core/mds/include/mds_dt_tcp_disc.h | 1 -
osaf/libs/core/mds/include/mds_dt_tipc.h | 2 +
osaf/libs/core/mds/mds_c_api.c | 20 ++++++++
osaf/libs/core/mds/mds_c_db.c | 83
+++++++++++++++++++++++++++++++++++
osaf/libs/core/mds/mds_c_sndrcv.c | 2 +-
osaf/libs/core/mds/mds_dt_tcp.c | 20 +++++++-
osaf/libs/core/mds/mds_dt_tipc.c | 206
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++------
osaf/libs/core/mds/mds_dt_trans.c | 18 ++++---
osaf/services/infrastructure/dtms/config/dtmd.conf | 23 +++++++++
osaf/services/infrastructure/dtms/dtm/dtm_intra.c | 13 ++---
osaf/services/infrastructure/dtms/dtm/dtm_node_sockets.c | 44
+++++++++---------
osaf/services/infrastructure/dtms/dtm/dtm_read_config.c | 30 ++++++++++++
osaf/services/infrastructure/dtms/include/dtm.h | 2 +
osaf/services/infrastructure/dtms/include/dtm_cb.h | 3 +
osaf/services/infrastructure/dtms/include/dtm_intra_disc.h | 2 -
osaf/services/infrastructure/nid/scripts/opensafd.in | 1 +
osaf/services/saf/immsv/immloadd/imm_loader.cc | 1 +
osaf/services/saf/immsv/immnd/immnd_evt.c | 9 ++-
24 files changed, 487 insertions(+), 71 deletions(-)
Testing Commands:
-----------------
Test any some application(may be cpsv) which has large ckpt data to be sync to
across the node node
before and after the patch .
Testing, Expected Results:
--------------------------
Should see Overall OpenSaf performence improvement
Conditions of Submission:
-------------------------
Ack from Hans/Anders BJ/Mathi
Arch Built Started Linux distro
-------------------------------------------
mips n n
mips64 n n
x86 n n
x86_64 y y
powerpc n n
powerpc64 n n
Reviewer Checklist:
-------------------
[Submitters: make sure that your review doesn't trigger any checkmarks!]
Your checkin has not passed review because (see checked entries):
___ Your RR template is generally incomplete; it has too many blank entries
that need proper data filled in.
___ You have failed to nominate the proper persons for review and push.
___ Your patches do not have proper short+long header
___ You have grammar/spelling in your header that is unacceptable.
___ You have exceeded a sensible line length in your headers/comments/text.
___ You have failed to put in a proper Trac Ticket # into your commits.
___ You have incorrectly put/left internal data in your comments/files
(i.e. internal bug tracking tool IDs, product names etc)
___ You have not given any evidence of testing beyond basic build tests.
Demonstrate some level of runtime or other sanity testing.
___ You have ^M present in some of your files. These have to be removed.
___ You have needlessly changed whitespace or added whitespace crimes
like trailing spaces, or spaces before tabs.
___ You have mixed real technical changes with whitespace and other
cosmetic code cleanup changes. These have to be separate commits.
___ You need to refactor your submission into logical chunks; there is
too much content into a single commit.
___ You have extraneous garbage in your review (merge commits etc)
___ You have giant attachments which should never have been sent;
Instead you should place your content in a public tree to be pulled.
___ You have too many commits attached to an e-mail; resend as threaded
commits, or place in a public tree for a pull.
___ You have resent this content multiple times without a clear indication
of what has changed between each re-send.
___ You have failed to adequately and individually address all of the
comments and change requests that were proposed in the initial review.
___ You have a misconfigured ~/.hgrc file (i.e. username, email etc)
___ Your computer have a badly configured date and time; confusing the
the threaded patch review.
___ Your changes affect IPC mechanism, and you don't present any results
for in-service upgradability test.
___ Your changes affect user manual and documentation, your patch series
do not contain the patch that updates the Doxygen manual.
------------------------------------------------------------------------------
WatchGuard Dimension instantly turns raw network data into actionable
security intelligence. It gives you real-time visual feedback on key
security issues and trends. Skip the complicated setup - simply import
a virtual appliance and go from zero to informed in seconds.
http://pubads.g.doubleclick.net/gampad/clk?id=123612991&iu=/4140/ostg.clktrk
_______________________________________________
Opensaf-devel mailing list
Opensaf-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/opensaf-devel
