Julian Santander created XERCESC-2037:
-----------------------------------------
Summary: Excessive locking in IconvGNUTranscoders
Key: XERCESC-2037
URL: https://issues.apache.org/jira/browse/XERCESC-2037
Project: Xerces-C++
Issue Type: Improvement
Components: Utilities
Affects Versions: 3.0.1
Environment: Linux x86_64
Reporter: Julian Santander
Priority: Minor
Fix For: 3.0.1
We have a multithreaded Linux application that was using Xerces C++ 2.7 running
on multicore (32) hardware.
Recently, we moved (forced by a change in our supporting platform sofware and
libraries) to Xerces C++ 3.0. Unfortunately this caused a bad impact in the
throughput of our application.
After some investigation, we found that our application was locking too much,
specially on {{IconvGNULCPTranscoder}}
{{XMLInitializer}} will create a default transcoder, and set this as the
transcoder that the XMLString uses in its static methods. Now this is a single
transcoder that is used system-wide.
All the {{IconvGNUxxx}} objects inherit from {{IconGNUWrapper}}. This wrapper
contains a mutex that it is used to protect {{iconv_t}} handlers used for the
to and from conversions. The wrapper locks the mutex on all calls to the iconv
library that uses the handlers. This is unavoidable as while the iconv library
seem to be thread-safe, the handlers keep state of the conversion, so they can
not be used simultaneously on multiple threads.
As a summary, the situation is that we have one object (default
{{IconvGNULCPTranscoder}}) that it is used system wide and that locks
processing for the duration of the transcoding (and apparently there are *many*
transcoding operations going on), this is bad for multithreading operation as
it reduces the number of simultaneous parsing to 1.
-----
Our solution so far has been:
# Make the IconvGNUWrapper store the to/from locales instead of the to/from
iconv_t
# Make the IconvGNUWrapper to keep a map of the to/from iconv_t indexed by
thread ID.
# When requested an iconv operation, the wrapper retrieves the handlers for the
current thread or allocates new ones if this is the first time.
# The 3) is protected by a Read/Write mutex. Note that there's still some
locking while dealing with searching/inserting to the map, but the heavy
operation (transcoding) can now run outside the mutex, which greatly reduces
the contention.
# plus some additional changes to clean up and wire in the above changes.
The above seems to have reduced contention to acceptable levels.
I can post a patch if desired, however in its current form it might not be
suitable for your code as it uses STL for the {{std::map}} and {{std::string}}
and it uses [ACE library|http://www.cs.wustl.edu/~schmidt/ACE.html] for the RW
mutex.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
