On Tue, 2015-02-24 at 09:34 -0800, Thiago Macieira wrote: > Hi guys > > Felix and I discussed for a while at the OIC members meeting and we came to > the following suggestion on how to achieve thread-safety for the context: > > 1) the context is reference-counted, so we will provide functions like: > * ITVTContext_context_create() (ref counts up) > * ITVTContext_context_unref() > * ITVTContext_context_ref(ITVTContext *) > > On platforms that support so, the refcounter will be atomic (x86, ARMv7 and > up, etc.). > > I have the code for this for GCC, ICC and Clang at > http://code.qt.io/cgit/qt/qtbase.git/tree/src/3rdparty/forkfd/forkfd_gcc.h?h=5.5 > > (see ffd_atomic_add_fetch). It's BSD-licensed, which is compatible with > Apache2, but since it's copyright by Intel, Intel is allowed to relicense.
Is the ref-counting overkill here? I would suspect that just allowing the consumer to control the lifetime would be a simpler implementation that gets us just about everything. > > 2) on multithreaded systems, we'll do a simple producer-consumer solution for > handling callbacks. The thread handling the socket parses the incoming CoAP > message and determines what callback to activate. It then ref counts up the > context and posts the callback into a thread-safe queue. > > A thread or a thread pool will read from the queue and dispatch the callback. > This way, we ensure that the context cannot get freed until the callbacks > have > been handled and we ensure that the user callback taking a long time will not > block socket handling. Additionally, since no locks were held prior to > calling > the user code, there should be no deadlocks in reentering the ITVTContext API. > > This solution requires memory allocation for the queue and a semaphore. > > 3) on thread-incapable systems, we will still use a queue but there won't be > a > thread or thread pool to consume the queue. Instead, shortly before going > back to poll(2)ing the socket, the main socket loop will empty the queue. > > If memory is an issue, this code can also be modified to use a fixed-size and > pre-allocated queue, so it will not pop and parse messages from the socket if > the queue is full. > > 4) the main ITVTContext API needs to be protected by a mutex on systems that > are thread-capable. On Linux, pthread_mutex is extremely lightweight. If need > be, I can write code to use futex(2) directly, at the expense of not being > able to use wait conditions. > We are already (for better or worse) using glib2's mutex library, I would suggest selecting 1 mutex for the entire C stack, and would prefer something more cross-platform than pthread_mutex. > We discussed whether we should use a RW Lock instead and came to the > conclusion that most calls into the API are expected to be mutating ones, so > there isn't much value in having the more complex RW Lock.
