Disclaimer: so far I never had to use a library like LMAX disruptor. After a lot of brain that I spent into the new async/await approach that's available today I even think that a truely high performance .net application has no need for such library. The following hopefully explains the why's.
To me there are mainly two aspects of asynch operations. One is the asynch nature of multithreading where computational expensive operations are offloaded to background threads. The other is async io which allows the cpu to continue doing other tasks when the, compared to the cpu cycling on its calculations, veeeery slooooow io like networking is involved. Asyc/await with tasks provides, from an api point of view, both. Traditionally an io operation would either block the cpu while waiting for the io to complete or be buffered/offloaded to a background thread and finished there. The downside of such an approach is that this involves cross thread synchronization points. The actual problem we need to solve is that we do want the cpu to wait for the slow io. This is where the async/await comes into play. async/await allows the io operation to start and the cpu to continue its task. When the async io is complete an event fired by the io will trigger the cpu to continue its work on a synchronization point that is chosen with an await. While this works best with io, it also works with cpu intensive tasks that need to be run on background threads. But using this for computational expensive cpu tasks only pays off it the costs of synchronization and context switches is insignificant with respect to the actual task. This said, if an appender involves IO, a client application could ultimately choose to either fire and forget, wait for the io or continue and synchronize later if we provided an async api. This however requires us to provide both a "normal" api and an async api. But doing so rewards with a truely async io. Note that this is something what nginx makes heavy use of. libuv is a library that provides a few aspects of io as an event based api. On Wed, 9 May 2018, 16:56 Matt Sicker, <[email protected]> wrote: > I'd be interesting in hearing about high performant .NET applications that > would necessitate the creation of libraries like LMAX Disruptor. AFAIK, > that's generally a C++ and Java world. > > On 9 May 2018 at 08:47, Remko Popma <[email protected]> wrote: > > > In the log4j world, async logging means adding the information to be > > logged to some data structure, whereupon the application thread returns > > immediately to do other work. > > In the background, another thread reads the information to be logged from > > the data structure, potentially transforms it, then renders it to the > > configured layout format and writes it to the configured appender(s). > > > > The data structure may be a standard queue, in which case the > “information > > to be logged” is often a LogEvent instance, or it could be a data > structure > > that is optimized for non-blocking inter-thread handovers, like the LMAX > > Disruptor. I don’t know what the equivalent of the latter is in the .NET > > world. > > > > It seems that concurrent queues in .net may use Async/await under the > > hood. (Based on what I see on SO, like https://stackoverflow.com/ > > questions/7863573/awaitable-task-based-queue) > > > > Not sure if lock-free mechanisms like the lmax disruptor exist. Be aware > > that the background thread needs to employ some waiting strategy until > work > > arrives. The simplest thing is to use some block-notify mechanism: the > > background thread is suspended and woken up by the operating system when > > notified. I assume this is what async/await uses. To be completely > > lock-free, an alternative wait strategy is to busy-spin but this means > > dedicating a core to logging which is a hefty price. In the disruptor > this > > is configurable so if log4j users really want to they can have lock-free > > logging in return for dedicating a cpu core. You may not want or need to > go > > that far. > > > > Remko > > > > (Shameless plug) Every java main() method deserves http://picocli.info > > > > > On May 9, 2018, at 22:06, Dominik Psenner <[email protected]> wrote: > > > > > > When implementing the async/await paradigm it would have to be provided > > as a logging event api and continuously invoked with async down to the > > appender implementations in order for the application code to benefit > from > > true async behavior. Or am I wrong here? > > > > > > > > >> On 2018-05-09 13:48, William Davis wrote: > > >> Jochen, I dont believe that appender has been ported to Log4Net. Maybe > > >> thats what we should do first? Im sure there are other uses cases out > > there > > >> though, which is why we've seen several people roll async appenders in > > the > > >> first place (although it could be a fundamental lack of understanding) > > >> > > >> On Wed, May 9, 2018 at 7:00 AM, Jochen Wiedmann < > > [email protected]> > > >> wrote: > > >> > > >>> On Mon, May 7, 2018 at 2:15 PM William Davis < > > [email protected]> > > >>> wrote: > > >>> > > >>>> I've noticed that there are several Async implementations of > standard > > >>>> appenders out in the wild. Is there a reason none of these have made > > >>> there > > >>>> way into the core product? Is it just b/c no one has taken the time > > to do > > >>> a > > >>>> pull request, or is there some other reason? > > >>> I wonder, why one would create a special async version, when all you > > need > > >>> to do is to put a standard async logger in front of the sync logger > > [1]? > > >>> > > >>> Jochen > > >>> > > >>> 1: https://logging.apache.org/log4j/2.x/manual/async.html# > > MixedSync-Async > > >>> > > > > > > > > > -- > Matt Sicker <[email protected]> >
