> -----Original Message----- > From: [email protected] [mailto:ceph-devel- > > I'm digging into perf and the code to see here/how I might be able to > improve performance for small I/O around 16K. > > I ran fio with rados and used perf to record. Looking through the report, > there is a very substantial amount of time creating threads (or so it looks, > but > I'm really new to perf). It seems to point to messenger, so I looked in the > code. From perf if looks like thread pooling isn't happening, but from what I > can gather from the code, it should. > [..]
This is so because you use SimpleMessenger, which can't handle small I/O well. Indeed, threads are problematic with it, as well as memory allocation. I did some benchmarking some time ago and the gist of it is that you could try going for AsyncMessenger and see if it helps. You can also see my results here: http://stuff.predictor.org.pl/chunksize.xlsx From there you can see that most of the time of small I/Os in SimpleMessenger Is spent in tcmalloc code, and also there's a performance drop around 64k Blocksize in Async Messenger. With best regards / Pozdrawiam Piotr Dałek
