Earlier I reported a very gradual slowdown in the performance
of my simple 1-sender 1-receiver test, on RHEL 7.0 and Fedora 20
but not on RHEL 6.3 .
The slowdown caused the test to end up running at half speed after
a billion or two billion messages. ( Which took hours to run. )
I now know how to cause this slowdown to happen any time, and
it works just as well on RHEL 6.3 as it does on RHEL 7.0 .
All I have to do is make the machine busy. Even though I do not
swamp all processors -- in fact, I leave a couple processors idle --
my receiver program slows down when the machine becomes busy --
***and it never recovers***.
I have been doing qpid builds, for example. Even after I interrupt
the build -- many minutes later, long after the box has become
idle but for my sender and receiver, the receiver's CPU usage
is still depressed, and performance has been cut to 1/2 or 1/3
of what it was at the beginning of the test.
It never comes back.
In fact, I can make it ratchet down again by running another build.
This is nothing magic about builds -- that was just a convenient
way of making the box busy.
I will be making a Jira for this later today.
I was able to make a callgrind picture for the receiver from when
the test was fast and from when it was slow. I will attach all
my info to the Jira.