On Tuesday, 17 October 2017 at 13:09:37 UTC, Steven Schveighoffer
wrote:
Ouch! I had an experience like that once.
I worked at a company that bought a one-man show's company who
had an impressive load-balancing software we wanted to
incorporate in our system.
About 1-2 years into him working at our company, one of our
developers tested it using webbench (all testing had been done
by this guy previously), and was getting terrible numbers. But
his tests always showed really good numbers.
Turns out he was "timing" his benchmarks by starting a separate
thread, then sleeping for 1 second, and then measuring how many
requests he handled in that "1 second". But of course, the
system was super-loaded, so the sleep was going way longer than
1 second, and his numbers looked great! After we fixed it, the
numbers looked horrific and matched webbench.
When this was found out, we kind of moved away from that
software, as we were moving our focus to hardware. I can't
imagine how that must have felt, though.
-Steve
This is just plain negligence on upper management's part. I can't
believe they got that far without doing due diligence to verify
his results.
