richardcocks commented on issue #4066:
URL: https://github.com/apache/iggy/issues/4066#issuecomment-5555182247
As I worked through what the `Math.Random` involved, I really hated it,
since `Math.Random` doesn't have any nice APIs for getting out bytes.
So I went back and had a re-think, and after some work, even with the crypto
API, I'm now at ~2.5x throughput on my local benchmarks:
| scheme | msg/s | note |
|---|--:|--:|
| CSPRNG UUID | 192,136 | original |
| Math.random ×4 | 475,669 | non-crypto |
| fastbytes anchor | 429,464 | external fed numbers |
| Pool copy-out | 504,958 | CSPRNG, +6.2% over Math.random, +163% over
baseline |
My local throughput is around 2.5x the `master` baseline now, but it comes
at the cost of complexity. The call to the crypto random API is an order of
magnitude slower than I think is reasonable, but we can take 32KB for almost
the same cost of taking 16 bytes, and so we can buffer them for use in several
thousand messages.
This gives us ~2.5x throughput with high quality random numbers.
Some alternatives to this entropy pool approach are:
- `Math.Random` which involves scaling it up to [0, 2^32) then
truncating it back to UInt32, then repeating this 4 times to stuff it into the
buffer.
- Using `crypto.fillRandomSync` on every call and accepting the
performance hit for hot clients.
- Vendoring in a PRNG that is faster at producing 16 bytes on demand.
I will finish off preparing a PR for this, I still need to clean it up, but
I should have something ready soon, and fingers crossed that it meets the
standard required.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
