OK, so here's a few different takes on numbers.
First graphs. Graph one is with the macro on:
https://g.defau.lt/#gluSzlb39ScnZU4FS2AtrYUF31IYct
Graph2 is with the macro off:
https://g.defau.lt/#6We3VgDfcXi7zkhMMzdesQLKD5iMUd
The colors are reads, read bytes, writes, write bytes. You can see the load
spreads out as expected
We also managed to do a synthetic test which was interesting and showed that for
our dominant use-case of RPMs with lots of small files this is a big win. This
is the mount of time the storage system was not doing work:
# fb-dba-core:
## With macro - many smaller files, one huge file
2017-04-06 16:26:57.722013873 -0700 PDT 5.73155ms 104.968µs 6
2017-04-06 16:26:58.722147493 -0700 PDT 5.252973ms 117.358µs 5
2017-04-06 16:26:59.722343955 -0700 PDT 7.422587ms 139.914µs 8
2017-04-06 16:27:00.72254866 -0700 PDT 6.109201ms 122.32µs 6
2017-04-06 16:27:01.722734509 -0700 PDT 7.40091ms 155.112µs 8
2017-04-06 16:27:02.722875662 -0700 PDT 7.794604ms 148.676µs 8
2017-04-06 16:27:12.723782443 -0700 PDT 208.33143ms 25.153745ms 202
2017-04-06 16:27:13.723915361 -0700 PDT 5.998775ms 133.564µs 7
2017-04-06 16:27:14.724062693 -0700 PDT 5.89304ms 132.267µs 7
2017-04-06 16:27:15.724212253 -0700 PDT 6.98323ms 117.4µs 8
2017-04-06 16:27:16.724324089 -0700 PDT 5.402875ms 132.129µs 5
2017-04-06 16:27:17.724437579 -0700 PDT 5.926757ms 127.274µs 7
2017-04-06 16:27:18.724621554 -0700 PDT 5.837904ms 121.583µs 6
2017-04-06 16:27:19.724740404 -0700 PDT 5.080387ms 115.304µs 5
2017-04-06 16:27:20.724855913 -0700 PDT 5.573811ms 131.026µs 6
2017-04-06 16:27:21.725034955 -0700 PDT 3.856319ms 102.18µs 4
2017-04-06 16:27:22.725188062 -0700 PDT 4.405144ms 111.026µs 5
## Without macro
2017-04-06 16:27:55.728507697 -0700 PDT 27.318392ms 885.188µs 26
2017-04-06 16:28:02.729271342 -0700 PDT 7.49606ms 116.632µs 8
2017-04-06 16:28:15.730604087 -0700 PDT 84.956988ms 10.598699ms 83
2017-04-06 16:28:20.73111897 -0700 PDT 48.069056ms 457.543µs 53
2017-04-06 16:28:21.731312738 -0700 PDT 65.498909ms 2.925142ms 63
# fb-gcc-5-glibc-2.23-runtime-6-1.x86_64.rpm, some 100MB files
## With macro
2017-04-06 16:33:41.765007508 -0700 PDT 1.320882ms 88.294µs 3
2017-04-06 16:33:42.765164056 -0700 PDT 13.984165ms 279.486µs 13
2017-04-06 16:33:43.765326558 -0700 PDT 1.890576ms 80.318µs 2
2017-04-06 16:33:46.765670291 -0700 PDT 39.313771ms 1.022946ms 38
2017-04-06 16:33:47.765840591 -0700 PDT 3.801104ms 94.404µs 5
2017-04-06 16:33:51.766274676 -0700 PDT 67.966698ms 2.87646ms 67
2017-04-06 16:33:52.766424373 -0700 PDT 1.996679ms 90.686µs 2
2017-04-06 16:33:56.766825593 -0700 PDT 101.579654ms 5.967763ms 98
2017-04-06 16:34:02.767494189 -0700 PDT 19.626819ms 332.579µs 20
2017-04-06 16:34:03.767628848 -0700 PDT 4.940615ms 116.663µs 5
2017-04-06 16:34:04.767777287 -0700 PDT 2.480112ms 97.978µs 4
2017-04-06 16:34:05.7679422 -0700 PDT 1.551916ms 62.483µs 2
2017-04-06 16:34:06.768067462 -0700 PDT 1.756435ms 68.526µs 2
2017-04-06 16:34:07.768176248 -0700 PDT 1.319394ms 87.179µs 2
2017-04-06 16:33:02.761068223 -0700 PDT 6.764001ms 99.465µs 7
## Without macro
2017-04-06 16:34:59.773285651 -0700 PDT 1.695824ms 87.892µs 2
2017-04-06 16:35:02.773564104 -0700 PDT 5.68678ms 105.131µs 6
2017-04-06 16:35:12.774532746 -0700 PDT 65.836492ms 1.853472ms 70
2017-04-06 16:35:13.774667005 -0700 PDT 82.757401ms 8.41281ms 80
2017-04-06 16:35:18.77511384 -0700 PDT 73.289284ms 3.436747ms 71
2017-04-06 16:35:24.775796768 -0700 PDT 1.199151ms 85.286µs 2
2017-04-06 16:35:26.77602982 -0700 PDT 1.203794ms 105.355µs 2
2017-04-06 16:35:43.777684277 -0700 PDT 51.357162ms 2.053203ms 50
So large enough files can still cause problems, but it's not perfect.
Madvise isn't quite the same here... though certainly keeping track and always
writing after a certain number of bytes would be even better, but I think this
shows this diff is a good start.
--
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/rpm-software-management/rpm/pull/187#issuecomment-292681920
_______________________________________________
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint
