Re: clearing the disk cache
Thanks. :)
But isn't "bufcachepercent < 5" there for a reason? Will my machine
now catch fire?
On Wed, Jul 4, 2018 at 10:23 AM, Boudewijn Dijkstra
wrote:
> Op Tue, 03 Jul 2018 16:06:37 +0200 schreef Maximilian Pichler
> :
>>
>> Now I'm resorting to "sysctl kern.bufcachepercent=5; sysctl
>> kern.bufcachepercent=90" to "almost" clear the cache. If only setting
>> it to 0 were allowed...
>
>
> --- sys/kern/kern_sysctl.c.orig Mon Feb 19 09:59:52 2018
> +++ sys/kern/kern_sysctl.c Wed Jul 4 10:20:53 2018
> @@ -602,7 +602,7 @@
> );
> if (error)
> return(error);
> - if (bufcachepercent > 90 || bufcachepercent < 5) {
> + if (bufcachepercent > 90) {
> bufcachepercent = opct;
> return (EINVAL);
> }
>
>
> --
> Gemaakt met Opera's e-mailprogramma: http://www.opera.com/mail/
>
Re: clearing the disk cache
Op Tue, 03 Jul 2018 16:06:37 +0200 schreef Maximilian Pichler
:
Now I'm resorting to "sysctl kern.bufcachepercent=5; sysctl
kern.bufcachepercent=90" to "almost" clear the cache. If only setting
it to 0 were allowed...
--- sys/kern/kern_sysctl.c.orig Mon Feb 19 09:59:52 2018
+++ sys/kern/kern_sysctl.c Wed Jul 4 10:20:53 2018
@@ -602,7 +602,7 @@
);
if (error)
return(error);
- if (bufcachepercent > 90 || bufcachepercent < 5) {
+ if (bufcachepercent > 90) {
bufcachepercent = opct;
return (EINVAL);
}
--
Gemaakt met Opera's e-mailprogramma: http://www.opera.com/mail/
Re: clearing the disk cache
Thanks for explaining! Looks like this is too complex to be reliably tricked by a few dd commands. Now I'm resorting to "sysctl kern.bufcachepercent=5; sysctl kern.bufcachepercent=90" to "almost" clear the cache. If only setting it to 0 were allowed... On Tue, Jul 3, 2018 at 3:06 PM, Claudio Jeker wrote: > On Tue, Jul 03, 2018 at 01:30:20PM +0200, Maximilian Pichler wrote: >> On Tue, Jul 3, 2018 at 11:47 AM, Janne Johansson wrote: >> > https://www.tedunangst.com/flak/post/2Q-buffer-cache-algorithm >> >> Thanks. If I'm reading this correctly upon access (read or write), an >> action is performed depending on what queue a buffer is in: >> - none: Take a buffer from the tail of the cold queue and insert it at >> the front of the hot queue. >> - hot: Keep it there, but move to the front. >> - cold: Move it to the front of the warm queue. >> - warm: Keep it there, but move to the front. >> >> If a buffer reaches the tail of a queue it moves on like this: >> hot -> cold >> cold -> (delete) >> warm -> cold >> >> Based on this understanding, let me describe my (failed) attempt to >> remove a file from cache. >> >> First, let's use 90%: >> $ doas sysctl kern.bufcachepercent >> kern.bufcachepercent=90 >> >> My machine has 16GB of RAM, but the end of the blog post says >> something about himem, so maybe this means only 0.9*4GB=3.6GB are >> used. >> >> Now let's create a new file t1, of size 512MB: >> $ dd bs=1m count=512 if=/dev/zero of=t1 >> 536870912 bytes transferred in 3.458 secs (155210354 bytes/sec) >> >> t1 should be in the hot queue now. It can be read at 2GB/s (way faster >> than disk), so at least we can be sure it is in *some* queue: >> $ dd bs=1m if=t1 of=/dev/null >> 536870912 bytes transferred in 0.263 secs (2036532935 bytes/sec) >> >> Now let's fill the hot and cold queues with something else: >> $ dd bs=1m count=16384 if=/dev/zero of=t2 >> 17179869184 bytes transferred in 57.210 secs (300290968 bytes/sec) >> >> This should have moved t1 first from the warm to the cold queue and >> then removed it from the cold queue. But strangely it can still be >> read at 2GB/s: >> $ dd bs=1m if=t1 of=/dev/null >> 536870912 bytes transferred in 0.251 secs (2135681199 bytes/sec) >> >> Why is this? How come t1 is still in the cache? >> > > Because it is more complicated. Because buffers are flipped from DMA mem > high mem in some situations. IIRC if the DMA hot queue is full the tail > buffers are flipped. Also just doing one dd is not enough to move buffers > between queues. For that they need to be read multiple times. Also write > buffers are never put in high mem or actually flipped down when written. > > In short the buffer cache is a complex beast and the few statistic numbers > systat are not enough to correctly understand the various states buffers > can be in. > > -- > :wq Claudio >
Re: clearing the disk cache
On Tue, Jul 03, 2018 at 01:30:20PM +0200, Maximilian Pichler wrote: > On Tue, Jul 3, 2018 at 11:47 AM, Janne Johansson wrote: > > https://www.tedunangst.com/flak/post/2Q-buffer-cache-algorithm > > Thanks. If I'm reading this correctly upon access (read or write), an > action is performed depending on what queue a buffer is in: > - none: Take a buffer from the tail of the cold queue and insert it at > the front of the hot queue. > - hot: Keep it there, but move to the front. > - cold: Move it to the front of the warm queue. > - warm: Keep it there, but move to the front. > > If a buffer reaches the tail of a queue it moves on like this: > hot -> cold > cold -> (delete) > warm -> cold > > Based on this understanding, let me describe my (failed) attempt to > remove a file from cache. > > First, let's use 90%: > $ doas sysctl kern.bufcachepercent > kern.bufcachepercent=90 > > My machine has 16GB of RAM, but the end of the blog post says > something about himem, so maybe this means only 0.9*4GB=3.6GB are > used. > > Now let's create a new file t1, of size 512MB: > $ dd bs=1m count=512 if=/dev/zero of=t1 > 536870912 bytes transferred in 3.458 secs (155210354 bytes/sec) > > t1 should be in the hot queue now. It can be read at 2GB/s (way faster > than disk), so at least we can be sure it is in *some* queue: > $ dd bs=1m if=t1 of=/dev/null > 536870912 bytes transferred in 0.263 secs (2036532935 bytes/sec) > > Now let's fill the hot and cold queues with something else: > $ dd bs=1m count=16384 if=/dev/zero of=t2 > 17179869184 bytes transferred in 57.210 secs (300290968 bytes/sec) > > This should have moved t1 first from the warm to the cold queue and > then removed it from the cold queue. But strangely it can still be > read at 2GB/s: > $ dd bs=1m if=t1 of=/dev/null > 536870912 bytes transferred in 0.251 secs (2135681199 bytes/sec) > > Why is this? How come t1 is still in the cache? > Because it is more complicated. Because buffers are flipped from DMA mem high mem in some situations. IIRC if the DMA hot queue is full the tail buffers are flipped. Also just doing one dd is not enough to move buffers between queues. For that they need to be read multiple times. Also write buffers are never put in high mem or actually flipped down when written. In short the buffer cache is a complex beast and the few statistic numbers systat are not enough to correctly understand the various states buffers can be in. -- :wq Claudio
Re: clearing the disk cache
On Tue, Jul 3, 2018 at 11:47 AM, Janne Johansson wrote: > https://www.tedunangst.com/flak/post/2Q-buffer-cache-algorithm Thanks. If I'm reading this correctly upon access (read or write), an action is performed depending on what queue a buffer is in: - none: Take a buffer from the tail of the cold queue and insert it at the front of the hot queue. - hot: Keep it there, but move to the front. - cold: Move it to the front of the warm queue. - warm: Keep it there, but move to the front. If a buffer reaches the tail of a queue it moves on like this: hot -> cold cold -> (delete) warm -> cold Based on this understanding, let me describe my (failed) attempt to remove a file from cache. First, let's use 90%: $ doas sysctl kern.bufcachepercent kern.bufcachepercent=90 My machine has 16GB of RAM, but the end of the blog post says something about himem, so maybe this means only 0.9*4GB=3.6GB are used. Now let's create a new file t1, of size 512MB: $ dd bs=1m count=512 if=/dev/zero of=t1 536870912 bytes transferred in 3.458 secs (155210354 bytes/sec) t1 should be in the hot queue now. It can be read at 2GB/s (way faster than disk), so at least we can be sure it is in *some* queue: $ dd bs=1m if=t1 of=/dev/null 536870912 bytes transferred in 0.263 secs (2036532935 bytes/sec) Now let's fill the hot and cold queues with something else: $ dd bs=1m count=16384 if=/dev/zero of=t2 17179869184 bytes transferred in 57.210 secs (300290968 bytes/sec) This should have moved t1 first from the warm to the cold queue and then removed it from the cold queue. But strangely it can still be read at 2GB/s: $ dd bs=1m if=t1 of=/dev/null 536870912 bytes transferred in 0.251 secs (2135681199 bytes/sec) Why is this? How come t1 is still in the cache? (Disclaimer: I've omitted the boring 'records' stats output by dd everywhere.)
Re: clearing the disk cache
Den tis 3 juli 2018 kl 10:59 skrev Maximilian Pichler < maxim.pich...@gmail.com>: > > > The buffer cache is implemented as two 2-queue and therefor a simple cat > > bigfile will not fill the cache. > > What sort of data structure or algorithm is this? Any reference would > be much appreciated. > > > 2Q https://www.tedunangst.com/flak/post/2Q-buffer-cache-algorithm -- May the most significant bit of your life be positive.
Re: clearing the disk cache
On Tue, Jul 3, 2018 at 9:48 AM, Claudio Jeker wrote: > or reboot the system after every run. Would unmounting and remounting not be good enough? (At least for a FAT-formatted SD card this appears to work, though it could be caused by something else). In fact, at what level does caching happen? Is it chunks of the disk that are cached, or chunks of files? > The buffer cache is implemented as two 2-queue and therefor a simple cat > bigfile will not fill the cache. What sort of data structure or algorithm is this? Any reference would be much appreciated. Thanks!
Re: clearing the disk cache
On Tue, 3 Jul 2018 09:42:46 +0200 Maximilian Pichler wrote: > I'm doing some performance tests that include reading files from disk > and want to make sure that each test takes place under similar > conditions. > > In particular, how can one clear the disk cache? (I want to make sure > that the second test isn't faster than the first one, just because > some files they both use are still in cache.) > > Right now I'm doing: > $ cat some_file_the_size_of_RAM > /dev/null > > Does this indeed clear the cache? Is there a better way? $ doas sysctl kern.bufcachepercent=5 This will knock the disk cache down to 5% of RAM space. You can not go lower, so a reboot between tests is the best way to have a clean state. Default is 20%, max. is 90% (which I use). > Also, are there several level of file/disk caches or just one? Just the one buffercache.
Re: clearing the disk cache
On Tue, Jul 03, 2018 at 09:42:46AM +0200, Maximilian Pichler wrote: > I'm doing some performance tests that include reading files from disk > and want to make sure that each test takes place under similar > conditions. > > In particular, how can one clear the disk cache? (I want to make sure > that the second test isn't faster than the first one, just because > some files they both use are still in cache.) > > Right now I'm doing: > $ cat some_file_the_size_of_RAM > /dev/null > > Does this indeed clear the cache? Is there a better way? > > Also, are there several level of file/disk caches or just one? > > As always, thanks for your advice! > This will not clear the cache. Either run the test multiple times and remove the first run or reboot the system after every run. The buffer cache is implemented as two 2-queue and therefor a simple cat bigfile will not fill the cache. You would need to read the file multiple times and even then you may not manage. -- :wq Claudio
clearing the disk cache
I'm doing some performance tests that include reading files from disk and want to make sure that each test takes place under similar conditions. In particular, how can one clear the disk cache? (I want to make sure that the second test isn't faster than the first one, just because some files they both use are still in cache.) Right now I'm doing: $ cat some_file_the_size_of_RAM > /dev/null Does this indeed clear the cache? Is there a better way? Also, are there several level of file/disk caches or just one? As always, thanks for your advice!
