Re: More disc cache improvements
In message ac1710c154.andrew-...@waitrose.com on 9 May 2015 Andrew Pinder andrew.pin...@tiscali.co.uk wrote: In message 20150508154706.gb2...@kyllikki.org on 8 May 2015 Vincent Sanders vi...@netsurf-browser.org wrote: The information I want is contained in the log file at exit as most of you have discovered. I have improved whats reported with build CI 2774 and would appreciate two lines of output which appear all together near the end of the log. An example is: content/fs_backing_store.c finalise 1613: Cache total/hit/miss/fail (counts) 2620/240/2380/0 (100%/9%/90%/0%) content/llcache.c llcache_finalise 3361: Backing store wrote 96531600 bytes in 6617 ms average 14626000 bytes/second ARMini RO 5.22 with !Boot and !NetSurf #2774 on the SD card. It has returned a cache too slow message in the two sessions since installing #2774. Results from the most recent one are: (10335.07) content/fs_backing_store.c finalise 1613: Cache total/hit/miss/fail (counts) 523/88/435/0 (100%/16%/83%/0%) (20076.05) content/llcache.c llcache_finalise 3361: Backing store wrote 577764 bytes in 0 ms average 4530 bytes/second After a subsequent lengthy but intermittent session I eventually got the bandwidth warning so then quit and got: (17974.94) content/fs_backing_store.c finalise 1613: Cache total/hit/miss/fail (counts) 563/78/485/0 (100%/13%/86%/0%) (17990.75) content/llcache.c llcache_finalise 3361: Backing store wrote 1222126 bytes in 0 ms average 9334 bytes/second Regards Andrew -- Andrew Pinder
Re: More disc cache improvements
In message 20150508154706.gb2...@kyllikki.org Vincent Sanders vi...@netsurf-browser.org wrote: I would like these two lines from the logfile along with the OS and hardware spec of the system. E.g. RISC OS 5 on Iyonix with FAT formatted hard drive or ROOL beta on Raspberry Pi 2 with FAT formatted SD card (7033.74) content/fs_backing_store.c finalise 1613: Cache total/hit/miss/fail (counts) 2461/687/1774/0 (100%/27%/72%/0%) (7033.74) content/llcache.c llcache_finalise 3361: Backing store wrote 22931046 bytes in 0 ms average 71300 bytes/second Iyonix, RISC OS 5.22, 38 GiB spinning rust, Filecore formatted. NS 2774. Dave Can't remember your password? Do you need a strong and secure password? Use Password manager! It stores your passwords protects your account. Check it out at http://mysecurelogon.com/password-manager
Re: More disc cache improvements
In message 20150508154706.gb2...@kyllikki.org on 8 May 2015 Vincent Sanders vi...@netsurf-browser.org wrote: OK perhaps I needed to be more explicit in what I wanted to achieve from this testing. Thanks for the clarification [snip] The information I want is contained in the log file at exit as most of you have discovered. I have improved whats reported with build CI 2774 and would appreciate two lines of output which appear all together near the end of the log. An example is: content/fs_backing_store.c finalise 1613: Cache total/hit/miss/fail (counts) 2620/240/2380/0 (100%/9%/90%/0%) content/llcache.c llcache_finalise 3361: Backing store wrote 96531600 bytes in 6617 ms average 14626000 bytes/second [Snip] I would like these two lines from the logfile along with the OS and hardware spec of the system. E.g. RISC OS 5 on Iyonix with FAT formatted hard drive or ROOL beta on Raspberry Pi 2 with FAT formatted SD card ARMini RO 5.22 with !Boot and !NetSurf #2774 on the SD card. It has returned a cache too slow message in the two sessions since installing #2774. Results from the most recent one are: (10335.07) content/fs_backing_store.c finalise 1613: Cache total/hit/miss/fail (counts) 523/88/435/0 (100%/16%/83%/0%) (20076.05) content/llcache.c llcache_finalise 3361: Backing store wrote 577764 bytes in 0 ms average 4530 bytes/second Regards Andrew -- Andrew Pinder
Re: More disc cache improvements
In message e75637c054.andrew-...@waitrose.com on 7 May 2015 Andrew Pinder andrew.pin...@tiscali.co.uk wrote: I wonder how stable the measurements are. I've just upgraded this ARMini to RO 5.22 so have redone the measurements, still with NS#2771: 161126 bytes/second. That's a massive increase. I had visited a number of websites for that. Trying going to just www.buxtonweather.co.uk (my homepage) as before now gives 154588 bytes/second - only 4% different. Have I missed something, or have ROOL done something spectacular in the changes from 5.20 to 5.22? More measurements: Various sites including BBC election coverage: 212787 bytes/second Relaunch, load home page only and quit: 0 bytes/second ??? Same again, but force a refresh and then quit: 31146 bytes/second Same again, but after the buxtonweather.co.uk site has done its regular update (every five minutes): 0 bytes/second A wide range of sites, mostly new to me: 328299 bytes/second I'm curious that there is so much variability! Regards Andrew -- Andrew Pinder
Re: More disc cache improvements
In article 0c9680c054.andrew-...@waitrose.com, Andrew Pinder andrew.pin...@tiscali.co.uk wrote: I'm curious that there is so much variability! Will some of this not be due to the fact that stuff is already in the cache and won't be saved again? -- Chris Johnson
Re: More disc cache improvements
In message 54c085ed15ch...@chris-johnson.org.uk on 8 May 2015 cj ch...@chris-johnson.org.uk wrote: In article 0c9680c054.andrew-...@waitrose.com, Andrew Pinder andrew.pin...@tiscali.co.uk wrote: I'm curious that there is so much variability! Will some of this not be due to the fact that stuff is already in the cache and won't be saved again? I expect so, but if so it makes it difficult to assess the performance of the cache without explicit instructions on whether it should be cleared. Regards Andrew -- Andrew Pinder
Re: More disc cache improvements
In article 0c9680c054.andrew-...@waitrose.com, [snip] More measurements: Various sites including BBC election coverage: 212787 bytes/second Relaunch, load home page only and quit: 0 bytes/second ??? Same again, but force a refresh and then quit: 31146 bytes/second Same again, but after the buxtonweather.co.uk site has done its regular update (every five minutes): 0 bytes/second A wide range of sites, mostly new to me: 328299 bytes/second I'm curious that there is so much variability! Yes, me too. Seemingly, large variance here too. Kind regards Brian
Re: More disc cache improvements
In message 019bfbbe54.andrew-...@waitrose.com on 5 May 2015 Andrew Pinder andrew.pin...@tiscali.co.uk wrote: In message 20150505103028.gg19...@kyllikki.org on 5 May 2015 Vincent Sanders vi...@netsurf-browser.org wrote: Further to my previous efforts I have made an attempt to improve the disc cache performance even more. I would again be grateful if suitably interested users could try test CI build 2771 or later. [snip] I have also adjusted how the computation of low bandwidth is made to try and make it more stable and less trigger happy. I would be interested in getting the details from using the updated test release in the same way as previously detailed. (The overall bandwidth message from the log) I am especially interested in testing from the Iyonix as this was right on the edge of usefulness previously. It is probably not worth bothering with the RPi and ARM mini platforms as their reported bandwidth last time was awful and I do not envisage an improvement of that many orders of magnitude. As you expected, no benefit for ARMini users: ARMini with RO 5.20 (10 June 2013) and !Boot on the SD card, not the USB drive. NS #2771 There was no complaint about the bandwidth being too low but the reported average from going only to the www.buxtonweather.co.uk site was 27269 bytes/second. The previous figure I reported was 33092 bytes/second (NS#2697). I wonder how stable the measurements are. I've just upgraded this ARMini to RO 5.22 so have redone the measurements, still with NS#2771: 161126 bytes/second. That's a massive increase. I had visited a number of websites for that. Trying going to just www.buxtonweather.co.uk (my homepage) as before now gives 154588 bytes/second - only 4% different. Have I missed something, or have ROOL done something spectacular in the changes from 5.20 to 5.22? Regards Andrew -- Andrew Pinder
Re: More disc cache improvements
Vincent Sanders, on 5 May, wrote: Further to my previous efforts I have made an attempt to improve the disc cache performance even more. I would again be grateful if suitably interested users could try test CI build 2771 or later. [snip] I am especially interested in testing from the Iyonix as this was right on the edge of usefulness previously. [snip] NetSurf #2771 From the Iyonix on heavy duty news sites, 347744, 375645, 366788 and 537769 bytes/sec. Lighter sites with presumably less to cache returned 56655 and 44821 bytes/sec, but the too slow warning never appeared. On VRPC OS4.39 on Windows 7, 508804, 1033119 and 1119900 bytes/sec which is better than previously. -- David Pitt
Re: More disc cache improvements
In article 20150505103028.gg19...@kyllikki.org, Vincent Sanders vi...@netsurf-browser.org wrote: I am especially interested in testing from the Iyonix as this was right on the edge of usefulness previously. Just loaded a few random pages from BBC and from ROOL on the Iyonix. I did have the disc cache disabled before, because of bandwidth warnings etc. Cache now set to 1 GB. This is the result: (868.07) content/llcache.c llcache_finalise 3361: Backing store average bandwidth 652836 bytes/second This is certainly better than the last time I tried this test, probably getting on for about double the bandwidth. I'll try on the PB later. -- Chris Johnson
Re: More disc cache improvements
Using a PandaBoard with the cache on a Fat32 formatted SSD (not the SD card) gave the following: (663.19) content/llcache.c llcache_finalise 3361: Backing store average bandwidth 414186 bytes/second -- Chris Johnson
More disc cache improvements
Further to my previous efforts I have made an attempt to improve the disc cache performance even more. I would again be grateful if suitably interested users could try test CI build 2771 or later. The previous changes switched to using a small number of large files to hold all the small entries within the cache instead of them using individual files. This drastically improved the cache performance on several RISC OS machines. The new update extended this approach so that these block files get created with their maximum size instead of being grown every time a new cache entry is added. This change should be beneficial to RISC OS users as filecore is (apparently) dreadful at this kind of usage pattern. I have also adjusted how the computation of low bandwidth is made to try and make it more stable and less trigger happy. I would be interested in getting the details from using the updated test release in the same way as previously detailed. (The overall bandwidth message from the log) I am especially interested in testing from the Iyonix as this was right on the edge of usefulness previously. It is probably not worth bothering with the RPi and ARM mini platforms as their reported bandwidth last time was awful and I do not envisage an improvement of that many orders of magnitude. -- Regards Vincent http://www.kyllikki.org/
Re: More disc cache improvements
In article 20150505103028.gg19...@kyllikki.org, Vincent Sanders vi...@netsurf-browser.org wrote: This change should be beneficial to RISC OS users as filecore is (apparently) dreadful at this kind of usage pattern. I wonder if this is because RISC OS files are 'defragmented' all the time - I think I am correct in saying that when a file becomes too big for the contiguous space it currently occupies, it is copied into a larger block of free space. This means that as the file grows, it may be regularly being copied somewhere else on the disc, and with files that may be hundreds of MB in size, this will be slow - very much so when the drive is an SD card. -- Chris Johnson
Re: More disc cache improvements
On Tue, May 05, 2015 at 11:48:46AM +0100, cj wrote: In article 20150505103028.gg19...@kyllikki.org, Vincent Sanders vi...@netsurf-browser.org wrote: This change should be beneficial to RISC OS users as filecore is (apparently) dreadful at this kind of usage pattern. I wonder if this is because RISC OS files are 'defragmented' all the time - I think I am correct in saying that when a file becomes too big for the contiguous space it currently occupies, it is copied into a larger block of free space. This means that as the file grows, it may be regularly being copied somewhere else on the disc, and with files that may be hundreds of MB in size, this will be slow - very much so when the drive is an SD card. Ish. FileCore has supported fragmented files for a long time (ISTR E implements this). It was only earlier versions that didn't support fragmented files and thus had to rearrange on file growth. B.
Re: More disc cache improvements
In article 20150505103028.gg19...@kyllikki.org, Vincent Sanders vi...@netsurf-browser.org wrote: Further to my previous efforts I have made an attempt to improve the disc cache performance even more. I would again be grateful if suitably interested users could try test CI build 2771 or later. [snip] I would be interested in getting the details from using the updated test release in the same way as previously detailed. (The overall bandwidth message from the log) OK. VRPC, 4.02, content/llcache.c llcache_finalise 3361: Backing store average bandwidth 15983181 bytes/second Hope that this is useful. I am especially interested in testing from the Iyonix as this was right on the edge of usefulness previously. It is probably not worth bothering with the RPi and ARM mini platforms as their reported bandwidth last time was awful and I do not envisage an improvement of that many orders of magnitude.
Re: More disc cache improvements
In message 20150505103028.gg19...@kyllikki.org on 5 May 2015 Vincent Sanders vi...@netsurf-browser.org wrote: Further to my previous efforts I have made an attempt to improve the disc cache performance even more. I would again be grateful if suitably interested users could try test CI build 2771 or later. The previous changes switched to using a small number of large files to hold all the small entries within the cache instead of them using individual files. This drastically improved the cache performance on several RISC OS machines. The new update extended this approach so that these block files get created with their maximum size instead of being grown every time a new cache entry is added. This change should be beneficial to RISC OS users as filecore is (apparently) dreadful at this kind of usage pattern. I have also adjusted how the computation of low bandwidth is made to try and make it more stable and less trigger happy. I would be interested in getting the details from using the updated test release in the same way as previously detailed. (The overall bandwidth message from the log) I am especially interested in testing from the Iyonix as this was right on the edge of usefulness previously. It is probably not worth bothering with the RPi and ARM mini platforms as their reported bandwidth last time was awful and I do not envisage an improvement of that many orders of magnitude. As you expected, no benefit for ARMini users: ARMini with RO 5.20 (10 June 2013) and !Boot on the SD card, not the USB drive. NS #2771 There was no complaint about the bandwidth being too low but the reported average from going only to the www.buxtonweather.co.uk site was 27269 bytes/second. The previous figure I reported was 33092 bytes/second (NS#2697). Regards Andrew -- Andrew Pinder
