Re: Updated disc cache
On 3 Apr 2015 as I do recall, Vincent Sanders wrote: [snip] If you are feeling very adventurous you can report the bandwidth achieved. This is a line in the debug Log file held in scrap *after* the browser has been quit. The last line of the Log will read something like: (2298.806358) desktop/netsurf.c netsurf_exit 294: Exited successfully The bandwidth line will be about 20 lines from the end of the log and look like (2298.804881) content/llcache.c llcache_finalise 3352: Backing store average bandwidth 128324035 bytes/second Results with Netsurf 2687 on Iyonix: the program now *sometimes* doesn't complain about disc cache bandwidth, whereas previously it invariably complained! It seems to be completely random: here are results from three runs done within minutes of each other, of which the cache failed on two almost instantly and gave no trouble on the third, which was image-heavy. (75.12) content/llcache.c llcache_finalise 3352: Backing store average bandwidth 13140 bytes/second (84.50) content/llcache.c llcache_finalise 3352: Backing store average bandwidth 4887 bytes/second (239.03) content/llcache.c llcache_finalise 3352: Backing store average bandwidth 210663 bytes/second -- Harriet Bazley == Loyaulte me lie == People who live in stone houses shouldn't throw glasses.
Re: Updated disc cache summary
In article 55256ea0.8010...@netsurf-browser.org, Michael Drake t...@netsurf-browser.org wrote: On 08/04/15 12:41, Chris Newman wrote: So given all this, on my RiscPC Strong ARMv4 Adjust 4.39 with Unipod, to what should I set the Cache parameters in NetSurf Choices? Too slow to be useful. Set disc cache size to 0. Ta. -- Chris
Re: Updated disc cache summary
In article 20150408104544.gg18...@kyllikki.org, Vincent Sanders vi...@netsurf-browser.org wrote: Just to summarise the outcome of all the observations on the improved disc cache. The improvements make the cache viable on many more supported systems, including more RISC OS systems. On PC with modern OS it made no great improvement as their OS could already cope with the directory layout and had a plentiful write speed already. No test I ran on PC saw the write rate, before or after the changes, drop below 30 megabytes a second. Although the updated cache does use less processor time to achieve its results. On RISC OS the benefit is stark and very clear. Most tests show a five fold or more improvement in write performace using the new code and many fewer directories created. Big snip So given all this, on my RiscPC Strong ARMv4 Adjust 4.39 with Unipod, to what should I set the Cache parameters in Netsurf Choices? Will the cache automaticaly be made in IDEFS::h-4.$.!Boot.Choices.Users.Single.WWW.NetSurf Regards, -- Chris
Re: Updated disc cache summary
On 08/04/15 12:41, Chris Newman wrote: So given all this, on my RiscPC Strong ARMv4 Adjust 4.39 with Unipod, to what should I set the Cache parameters in NetSurf Choices? Too slow to be useful. Set disc cache size to 0. -- Michael Drake http://www.netsurf-browser.org/
Re: Updated disc cache
In message 20150403111441.gb18...@kyllikki.org on 3 Apr 2015 Vincent Sanders vi...@netsurf-browser.org wrote: I know several RISC OS users regularly use the CI builds and have had issues with the disc cache. This is partly a request for assistance and partly a warning. I have recently changed the disc based caching to use fewer small files. This change is not backwards compatible and will leave the old cache files behind. I would suggest that any of you using the disc cache to delete it before running a NetSurf CI version after #2696 NetSurf will continue to run just fine if you do not but all the old cache files will be left behind and never cleaned up. The upside of this change is that it *may* help with performance for those of you that were seeing repeated warnings about insufficient disc bandwidth. As I have explained previously on several occasions the RISC OS filesystem performance appears to be very poor when using several small files, the new system uses a handful of large files as well to remove this as an issue. If you have previously disabled the cache please can I ask you to retry with the newer versions and see if the performance has improved? If you are feeling very adventurous you can report the bandwidth achieved. This is a line in the debug Log file held in scrap *after* the browser has been quit. The last line of the Log will read something like: (2298.806358) desktop/netsurf.c netsurf_exit 294: Exited successfully The bandwidth line will be about 20 lines from the end of the log and look like (2298.804881) content/llcache.c llcache_finalise 3352: Backing store average bandwidth 128324035 bytes/second ARMini with RO 5.20 (10 June 2013) NS #2697 I went only to the www.buxtonweather.co.uk website. In the log I found these (20.42) content/llcache.c llcache_persist 2494: Overran timeslot (30.88) content/llcache.c llcache_persist_slowcheck 2438: Cannot write minimum bandwidth (59.82) content/llcache.c llcache_finalise 3352: Backing store average bandwidth 33092 bytes/second That last line was many more than 20 lines from the end of the file Regards Andrew -- Andrew Pinder
Re: Updated disc cache
In article 20150403135750.ge18...@kyllikki.org, Vincent Sanders vi...@netsurf-browser.org wrote: snip I suspect much of the delay for small files is due to checking, creating, and traversing directories! The depth was chosen so it would work on poor-quality file systems that only allow a handful of entries in a directory, such as FileCore :) It is a shame that there is no simple way to discover if 'big directories' are being used with no such limitations, as they have been here for many many years. A bit technical for this list but http://git.netsurf-browser.org/netsurf.git/tree/content/fs_backing_store.c#n326 explains ... I did have a look, but clear as mud to me. I have just deleted my cache from Netsurf v3.3 - about 2,500 files ... and about 12,500 directories! That is about 5 directories for each file, and I would hope for at least 50 files in each directory! But I do understand that it is not simple task!
Re: Updated disc cache
In message 54af215e1bch...@chris-johnson.org.uk on 4 Apr 2015 cj ch...@chris-johnson.org.uk wrote: In article 54aec5195fstuartli...@orpheusinternet.co.uk, lists stuartli...@orpheusinternet.co.uk wrote: Average bandwidth 355822 bytes/second NetSurf CI #2680 ARMX6 So nothing much to write home about there, considering some of the hype surrounding the disc speed of the ARMX6. Stuart is reporting values from a version (#2680) before the upgrade that improves things (after #2696). Regards Andrew -- Andrew Pinder
Re: Updated disc cache
In article e26e3db054.andrew-...@waitrose.com, Andrew Pinder andrew.pin...@tiscali.co.uk wrote: In article 54aec5195fstuartli...@orpheusinternet.co.uk, lists stuartli...@orpheusinternet.co.uk wrote: Average bandwidth 355822 bytes/second NetSurf CI #2680 ARMX6 So nothing much to write home about there, considering some of the hype surrounding the disc speed of the ARMX6. Stuart is reporting values from a version (#2680) before the upgrade that improves things (after #2696). Very quick look at the BBC site (I got a couple of time-outs from the daily mail before giving up) 2790273 with version CI # 2967. -- Stuart Winsor Tools With A Mission sending tools across the world http://www.twam.co.uk/
Re: Updated disc cache
In article 20150403111441.gb18...@kyllikki.org, Vincent Sanders vi...@netsurf-browser.org wrote: [Snip] If you are feeling very adventurous you can report the bandwidth achieved. [Snip] (152.54) content/llcache.c llcache_finalise 3352: Backing store average bandwidth 561256 bytes/second -- _ Brian Jordan Virtual RPC-AdjustSA on Windows 8.1 Pro RISC OS 6.20 _
Re: Updated disc cache
[snip] I would suggest that any of you using the disc cache to delete it before running a NetSurf CI version after #2696 NetSurf will continue to run just fine if you do not but all the old cache files will be left behind and never cleaned up. Is there a ',' or an '.' missing somewhere? The meaning is slightly fuzzy. [snip]
Re: Updated disc cache
In article 54af215e1bch...@chris-johnson.org.uk, cj ch...@chris-johnson.org.uk wrote: NetSurf CI #2680 ARMX6 So nothing much to write home about there, considering some of the hype surrounding the disc speed of the ARMX6. I'm not sure how much the download speed affects the results; I had several timeouts during the session. Perhaps we all need to try with the same site to get a relative measure. I would say the ARMX6 is pretty fast. When !Pluto is loaded it expires old articles. I always used to see the message expiring... and it took several seconds. I don't see the message anymore.anymore. -- Stuart Winsor Tools With A Mission sending tools across the world http://www.twam.co.uk/
Re: Updated disc cache
In article 54aec5195fstuartli...@orpheusinternet.co.uk, lists stuartli...@orpheusinternet.co.uk wrote: Average bandwidth 355822 bytes/second NetSurf CI #2680 ARMX6 So nothing much to write home about there, considering some of the hype surrounding the disc speed of the ARMX6. -- Chris Johnson
Re: Updated disc cache
In article 20150403111441.gb18...@kyllikki.org, Vincent Sanders vi...@netsurf-browser.org wrote: (2298.804881) content/llcache.c llcache_finalise 3352: Backing store average bandwidth 128324035 bytes/second Hells bells - you'll be lucky to a tenth of that speed on RISC OS hardware and probably less on usb or sdfs based drives. I'll certainly check it out on my machines. -- Chris Johnson
Re: Updated disc cache
Vincent Sanders, on 3 Apr, wrote: [snip - cache bandwidth] NetSurf 2696 RPi2 SDFS 6067 bytes/s RPi2 Fat32FS 15220 bytes/s Iyonix320252 bytes/s A9home509265 bytes/s VRPC W7 SSD 605771 bytes/s -- David Pitt
Re: Updated disc cache
cj, on 3 Apr, wrote: In article 20150403111441.gb18...@kyllikki.org, Vincent Sanders vi...@netsurf-browser.org wrote: The bandwidth line will be about 20 lines from the end of the log I restarted Netsurf with cache enabled on the Iyonix. Loaded up the ROOL forum. Message came up almost immediately that the cache was being disabled. Can't say that I blame it! The ROOL forum content is particularly turgid at the moment, no sensible software would see any purpose in cacheing that. Quit again. Write speed details. (169.28) content/llcache.c llcache_finalise 3352: Backing store average bandwidth 95831 bytes/second Doesn't look very hopeful does it. Hmm! My Iyonix did over three time better than that, and there was no too slow message. My test piece was http://www.dailymail.co.uk because that is a particularly heavy duty site. -- David Pitt
Re: Updated disc cache
On Fri, Apr 03, 2015 at 01:30:14PM +0100, nets...@avisoft.f9.co.uk wrote: In article 54ae82a927ch...@chris-johnson.org.uk, cj ch...@chris-johnson.org.uk wrote: I can see why RISC OS gets indigestion with the cache. Have just deleted the cache on the Iyonix, and there were over 21,000 directories and over 19,000 files. Yes, there seem to be lots of directories - many empty. The non-empty ones often only contain directories. And the ones containing files rarely have more than 1 file in! Strikes me that there are too many directory levels. Insetad of having !Cache.Caches.Default.NetSurf.m.A.3.G.A3GKETD it may be better to have !Cache.Caches.Default.NetSurf.mA3G.A3GKETD I suspect much of the delay for small files is due to checking, creating, and traversing directories! The depth was chosen so it would work on poor-quality file systems that only allow a handful of entries in a directory, such as FileCore :) B.
Re: Updated disc cache
Vincent Sanders wrote on 3 Apr: If you are feeling very adventurous you can report the bandwidth achieved. This is a line in the debug Log file held in scrap *after* the browser has been quit. The last line of the Log will read something like: (2298.806358) desktop/netsurf.c netsurf_exit 294: Exited successfully The bandwidth line will be about 20 lines from the end of the log and look like (2298.804881) content/llcache.c llcache_finalise 3352: Backing store average bandwidth 128324035 bytes/second Thanks for the work. Where do we report it TO? Can you give a quicklink, please? -- Jim Nagelwww.archivemag.co.uk
Re: Updated disc cache
On Fri, Apr 03, 2015 at 12:48:39PM +0100, Jim Nagel wrote: Vincent Sanders wrote on 3 Apr: If you are feeling very adventurous you can report the bandwidth achieved. This is a line in the debug Log file held in scrap *after* the browser has been quit. The last line of the Log will read something like: (2298.806358) desktop/netsurf.c netsurf_exit 294: Exited successfully The bandwidth line will be about 20 lines from the end of the log and look like (2298.804881) content/llcache.c llcache_finalise 3352: Backing store average bandwidth 128324035 bytes/second Thanks for the work. Where do we report it TO? Can you give a quicklink, please? this is an informal poll, so to the list is fine. -- Jim Nagelwww.archivemag.co.uk -- Regards Vincent http://www.kyllikki.org/
Re: Updated disc cache
I can see why RISC OS gets indigestion with the cache. Have just deleted the cache on the Iyonix, and there were over 21,000 directories and over 19,000 files. -- Chris Johnson
Re: Updated disc cache
In article 20150403111441.gb18...@kyllikki.org, Vincent Sanders vi...@netsurf-browser.org wrote: The bandwidth line will be about 20 lines from the end of the log I restarted Netsurf with cache enabled on the Iyonix. Loaded up the ROOL forum. Message came up almost immediately that the cache was being disabled. Quit again. Write speed details. (169.28) content/llcache.c llcache_finalise 3352: Backing store average bandwidth 95831 bytes/second Doesn't look very hopeful does it. I'll try on the PandaBoard later. -- Chris Johnson
Re: Updated disc cache
I have now tried on the PandaBoard. Used random pages from the Daily Mail site (not much content if you are not interested in celebrates!). The first time I tried I fairly quickly ended up with the cache being disabled - the logged average speed was not much over 100 KB/s. However, I then reran Netsurf and using the same site but different pages, there was no disabling and after wasting about 20 mins of my life, the average speed logged was close on 500 KB/s, which was much better. This is on a usb connected SSD, formated to filecore, which is not as fast as Fat32fs formatted. Is the caching possibly slower at first when things are being set up, and faster afterwards? The pages do render much faster on the PB than on the Iyonix. -- Chris Johnson
Re: Updated disc cache
In article 20150403111441.gb18...@kyllikki.org, Vincent Sanders vi...@netsurf-browser.org wrote: If you are feeling very adventurous you can report the bandwidth achieved. This is a line in the debug Log file held in scrap *after* the browser has been quit. The last line of the Log will read something like: (2298.806358) desktop/netsurf.c netsurf_exit 294: Exited successfully Average bandwidth 355822 bytes/second NetSurf CI #2680 ARMX6 -- Stuart Winsor Tools With A Mission sending tools across the world http://www.twam.co.uk/
Re: Updated disc cache
cj, on 3 Apr, wrote: In article mpro.nm8dx001qojsl00l7.pit...@pittdj.co.uk, David Pitt pit...@pittdj.co.uk wrote: Hmm! My Iyonix did over three time better than that, and there was no too slow message. My test piece was http://www.dailymail.co.uk because that is a particularly heavy duty site. OK. A lot of random browsing around that site led to: (5743.13) content/llcache.c llcache_finalise 3352: Backing store average bandwidth 531777 bytes/second which is over 5 times faster. However, I thought we would be talking drive speed, which shouldn't be affected by the download speed of any particular site, or am I completely up the wrong alley? I have just a bit of a take two on the Daily Mail site, a longer session, and this time got an average bandwidth of 447397bytes/second on the Iyonix. I too would think that cacheing would mainly be about disc speed but I also think NetSurf's performance is severely hampered by the slow processors available to RISC OS. -- David Pitt
Re: Updated disc cache
On Fri, Apr 03, 2015 at 02:39:05PM +0100, cj wrote: In article mpro.nm8dx001qojsl00l7.pit...@pittdj.co.uk, David Pitt pit...@pittdj.co.uk wrote: Hmm! My Iyonix did over three time better than that, and there was no too slow message. My test piece was http://www.dailymail.co.uk because that is a particularly heavy duty site. OK. A lot of random browsing around that site led to: (5743.13) content/llcache.c llcache_finalise 3352: Backing store average bandwidth 531777 bytes/second which is over 5 times faster. However, I thought we would be talking drive speed, which shouldn't be affected by the download speed of any particular site, or am I completely up the wrong alley? That value is *purely* the total amount *written* to disc divided by how long the write operations took. The write time includes all directory creation/seek operations etc. rather than just the raw disc write performance. Anything above a megabit a second (125000 bytes/second) will not trigger the warning about low write speed. I set it there because below that value the overheads of disc caching exceed the benefit of simply fetching data from the network. -- Chris Johnson -- Regards Vincent http://www.kyllikki.org/
Re: Updated disc cache
In article mpro.nm8dx001qojsl00l7.pit...@pittdj.co.uk, David Pitt pit...@pittdj.co.uk wrote: Hmm! My Iyonix did over three time better than that, and there was no too slow message. My test piece was http://www.dailymail.co.uk because that is a particularly heavy duty site. OK. A lot of random browsing around that site led to: (5743.13) content/llcache.c llcache_finalise 3352: Backing store average bandwidth 531777 bytes/second which is over 5 times faster. However, I thought we would be talking drive speed, which shouldn't be affected by the download speed of any particular site, or am I completely up the wrong alley? -- Chris Johnson
Re: Updated disc cache
In article 20150403131050.gq29...@platypus.pepperfish.net, Rob Kendrick r...@netsurf-browser.org wrote: On Fri, Apr 03, 2015 at 01:30:14PM +0100, nets...@avisoft.f9.co.uk wrote: In article 54ae82a927ch...@chris-johnson.org.uk, cj ch...@chris-johnson.org.uk wrote: I can see why RISC OS gets indigestion with the cache. Have just deleted the cache on the Iyonix, and there were over 21,000 directories and over 19,000 files. Yes, there seem to be lots of directories - many empty. The non-empty ones often only contain directories. And the ones containing files rarely have more than 1 file in! Strikes me that there are too many directory levels. Insetad of having !Cache.Caches.Default.NetSurf.m.A.3.G.A3GKETD it may be better to have !Cache.Caches.Default.NetSurf.mA3G.A3GKETD I suspect much of the delay for small files is due to checking, creating, and traversing directories! The depth was chosen so it would work on poor-quality file systems that only allow a handful of entries in a directory, such as FileCore :) It is a shame that there is no simple way to discover if 'big directories' are being used with no such limitations, as they have been here for many many years. Martin
Re: Updated disc cache
snip I suspect much of the delay for small files is due to checking, creating, and traversing directories! The depth was chosen so it would work on poor-quality file systems that only allow a handful of entries in a directory, such as FileCore :) It is a shame that there is no simple way to discover if 'big directories' are being used with no such limitations, as they have been here for many many years. A bit technical for this list but http://git.netsurf-browser.org/netsurf.git/tree/content/fs_backing_store.c#n326 explains all the constraints from all the different systems the cache must deal with, the result is lowest common denominator. Beleive me when I say working out that limit set from a many dimensional dataset like that was not easy With the changes I have just made however more than 90% of metadata and 70% of actual data ends up in the large block files and causes far fewer directories to be created substantially reducing overheads. -- Regards Vincent http://www.kyllikki.org/
Re: Updated disc cache
On Fri, Apr 03, 2015 at 03:13:17PM +0100, David Pitt wrote: I also think NetSurf's performance is severely hampered by the slow processors available to RISC OS. No, the CPUs are perfectly adequately fast. A Raspberry Pi can do many megabytes a second when running Linux. RISC OS's IO layer and file system stack is simply too old-fashioned and simplistic (even by 1980s standards). B.
