Updated disc cache
I know several RISC OS users regularly use the CI builds and have had issues with the disc cache. This is partly a request for assistance and partly a warning. I have recently changed the disc based caching to use fewer small files. This change is not backwards compatible and will leave the old cache files behind. I would suggest that any of you using the disc cache to delete it before running a NetSurf CI version after #2696 NetSurf will continue to run just fine if you do not but all the old cache files will be left behind and never cleaned up. The upside of this change is that it *may* help with performance for those of you that were seeing repeated warnings about insufficient disc bandwidth. As I have explained previously on several occasions the RISC OS filesystem performance appears to be very poor when using several small files, the new system uses a handful of large files as well to remove this as an issue. If you have previously disabled the cache please can I ask you to retry with the newer versions and see if the performance has improved? If you are feeling very adventurous you can report the bandwidth achieved. This is a line in the debug Log file held in scrap *after* the browser has been quit. The last line of the Log will read something like: (2298.806358) desktop/netsurf.c netsurf_exit 294: Exited successfully The bandwidth line will be about 20 lines from the end of the log and look like (2298.804881) content/llcache.c llcache_finalise 3352: Backing store average bandwidth 128324035 bytes/second -- Regards Vincent http://www.kyllikki.org/
Re: Updated disc cache
In article 20150403111441.gb18...@kyllikki.org, Vincent Sanders vi...@netsurf-browser.org wrote: (2298.804881) content/llcache.c llcache_finalise 3352: Backing store average bandwidth 128324035 bytes/second Hells bells - you'll be lucky to a tenth of that speed on RISC OS hardware and probably less on usb or sdfs based drives. I'll certainly check it out on my machines. -- Chris Johnson
Re: Updated disc cache
Vincent Sanders, on 3 Apr, wrote: [snip - cache bandwidth] NetSurf 2696 RPi2 SDFS 6067 bytes/s RPi2 Fat32FS 15220 bytes/s Iyonix320252 bytes/s A9home509265 bytes/s VRPC W7 SSD 605771 bytes/s -- David Pitt
Re: Updated disc cache
cj, on 3 Apr, wrote: In article 20150403111441.gb18...@kyllikki.org, Vincent Sanders vi...@netsurf-browser.org wrote: The bandwidth line will be about 20 lines from the end of the log I restarted Netsurf with cache enabled on the Iyonix. Loaded up the ROOL forum. Message came up almost immediately that the cache was being disabled. Can't say that I blame it! The ROOL forum content is particularly turgid at the moment, no sensible software would see any purpose in cacheing that. Quit again. Write speed details. (169.28) content/llcache.c llcache_finalise 3352: Backing store average bandwidth 95831 bytes/second Doesn't look very hopeful does it. Hmm! My Iyonix did over three time better than that, and there was no too slow message. My test piece was http://www.dailymail.co.uk because that is a particularly heavy duty site. -- David Pitt
Re: Updated disc cache
On Fri, Apr 03, 2015 at 01:30:14PM +0100, nets...@avisoft.f9.co.uk wrote: In article 54ae82a927ch...@chris-johnson.org.uk, cj ch...@chris-johnson.org.uk wrote: I can see why RISC OS gets indigestion with the cache. Have just deleted the cache on the Iyonix, and there were over 21,000 directories and over 19,000 files. Yes, there seem to be lots of directories - many empty. The non-empty ones often only contain directories. And the ones containing files rarely have more than 1 file in! Strikes me that there are too many directory levels. Insetad of having !Cache.Caches.Default.NetSurf.m.A.3.G.A3GKETD it may be better to have !Cache.Caches.Default.NetSurf.mA3G.A3GKETD I suspect much of the delay for small files is due to checking, creating, and traversing directories! The depth was chosen so it would work on poor-quality file systems that only allow a handful of entries in a directory, such as FileCore :) B.
Re: Updated disc cache
Vincent Sanders wrote on 3 Apr: If you are feeling very adventurous you can report the bandwidth achieved. This is a line in the debug Log file held in scrap *after* the browser has been quit. The last line of the Log will read something like: (2298.806358) desktop/netsurf.c netsurf_exit 294: Exited successfully The bandwidth line will be about 20 lines from the end of the log and look like (2298.804881) content/llcache.c llcache_finalise 3352: Backing store average bandwidth 128324035 bytes/second Thanks for the work. Where do we report it TO? Can you give a quicklink, please? -- Jim Nagelwww.archivemag.co.uk
Re: Updated disc cache
On Fri, Apr 03, 2015 at 12:48:39PM +0100, Jim Nagel wrote: Vincent Sanders wrote on 3 Apr: If you are feeling very adventurous you can report the bandwidth achieved. This is a line in the debug Log file held in scrap *after* the browser has been quit. The last line of the Log will read something like: (2298.806358) desktop/netsurf.c netsurf_exit 294: Exited successfully The bandwidth line will be about 20 lines from the end of the log and look like (2298.804881) content/llcache.c llcache_finalise 3352: Backing store average bandwidth 128324035 bytes/second Thanks for the work. Where do we report it TO? Can you give a quicklink, please? this is an informal poll, so to the list is fine. -- Jim Nagelwww.archivemag.co.uk -- Regards Vincent http://www.kyllikki.org/
Re: Updated disc cache
I can see why RISC OS gets indigestion with the cache. Have just deleted the cache on the Iyonix, and there were over 21,000 directories and over 19,000 files. -- Chris Johnson
Re: Updated disc cache
In article 20150403111441.gb18...@kyllikki.org, Vincent Sanders vi...@netsurf-browser.org wrote: The bandwidth line will be about 20 lines from the end of the log I restarted Netsurf with cache enabled on the Iyonix. Loaded up the ROOL forum. Message came up almost immediately that the cache was being disabled. Quit again. Write speed details. (169.28) content/llcache.c llcache_finalise 3352: Backing store average bandwidth 95831 bytes/second Doesn't look very hopeful does it. I'll try on the PandaBoard later. -- Chris Johnson
Re: unreadably narrow column in rendering of page
On 31 March 2015 at 16:44, Jim Nagel nets...@abbeypress.co.uk wrote: These pages from local newspaper takes ages in the fetching-processing stage, and then finally displays its text in a pane that is too narrow to read. http://www.centralsomersetgazette.co.uk/Just-doctor-ordered/story-26229361-detail/story.html http://www.centralsomersetgazette.co.uk/Street-schoolgirl-gains-place-national-theatre/story-26181050-detail/story.html Dunno if the fault is Netsurf or the designer of the page. Firefox doesn't seem to be doing a much better job. It doesn't look like a NetSurf issue. Cheers, JFL -- Jean-François Lemaire
Re: Updated disc cache
I have now tried on the PandaBoard. Used random pages from the Daily Mail site (not much content if you are not interested in celebrates!). The first time I tried I fairly quickly ended up with the cache being disabled - the logged average speed was not much over 100 KB/s. However, I then reran Netsurf and using the same site but different pages, there was no disabling and after wasting about 20 mins of my life, the average speed logged was close on 500 KB/s, which was much better. This is on a usb connected SSD, formated to filecore, which is not as fast as Fat32fs formatted. Is the caching possibly slower at first when things are being set up, and faster afterwards? The pages do render much faster on the PB than on the Iyonix. -- Chris Johnson
Re: unreadably narrow column in rendering of page
On Tue, Mar 31, 2015 at 03:44:06PM +0100, Jim Nagel wrote: These pages from local newspaper takes ages in the fetching-processing stage, and then finally displays its text in a pane that is too narrow to read. http://www.centralsomersetgazette.co.uk/Just-doctor-ordered/story-26229361-detail/story.html http://www.centralsomersetgazette.co.uk/Street-schoolgirl-gains-place-national-theatre/story-26181050-detail/story.html Dunno if the fault is Netsurf or the designer of the page. The wife (voice teacher for pupils in stories) is not impressed. Netsurf #2644 on Iyonix (RiscOS 5.18). -- Jim Nagelwww.archivemag.co.uk I reported that in the bug database for you as bug number 2303. -- Regards Vincent http://www.kyllikki.org/
Re: Updated disc cache
In article 20150403111441.gb18...@kyllikki.org, Vincent Sanders vi...@netsurf-browser.org wrote: If you are feeling very adventurous you can report the bandwidth achieved. This is a line in the debug Log file held in scrap *after* the browser has been quit. The last line of the Log will read something like: (2298.806358) desktop/netsurf.c netsurf_exit 294: Exited successfully Average bandwidth 355822 bytes/second NetSurf CI #2680 ARMX6 -- Stuart Winsor Tools With A Mission sending tools across the world http://www.twam.co.uk/
Re: Updated disc cache
cj, on 3 Apr, wrote: In article mpro.nm8dx001qojsl00l7.pit...@pittdj.co.uk, David Pitt pit...@pittdj.co.uk wrote: Hmm! My Iyonix did over three time better than that, and there was no too slow message. My test piece was http://www.dailymail.co.uk because that is a particularly heavy duty site. OK. A lot of random browsing around that site led to: (5743.13) content/llcache.c llcache_finalise 3352: Backing store average bandwidth 531777 bytes/second which is over 5 times faster. However, I thought we would be talking drive speed, which shouldn't be affected by the download speed of any particular site, or am I completely up the wrong alley? I have just a bit of a take two on the Daily Mail site, a longer session, and this time got an average bandwidth of 447397bytes/second on the Iyonix. I too would think that cacheing would mainly be about disc speed but I also think NetSurf's performance is severely hampered by the slow processors available to RISC OS. -- David Pitt
Re: unreadably narrow column in rendering of page
J. F. Lemaire, on 3 Apr, wrote: On 31 March 2015 at 16:44, Jim Nagel nets...@abbeypress.co.uk wrote: These pages from local newspaper takes ages in the fetching-processing stage, and then finally displays its text in a pane that is too narrow to read. http://www.centralsomersetgazette.co.uk/Just-doctor-ordered/story-26229361-detail/story.html http://www.centralsomersetgazette.co.uk/Street-schoolgirl-gains-place-national-theatre/story-26181050-detail/story.html Dunno if the fault is Netsurf or the designer of the page. Firefox doesn't seem to be doing a much better job. It doesn't look like a NetSurf issue. All good here in Firefox 36.0.4. In NetSurf #2696 there is a large blank space to the left of the story. -- David Pitt
Re: Updated disc cache
On Fri, Apr 03, 2015 at 02:39:05PM +0100, cj wrote: In article mpro.nm8dx001qojsl00l7.pit...@pittdj.co.uk, David Pitt pit...@pittdj.co.uk wrote: Hmm! My Iyonix did over three time better than that, and there was no too slow message. My test piece was http://www.dailymail.co.uk because that is a particularly heavy duty site. OK. A lot of random browsing around that site led to: (5743.13) content/llcache.c llcache_finalise 3352: Backing store average bandwidth 531777 bytes/second which is over 5 times faster. However, I thought we would be talking drive speed, which shouldn't be affected by the download speed of any particular site, or am I completely up the wrong alley? That value is *purely* the total amount *written* to disc divided by how long the write operations took. The write time includes all directory creation/seek operations etc. rather than just the raw disc write performance. Anything above a megabit a second (125000 bytes/second) will not trigger the warning about low write speed. I set it there because below that value the overheads of disc caching exceed the benefit of simply fetching data from the network. -- Chris Johnson -- Regards Vincent http://www.kyllikki.org/
Re: Updated disc cache
In article mpro.nm8dx001qojsl00l7.pit...@pittdj.co.uk, David Pitt pit...@pittdj.co.uk wrote: Hmm! My Iyonix did over three time better than that, and there was no too slow message. My test piece was http://www.dailymail.co.uk because that is a particularly heavy duty site. OK. A lot of random browsing around that site led to: (5743.13) content/llcache.c llcache_finalise 3352: Backing store average bandwidth 531777 bytes/second which is over 5 times faster. However, I thought we would be talking drive speed, which shouldn't be affected by the download speed of any particular site, or am I completely up the wrong alley? -- Chris Johnson
Re: Updated disc cache
In article 20150403131050.gq29...@platypus.pepperfish.net, Rob Kendrick r...@netsurf-browser.org wrote: On Fri, Apr 03, 2015 at 01:30:14PM +0100, nets...@avisoft.f9.co.uk wrote: In article 54ae82a927ch...@chris-johnson.org.uk, cj ch...@chris-johnson.org.uk wrote: I can see why RISC OS gets indigestion with the cache. Have just deleted the cache on the Iyonix, and there were over 21,000 directories and over 19,000 files. Yes, there seem to be lots of directories - many empty. The non-empty ones often only contain directories. And the ones containing files rarely have more than 1 file in! Strikes me that there are too many directory levels. Insetad of having !Cache.Caches.Default.NetSurf.m.A.3.G.A3GKETD it may be better to have !Cache.Caches.Default.NetSurf.mA3G.A3GKETD I suspect much of the delay for small files is due to checking, creating, and traversing directories! The depth was chosen so it would work on poor-quality file systems that only allow a handful of entries in a directory, such as FileCore :) It is a shame that there is no simple way to discover if 'big directories' are being used with no such limitations, as they have been here for many many years. Martin
Re: Updated disc cache
snip I suspect much of the delay for small files is due to checking, creating, and traversing directories! The depth was chosen so it would work on poor-quality file systems that only allow a handful of entries in a directory, such as FileCore :) It is a shame that there is no simple way to discover if 'big directories' are being used with no such limitations, as they have been here for many many years. A bit technical for this list but http://git.netsurf-browser.org/netsurf.git/tree/content/fs_backing_store.c#n326 explains all the constraints from all the different systems the cache must deal with, the result is lowest common denominator. Beleive me when I say working out that limit set from a many dimensional dataset like that was not easy With the changes I have just made however more than 90% of metadata and 70% of actual data ends up in the large block files and causes far fewer directories to be created substantially reducing overheads. -- Regards Vincent http://www.kyllikki.org/
Re: Updated disc cache
On Fri, Apr 03, 2015 at 03:13:17PM +0100, David Pitt wrote: I also think NetSurf's performance is severely hampered by the slow processors available to RISC OS. No, the CPUs are perfectly adequately fast. A Raspberry Pi can do many megabytes a second when running Linux. RISC OS's IO layer and file system stack is simply too old-fashioned and simplistic (even by 1980s standards). B.
