Andrew M. Bishop
Mon, 10 Sep 2007 21:06:45 -0700
Ian Stirling <[EMAIL PROTECTED]> writes: > >>So, naturally, when I realised a need for a possibly related bit of > >>software, > >>and lacking results from google. I started wondering if anyone had thought > >>of, > >>or knows of an implementation or proof of concept with wwwoffled. > >> > >>Basically, it's a two part web proxy to drastically reduce web usage > >>bandwidth. > >>One part resides on a mobile device with a (usually) poor bandwidth link, > >>but > >>relatively large amount of storage, that may occasionally be plugged into a > >>high > >>speed network. > >> > >>The other part is on server, connected via a fast connection to the > >>internet. > > Have you seen the (no longer maintained) rsync-over-HTTP protocol that > > is at http://rproxy.samba.org/. This combines rsync and HTTP to be > > able to only transfer the updates just like you want. > > They describe just what you are looking for, but since nobody was > > interested about using it they seem to have given up. > > Interesting, thanks! > Googling had found me little. I remembered having seen the combination of rsync and HTTP before so it was easy for me to find. > > Before you ask, I don't plan to add it into WWWOFFLE. > > Initial design was planned to be a little server on host and client, that > implemented this by changing the files in the wwwoffle cache, rather than > properly implementing stuff. > > If eventually I generated a patch for WWWOFFLE, that mostly 'just worked', > without greatly complicating the core logic, are there any circumstances in > which it might be included? The reason that I say I don't plan to add it in to WWWOFFLE is that this is my standard reply when people ask me to extend WWWOFFLE way beyond its primary task. WWWOFFLE is a proxy cache that keeps a copy for offline viewing, allows offline requesting and automated online fetching and to help with this provides lots of options that reduce re-requesting cached URLs. There only needs to be one WWWOFFLE proxy and this resides at the user's end of the link. What you are suggesting is a pair of proxies, one each end of the low bandwidth link, that act together to minimise the data flowing over that link. The one on the internet end of the link (not the client end) could get new pages from the internet, or from WWWOFFLE or use the WWWOFFLE cache directly. The most general way of implementing it is for it to have an upstream proxy that it gets pages from. There is no need for it to keep cached pages itself, it only needs to have an accessible source of pages which could be WWWOFFLE or could be an uncached internet connection. For the client end of the link there does need to be a store of pages so a local proxy function would be needed. If you are planning something that requires an instance of WWWOFFLE at each of the link then I don't see the point in it, it is all too specific. If you are thinking of making it general so that WWWOFFLE is used at the client end and the existing rproxy implementation at the internet end then it would be much better (and it would close Debian bug number 150716). The problem is that all this is rather specific to the rsync HTTP extensions which seem to be unsupported due to lack of interest. There seem to be very few cases where a user would have access to an rsync supporting proxy at the the internet end to make this useful. -- Andrew. ---------------------------------------------------------------------- Andrew M. Bishop [EMAIL PROTECTED] http://www.gedanken.demon.co.uk/ WWWOFFLE users page: http://www.gedanken.demon.co.uk/wwwoffle/version-2.9/user.html