Yes, I have a very similar idea. At the moment you select a few folders to sync but in the future you want to have your complete home directory synced/available. Of course I don't have enough space on some of my devices like small netbooks, tablets, phones or even mig machines to hold all my movies, music, photos documents that I have on my big ownCloud. So the local storage should become something like a cache for the ownCloud server. You _see_ all the files always but if you access a file than it is checked if the file exists as a local cache file or if it has to be fetched from the server over the network. The files that you use most often are stored locally. Examples are your favorite music, movies and files.
The challenge is that this requires a custom filesystem. It is relatively easy on linux with fuse to implement. But we want to support Windows, Mac, Android and other devices/operating systems too. So this is a challenge. But definitely something that we want for the future. A single sync folder is so old-school ;-) Frank On 16.05.2013, at 13:03, Bjorn Madsen <[email protected]> wrote: > Hi I looked into the storage problem some time ago, and I think Klass could > solve it with some evolution of the current client (and its local usage of > SQLite), but to justify my thinking a little bit of background analysis is > required: > > File usage has some pretty consistent characteristics amongst my near peers: > > There is a high frequency usage of small files and low frequency usage of > large files i.e. powerlaw relationship: > 1. At the one end of the spectrum I want files that I use frequently on my > local device(s), so I do not have to wait for download. > 2. At the other end of the spectrum I have files that I rarely use, but want > access to at some unpredictable point in time within the next two decades. > 3. Files I use often are relatively small, and in contrast, the files I > rarely use are rather large. > 4. There are lots of small files and less larger ones. > > Behaviour: > 5. Users want access to their files from multiple devices of different > specification; a personal example is: > 5.1 I have* 2Gb for files on my mobile > 5.2 I have* 70Gb for files of 128 Gb SSD on my laptop > 5.3 I have* 90Gb for files of 160Gb desktop. > 5.4 I have* 3 Tb for files on my OwnCloud server (always online, with > fixed tcp/ip) - this could very well be other scale out services as well. > 5.5 Each device will be online at random times, and they can connect > securely together: > > The trick that makes it irrelevant how much local disk space I have, is that > I permit LOCAL OVERWRITE IFF there is GLOBAL REDUNDANCY (via OwnCloud). > > If a file is updated on the one device I want it up-to-date on the other > devices as soon as possible (via web), but only IF it is used MORE THAN ONCE > on the other devices. This means all files are listed (A lot of JSON-type > symbolic file-links that take up 1-2kb max), but all files are as JSON-links > until used the first time (just like as with google drive installed on > windows). > > This requires that the OwnCloud client monitors the file system (iNotify or > similar), and uses statistics of file-usage PER DEVICE PER USER to stay > up-to-date. > First it must assure that 1 copy of local files are up-to-date on the > OwnCloud Server whilst the OwnCloud must have sufficient capacity to "back > the local file up". > Second it must prioritize synchronization as a schedule, and not simply > perform a braindead mirror of N nodes (as it does currently). > > For practical purpose of such prioritization a vector that contains > exponential function of "time since last usage" x "frequency of usage" would > suffice. > > The point here that the user NEVER runs out of space on a single device, > because the files which are not used after a certain amount of time are > release for overwrite the moment the space is needed for other - arguably > more relevant files - or they are simply not downloaded to the local device > (because they exist locally as pointer with JSON link). > > Please notice that overwritten in this context means that the > "111010101"-blob on your LOCAL device is substituted with a JSON link to the > copy on OwnCloud (if this is more feasible utilisation of the limited space > on the client). > The local pointer to the file X is only change from the pointer named X that > refers to iNode so-and-so-much to a pointer named X that refers to JSON that > points to OwnCloud. > Hereby the local file X is released to be deleted/overwritten (depending on > the HW-driver setting) whilst UI the content underneath the UI simply changes > to point at the OwnCloud. Obviously this overwrite is only permitted IFF the > file exists on the OwnCloud (i.e. has been uploaded). > > So the trick in summary: Files which occupy a higher frequency of usage or > generally just are new on the LOCAL devices, everything that does not fit in > the space available locally exists on the OwnCloud, but is visible on the > local device as a JSON-link that from the UI is visually identical the real > file. > > For Klaas it means that iNode will change a lot (sorry) and that sha-256 > should be packed into the JSON link or the sqlite db. I know Klass is working > on that already. > Also the client could read the sqlite database for updates instead of > visiting the filesystem. To check if two databases are up-to-date a sha-256 > of the database itself may be exchanged which drops the required > communication (client-server) significantly at the first connection. The > knowledge alone can permit a less power-consuming startup of connectivity in > stead of the current where bandwidth is consumed naively. However these > details are just the top of the iceberg. But I think OwnCloud client is the > best match "out-there" for taking a step in that direction. > > A wider intellectual challenge is to predict behaviour across devices which > are idle are predict which files are to be used so that local devices are > up-to-date using low-power connection BEFORE you pick them up and use them > but I'm still working on that :-) > > Kind regards, > Bjorn > > > > > On 16 May 2013 07:52, Jan-Christoph Borchardt <[email protected]> wrote: > As a Chromebook owner who put Ubuntu on it: The main concern or solution here > seems to be to make ownCloud cope better with being offline. Use more client > (JS) and less server (PHP) logic, use local browser databases for data > caching (localStorage, indexedDB) and cache the apps themselves (AppCache). > > That way there's also less dependence on the server and better performance > because not everything has to be loaded again so often. > > Of course this involves major code forward, but the Appframework and newer > apps already are on the path to this. Also solves one of my pet peeves: the > files app reloads every time you click a folder … > > This would also make ownCloud better compatible with Firefox OS. Would be > quite strange to make "thin client apps" for the "fat web frontend" instead > of just improving it for everyone. > _______________________________________________ > Owncloud mailing list > [email protected] > https://mail.kde.org/mailman/listinfo/owncloud > > > > -- > Bjorn Madsen > Researcher Complex Systems Research > Ph.: (+44) 0 7792 030 720 > [email protected] > > _______________________________________________ > Owncloud mailing list > [email protected] > https://mail.kde.org/mailman/listinfo/owncloud _______________________________________________ Owncloud mailing list [email protected] https://mail.kde.org/mailman/listinfo/owncloud
