On Oct 31 2015, Cameron Boehmer <[email protected]> wrote:
> Hey guys,
>
> First, this project looks incredible! Thanks to all the contributors :)
> Second, after a brief encounter with the FAQ, I don't think s3ql does what
> I want, but it's *so close *that I want to lay out my goals—I can't be the
> first seeker on this path...
Let's see what we can do. I think your main problem is that your goals
are a little ill-defined:
> My goal: the holy grail of personal storage.
>
> - local file system is effectively infinite (backed by s3, etc)
That is already a contradiction. Either the file system is *local*, then
it's finite and *not* backed by a remote service, or it is *remote*.
> - reading a file stored on a remote fs downloads that file to the local
> filesystem *at the same path*
That is impossible by definition. On any path there can be only one file
system mounted.
> - another way to say this is that the local filesystem acts a bit
> like an LRU cache for remote paths
..and this is mostly what S3Ql does, only the cache is not in the same
location (because that's impossible by definition), and the cache is
flushed on umount.
> s3ql looks perfect *except* for the last point. The FAQ says that the
> closest it comes to this is using rsync to copy local data into an s3ql
> mount, but that won't let you see remote files next to local ones, let
> alone make them local transparently.
I'm going to take a wild guess here and describe in my words what I
think you want:
* You want two file systems, one locally and one remote
* You want to have an union file system (like unionfs), that "merges"
the two file systems, and where files from the local fs take
precedence.
* You want writes to the union to be propagated to both the local and
remote fs
* Occasionally, you want to delete files from the local fs that have
not been accessed for a while
Is that correct? In that case (as the description already implies), I
would take a look at the various union/overlay file systems to see if
any of them already does what you want. The only thing that might be
missing is the ability to write to both backing file systems. You could
either hack that yourself, or periodically run rsync from the local
backing filesystem to the remote backing filesystem. For the expiration,
a cron-job would do.
Best,
-Nikolaus
--
GPG encrypted emails preferred. Key id: 0xD113FCAC3C4E599F
Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F
»Time flies like an arrow, fruit flies like a Banana.«
--
You received this message because you are subscribed to the Google Groups
"s3ql" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
For more options, visit https://groups.google.com/d/optout.
