Just wanted to give you a heads up about this cool program I have been
playing with. It makes a virtual disk that is hosted on amazon s3.
http://code.google.com/p/s3backer/
I'll just paste their description:
"*s3backer* is a filesystem that contains a single file backed by the
Amazon Simple Storage Service <http://aws.amazon.com/s3> (Amazon S3). As
a filesystem, it is very simple: it provides a single normal file having
a fixed size. Underneath, the file is divided up into blocks, and the
content of each block is stored in a unique Amazon S3 object. In other
words, what *s3backer* provides is really more like an S3-backed virtual
hard disk device, rather than a filesystem.
In typical usage, a normal filesystem is mounted on top of the file
exported by the *s3backer* filesystem using a loopback mount (or disk
image mount on Mac OS X).
This arrangement has several benefits compared to more complete S3
filesystem implementations:
* By not attempting to implement a complete filesystem, which is a
complex undertaking and difficult to get right, *s3backer* can
stay very lightweight and simple. Only three HTTP operations are
used: GET, PUT, and DELETE. All of the experience and knowledge
about how to properly implement filesystems that already exists
can be reused.
* By utilizing existing filesystems, you get full UNIX filesystem
semantics. Subtle bugs or missing functionality relating to hard
links, extended attributes, POSIX locking, etc. are avoided.
* The gap between normal filesystem semantics and Amazon S3
``eventual consistency'' is more easily and simply solved when one
can interpret S3 objects as simple device blocks rather than
filesystem objects (see below).
* When storing your data on Amazon S3 servers, which are not under
your control, the ability to encrypt data becomes a critical
issue. *s3backer* supports secure encryption and authentication.
Alternately, the encryption capability built into the Linux
loopback device can be used.
* Since S3 data is accessed over the network, local caching is also
very important for performance reasons. Since *s3backer* presents
the equivalent of a virtual hard disk to the kernel, most of the
filesystem caching can be done where it should be: in the kernel,
via the kernel's page cache. However *s3backer* also includes its
own internal block cache for increased performance, using
asynchronous worker threads to take advantage of the parallelism
inherent in the network."
Basically you get a network hosted disk that is mounted locally through
loopback. It works great (I am hosting videos that are played straight
off the drive to the web), is relatively simple to set up and use, and
of course it is cloud-hosted so you can use it on more than one system.
The cost of S3 is very reasonable, it is a compelling solution for
backups, web storage, etc.
Jeremy
_______________________________________________
mlug mailing list
[email protected]
https://listes.koumbit.net/cgi-bin/mailman/listinfo/mlug-listserv.mlug.ca
