On Sat, May 7, 2016 at 7:05 PM, Alok Parlikar <[email protected]> wrote:
> Probably an FAQ, and I did browse the group archives, but might still need > a little handholding in setting up my workflow. > > I have two linux laptops, an android phone, a RaspberryPi and a host with > linode. The linode box currently has owncloud installed. My backup workflow > currently just has duplicity with external hdds. > > I'd like to switch to camlistore -- looking at all the docs and videos, it > was easy to make up my mind. But I'm stuck as to how to actually set up my > workflow. > > (1) Which machine(s) should run camlistored? One of my laptops? The > linode? Or should I switch to GCE/AWS instead of linode and use cloud > storage for backup? > I'd recommend running a primary instance on an always-connected, always-on host. I have in the past run a secondary instance on my laptop for local access, but there is not yet support for syncing subsets of your data (for example your "Documents" folder) from a main instance to a satellite instance, so for now I'd recommend just accessing your store as a client from your laptop. > (2) Can the raspberry pi also be a local cache of my store on my network? I had an instance on a Beaglebone Black (~comparable to a Pi v2). It was fine for serving content, but too slow to keep up with chunking uploaded data and indexing. The Pi3 may be fast enough, but I haven't tried it yet. If the node isn't taking uploads directly, just syncing from another instance, it would only have to deal with indexing. It might also be possible to replicate the index from your primary node if you use an index backend with replication (mongo etc) but I haven't tried this. > (3) Any tips for how my backup workflow should look like -- if I use > s3/GCS, should I still back up my laptop periodically on disks? If so, > what's the best way to back up the camlistore data? > Are you planning to take periodic snapshots of your homedir using "camput file ~"? I don't use this functionality, so I can't advise. My impression is it's quite reliable. The backup method for Camlistore data depends on the storage backend. The different cloud storage providers all have their own best practices for backups. There are three different local filesystem backends (localdisk: one file per blob; diskpacked: 4GB journal files; and blobpacked, which stores blobs loose initially and then repacks for efficient sequential reads). I think for all of them you'd be safest to shut down the service during a backup. Modern filesystems like ZFS and BTRFS support online snapshots which is safest of all. > (4) I think my $HOME should still be on disk, and not use the fuse mount; > in that case, how easy is it to keep it in sync with camlistore? > There are a couple of ways to map a directory structure into Camlistore. (a) camput file $PATH creates a static (deduplicated) snapshot of the path. It's great for backups, and is the foundation on which the other mode is built. You can restore with camget -o $OUTDIR $REF. (b) camput file -filenodes $PATH creates a mutable "permanode" for each file/directory under the path. This mode is appropriate when you plan to treat Camlistore as the source of truth going forward. Unlike (a), the command isn't idempotent; each run creates a new set of permanodes. You can also create permanodes by creating/copying directories under "roots/" in the FUSE mount. If you are talking about bidirectional sync between the local filesystem and permanodes, it's possible but requires some care. Always test first on a throwaway instance (devcam server -wipe is your friend). The main thing to be aware of: lots of user space programs spam temp files, and cammount doesn't filter them out (yet). Also, depending on how a sync program operates, it may remove the original file and create a new one. I'd like to document which sync software works, and flags are needed. As Mathieu said, you may want to wait. Help wanted <https://github.com/camlistore/camlistore/issues?q=is%3Aissue+is%3Aopen+fs+label%3AFilesystem> :) -- You received this message because you are subscribed to the Google Groups "Camlistore" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
