I'm just curious then. Is anyone actually using camlistore to do production backups? And if so, does anyone have some scripts to share? The current raw low-level interface seems to be roughly equivalent to that of "git plumbing" --- e.g., harking back to the the early days of git when Linus was writing shell scripts that used "git write-tree", "git ls-tree", "git commit-tree", "git update-ref", etc. when he was first boot-strapping git. The thing is, git's "porcelain" came very quickly, and camilstore has been around for while (people have been asking questions about workflow for doing backups going back to 2014, according to searches on this mailing list). So before I go and re-invent the wheel --- does anyone have any scripts they could share about how they are actually using camilstore to do their own backups?
I just find it hard to believe people are just using things like "camput file -permanode -title="home-backup-2016-10-1" -tag backups $HOME" and just calling it a day? Or is that really all people are doing? Thanks, -- Ted On Saturday, October 1, 2016 at 1:48:57 PM UTC-4, mpl wrote: > > Hi, > > camtool is indeed not the most user-friendly of our tools. When I > forget how it works, (as I just had now), I just "reverse-engineer" > it. That's to say, if you create (mkdir, cp) new nodes in roots, > you'll see (with some devcam get, devcam tool describe, or in the web > UI) that the relation created is a camliPath. Same goes for the > children of these nodes. So: > > devcam put permanode > -> sha1-500520ae4269b7ac7b20f0a42ba57650450ddb02 > devcam put attr camliRoot > sha1-500520ae4269b7ac7b20f0a42ba57650450ddb02 someRootName > devcam put permanode > -> sha1-c8db36279ee03f5daf69a3d2885ef5d8c0111f05 > devcam put attr sha1-500520ae4269b7ac7b20f0a42ba57650450ddb02 > camliPath:TODO.txt sha1-c8db36279ee03f5daf69a3d2885ef5d8c0111f05 > devcam put file /home/mpl/docs/TODO.txt > -> sha1-4c8b980f2c5faa49e687c8f1235ed17b2818691c > devcam put attr sha1-c8db36279ee03f5daf69a3d2885ef5d8c0111f05 > camliContent sha1-4c8b980f2c5faa49e687c8f1235ed17b2818691c > > > ll /tmp/cammount-dir/roots/someRootName/TODO.txt > -rw------- 1 mpl mpl 1654 Oct 1 19:14 > /tmp/cammount-dir/roots/someRootName/TODO.txt > > I'd have to recheck, but I don't think the navigation from the roots > are following other relations than camliPath and camliContent, so > they're not following the static-set relation that you get betwen a > dir and its children when you camput a directory. And I don't know > what's the roadmap on that. > > As (I think) you have found out, you can still browse a directory if > you know its hash, as in: > > devcam put file /home/mpl/docs > -> sha1-339cf73c02ea5981b71f3aeaaf8864dbfbedafa5 > cd /tmp/cammount-dir/sha1-339cf73c02ea5981b71f3aeaaf8864dbfbedafa5 > > hth, > Mathieu > > > On 1 October 2016 at 18:52, Theodore Ts'o <[email protected] > <javascript:>> wrote: > > I'm really confused about how to take a backup and make it available in > > Roots so I can browse it using the cammount. I don't know if this > releated > > to the fact that I'm using ./bin/devcam, but the documentation on how > Roots > > and work is very sparse. :-( > > > > OK, so starting with "./bin/devcam server --wipe" in one window, I then > do a > > backup: > > > > % ./bin/devcam put file /tmp/test-dir > > > > So I get back a sha1-sig which looks like this: > > > > > > % ./bin/devcam tool describe > sha1-0609bc86627d0fa3067d45c09ede479c7e8643f8 > > { > > "meta": { > > "sha1-01b8821894b8de827984b1b9f59671bf0cffc036": { > > "blobRef": "sha1-01b8821894b8de827984b1b9f59671bf0cffc036", > > "camliType": "directory", > > "size": 306, > > "dir": { > > "fileName": "subdir", > > "size": 1, > > "wholeRef": null > > }, > > "dirChildren": [ > > "sha1-9999264baa2c936c30b509be0f0f77c1e39615f6" > > ] > > }, > > "sha1-0609bc86627d0fa3067d45c09ede479c7e8643f8": { > > "blobRef": "sha1-0609bc86627d0fa3067d45c09ede479c7e8643f8", > > "camliType": "directory", > > "size": 309, > > "dir": { > > "fileName": "test-dir", > > "size": 3, > > "wholeRef": null > > }, > > "dirChildren": [ > > "sha1-01b8821894b8de827984b1b9f59671bf0cffc036", > > "sha1-7ecba772756f1b4fbe0f3179cc5ad02e087aad75", > > "sha1-8256107f4be52db012e4e80d2e43a71b542769d7" > > ] > > }, > > "sha1-7ecba772756f1b4fbe0f3179cc5ad02e087aad75": { > > "blobRef": "sha1-7ecba772756f1b4fbe0f3179cc5ad02e087aad75", > > "camliType": "file", > > "size": 352, > > "file": { > > "fileName": "issue", > > "size": 36, > > "time": "2016-10-01T14:21:26.091166128Z", > > "wholeRef": "sha1-309e70c09b04eb5e144b29b21e5ecec2d3289628" > > } > > }, > > "sha1-8256107f4be52db012e4e80d2e43a71b542769d7": { > > "blobRef": "sha1-8256107f4be52db012e4e80d2e43a71b542769d7", > > "camliType": "file", > > "size": 352, > > "file": { > > "fileName": "motd", > > "size": 286, > > "time": "2016-10-01T14:21:29.771162244Z", > > "wholeRef": "sha1-8b55aac644e9e6f2701805584cc391ff81d3ecec" > > } > > }, > > "sha1-9999264baa2c936c30b509be0f0f77c1e39615f6": { > > "blobRef": "sha1-9999264baa2c936c30b509be0f0f77c1e39615f6", > > "camliType": "file", > > "size": 350, > > "file": { > > "fileName": "date", > > "size": 29, > > "time": "2016-10-01T15:09:46.36563753Z", > > "wholeRef": "sha1-fc866bc41a833bed829bf1580ce11113580fd4b0" > > } > > } > > } > > } > > > > so the first weird thing is that the docs imply that this should work: > > > > % ./bin/devcam mount sha1-0609bc86627d0fa3067d45c09ede479c7e8643f8 > > Cammount running with mountpoint /tmp/cammount-dir. Press 'q' <enter> or > > ctrl-c to shut down. > > 2016/10/01 12:32:19 root specified is not a blobref or name of a root: > > "/tmp/cammount-dir" > > > > Hmm... so it claims it wants the "name of a root". how about the > > pre-created dev-pics-root? > > > > % ./bin/devcam mount /dev-pics-root > > Cammount running with mountpoint /tmp/cammount-dir. Press 'q' <enter> or > > ctrl-c to shut down. > > 2016/10/01 12:33:45 root specified is not a blobref or name of a root: > > "/tmp/cammount-dir" > > > > Nope. OK, so I'll just use "./bin/devcam mount" and get the full > default > > top-level: > > > > % ls /tmp/cammount-dir > > total 0 > > 0 at/ 0 recent/ 0 sha1-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx/ 0 > > WELCOME.txt > > 0 date/ 0 roots/ 0 tag/ > > % ls /tmp/cammount-dir/roots > > total 0 > > 0 dev-pics-root/ > > > > Let's create a new root: > > > > % ./bin/devcam put permanode 2> /dev/null > > sha1-19d8e08e5660275357b072edae6741b14db0c882 > > % ./bin/devcam put attr sha1-19d8e08e5660275357b072edae6741b14db0c882 > title > > testroot 2> /dev/null > > sha1-608d3485eb205e41e3db9ba2c7b498eff1797f3f > > % ./bin/devcam put attr sha1-19d8e08e5660275357b072edae6741b14db0c882 > > camliRoot testroot 2> /dev/null > > sha1-10b885cd8630331db50648cc10242cb6e0edadb1 > > > > % ls /tmp/cammount-dir/roots/ > > total 0 > > 0 dev-pics-root/ 0 testroot/ > > > > Yay! It works. Now I try connecting the backup I made above > > (sha1-0609bc86627d0fa3067d45c09ede479c7e8643f8) with the testroot.... > and > > everything fails. My best guess, which I fully expected to work was: > > > > % ./bin/devcam put attr sha1-19d8e08e5660275357b072edae6741b14db0c882 > > camliPath:testdir sha1-0609bc86627d0fa3067d45c09ede479c7e8643f8 > > sha1-aaf244b34b412331789caa86eaf09443468a22ee > > > > But alas, no dice: > > > > % ./bin/devcam mount > > Cammount running with mountpoint /tmp/cammount-dir. Press 'q' <enter> or > > ctrl-c to shut down. > > > > <in another window> > > % ls /tmp/cammount-dir/roots/testroot/ > > total 0 > > > > Argh, so let's try creating via fuse: > > > > % mkdir -p /tmp/cammount-dir/roots/testroot/foo/bar/baz > > > > <back to the original window after shutting down cammount> > > > > % ./bin/devcam tool describe > sha1-19d8e08e5660275357b072edae6741b14db0c882{ > > "meta": { > > "sha1-19d8e08e5660275357b072edae6741b14db0c882": { > > "blobRef": "sha1-19d8e08e5660275357b072edae6741b14db0c882", > > "camliType": "permanode", > > "size": 562, > > "permanode": { > > "attr": { > > "camliPath:foo": [ > > "sha1-fc4e1bf59ba29d2e13f73d685c09d2cd26674bdc" > > ], > > "camliPath:testdir": [ > > "sha1-0609bc86627d0fa3067d45c09ede479c7e8643f8" > > ], > > "camliRoot": [ > > "testroot" > > ], > > "title": [ > > "testroot" > > ] > > }, > > "modtime": "2016-10-01T16:44:00.886392008Z" > > } > > } > > } > > } > > > > Hmm, let's look at what the references for "foo" and "testdir" look > like: > > > > % ./bin/devcam tool describe > sha1-fc4e1bf59ba29d2e13f73d685c09d2cd26674bdc > > { > > "meta": { > > "sha1-fc4e1bf59ba29d2e13f73d685c09d2cd26674bdc": { > > "blobRef": "sha1-fc4e1bf59ba29d2e13f73d685c09d2cd26674bdc", > > "camliType": "permanode", > > "size": 562, > > "permanode": { > > "attr": { > > "camliNodeType": [ > > "directory" > > ], > > "camliPath:bar": [ > > "sha1-b0dc6213dfbc7b004304fe46ebd667a9529dd928" > > ], > > "title": [ > > "foo" > > ] > > }, > > "modtime": "2016-10-01T16:44:00.905961518Z" > > } > > } > > } > > } > > > > and the description of "testdir" is above. > > > > And the first thing I notice is they look totally difference. For the > > subdir created via fuse, it uses camliPath to name the subdirectories. > But > > in testdir, the directory contents are described very differently: > > > > "sha1-0609bc86627d0fa3067d45c09ede479c7e8643f8": { > > "blobRef": "sha1-0609bc86627d0fa3067d45c09ede479c7e8643f8", > > "camliType": "directory", > > "size": 309, > > "dir": { > > "fileName": "test-dir", > > "size": 3, > > "wholeRef": null > > }, > > "dirChildren": [ > > "sha1-01b8821894b8de827984b1b9f59671bf0cffc036", > > "sha1-7ecba772756f1b4fbe0f3179cc5ad02e087aad75", > > "sha1-8256107f4be52db012e4e80d2e43a71b542769d7" > > ] > > }, > > > > It doesn't have the camliPath attributes at all! > > > > Yet cammount can understand the other scheme, so long as I remember the > sha1 > > hash: > > > > % ls /tmp/cammount-dir/sha1-0609bc86627d0fa3067d45c09ede479c7e8643f8 > > total 0 > > 0 issue 0 motd 0 subdir/ > > > > But of course I don't want to get into the business of memorizing sha1 > > hashes. And if I create a permanode, e.g: > > > > % ./bin/devcam put file --permanode /tmp/test-dir > > > > The permanode doesn't show up in FUSE hierarchy exported by cammount > > (although it does show up in the web ui). > > > > Argh!!!! Very frustrating. > > > > What am I missing? > > > > Thanks, > > > > -- Ted > > > > > > > > -- > > You received this message because you are subscribed to the Google > Groups > > "Camlistore" group. > > To unsubscribe from this group and stop receiving emails from it, send > an > > email to [email protected] <javascript:>. > > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "Camlistore" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
