Well, I thought I would respond back to this so people don't waste their time.
I'd recommend not using s3ql on top of acd_cli, it's too unstable. I found myself with a crashed file system and having to run fsck everyday. My reasons for wanting to use s3ql were: 1. The chunked storage format I thought would be better for doing things like resuming movies in the middle on my plex server 2. The encryption it provides makes me feel like a sneaky secrete agent. Instead, I recommend you just upload your movies normally and mount using acd_cli. For #1, chunking the files doesn't seem necessary. I don't know what black magic the acd_cli mount is doing, but my movies resume from the middle just fine, and I'd actually say that they play a little faster, s3ql seems to add some overhead. I do miss out on the encryption, but I really don't think amazon cares about what I upload. The other thing you need to consider with encryption and s3ql in general is that you're locking yourself in to a particular technology. Right now I'm running plex on linux, but if I ever decide to buy a windows license so I can watch netflix on it, I'd have to re-download all my movies then re-upload them in some new format. Finally, if encryption is really important to you, there's a pull request for supporting encryptfs with acd_cli that seems a lot more mature then the pull request here for supporting s3ql. Some more details on what I ended up doing: 1. I am still using union-fs. I keep everything on the local disk until I need to make some room, then I archive stuff to acd. It's sorta like a poor mans version of the disk cache s3ql provides. 2. rclone is amazing. Instead of using acd_cli to upload which I find to be a little slow, I use rclone. I actually got banned for a little bit from my vps until I throttled it because I was uploading at 50-80 MBps (capital B is intentional!), which was most of their 1Gbps pipe. Now I upload at 20MBps consistently. Rclone also seems a little bit more stable then acd_cli, although their mount stuff is not currently as good. Anyways, the caching + encryption stuff that s3ql does is cool and I might revisit if it ever gets official support. It would be nice if it did because then I could remove all my unionfs and acd_cli stuff and just use s3ql. But what I have right now is good enough for my needs. I've had 1 file system crash in the last 2 weeks, and recovery from that is a lot faster since I don't have to run fsck. I know a lot of people are trying to get this combo working, so hopefully my post helps. On Wednesday, October 26, 2016 at 3:04:25 PM UTC-4, Nikolaus Rath wrote: > > On Oct 26 2016, Jonas Lippuner <[email protected] <javascript:>> wrote: > > I'm using a similar setup. Mount ACD locally with acd_cli and then use > an > > s3ql volume inside the mounted ACD path. The reason for using ACD rather > > than S3 is very simple: cost. > > Well, that explains it. You get what you pay for :-). > > [...] > > Here's the backtrace (from mount.log) of the error I usually see: > [...] > > > > "/usr/local/lib/python3.5/dist-packages/s3ql-2.20-py3.5-linux-x86_64.egg/s3ql/backends/local.py", > > > > line 304, in __init__ > > self.fh = open(name, 'wb', buffering=0) > > OSError: [Errno 14] Bad address: > > > '/home/jlippuner/cloud_storage/ACD/remote/s3ql_backup/s3ql_data_/698/s3ql_data_69888#7693-140501390976768.tmp' > > > > Yes, that's ACD being buggy. > > > I wonder how hard it would be to add a native ACD interface to S3QL... > > Some people are trying, see https://github.com/s3ql/s3ql/pull/7 > > > Best, > -Nikolaus > -- > GPG encrypted emails preferred. Key id: 0xD113FCAC3C4E599F > Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F > > »Time flies like an arrow, fruit flies like a Banana.« > -- You received this message because you are subscribed to the Google Groups "s3ql" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
