> On Jul 2, 2016, at 9:01 AM, Nikolaus Rath <[email protected]> wrote: > > On Jul 01 2016, Peter Auyeung <[email protected]> wrote: >>> On Jul 1, 2016, at 1:43 PM, Nikolaus Rath <[email protected]> wrote: >>> >>> On Jul 01 2016, Peter Auyeung <[email protected]> wrote: >>>>>>> On Jun 29 2016, Peter Auyeung <[email protected]> wrote: >>>>>>> On Wednesday, June 29, 2016 at 11:53:06 AM UTC-7, Nikolaus Rath wrote: >>>>>>> On Jun 29 2016, Peter Auyeung <[email protected] <javascript:>> >>>>>>> wrote: >>>>>>>> Is there a way to backup and restore metadata of s3ql? >>>>>>> >>>>>>> You could copy/restore the s3ql_metadata object in your storage >>>>>>> backend, >>>>>>> or the *.db file in your --cachedir. >>>>>> >>>>>> I did a copy and restore of the backend of s3ql on local storage and >>>>>> getting the following error: >>>>>> >>>>>> # mount.s3ql local:///ntap4/restore/ /s3ql/restore/ >>>>>> Using 10 upload threads. >>>>>> Autodetected 4034 file descriptors available for cache entries >>>>>> Enter file system encryption passphrase: >>>>>> ERROR: Uncaught top-level exception: >>>>>> Traceback (most recent call last): >>>>>> File "/usr/bin/mount.s3ql", line 9, in <module> >>>>>> load_entry_point('s3ql==2.18', 'console_scripts', 'mount.s3ql')() >>>>>> File "/usr/lib/s3ql/s3ql/mount.py", line 129, in main >>>>>> (param, db) = get_metadata(backend, cachepath) >>>>>> File "/usr/lib/s3ql/s3ql/mount.py", line 374, in get_metadata >>>>>> param = backend.lookup('s3ql_metadata') >>>>>> File "/usr/lib/s3ql/s3ql/backends/comprenc.py", line 77, in lookup >>>>>> return self._verify_meta(key, meta_raw)[1] >>>>>> File "/usr/lib/s3ql/s3ql/backends/comprenc.py", line 137, in _verify_meta >>>>>> raise CorruptedObjectError('HMAC mismatch') >>>>>> s3ql.backends.common.CorruptedObjectError: HMAC mismatch >>>>> >>>>> >>>>> Well.. you did something wrong. Maybe you copied one of the >>>>> s3ql_metadata_bak objects to s3ql_metadata? This will give you a >>>>> checksum error. >>>>> >>>>> That said, I think you should explain what problem you are trying to >>>>> solve. I don't think what you're trying to do is the right solution. >>>> I am trying to backup a s3ql to google clould either on s3ql or not >>> >>> This doesn't make sense. If you're trying to backup to Google cloud, >>> what are you trying to do above? Clearly you're trying to mount >>> something from local storage. >> That would be s3ql on local storage >> Trying to backup locally to s3ql and snapshots and replicate to google > > So why are you talking about metadata then? You should be replicating > the entire folder that you pass to the local backend. > > Also, if you correctly backup and restore then by definition S3QL can't > even tell that anything happened. If after the restore S3QL doesn't work > anymore, then that's not a problem with S3QL but with your > backup/restore procedure. > You are correct, i was messed up with the db and param file. The restore is now working.
>>>> I was trying to rsync s3ql locally to the one on google storage >>> >>> You can do that, but then you won't be able to mount the file system >>> using the Google Storage backend (the backends use different file >>> formats). contrib/clone_fs.py can do the conversion though. >> Does clone_fs.py incrementally sync two s3ql? > > No. > >> I am trying to reduce the cloud put get traffic during incremental > > Then you should be working with two mounted S3QL file systems (one > using the local backend and one using the Google Storage backend) and do > the synchronization using e.g. rsync. > I was using rsync. As I asked in a different thread. Due to the amount of small files in s3ql. The rsync generated lots of put get traffic during incremental. > > Best, > -Nikolaus > > -- > GPG encrypted emails preferred. Key id: 0xD113FCAC3C4E599F > Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F > > »Time flies like an arrow, fruit flies like a Banana.« > > -- > You received this message because you are subscribed to a topic in the Google > Groups "s3ql" group. > To unsubscribe from this topic, visit > https://groups.google.com/d/topic/s3ql/DBmekkt-Syk/unsubscribe. > To unsubscribe from this group and all its topics, send an email to > [email protected]. > For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups "s3ql" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
