On Tuesday, August 11, 2015 at 1:42:15 PM UTC-4, Nikolaus Rath wrote: > > On Aug 11 2015, Keebs <[email protected] <javascript:>> wrote: > > I have a backup saving to a samba share backed by S3QL. The file "saves" > > successfully to the share and S3QL starts the upload to S3 as normal. > After > > the save to the share completes a verify is run on the file, but this is > > where it fails. > > What exactly fails? >
I wish I could answer this in more detail. Backup Exec reports Error V-79-57344-33992 which correlates to: *Storage device has reported an error on a request to read data from media. Error reported: The request could not be performed because of an I/O device error.* The subcode for the failure is: *Error: e00084c8 - The backup storage device has failed* > > I understand the eventual consistency issues, so I enlarged > > the s3ql cache to exceed the size of the file being saved thinking > > that > [...] > > No, quite obviously you don't understand the eventual consistency issues > :-). As long as the file system is mounted, they can not cause you any > problems. > > The way I understand it: Let's say I have an S3QL cache size of 2GB. Now I write a 4GB file. The first 2GB will copy quickly and fill the cache while the latter 2GB of the file will slowly save as cache is freed. Once the file has finished copying locally, the S3QL cache will still be full with the final 2GB of the file as it continues to upload to S3. Immediately after the backup thinks the file has been saved, it wants to verify the file so it is now requesting the file to be read back from the beginning, but the beginning of the file is no longer in the S3QL cache because the final 2gb of the file has the S3QL cache consumed until it frees some space to re-download the beginning of the file from S3 - which is where the eventual consistency could theoretically come into play. Until the time cache has been freed, the beginning of the file will not be available to be read. What am I misunderstanding? This is why I increased the S3QL cache well above the 4GB. In my test, I am copying a 4.1GB file and the verify is failing after reading less than 100meg according to BackupExec. > > > Any idea why the immediate verify after write is failing? > > It works for me: > > $ cd $s3ql_mountpoint > $ cp /usr/bin/emacs . > $ cmp /usr/bin/emacs emacs > $ > > Obviously you're doing something different, so you'll need to describe > (exactly!) what you are doing The best explanation I can find from Symantec is: *When the backup is done, some checksums are calculated from the data and stored in the backup set. Verify reads back the data from the backup set, recalculate the checksums and compass them with the stored checksums. If they matches, verify passes.* If I can get Symantec to explain more about exactly what their "verify" operation does, I will be happy to relay more information. I'm not sure what else I can provide at the moment. -- You received this message because you are subscribed to the Google Groups "s3ql" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
