Not quite. A checkpoint forces a break in the data stream; on tape for will get a EOF mark written; the tape will stop; the media server has a bit of a chat with the Master Server, flushing out data about the files backed up in the file just closed, etc. That is how it knows if there is a failure and restart, what is still to back up.
If you are writing to a disk target you will again split the backup into another fragment I think. So if you say set a 32GB fragment size, the on-disk files are split every 32GB and also I believe when a checkpoint is hit, every 15 minutes or whatever you configure. As to effect on deduplication % I would not expect it to be huge, though it will force the software to start calculating hashes from the start of each new file. William D L Brown From: veritas-bu-boun...@mailman.eng.auburn.edu [mailto:veritas-bu-boun...@mailman.eng.auburn.edu] On Behalf Of scott.geo...@parker.com Sent: 08 September 2011 12:19 To: veritas-bu@mailman.eng.auburn.edu Subject: Re: [Veritas-bu] Dedupe ratios and checkpoint backups I think the question was: where is that file located when it is created, and does it then get backed up by the job doing the backup itself? From: David Stanaway <da...@stanaway.net> To: veritas-bu@mailman.eng.auburn.edu Date: 09/07/2011 09:11 PM Subject: Re: [Veritas-bu] Dedupe ratios and checkpoint backups Sent by: veritas-bu-boun...@mailman.eng.auburn.edu ________________________________ Checkpoints create a new file for each checkpoint. On 9/7/2011 1:17 PM, Rusty Major wrote: I have always understood it that Checkpoints were saved in a log on the client and, therefore, wouldn't affect dedupe ratios at all. I haven't ever verified that, nor did a quick search yield anything. -Rusty From: veritas-bu-boun...@mailman.eng.auburn.edu<mailto:veritas-bu-boun...@mailman.eng.auburn.edu> [mailto:veritas-bu-boun...@mailman.eng.auburn.edu<mailto:veritas-bu-boun...@mailman.eng.auburn.edu>] On Behalf Of Lightner, Jeff Sent: Tuesday, September 06, 2011 3:23 PM To: scott.geo...@parker.com<mailto:scott.geo...@parker.com>; veritas-bu@mailman.eng.auburn.edu<mailto:veritas-bu@mailman.eng.auburn.edu> Subject: Re: [Veritas-bu] Dedupe ratios and checkpoint backups Haven't done any comparisons but we use checkpoint for our big ERP DB (near 6 TB) backup and still get good compression ratios. There is nothing that has made me think I need to look at it or tweak it to get better. I'd say the benefit of not having to restart a huge backup from scratch offered by checkpoints would outweigh deduplication ratio issues unless you have infinite time to run backups. ________________________________ From: veritas-bu-boun...@mailman.eng.auburn.edu<mailto:veritas-bu-boun...@mailman.eng.auburn.edu> [mailto:veritas-bu-boun...@mailman.eng.auburn.edu<mailto:veritas-bu-boun...@mailman.eng.auburn.edu>] On Behalf Of scott.geo...@parker.com<mailto:scott.geo...@parker.com> Sent: Tuesday, September 06, 2011 3:06 PM To: veritas-bu@mailman.eng.auburn.edu<mailto:veritas-bu@mailman.eng.auburn.edu> Subject: Re: [Veritas-bu] Dedupe ratios and checkpoint backups To answer your question, we tested both the NBU 5000 (Symantec) and the DD860 (Data Domain) and the differences couldn't be more stark. I was using a 30 minute checkpoint interval. I never achieved anything better than 15:1 from the NBU5000 (which isn't bad). The DD860 hit 39:1 at the time I disabled all of the policies. This was running daily full backups on an array of different servers (DB2, Windows file servers, UNIX file servers, and Siebel app servers) over the course of 2 months. The NBU5000 is a fixed block device, the DD860 a variable block device. There is no way of knowing if checkpoints were the culprit for the NBU5000 getting the lesser ratio, but it does present a plausible theory. From: scott.geo...@parker.com<mailto:scott.geo...@parker.com> To: "veritas-bu@mailman.eng.auburn.edu<mailto:veritas-bu@mailman.eng.auburn.edu>" <veritas-bu@mailman.eng.auburn.edu<mailto:veritas-bu@mailman.eng.auburn.edu>> Date: 09/06/2011 02:54 PM Subject: Re: [Veritas-bu] Dedupe ratios and checkpoint backups Sent by: veritas-bu-boun...@mailman.eng.auburn.edu<mailto:veritas-bu-boun...@mailman.eng.auburn.edu> ________________________________ If at all, this would probably affect fixed-block solutions more than the variable-block ones. The variable-blocked solutions would continue to look for identical blocks, but in different positions of the data stream. From: "Stafford, Geoff" <gstaff...@barclaycardus.com<mailto:gstaff...@barclaycardus.com>> To: "veritas-bu@mailman.eng.auburn.edu<mailto:veritas-bu@mailman.eng.auburn.edu>" <veritas-bu@mailman.eng.auburn.edu<mailto:veritas-bu@mailman.eng.auburn.edu>> Date: 09/06/2011 02:45 PM Subject: [Veritas-bu] Dedupe ratios and checkpoint backups Sent by: veritas-bu-boun...@mailman.eng.auburn.edu<mailto:veritas-bu-boun...@mailman.eng.auburn.edu> ________________________________ Just wondering if anyone has done testing or seen documentation (from any dedupe vendor) regarding the usage of enabling checkpoints on backups that are being deduplicated? I would think that the introduction of checkpoints every X minutes into the datastream would interrupt the continuity of the data and make it seem more unique thus negatively affecting dedupe ratios but I'm wondering by how much. Most, if not all, of the variable length guys have the ability to 're-align' themselves to the start of the files so I would think it might be more pronounced on large files vs your average server but I'm just thinking out loud. Anyone seen a recommendation or actually tested themselves? _______________________________________________________ Barclays www.barclaycardus.com _______________________________________________________ This e-mail and any files transmitted with it may contain confidential and/or proprietary information. It is intended solely for the use of the individual or entity who is the intended recipient. Unauthorized use of this information is prohibited. If you have received this in error, please contact the sender by replying to this message and delete this material from any system it may be on. _______________________________________________ Veritas-bu maillist - Veritas-bu@mailman.eng.auburn.edu<mailto:Veritas-bu@mailman.eng.auburn.edu> http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu_______________________________________________ Veritas-bu maillist - Veritas-bu@mailman.eng.auburn.edu<mailto:Veritas-bu@mailman.eng.auburn.edu> http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu Proud partner. Susan G. Komen for the Cure. Please consider our environment before printing this e-mail or attachments. ---------------------------------- CONFIDENTIALITY NOTICE: This e-mail may contain privileged or confidential information and is for the sole use of the intended recipient(s). If you are not the intended recipient, any disclosure, copying, distribution, or use of the contents of this information is prohibited and may be unlawful. If you have received this electronic transmission in error, please reply immediately to the sender that you have received the message in error, and delete it. Thank you. ---------------------------------- _______________________________________________ Veritas-bu maillist - Veritas-bu@mailman.eng.auburn.edu<mailto:Veritas-bu@mailman.eng.auburn.edu> http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu _______________________________________________ Veritas-bu maillist - Veritas-bu@mailman.eng.auburn.edu http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu ________________________________ This e-mail was sent by GlaxoSmithKline Services Unlimited (registered in England and Wales No. 1047315), which is a member of the GlaxoSmithKline group of companies. The registered address of GlaxoSmithKline Services Unlimited is 980 Great West Road, Brentford, Middlesex TW8 9GS.
_______________________________________________ Veritas-bu maillist - Veritas-bu@mailman.eng.auburn.edu http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu
