Thanks JayJay, Wol, Jerry & Brian for enlightening. The UV docs on distributed files are extremely sparse, so the peer experiences are essential to a good strategy.
JJ's comment about adding archive part files whose key does not fit the algorithm already in place is what I ran into. -Baker -----Original Message----- From: [email protected] [mailto:[email protected]] On Behalf Of John Jenkins Sent: Thursday, May 28, 2009 4:20 PM To: [email protected] Subject: RE: [U2] Best algorithm for UV part files It's really useful for archiving if the key contains a date - build the algorithm on the date in the key and have a separate partfile for each year (or quarter or month - you get the idea), When you want to archive old data you just drop the partfile concerned - remembering to add new partfiles for new dates as you go forward (this bit can be automated), ALWAYS remember to have an "ELSE" or "CASE 1" on your algorithm just in case, and monitor the specified partfile to fix whatever problem caused data to be placed in there (incorrectly - or maybe you just forgot to create a new partfile soon enough). Caution - I have seen some grief where partfiles are added that already contain data - and data that does not hash to this partfile under the distribution algorithm rules. OK - it doesn't happen often - but it can cause grief and confusion - just be careful. Which algorithm to use ? sorry - answer is "it depends"..... 1/ do you want even distribution - ? 2/ do you want context distribution (such as a date or a zip code)? 3/ How unique is your key? 4/ what are your data volumes? It all depends......... Regards JayJay ------- u2-users mailing list [email protected] To unsubscribe please visit http://listserver.u2ug.org/ This communication, its contents and any file attachments transmitted with it are intended solely for the addressee(s) and may contain confidential proprietary information. Access by any other party without the express written permission of the sender is STRICTLY PROHIBITED. If you have received this communication in error you may not copy, distribute or use the contents, attachments or information in any way. Please destroy it and contact the sender. ------- u2-users mailing list [email protected] To unsubscribe please visit http://listserver.u2ug.org/
