Re: [U2] Really trying to understand dynamic file sizing

2012-06-29 Thread Dave Laansma
Products, Services and Innovative Solutions" -Original Message- From: u2-users-boun...@listserver.u2ug.org [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Doug Averch Sent: Friday, June 29, 2012 4:17 PM To: U2 Users List Subject: Re: [U2] Really trying to understand dynamic

Re: [U2] Really trying to understand dynamic file sizing

2012-06-29 Thread Doug Averch
Hi Dave and others: Having worked for Wally many years ago at Unidata, his knowledge is unparalleled about Unidata. So given that I've been doing this for a number of years, I'm not sure where the 100 number is coming. The other numbers I have reams of paper, CD's, and thumb drives backing up al

Re: [U2] Really trying to understand dynamic file sizing

2012-06-29 Thread Dave Laansma
2012 3:06 PM To: U2 Users List Subject: Re: [U2] Really trying to understand dynamic file sizing I wouldn't go below 10%. You could end up with lots of splitting and very sparsely populated groups. Wally Terhune Technical Support Engineer Rocket Software 4600 South Ulster Street, Suite 1100 **D

Re: [U2] Really trying to understand dynamic file sizing

2012-06-29 Thread Wally Terhune
e.com/u2 -Original Message- From: u2-users-boun...@listserver.u2ug.org [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Doug Averch Sent: Friday, June 29, 2012 11:38 AM To: U2 Users List Subject: Re: [U2] Really trying to understand dynamic file sizing Hi Dave: You cannot ge

Re: [U2] Really trying to understand dynamic file sizing

2012-06-29 Thread Dave Laansma
org [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Doug Averch Sent: Friday, June 29, 2012 1:38 PM To: U2 Users List Subject: Re: [U2] Really trying to understand dynamic file sizing Hi Dave: You cannot get any lower than one percent. If you set your block size to 2K that will fix the spl

Re: [U2] Really trying to understand dynamic file sizing

2012-06-29 Thread Doug Averch
Hi Dave: You cannot get any lower than one percent. If you set your block size to 2K that will fix the split problem making about 6%. Your average record size is 1537 which means you will get about one record per block so split by key will be worthless and you will be into Level 2 and/or Level 1

Re: [U2] Really trying to understand dynamic file sizing

2012-06-29 Thread Mecki Foerthmann
org [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Doug Averch Sent: Thursday, June 28, 2012 8:56 PM To: U2 Users List Subject: Re: [U2] Really trying to understand dynamic file sizing On Thu, Jun 28, 2012 at 6:50 PM, Doug Averch wrote: 126/4064 (Correction, too late to be working on this) 1

Re: [U2] Really trying to understand dynamic file sizing

2012-06-29 Thread Dave Laansma
: u2-users-boun...@listserver.u2ug.org [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Doug Averch Sent: Thursday, June 28, 2012 8:56 PM To: U2 Users List Subject: Re: [U2] Really trying to understand dynamic file sizing On Thu, Jun 28, 2012 at 6:50 PM, Doug Averch wrote: > 126/4064 (Correcti

Re: [U2] Really trying to understand dynamic file sizing

2012-06-28 Thread Doug Averch
On Thu, Jun 28, 2012 at 6:50 PM, Doug Averch wrote: > 126/4064 (Correction, too late to be working on this) > 1 (Correction average Load before split) And that matches your numbers Regards, Doug www.u2logic.com ___ U2-Users mai

Re: [U2] Really trying to understand dynamic file sizing

2012-06-28 Thread Doug Averch
Here is the math from how I read the manual on KEYONLY Split/Merge. 4096 (Group Size from Guide) -32 (Unidata Overhead from documentation) 4064 (Usable Space) 6(Keys per group from Guide) 13 (Key size from Guide) = 78 (Key size on average) 6(Keys per Group from Guide) 8

Re: [U2] Really trying to understand dynamic file sizing

2012-06-28 Thread Baakkonen, Rodney A (Rod) 46K
idx011 -rw-rw-rw- 1 udtcron mcc 1765761024 Jun 26 15:17 idx012 -rwxrwxr-x 1 root mcc 954723328 Jun 28 09:02 over001 I haven't had to do anything to this file for a couple of years. -Original Message- From: u2-users-boun...@listserver.u2ug.org [mailto:u2-users-bo

[U2] Really trying to understand dynamic file sizing

2012-06-28 Thread Dave Laansma
I've only got a handful of dynamic files but of course they're huge and have a big impact on our daily and monthly processing. I'd REALLY like to understand the tuning mechanisms for these files, specifically SPLIT/MERGE. The formulas that I got on previous responses just don't seem to make sen