Re: [U2] static/dynamic file(s) opinions wanted!
Hi. Everybody talks about performance and maintenance. But, what about fragility? Years ago we move towards dynamic files. But many of our customer's servers didn't have UPS. So, a single storm or power supply failure could result in file corruption. Today we don't have dynamic files at all, except from those servers that are hosted in a very well conditioned environment. And yes, the resize of static files is absollutly necessary. We do it with a self made basic program. But we do it at night, because we had bad experiences with the CONCURRENT clause. By the way, could someone tell me from wich release of UniVerse the CONCURRENT clause is sure enough? Thanks, __ Augusto Alonso I.T. Manager Quiter Servicios Centrales Tel: +34 902 23 33 23 Fax: +34 902 23 42 80 [EMAIL PROTECTED] www.quiter.com __ - Original Message - From: Ray Wurlod [EMAIL PROTECTED] To: u2-users@listserver.u2ug.org Sent: Friday, July 25, 2008 7:51 AM Subject: Re: [U2] static/dynamic file(s) opinions wanted! UniVerse Dynamic files do NOT resize every time they split. They only resize a single group at each split. The next group to split is allocated on a round robin basis from group number 1 up to the group number that is the highest power of 2 less than or equal to the current modulus. So, for a large dynamic file, it can be a long interval between when a group is resized and the next time it is resized. This is why we advocate resizing Dynamic files periodically. Yes, they will work OK if you don't. - Original Message - From: JPB-U2UG [EMAIL PROTECTED] To: u2-users@listserver.u2ug.org Subject: Re: [U2] static/dynamic file(s) opinions wanted! Date: Thu, 24 Jul 2008 22:23:51 -0500 Tom, The point is that it doesn't work with dynamic files. David, I don't really think you need to resize dynamic files because they are always resizing themselves, unless they are not changing size. This type of resizing is equivalent to defragging on a windows system. Since dynamic files resize every time they split then it is not necessary. --- u2-users mailing list u2-users@listserver.u2ug.org To unsubscribe please visit http://listserver.u2ug.org/ --- u2-users mailing list u2-users@listserver.u2ug.org To unsubscribe please visit http://listserver.u2ug.org/
RE: [U2] static/dynamic file(s) opinions wanted!
Thanks Jeff, Or... You could call me by my nickname of LOL. Sincerely, AFAIK -- Original message -- From: Jeff Fitzgerald [EMAIL PROTECTED] Gee, with all that, shouldn't the [ad] flag be in uppercase, underlined and bolded? Perhaps HTH will download your stuff and post a review. I think you should address him formally as Mr. Helps, though. Later, when you are friends you could use his first name, Hope. ;-) Jeff Fitzgerald Fitzgerald Long, Inc. -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of daverch Sent: Thursday, July 24, 2008 1:10 PM To: u2-users@listserver.u2ug.org Subject: RE: [U2] static/dynamic file(s) opinions wanted! [ad] Hi HTH, U2logic has created a resize plug-in for Eclipse IDE that surpasses their functionality. We do not ...snip... [/ad] -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of [EMAIL PROTECTED] Sent: Wednesday, July 23, 2008 10:25 PM To: u2-users@listserver.u2ug.org Subject: RE: [U2] static/dynamic file(s) opinions wanted! Hi Doug, In addition to the great info from this group (like Dan provides), this site http://www.fitzlong.com/ provides great insight into U2 files. You should take some time to read the papers that they've provided. Also, they have an incredible product called fast that can really help you eliminate a lot of the maintenance headaches and increase your system performance. HTH --- u2-users mailing list u2-users@listserver.u2ug.org To unsubscribe please visit http://listserver.u2ug.org/ --- u2-users mailing list u2-users@listserver.u2ug.org To unsubscribe please visit http://listserver.u2ug.org/
RE: [U2] static/dynamic file(s) opinions wanted!
All I've missed the start of this thread, so apologies to all if this has been mentioned before. On a 24x7 site, dynamic files should be the most useful (because there is no time available for resizing static files) but AFAIR beause the current load etc are held in shared memory, this prevents you simply pausing the database momentarily whilst splitting a mirror and then backing up that mirror. Unless of course the dbpause updates any open dynamic files with the the content of the shared memory stats before it exits (which would be sensible)... does anyone know whether it does? Brian --- u2-users mailing list u2-users@listserver.u2ug.org To unsubscribe please visit http://listserver.u2ug.org/
RE: [U2] static/dynamic file(s) opinions wanted!
[ad] Hi HTH, U2logic has created a resize plug-in for Eclipse IDE that surpasses their functionality. We do not need any downloaded programs, directories or data files to run. We do not license the product by CPU or number of Universe or Unidata users. We license XLr8 Resizer by your individual workstation. We use standard Universe and Unidata utilities to resize you files. XLr8 Resizer is was built on current technology (Eclipse 3.3 or 3.4 and Java VM 1.6). If you want to stick with something that was developed many years ago, looks outdated, and is very pricey, then stay where you are. Otherwise, go with the leader in U2 tools based on Eclipse technology: U2logic. We are a green software product. 1) Software is only downloadable via our web site. 2) The software has not paper output, everything is displayed in Eclipse. 3) Our documentation is in PDF or Word format. 4) Payment for our software is Pay Pal which is paperless. We would recommend that you download our package and see the differences. Or at the very least, read our product information on our web site at www.u2logic.com Regards, Doug BTW: Our other plug-in's for the U2 environment are an editor, a dictionary editor, an installer, an object editor, and a Web Developer. [/ad] -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of [EMAIL PROTECTED] Sent: Wednesday, July 23, 2008 10:25 PM To: u2-users@listserver.u2ug.org Subject: RE: [U2] static/dynamic file(s) opinions wanted! Hi Doug, In addition to the great info from this group (like Dan provides), this site http://www.fitzlong.com/ provides great insight into U2 files. You should take some time to read the papers that they've provided. Also, they have an incredible product called fast that can really help you eliminate a lot of the maintenance headaches and increase your system performance. HTH -- Original message -- From: Doug Chanco [EMAIL PROTECTED] Dan thanks a million this is incredibly helpful! Dougc --- u2-users mailing list u2-users@listserver.u2ug.org To unsubscribe please visit http://listserver.u2ug.org/ --- u2-users mailing list u2-users@listserver.u2ug.org To unsubscribe please visit http://listserver.u2ug.org/ --- u2-users mailing list u2-users@listserver.u2ug.org To unsubscribe please visit http://listserver.u2ug.org/
RE: [U2] static/dynamic file(s) opinions wanted!
This is the first time I have encountered a restriction on using CONCURRENT on Type 30 files. It comes as quite a surprise, because 24x7 sites like the Ambulance Service need it. I suspect that the error on USING is a subsequent error from the first one. You should always be able to specify an alternative file system, against the possibility that there's not enough space on the current file system for the second copy of the file. Would be interested to hear what IBM support has to say on this. --- u2-users mailing list u2-users@listserver.u2ug.org To unsubscribe please visit http://listserver.u2ug.org/
RE: [U2] static/dynamic file(s) opinions wanted!
-Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Dan Fitzgerald Sent: 23 July 2008 00:28 To: u2-users@listserver.u2ug.org Subject: RE: [U2] static/dynamic file(s) opinions wanted! snip I saw a 1Tb drive at Fry's for $170 the other day, so space isn't an issue anymore, either. /snip Actually space can be an issue, most 'servers' will be raid and 1 or 2 u and probably have all disks in placem and be in a hosting center - to increase the disk space is a matter of a san, or changing all the disks in the box, or extending partitions and rebuilding - certainly quite a chunk of downtime Symeon. --- u2-users mailing list u2-users@listserver.u2ug.org To unsubscribe please visit http://listserver.u2ug.org/
RE: [U2] static/dynamic file(s) opinions wanted!
Wow, Super detailed explanation. I want to avoid the Katrina effect but 1. I am not sure what to check for, I can run ANALYZE.FILE and get the below results (i his the right way to inspect my levees or should I be using something else? I am just used to using jRF -R jASE report only) and HASH.HELP and I am not sure what to use for dynamic files to inspect them. ANALYZE.FILE PARTS File name .. PARTS Pathname ... PARTS File type .. DYNAMIC Hashing Algorithm .. GENERAL No. of groups (modulus) 240052 current ( minimum 1 ) Large record size .. 1600 bytes Group size . 2048 bytes Load factors ... 80% (split), 50% (merge) and 35% (actual) Total size . 907905024 bytes Thanks for the great explanation Dougc Ps On a semi related subject, I sure do miss fry's, there is not one here in NC . -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Dan Fitzgerald Sent: Tuesday, July 22, 2008 7:28 PM To: u2-users@listserver.u2ug.org Subject: RE: [U2] static/dynamic file(s) opinions wanted! And once a dynamic file starts splitting, writes become very inefficient. I'd say that the most efficient file is a well-sized static one. You want it wide shallow. I'm not against having a mod larger than the number of items anymore, especially if the file is going to grow. If I know that the file is going to grow to 1.5Gb, I'll create it that large. With today's storage technologies, having empty groups isn't all that bad a thing. The old empty groups impact SELECT statements isn't as true as it once was, and if you do so many selects that it's an issue, you might want to look into indices. I saw a 1Tb drive at Fry's for $170 the other day, so space isn't an issue anymore, either. Dynamic files used to be sold as you don't have to maintain them. You do, though. If your current mod is GT your minimum mod, you should resize up with a different min mod. That's maintenance. The reason is that your file has reached a point where you're adding data to a file with an increasing chance that the group is already near the split %. When that happens, your write turns into a beast. It has to go to whatever mechanism your OS has for finding free space, attaching that, updating that table, dividing the data in RAM between the two groups (old new), read in the header ( there's only one per file, no matter how many users want it), lock it, update it with the new structure, write it, unlock it, write out the old group, write out the new group. Undersized static files can be as bad, but dynamic files have the Katrina effect. Everybody expects that the levees will hold, and FEMA will rescue them, and dynamic files don't need maintenance, so they don't check to be sure... From: [EMAIL PROTECTED] To: u2-users@listserver.u2ug.org Subject: RE: [U2] static/dynamic file(s) opinions wanted! Date: Tue, 22 Jul 2008 18:25:50 -0400 Thanks Kevin but I am still missing something, if dynamic files are well dynamic why would one need to resize them and how would one know what to resize them to? I am sorry if this is basic stuff but I come from a jBASE world and have never really used dynamic files. you cannot run HASH.HELP on a dynamic file and ANALYZE.FILE does not seem to return anything useful in resizing, so how would one determaine (without using a rool like the ones previously mentioned). I LOVE to use tools like these BUT I also like to know how to do it manually in case I cannot use these tool(s) (for whatever reason). I guess the gist of my question(s) would be 1. how to tell if a dynamic file needs to be resized 2. my other question about which is better seems to be that dynamic files are if you have LOTS of files and they grow a lot and you cannot maintain them thanks everyone! dougc -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Kevin King Sent: Tuesday, July 22, 2008 4:36 PM To: u2-users@listserver.u2ug.org Subject: Re: [U2] static/dynamic file(s) opinions wanted! Files that grow at a controlled rate and especially files that could exceed 2G are good candidates for dynamic files. Files that are cleared, or files that have masses of data loaded or removed from them, are not. Work files, for example, can be a horrible use of dynamic files. As was stated earlier, it's crucial that the original block size of the dynamic file be set properly, otherwise the file could split way too often. On Tue, Jul 22, 2008 at 1:02 PM, Symeon Breen [EMAIL PROTECTED] wrote: We use dynamic files no problem - yes I suppose in certain circumstances there is an overhead, but it would still be faster than a badly sized static file. The conclusion we have is if you are really on top of your file sizes and administrating things daily there is probably less need for dynamic files. If however you have hundreds of accounts and files
RE: [U2] static/dynamic file(s) opinions wanted!
See the minimum modulus of 1 the current mod of 240052? your file has split 240051 times, and a lot of your writes are probably going to involve a split operation. Check this again in a couple of days to see what the current mod is to see how many split operations you've done. An interesting thing to see would be the sizes of the DATA.30 OVER.30 files; with a mod of 240052 a group size of 2048, the file is sized for 469Mb, yet the actual data size is 860Gb. My first guess would be that you are storing a lot of oversized items (1600b, the Large Record Size) - nearly 400Mb. IIRC, UV puts a pointer in the primary group to the overflow, so each I/O will be at least 2 disk accesses for an oversized item (this is efficient: if it tried to store the first 1600 bytes here in primary space, you'd have to read all of the oversized items to get to the most recently accessed items at the end of the group). If a large fraction of your oversized items are under, say 3200b, doubling the group size might help significantly, bearing in mind that you don't want to exceed the page size for your OS (typically 4K)(think about it a moment: if data is stored on disk in 4k pages, and you have a group size above that, every read is two physical disk reads, cutting your throughput capacity in half). To look at this, I use a small program that opens the file, loops through a basic select while readnext, reads each record, does a len(rec), then increments a counter in an attribute where 1 is a count of records from 0-500b in size, 2 is 501-1000, 3 is 1001-1500, etc. I've often found it useful to also make from attribute 20 (30? 40? your choice) on up multivalued, with the 1st mv being the counter then a subvalue list of item ids in the 2nd mv. This way, you can see a histogram of your record sizes, and make more informed file sizing/typing decisions. Subject: RE: [U2] static/dynamic file(s) opinions wanted! Date: Wed, 23 Jul 2008 07:55:32 -0400 From: [EMAIL PROTECTED] To: u2-users@listserver.u2ug.org Wow, Super detailed explanation. I want to avoid the Katrina effect but 1. I am not sure what to check for, I can run ANALYZE.FILE and get the below results (i his the right way to inspect my levees or should I be using something else? I am just used to using jRF -R jASE report only) and HASH.HELP and I am not sure what to use for dynamic files to inspect them. ANALYZE.FILE PARTS File name .. PARTS Pathname ... PARTS File type .. DYNAMIC Hashing Algorithm .. GENERAL No. of groups (modulus) 240052 current ( minimum 1 ) Large record size .. 1600 bytes Group size . 2048 bytes Load factors ... 80% (split), 50% (merge) and 35% (actual) Total size . 907905024 bytes Thanks for the great explanation Dougc Ps On a semi related subject, I sure do miss fry's, there is not one here in NC . -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Dan Fitzgerald Sent: Tuesday, July 22, 2008 7:28 PM To: u2-users@listserver.u2ug.org Subject: RE: [U2] static/dynamic file(s) opinions wanted! And once a dynamic file starts splitting, writes become very inefficient. I'd say that the most efficient file is a well-sized static one. You want it wide shallow. I'm not against having a mod larger than the number of items anymore, especially if the file is going to grow. If I know that the file is going to grow to 1.5Gb, I'll create it that large. With today's storage technologies, having empty groups isn't all that bad a thing. The old empty groups impact SELECT statements isn't as true as it once was, and if you do so many selects that it's an issue, you might want to look into indices. I saw a 1Tb drive at Fry's for $170 the other day, so space isn't an issue anymore, either. Dynamic files used to be sold as you don't have to maintain them. You do, though. If your current mod is GT your minimum mod, you should resize up with a different min mod. That's maintenance. The reason is that your file has reached a point where you're adding data to a file with an increasing chance that the group is already near the split %. When that happens, your write turns into a beast. It has to go to whatever mechanism your OS has for finding free space, attaching that, updating that table, dividing the data in RAM between the two groups (old new), read in the header ( there's only one per file, no matter how many users want it), lock it, update it with the new structure, write it, unlock it, write out the old group, write out the new group. Undersized static files can be as bad, but dynamic files have the Katrina effect. Everybody expects that the levees will hold, and FEMA will rescue them, and dynamic files don't need maintenance, so they don't check to be sure... From: [EMAIL PROTECTED] To: u2-users@listserver.u2ug.org Subject: RE: [U2] static/dynamic file(s) opinions wanted! Date: Tue, 22 Jul 2008 18
RE: [U2] static/dynamic file(s) opinions wanted!
Dan thanks a million this is incredibly helpful! Dougc --- u2-users mailing list u2-users@listserver.u2ug.org To unsubscribe please visit http://listserver.u2ug.org/
RE: [U2] static/dynamic file(s) opinions wanted!
Hi Doug, In addition to the great info from this group (like Dan provides), this site http://www.fitzlong.com/ provides great insight into U2 files. You should take some time to read the papers that they've provided. Also, they have an incredible product called fast that can really help you eliminate a lot of the maintenance headaches and increase your system performance. HTH -- Original message -- From: Doug Chanco [EMAIL PROTECTED] Dan thanks a million this is incredibly helpful! Dougc --- u2-users mailing list u2-users@listserver.u2ug.org To unsubscribe please visit http://listserver.u2ug.org/ --- u2-users mailing list u2-users@listserver.u2ug.org To unsubscribe please visit http://listserver.u2ug.org/
RE: [U2] static/dynamic file(s) opinions wanted!
I have always preferred dynamic files. In the long run, they are easier to deal with. You should still determine the best block size. Also, if you are going to bulk load it, you should set the min mod so that it does not start splitting right off the bat. You still want the basic size to be accurate if you can determine that up front. That being said, there is the nasty blink error (backward link). The only time I have seen these were in a vendor's proprietary index files, but when we got them, the file was worthless. Fortunately, since they were not data, we could rebuild these files. John Israel -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of doug chanco Sent: Tuesday, July 22, 2008 11:36 AM To: u2-users@listserver.u2ug.org Subject: [U2] static/dynamic file(s) opinions wanted! hey all, I have heard bad things about using dynamic files versus hashed/static ones. Can anyone share any thoughts on which is better (in particular on a system where the files grow at a fairly steady rate). I always understood that dynamic files were best on files that did not change that much that fast as the constant need to resize would outweigh the manual effort of resizing the files manually (or with a program). I am looking for insight (or where to find some insight) on universe and best file practices (right now I am reading the system description manual and its helping but lacks insights that I am sure some of the old pickies on here have) so any thoughts/suggestions/ideas/comments are welcomed! thanks dougc ps universe 10.1 and aix 5.2 --- u2-users mailing list u2-users@listserver.u2ug.org To unsubscribe please visit http://listserver.u2ug.org/ --- u2-users mailing list u2-users@listserver.u2ug.org To unsubscribe please visit http://listserver.u2ug.org/
RE: [U2] static/dynamic file(s) opinions wanted!
We use dynamic files no problem - yes I suppose in certain circumstances there is an overhead, but it would still be faster than a badly sized static file. The conclusion we have is if you are really on top of your file sizes and administrating things daily there is probably less need for dynamic files. If however you have hundreds of accounts and files then dynamic files are easier to admin and hence probably faster in the long term. Symeon. -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of doug chanco Sent: 22 July 2008 16:36 To: u2-users@listserver.u2ug.org Subject: [U2] static/dynamic file(s) opinions wanted! hey all, I have heard bad things about using dynamic files versus hashed/static ones. Can anyone share any thoughts on which is better (in particular on a system where the files grow at a fairly steady rate). I always understood that dynamic files were best on files that did not change that much that fast as the constant need to resize would outweigh the manual effort of resizing the files manually (or with a program). I am looking for insight (or where to find some insight) on universe and best file practices (right now I am reading the system description manual and its helping but lacks insights that I am sure some of the old pickies on here have) so any thoughts/suggestions/ideas/comments are welcomed! thanks dougc ps universe 10.1 and aix 5.2 --- u2-users mailing list u2-users@listserver.u2ug.org To unsubscribe please visit http://listserver.u2ug.org/ No virus found in this incoming message. Checked by AVG - http://www.avg.com Version: 8.0.138 / Virus Database: 270.5.3/1565 - Release Date: 7/21/2008 6:36 PM --- u2-users mailing list u2-users@listserver.u2ug.org To unsubscribe please visit http://listserver.u2ug.org/
RE: [U2] static/dynamic file(s) opinions wanted!
Thanks for the info, so you do any maintenance on the files or does universe do a good job maintaining the files? The reason I ask is that we resized all our files back in DEC (using fitzgerald and long's fast tool) and our nightly process went from finishing @6am to @3am and we are now back to 6am finish times, since I get a daily report of processing time I was able to watch it creeping Now I know that there are a lot more items to process but my gut feeling is that the files are in need of resizing but since they are dynamic maybe I am wrong dougc __ Information from ESET NOD32 Antivirus, version of virus signature database 3288 (20080722) __ The message was checked by ESET NOD32 Antivirus. http://www.eset.com --- u2-users mailing list u2-users@listserver.u2ug.org To unsubscribe please visit http://listserver.u2ug.org/
Re: [U2] static/dynamic file(s) opinions wanted!
Files that grow at a controlled rate and especially files that could exceed 2G are good candidates for dynamic files. Files that are cleared, or files that have masses of data loaded or removed from them, are not. Work files, for example, can be a horrible use of dynamic files. As was stated earlier, it's crucial that the original block size of the dynamic file be set properly, otherwise the file could split way too often. On Tue, Jul 22, 2008 at 1:02 PM, Symeon Breen [EMAIL PROTECTED] wrote: We use dynamic files no problem - yes I suppose in certain circumstances there is an overhead, but it would still be faster than a badly sized static file. The conclusion we have is if you are really on top of your file sizes and administrating things daily there is probably less need for dynamic files. If however you have hundreds of accounts and files then dynamic files are easier to admin and hence probably faster in the long term. Symeon. -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of doug chanco Sent: 22 July 2008 16:36 To: u2-users@listserver.u2ug.org Subject: [U2] static/dynamic file(s) opinions wanted! hey all, I have heard bad things about using dynamic files versus hashed/static ones. Can anyone share any thoughts on which is better (in particular on a system where the files grow at a fairly steady rate). I always understood that dynamic files were best on files that did not change that much that fast as the constant need to resize would outweigh the manual effort of resizing the files manually (or with a program). I am looking for insight (or where to find some insight) on universe and best file practices (right now I am reading the system description manual and its helping but lacks insights that I am sure some of the old pickies on here have) so any thoughts/suggestions/ideas/comments are welcomed! thanks dougc ps universe 10.1 and aix 5.2 --- u2-users mailing list u2-users@listserver.u2ug.org To unsubscribe please visit http://listserver.u2ug.org/ No virus found in this incoming message. Checked by AVG - http://www.avg.com Version: 8.0.138 / Virus Database: 270.5.3/1565 - Release Date: 7/21/2008 6:36 PM --- u2-users mailing list u2-users@listserver.u2ug.org To unsubscribe please visit http://listserver.u2ug.org/ -- -Kevin http://www.PrecisOnline.com --- u2-users mailing list u2-users@listserver.u2ug.org To unsubscribe please visit http://listserver.u2ug.org/
RE: [U2] static/dynamic file(s) opinions wanted!
Symeon, In today's world of faster and faster processors with more and more disk space, we always use dynamic files for transactions based records and static for code files or files that have little growth. We work in both Unidata and Universe and find they both have drawback and pluses which would take pages and pages to hash out the differences. Resizing on static files if the are used as described above ends up being a annual or semi-annual task. Dynamic files for Universe, we resize on a bimonthly basis. Unidata files are resized on a monthly basis. We rarely see a slow down in any of our client sites, since we are on top of our clients file systems. [ad] Of course, we use our own resizing tool based on the www.eclipse.org tool. It is the only new tool that resizes both Unidata and Universe files fast and effectively. This tool supports multiple accounts and multiple customers simultaneously without the need to download anything software on their machines. You can run this tool from any machine that has Eclipse installed on it. So if you like Mac's or Linux, and even Windows you can use our tool. [/ad] Regards, Doug www.u2logic.com - Original Message - From: Symeon Breen [EMAIL PROTECTED] To: u2-users@listserver.u2ug.org Sent: 07/22/2008 3:02 PM Subject: RE: [U2] static/dynamic file(s) opinions wanted! We use dynamic files no problem - yes I suppose in certain circumstances there is an overhead, but it would still be faster than a badly sized static file. The conclusion we have is if you are really on top of your file sizes and administrating things daily there is probably less need for dynamic files. If however you have hundreds of accounts and files then dynamic files are easier to admin and hence probably faster in the long term. Symeon. -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of doug chanco Sent: 22 July 2008 16:36 To: u2-users@listserver.u2ug.org Subject: [U2] static/dynamic file(s) opinions wanted! hey all, I have heard bad things about using dynamic files versus hashed/static ones. Can anyone share any thoughts on which is better (in particular on a system where the files grow at a fairly steady rate). I always understood that dynamic files were best on files that did not change that much that fast as the constant need to resize would outweigh the manual effort of resizing the files manually (or with a program). I am looking for insight (or where to find some insight) on universe and best file practices (right now I am reading the system description manual and its helping but lacks insights that I am sure some of the old pickies on here have) so any thoughts/suggestions/ideas/comments are welcomed! thanks dougc ps universe 10.1 and aix 5.2 --- u2-users mailing list u2-users@listserver.u2ug.org To unsubscribe please visit http://listserver.u2ug.org/ No virus found in this incoming message. Checked by AVG - http://www.avg.com Version: 8.0.138 / Virus Database: 270.5.3/1565 - Release Date: 7/21/2008 6:36 PM --- u2-users mailing list u2-users@listserver.u2ug.org To unsubscribe please visit http://listserver.u2ug.org/ --- u2-users mailing list u2-users@listserver.u2ug.org To unsubscribe please visit http://listserver.u2ug.org/ --- u2-users mailing list u2-users@listserver.u2ug.org To unsubscribe please visit http://listserver.u2ug.org/
RE: [U2] static/dynamic file(s) opinions wanted!
And once a dynamic file starts splitting, writes become very inefficient. I'd say that the most efficient file is a well-sized static one. You want it wide shallow. I'm not against having a mod larger than the number of items anymore, especially if the file is going to grow. If I know that the file is going to grow to 1.5Gb, I'll create it that large. With today's storage technologies, having empty groups isn't all that bad a thing. The old empty groups impact SELECT statements isn't as true as it once was, and if you do so many selects that it's an issue, you might want to look into indices. I saw a 1Tb drive at Fry's for $170 the other day, so space isn't an issue anymore, either. Dynamic files used to be sold as you don't have to maintain them. You do, though. If your current mod is GT your minimum mod, you should resize up with a different min mod. That's maintenance. The reason is that your file has reached a point where you're adding data to a file with an increasing chance that the group is already near the split %. When that happens, your write turns into a beast. It has to go to whatever mechanism your OS has for finding free space, attaching that, updating that table, dividing the data in RAM between the two groups (old new), read in the header ( there's only one per file, no matter how many users want it), lock it, update it with the new structure, write it, unlock it, write out the old group, write out the new group. Undersized static files can be as bad, but dynamic files have the Katrina effect. Everybody expects that the levees will hold, and FEMA will rescue them, and dynamic files don't need maintenance, so they don't check to be sure... From: [EMAIL PROTECTED] To: u2-users@listserver.u2ug.org Subject: RE: [U2] static/dynamic file(s) opinions wanted! Date: Tue, 22 Jul 2008 18:25:50 -0400 Thanks Kevin but I am still missing something, if dynamic files are well dynamic why would one need to resize them and how would one know what to resize them to? I am sorry if this is basic stuff but I come from a jBASE world and have never really used dynamic files. you cannot run HASH.HELP on a dynamic file and ANALYZE.FILE does not seem to return anything useful in resizing, so how would one determaine (without using a rool like the ones previously mentioned). I LOVE to use tools like these BUT I also like to know how to do it manually in case I cannot use these tool(s) (for whatever reason). I guess the gist of my question(s) would be 1. how to tell if a dynamic file needs to be resized 2. my other question about which is better seems to be that dynamic files are if you have LOTS of files and they grow a lot and you cannot maintain them thanks everyone! dougc -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Kevin King Sent: Tuesday, July 22, 2008 4:36 PM To: u2-users@listserver.u2ug.org Subject: Re: [U2] static/dynamic file(s) opinions wanted! Files that grow at a controlled rate and especially files that could exceed 2G are good candidates for dynamic files. Files that are cleared, or files that have masses of data loaded or removed from them, are not. Work files, for example, can be a horrible use of dynamic files. As was stated earlier, it's crucial that the original block size of the dynamic file be set properly, otherwise the file could split way too often. On Tue, Jul 22, 2008 at 1:02 PM, Symeon Breen [EMAIL PROTECTED] wrote: We use dynamic files no problem - yes I suppose in certain circumstances there is an overhead, but it would still be faster than a badly sized static file. The conclusion we have is if you are really on top of your file sizes and administrating things daily there is probably less need for dynamic files. If however you have hundreds of accounts and files then dynamic files are easier to admin and hence probably faster in the long term. Symeon. -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of doug chanco Sent: 22 July 2008 16:36 To: u2-users@listserver.u2ug.org Subject: [U2] static/dynamic file(s) opinions wanted! hey all, I have heard bad things about using dynamic files versus hashed/static ones. Can anyone share any thoughts on which is better (in particular on a system where the files grow at a fairly steady rate). I always understood that dynamic files were best on files that did not change that much that fast as the constant need to resize would outweigh the manual effort of resizing the files manually (or with a program). I am looking for insight (or where to find some insight) on universe and best file practices (right now I am reading the system description manual and its helping but lacks insights that I am sure some of the old pickies on here have) so any thoughts/suggestions/ideas/comments are welcomed! thanks dougc ps universe 10.1 and aix 5.2 --- u2-users mailing list u2-users@listserver.u2ug.org
Re: [U2] static/dynamic file(s) opinions wanted!
Ray, is there any problem to doing this while people are logged in and might write to the file Assuming the file is not in common jak - Original Message - From: Ray Wurlod [EMAIL PROTECTED] To: u2-users@listserver.u2ug.org Sent: Wednesday, July 23, 2008 9:09 AM Subject: RE: [U2] static/dynamic file(s) opinions wanted! Occasionally RESIZE filename * * * is a good thing to do with dynamic files. This re-packs each group, consolidating fragmented free space (from deleted/reduced records) in (at the end of) each group buffer. It may, if you're lucky, also bring some groups out of overflow. Dynamic files do not, of themselves, re-pack groups in normal operation unless by doing so they would avoid extending the group into overflow. - Original Message - From: Doug Chanco [EMAIL PROTECTED] To: u2-users@listserver.u2ug.org Subject: RE: [U2] static/dynamic file(s) opinions wanted! Date: Tue, 22 Jul 2008 18:25:50 -0400 Thanks Kevin but I am still missing something, if dynamic files are well dynamic why would one need to resize them and how would one know what to resize them to? I am sorry if this is basic stuff but I come from a jBASE world and have never really used dynamic files. --- u2-users mailing list u2-users@listserver.u2ug.org To unsubscribe please visit http://listserver.u2ug.org/ --- u2-users mailing list u2-users@listserver.u2ug.org To unsubscribe please visit http://listserver.u2ug.org/
Re: [U2] static/dynamic file(s) opinions wanted!
RESIZE filename * * * CONCURRENT USING tempdirpath will be fine even if the dynamic hashed file is open in COMMON. Make sure that the combination tempdirpath/filename is less than 128 characters long. They have put a deal of work into making CONCURRENT safe over the years. Any bad press you read is probably old. It's still true that you should avoid INPLACE at all costs, especially for dynamic files! - Original Message - From: John Kent [EMAIL PROTECTED] To: u2-users@listserver.u2ug.org Subject: Re: [U2] static/dynamic file(s) opinions wanted! Date: Wed, 23 Jul 2008 09:48:50 +1000 Ray, is there any problem to doing this while people are logged in and might write to the file Assuming the file is not in common jak - Original Message - From: Ray Wurlod [EMAIL PROTECTED] To: u2-users@listserver.u2ug.org Sent: Wednesday, July 23, 2008 9:09 AM Subject: RE: [U2] static/dynamic file(s) opinions wanted! Occasionally RESIZE filename * * * is a good thing to do with dynamic files. This re-packs each group, consolidating fragmented free space (from deleted/reduced records) in (at the end of) each group buffer. It may, if you're lucky, also bring some groups out of overflow. Dynamic files do not, of themselves, re-pack groups in normal operation unless by doing so they would avoid extending the group into overflow. - Original Message - From: Doug Chanco [EMAIL PROTECTED] To: u2-users@listserver.u2ug.org Subject: RE: [U2] static/dynamic file(s) opinions wanted! Date: Tue, 22 Jul 2008 18:25:50 -0400 Thanks Kevin but I am still missing something, if dynamic files are well dynamic why would one need to resize them and how would one know what to resize them to? I am sorry if this is basic stuff but I come from a jBASE world and have never really used dynamic files. --- u2-users mailing list u2-users@listserver.u2ug.org To unsubscribe please visit http://listserver.u2ug.org/ --- u2-users mailing list u2-users@listserver.u2ug.org To unsubscribe please visit http://listserver.u2ug.org/ --- u2-users mailing list u2-users@listserver.u2ug.org To unsubscribe please visit http://listserver.u2ug.org/
