Re: [U2] Sequentially Hashed Files
In message 6e8c6c3ab287df45837a5816ce37531d68d6544...@daconosbserver.daconosbs.loca l, David Jordan da...@dacono.com.au writes Hi Ray Are you using UniVerse or UniData. I use dynamic files and never have to resize them in UniVerse. The OP said that the files keep on growing and shrinking - and that is not good for dynamic files. Although I would ask why you have to keep resizing and generally maintaining the files? You shouldn't have to do that with dynamic files (unless, of course, you're short of disk space and overflow wastes huge amounts). I'd go with Adrian's suggestion and use MINIMUM.MODULUS. *However* you did say the files regularly get emptied. Do you have an app that processes the file and empties it? If it's disk space you're worried about, just do a CLEAR.FILE inside your app once you've emptied the file - I think you can do it as a basic statement rather than an execute. You'll need to look at how to do it (it should be very simple) but I'd be inclined to do the below on every file after it's processed lock file clearselect select file if count = 0 then clearfile release That's all your maintenance and disk freeing automated for you... Cheers, Wol -- Anthony W. Youngman pi...@thewolery.demon.co.uk 'Yings, yow graley yin! Suz ae rikt dheu,' said the blue man, taking the thimble. 'What *is* he?' said Magrat. 'They're gnomes,' said Nanny. The man lowered the thimble. 'Pictsies!' Carpe Jugulum, Terry Pratchett 1998 Visit the MaVerick web-site - http://www.maverick-dbms.org Open Source Pick ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
Re: [U2] Sequentially Hashed Files
I agree with both setting a min modulus AND doing a CLEARFILE if the file is empty. Just remember that if an empty file has a big modulus, it can still take some time for the select to chug through all the groups. That time will be determined by the mod and the performance of your machine. John Israel Senior Programmer/Analyst Dayton Superior Corporation 721 Richard St. Miamisburg, OHÂ 45342 937-866-0711 x44380 -Original Message- From: u2-users-boun...@listserver.u2ug.org [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Anthony W. Youngman Sent: Wednesday, October 21, 2009 3:52 AM To: u2-users@listserver.u2ug.org Subject: Re: [U2] Sequentially Hashed Files In message 6e8c6c3ab287df45837a5816ce37531d68d6544...@daconosbserver.daconosbs.loca l, David Jordan da...@dacono.com.au writes Hi Ray Are you using UniVerse or UniData. I use dynamic files and never have to resize them in UniVerse. The OP said that the files keep on growing and shrinking - and that is not good for dynamic files. Although I would ask why you have to keep resizing and generally maintaining the files? You shouldn't have to do that with dynamic files (unless, of course, you're short of disk space and overflow wastes huge amounts). I'd go with Adrian's suggestion and use MINIMUM.MODULUS. *However* you did say the files regularly get emptied. Do you have an app that processes the file and empties it? If it's disk space you're worried about, just do a CLEAR.FILE inside your app once you've emptied the file - I think you can do it as a basic statement rather than an execute. You'll need to look at how to do it (it should be very simple) but I'd be inclined to do the below on every file after it's processed lock file clearselect select file if count = 0 then clearfile release That's all your maintenance and disk freeing automated for you... Cheers, Wol -- Anthony W. Youngman pi...@thewolery.demon.co.uk 'Yings, yow graley yin! Suz ae rikt dheu,' said the blue man, taking the thimble. 'What *is* he?' said Magrat. 'They're gnomes,' said Nanny. The man lowered the thimble. 'Pictsies!' Carpe Jugulum, Terry Pratchett 1998 Visit the MaVerick web-site - http://www.maverick-dbms.org Open Source Pick ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
[U2] Sequentially Hashed Files
Does anyone have any experience using Sequentially hashed files? The documentation on Sequentially hashed files is not very explicit about what kinds of benefits or problems can be expected / experienced.. It also only hints towards one use case, but IMO doesn't do enough to clearly define when these types of files are actually beneficial and why... Thanks in advance. Ray ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
Re: [U2] Sequentially Hashed Files
In message 5e5ef002669716488cf5f69288ba6b093bf...@msgl-lon01-es01.msgluk.local, Raymond P. de Bourbon rdebour...@msgl.com writes Does anyone have any experience using Sequentially hashed files? The documentation on Sequentially hashed files is not very explicit about what kinds of benefits or problems can be expected / experienced.. It also only hints towards one use case, but IMO doesn't do enough to clearly define when these types of files are actually beneficial and why... I've never had any need to use sequentially hashed files, but I'm quite happy to try and explain it ... Two questions to start with: Firstly, is your app generating sequential numeric keys? And do you only create records, never delete them? Secondly (less importantly), are your records of a roughly consistent size? The point of sequential files is that, if your answer to both questions is yes, a sequential file is trivially and perfectly balanced. And such files are common enough to make creating a special file type worth doing. Rather than trying to find some hash algorithm that does a good job, sequential files use the key value as the hash value. So if each record fills a group, record 1 goes in group 1, record 2 goes in group 2, etc etc. If two records fill a group, then it goes 1, 1:2, 1:3,2, 1:3,2:4, etc etc. I would have thought that if the typical record ended up in overflow, then sequential would be a good match for sequential keys too regardless of whether the size is consistent. However, DON'T take my GUESS on that. And bear in mind the advice is, if the answer to the first question is no, then DON'T go near sequential files. They're a special case, for a special-case situation. It's just that that special case is actually quite common. Cheers, Wol -- Anthony W. Youngman pi...@thewolery.demon.co.uk 'Yings, yow graley yin! Suz ae rikt dheu,' said the blue man, taking the thimble. 'What *is* he?' said Magrat. 'They're gnomes,' said Nanny. The man lowered the thimble. 'Pictsies!' Carpe Jugulum, Terry Pratchett 1998 Visit the MaVerick web-site - http://www.maverick-dbms.org Open Source Pick ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
Re: [U2] Sequentially Hashed Files
Thanks for the response.. Here's my situation: The app is generating records with sequential numeric keys and over time the records are deleted with the file returning to a count of zero.. The record sizes can vary considerably from record to record... Currently we are using dynamic files and are constantly having to resize and generally having to maintain the files, and it is becoming quite a headache for us .. Right now, we are just looking for other options and I had seen sequentially hashed files in the manual, and it said that they are bested suited for cases with sequential keys (but says nothing about create not delete etc) - so I thought I would ask... Looks like we may just have to spend time working on some admin scripts to maintain the dynamic files automatically... Ray -Original Message- From: u2-users-boun...@listserver.u2ug.org [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Anthony W. Youngman Sent: 20 October 2009 16:46 To: u2-users@listserver.u2ug.org Subject: Re: [U2] Sequentially Hashed Files In message 5e5ef002669716488cf5f69288ba6b093bf...@msgl-lon01-es01.msgluk.local, Raymond P. de Bourbon rdebour...@msgl.com writes Does anyone have any experience using Sequentially hashed files? The documentation on Sequentially hashed files is not very explicit about what kinds of benefits or problems can be expected / experienced.. It also only hints towards one use case, but IMO doesn't do enough to clearly define when these types of files are actually beneficial and why... I've never had any need to use sequentially hashed files, but I'm quite happy to try and explain it ... Two questions to start with: Firstly, is your app generating sequential numeric keys? And do you only create records, never delete them? Secondly (less importantly), are your records of a roughly consistent size? The point of sequential files is that, if your answer to both questions is yes, a sequential file is trivially and perfectly balanced. And such files are common enough to make creating a special file type worth doing. Rather than trying to find some hash algorithm that does a good job, sequential files use the key value as the hash value. So if each record fills a group, record 1 goes in group 1, record 2 goes in group 2, etc etc. If two records fill a group, then it goes 1, 1:2, 1:3,2, 1:3,2:4, etc etc. I would have thought that if the typical record ended up in overflow, then sequential would be a good match for sequential keys too regardless of whether the size is consistent. However, DON'T take my GUESS on that. And bear in mind the advice is, if the answer to the first question is no, then DON'T go near sequential files. They're a special case, for a special-case situation. It's just that that special case is actually quite common. Cheers, Wol -- Anthony W. Youngman pi...@thewolery.demon.co.uk 'Yings, yow graley yin! Suz ae rikt dheu,' said the blue man, taking the thimble. 'What *is* he?' said Magrat. 'They're gnomes,' said Nanny. The man lowered the thimble. 'Pictsies!' Carpe Jugulum, Terry Pratchett 1998 Visit the MaVerick web-site - http://www.maverick-dbms.org Open Source Pick ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
Re: [U2] Sequentially Hashed Files
Hi, Creating a type-19 file (or type-1) has advantages with large records IF you are not doing many reads and writes to the file. If you are updating a type of log record, look into using OPENSEQ, READSEQ, WRITESEQ, instead of READs and WRITEs. With the SEQ functions, you are appending everything at the bottom of the record. Tom RATEX Business Solutions -Original Message- From: u2-users-boun...@listserver.u2ug.org [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Raymond P. de Bourbon Sent: Tuesday, October 20, 2009 11:58 AM To: U2 Users List Subject: Re: [U2] Sequentially Hashed Files Thanks for the response.. Here's my situation: The app is generating records with sequential numeric keys and over time the records are deleted with the file returning to a count of zero.. The record sizes can vary considerably from record to record... Currently we are using dynamic files and are constantly having to resize and generally having to maintain the files, and it is becoming quite a headache for us .. Right now, we are just looking for other options and I had seen sequentially hashed files in the manual, and it said that they are bested suited for cases with sequential keys (but says nothing about create not delete etc) - so I thought I would ask... Looks like we may just have to spend time working on some admin scripts to maintain the dynamic files automatically... Ray ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
Re: [U2] Sequentially Hashed Files
Hi Ray Are you using UniVerse or UniData. I use dynamic files and never have to resize them in UniVerse. Regards David Jordan ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
Re: [U2] Sequentially Hashed Files
Have you tried setting MINIMUM.MODULUS to a high value on your dynamic file. This prevents the file from merging below the size you've set. Although it can still split to larger than that size. You can use the CONFIGURE.FILE command to set MINIMUM.MODULUS - but it won't take effect until the next time you use CLEAR.FILE. It may also be useful to set LARGE.RECORD size (see the help for CONFIGURE.FILE). -Original Message- From: u2-users-boun...@listserver.u2ug.org [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Raymond P. de Bourbon Sent: Tuesday, 20 October 2009 11:58 PM To: U2 Users List Subject: Re: [U2] Sequentially Hashed Files Thanks for the response.. Here's my situation: The app is generating records with sequential numeric keys and over time the records are deleted with the file returning to a count of zero.. The record sizes can vary considerably from record to record... Currently we are using dynamic files and are constantly having to resize and generally having to maintain the files, and it is becoming quite a headache for us .. Right now, we are just looking for other options and I had seen sequentially hashed files in the manual, and it said that they are bested suited for cases with sequential keys (but says nothing about create not delete etc) - so I thought I would ask... Looks like we may just have to spend time working on some admin scripts to maintain the dynamic files automatically... Ray DISCLAIMER: Disclaimer. This e-mail is private and confidential. If you are not the intended recipient, please advise us by return e-mail immediately, and delete the e-mail and any attachments without using or disclosing the contents in any way. The views expressed in this e-mail are those of the author, and do not represent those of this company unless this is clearly indicated. You should scan this e-mail and any attachments for viruses. This company accepts no liability for any direct or indirect damage or loss resulting from the use of any attachments to this e-mail. ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
