Re: [U2] [UV] Marking New Index as Built
I tried this at U2 university on 11.1 in December with problems similar to those in my earlier message. -Rick On May 6, 2011, at 6:09 AM, "Kurt Neumann" wrote: > Hi Perry > > One of the new features in UniVerse 11.1 is the concurrent build of indices. > I would suggest being very careful in using it as, to my knowledge, it could > have an impact on your performance. From the new features Pdf: BUILD.INDEX > filename {attribute [attribute...] | ALL } [CONCURRENT] > > Regards > Kurt Neumann; U2 Distributor in Africa > > > > -Original Message- > From: u2-users-boun...@listserver.u2ug.org > [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Perry Taylor > Sent: 06 May 2011 02:55 PM > To: U2 Users List > Subject: Re: [U2] [UV] Marking New Index as Built > > In the end I took down the application that updates the indexed field and did > a BUILD.INDEX at 1am. It took approximately one hour to build but > unfortunately another process that accesses that file shutdown in error as > the BUILD.INDEX had the FX lock on the file. No harm done but again without > a truly online build this will continue to be a messy operation for 7X24 > shops. Sure wish Rocket would take this seriously and give us a better tool. > > Perry > > -Original Message- > From: u2-users-boun...@listserver.u2ug.org > [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Charles Stevenson > Sent: Thursday, May 05, 2011 9:21 PM > To: U2 Users List > Subject: Re: [U2] [UV] Marking New Index as Built > > So, Perry, what did you finally decide to do? > ___ > U2-Users mailing list > U2-Users@listserver.u2ug.org > http://listserver.u2ug.org/mailman/listinfo/u2-users > > CONFIDENTIALITY NOTICE: This e-mail message, including any attachments, is > for the sole use of the intended recipient(s) and may contain confidential > and privileged information. Any unauthorized review, use, disclosure or > distribution is prohibited. ZirMed, Inc. has strict policies regarding the > content of e-mail communications, specifically Protected Health Information, > any communications containing such material will be returned to the > originating party with such advisement noted. If you are not the intended > recipient, please contact the sender by reply e-mail and destroy all copies > of the original message. > ___ > U2-Users mailing list > U2-Users@listserver.u2ug.org > http://listserver.u2ug.org/mailman/listinfo/u2-users > > > > ___ > U2-Users mailing list > U2-Users@listserver.u2ug.org > http://listserver.u2ug.org/mailman/listinfo/u2-users ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
Re: [U2] [UV] Marking New Index as Built
Hi Perry One of the new features in UniVerse 11.1 is the concurrent build of indices. I would suggest being very careful in using it as, to my knowledge, it could have an impact on your performance. From the new features Pdf: BUILD.INDEX filename {attribute [attribute...] | ALL } [CONCURRENT] Regards Kurt Neumann; U2 Distributor in Africa -Original Message- From: u2-users-boun...@listserver.u2ug.org [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Perry Taylor Sent: 06 May 2011 02:55 PM To: U2 Users List Subject: Re: [U2] [UV] Marking New Index as Built In the end I took down the application that updates the indexed field and did a BUILD.INDEX at 1am. It took approximately one hour to build but unfortunately another process that accesses that file shutdown in error as the BUILD.INDEX had the FX lock on the file. No harm done but again without a truly online build this will continue to be a messy operation for 7X24 shops. Sure wish Rocket would take this seriously and give us a better tool. Perry -Original Message- From: u2-users-boun...@listserver.u2ug.org [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Charles Stevenson Sent: Thursday, May 05, 2011 9:21 PM To: U2 Users List Subject: Re: [U2] [UV] Marking New Index as Built So, Perry, what did you finally decide to do? ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users CONFIDENTIALITY NOTICE: This e-mail message, including any attachments, is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. ZirMed, Inc. has strict policies regarding the content of e-mail communications, specifically Protected Health Information, any communications containing such material will be returned to the originating party with such advisement noted. If you are not the intended recipient, please contact the sender by reply e-mail and destroy all copies of the original message. ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
Re: [U2] [UV] Marking New Index as Built
In the end I took down the application that updates the indexed field and did a BUILD.INDEX at 1am. It took approximately one hour to build but unfortunately another process that accesses that file shutdown in error as the BUILD.INDEX had the FX lock on the file. No harm done but again without a truly online build this will continue to be a messy operation for 7X24 shops. Sure wish Rocket would take this seriously and give us a better tool. Perry -Original Message- From: u2-users-boun...@listserver.u2ug.org [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Charles Stevenson Sent: Thursday, May 05, 2011 9:21 PM To: U2 Users List Subject: Re: [U2] [UV] Marking New Index as Built So, Perry, what did you finally decide to do? ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users CONFIDENTIALITY NOTICE: This e-mail message, including any attachments, is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. ZirMed, Inc. has strict policies regarding the content of e-mail communications, specifically Protected Health Information, any communications containing such material will be returned to the originating party with such advisement noted. If you are not the intended recipient, please contact the sender by reply e-mail and destroy all copies of the original message. ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
Re: [U2] [UV] Marking New Index as Built
So, Perry, what did you finally decide to do? ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
Re: [U2] [UV] Marking New Index as Built
Rick's right, users with the file already open won't know about the new index. In Perry's case, that's easy to work around, so I didn't think about that. I still think mucking with the right bits or bytes in INDEX.MAP will put the index in the right state to fake out any process that _subsequently_ opens the file so they will update the index going forward. Then you can go backpopulate the index either thru your own program that writes to the INDEX.nnn file directly, or maybe a controlled record-by-record delete & restore of the data records. On 4/28/2011 1:29 PM, Rick Nuckolls wrote: I did some investigation of this in the past, and it is a nightmare. I seem to recall that for a given process to correctly update an index, the index had to have been created and enabled at the time that the file was opened by that user. Otherwise, a user that had the file open prior to the build.index would not update the newly enabled index. The implication is that you want to have everyone off of the system while doing a BUILD.INDEX. Alternately, create the index, build it, then rebuild it after the next restart. Building it will cause it to be "enabled", which ensures that all of the processes treat it that way at the restart. An enabled, partially built index will cause all sorts of anomalous behavior unless you specify "NO.INDEX" in all the right places. I did submit this as a problem, complete with a demo program, to Rocket a while ago. But I have no reason to believe that it has been fixed. If you are really willing to work with a partially built index... Another possible trick, if your index is "NO.NULLS" , and is built on a i-type that calls a subroutine, would be to make the return value of the subroutine blank during the first build. This will cause the BUILD.INDEX process to functionally skip the list creation for all records and enable the index immediately after the select. Once that is done, recatalog the program with the real code, but do not change the calling i-type. Thereafter, the index should be updated, at least for those processes that opened the file after the first, null, BUILD.INDEX. Depending on your OS / hardware configuration, it can sometimes be worthwhile to COUNT the file prior to the first BUILD.INDEX; causing the caching of most of the file in memory. (You can test this by COUNTing the file twice, and seeing if the second one goes appreciably faster than the first.) Indeed, once you get all of the processes updating the index, you could probably write a little program to figure out all of the ids that are indexed, MERGE.LIST INTERSECT this against the id's that should be indexed on the file, and then readu the record, blank out the reference field(s) (saving the correct record) writeu the record to create an "old version" of the record on file write the original record, thus updating the index to the current version. Rick Nuckolls Lynden Inc. On Apr 28, 2011, at 10:38 AM, Perry Taylor wrote: Hey Chuck. The CONCURRENT option is actually there although it's not documented (and I suspect not officially supported). All it really does it not lock the file which doesn't happen until the select is completed anyway. One would have to make sure no updates were being made to the file until the select finishes or the index build may not be complete and accurate or may include entries for deleted records. At this point not taking a file lock on the data file seems less useful as the index file is going to be cleared prior to population and the population doesn't really take all that long in comparison to the select in most cases. In any event seems very messy to me. Traditional relational databases have the option to build/rebuild indexes while the table is in use. In the end it seems to me that Rocket needs to come up with a completely new strategy for building/rebuilding an index. Perry -Original Message- From: u2-users-boun...@listserver.u2ug.org [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Charles Stevenson Sent: Thursday, April 28, 2011 10:53 AM To: U2 Users List Subject: Re: [U2] [UV] Marking New Index as Built That should work. And then you can write your own update program to back-populate the index w/o having to do a build at all. But if your& my connivings are true, then why doesn't RocketSQ give that to us as a concurrent build option? A few years ago they (as IBM) declared that 7x24 uptime was a big goal for U2. Warm regards, Chuck On 4/27/2011 2:24 PM, Perry Taylor wrote: I have a need in UniVerse to add a new index on a large file which has existing indexes. I don't have a time window in the coming days in which to build the index but need to use the index for new records which will be added to the file after the index has been created. Is there a way to force UniVerse to think that
Re: [U2] [UV] Marking New Index as Built
Thinking back to Prime INFORMATION, that was the default behaviour. You could have deferred updates pending and still have new transactions update the indexes. But, then, in Prime INFORMATION BUILD.INDEX and UPDATE.INDEX were two separate commands, unlike the UniVerse implementation. There's probably something you could tweak in the INDEX.MAP file to get this to work. Unfortunately I am in an airport lounge (again!) and not carrying my UniVerse Internals training class notes. = ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
Re: [U2] [UV] Marking New Index as Built
Yup... works on type 30... >CREATE.FILE ITEST 1 1,1,30 Creating file "ITEST" as Type 30. Creating file "D_ITEST" as Type 18, Modulo 1, Separation 1. Added "@ID", the default record for RetrieVe, to "D_ITEST". >CREATE.INDEX ITEST @ID >LIST.INDEX ITEST ALL Alternate Key Index Summary for file ITEST File... ITEST Indices 1 (0 A-type, 0 C-type, 1 D-type, 0 I-type, 0 SQL, 0 S-type) Index Updates.. Enabled, No updates pending Index name Type BuildNulls In DICT S/M Just Unique Field num/I-type @ID DNot Reqd YesYes SL N0 Perry -Original Message- From: u2-users-boun...@listserver.u2ug.org [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Wols Lists Sent: Friday, April 29, 2011 9:39 AM To: u2-users@listserver.u2ug.org Subject: Re: [U2] [UV] Marking New Index as Built On 29/04/11 02:37, Perry Taylor wrote: > This is on a fairly old release 10.1.21... :-) I'm not sure what the latest version I really played with on this score was. Most of my experience was 9.6 and 10.0. The other thing is, I've normally only ever used type 30 files ... So does it work on 18s but not 30s, or has it been fixed, or or or ...? Cheers, Wol > >> CREATE.FILE ITEST 1 1 > Creating file "ITEST" as Type 18, Modulo 1, Separation 1. > Creating file "D_ITEST" as Type 18, Modulo 1, Separation 1. > Added "@ID", the default record for RetrieVe, to "D_ITEST". >> CREATE.INDEX ITEST @ID >> LIST.INDEX ITEST ALL > Alternate Key Index Summary for file ITEST > File... ITEST > Indices 1 (0 A-type, 0 C-type, 1 D-type, 0 I-type, 0 SQL, 0 > S-type) > Index Updates.. Enabled, No updates pending > > Index name Type BuildNulls In DICT S/M Just Unique Field > num/I-type > @ID DNot Reqd YesYes SL N0 > > > However if there are any records in the file... > >> DELETE.INDEX ITEST @ID > Removing index file @ID. >> COPY VOC RELLEVEL > TO:(ITEST > 1 RELLEVEL copied to RELLEVEL in file "ITEST". > > > 1 record copied. >> CREATE.INDEX ITEST @ID >> LIST.INDEX ITEST ALL > Alternate Key Index Summary for file ITEST > File... ITEST > Indices 1 (0 A-type, 0 C-type, 1 D-type, 0 I-type, 0 SQL, 0 > S-type) > Index Updates.. Enabled, No updates pending > > Index name Type BuildNulls In DICT S/M Just Unique Field > num/I-type > @ID DRequired YesYes SL N0 > > Perry > -----Original Message- > From: u2-users-boun...@listserver.u2ug.org > [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Wols Lists > Sent: Thursday, April 28, 2011 5:24 PM > To: u2-users@listserver.u2ug.org > Subject: Re: [U2] [UV] Marking New Index as Built > > On 29/04/11 00:08, Perry Taylor wrote: >> Actually I think UV does this. If I create an index on an empty file > the index does not have to be built. >> > Are you sure? Or have they changed it recently? > > I went from Prime to UV, and that was one of the things I noticed very > quickly. To the best of my knowledge it was never fixed. Anyways, I've > always built an index on UV after creating it, even with an empty file, > and always because UV complained and demanded I do it. > >> Perry > > Cheers, > Wol > ___ > U2-Users mailing list > U2-Users@listserver.u2ug.org > http://listserver.u2ug.org/mailman/listinfo/u2-users > > CONFIDENTIALITY NOTICE: This e-mail message, including any > attachments, is for the sole use of the intended recipient(s) > and may contain confidential and privileged information. Any > unauthorized review, use, disclosure or distribution is > prohibited. ZirMed, Inc. has strict policies regarding the > content of e-mail communications, specifically Protected Health > Information, any communications containing such material will > be returned to the originating party with such advisement > noted. If you are not the intended recipient, please contact > the sender by reply e-mail and destroy all copies of the > original message. > ___ > U2-Users mailing list > U2-Users@listserver.u2ug.org > http://listserver.u2ug.org/mailman/listinfo/u2-users > ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
Re: [U2] [UV] Marking New Index as Built
On 29/04/11 02:37, Perry Taylor wrote: > This is on a fairly old release 10.1.21... :-) I'm not sure what the latest version I really played with on this score was. Most of my experience was 9.6 and 10.0. The other thing is, I've normally only ever used type 30 files ... So does it work on 18s but not 30s, or has it been fixed, or or or ...? Cheers, Wol > >> CREATE.FILE ITEST 1 1 > Creating file "ITEST" as Type 18, Modulo 1, Separation 1. > Creating file "D_ITEST" as Type 18, Modulo 1, Separation 1. > Added "@ID", the default record for RetrieVe, to "D_ITEST". >> CREATE.INDEX ITEST @ID >> LIST.INDEX ITEST ALL > Alternate Key Index Summary for file ITEST > File... ITEST > Indices 1 (0 A-type, 0 C-type, 1 D-type, 0 I-type, 0 SQL, 0 > S-type) > Index Updates.. Enabled, No updates pending > > Index name Type BuildNulls In DICT S/M Just Unique Field > num/I-type > @ID DNot Reqd YesYes SL N0 > > > However if there are any records in the file... > >> DELETE.INDEX ITEST @ID > Removing index file @ID. >> COPY VOC RELLEVEL > TO:(ITEST > 1 RELLEVEL copied to RELLEVEL in file "ITEST". > > > 1 record copied. >> CREATE.INDEX ITEST @ID >> LIST.INDEX ITEST ALL > Alternate Key Index Summary for file ITEST > File... ITEST > Indices 1 (0 A-type, 0 C-type, 1 D-type, 0 I-type, 0 SQL, 0 > S-type) > Index Updates.. Enabled, No updates pending > > Index name Type BuildNulls In DICT S/M Just Unique Field > num/I-type > @ID DRequired YesYes SL N0 > > Perry > -Original Message- > From: u2-users-boun...@listserver.u2ug.org > [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Wols Lists > Sent: Thursday, April 28, 2011 5:24 PM > To: u2-users@listserver.u2ug.org > Subject: Re: [U2] [UV] Marking New Index as Built > > On 29/04/11 00:08, Perry Taylor wrote: >> Actually I think UV does this. If I create an index on an empty file > the index does not have to be built. >> > Are you sure? Or have they changed it recently? > > I went from Prime to UV, and that was one of the things I noticed very > quickly. To the best of my knowledge it was never fixed. Anyways, I've > always built an index on UV after creating it, even with an empty file, > and always because UV complained and demanded I do it. > >> Perry > > Cheers, > Wol > ___ > U2-Users mailing list > U2-Users@listserver.u2ug.org > http://listserver.u2ug.org/mailman/listinfo/u2-users > > CONFIDENTIALITY NOTICE: This e-mail message, including any > attachments, is for the sole use of the intended recipient(s) > and may contain confidential and privileged information. Any > unauthorized review, use, disclosure or distribution is > prohibited. ZirMed, Inc. has strict policies regarding the > content of e-mail communications, specifically Protected Health > Information, any communications containing such material will > be returned to the originating party with such advisement > noted. If you are not the intended recipient, please contact > the sender by reply e-mail and destroy all copies of the > original message. > ___ > U2-Users mailing list > U2-Users@listserver.u2ug.org > http://listserver.u2ug.org/mailman/listinfo/u2-users > ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
Re: [U2] [UV] Marking New Index as Built
This is on a fairly old release 10.1.21... >CREATE.FILE ITEST 1 1 Creating file "ITEST" as Type 18, Modulo 1, Separation 1. Creating file "D_ITEST" as Type 18, Modulo 1, Separation 1. Added "@ID", the default record for RetrieVe, to "D_ITEST". >CREATE.INDEX ITEST @ID >LIST.INDEX ITEST ALL Alternate Key Index Summary for file ITEST File... ITEST Indices 1 (0 A-type, 0 C-type, 1 D-type, 0 I-type, 0 SQL, 0 S-type) Index Updates.. Enabled, No updates pending Index name Type BuildNulls In DICT S/M Just Unique Field num/I-type @ID DNot Reqd YesYes SL N0 However if there are any records in the file... >DELETE.INDEX ITEST @ID Removing index file @ID. >COPY VOC RELLEVEL TO:(ITEST 1 RELLEVEL copied to RELLEVEL in file "ITEST". 1 record copied. >CREATE.INDEX ITEST @ID >LIST.INDEX ITEST ALL Alternate Key Index Summary for file ITEST File... ITEST Indices 1 (0 A-type, 0 C-type, 1 D-type, 0 I-type, 0 SQL, 0 S-type) Index Updates.. Enabled, No updates pending Index name Type BuildNulls In DICT S/M Just Unique Field num/I-type @ID DRequired YesYes SL N0 Perry -Original Message- From: u2-users-boun...@listserver.u2ug.org [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Wols Lists Sent: Thursday, April 28, 2011 5:24 PM To: u2-users@listserver.u2ug.org Subject: Re: [U2] [UV] Marking New Index as Built On 29/04/11 00:08, Perry Taylor wrote: > Actually I think UV does this. If I create an index on an empty file the index does not have to be built. > Are you sure? Or have they changed it recently? I went from Prime to UV, and that was one of the things I noticed very quickly. To the best of my knowledge it was never fixed. Anyways, I've always built an index on UV after creating it, even with an empty file, and always because UV complained and demanded I do it. > Perry Cheers, Wol ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users CONFIDENTIALITY NOTICE: This e-mail message, including any attachments, is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. ZirMed, Inc. has strict policies regarding the content of e-mail communications, specifically Protected Health Information, any communications containing such material will be returned to the originating party with such advisement noted. If you are not the intended recipient, please contact the sender by reply e-mail and destroy all copies of the original message. ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
Re: [U2] [UV] Marking New Index as Built
On 29/04/11 00:08, Perry Taylor wrote: > Actually I think UV does this. If I create an index on an empty file the > index does not have to be built. > Are you sure? Or have they changed it recently? I went from Prime to UV, and that was one of the things I noticed very quickly. To the best of my knowledge it was never fixed. Anyways, I've always built an index on UV after creating it, even with an empty file, and always because UV complained and demanded I do it. > Perry Cheers, Wol ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
Re: [U2] [UV] Marking New Index as Built
Actually I think UV does this. If I create an index on an empty file the index does not have to be built. Perry - Original Message - From: Wols Lists [mailto:antli...@youngman.org.uk] Sent: Thursday, April 28, 2011 05:57 PM To: u2-users@listserver.u2ug.org Subject: Re: [U2] [UV] Marking New Index as Built On 28/04/11 17:53, Charles Stevenson wrote: > That should work. > And then you can write your own update program to back-populate the > index w/o having to do a build at all. > > But if your & my connivings are true, then why doesn't RocketSQ give > that to us as a concurrent build option? A few years ago they (as IBM) > declared that 7x24 uptime was a big goal for U2. > Another thing they should do, is what Prime did long ago. INFORMATION actually checked whether there were any records in the file! It's a right pain to have to rebuild all your new indices when you create a file, add the indices, and then add the data, in that order. INFORMATION just enabled the index and marked it "up to date" on creation if there were no records in the file. Cheers, Wol ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users CONFIDENTIALITY NOTICE: This e-mail message, including any attachments, is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. ZirMed, Inc. has strict policies regarding the content of e-mail communications, specifically Protected Health Information, any communications containing such material will be returned to the originating party with such advisement noted. If you are not the intended recipient, please contact the sender by reply e-mail and destroy all copies of the original message. ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
Re: [U2] [UV] Marking New Index as Built
On 28/04/11 17:53, Charles Stevenson wrote: > That should work. > And then you can write your own update program to back-populate the > index w/o having to do a build at all. > > But if your & my connivings are true, then why doesn't RocketSQ give > that to us as a concurrent build option? A few years ago they (as IBM) > declared that 7x24 uptime was a big goal for U2. > Another thing they should do, is what Prime did long ago. INFORMATION actually checked whether there were any records in the file! It's a right pain to have to rebuild all your new indices when you create a file, add the indices, and then add the data, in that order. INFORMATION just enabled the index and marked it "up to date" on creation if there were no records in the file. Cheers, Wol ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
Re: [U2] [UV] Marking New Index as Built
I did some investigation of this in the past, and it is a nightmare. I seem to recall that for a given process to correctly update an index, the index had to have been created and enabled at the time that the file was opened by that user. Otherwise, a user that had the file open prior to the build.index would not update the newly enabled index. The implication is that you want to have everyone off of the system while doing a BUILD.INDEX. Alternately, create the index, build it, then rebuild it after the next restart. Building it will cause it to be "enabled", which ensures that all of the processes treat it that way at the restart. An enabled, partially built index will cause all sorts of anomalous behavior unless you specify "NO.INDEX" in all the right places. I did submit this as a problem, complete with a demo program, to Rocket a while ago. But I have no reason to believe that it has been fixed. If you are really willing to work with a partially built index... Another possible trick, if your index is "NO.NULLS" , and is built on a i-type that calls a subroutine, would be to make the return value of the subroutine blank during the first build. This will cause the BUILD.INDEX process to functionally skip the list creation for all records and enable the index immediately after the select. Once that is done, recatalog the program with the real code, but do not change the calling i-type. Thereafter, the index should be updated, at least for those processes that opened the file after the first, null, BUILD.INDEX. Depending on your OS / hardware configuration, it can sometimes be worthwhile to COUNT the file prior to the first BUILD.INDEX; causing the caching of most of the file in memory. (You can test this by COUNTing the file twice, and seeing if the second one goes appreciably faster than the first.) Indeed, once you get all of the processes updating the index, you could probably write a little program to figure out all of the ids that are indexed, MERGE.LIST INTERSECT this against the id's that should be indexed on the file, and then readu the record, blank out the reference field(s) (saving the correct record) writeu the record to create an "old version" of the record on file write the original record, thus updating the index to the current version. Rick Nuckolls Lynden Inc. On Apr 28, 2011, at 10:38 AM, Perry Taylor wrote: > Hey Chuck. > > The CONCURRENT option is actually there although it's not documented > (and I suspect not officially supported). All it really does it not > lock the file which doesn't happen until the select is completed anyway. > One would have to make sure no updates were being made to the file until > the select finishes or the index build may not be complete and accurate > or may include entries for deleted records. At this point not taking a > file lock on the data file seems less useful as the index file is going > to be cleared prior to population and the population doesn't really take > all that long in comparison to the select in most cases. In any event > seems very messy to me. > > Traditional relational databases have the option to build/rebuild > indexes while the table is in use. In the end it seems to me that > Rocket needs to come up with a completely new strategy for > building/rebuilding an index. > > Perry > > -Original Message- > From: u2-users-boun...@listserver.u2ug.org > [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Charles > Stevenson > Sent: Thursday, April 28, 2011 10:53 AM > To: U2 Users List > Subject: Re: [U2] [UV] Marking New Index as Built > > That should work. > And then you can write your own update program to back-populate the > index w/o having to do a build at all. > > But if your & my connivings are true, then why doesn't RocketSQ give > that to us as a concurrent build option? A few years ago they (as IBM) > > declared that 7x24 uptime was a big goal for U2. > > Warm regards, > Chuck > > On 4/27/2011 2:24 PM, Perry Taylor wrote: >> I have a need in UniVerse to add a new index on a large file which has >> existing indexes. I don't have a time window in the coming days in >> which to build the index but need to use the index for new records > which >> will be added to the file after the index has been created. Is there > a >> way to force UniVerse to think that the index has been built so that > it >> can be used for newer records allowing me to defer the build to a > later >> date? >> >> Thanks. >> Perry Taylor >> Zirmed, Inc. >> >> >> >> >> CONFIDENTIALITY NOTICE: This e-mail message, including any >> attachments, is for the sole use of th
Re: [U2] [UV] Marking New Index as Built
Hey Chuck. The CONCURRENT option is actually there although it's not documented (and I suspect not officially supported). All it really does it not lock the file which doesn't happen until the select is completed anyway. One would have to make sure no updates were being made to the file until the select finishes or the index build may not be complete and accurate or may include entries for deleted records. At this point not taking a file lock on the data file seems less useful as the index file is going to be cleared prior to population and the population doesn't really take all that long in comparison to the select in most cases. In any event seems very messy to me. Traditional relational databases have the option to build/rebuild indexes while the table is in use. In the end it seems to me that Rocket needs to come up with a completely new strategy for building/rebuilding an index. Perry -Original Message- From: u2-users-boun...@listserver.u2ug.org [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Charles Stevenson Sent: Thursday, April 28, 2011 10:53 AM To: U2 Users List Subject: Re: [U2] [UV] Marking New Index as Built That should work. And then you can write your own update program to back-populate the index w/o having to do a build at all. But if your & my connivings are true, then why doesn't RocketSQ give that to us as a concurrent build option? A few years ago they (as IBM) declared that 7x24 uptime was a big goal for U2. Warm regards, Chuck On 4/27/2011 2:24 PM, Perry Taylor wrote: > I have a need in UniVerse to add a new index on a large file which has > existing indexes. I don't have a time window in the coming days in > which to build the index but need to use the index for new records which > will be added to the file after the index has been created. Is there a > way to force UniVerse to think that the index has been built so that it > can be used for newer records allowing me to defer the build to a later > date? > > Thanks. > Perry Taylor > Zirmed, Inc. > > > > > CONFIDENTIALITY NOTICE: This e-mail message, including any > attachments, is for the sole use of the intended recipient(s) > and may contain confidential and privileged information. Any > unauthorized review, use, disclosure or distribution is > prohibited. ZirMed, Inc. has strict policies regarding the > content of e-mail communications, specifically Protected Health > Information, any communications containing such material will > be returned to the originating party with such advisement > noted. If you are not the intended recipient, please contact > the sender by reply e-mail and destroy all copies of the > original message. > ___ > U2-Users mailing list > U2-Users@listserver.u2ug.org > http://listserver.u2ug.org/mailman/listinfo/u2-users > ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
Re: [U2] [UV] Marking New Index as Built
That should work. And then you can write your own update program to back-populate the index w/o having to do a build at all. But if your & my connivings are true, then why doesn't RocketSQ give that to us as a concurrent build option? A few years ago they (as IBM) declared that 7x24 uptime was a big goal for U2. Warm regards, Chuck On 4/27/2011 2:24 PM, Perry Taylor wrote: I have a need in UniVerse to add a new index on a large file which has existing indexes. I don't have a time window in the coming days in which to build the index but need to use the index for new records which will be added to the file after the index has been created. Is there a way to force UniVerse to think that the index has been built so that it can be used for newer records allowing me to defer the build to a later date? Thanks. Perry Taylor Zirmed, Inc. CONFIDENTIALITY NOTICE: This e-mail message, including any attachments, is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. ZirMed, Inc. has strict policies regarding the content of e-mail communications, specifically Protected Health Information, any communications containing such material will be returned to the originating party with such advisement noted. If you are not the intended recipient, please contact the sender by reply e-mail and destroy all copies of the original message. ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
Re: [U2] [UV] Marking New Index as Built
I think I have found an "unsupported" way to doing this. It appears there is a flag in the INDEX.MAP file that controls this and flipping it does the trick. I'm hoping there is an "official" way to do this so I don't have any surprises when I muck with the internals. Any ideas? Thanks. Perry -Original Message- From: u2-users-boun...@listserver.u2ug.org [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Perry Taylor Sent: Wednesday, April 27, 2011 1:24 PM To: U2-Users List Subject: [U2] [UV] Marking New Index as Built I have a need in UniVerse to add a new index on a large file which has existing indexes. I don't have a time window in the coming days in which to build the index but need to use the index for new records which will be added to the file after the index has been created. Is there a way to force UniVerse to think that the index has been built so that it can be used for newer records allowing me to defer the build to a later date? Thanks. Perry Taylor Zirmed, Inc. CONFIDENTIALITY NOTICE: This e-mail message, including any attachments, is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. ZirMed, Inc. has strict policies regarding the content of e-mail communications, specifically Protected Health Information, any communications containing such material will be returned to the originating party with such advisement noted. If you are not the intended recipient, please contact the sender by reply e-mail and destroy all copies of the original message. ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
[U2] [UV] Marking New Index as Built
I have a need in UniVerse to add a new index on a large file which has existing indexes. I don't have a time window in the coming days in which to build the index but need to use the index for new records which will be added to the file after the index has been created. Is there a way to force UniVerse to think that the index has been built so that it can be used for newer records allowing me to defer the build to a later date? Thanks. Perry Taylor Zirmed, Inc. CONFIDENTIALITY NOTICE: This e-mail message, including any attachments, is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. ZirMed, Inc. has strict policies regarding the content of e-mail communications, specifically Protected Health Information, any communications containing such material will be returned to the originating party with such advisement noted. If you are not the intended recipient, please contact the sender by reply e-mail and destroy all copies of the original message. ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users