Re: [Fedora-directory-users] Migration from i-planet 52

2006-12-18 Thread David Boreham 

Eddie C wrote:


I ran
DB ERROR: db_verify: Page 30: out-of-order key at entry 498
DB ERROR: db_verify: DB->verify: db/o_com/channelcontentowner.db4: 
DB_VERIFY_BAD: Database verification failed


I'm assuming that you are running the correct version of db_verify (it 
should

perform a version check on the magic number in the region files, so I think
it has to be the right one).

My best guess here is that the process by which a new file is added to
a running DB environment went wrong somehow. The details have changed
a few times over the years and it is just possible that the current FDS
code is not up to date. Potentially the 'problem' would be fixed by
simply stopping and re-starting the server (because the environment
would see all the files closed and then re-opened). I do know that the
process for adding a new file to the environment is correctly followed
in the newer code that deals with individual back-end restore from archive.
Perhaps the online index code is not using the same underlying function.
I haven't taken the time to examine the code though.

The problem I'm thinking about arises when a file that was created
using one db environment (a temporary one, such as is used when building
an index), is opened within another environment, 'bad stuff' can happen
along the lines of what you are seeing. There is state in the file that
references the old environment, which is now stale vs. the new one.
db_verify sees that and barfs. It is possible to avoid this happening
but one has to be careful to make the right calls in the correct order
(the details of which I forget now).

As Rich and Noriko said, if you just re-import after creating the indices
this issue (if it is indeed the one I'm thinking of) will be avoided because
all the files are created at the same time under the same db environment.










--
Fedora-directory-users mailing list
Fedora-directory-users@redhat.com
https://www.redhat.com/mailman/listinfo/fedora-directory-users

Re: [Fedora-directory-users] Migration from i-planet 52

2006-12-18 Thread Richard Megginson 

Eddie C wrote:
The document I had suggested using ldapsearch and ldapadd to migrate 
data. If lidf2db commands are faster/better I will use them.


>> Try creating all of the required indexes first, then doing the 
import of

>> your original LDIF.

I am willing to try this, but It is scary to me. I would have rather 
you said I must be doing something wrong...because


Our LDAP database has been in production for 6 years. We add indexes 
to our i-planet on average twice a year due to new software or new 
features. Your advice is almost suggesting that adding new indexes can 
corrupt the database.
No, not exactly.  I'm not really sure what went wrong.  Feel free to try 
it again.  It's just that for the initial data import, it's much faster 
to configure the indexes first, then use ldif2db to import the data and 
create the indexes at the same time.


I will try again from scratch using everyones advice of course.

Thank you,
Edward





On 12/16/06, *Noriko Hosoi* < [EMAIL PROTECTED] 
> wrote:


Richard Megginson wrote:

> Eddie C wrote:
>
>> I recently did an ldif backup of our iplanet 52 database. Its about
>> an 88 MB ldif file.
>> I took this to a new FDS server Dell 850 3 ghz duel core 2 sata
hard
>> disks.
>> I ran an ldapadd  the data imported perfectly.
>
Are there any reason to use ldapadd instead of ldif2db?  ldif2db
should
be much faster...

>> Then I tried to cutover some systems and give the database some
load.
>>
>> System went 200% processor
>>
>> Eventually I realized I was missing indexes so I added them through
>> the graphical tool.
>>
>> The log seemed to do something like this
>> generating index 1%
>> generating index 2%
>> 
>> generating index 49%
>> Done
>> Seemed weird that they would jump from 49% to Done
>> At this point the new system was running at 100% processor
>> But the queries are running faster on our old 440 MHZ sparc t1
>> server52 database
>>
>> I ran
>> DB ERROR: db_verify: Page 30: out-of-order key at entry 498
>> DB ERROR: db_verify: DB->verify: db/o_com/channelcontentowner.db4:
>> DB_VERIFY_BAD: Database verification failed
>>
>> then I tried db2_index. The program seemed to be in a tight loop
>> complaining about 1 missing entry.
>>
>> I do not realize how the data can be so corrupted right after
an import.
>>
>> These are someone generic symptoms. Any ideas? Thanks
>
> Try creating all of the required indexes first, then doing the
import
> of your original LDIF.  Not only will the import+index creation be
> much faster (than doing the import then creating the indexes one
at a
> time), but I think your database corruption problems will vanish.
>
>>


>>
>> --
>> Fedora-directory-users mailing list
>> Fedora-directory-users@redhat.com

>> https://www.redhat.com/mailman/listinfo/fedora-directory-users
>>
>
>
>
>--
>Fedora-directory-users mailing list
>Fedora-directory-users@redhat.com

>https://www.redhat.com/mailman/listinfo/fedora-directory-users

>
>



--
Fedora-directory-users mailing list
Fedora-directory-users@redhat.com

https://www.redhat.com/mailman/listinfo/fedora-directory-users






--
Fedora-directory-users mailing list
Fedora-directory-users@redhat.com
https://www.redhat.com/mailman/listinfo/fedora-directory-users
  


smime.p7s
Description: S/MIME Cryptographic Signature
--
Fedora-directory-users mailing list
Fedora-directory-users@redhat.com
https://www.redhat.com/mailman/listinfo/fedora-directory-users

Re: [Fedora-directory-users] Migration from i-planet 52

2006-12-17 Thread Eddie C 

The document I had suggested using ldapsearch and ldapadd to migrate data.
If lidf2db commands are faster/better I will use them.


Try creating all of the required indexes first, then doing the import of
your original LDIF.


I am willing to try this, but It is scary to me. I would have rather you
said I must be doing something wrong...because

Our LDAP database has been in production for 6 years. We add indexes to our
i-planet on average twice a year due to new software or new features. Your
advice is almost suggesting that adding new indexes can corrupt the
database.

I will try again from scratch using everyones advice of course.

Thank you,
Edward





On 12/16/06, Noriko Hosoi <[EMAIL PROTECTED]> wrote:


Richard Megginson wrote:

> Eddie C wrote:
>
>> I recently did an ldif backup of our iplanet 52 database. Its about
>> an 88 MB ldif file.
>> I took this to a new FDS server Dell 850 3 ghz duel core 2 sata hard
>> disks.
>> I ran an ldapadd  the data imported perfectly.
>
Are there any reason to use ldapadd instead of ldif2db?  ldif2db should
be much faster...

>> Then I tried to cutover some systems and give the database some load.
>>
>> System went 200% processor
>>
>> Eventually I realized I was missing indexes so I added them through
>> the graphical tool.
>>
>> The log seemed to do something like this
>> generating index 1%
>> generating index 2%
>> 
>> generating index 49%
>> Done
>> Seemed weird that they would jump from 49% to Done
>> At this point the new system was running at 100% processor
>> But the queries are running faster on our old 440 MHZ sparc t1
>> server52 database
>>
>> I ran
>> DB ERROR: db_verify: Page 30: out-of-order key at entry 498
>> DB ERROR: db_verify: DB->verify: db/o_com/channelcontentowner.db4:
>> DB_VERIFY_BAD: Database verification failed
>>
>> then I tried db2_index. The program seemed to be in a tight loop
>> complaining about 1 missing entry.
>>
>> I do not realize how the data can be so corrupted right after an
import.
>>
>> These are someone generic symptoms. Any ideas? Thanks
>
> Try creating all of the required indexes first, then doing the import
> of your original LDIF.  Not only will the import+index creation be
> much faster (than doing the import then creating the indexes one at a
> time), but I think your database corruption problems will vanish.
>
>>

>>
>> --
>> Fedora-directory-users mailing list
>> Fedora-directory-users@redhat.com
>> https://www.redhat.com/mailman/listinfo/fedora-directory-users
>>
>
>
>
>--
>Fedora-directory-users mailing list
>Fedora-directory-users@redhat.com
>https://www.redhat.com/mailman/listinfo/fedora-directory-users
>
>



--
Fedora-directory-users mailing list
Fedora-directory-users@redhat.com
https://www.redhat.com/mailman/listinfo/fedora-directory-users




--
Fedora-directory-users mailing list
Fedora-directory-users@redhat.com
https://www.redhat.com/mailman/listinfo/fedora-directory-users

Re: [Fedora-directory-users] Migration from i-planet 52

2006-12-16 Thread Noriko Hosoi 

Richard Megginson wrote:


Eddie C wrote:

I recently did an ldif backup of our iplanet 52 database. Its about 
an 88 MB ldif file.
I took this to a new FDS server Dell 850 3 ghz duel core 2 sata hard 
disks.

I ran an ldapadd  the data imported perfectly.


Are there any reason to use ldapadd instead of ldif2db?  ldif2db should 
be much faster...



Then I tried to cutover some systems and give the database some load.
 
System went 200% processor
 
Eventually I realized I was missing indexes so I added them through 
the graphical tool.
 
The log seemed to do something like this

generating index 1%
generating index 2%

generating index 49%
Done
Seemed weird that they would jump from 49% to Done
At this point the new system was running at 100% processor
But the queries are running faster on our old 440 MHZ sparc t1 
server52 database
 
I ran

DB ERROR: db_verify: Page 30: out-of-order key at entry 498
DB ERROR: db_verify: DB->verify: db/o_com/channelcontentowner.db4: 
DB_VERIFY_BAD: Database verification failed
 
then I tried db2_index. The program seemed to be in a tight loop 
complaining about 1 missing entry.
 
I do not realize how the data can be so corrupted right after an import.
 
These are someone generic symptoms. Any ideas? Thanks


Try creating all of the required indexes first, then doing the import 
of your original LDIF.  Not only will the import+index creation be 
much faster (than doing the import then creating the indexes one at a 
time), but I think your database corruption problems will vanish.





--
Fedora-directory-users mailing list
Fedora-directory-users@redhat.com
https://www.redhat.com/mailman/listinfo/fedora-directory-users
  




--
Fedora-directory-users mailing list
Fedora-directory-users@redhat.com
https://www.redhat.com/mailman/listinfo/fedora-directory-users
 





smime.p7s
Description: S/MIME Cryptographic Signature
--
Fedora-directory-users mailing list
Fedora-directory-users@redhat.com
https://www.redhat.com/mailman/listinfo/fedora-directory-users

Re: [Fedora-directory-users] Migration from i-planet 52

2006-12-15 Thread Richard Megginson 

Eddie C wrote:
I recently did an ldif backup of our iplanet 52 database. Its about an 
88 MB ldif file.
I took this to a new FDS server Dell 850 3 ghz duel core 2 sata hard 
disks.

I ran an ldapadd  the data imported perfectly.
Then I tried to cutover some systems and give the database some load.
 
System went 200% processor
 
Eventually I realized I was missing indexes so I added them through 
the graphical tool.
 
The log seemed to do something like this

generating index 1%
generating index 2%

generating index 49%
Done
Seemed weird that they would jump from 49% to Done
At this point the new system was running at 100% processor
But the queries are running faster on our old 440 MHZ sparc t1 
server52 database
 
I ran

DB ERROR: db_verify: Page 30: out-of-order key at entry 498
DB ERROR: db_verify: DB->verify: db/o_com/channelcontentowner.db4: 
DB_VERIFY_BAD: Database verification failed
 
then I tried db2_index. The program seemed to be in a tight loop 
complaining about 1 missing entry.
 
I do not realize how the data can be so corrupted right after an import.
 
These are someone generic symptoms. Any ideas? Thanks
Try creating all of the required indexes first, then doing the import of 
your original LDIF.  Not only will the import+index creation be much 
faster (than doing the import then creating the indexes one at a time), 
but I think your database corruption problems will vanish.



--
Fedora-directory-users mailing list
Fedora-directory-users@redhat.com
https://www.redhat.com/mailman/listinfo/fedora-directory-users
  


smime.p7s
Description: S/MIME Cryptographic Signature
--
Fedora-directory-users mailing list
Fedora-directory-users@redhat.com
https://www.redhat.com/mailman/listinfo/fedora-directory-users

[Fedora-directory-users] Migration from i-planet 52

2006-12-15 Thread Eddie C 

I recently did an ldif backup of our iplanet 52 database. Its about an 88 MB
ldif file.
I took this to a new FDS server Dell 850 3 ghz duel core 2 sata hard disks.
I ran an ldapadd  the data imported perfectly.
Then I tried to cutover some systems and give the database some load.

System went 200% processor

Eventually I realized I was missing indexes so I added them through the
graphical tool.

The log seemed to do something like this
generating index 1%
generating index 2%

generating index 49%
Done
Seemed weird that they would jump from 49% to Done
At this point the new system was running at 100% processor
But the queries are running faster on our old 440 MHZ sparc t1 server52
database

I ran
DB ERROR: db_verify: Page 30: out-of-order key at entry 498
DB ERROR: db_verify: DB->verify: db/o_com/channelcontentowner.db4:
DB_VERIFY_BAD: Database verification failed

then I tried db2_index. The program seemed to be in a tight loop complaining
about 1 missing entry.

I do not realize how the data can be so corrupted right after an import.

These are someone generic symptoms. Any ideas? Thanks
--
Fedora-directory-users mailing list
Fedora-directory-users@redhat.com
https://www.redhat.com/mailman/listinfo/fedora-directory-users