Re: [Freeipa-users] backup/restore IPA servers with db2ldap.pl, ldap2db.pl ???
Please feel free to do it. Thanks. --David From: Dmitri Pal To: Rich Megginson Cc: David Copperfield ; Rob Crittenden ; E Deon Lackey ; "freeipa-users@redhat.com" Sent: Friday, May 11, 2012 2:53 PM Subject: Re: [Freeipa-users] backup/restore IPA servers with db2ldap.pl, ldap2db.pl ??? On 05/10/2012 10:54 PM, Rich Megginson wrote: On 05/10/2012 07:54 PM, David Copperfield wrote: >OK, >> >> >> that means the steps below: >> >> >>1) on IPA replica, lets create 4 IPA users: A,B,C and D. Now make a backup >>with 'db2ldif.pl -r ...' >> >> >>2) on IPA replica, delete the user D. 'ipa user-del D'. >> >> >>3, on IPA master, delete the user C. 'ipa user-del C'. >> >> >>4, now check on other IPA master and IPA replica, both shows only two users >>'A' and 'B'. this is expected. >> >> >>5, now on IPA replica, restore the backup with 'ldif2db.pl' >> >> >>6, check on IPA replica immediately, 'ipa user-find' shows 4 users 'A, B, C, >>D' at the beginning. >> >> >>7, check IPA Master, 'ipa user-find' shows still only two users 'A, B'. >> >> >>8, wait 3 minutes or so, check on IPA replica, and found that there are only >>THREE users 'A, B, D'. The users 'C' is deleted now -- change propagated from >>IPA Master. >> >> >>9, check on IPA Master again and again, there are still only two users 'A, B'. >> >> >>10, check on IPA Replica again and again, there are still three users 'A, >>B,D'. --- this status is different from IPA Master's 'A,B', or backup's 'A, >>B, C, D'. >> >> >> >> >>If backup was created without '-r' option, then the step 8 above will always >>show 'A,B,C,D', the same as backup. with '-r' option make the final result >>between. >> >> >> >> >>Hope I have explained it clearly. Please advice something like ipa2ldif.pl >>and ldif2ipa.pl tools. There are really the key useful feature for serious >>production IPA deployment, which is definitely of much higher priority than >>dogtag. >Sounds like a bug. What should happen is that the deletion of C and D should be propagated to replica. > Was a bug or a ticket filed? > > >> >>Thanks a lot. >> >> >>--David >> >> >> >> >> >> >> >> >> From: Rich Megginson >>To: David Copperfield >>Cc: E Deon Lackey ; Petr Spacek ; Rob >>Crittenden ; "freeipa-users@redhat.com" >> >>Sent: Thursday, May 10, 2012 6:37 PM >>Subject: Re: [Freeipa-users] backup/restore IPA servers with db2ldap.pl, >>ldap2db.pl ??? >> >> >>On 05/10/2012 07:32 PM, David Copperfield wrote: >>Hi Rich and all, >>> >>> >>>the '-r' option to db2ldif.pl doesn't work neither, it make few difference. >>> >>> >>>My command, backup and restore commands on the IPA replica are: >>> >>> >>>db2ldif.pl -D 'cn=Directory Manager' -w - -r -s 'dc=example,dc=com' >>> >>> >>>ldif2db.pl -D 'cn=Directory Manager' -w - -i >>> >>> >>>The only difference is: after IPA master restart (restart happens after IPA >>>replica's restore operation), the changes -- which applied on IPA master >>>before backup -- are propagated to IPA replica. Which is in fact, make the >>>restoration test end up with a result completely unusable on IPA replica, an >>>result that is different from backup, and different from IPA master. >>> >>I don't quite understand what you mean. >> >> >> >>> >>>Please let me know if there are any other options/steps to follow. Thanks. >>Not sure what else to try. >> >> >> >>> >>>--David >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> From: Rich Megginson >>>To: David Copperfield >>>Cc: "freeipa-users@redhat.com" ; Rob Crittenden >>>; Petr Spacek >>>Sent: Thursday, May 10, 2012 5:28 PM >>>Subject: Re: [Freeipa-users] backup/restore IPA servers with db2ldap.pl, >>>ldap2db.pl ??? >>> >&
Re: [Freeipa-users] backup/restore IPA servers with db2ldap.pl, ldap2db.pl ???
On 05/10/2012 10:54 PM, Rich Megginson wrote: > On 05/10/2012 07:54 PM, David Copperfield wrote: >> OK, >> >> that means the steps below: >> >> 1) on IPA replica, lets create 4 IPA users: A,B,C and D. Now make a >> backup with 'db2ldif.pl -r ...' >> >> 2) on IPA replica, delete the user D. 'ipa user-del D'. >> >> 3, on IPA master, delete the user C. 'ipa user-del C'. >> >> 4, now check on other IPA master and IPA replica, both shows only two >> users 'A' and 'B'. this is expected. >> >> 5, now on IPA replica, restore the backup with 'ldif2db.pl' >> >> 6, check on IPA replica immediately, 'ipa user-find' shows 4 users >> 'A, B, C, D' at the beginning. >> >> 7, check IPA Master, 'ipa user-find' shows still only two users 'A, B'. >> >> 8, wait 3 minutes or so, check on IPA replica, and found that there >> are only THREE users 'A, B, D'. The users 'C' is deleted now -- >> change propagated from IPA Master. >> >> 9, check on IPA Master again and again, there are still only two >> users 'A, B'. >> >> 10, check on IPA Replica again and again, there are still three users >> 'A, B,D'. --- this status is different from IPA Master's 'A,B', or >> backup's 'A, B, C, D'. >> >> >> If backup was created without '-r' option, then the step 8 above will >> always show 'A,B,C,D', the same as backup. with '-r' option make the >> final result between. >> >> >> Hope I have explained it clearly. Please advice something like >> ipa2ldif.pl and ldif2ipa.pl tools. There are really the key useful >> feature for serious production IPA deployment, which is definitely of >> much higher priority than dogtag. > > Sounds like a bug. What should happen is that the deletion of C and D > should be propagated to replica. Was a bug or a ticket filed? > >> >> Thanks a lot. >> >> --David >> >> >> >> >> *From:* Rich Megginson >> *To:* David Copperfield >> *Cc:* E Deon Lackey ; Petr Spacek >> ; Rob Crittenden ; >> "freeipa-users@redhat.com" >> *Sent:* Thursday, May 10, 2012 6:37 PM >> *Subject:* Re: [Freeipa-users] backup/restore IPA servers with >> db2ldap.pl, ldap2db.pl ??? >> >> On 05/10/2012 07:32 PM, David Copperfield wrote: >>> Hi Rich and all, >>> >>> the '-r' option to db2ldif.pl <http://db2ldif.pl> doesn't work >>> neither, it make few difference. >>> >>> My command, backup and restore commands on the IPA replica are: >>> >>> db2ldif.pl -D 'cn=Directory Manager' -w - -r -s 'dc=example,dc=com' >>> >>> ldif2db.pl <http://ldif2db.pl> -D 'cn=Directory Manager' -w - -i >>> >>> >>> The only difference is: after IPA master restart (restart happens >>> after IPA replica's restore operation), the changes -- which applied >>> on IPA master before backup -- are propagated to IPA replica. Which >>> is in fact, make the restoration test end up with a result >>> completely unusable on IPA replica, an result that is different from >>> backup, and different from IPA master. >> >> I don't quite understand what you mean. >> >>> >>> Please let me know if there are any other options/steps to follow. >>> Thanks. >> >> Not sure what else to try. >> >>> >>> --David >>> >>> >>> >>> >>> >>> *From:* Rich Megginson >>> <mailto:rmegg...@redhat.com> >>> *To:* David Copperfield <mailto:cao2...@yahoo.com> >>> *Cc:* "freeipa-users@redhat.com" <mailto:freeipa-users@redhat.com> >>> <mailto:freeipa-users@redhat.com>; Rob >>> Crittenden <mailto:rcrit...@redhat.com>; Petr >>> Spacek <mailto:pspa...@redhat.com> >>> *Sent:* Thursday, May 10, 2012 5:28 PM >>> *Subject:* Re: [Freeipa-users] backup/restore IPA servers with >>> db2ldap.pl <http://db2ldap.pl>, ldap2db.pl <http://ldap2db.pl> ??? >>> >>> On 05/10/2012 04:37 PM, David Copperfield wrote: >>>> Hi Rich and all, >>>> >>>> Thanks for correct
Re: [Freeipa-users] backup/restore IPA servers with db2ldap.pl, ldap2db.pl ???
On 05/10/2012 07:54 PM, David Copperfield wrote: OK, that means the steps below: 1) on IPA replica, lets create 4 IPA users: A,B,C and D. Now make a backup with 'db2ldif.pl -r ...' 2) on IPA replica, delete the user D. 'ipa user-del D'. 3, on IPA master, delete the user C. 'ipa user-del C'. 4, now check on other IPA master and IPA replica, both shows only two users 'A' and 'B'. this is expected. 5, now on IPA replica, restore the backup with 'ldif2db.pl' 6, check on IPA replica immediately, 'ipa user-find' shows 4 users 'A, B, C, D' at the beginning. 7, check IPA Master, 'ipa user-find' shows still only two users 'A, B'. 8, wait 3 minutes or so, check on IPA replica, and found that there are only THREE users 'A, B, D'. The users 'C' is deleted now -- change propagated from IPA Master. 9, check on IPA Master again and again, there are still only two users 'A, B'. 10, check on IPA Replica again and again, there are still three users 'A, B,D'. --- this status is different from IPA Master's 'A,B', or backup's 'A, B, C, D'. If backup was created without '-r' option, then the step 8 above will always show 'A,B,C,D', the same as backup. with '-r' option make the final result between. Hope I have explained it clearly. Please advice something like ipa2ldif.pl and ldif2ipa.pl tools. There are really the key useful feature for serious production IPA deployment, which is definitely of much higher priority than dogtag. Sounds like a bug. What should happen is that the deletion of C and D should be propagated to replica. Thanks a lot. --David -------- *From:* Rich Megginson *To:* David Copperfield *Cc:* E Deon Lackey ; Petr Spacek ; Rob Crittenden ; "freeipa-users@redhat.com" *Sent:* Thursday, May 10, 2012 6:37 PM *Subject:* Re: [Freeipa-users] backup/restore IPA servers with db2ldap.pl, ldap2db.pl ??? On 05/10/2012 07:32 PM, David Copperfield wrote: Hi Rich and all, the '-r' option to db2ldif.pl <http://db2ldif.pl> doesn't work neither, it make few difference. My command, backup and restore commands on the IPA replica are: db2ldif.pl -D 'cn=Directory Manager' -w - -r -s 'dc=example,dc=com' ldif2db.pl <http://ldif2db.pl> -D 'cn=Directory Manager' -w - -i The only difference is: after IPA master restart (restart happens after IPA replica's restore operation), the changes -- which applied on IPA master before backup -- are propagated to IPA replica. Which is in fact, make the restoration test end up with a result completely unusable on IPA replica, an result that is different from backup, and different from IPA master. I don't quite understand what you mean. Please let me know if there are any other options/steps to follow. Thanks. Not sure what else to try. --David *From:* Rich Megginson <mailto:rmegg...@redhat.com> *To:* David Copperfield <mailto:cao2...@yahoo.com> *Cc:* "freeipa-users@redhat.com" <mailto:freeipa-users@redhat.com> <mailto:freeipa-users@redhat.com>; Rob Crittenden <mailto:rcrit...@redhat.com>; Petr Spacek <mailto:pspa...@redhat.com> *Sent:* Thursday, May 10, 2012 5:28 PM *Subject:* Re: [Freeipa-users] backup/restore IPA servers with db2ldap.pl <http://db2ldap.pl>, ldap2db.pl <http://ldap2db.pl> ??? On 05/10/2012 04:37 PM, David Copperfield wrote: Hi Rich and all, Thanks for correction. They are db2ldif.pl <http://db2ldif.pl> and ldif2db.pl <http://ldif2db.pl> scripts, which are originally for 389 Directory Servers' backup and restore purposes. There are no IPA tools for IPA system backup and restore. Is there a plan to develop tools like ipa2ldif.pl <http://ipa2ldif.pl> and ldif2ipa.pl <http://ldif2ipa.pl> soon? or, at least, whether it is in IPA roadmap? For the second question: I use the simple way: ipa user-add/user-delete/user-find to see whether data is propagated. My testing steps are like this: 1, run 'ipa user-add testuser' on IPA replica, check it on IPA master with 'ipa user-find testuser' and it is found in a few seconds -- not 5 minutes. 2, run 'db2ldif.pl on IPA replica to save a backup. 3, run 'ipa user-del testuser' on IPA replica, then 'ipa user-find' on IPA replica, and it shows that the user is deleted. 4, double check 'ipa user-find test user' on IPA master, and it is found deleted, which is as expected and it is propagated in just a few seconds. 5, run 'ldif2db.pl' on the same IPA replica
Re: [Freeipa-users] backup/restore IPA servers with db2ldap.pl, ldap2db.pl ???
OK, that means the steps below: 1) on IPA replica, lets create 4 IPA users: A,B,C and D. Now make a backup with 'db2ldif.pl -r ...' 2) on IPA replica, delete the user D. 'ipa user-del D'. 3, on IPA master, delete the user C. 'ipa user-del C'. 4, now check on other IPA master and IPA replica, both shows only two users 'A' and 'B'. this is expected. 5, now on IPA replica, restore the backup with 'ldif2db.pl' 6, check on IPA replica immediately, 'ipa user-find' shows 4 users 'A, B, C, D' at the beginning. 7, check IPA Master, 'ipa user-find' shows still only two users 'A, B'. 8, wait 3 minutes or so, check on IPA replica, and found that there are only THREE users 'A, B, D'. The users 'C' is deleted now -- change propagated from IPA Master. 9, check on IPA Master again and again, there are still only two users 'A, B'. 10, check on IPA Replica again and again, there are still three users 'A, B,D'. --- this status is different from IPA Master's 'A,B', or backup's 'A, B, C, D'. If backup was created without '-r' option, then the step 8 above will always show 'A,B,C,D', the same as backup. with '-r' option make the final result between. Hope I have explained it clearly. Please advice something like ipa2ldif.pl and ldif2ipa.pl tools. There are really the key useful feature for serious production IPA deployment, which is definitely of much higher priority than dogtag. Thanks a lot. --David ________ From: Rich Megginson To: David Copperfield Cc: E Deon Lackey ; Petr Spacek ; Rob Crittenden ; "freeipa-users@redhat.com" Sent: Thursday, May 10, 2012 6:37 PM Subject: Re: [Freeipa-users] backup/restore IPA servers with db2ldap.pl, ldap2db.pl ??? On 05/10/2012 07:32 PM, David Copperfield wrote: Hi Rich and all, > > >the '-r' option to db2ldif.pl doesn't work neither, it make few difference. > > >My command, backup and restore commands on the IPA replica are: > > >db2ldif.pl -D 'cn=Directory Manager' -w - -r -s 'dc=example,dc=com' > > >ldif2db.pl -D 'cn=Directory Manager' -w - -i > > >The only difference is: after IPA master restart (restart happens after IPA >replica's restore operation), the changes -- which applied on IPA master >before backup -- are propagated to IPA replica. Which is in fact, make the >restoration test end up with a result completely unusable on IPA replica, an >result that is different from backup, and different from IPA master. > I don't quite understand what you mean. > >Please let me know if there are any other options/steps to follow. Thanks. Not sure what else to try. > >--David > > > > > > > > > > > From: Rich Megginson >To: David Copperfield >Cc: "freeipa-users@redhat.com" ; Rob Crittenden >; Petr Spacek >Sent: Thursday, May 10, 2012 5:28 PM >Subject: Re: [Freeipa-users] backup/restore IPA servers with db2ldap.pl, >ldap2db.pl ??? > > >On 05/10/2012 04:37 PM, David Copperfield wrote: >Hi Rich and all, >> >> >>Thanks for correction. They are db2ldif.pl and ldif2db.pl scripts, which are >>originally for 389 Directory Servers' backup and restore purposes. >> >> >>There are no IPA tools for IPA system backup and restore. Is there a plan to >>develop tools like ipa2ldif.pl and ldif2ipa.pl soon? or, at least, whether it >>is in IPA roadmap? >> >> >>For the second question: I use the simple way: ipa >>user-add/user-delete/user-find to see whether data is propagated. My testing >>steps are like this: >> >> >> 1, run 'ipa user-add testuser' on IPA replica, check it on IPA master with >>'ipa user-find testuser' and it is found in a few seconds -- not 5 minutes. >> >> >> 2, run 'db2ldif.pl on IPA replica to save a backup. >> >> >> 3, run 'ipa user-del testuser' on IPA replica, then 'ipa user-find' on IPA >>replica, and it shows that the user is deleted. >> >> >> 4, double check 'ipa user-find test user' on IPA master, and it is found >>deleted, which is as expected and it is propagated in just a few seconds. >> >> >> 5, run 'ldif2db.pl' on the same IPA replica where the backup was created. >> >> >> 6, run 'ipa user-find testuser' on IPA replica and it is found that the user >>testuser is alive again. >> >> 7, run 'ipa user-find testuser' on
Re: [Freeipa-users] backup/restore IPA servers with db2ldap.pl, ldap2db.pl ???
On 05/10/2012 07:32 PM, David Copperfield wrote: Hi Rich and all, the '-r' option to db2ldif.pl doesn't work neither, it make few difference. My command, backup and restore commands on the IPA replica are: db2ldif.pl -D 'cn=Directory Manager' -w - -r -s 'dc=example,dc=com' ldif2db.pl -D 'cn=Directory Manager' -w - -i The only difference is: after IPA master restart (restart happens after IPA replica's restore operation), the changes -- which applied on IPA master before backup -- are propagated to IPA replica. Which is in fact, make the restoration test end up with a result completely unusable on IPA replica, an result that is different from backup, and different from IPA master. I don't quite understand what you mean. Please let me know if there are any other options/steps to follow. Thanks. Not sure what else to try. --David *From:* Rich Megginson *To:* David Copperfield *Cc:* "freeipa-users@redhat.com" ; Rob Crittenden ; Petr Spacek *Sent:* Thursday, May 10, 2012 5:28 PM *Subject:* Re: [Freeipa-users] backup/restore IPA servers with db2ldap.pl, ldap2db.pl ??? On 05/10/2012 04:37 PM, David Copperfield wrote: Hi Rich and all, Thanks for correction. They are db2ldif.pl <http://db2ldif.pl> and ldif2db.pl <http://ldif2db.pl> scripts, which are originally for 389 Directory Servers' backup and restore purposes. There are no IPA tools for IPA system backup and restore. Is there a plan to develop tools like ipa2ldif.pl <http://ipa2ldif.pl> and ldif2ipa.pl <http://ldif2ipa.pl> soon? or, at least, whether it is in IPA roadmap? For the second question: I use the simple way: ipa user-add/user-delete/user-find to see whether data is propagated. My testing steps are like this: 1, run 'ipa user-add testuser' on IPA replica, check it on IPA master with 'ipa user-find testuser' and it is found in a few seconds -- not 5 minutes. 2, run 'db2ldif.pl on IPA replica to save a backup. 3, run 'ipa user-del testuser' on IPA replica, then 'ipa user-find' on IPA replica, and it shows that the user is deleted. 4, double check 'ipa user-find test user' on IPA master, and it is found deleted, which is as expected and it is propagated in just a few seconds. 5, run 'ldif2db.pl' on the same IPA replica where the backup was created. 6, run 'ipa user-find testuser' on IPA replica and it is found that the user testuser is alive again. 7, run 'ipa user-find testuser' on IPA master. 1/3 times we can find it -- and in just a few seconds. other 2/3 times it could not be found even after HALF HOUR. Please have a quick duplicate tests at your side and advice what normal users should do, because a reliable backup/restore solution is definitely one of the key criteria. Thanks a lot. Ok, I see. The problem is that a regular db2ldif[.pl] does not save the replication meta-data. You must use the -r option to generate an ldif file with the replication meta-data. ldif2db[.pl] is destructive - it wipes out your database completely and replaces it, wiping out any replication meta-data in the process. If you ldif2db[.pl] a file exported with db2ldif[.pl] -r, it will replace the replication meta-data too. See http://docs.redhat.com/docs/en-US/Red_Hat_Directory_Server/9.0/html/Administration_Guide/Managing_Replication-Initializing_Consumers.html#Initializing_Consumers-Manual_Consumer_Initialization_Using_the_Command_Line --David *From:* Rich Megginson <mailto:rmegg...@redhat.com> *To:* David Copperfield <mailto:cao2...@yahoo.com> *Cc:* "freeipa-users@redhat.com" <mailto:freeipa-users@redhat.com> <mailto:freeipa-users@redhat.com>; Rob Crittenden <mailto:rcrit...@redhat.com>; Petr Spacek <mailto:pspa...@redhat.com> *Sent:* Thursday, May 10, 2012 3:19 PM *Subject:* Re: [Freeipa-users] backup/restore IPA servers with db2ldap.pl <http://db2ldap.pl>, ldap2db.pl <http://ldap2db.pl> ??? On 05/10/2012 03:57 PM, David Copperfield wrote: Hi Rob, Petr and all, Because recently crashes of my IPA master and IPA replicas servers, I'm thinking of methods of backup/restore IPA user data: users, groups, host and server certificates etc. It's said that the only official way is to create an extra IPA replica and backup/snapshot that replica all the way. But there still has a big chance that some mistakes propagate for a to whole IPA domain/realm before the IAP administrator find it and data got lost forever and some may not even be recovered. What I think is because both Dogtag and IPA store data in backend 389 directory servers separately, then if I f
Re: [Freeipa-users] backup/restore IPA servers with db2ldap.pl, ldap2db.pl ???
Hi Rich and all, the '-r' option to db2ldif.pl doesn't work neither, it make few difference. My command, backup and restore commands on the IPA replica are: db2ldif.pl -D 'cn=Directory Manager' -w - -r -s 'dc=example,dc=com' ldif2db.pl -D 'cn=Directory Manager' -w - -i The only difference is: after IPA master restart (restart happens after IPA replica's restore operation), the changes -- which applied on IPA master before backup -- are propagated to IPA replica. Which is in fact, make the restoration test end up with a result completely unusable on IPA replica, an result that is different from backup, and different from IPA master. Please let me know if there are any other options/steps to follow. Thanks. --David From: Rich Megginson To: David Copperfield Cc: "freeipa-users@redhat.com" ; Rob Crittenden ; Petr Spacek Sent: Thursday, May 10, 2012 5:28 PM Subject: Re: [Freeipa-users] backup/restore IPA servers with db2ldap.pl, ldap2db.pl ??? On 05/10/2012 04:37 PM, David Copperfield wrote: Hi Rich and all, > > >Thanks for correction. They are db2ldif.pl and ldif2db.pl scripts, which are >originally for 389 Directory Servers' backup and restore purposes. > > >There are no IPA tools for IPA system backup and restore. Is there a plan to >develop tools like ipa2ldif.pl and ldif2ipa.pl soon? or, at least, whether it >is in IPA roadmap? > > >For the second question: I use the simple way: ipa >user-add/user-delete/user-find to see whether data is propagated. My testing >steps are like this: > > > 1, run 'ipa user-add testuser' on IPA replica, check it on IPA master with >'ipa user-find testuser' and it is found in a few seconds -- not 5 minutes. > > > 2, run 'db2ldif.pl on IPA replica to save a backup. > > > 3, run 'ipa user-del testuser' on IPA replica, then 'ipa user-find' on IPA >replica, and it shows that the user is deleted. > > > 4, double check 'ipa user-find test user' on IPA master, and it is found >deleted, which is as expected and it is propagated in just a few seconds. > > > 5, run 'ldif2db.pl' on the same IPA replica where the backup was created. > > > 6, run 'ipa user-find testuser' on IPA replica and it is found that the user >testuser is alive again. > > 7, run 'ipa user-find testuser' on IPA master. 1/3 times we can find it -- and in just a few seconds. other 2/3 times it could not be found even after HALF HOUR. > > >Please have a quick duplicate tests at your side and advice what normal users >should do, because a reliable backup/restore solution is definitely one of the >key criteria. Thanks a lot. > > Ok, I see. The problem is that a regular db2ldif[.pl] does not save the replication meta-data. You must use the -r option to generate an ldif file with the replication meta-data. ldif2db[.pl] is destructive - it wipes out your database completely and replaces it, wiping out any replication meta-data in the process. If you ldif2db[.pl] a file exported with db2ldif[.pl] -r, it will replace the replication meta-data too. See http://docs.redhat.com/docs/en-US/Red_Hat_Directory_Server/9.0/html/Administration_Guide/Managing_Replication-Initializing_Consumers.html#Initializing_Consumers-Manual_Consumer_Initialization_Using_the_Command_Line --David > > > > > > > > > > > > > > > From: Rich Megginson >To: David Copperfield >Cc: "freeipa-users@redhat.com" ; Rob Crittenden >; Petr Spacek >Sent: Thursday, May 10, 2012 3:19 PM >Subject: Re: [Freeipa-users] backup/restore IPA servers with db2ldap.pl, >ldap2db.pl ??? > > >On 05/10/2012 03:57 PM, David Copperfield wrote: >Hi Rob, Petr and all, >> >> >>Because recently crashes of my IPA master and IPA replicas servers, I'm >>thinking of methods of backup/restore IPA user data: users, groups, host and >>server certificates etc. >> >> >>It's said that the only official way is to create an extra IPA replica and >>backup/snapshot that replica all the way. But there still has a big chance >>that some mistakes propagate for a to whole IPA domain/realm before the IAP >>administrator find it and data got lost forever and some may not even be >>recovered. >> >> >>What I think is because both Dogtag and IPA store data in backend 389 >>directory servers separately, then if I freeze the change on one IPA replica >>for a few minutes first, then run db2ldap.pl for both 389 ldap backends, then >>un-freeze the IPA replica t
Re: [Freeipa-users] backup/restore IPA servers with db2ldap.pl, ldap2db.pl ???
On 05/10/2012 04:37 PM, David Copperfield wrote: Hi Rich and all, Thanks for correction. They are db2ldif.pl and ldif2db.pl scripts, which are originally for 389 Directory Servers' backup and restore purposes. There are no IPA tools for IPA system backup and restore. Is there a plan to develop tools like ipa2ldif.pl and ldif2ipa.pl soon? or, at least, whether it is in IPA roadmap? For the second question: I use the simple way: ipa user-add/user-delete/user-find to see whether data is propagated. My testing steps are like this: 1, run 'ipa user-add testuser' on IPA replica, check it on IPA master with 'ipa user-find testuser' and it is found in a few seconds -- not 5 minutes. 2, run 'db2ldif.pl on IPA replica to save a backup. 3, run 'ipa user-del testuser' on IPA replica, then 'ipa user-find' on IPA replica, and it shows that the user is deleted. 4, double check 'ipa user-find test user' on IPA master, and it is found deleted, which is as expected and it is propagated in just a few seconds. 5, run 'ldif2db.pl' on the same IPA replica where the backup was created. 6, run 'ipa user-find testuser' on IPA replica and it is found that the user testuser is alive again. 7, run 'ipa user-find testuser' on IPA master. 1/3 times we can find it -- and in just a few seconds. other 2/3 times it could not be found even after HALF HOUR. Please have a quick duplicate tests at your side and advice what normal users should do, because a reliable backup/restore solution is definitely one of the key criteria. Thanks a lot. Ok, I see. The problem is that a regular db2ldif[.pl] does not save the replication meta-data. You must use the -r option to generate an ldif file with the replication meta-data. ldif2db[.pl] is destructive - it wipes out your database completely and replaces it, wiping out any replication meta-data in the process. If you ldif2db[.pl] a file exported with db2ldif[.pl] -r, it will replace the replication meta-data too. See http://docs.redhat.com/docs/en-US/Red_Hat_Directory_Server/9.0/html/Administration_Guide/Managing_Replication-Initializing_Consumers.html#Initializing_Consumers-Manual_Consumer_Initialization_Using_the_Command_Line --David *From:* Rich Megginson *To:* David Copperfield *Cc:* "freeipa-users@redhat.com" ; Rob Crittenden ; Petr Spacek *Sent:* Thursday, May 10, 2012 3:19 PM *Subject:* Re: [Freeipa-users] backup/restore IPA servers with db2ldap.pl, ldap2db.pl ??? On 05/10/2012 03:57 PM, David Copperfield wrote: Hi Rob, Petr and all, Because recently crashes of my IPA master and IPA replicas servers, I'm thinking of methods of backup/restore IPA user data: users, groups, host and server certificates etc. It's said that the only official way is to create an extra IPA replica and backup/snapshot that replica all the way. But there still has a big chance that some mistakes propagate for a to whole IPA domain/realm before the IAP administrator find it and data got lost forever and some may not even be recovered. What I think is because both Dogtag and IPA store data in backend 389 directory servers separately, then if I freeze the change on one IPA replica for a few minutes first, then run db2ldap.pl <http://db2ldap.pl> for both 389 ldap backends, then un-freeze the IPA replica to get sync from master. When data needs to be restored because of disasters, the backup files(in LDIF format -- for easy to read) can be restored to the two 389 LDAP backends on IPA replica with command ldap2db.pl <http://ldap2db.pl> during the freezing period. It's ldif2db.pl <http://ldif2db.pl> db2ldif.pl <http://db2ldif.pl> not ldap Have anyone tried this solution yet? Is there any limitations? My experiences showed that the IPA replica did get data restored successfully (no dogtag is involved so only one LDAP backend is saved/restored). But the IPA master some times didn't get the data synced from IPA replica ( 1/3 times it is synced, 2/3 times needs manual command 'ipa-replica-manage force-sync --from ' ). How did you verify that the data was synced? Note that if a server has been down for a while, it will take the supplier up to 5 minutes to recognize that the consumer is up again, without force sync. Please shed a light in this area, as backup/restore of IPA master/replica is even not mentioned on the IPA document at all. Thanks a lot. --David ___ Freeipa-users mailing list Freeipa-users@redhat.com <mailto:Freeipa-users@redhat.com> https://www.redhat.com/mailman/listinfo/freeipa-users ___ Freeipa-users mailing list Freeipa-users@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-users
Re: [Freeipa-users] backup/restore IPA servers with db2ldap.pl, ldap2db.pl ???
Hi Rich and all, Thanks for correction. They are db2ldif.pl and ldif2db.pl scripts, which are originally for 389 Directory Servers' backup and restore purposes. There are no IPA tools for IPA system backup and restore. Is there a plan to develop tools like ipa2ldif.pl and ldif2ipa.pl soon? or, at least, whether it is in IPA roadmap? For the second question: I use the simple way: ipa user-add/user-delete/user-find to see whether data is propagated. My testing steps are like this: 1, run 'ipa user-add testuser' on IPA replica, check it on IPA master with 'ipa user-find testuser' and it is found in a few seconds -- not 5 minutes. 2, run 'db2ldif.pl on IPA replica to save a backup. 3, run 'ipa user-del testuser' on IPA replica, then 'ipa user-find' on IPA replica, and it shows that the user is deleted. 4, double check 'ipa user-find test user' on IPA master, and it is found deleted, which is as expected and it is propagated in just a few seconds. 5, run 'ldif2db.pl' on the same IPA replica where the backup was created. 6, run 'ipa user-find testuser' on IPA replica and it is found that the user testuser is alive again. 7, run 'ipa user-find testuser' on IPA master. 1/3 times we can find it -- and in just a few seconds. other 2/3 times it could not be found even after HALF HOUR. Please have a quick duplicate tests at your side and advice what normal users should do, because a reliable backup/restore solution is definitely one of the key criteria. Thanks a lot. --David From: Rich Megginson To: David Copperfield Cc: "freeipa-users@redhat.com" ; Rob Crittenden ; Petr Spacek Sent: Thursday, May 10, 2012 3:19 PM Subject: Re: [Freeipa-users] backup/restore IPA servers with db2ldap.pl, ldap2db.pl ??? On 05/10/2012 03:57 PM, David Copperfield wrote: Hi Rob, Petr and all, > > >Because recently crashes of my IPA master and IPA replicas servers, I'm >thinking of methods of backup/restore IPA user data: users, groups, host and >server certificates etc. > > >It's said that the only official way is to create an extra IPA replica and >backup/snapshot that replica all the way. But there still has a big chance >that some mistakes propagate for a to whole IPA domain/realm before the IAP >administrator find it and data got lost forever and some may not even be >recovered. > > >What I think is because both Dogtag and IPA store data in backend 389 >directory servers separately, then if I freeze the change on one IPA replica >for a few minutes first, then run db2ldap.pl for both 389 ldap backends, then >un-freeze the IPA replica to get sync from master. > > > When data needs to be restored because of disasters, the backup files(in LDIF >format -- for easy to read) can be restored to the two 389 LDAP backends on >IPA replica with command ldap2db.pl during the freezing period. It's ldif2db.pl db2ldif.pl not ldap > > Have anyone tried this solution yet? Is there any limitations? > > >My experiences showed that the IPA replica did get data restored successfully >(no dogtag is involved so only one LDAP backend is saved/restored). But the >IPA master some times didn't get the data synced from IPA replica ( 1/3 times >it is synced, 2/3 times needs manual command 'ipa-replica-manage force-sync >--from ' ). How did you verify that the data was synced? Note that if a server has been down for a while, it will take the supplier up to 5 minutes to recognize that the consumer is up again, without force sync. > >Please shed a light in this area, as backup/restore of IPA master/replica is >even not mentioned on the IPA document at all. > > >Thanks a lot. > > >--David > > > > > > > > > > > > > > > > >___ Freeipa-users mailing list Freeipa-users@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-users___ Freeipa-users mailing list Freeipa-users@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-users
Re: [Freeipa-users] backup/restore IPA servers with db2ldap.pl, ldap2db.pl ???
On 05/10/2012 03:57 PM, David Copperfield wrote: Hi Rob, Petr and all, Because recently crashes of my IPA master and IPA replicas servers, I'm thinking of methods of backup/restore IPA user data: users, groups, host and server certificates etc. It's said that the only official way is to create an extra IPA replica and backup/snapshot that replica all the way. But there still has a big chance that some mistakes propagate for a to whole IPA domain/realm before the IAP administrator find it and data got lost forever and some may not even be recovered. What I think is because both Dogtag and IPA store data in backend 389 directory servers separately, then if I freeze the change on one IPA replica for a few minutes first, then run db2ldap.pl for both 389 ldap backends, then un-freeze the IPA replica to get sync from master. When data needs to be restored because of disasters, the backup files(in LDIF format -- for easy to read) can be restored to the two 389 LDAP backends on IPA replica with command ldap2db.pl during the freezing period. It's ldif2db.pl db2ldif.pl not ldap Have anyone tried this solution yet? Is there any limitations? My experiences showed that the IPA replica did get data restored successfully (no dogtag is involved so only one LDAP backend is saved/restored). But the IPA master some times didn't get the data synced from IPA replica ( 1/3 times it is synced, 2/3 times needs manual command 'ipa-replica-manage force-sync --from ' ). How did you verify that the data was synced? Note that if a server has been down for a while, it will take the supplier up to 5 minutes to recognize that the consumer is up again, without force sync. Please shed a light in this area, as backup/restore of IPA master/replica is even not mentioned on the IPA document at all. Thanks a lot. --David ___ Freeipa-users mailing list Freeipa-users@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-users ___ Freeipa-users mailing list Freeipa-users@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-users
[Freeipa-users] backup/restore IPA servers with db2ldap.pl, ldap2db.pl ???
Hi Rob, Petr and all, Because recently crashes of my IPA master and IPA replicas servers, I'm thinking of methods of backup/restore IPA user data: users, groups, host and server certificates etc. It's said that the only official way is to create an extra IPA replica and backup/snapshot that replica all the way. But there still has a big chance that some mistakes propagate for a to whole IPA domain/realm before the IAP administrator find it and data got lost forever and some may not even be recovered. What I think is because both Dogtag and IPA store data in backend 389 directory servers separately, then if I freeze the change on one IPA replica for a few minutes first, then run db2ldap.pl for both 389 ldap backends, then un-freeze the IPA replica to get sync from master. When data needs to be restored because of disasters, the backup files(in LDIF format -- for easy to read) can be restored to the two 389 LDAP backends on IPA replica with command ldap2db.pl during the freezing period. Have anyone tried this solution yet? Is there any limitations? My experiences showed that the IPA replica did get data restored successfully (no dogtag is involved so only one LDAP backend is saved/restored). But the IPA master some times didn't get the data synced from IPA replica ( 1/3 times it is synced, 2/3 times needs manual command 'ipa-replica-manage force-sync --from ' ). Please shed a light in this area, as backup/restore of IPA master/replica is even not mentioned on the IPA document at all. Thanks a lot. --David___ Freeipa-users mailing list Freeipa-users@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-users
