Sorry Thierry, any news for my issue? Did you see new logs? Could I use the command *ipa-replica-manage re-initialize --from srv02.ipa.mydomain.com <http://srv02.ipa.mydomain.com>* in the First Master server? Or is it better to solve the issue in another way?
Please let me know, thanks. Bye Il giorno gio 23 apr 2020 alle ore 09:54 Morgan Marodin <mor...@marodin.it> ha scritto: > Hi Theirry. > > To tell the truth my configuration was already set to on, on both VMs: > > *[root@srv01 ~]# ldapsearch -D "cn=Directory Manager" -h localhost -b > "cn=config" -w $PASS | grep > nsslapd-ignore-time-skewnsslapd-ignore-time-skew: on* > > > *[root@srv02 ~]# ldapsearch -D "cn=Directory Manager" -h localhost -b > "cn=config" -w $PASS | grep > nsslapd-ignore-time-skewnsslapd-ignore-time-skew: on* > > Anyway, I tried to set up it to *off* and then to *on* again, but now I > have a new issue into logs of the 2nd server :( > > Srv01 logs are similar as before: > *[23/Apr/2020:09:28:45.958922636 +0200] - WARN - csngen_new_csn - Too much > time skew (-22777212 secs). Current seqnum=5ca0* > > Srv02 logs now are like these: > > > *[23/Apr/2020:09:32:14.919328803 +0200] - ERR - > agmt="cn=meTosrv01.ipa.mydomain.com <http://meTosrv01.ipa.mydomain.com>" > (srv01:389) - clcache_load_buffer - Can't locate CSN 5e7cfe03000400030000 > in the changelog (DB rc=-30988). If replication stops, the consumer may > need to be reinitialized.[23/Apr/2020:09:32:14.920873489 +0200] - ERR - > NSMMReplicationPlugin - changelog program - repl_plugin_name_cl - > agmt="cn=meTosrv01.ipa.mydomain.com <http://meTosrv01.ipa.mydomain.com>" > (srv01:389): CSN 5e7cfe03000400030000 not found, we aren't as up to date, > or we purged[23/Apr/2020:09:32:14.922161821 +0200] - ERR - > NSMMReplicationPlugin - send_updates - agmt="cn=meTosrv01.ipa.mydomain.com > <http://meTosrv01.ipa.mydomain.com>" (srv01:389): Data required to update > replica has been purged from the changelog. If the error persists the > replica must be reinitialized.* > > I have just tried to force a replica on both sides, without success: > > > > > > > > > *[root@srv01 ~]# ipa-replica-manage force-sync --from > srv02.ipa.mydomain.com <http://srv02.ipa.mydomain.com>No status yetNo > status yetNo status yet[root@srv02 ~]# ipa-replica-manage force-sync --from > srv01.ipa.mydomain.com <http://srv01.ipa.mydomain.com>No status yetNo > status yetNo status yet* > > What could I do now? > Thanks, bye > > Il giorno mer 22 apr 2020 alle ore 17:08 thierry bordaz via FreeIPA-users < > freeipa-users@lists.fedorahosted.org> ha scritto: > >> Hi Morgan, >> >> Sure. The most immediate and safest action is to do >> >> dn: cn=config >> changetype: modify >> replace: nsslapd-ignore-time-skew >> nsslapd-ignore-time-skew: on >> >> >> >> On all servers in the topology (no need to restart). Then monitor if >> replication is catching up. >> Okay NTP issues is likely the RC of your time skew but there is not easy >> way to prove it if any. >> >> best regards >> theirry >> >> >> >> On 4/22/20 3:16 PM, Morgan Marodin via FreeIPA-users wrote: >> >> Hi. >> >> I don't have access to RedHat portal :( >> There are similar articles in a public forum? >> >> Anyway ... could I stop ipa-server, change the value of >> *nsslapd-ignore-time-skew* into >> */etc/dirsrv/slapd-IPA-MYDOMAIN-COM/dse.ldif* and start again the server? >> Or is more complicated to change the configuration? >> >> VMs are local, but the cluster where the 1st server is running is >> affected by NTP problems ... >> For this reason I want to remove the First Master and install another >> replica in the new cluster. >> >> Thanks, bye. >> Morgan >> >> Il giorno mer 22 apr 2020 alle ore 11:33 thierry bordaz via FreeIPA-users >> <freeipa-users@lists.fedorahosted.org> ha scritto: >> >>> Hi, >>> >>> CSN generator time skew is a pending issue still under investigation. >>> >>> At the moment the way your csn generator is messed up looks not fatal. >>> You can allow replication to continue with the setting of >>> nsslapd-ignore-time-skew on all servers. ( >>> https://access.redhat.com/solutions/1162703) >>> >>> If it does not allow replication to continue there is a recovery >>> procedure but I would recommend to first try ignore-time-skew ( >>> https://access.redhat.com/solutions/3543811) >>> >>> NTP tuning or specific VMs are suspected to contribute to time skew. >>> What type of VMs are you using (local or cloud (AWS)) ? >>> >>> best regards >>> thierry >>> >>> On 4/21/20 5:42 PM, Morgan Marodin via FreeIPA-users wrote: >>> >>> Hi. >>> >>> Into my environment I have two IPA server, replicating each other. >>> They are both 7.6 OS systems, ipa-server RPM version is >>> 4.6.4-10.0.1.el7_6.2.x86_64. >>> >>> The first server installed was srv01 (many years ago), then I installed >>> the replica into srv02 (like a year later the 1st node). >>> When I had a single server I did also a trust with my corporate Active >>> Directory. >>> VMs are running in 2 different hypervisor clusters. >>> >>> Now the replication doesn't works. Into log files I have this error: >>> >>> >>> *[16/Apr/2020:12:25:36.856632697 +0200] - ERR - csngen_adjust_time - >>> Adjustment limit exceeded; value - 23221226, limit - 86400 >>> [16/Apr/2020:12:25:36.857909222 +0200] - ERR - NSMMReplicationPlugin - >>> repl5_inc_run - agmt="cn=meTosrv01.ipa.mydomain.com >>> <http://meTosrv01.ipa.mydomain.com>" (srv01:389): Fatal error - too much >>> time skew between replicas! [16/Apr/2020:12:25:36.862233147 +0200] - ERR - >>> NSMMReplicationPlugin - repl5_inc_run - agmt="cn=meTosrv01.ipa.mydomain.com >>> <http://meTosrv01.ipa.mydomain.com>" (srv01:389): Incremental update failed >>> and requires administrator action* >>> >>> I tried to force the replica, but the limit exceeded problem doesn't >>> allow the sync. >>> I know that the problem is that CSN generator has become grossly skewed. >>> Using the external script readNsState.py I found that there was as >>> offset time for about a month, so ... I waited for a month and then the >>> issue disappeared. >>> But now the offset is about 9 months ... I can't wait so much time :) >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> *[root@srv01 scripts]# ./readNsState.py >>> /etc/dirsrv/slapd-IPA-MYDOMAIN-COM/dse.ldif nsState is >>> BAAAAAAAAACCN/xfAAAAAHbiBAAAAAAABCgAAAAAAAANdQAAAAAAAA== Little Endian For >>> replica cn=replica,cn=dc\3Dipa\2Cdc\3Dmydomain\2Cdc\3Dcom,cn=mapping >>> tree,cn=con fmtstr=[H6x3QH6x] size=40 len of nsstate is 40 CSN >>> generator state: Replica ID : 4 Sampled Time : 1610364802 >>> Gen as csn : 5ffc37822996500040000 Time as str : Mon Jan 11 >>> 12:33:22 2021 Local Offset : 320118 Remote Offset : 10244 Seq. >>> num : 29965 System time : Tue Apr 21 15:03:45 2020 Diff in >>> sec. : -22890577 Day:sec diff : -265:5423 nsState is >>> YAAAAAAAAAADLZheAAAAAAAAAAAAAAAAXSgAAAAAAAATAAAAAAAAAA== Little Endian For >>> replica cn=replica,cn=o\3Dipaca,cn=mapping tree,cn=config >>> fmtstr=[H6x3QH6x] size=40 len of nsstate is 40 CSN generator state: >>> Replica ID : 96 Sampled Time : 1587031299 Gen as csn : >>> 5e982d03001900960000 Time as str : Thu Apr 16 12:01:39 2020 Local >>> Offset : 0 Remote Offset : 10333 Seq. num : 19 System >>> time : Tue Apr 21 15:03:45 2020 Diff in sec. : 442926 Day:sec >>> diff : 5:10926 [root@srv02 scripts]# ./readNsState.py >>> /etc/dirsrv/slapd-IPA-MYDOMAIN-COM/dse.ldif nsState is >>> AwAAAAAAAABU7p5eAAAAAAAAAAAAAAAAsVNiAQAAAAAAAAAAAAAAAA== Little Endian For >>> replica cn=replica,cn=dc\3Dipa\2Cdc\3Dmydomain\2Cdc\3Dcom,cn=mapping >>> tree,cn=con fmtstr=[H6x3QH6x] size=40 len of nsstate is 40 CSN >>> generator state: Replica ID : 3 Sampled Time : 1587474004 >>> Gen as csn : 5e9eee54000000030000 Time as str : Tue Apr 21 >>> 15:00:04 2020 Local Offset : 0 Remote Offset : 23221169 Seq. >>> num : 0 System time : Tue Apr 21 15:02:38 2020 Diff in sec. >>> : 154 Day:sec diff : 0:154 nsState is >>> YQAAAAAAAAAuLZheAAAAAEUBAAAAAAAA7SYAAAAAAAASAAAAAAAAAA== Little Endian For >>> replica cn=replica,cn=o\3Dipaca,cn=mapping tree,cn=config >>> fmtstr=[H6x3QH6x] size=40 len of nsstate is 40 CSN generator state: >>> Replica ID : 97 Sampled Time : 1587031342 Gen as csn : >>> 5e982d2e001800970000 Time as str : Thu Apr 16 12:02:22 2020 Local >>> Offset : 325 Remote Offset : 9965 Seq. num : 18 System >>> time : Tue Apr 21 15:02:38 2020 Diff in sec. : 442816 Day:sec >>> diff : 5:10816* >>> >>> As you can see in the 1st node the Time as str is Jan 11 of 2021. >>> With timedatectl command I see that both VMs use the same Time zone and >>> the clock is correct. >>> >>> I found this old article to fix my issue: >>> *https://www.redhat.com/archives/freeipa-users/2014-February/msg00007.html >>> <https://www.redhat.com/archives/freeipa-users/2014-February/msg00007.html>* >>> >>> But ... I had the same issue in the past, always in the 1st server. So, >>> in my mind I don't want to try to use that fix. >>> I have a new hypervisor cluster, so I would prefer to reinstall the 1st >>> server, using these steps: >>> >>> 1) check if all roles (also the CA) is installed in srv02 >>> You can find here some data about the VMs: >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> *[root@srv01 ~]# ipa server-show srv01.ipa.mydomain.com >>> <http://srv01.ipa.mydomain.com> Server name: srv01.ipa.mydomain.com >>> <http://srv01.ipa.mydomain.com> Managed suffixes: domain, ca Min domain >>> level: 0 Max domain level: 1 Enabled server roles: CA server, IPA >>> master, DNS server, NTP server, AD trust controller [root@srv02 ~]# ipa >>> server-show srv02.ipa.mydomain.com <http://srv02.ipa.mydomain.com> Server >>> name: srv02.ipa.mydomain.com <http://srv02.ipa.mydomain.com> Managed >>> suffixes: domain, ca Min domain level: 0 Max domain level: 1 Enabled >>> server roles: CA server, IPA master, DNS server, NTP server [root@srv01 ~]# >>> ipa config-show Maximum username length: 32 Home directory base: /home >>> Default shell: /bin/bash Default users group: ipausers Default e-mail >>> domain: ipa.mydomain.com <http://ipa.mydomain.com> Search time limit: 2 >>> Search size limit: 100 User search fields: >>> uid,givenname,sn,telephonenumber,ou,title Group search fields: >>> cn,description Enable migration mode: FALSE Certificate Subject base: >>> O=IPA.MYDOMAIN.COM <http://IPA.MYDOMAIN.COM> Password Expiration >>> Notification (days): 4 Password plugin features: AllowNThash SELinux >>> user map order: >>> guest_u:s0$xguest_u:s0$user_u:s0$staff_u:s0-s0:c0.c1023$unconfined_u:s0-s0:c0.c1023 >>> Default SELinux user: unconfined_u:s0-s0:c0.c1023 Default PAC types: >>> MS-PAC, nfs:NONE IPA masters: srv01.ipa.mydomain.com >>> <http://srv01.ipa.mydomain.com>, srv02.ipa.mydomain.com >>> <http://srv02.ipa.mydomain.com> IPA CA servers: srv01.ipa.mydomain.com >>> <http://srv01.ipa.mydomain.com>, srv02.ipa.mydomain.com >>> <http://srv02.ipa.mydomain.com> IPA NTP servers: srv01.ipa.mydomain.com >>> <http://srv01.ipa.mydomain.com>, srv02.ipa.mydomain.com >>> <http://srv02.ipa.mydomain.com> IPA CA renewal master: >>> srv01.ipa.mydomain.com <http://srv01.ipa.mydomain.com> [root@srv02 ~]# ipa >>> config-show Maximum username length: 32 Home directory base: /home >>> Default shell: /bin/bash Default users group: ipausers Default e-mail >>> domain: ipa.mydomain.com <http://ipa.mydomain.com> Search time limit: 2 >>> Search size limit: 100 User search fields: >>> uid,givenname,sn,telephonenumber,ou,title Group search fields: >>> cn,description Enable migration mode: FALSE Certificate Subject base: >>> O=IPA.MYDOMAIN.COM <http://IPA.MYDOMAIN.COM> Password Expiration >>> Notification (days): 4 Password plugin features: AllowNThash SELinux >>> user map order: >>> guest_u:s0$xguest_u:s0$user_u:s0$staff_u:s0-s0:c0.c1023$unconfined_u:s0-s0:c0.c1023 >>> Default SELinux user: unconfined_u:s0-s0:c0.c1023 Default PAC types: >>> MS-PAC, nfs:NONE IPA masters: srv01.ipa.mydomain.com >>> <http://srv01.ipa.mydomain.com>, srv02.ipa.mydomain.com >>> <http://srv02.ipa.mydomain.com> IPA CA servers: srv01.ipa.mydomain.com >>> <http://srv01.ipa.mydomain.com>, srv02.ipa.mydomain.com >>> <http://srv02.ipa.mydomain.com> IPA NTP servers: srv01.ipa.mydomain.com >>> <http://srv01.ipa.mydomain.com>, srv02.ipa.mydomain.com >>> <http://srv02.ipa.mydomain.com> IPA CA renewal master: >>> srv01.ipa.mydomain.com <http://srv01.ipa.mydomain.com> [root@srv01 ~]# >>> ipactl status Directory Service: RUNNING krb5kdc Service: RUNNING kadmin >>> Service: RUNNING named Service: RUNNING httpd Service: RUNNING ipa-custodia >>> Service: RUNNING ntpd Service: RUNNING pki-tomcatd Service: STOPPED smb >>> Service: RUNNING winbind Service: RUNNING ipa-otpd Service: RUNNING >>> ipa-dnskeysyncd Service: RUNNING ipa: INFO: The ipactl command was >>> successful [root@srv02 ~]# ipactl status Directory Service: RUNNING krb5kdc >>> Service: RUNNING kadmin Service: RUNNING named Service: RUNNING httpd >>> Service: RUNNING ipa-custodia Service: RUNNING ntpd Service: RUNNING >>> pki-tomcatd Service: STOPPED ipa-otpd Service: RUNNING ipa-dnskeysyncd >>> Service: RUNNING ipa: INFO: The ipactl command was successful [root@srv01 >>> ~]# certutil -L -d /etc/pki/pki-tomcat/alias Certificate Nickname >>> Trust Attributes >>> SSL,S/MIME,JAR/XPI Server-Cert cert-pki-ca >>> u,u,u subsystemCert cert-pki-ca >>> u,u,u caSigningCert cert-pki-ca >>> CTu,Cu,Cu ocspSigningCert cert-pki-ca >>> u,u,u auditSigningCert cert-pki-ca >>> u,u,Pu [root@srv02 ~]# certutil -L -d /etc/pki/pki-tomcat/alias >>> Certificate Nickname Trust >>> Attributes >>> SSL,S/MIME,JAR/XPI Server-Cert cert-pki-ca >>> u,u,u subsystemCert cert-pki-ca >>> u,u,u caSigningCert cert-pki-ca CTu,u,u >>> ocspSigningCert cert-pki-ca u,u,u >>> auditSigningCert cert-pki-ca u,u,Pu* >>> >>> >>> It seems that AD trust controller role, IPA CA renewal master, smb and >>> windbind are only in the 1st server. >>> And also caSigningCert cert-pki-ca entry is different (CTu,Cu,Cu vs >>> CTu,u,u). >>> >>> I can see only in the 1st server these DNS records: >>> >>> >>> >>> >>> >>> *_kerberos._tcp.Default-First-Site-Name._sites.dc._msdcs SRV 0 100 88 >>> srv01 _kerberos._tcp.dc._msdcs SRV 0 100 88 srv01 >>> _kerberos._udp.Default-First-Site-Name._sites.dc._msdcs SRV 0 100 88 srv01 >>> _kerberos._udp.dc._msdcs SRV 0 100 88 srv01 >>> _ldap._tcp.Default-First-Site-Name._sites.dc._msdcs 0 100 389 srv01 >>> _ldap._tcp.dc._msdcs 0 100 389 srv01* >>> >>> Srv01 is the first master, I know, but is the server VM that has clock >>> problems, in both situations. >>> So I want to keep srv02 and install a new one. >>> >>> What do I have to do to let the 2nd VM be a single server? >>> Could I use these URLs? >>> >>> *https://www.freeipa.org/page/Backup_and_Restore#One_Server_Loss_-_First_Master >>> <https://www.freeipa.org/page/Backup_and_Restore#One_Server_Loss_-_First_Master> >>> https://www.freeipa.org/page/V4/Server_Roles#Upgrade >>> <https://www.freeipa.org/page/V4/Server_Roles#Upgrade>* >>> >>> >>> 2) uninstall ipa-server from the 1st server (srv01) and then powering >>> off it, assuming that all data into the 2nd one are ok (srv02) >>> >>> 3) update freeipa and all other RPM packages into the VM srv02 >>> >>> 4) install a new fresh VM, always with 7 release, and create a new >>> replica >>> Could I use the same old hostname (srv01) and IP address for this new >>> VM? Or is better to use the same IP but a new name, like srv03? >>> >>> >>> Do you think this is the right way to solve my issue? >>> Or do you have any better idea? >>> >>> Please let me know, thanks. >>> Bye, Morgan >>> >>> _______________________________________________ >>> FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org >>> To unsubscribe send an email to freeipa-users-le...@lists.fedorahosted.org >>> Fedora Code of Conduct: >>> https://docs.fedoraproject.org/en-US/project/code-of-conduct/ >>> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines >>> List Archives: >>> https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedorahosted.org >>> >>> >>> _______________________________________________ >>> FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org >>> To unsubscribe send an email to >>> freeipa-users-le...@lists.fedorahosted.org >>> Fedora Code of Conduct: >>> https://docs.fedoraproject.org/en-US/project/code-of-conduct/ >>> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines >>> List Archives: >>> https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedorahosted.org >>> >> >> >> _______________________________________________ >> FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org >> To unsubscribe send an email to freeipa-users-le...@lists.fedorahosted.org >> Fedora Code of Conduct: >> https://docs.fedoraproject.org/en-US/project/code-of-conduct/ >> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines >> List Archives: >> https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedorahosted.org >> >> >> _______________________________________________ >> FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org >> To unsubscribe send an email to >> freeipa-users-le...@lists.fedorahosted.org >> Fedora Code of Conduct: >> https://docs.fedoraproject.org/en-US/project/code-of-conduct/ >> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines >> List Archives: >> https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedorahosted.org >> >
_______________________________________________ FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org To unsubscribe send an email to freeipa-users-le...@lists.fedorahosted.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedorahosted.org
