Re: Bug#935080: slapcat used in gosa hook script gosa-modify-host
Moin Mike, [ Mike Gabriel, 2020-12-07 ] > Instead of always fixing gosa-modify-host again after debian-edu-config > Update from buster/updates, I have now prepare a patch for the next deb10u7 > upload. Very good, thanks. Wolfgang signature.asc Description: PGP signature
Bug#935080: slapcat used in gosa hook script gosa-modify-host
On Fri, May 01, 2020 at 06:26:52AM +, Mike Gabriel wrote:
> Here is a slight change...
>
>
> ```
> #!/bin/bash
>
> set -e
>
> declare -a hosts
> num_hosts=0
> while read KEY VALUE ; do
> case "$KEY" in
> dn:) let "num_hosts+=1" ;;
> cn:) hosts[$(($num_hosts-1))]="$VALUE" ;;
> "")
> :
> ;;
> esac
> done < <(ldapsearch -xLLL "objectclass=dhcpHost")
>
> # add gateway host manually
> hosts[$num_hosts]=gateway
>
> # and also tjener...
> hosts[$num_hosts+1]=tjener
>
> printf -v hosts_str -- ',,%q' "${hosts[@]}"
> hosts_str=$(echo $hosts_str | tr 'A-Z' 'a-z')
>
> for i in $(basename -a /etc/debian-edu/host-keytabs/* | sed
> 's#.intern.keytab##') ; do
> match_value=$(echo $i | tr 'A-Z' 'a-z')
> if [[ ! "${hosts_str},," =~ ",,${match_value},," ]]; then
> kadmin.local delprinc host/$i.intern@INTERN
> kadmin.local delprinc nfs/$i.intern@INTERN
> rm /etc/debian-edu/host-keytabs/$i.intern.keytab
> fi
> done
> ```
>
> The match_value part in the loop got introduced. We need to keep the
> case-sensitive string for exactly matching the file name and the principal
> name. But for matching, we want all to be lower-case.
Good. Please commit.
Wolfgang
signature.asc
Description: PGP signature
Bug#935080: slapcat used in gosa hook script gosa-modify-host
Moin Mike, On Fri, May 01, 2020 at 06:18:42AM +, Mike Gabriel wrote: > I looked at your script snippet and modified it and tested it on a live > system with many hosts and LDAP being out-of-sync with the keytab files: [..] > Changes: > > * avoid the need of a TEMP file > * match LDAP hosts case insensitive Nice. > I wonder if this script (I tested it as a standalone script) should > not be better placed as a daily CRON job on TJENER (?) instead of it > being part of the gosa-modify-host hook script. Yes, could be. IIRC I opted against relying on a cron job and preferred to clean it up directly. (Maybe cron would not run in each case because people shut down even the main server at the end of a school day.) Wolfgang signature.asc Description: PGP signature
Bug#935080: slapcat used in gosa hook script gosa-modify-host
Hi again,
On Mo 27 Apr 2020 12:31:41 CEST, Wolfgang Schweer wrote:
Maybe you can test a follow-up one, based on your proposal to query LDAP
only once. (Host gateway considered too, just in case someone has added
a keytab.)
Here is a slight change...
```
#!/bin/bash
set -e
declare -a hosts
num_hosts=0
while read KEY VALUE ; do
case "$KEY" in
dn:) let "num_hosts+=1" ;;
cn:) hosts[$(($num_hosts-1))]="$VALUE" ;;
"")
:
;;
esac
done < <(ldapsearch -xLLL "objectclass=dhcpHost")
# add gateway host manually
hosts[$num_hosts]=gateway
# and also tjener...
hosts[$num_hosts+1]=tjener
printf -v hosts_str -- ',,%q' "${hosts[@]}"
hosts_str=$(echo $hosts_str | tr 'A-Z' 'a-z')
for i in $(basename -a /etc/debian-edu/host-keytabs/* | sed
's#.intern.keytab##') ; do
match_value=$(echo $i | tr 'A-Z' 'a-z')
if [[ ! "${hosts_str},," =~ ",,${match_value},," ]]; then
kadmin.local delprinc host/$i.intern@INTERN
kadmin.local delprinc nfs/$i.intern@INTERN
rm /etc/debian-edu/host-keytabs/$i.intern.keytab
fi
done
```
The match_value part in the loop got introduced. We need to keep the
case-sensitive string for exactly matching the file name and the
principal name. But for matching, we want all to be lower-case.
Mike
--
DAS-NETZWERKTEAM
c\o Technik- und Ökologiezentrum Eckernförde
Mike Gabriel, Marienthaler Str. 17, 24340 Eckernförde
mobile: +49 (1520) 1976 148
landline: +49 (4351) 850 8940
GnuPG Fingerprint: 9BFB AEE8 6C0A A5FF BF22 0782 9AF4 6B30 2577 1B31
mail: mike.gabr...@das-netzwerkteam.de, http://das-netzwerkteam.de
pgpPRcW5phv_j.pgp
Description: Digitale PGP-Signatur
Bug#935080: slapcat used in gosa hook script gosa-modify-host
HI Wolfgang,
On Mo 27 Apr 2020 12:31:41 CEST, Wolfgang Schweer wrote:
Moin Mike,
On Sun, Apr 19, 2020 at 11:43:24AM +, Mike Gabriel wrote:
On my customer site, we have 300 host entries or so. Doing the above
loop for every host change in LDAP will let the script walk over 300
LDAP queries. The response time of GOsa becomes awful with that.
[..]
During the next days, I can test your patch, either the one or above
or a follow-up version.
Maybe you can test a follow-up one, based on your proposal to query LDAP
only once. (Host gateway considered too, just in case someone has added
a keytab.)
Wolfgang
I looked at your script snippet and modified it and tested it on a
live system with many hosts and LDAP being out-of-sync with the keytab
files:
```
#!/bin/bash
set -e
declare -a hosts
num_hosts=0
while read KEY VALUE ; do
case "$KEY" in
dn:) let "num_hosts+=1" ;;
cn:) hosts[$(($num_hosts-1))]="$VALUE" ;;
"")
:
;;
esac
done < <(ldapsearch -xLLL "objectclass=dhcpHost")
# add gateway host manually
hosts[$num_hosts]=gateway
# and also tjener...
hosts[$num_hosts+1]=tjener
printf -v hosts_str -- ',,%q' "${hosts[@]}"
hosts_str=$(echo $hosts_str | tr 'A-Z' 'a-z')
for i in $(basename -a /etc/debian-edu/host-keytabs/* | sed
's#.intern.keytab##' | tr 'A-Z' 'a-z') ; do
if [[ ! "${hosts_str},," =~ ",,$i,," ]]; then
kadmin.local delprinc host/$i.intern@INTERN
kadmin.local delprinc nfs/$i.intern@INTERN
rm /etc/debian-edu/host-keytabs/$i.intern.keytab
fi
done
```
Changes:
* avoid the need of a TEMP file
* match LDAP hosts case insensitive
I wonder if this script (I tested it as a standalone script) should
not be better placed as a daily CRON job on TJENER (?) instead of it
being part of the gosa-modify-host hook script.
Mike
--
DAS-NETZWERKTEAM
c\o Technik- und Ökologiezentrum Eckernförde
Mike Gabriel, Marienthaler Str. 17, 24340 Eckernförde
mobile: +49 (1520) 1976 148
landline: +49 (4351) 850 8940
GnuPG Fingerprint: 9BFB AEE8 6C0A A5FF BF22 0782 9AF4 6B30 2577 1B31
mail: mike.gabr...@das-netzwerkteam.de, http://das-netzwerkteam.de
pgppTCInoZZpn.pgp
Description: Digitale PGP-Signatur
Bug#935080: slapcat used in gosa hook script gosa-modify-host
Moin Mike,
On Sun, Apr 19, 2020 at 11:43:24AM +, Mike Gabriel wrote:
> On my customer site, we have 300 host entries or so. Doing the above
> loop for every host change in LDAP will let the script walk over 300
> LDAP queries. The response time of GOsa becomes awful with that.
[..]
> During the next days, I can test your patch, either the one or above
> or a follow-up version.
Maybe you can test a follow-up one, based on your proposal to query LDAP
only once. (Host gateway considered too, just in case someone has added
a keytab.)
Wolfgang
#!/bin/sh
set -e
## This script is run by www-data using sudo. Keep that in mind!
## Make sure that malicious execution cannot hurt.##
HOST="$1"
kadmin.local -q "add_principal -policy hosts -randkey host/$HOST.intern"
kadmin.local -q "ktadd -k /etc/debian-edu/host-keytabs/$HOST.intern.keytab
host/$HOST.intern"
kadmin.local -q "add_principal -policy hosts -randkey nfs/$HOST.intern"
kadmin.local -q "ktadd -k /etc/debian-edu/host-keytabs/$HOST.intern.keytab
nfs/$HOST.intern"
logger -t gosa-modify-host -p notice Krb5 principals and keytab file for host
\'$HOST\' created.
# update services:
/usr/share/debian-edu-config/tools/gosa-sync-dns-nfs
# cleanup from leftover host principals and keytab file:
TEMPDIR=$(mktemp -d)
ldapsearch -xLLL "objectclass=dhcpHost" | grep ^cn: | cut -d' ' -f2 >
${TEMPDIR}/hosts
echo gateway >> ${TEMPDIR}/hosts
for i in $(basename -a /etc/debian-edu/host-keytabs/* | sed
's#.intern.keytab##') ; do
if ! grep -q $i ${TEMPDIR}/hosts ; then
kadmin.local delprinc host/$i.intern@INTERN
kadmin.local delprinc nfs/$i.intern@INTERN
rm /etc/debian-edu/host-keytabs/$i.intern.keytab
fi
done
exit 0
signature.asc
Description: PGP signature
Bug#935080: slapcat used in gosa hook script gosa-modify-host
Hi Wolfgang, thanks for working on this. On So 19 Apr 2020 11:40:51 CEST, Wolfgang Schweer wrote: On Mon, Aug 19, 2019 at 01:33:31PM +, Mike Gabriel wrote: On Mo 19 Aug 2019 12:56:11 CEST, Petter Reinholdtsen wrote: > Note, I have no idea why slapcat is used in the script to locate hosts: > > # cleanup from leftover host principals and keytab file: > for i in $(basename -a /etc/debian-edu/host-keytabs/* | sed > 's#.intern.keytab##') ; do > if slapcat | grep $i | grep -q dhcp ; then > : > else > kadmin.local delprinc host/$i.intern@INTERN > kadmin.local delprinc nfs/$i.intern@INTERN > rm /etc/debian-edu/host-keytabs/$i.intern.keytab > fi > done Using slapcat here is wrong, it should be proper LDAP db queries with specific search pattern. Maybe this would be better: # cleanup from leftover host principals and keytab file: for i in $(basename -a /etc/debian-edu/host-keytabs/* | sed 's#.intern.keytab##') ; do if [ "" = "$(ldapsearch -xLLL "(&(objectclass=dhcpHost)(cn=$i))")" ] ; then kadmin.local delprinc host/$i.intern@INTERN kadmin.local delprinc nfs/$i.intern@INTERN rm /etc/debian-edu/host-keytabs/$i.intern.keytab fi done Works for me, expert feedback appreciated. The problem with above approach (without having tested it) is scalability. For each host keytab you do an ldapsearch query. IMHO it would be much better to either (a) do it and fork that part of the script to background ( (...)& semantics in bash) or (b) do one LDAP query and try to cleanup host keytabs from that In fact, doing both might do the trick. On my customer site, we have 300 host entries or so. Doing the above loop for every host change in LDAP will let the script walk over 300 LDAP queries. The response time of GOsa becomes awful with that. During the next days, I can test your patch, either the one or above or a follow-up version. > I have no idea why Wolfgang decided to use slapcat instead of ldapsearch > here. Perhaps to make sure he is operating on the local LDAP database, > or because he did not have the LDAP connection details available in the > script? I wasn't aware of possible drawbacks like the ones Mike reported. ;-) IMHO, the LDAP db will answer anonymous queries just right when it comes to DHCP hosts. Yes, seems to be so, see the proposed change above. ;-) Mike -- DAS-NETZWERKTEAM c\o Technik- und Ökologiezentrum Eckernförde Mike Gabriel, Marienthaler Str. 17, 24340 Eckernförde mobile: +49 (1520) 1976 148 landline: +49 (4351) 850 8940 GnuPG Fingerprint: 9BFB AEE8 6C0A A5FF BF22 0782 9AF4 6B30 2577 1B31 mail: mike.gabr...@das-netzwerkteam.de, http://das-netzwerkteam.de pgpzx34m4nQUU.pgp Description: Digitale PGP-Signatur
Bug#935080: slapcat used in gosa hook script gosa-modify-host
On Mon, Aug 19, 2019 at 01:33:31PM +, Mike Gabriel wrote: > On Mo 19 Aug 2019 12:56:11 CEST, Petter Reinholdtsen wrote: > > Note, I have no idea why slapcat is used in the script to locate hosts: > > > > # cleanup from leftover host principals and keytab file: > > for i in $(basename -a /etc/debian-edu/host-keytabs/* | sed > > 's#.intern.keytab##') ; do > > if slapcat | grep $i | grep -q dhcp ; then > > : > > else > > kadmin.local delprinc host/$i.intern@INTERN > > kadmin.local delprinc nfs/$i.intern@INTERN > > rm /etc/debian-edu/host-keytabs/$i.intern.keytab > > fi > > done > > Using slapcat here is wrong, it should be proper LDAP db queries with > specific search pattern. Maybe this would be better: # cleanup from leftover host principals and keytab file: for i in $(basename -a /etc/debian-edu/host-keytabs/* | sed 's#.intern.keytab##') ; do if [ "" = "$(ldapsearch -xLLL "(&(objectclass=dhcpHost)(cn=$i))")" ] ; then kadmin.local delprinc host/$i.intern@INTERN kadmin.local delprinc nfs/$i.intern@INTERN rm /etc/debian-edu/host-keytabs/$i.intern.keytab fi done Works for me, expert feedback appreciated. > > I have no idea why Wolfgang decided to use slapcat instead of ldapsearch > > here. Perhaps to make sure he is operating on the local LDAP database, > > or because he did not have the LDAP connection details available in the > > script? I wasn't aware of possible drawbacks like the ones Mike reported. > IMHO, the LDAP db will answer anonymous queries just right when it comes to > DHCP hosts. Yes, seems to be so, see the proposed change above. Wolfgang signature.asc Description: PGP signature
Bug#935080: slapcat used in gosa hook script gosa-modify-host
Hi Petter, On Mo 19 Aug 2019 12:56:11 CEST, Petter Reinholdtsen wrote: [Mike Gabriel] The slapcat tool is an offline administration tool for LDAP and should not be used for day-to-day online tasks. Care to explain this argument a bit more? I fail to see why slapcat should have a different status from any other tools available, for use in day-to-day tasks as the developer see fit. When using slapcat, you always dump the full local DB rather than searching for what you are looking for and leave the search to the DB. The loop quoted below dumps the full LDAP db for each host listed in /etc/debian-edu/host-keytabs. This is something between 80-100 hosts here. And this happens whenever the admin clicks "OK" on a GOsa system (for modifications). So, this scales badly. Is there some other reason not to use slapcat, in addition to it 'should not be used for day-to-day online tasks'? The script runs as "root" and everything in LDAP gets shown to the script with slapcat (and is grepped out, but still). Furthermore, the full DB gets dumped several times repetetively and the script runs (after clicking "OK" on a GOsa system) whenever a host gets modified (which causes a long delay on a populated school LDAP DB. Note, I have no idea why slapcat is used in the script to locate hosts: # cleanup from leftover host principals and keytab file: for i in $(basename -a /etc/debian-edu/host-keytabs/* | sed 's#.intern.keytab##') ; do if slapcat | grep $i | grep -q dhcp ; then : else kadmin.local delprinc host/$i.intern@INTERN kadmin.local delprinc nfs/$i.intern@INTERN rm /etc/debian-edu/host-keytabs/$i.intern.keytab fi done Using slapcat here is wrong, it should be proper LDAP db queries with specific search pattern. I have no idea why Wolfgang decided to use slapcat instead of ldapsearch here. Perhaps to make sure he is operating on the local LDAP database, or because he did not have the LDAP connection details available in the script? IMHO, the LDAP db will answer anonymous queries just right when it comes to DHCP hosts. @Wolfgang: feedback? Mike -- DAS-NETZWERKTEAM c\o Technik- und Ökologiezentrum Eckernförde Mike Gabriel, Marienthaler str. 17, 24340 Eckernförde mobile: +49 (1520) 1976 148 landline: +49 (4351) 850 8940 GnuPG Fingerprint: 9BFB AEE8 6C0A A5FF BF22 0782 9AF4 6B30 2577 1B31 mail: mike.gabr...@das-netzwerkteam.de, http://das-netzwerkteam.de pgpD8BmzXXSu5.pgp Description: Digitale PGP-Signatur
Bug#935080: slapcat used in gosa hook script gosa-modify-host
[Mike Gabriel] > The slapcat tool is an offline administration tool for LDAP and should > not be used for day-to-day online tasks. Care to explain this argument a bit more? I fail to see why slapcat should have a different status from any other tools available, for use in day-to-day tasks as the developer see fit. Is there some other reason not to use slapcat, in addition to it 'should not be used for day-to-day online tasks'? Note, I have no idea why slapcat is used in the script to locate hosts: # cleanup from leftover host principals and keytab file: for i in $(basename -a /etc/debian-edu/host-keytabs/* | sed 's#.intern.keytab##') ; do if slapcat | grep $i | grep -q dhcp ; then : else kadmin.local delprinc host/$i.intern@INTERN kadmin.local delprinc nfs/$i.intern@INTERN rm /etc/debian-edu/host-keytabs/$i.intern.keytab fi done I have no idea why Wolfgang decided to use slapcat instead of ldapsearch here. Perhaps to make sure he is operating on the local LDAP database, or because he did not have the LDAP connection details available in the script? -- Vennlig hilsen Petter Reinholdtsen
Bug#935080: slapcat used in gosa hook script gosa-modify-host
Package: debian-edu-config Severity: important Version: 2.10.67 Hi all, the gosa-modify-host script utilizes slapcat to check LDAP for removed hosts: ``` www-data@tjener:~$ /usr/bin/sudo /usr/share/debian-edu-config/tools/gosa-modify-host && echo TRUE + slapcat + grep dell007.intern. + grep -q dhcp + kadmin.local delprinc host/dell007.intern..intern@INTERN delete_principal: Principal does not exist while deleting principal "host/dell007.intern..intern@INTERN" ``` 1. The slapcat tool is an offline administration tool for LDAP and should not be used for day-to-day online tasks. The part where it is used is about cleaning up the /etc/debian-edu/host-keytabs/ directory. The better approach here is going over all files in that folder and doing an ldapsearch query for each host. 2. Furthermore, this should not be done in gosa-modify-host, but in a separate CRON job that does not inflict on the execution time of the script. 3. Last but not least, the loop over all slapcat'ed hosts aborts if some error occurs and thus ends in the middle of the run. This needs some more love, I guess. Mike -- DAS-NETZWERKTEAM c\o Technik- und Ökologiezentrum Eckernförde Mike Gabriel, Marienthaler str. 17, 24340 Eckernförde mobile: +49 (1520) 1976 148 landline: +49 (4351) 850 8940 GnuPG Fingerprint: 9BFB AEE8 6C0A A5FF BF22 0782 9AF4 6B30 2577 1B31 mail: mike.gabr...@das-netzwerkteam.de, http://das-netzwerkteam.de pgpzOM8aFsrjh.pgp Description: Digitale PGP-Signatur
