[389-users] Re: unattended request cert process

[Fong, Trevor]

Hi abosch,

Have you looked at preconfiguring the cert db's with a wildcard cert from a CA 
to avoid the request/sign dance?  

Trev


On 2020-12-02, 1:04 AM, "Angel Bosch Mora"  wrote:

[CAUTION: Non-UBC Email]

> depending on your version of 389, look at "dsctl  tls
> import-ca"
> 
> {william@ldapkdc 9:12} ~/development $ dsctl localhost tls import-ca
> --help
> usage: dsctl [instance] tls import-ca [-h] cert_path nickname
> 
> positional arguments:
>   cert_path   The path to the x509 cert to import as a server CA
>   nicknameThe name of the certificate once imported
> 
> optional arguments:
>   -h, --help  show this help message and exit
> 
> This allows you to import a PEM  CA file. There are a number of other
> helpers under the tls subcommand to make cert management easier.
>


all this is pretty new, right?

I can't recall reading this last time I checked docs.

anyway, my main problem is that to deploy a node in a truly unattended mode 
It shouldn't pause at CSR request and continue when CA sign certificates, so 
I'm trying to have some preconfigured cert databases and signed certs.

If there's no way to do that, I can't dynamically create and destroy nodes.

the other option is letting the loadbalancer handle encryption, but 
official docs are very aggressive against that option, but I wonder if I should 
ignore that recommendation and encrypt at LB level.
any hints?

abosch
-- Institut Mallorqui d'Afers Socials. Aquest missatge, i si escau, 
qualsevol fitxer annex, es dirigeix exclusivament a la persona que n'es 
destinataria i pot contenir informacio confidencial. En cap cas no heu de 
copiar aquest missatge ni lliurar-lo a terceres persones sense permis expres de 
l'IMAS. Si no sou la persona destinataria que s'hi indica (o la responsable de 
lliurar-l'hi) us demanam que ho notifiqueu immediatament a l'adreca electronica 
de la persona remitent. Abans d'imprimir aquest missatge, pensau si es realment 
necessari.
___
389-users mailing list -- 389-users@lists.fedoraproject.org
To unsubscribe send an email to 389-users-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/389-users@lists.fedoraproject.org


___
389-users mailing list -- 389-users@lists.fedoraproject.org
To unsubscribe send an email to 389-users-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/389-users@lists.fedoraproject.org

[389-users] Re: Re-initialization Failure: Bulk Import Abandoned, Thread Monitoring Returned -23

[Fong, Trevor]

Thanks William.
Just wanted to make an update to report that recovery via ldif2db was 
successful last night.

Thanks,
Trev

On 2020-09-17, 7:37 PM, "William Brown"  wrote:


> I don't want to do an db2ldif during production time since it will put 
the db into read-only mode.
> The primary provider will do a backup at 01:00 (with -r) so I'll grab 
that and try to ldif2db the secondary provider with it.
> 

Yep, that seems like your best course of action at this point. :) 

—
Sincerely,

William Brown

Senior Software Engineer, 389 Directory Server
SUSE Labs
___
389-users mailing list -- 389-users@lists.fedoraproject.org
To unsubscribe send an email to 389-users-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/389-users@lists.fedoraproject.org


___
389-users mailing list -- 389-users@lists.fedoraproject.org
To unsubscribe send an email to 389-users-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/389-users@lists.fedoraproject.org

[389-users] Re: Re-initialization Failure: Bulk Import Abandoned, Thread Monitoring Returned -23

[Fong, Trevor]

The below bug linked to this bug: https://pagure.io/389-ds-base/issue/49915
Running "top -H", I could see that a thread hit 100% CPU on the primary 
provider during re-initialization of the secondary provider:

top - 19:14:37 up 13 days,  5:00,  2 users,  load average: 0.75, 0.39, 0.64
Threads: 406 total,   2 running, 404 sleeping,   0 stopped,   0 zombie
%Cpu(s): 29.4 us,  0.2 sy,  0.0 ni, 70.4 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
KiB Mem : 41027760 total,   274436 free, 26755384 used, 13997940 buff/cache
KiB Swap:  4095996 total,  4007932 free,88064 used. 13755740 avail Mem

  PID USER  PR  NIVIRTRESSHR S %CPU %MEM TIME+ COMMAND
1324 dirsrv20   0   29.3g  26.4g   1.7g S  0.0 67.5  32:09.15 ns-slapd
1422 dirsrv20   0   29.3g  26.4g   1.7g S  0.0 67.5   7:13.29 ns-slapd
1423 dirsrv20   0   29.3g  26.4g   1.7g S  0.0 67.5   4:49.31 ns-slapd
1424 dirsrv20   0   29.3g  26.4g   1.7g S 17.6 67.5   3440:05 ns-slapd
1425 dirsrv20   0   29.3g  26.4g   1.7g S  0.0 67.5   0:38.04 ns-slapd
1427 dirsrv20   0   29.3g  26.4g   1.7g S  0.0 67.5   0:00.00 ns-slapd
1428 dirsrv20   0   29.3g  26.4g   1.7g S  0.0 67.5   0:13.75 ns-slapd
1429 dirsrv20   0   29.3g  26.4g   1.7g S  0.0 67.5   1:58.45 ns-slapd
1430 dirsrv20   0   29.3g  26.4g   1.7g S  0.0 67.5   1:53.75 ns-slapd
1431 dirsrv20   0   29.3g  26.4g   1.7g S  0.0 67.5   1:55.51 ns-slapd
1432 dirsrv20   0   29.3g  26.4g   1.7g S  0.0 67.5   1:56.32 ns-slapd
1433 dirsrv20   0   29.3g  26.4g   1.7g S  0.0 67.5   1:55.66 ns-slapd
1434 dirsrv20   0   29.3g  26.4g   1.7g S  0.0 67.5   2:11.95 ns-slapd
1435 dirsrv20   0   29.3g  26.4g   1.7g R 99.9 67.5  97:37.64 ns-slapd
1436 dirsrv20   0   29.3g  26.4g   1.7g S  0.0 67.5   0:00.00 ns-slapd
1437 dirsrv20   0   29.3g  26.4g   1.7g S  0.0 67.5   0:00.00 ns-slapd

I don't want to do an db2ldif during production time since it will put the db 
into read-only mode.
The primary provider will do a backup at 01:00 (with -r) so I'll grab that and 
try to ldif2db the secondary provider with it.

Thanks,
Trev


From: "Fong, Trevor" 
Reply-To: "389-users@lists.fedoraproject.org" 
<389-users@lists.fedoraproject.org>
Date: Thursday, September 17, 2020 at 7:02 PM
To: "389-users@lists.fedoraproject.org" <389-users@lists.fedoraproject.org>
Subject: [389-users] Re: Re-initialization Failure: Bulk Import Abandoned, 
Thread Monitoring Returned -23

Hi Marc,

Thanks for your suggestions.
We had disabled autotuning, but even re-enabling autotuning we are seeing the 
same errors.

Just wondering if we are hitting this bug? 
https://pagure.io/389-ds-base/issue/49796

Thanks everyone!
Trev

From: Marc Sauton 
Reply-To: "389-users@lists.fedoraproject.org" 
<389-users@lists.fedoraproject.org>
Date: Thursday, September 17, 2020 at 5:23 PM
To: "389-users@lists.fedoraproject.org" <389-users@lists.fedoraproject.org>
Subject: [389-users] Re: Re-initialization Failure: Bulk Import Abandoned, 
Thread Monitoring Returned -23

The average rate and error code seem to point to a configuration issue.
Try to verify the import and cache settings in the console, they may be very 
small if the auto tuning feature is disabled.
for manual settings, the values depend on the amount of RAM available to the 
LDAP server, and the db, index sizes.

"small" db example of parameters and values:

dn: cn=config,cn=ldbm database,cn=plugins,cn=config
nsslapd-import-cachesize: 5

dn: cn=userroot,cn=ldbm database,cn=plugins,cn=config
nsslapd-cachememsize: 134217728

dn: cn=bdb,cn=config,cn=ldbm database,cn=plugins,cn=config
nsslapd-import-cache-autosize: 0
nsslapd-import-cachesize: 5
nsslapd-cache-autosize: 10
nsslapd-cache-autosize-split: 25
nsslapd-dbcachesize: 38314475

documentation references:
https://access.redhat.com/documentation/en-us/red_hat_directory_server/11/html/performance_tuning_guide/memoryusage#DB_and_entry_cache_RAM_usage
https://access.redhat.com/documentation/en-us/red_hat_directory_server/11/html/configuration_command_and_file_reference/database_plug_in_attributes#nsslapd_cache_autosize


Thanks,
M.

On Thu, Sep 17, 2020 at 4:10 PM Fong, Trevor 
mailto:trevor.f...@ubc.ca>> wrote:
Hi Everyone,

I'm having an issue re-initializing our secondary muti-master replicated 389 DS 
provider node via 389-Console > replication agreement > "Initialize Consumer".
It eventually aborts the update with an error "import userRoot: Thread 
monitoring returned: -23"
Would anyone know how to fix it?  Or what the issue may be?  What we could try?

Thanks in advance,
Trev

We're running 389-Directory/1.3.10.1<http://1.3.10.1> B2020.167.146
Here's that the error log says on the node receiving re-initialization:

[17/Sep/2020:15:21:07.489737002 -0700] - NOTICE - NSMMReplicationPlugin - 
multimaster_be_state_change - Replica dc= is going offline

[389-users] Re: Re-initialization Failure: Bulk Import Abandoned, Thread Monitoring Returned -23

[Fong, Trevor]

Hi Marc,

Thanks for your suggestions.
We had disabled autotuning, but even re-enabling autotuning we are seeing the 
same errors.

Just wondering if we are hitting this bug? 
https://pagure.io/389-ds-base/issue/49796

Thanks everyone!
Trev

From: Marc Sauton 
Reply-To: "389-users@lists.fedoraproject.org" 
<389-users@lists.fedoraproject.org>
Date: Thursday, September 17, 2020 at 5:23 PM
To: "389-users@lists.fedoraproject.org" <389-users@lists.fedoraproject.org>
Subject: [389-users] Re: Re-initialization Failure: Bulk Import Abandoned, 
Thread Monitoring Returned -23

The average rate and error code seem to point to a configuration issue.
Try to verify the import and cache settings in the console, they may be very 
small if the auto tuning feature is disabled.
for manual settings, the values depend on the amount of RAM available to the 
LDAP server, and the db, index sizes.

"small" db example of parameters and values:

dn: cn=config,cn=ldbm database,cn=plugins,cn=config
nsslapd-import-cachesize: 5

dn: cn=userroot,cn=ldbm database,cn=plugins,cn=config
nsslapd-cachememsize: 134217728

dn: cn=bdb,cn=config,cn=ldbm database,cn=plugins,cn=config
nsslapd-import-cache-autosize: 0
nsslapd-import-cachesize: 5
nsslapd-cache-autosize: 10
nsslapd-cache-autosize-split: 25
nsslapd-dbcachesize: 38314475

documentation references:
https://access.redhat.com/documentation/en-us/red_hat_directory_server/11/html/performance_tuning_guide/memoryusage#DB_and_entry_cache_RAM_usage
https://access.redhat.com/documentation/en-us/red_hat_directory_server/11/html/configuration_command_and_file_reference/database_plug_in_attributes#nsslapd_cache_autosize


Thanks,
M.

On Thu, Sep 17, 2020 at 4:10 PM Fong, Trevor 
mailto:trevor.f...@ubc.ca>> wrote:
Hi Everyone,

I'm having an issue re-initializing our secondary muti-master replicated 389 DS 
provider node via 389-Console > replication agreement > "Initialize Consumer".
It eventually aborts the update with an error "import userRoot: Thread 
monitoring returned: -23"
Would anyone know how to fix it?  Or what the issue may be?  What we could try?

Thanks in advance,
Trev

We're running 389-Directory/1.3.10.1<http://1.3.10.1> B2020.167.146
Here's that the error log says on the node receiving re-initialization:

[17/Sep/2020:15:21:07.489737002 -0700] - NOTICE - NSMMReplicationPlugin - 
multimaster_be_state_change - Replica dc= is going offline; disabling 
replication
[17/Sep/2020:15:21:07.514586270 -0700] - INFO - dblayer_instance_start - Import 
is running with nsslapd-db-private-import-mem on; No other process is allowed 
to access the database
[17/Sep/2020:15:21:27.554375232 -0700] - INFO - import_monitor_threads - import 
userRoot: Processed 1 entries -- average rate 0.1/sec, recent rate 0.0/sec, hit 
ratio 0%
[17/Sep/2020:15:21:47.593876983 -0700] - INFO - import_monitor_threads - import 
userRoot: Processed 1 entries -- average rate 0.0/sec, recent rate 0.0/sec, hit 
ratio 0%
[17/Sep/2020:15:22:07.630801176 -0700] - INFO - import_monitor_threads - import 
userRoot: Processed 1 entries -- average rate 0.0/sec, recent rate 0.0/sec, hit 
ratio 0%
[17/Sep/2020:15:22:27.667537260 -0700] - INFO - import_monitor_threads - import 
userRoot: Processed 1 entries -- average rate 0.0/sec, recent rate 0.0/sec, hit 
ratio 0%
[17/Sep/2020:15:22:47.704917493 -0700] - INFO - import_monitor_threads - import 
userRoot: Processed 1 entries -- average rate 0.0/sec, recent rate 0.0/sec, hit 
ratio 0%
[17/Sep/2020:15:23:07.746084506 -0700] - INFO - import_monitor_threads - import 
userRoot: Processed 1 entries -- average rate 0.0/sec, recent rate 0.0/sec, hit 
ratio 0%
[17/Sep/2020:15:23:27.785902082 -0700] - INFO - import_monitor_threads - import 
userRoot: Processed 1 entries -- average rate 0.0/sec, recent rate 0.0/sec, hit 
ratio 0%
[17/Sep/2020:15:23:47.830564570 -0700] - INFO - import_monitor_threads - import 
userRoot: Processed 1 entries -- average rate 0.0/sec, recent rate 0.0/sec, hit 
ratio 0%
[17/Sep/2020:15:24:07.868457613 -0700] - INFO - import_monitor_threads - import 
userRoot: Processed 1 entries -- average rate 0.0/sec, recent rate 0.0/sec, hit 
ratio 0%
[17/Sep/2020:15:24:27.907239617 -0700] - INFO - import_monitor_threads - import 
userRoot: Processed 1 entries -- average rate 0.0/sec, recent rate 0.0/sec, hit 
ratio 0%
[17/Sep/2020:15:24:47.948025735 -0700] - INFO - import_monitor_threads - import 
userRoot: Processed 1 entries -- average rate 0.0/sec, recent rate 0.0/sec, hit 
ratio 0%
[17/Sep/2020:15:25:07.986469285 -0700] - INFO - import_monitor_threads - import 
userRoot: Processed 1 entries -- average rate 0.0/sec, recent rate 0.0/sec, hit 
ratio 0%
[17/Sep/2020:15:25:28.022970040 -0700] - INFO - import_monitor_threads - import 
userRoot: Processed 1 entries -- average rate 0.0/sec, recent rate 0.0/sec, hit 
ratio 0%
[17/Sep/2020:15:25:48.055641123 -0700] - INFO - import_monitor_thre

[389-users] Re: Re-initialization Failure: Bulk Import Abandoned, Thread Monitoring Returned -23

[Fong, Trevor]

Hi William,

Connectivity between the two machines is good.
The replication agreement from primary provider to secondary provider is OK too.
It was working this morning until I decided to re-initialize the secondary 
provider to resolve some data inconsistencies.

Trev


On 2020-09-17, 5:05 PM, "William Brown"  wrote:

Yes, I think it does. So in that case it would be good to check the 
replication configurations and connectivity between the servers is good to 
eliminate that as a possible cause. 

> On 18 Sep 2020, at 10:02, Fong, Trevor  wrote:
> 
> Hi William,
> 
> Thanks for your suggestion about db2ldif.  
> Doesn't db2ldif put the server into read-only mode?  We would want to 
avoid that as that is our primary provider.
> 
> Thanks,
> Trev
> 
> On 2020-09-17, 4:52 PM, "William Brown"  wrote:
> 
    >    Hi there,
> 
>> On 18 Sep 2020, at 09:09, Fong, Trevor  wrote:
>> 
>> Hi Everyone,
>> 
>> I'm having an issue re-initializing our secondary muti-master replicated 
389 DS provider node via 389-Console > replication agreement > "Initialize 
Consumer".
>> It eventually aborts the update with an error "import userRoot: Thread 
monitoring returned: -23"
>> Would anyone know how to fix it?  Or what the issue may be?  What we 
could try?
> 
>Error -23 looks like "#define ERR_IMPORT_ABORTED -23". 
> 
>An alternate method to re-init the consumer is from the current good 
active server perform:
> 
>db2ldif -r 
> 
>On the server you want to initialise, take the ldif that is generated 
from the above command and do:
> 
>ldif2db 
> 
>The -r flag is the key, it adds the replication metadata here so it 
acts as a replica re-init.
> 
>It's probably worth also checking connectivity between the servers to 
be sure everything is working, it seems odd that your import only imported a 
single entry  Is the replication configs all consistent and complete in the 
topology? 
> 
>> 
>> Thanks in advance,
>> Trev
>> 
>> We're running 389-Directory/1.3.10.1 B2020.167.146
>> Here's that the error log says on the node receiving re-initialization:
>> 
>> [17/Sep/2020:15:21:07.489737002 -0700] - NOTICE - NSMMReplicationPlugin 
- multimaster_be_state_change - Replica dc= is going offline; 
disabling replication
>> [17/Sep/2020:15:21:07.514586270 -0700] - INFO - dblayer_instance_start - 
Import is running with nsslapd-db-private-import-mem on; No other process is 
allowed to access the database
>> [17/Sep/2020:15:21:27.554375232 -0700] - INFO - import_monitor_threads - 
import userRoot: Processed 1 entries -- average rate 0.1/sec, recent rate 
0.0/sec, hit ratio 0%
>> [17/Sep/2020:15:21:47.593876983 -0700] - INFO - import_monitor_threads - 
import userRoot: Processed 1 entries -- average rate 0.0/sec, recent rate 
0.0/sec, hit ratio 0%
>> [17/Sep/2020:15:22:07.630801176 -0700] - INFO - import_monitor_threads - 
import userRoot: Processed 1 entries -- average rate 0.0/sec, recent rate 
0.0/sec, hit ratio 0%
>> [17/Sep/2020:15:22:27.667537260 -0700] - INFO - import_monitor_threads - 
import userRoot: Processed 1 entries -- average rate 0.0/sec, recent rate 
0.0/sec, hit ratio 0%
>> [17/Sep/2020:15:22:47.704917493 -0700] - INFO - import_monitor_threads - 
import userRoot: Processed 1 entries -- average rate 0.0/sec, recent rate 
0.0/sec, hit ratio 0%
>> [17/Sep/2020:15:23:07.746084506 -0700] - INFO - import_monitor_threads - 
import userRoot: Processed 1 entries -- average rate 0.0/sec, recent rate 
0.0/sec, hit ratio 0%
>> [17/Sep/2020:15:23:27.785902082 -0700] - INFO - import_monitor_threads - 
import userRoot: Processed 1 entries -- average rate 0.0/sec, recent rate 
0.0/sec, hit ratio 0%
>> [17/Sep/2020:15:23:47.830564570 -0700] - INFO - import_monitor_threads - 
import userRoot: Processed 1 entries -- average rate 0.0/sec, recent rate 
0.0/sec, hit ratio 0%
>> [17/Sep/2020:15:24:07.868457613 -0700] - INFO - import_monitor_threads - 
import userRoot: Processed 1 entries -- average rate 0.0/sec, recent rate 
0.0/sec, hit ratio 0%
>> [17/Sep/2020:15:24:27.907239617 -0700] - INFO - import_monitor_threads - 
import userRoot: Processed 1 entries -- average rate 0.0/sec, recent rate 
0.0/sec, hit ratio 0%
>> [17/Sep/2020:15:24:47.948025735 -0700] - INFO - import_monitor_threads - 
import userRoot: Processed 1 entries -- average rate 0.0/sec, recent rate 
0.0/sec, hit ratio 0%
>> [17/Sep/2020:15:25:07.986469285 -0700] - INFO - import_monitor_threads - 
import userRoot:

[389-users] Re: Re-initialization Failure: Bulk Import Abandoned, Thread Monitoring Returned -23

[Fong, Trevor]

Hi William,

Thanks for your suggestion about db2ldif.  
Doesn't db2ldif put the server into read-only mode?  We would want to avoid 
that as that is our primary provider.

Thanks,
Trev

On 2020-09-17, 4:52 PM, "William Brown"  wrote:

Hi there,

> On 18 Sep 2020, at 09:09, Fong, Trevor  wrote:
> 
> Hi Everyone,
>  
> I'm having an issue re-initializing our secondary muti-master replicated 
389 DS provider node via 389-Console > replication agreement > "Initialize 
Consumer".
> It eventually aborts the update with an error "import userRoot: Thread 
monitoring returned: -23"
> Would anyone know how to fix it?  Or what the issue may be?  What we 
could try?

Error -23 looks like "#define ERR_IMPORT_ABORTED -23". 

An alternate method to re-init the consumer is from the current good active 
server perform:

db2ldif -r 

On the server you want to initialise, take the ldif that is generated from 
the above command and do:

ldif2db 

The -r flag is the key, it adds the replication metadata here so it acts as 
a replica re-init.

It's probably worth also checking connectivity between the servers to be 
sure everything is working, it seems odd that your import only imported a 
single entry  Is the replication configs all consistent and complete in the 
topology? 

>  
> Thanks in advance,
> Trev
>  
> We're running 389-Directory/1.3.10.1 B2020.167.146
> Here's that the error log says on the node receiving re-initialization:
>  
> [17/Sep/2020:15:21:07.489737002 -0700] - NOTICE - NSMMReplicationPlugin - 
multimaster_be_state_change - Replica dc= is going offline; disabling 
replication
> [17/Sep/2020:15:21:07.514586270 -0700] - INFO - dblayer_instance_start - 
Import is running with nsslapd-db-private-import-mem on; No other process is 
allowed to access the database
> [17/Sep/2020:15:21:27.554375232 -0700] - INFO - import_monitor_threads - 
import userRoot: Processed 1 entries -- average rate 0.1/sec, recent rate 
0.0/sec, hit ratio 0%
> [17/Sep/2020:15:21:47.593876983 -0700] - INFO - import_monitor_threads - 
import userRoot: Processed 1 entries -- average rate 0.0/sec, recent rate 
0.0/sec, hit ratio 0%
> [17/Sep/2020:15:22:07.630801176 -0700] - INFO - import_monitor_threads - 
import userRoot: Processed 1 entries -- average rate 0.0/sec, recent rate 
0.0/sec, hit ratio 0%
> [17/Sep/2020:15:22:27.667537260 -0700] - INFO - import_monitor_threads - 
import userRoot: Processed 1 entries -- average rate 0.0/sec, recent rate 
0.0/sec, hit ratio 0%
> [17/Sep/2020:15:22:47.704917493 -0700] - INFO - import_monitor_threads - 
import userRoot: Processed 1 entries -- average rate 0.0/sec, recent rate 
0.0/sec, hit ratio 0%
> [17/Sep/2020:15:23:07.746084506 -0700] - INFO - import_monitor_threads - 
import userRoot: Processed 1 entries -- average rate 0.0/sec, recent rate 
0.0/sec, hit ratio 0%
> [17/Sep/2020:15:23:27.785902082 -0700] - INFO - import_monitor_threads - 
import userRoot: Processed 1 entries -- average rate 0.0/sec, recent rate 
0.0/sec, hit ratio 0%
> [17/Sep/2020:15:23:47.830564570 -0700] - INFO - import_monitor_threads - 
import userRoot: Processed 1 entries -- average rate 0.0/sec, recent rate 
0.0/sec, hit ratio 0%
> [17/Sep/2020:15:24:07.868457613 -0700] - INFO - import_monitor_threads - 
import userRoot: Processed 1 entries -- average rate 0.0/sec, recent rate 
0.0/sec, hit ratio 0%
> [17/Sep/2020:15:24:27.907239617 -0700] - INFO - import_monitor_threads - 
import userRoot: Processed 1 entries -- average rate 0.0/sec, recent rate 
0.0/sec, hit ratio 0%
> [17/Sep/2020:15:24:47.948025735 -0700] - INFO - import_monitor_threads - 
import userRoot: Processed 1 entries -- average rate 0.0/sec, recent rate 
0.0/sec, hit ratio 0%
> [17/Sep/2020:15:25:07.986469285 -0700] - INFO - import_monitor_threads - 
import userRoot: Processed 1 entries -- average rate 0.0/sec, recent rate 
0.0/sec, hit ratio 0%
> [17/Sep/2020:15:25:28.022970040 -0700] - INFO - import_monitor_threads - 
import userRoot: Processed 1 entries -- average rate 0.0/sec, recent rate 
0.0/sec, hit ratio 0%
> [17/Sep/2020:15:25:48.055641123 -0700] - INFO - import_monitor_threads - 
import userRoot: Processed 1 entries -- average rate 0.0/sec, recent rate 
0.0/sec, hit ratio 0%
> [17/Sep/2020:15:26:08.091707793 -0700] - INFO - import_monitor_threads - 
import userRoot: Processed 1 entries -- average rate 0.0/sec, recent rate 
0.0/sec, hit ratio 0%
> [17/Sep/2020:15:26:08.093192417 -0700] - INFO - import_throw_in_towel - 
import userRoot: Decided to end this pass because the progress rate has dropped 
below the 50% threshold.
> [17/Sep/2020:15:26:08.094010231 -0700] - INFO - import_monitor_threads - 
i

[389-users] Re-initialization Failure: Bulk Import Abandoned, Thread Monitoring Returned -23

[Fong, Trevor]

Hi Everyone,

I'm having an issue re-initializing our secondary muti-master replicated 389 DS 
provider node via 389-Console > replication agreement > "Initialize Consumer".
It eventually aborts the update with an error "import userRoot: Thread 
monitoring returned: -23"
Would anyone know how to fix it?  Or what the issue may be?  What we could try?

Thanks in advance,
Trev

We're running 389-Directory/1.3.10.1 B2020.167.146
Here's that the error log says on the node receiving re-initialization:

[17/Sep/2020:15:21:07.489737002 -0700] - NOTICE - NSMMReplicationPlugin - 
multimaster_be_state_change - Replica dc= is going offline; disabling 
replication
[17/Sep/2020:15:21:07.514586270 -0700] - INFO - dblayer_instance_start - Import 
is running with nsslapd-db-private-import-mem on; No other process is allowed 
to access the database
[17/Sep/2020:15:21:27.554375232 -0700] - INFO - import_monitor_threads - import 
userRoot: Processed 1 entries -- average rate 0.1/sec, recent rate 0.0/sec, hit 
ratio 0%
[17/Sep/2020:15:21:47.593876983 -0700] - INFO - import_monitor_threads - import 
userRoot: Processed 1 entries -- average rate 0.0/sec, recent rate 0.0/sec, hit 
ratio 0%
[17/Sep/2020:15:22:07.630801176 -0700] - INFO - import_monitor_threads - import 
userRoot: Processed 1 entries -- average rate 0.0/sec, recent rate 0.0/sec, hit 
ratio 0%
[17/Sep/2020:15:22:27.667537260 -0700] - INFO - import_monitor_threads - import 
userRoot: Processed 1 entries -- average rate 0.0/sec, recent rate 0.0/sec, hit 
ratio 0%
[17/Sep/2020:15:22:47.704917493 -0700] - INFO - import_monitor_threads - import 
userRoot: Processed 1 entries -- average rate 0.0/sec, recent rate 0.0/sec, hit 
ratio 0%
[17/Sep/2020:15:23:07.746084506 -0700] - INFO - import_monitor_threads - import 
userRoot: Processed 1 entries -- average rate 0.0/sec, recent rate 0.0/sec, hit 
ratio 0%
[17/Sep/2020:15:23:27.785902082 -0700] - INFO - import_monitor_threads - import 
userRoot: Processed 1 entries -- average rate 0.0/sec, recent rate 0.0/sec, hit 
ratio 0%
[17/Sep/2020:15:23:47.830564570 -0700] - INFO - import_monitor_threads - import 
userRoot: Processed 1 entries -- average rate 0.0/sec, recent rate 0.0/sec, hit 
ratio 0%
[17/Sep/2020:15:24:07.868457613 -0700] - INFO - import_monitor_threads - import 
userRoot: Processed 1 entries -- average rate 0.0/sec, recent rate 0.0/sec, hit 
ratio 0%
[17/Sep/2020:15:24:27.907239617 -0700] - INFO - import_monitor_threads - import 
userRoot: Processed 1 entries -- average rate 0.0/sec, recent rate 0.0/sec, hit 
ratio 0%
[17/Sep/2020:15:24:47.948025735 -0700] - INFO - import_monitor_threads - import 
userRoot: Processed 1 entries -- average rate 0.0/sec, recent rate 0.0/sec, hit 
ratio 0%
[17/Sep/2020:15:25:07.986469285 -0700] - INFO - import_monitor_threads - import 
userRoot: Processed 1 entries -- average rate 0.0/sec, recent rate 0.0/sec, hit 
ratio 0%
[17/Sep/2020:15:25:28.022970040 -0700] - INFO - import_monitor_threads - import 
userRoot: Processed 1 entries -- average rate 0.0/sec, recent rate 0.0/sec, hit 
ratio 0%
[17/Sep/2020:15:25:48.055641123 -0700] - INFO - import_monitor_threads - import 
userRoot: Processed 1 entries -- average rate 0.0/sec, recent rate 0.0/sec, hit 
ratio 0%
[17/Sep/2020:15:26:08.091707793 -0700] - INFO - import_monitor_threads - import 
userRoot: Processed 1 entries -- average rate 0.0/sec, recent rate 0.0/sec, hit 
ratio 0%
[17/Sep/2020:15:26:08.093192417 -0700] - INFO - import_throw_in_towel - import 
userRoot: Decided to end this pass because the progress rate has dropped below 
the 50% threshold.
[17/Sep/2020:15:26:08.094010231 -0700] - INFO - import_monitor_threads - import 
userRoot: Ending pass number 1 ...
[17/Sep/2020:15:26:08.195022638 -0700] - INFO - import_monitor_threads - import 
userRoot: Foreman is done; waiting for workers to finish...
[17/Sep/2020:15:26:08.196221021 -0700] - INFO - import_monitor_threads - import 
userRoot: Workers finished; cleaning up...
[17/Sep/2020:15:26:08.397372463 -0700] - INFO - import_monitor_threads - import 
userRoot: Workers cleaned up.
[17/Sep/2020:15:26:08.398525016 -0700] - INFO - import_sweep_after_pass - 
import userRoot: Sweeping files for merging later...
[17/Sep/2020:15:26:08.409694998 -0700] - INFO - dblayer_instance_start - Import 
is running with nsslapd-db-private-import-mem on; No other process is allowed 
to access the database
[17/Sep/2020:15:26:08.411165210 -0700] - INFO - import_sweep_after_pass - 
import userRoot: Sweep done.
[17/Sep/2020:15:26:08.412153776 -0700] - INFO - import_main_offline - import 
userRoot: Beginning pass number 2
[17/Sep/2020:15:26:28.444537073 -0700] - INFO - import_monitor_threads - import 
userRoot: Processed 1 entries (pass 2) -- average rate 214748364.8/sec, recent 
rate 0.0/sec, hit ratio 0%
[17/Sep/2020:15:26:48.487800198 -0700] - INFO - import_monitor_threads - import 
userRoot: Processed 1 entries (pass 2) -- average rate 107374182.4/sec, recent 
rate 0.0/sec, hit ratio 0%

[389-users] Provider Node Not Restarting Following Failed Schema Update

[Fong, Trevor]

Hi There,

We tried to dynamically a new schema dynamically using 
/usr/lib64/dirsrv/slapd-eldapp1/schema-reload.pl
Unfortunately, (and unknown to us at the time) the objectClass definition 
misspelt a couple of the attribute names.
The schema reload process should have picked that up and refused it, but it 
didn't and so proceeded to update entries using the new schema.
That's when we started getting errors like the following in the error log:

[19/Jun/2020:10:28:08.390882389 -0700] - ERR - libdb - BDB0151 fsync: 
Input/output error
[19/Jun/2020:10:28:08.399523527 -0700] - ERR - libdb - BDB0151 fsync: 
Input/output error
[19/Jun/2020:10:28:08.404890880 -0700] - ERR - libdb - BDB0151 fsync: 
Input/output error
[19/Jun/2020:10:28:08.430284251 -0700] - ERR - libdb - BDB0151 fsync: 
Input/output error
[19/Jun/2020:10:28:08.466371449 -0700] - ERR - libdb - BDB0151 fsync: 
Input/output error
[19/Jun/2020:10:28:08.495859651 -0700] - ERR - libdb - BDB0151 fsync: 
Input/output error
[19/Jun/2020:10:28:08.522007224 -0700] - ERR - libdb - BDB0151 fsync: 
Input/output error
[19/Jun/2020:10:28:08.546930415 -0700] - ERR - libdb - BDB4519 txn_checkpoint: 
failed to flush the buffer cache: Input/output error
[19/Jun/2020:10:28:08.569781853 -0700] - CRIT - checkpoint_threadmain - Serious 
Error---Failed to checkpoint database, err=5 (Input/output error)

I tried restarting dirsrv and that's when it started giving errors about the 
unknown (misspelt) attributes in the new objectClass.
I fixed those errors in the schema and restarting dirsrv.
I saw the following message in the error log:

NOTICE - dblayer_start - Detected Disorderly Shutdown last time Directory 
Server was running, recovering database.

There was no further log, but the CPU utilization for ns-slapd was at 99.9% so 
I just let it run over night hoping that it wasn't stuck in a loop.
But there was no improvement the next morning, so I ordered a RAM increase from 
4 GB --> 16 GB hoping that would fix it, I let it run for a while with no 
indication of progress.
I also tried to run db2ldif to try to dump the db to an ldif file, but got the 
same "recovering database" message.  That's where it is now - I'll let it run 
for a few hours and hope it does something.

Would anyone be able to offer any further advice?
Is there any way to see how it's getting along with the database recovery?
Is this db well and truly hosed?
Unfortunately this system was spec'd for development so no backups were running 
so recovery from backup is not an option.

Thanks,
Trev


___
389-users mailing list -- 389-users@lists.fedoraproject.org
To unsubscribe send an email to 389-users-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/389-users@lists.fedoraproject.org

[389-users] Re: Setting Up 389 DS for Load Balancing with SSL Wildcard Certs

[Fong, Trevor]

Hi William,

Thanks very much for the clues.
I finally got it to work by:
1. Importing the wildcard cert's CA cert chain (I did each issuer as separate 
entries, but it might work concatenated into 1 file?)
  certutil -A -d . -n "CA ExternalCARoot" -t "CT,," -i ExternalCARoot.crt
  certutil -A -d . -n "CA Intermediate_CA" -t "CT,," -i IntermediateCA.crt
  certutil -A -d . -n "CA StandardSSLCA" -t "CT,," -i StandardSSLCA.crt
2. Convert wildcard cert's key to PKCS #12
  openssl pkcs12 -export -out star.example.com_key.pfx -inkey 
star.example.com.key -in star.example.com.crt -certfile StandardSSLCA2.crt
3. Import the wildcard cert's key
  pk12util -i star.example.com_key.pfx -d /etc/dirsrv/slapd-/ -W 
"password"
4. Find the name of the wildcard cert
  certutil -L -d . -f pwdfile.txt
5. Point 389 DS at the wildcard cert
  dn: cn=RSA,cn=encryption,cn=config
  changetype: modify
  replace: nsSSLPersonalitySSL
  nsSSLPersonalitySSL: *.example.com - CA
6. Restart 389 DS
  systemctl restart dirsrv.target

Thanks,
Trev



On 2019-07-26, 5:49 PM, "William Brown"  wrote:



> On 27 Jul 2019, at 07:50, Fong, Trevor  wrote:
> 
> Hi Everyone,
>  
> I've configured 2 new 389 DS hubs (eg new1.example.com, new2.example.com) 
and have connected them to our main 389 DS cluster.
> They each have their own self-signed certificate, and replication is 
working well.
>  
> I now want to load-balance these 2 nodes under their own VIP/hostname: 
downtown.example.com.
> I have added our wildcard cert for *.example.com to each node's NSS cert 
DB in /etc/dirsrv/slapd- to cover the "downtown.example.com" address.

You have to remove the existing "server-cert" alias in NSS DB, and add your 
wildcard cert/key with the name "server-cert". You'll need to make a P12 bundle 
with the right alias and then import it.


https://fy.blackhats.net.au/blog/html/pages/nss_and_openssl_command_reference.html

Check the "importing certificates to NSS" section, and the "basic listing" 
section. 

Hope that helps, 

>  
> However, querying the VIP's SSL, I see that the new node's self-signed 
cert is still presented instead of the wildcard:
>  
> $ echo | openssl s_client -connect downtown.example.com:636
> CONNECTED(0003)
> depth=1 CN = self-ca.example.com
> verify error:num=19:self signed certificate in certificate chain
> ---
> 
>  
> I thought that perhaps the node's own new1.example.com self-signed cert 
was taking precedence over the wildcard cert.
> But removing it resulted in:
>  
> $ echo | openssl s_client -connect downtown.example.com:636
> socket: Bad file descriptor
> connect:errno=9
>  
>  
> Would anyone be able to tell me how to achieve this correctly, or point 
me in the right/another direction?
>  
> Thanks a lot,
> Trev
>  
> ___
> 389-users mailing list -- 389-users@lists.fedoraproject.org
> To unsubscribe send an email to 389-users-le...@lists.fedoraproject.org
> Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives: 
https://lists.fedoraproject.org/archives/list/389-users@lists.fedoraproject.org

—
Sincerely,

William Brown

Senior Software Engineer, 389 Directory Server
SUSE Labs
___
389-users mailing list -- 389-users@lists.fedoraproject.org
To unsubscribe send an email to 389-users-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/389-users@lists.fedoraproject.org


___
389-users mailing list -- 389-users@lists.fedoraproject.org
To unsubscribe send an email to 389-users-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/389-users@lists.fedoraproject.org

[389-users] Setting Up 389 DS for Load Balancing with SSL Wildcard Certs

[Fong, Trevor]

Hi Everyone,

I've configured 2 new 389 DS hubs (eg new1.example.com, new2.example.com) and 
have connected them to our main 389 DS cluster.
They each have their own self-signed certificate, and replication is working 
well.

I now want to load-balance these 2 nodes under their own VIP/hostname: 
downtown.example.com.
I have added our wildcard cert for *.example.com to each node's NSS cert DB in 
/etc/dirsrv/slapd- to cover the "downtown.example.com" address.

However, querying the VIP's SSL, I see that the new node's self-signed cert is 
still presented instead of the wildcard:

$ echo | openssl s_client -connect downtown.example.com:636
CONNECTED(0003)
depth=1 CN = self-ca.example.com
verify error:num=19:self signed certificate in certificate chain
---


I thought that perhaps the node's own new1.example.com self-signed cert was 
taking precedence over the wildcard cert.
But removing it resulted in:

$ echo | openssl s_client -connect downtown.example.com:636
socket: Bad file descriptor
connect:errno=9


Would anyone be able to tell me how to achieve this correctly, or point me in 
the right/another direction?

Thanks a lot,
Trev

___
389-users mailing list -- 389-users@lists.fedoraproject.org
To unsubscribe send an email to 389-users-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/389-users@lists.fedoraproject.org

[389-users] Dynamic Group Query Not Returning Members

[Fong, Trevor]

Hi There,

I'm trying to set up dynamic groups with 389 DS (1.3.7.5 B2018.184.1411) but my 
queries against it don't seem to be returning any members.
I have a user set up like this:

objectClass: eduPerson
objectClass: inetLocalMailRecipient
objectClass: inetOrgPerson
objectClass: inetUser
objectClass: organizationalPerson
objectClass: person
objectClass: posixAccount
objectClass: top
cn: Test Abc
gidNumber: 
homeDirectory: /home/testabc
sn: Abc
uid: testabc
uidNumber: 
displayName: Test Abc
givenName: Test
loginShell: /bin/bash
userPassword:: 


I have my dynamic group set up like this:

objectClass: groupOfUniqueNames
objectClass: groupOfURLs
objectClass: top
cn: DynGroup-Test-UniqueMembers
description: uniqueMembers test
memberURL: ldap:///ou=PEOPLE,dc=test?objectClass?sub?(sn=Abc)


The memberURL:
ldap:///ou=PEOPLE,dc=test?objectClass?sub?(sn=Abc)
returns uid=testabc

However my query of the dynamic group doesn't return anything:
ldapsearch -x -D "cn=Directory Manager" -W -b " ou=Groups,dc=test" -s sub -a 
always -z 1000 "(uniqueMember=uid=testabc,ou=PEOPLE,dc=test)" "objectClass"


Thanks in advance for your help.
Trev

___
389-users mailing list -- 389-users@lists.fedoraproject.org
To unsubscribe send an email to 389-users-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/389-users@lists.fedoraproject.org

[389-users] Re: Replication Delay

[Fong, Trevor]

Hi Everyone,

Hazzah!  I've finally licked the slow (and erratic) replication between our 1.2 
-> 1.3 clusters! 
The problem was that when I was setting up the 1.3 cluster, I'd done it with a 
view to replace the 1.2 cluster.
In making that assumption, I'd set up the cluster in isolation.  Everything 
worked as it was supposed to, but I didn't occur to me to set the masters up 
with different replica ID's from those in the 1.2 cluster.  When I hooked the 
1.3 cluster up to the 1.2 cluster, replication into the 1.3 was slow and 
sometimes it would just break.

Rebuilding the 1.3 cluster with unique replica ID's for all master nodes across 
both clusters resolved the problem.

Thanks everyone for their helpful comments.
Trev 


On 2018-02-20, 4:13 PM, "Mark Reynolds" <mreyno...@redhat.com> wrote:



On 02/20/2018 06:53 PM, William Brown wrote:
> On Tue, 2018-02-20 at 23:36 +, Fong, Trevor wrote:
>> Hi William,
>>
>> Thanks a lot for your reply.
>>
>> That's correct - replication schedule is not enabled.
>> No - there are definitely changes to replicate - I know, I made the
>> change myself (
>> I changed the "description" attribute on an account, but it takes up
>> to 15 mins for the change to appear in the 1.3 master.
>> That master replicates to another master and a bunch of other hubs.
>> Those hubs replicate amongst themselves and a bunch of consumers.
> So to be correct in my understanding:
>
> 1.2 <-> 1.3 --> [ group of hubs/consumers ]
>
> Yes? 
>
>> The update can take up to 15 mins to make it from the 1.2 master,
>> into the 1.3 master; but once it hits the 1.3 master, it is
>> replicated around the 1.3 cluster within 1 sec.
>>
>> Only memberOf is disallowed for fractional replication.
>>
>> Can anyone give me any guidance as to the settings of the "backoff"
>> and other parameters?  Any doc links that may be useful?
> Mark? You wrote thisn, I can't remember what it's called 
Before we should adjust the back off min and max values, we need to
determine why 1.2.11 is having a hard time updating 1.3.6.  1.3.6 is
just receiving updates, so it's 1.2.11 that "seems" to be misbehaving. 
So... Is there anything in the errors log on 1.2.11?  It wouldn't hurt
to check 1.3.6, but I think 1.2.11 is where we will find our answer.

If there is noting in the log, then turn on replication logging and do
your test update.  Once the update hits 1.3.6 turn replication logging
off.  Then we can look at the logs and see what happens with your test
update.

But as requested here is the backoff min & max info:

http://www.port389.org/docs/389ds/design/replication-retry-settings.html
    
>
>> Thanks a lot,
>> Trev
>>
>>
>> On 2018-02-18, 3:32 PM, "William Brown" <will...@blackhats.net.au>
>> wrote:
>>
>> On Sat, 2018-02-17 at 01:49 +, Fong, Trevor wrote:
>> > Hi Everyone,
>> >  
>> > I’ve set up a new 389 DS cluster (389-Directory/1.3.6.1
>> > B2018.016.1710) and have set up a replication agreement from
>> our old
>> > cluster (389-Directory/1.2.11.15 B2014.300.2010) to a master
>> node in
>> > the new cluster.  Problem is that updates in the old cluster
>> take up
>> > to 15 mins to make it into the new cluster.  We need it to be
>> near
>> > instantaneous, like it normally is.  Any ideas what I can
>> check?
>> 
>> I am assuming you don't have a replication schedule enabled?
>> 
>> In LDAP replication is always "eventual". So a delay isn't
>> harmful.
>> 
>> But there are many things that can influence this. Ludwig is the
>> expert, and I expect he'll comment here. 
>> 
>> Only one master may be "replicating" to a server at a time. So if
>> your
>> 1.3 server is replicating with other servers, then your 1.2
>> server may
>> have to "wait it's turn".
>> 
>> There is a replication 'backoff' timer, that sets how long it
>> tries and
>> scales these attempts too. I'm not sure if 1.2 has this or not
>> though.
>> 
>> Another reason could be there are no changes to be replicated,
>> replication only runs when there is something t

[389-users] Re: Experiences with Large Groups (>100k Members)

[Fong, Trevor]

Thanks very much for your reply Trevor.
Just to expand a bit on my bit about sync’ing – We’ve been running on 389 DS 
for about 5 years now, and it has been solid for the most part.
Our LDAP cluster (multi-master replicated providers, hubs, and consumers) is 
sync’d to from our RDBMS based identity service via LSC (lsc-project.org).
LSC is normally trouble free, but it doesn’t do group replication very well due 
to it treating multi-value attributes in a monolithic manner (so we don’t do 
group sync yet).

Just wondering what others have encountered with large groups and syncing 
between LDAP <--> RDBMS / LDAP <--> LDAP.

Thanks,
Trev

From: "Wendt, Trevor" <trevor.we...@blackhillscorp.com>
Reply-To: "389-users@lists.fedoraproject.org" 
<389-users@lists.fedoraproject.org>
Date: Thursday, April 26, 2018 at 1:35 PM
To: "389-users@lists.fedoraproject.org" <389-users@lists.fedoraproject.org>
Subject: [389-users] Re: Experiences with Large Groups (>100k Members)

Trev, Was going to suggest splitting and batching your imports, good call.
We have had over 650k and not had any issues with 389ds. Started back on 
fedora-ds and migrated along with changes. Replication (master/master) is 
solid, keeps up with day to day changes fine, very stable. Overall no issues 
for 10+ years using it with steady growth. We are moving away to another 
solution but by not because of 389ds.  Good luck. -Trevor


From: Fong, Trevor [mailto:trevor.f...@ubc.ca]
Sent: Thursday, April 26, 2018 2:08 PM
To: General discussion list for the 389 Directory server project. 
<389-users@lists.fedoraproject.org>
Subject: [389-users] Re: Experiences with Large Groups (>100k Members)

Just an update:
I was successful in loading 532k members into a group, in ~45 mins, via 
ldapmodify by segmenting the ldif into 5 separate add:member sections, of ~100k 
each.  I also set nsslapd-db-locks in cn=config,cn=ldbm 
database,cn=plugins,cn=config to 40 – not sure which made any difference.

Still interested in other people’s experience with large groups.

From: "Fong, Trevor" <trevor.f...@ubc.ca>
Reply-To: "389-users@lists.fedoraproject.org" 
<389-users@lists.fedoraproject.org>
Date: Thursday, April 26, 2018 at 11:06 AM
To: "389-users@lists.fedoraproject.org" <389-users@lists.fedoraproject.org>
Subject: [389-users] Experiences with Large Groups (>100k Members)

Hi Everyone,

I was wondering what experiences people have had with large groups (> 100k 
members) in 389 DS?
Particularly interested in loading, managing and syncing them.
WRT syncing – how do people efficiently sync large groups?  Most sync utilities 
sync at the attribute level; if the changed attribute (eg member) is 
multivalued, it just replaces all values.  That’s OK if there’s only a few 
values, but is not efficient when there are a large number of them.  A more 
efficient way would be to diff the 2 attributes and add/delete the differences; 
does anyone know of any sync tools that do something like this?

Background:
I have a few particularly large groups of > 500k members that are currently 
handled in a DBMS, but want to migrate them to LDAP instead.
When I try to load them via ldapmodify, doing an add:member per member was 
going to take more than 24 hrs at rate of processing at the time of abort.
Trying to add all members instead, with a single add:member and listing all 
members after that instruction, eventually ended with an Operations Error.  
Turning on housekeeping error level showed it was getting “Lock table is out of 
available lock entries” error – I’m in the process of retrying with adjusted 
nsslapd-db-locks in cn=config,cn=ldbm database,cn=plugins,cn=config.

Thanks,
Trev


_
Trevor Fong
Senior Programmer Analyst
Information Technology | Engage. Envision. Enable.
The University of British Columbia
trevor.f...@ubc.ca<mailto:trevor.f...@ubc.ca> | 
1-604-827-5247 | it.ubc.ca<http://it.ubc.ca>




This electronic message transmission contains information from Black Hills 
Corporation, its affiliate or subsidiary, which may be confidential or 
privileged. The information is intended to be for the use of the individual or 
entity named above. If you are not the intended recipient, be aware the 
disclosure, copying, distribution or use of the contents of this information is 
prohibited. If you received this electronic transmission in error, please reply 
to sender immediately; then delete this message without copying it or further 
reading.

___
389-users mailing list -- 389-users@lists.fedoraproject.org
To unsubscribe send an email to 389-users-le...@lists.fedoraproject.org

[389-users] Re: Experiences with Large Groups (>100k Members)

[Fong, Trevor]

Just an update:
I was successful in loading 532k members into a group, in ~45 mins, via 
ldapmodify by segmenting the ldif into 5 separate add:member sections, of ~100k 
each.  I also set nsslapd-db-locks in cn=config,cn=ldbm 
database,cn=plugins,cn=config to 40 – not sure which made any difference.

Still interested in other people’s experience with large groups.

From: "Fong, Trevor" <trevor.f...@ubc.ca>
Reply-To: "389-users@lists.fedoraproject.org" 
<389-users@lists.fedoraproject.org>
Date: Thursday, April 26, 2018 at 11:06 AM
To: "389-users@lists.fedoraproject.org" <389-users@lists.fedoraproject.org>
Subject: [389-users] Experiences with Large Groups (>100k Members)

Hi Everyone,

I was wondering what experiences people have had with large groups (> 100k 
members) in 389 DS?
Particularly interested in loading, managing and syncing them.
WRT syncing – how do people efficiently sync large groups?  Most sync utilities 
sync at the attribute level; if the changed attribute (eg member) is 
multivalued, it just replaces all values.  That’s OK if there’s only a few 
values, but is not efficient when there are a large number of them.  A more 
efficient way would be to diff the 2 attributes and add/delete the differences; 
does anyone know of any sync tools that do something like this?

Background:
I have a few particularly large groups of > 500k members that are currently 
handled in a DBMS, but want to migrate them to LDAP instead.
When I try to load them via ldapmodify, doing an add:member per member was 
going to take more than 24 hrs at rate of processing at the time of abort.
Trying to add all members instead, with a single add:member and listing all 
members after that instruction, eventually ended with an Operations Error.  
Turning on housekeeping error level showed it was getting “Lock table is out of 
available lock entries” error – I’m in the process of retrying with adjusted 
nsslapd-db-locks in cn=config,cn=ldbm database,cn=plugins,cn=config.

Thanks,
Trev


_
Trevor Fong
Senior Programmer Analyst
Information Technology | Engage. Envision. Enable.
The University of British Columbia
trevor.f...@ubc.ca<mailto:trevor.f...@ubc.ca> | 
1-604-827-5247 | it.ubc.ca<http://it.ubc.ca>

___
389-users mailing list -- 389-users@lists.fedoraproject.org
To unsubscribe send an email to 389-users-le...@lists.fedoraproject.org

[389-users] Experiences with Large Groups (>100k Members)

[Fong, Trevor]

Hi Everyone,

I was wondering what experiences people have had with large groups (> 100k 
members) in 389 DS?
Particularly interested in loading, managing and syncing them.
WRT syncing – how do people efficiently sync large groups?  Most sync utilities 
sync at the attribute level; if the changed attribute (eg member) is 
multivalued, it just replaces all values.  That’s OK if there’s only a few 
values, but is not efficient when there are a large number of them.  A more 
efficient way would be to diff the 2 attributes and add/delete the differences; 
does anyone know of any sync tools that do something like this?

Background:
I have a few particularly large groups of > 500k members that are currently 
handled in a DBMS, but want to migrate them to LDAP instead.
When I try to load them via ldapmodify, doing an add:member per member was 
going to take more than 24 hrs at rate of processing at the time of abort.
Trying to add all members instead, with a single add:member and listing all 
members after that instruction, eventually ended with an Operations Error.  
Turning on housekeeping error level showed it was getting “Lock table is out of 
available lock entries” error – I’m in the process of retrying with adjusted 
nsslapd-db-locks in cn=config,cn=ldbm database,cn=plugins,cn=config.

Thanks,
Trev


_
Trevor Fong
Senior Programmer Analyst
Information Technology | Engage. Envision. Enable.
The University of British Columbia
trevor.f...@ubc.ca | 
1-604-827-5247 | it.ubc.ca

___
389-users mailing list -- 389-users@lists.fedoraproject.org
To unsubscribe send an email to 389-users-le...@lists.fedoraproject.org

[389-users] Re: Replication Delay

[Fong, Trevor]

Hi William,

Thanks a lot for your reply.

That's correct - replication schedule is not enabled.
No - there are definitely changes to replicate - I know, I made the change 
myself (
I changed the "description" attribute on an account, but it takes up to 15 mins 
for the change to appear in the 1.3 master.
That master replicates to another master and a bunch of other hubs.
Those hubs replicate amongst themselves and a bunch of consumers.

The update can take up to 15 mins to make it from the 1.2 master, into the 1.3 
master; but once it hits the 1.3 master, it is replicated around the 1.3 
cluster within 1 sec.

Only memberOf is disallowed for fractional replication.

Can anyone give me any guidance as to the settings of the "backoff" and other 
parameters?  Any doc links that may be useful?

Thanks a lot,
Trev


On 2018-02-18, 3:32 PM, "William Brown" <will...@blackhats.net.au> wrote:

    On Sat, 2018-02-17 at 01:49 +, Fong, Trevor wrote:
> Hi Everyone,
>  
> I’ve set up a new 389 DS cluster (389-Directory/1.3.6.1
> B2018.016.1710) and have set up a replication agreement from our old
> cluster (389-Directory/1.2.11.15 B2014.300.2010) to a master node in
> the new cluster.  Problem is that updates in the old cluster take up
> to 15 mins to make it into the new cluster.  We need it to be near
> instantaneous, like it normally is.  Any ideas what I can check?

I am assuming you don't have a replication schedule enabled?

In LDAP replication is always "eventual". So a delay isn't harmful.

But there are many things that can influence this. Ludwig is the
expert, and I expect he'll comment here. 

Only one master may be "replicating" to a server at a time. So if your
1.3 server is replicating with other servers, then your 1.2 server may
have to "wait it's turn".

There is a replication 'backoff' timer, that sets how long it tries and
scales these attempts too. I'm not sure if 1.2 has this or not though.

Another reason could be there are no changes to be replicated,
replication only runs when there is something to do. So your 1.2 server
may have no changes, or it could be eliminating the changes with
fractional replication.

Finally, it's very noisy but you could consider enabling replication
logging to check what's happening. 

I hope that helps,



>  
> Thanks a lot,
> Trev  
>  
> _
> Trevor Fong
> Senior Programmer Analyst
> Information Technology | Engage. Envision. Enable.
> The University of British Columbia
> trevor.f...@ubc.ca | 1-604-827-5247 | it.ubc.ca
>  
> ___
> 389-users mailing list -- 389-users@lists.fedoraproject.org
> To unsubscribe send an email to 389-users-leave@lists.fedoraproject.o
> rg
-- 
Thanks,

William Brown
___
389-users mailing list -- 389-users@lists.fedoraproject.org
To unsubscribe send an email to 389-users-le...@lists.fedoraproject.org


___
389-users mailing list -- 389-users@lists.fedoraproject.org
To unsubscribe send an email to 389-users-le...@lists.fedoraproject.org

[389-users] Replication Delay

[Fong, Trevor]

Hi Everyone,

I’ve set up a new 389 DS cluster (389-Directory/1.3.6.1 B2018.016.1710) and 
have set up a replication agreement from our old cluster 
(389-Directory/1.2.11.15 B2014.300.2010) to a master node in the new cluster.  
Problem is that updates in the old cluster take up to 15 mins to make it into 
the new cluster.  We need it to be near instantaneous, like it normally is.  
Any ideas what I can check?

Thanks a lot,
Trev

_
Trevor Fong
Senior Programmer Analyst
Information Technology | Engage. Envision. Enable.
The University of British Columbia
trevor.f...@ubc.ca | 
1-604-827-5247 | it.ubc.ca

___
389-users mailing list -- 389-users@lists.fedoraproject.org
To unsubscribe send an email to 389-users-le...@lists.fedoraproject.org

[389-users] Re: elapsed time gremlin

[Fong, Trevor]

Can we set nsIDListScanLimit=7 (or whatever) for a service account instead 
of changing the system nsslapd-idlistscanlimit setting?

Thanks,
Trev


From: Anthony Winstanley 
Reply-To: "389-users@lists.fedoraproject.org" 
<389-users@lists.fedoraproject.org>
Date: Friday, February 17, 2017 at 12:37 PM
To: "389-users@lists.fedoraproject.org" <389-users@lists.fedoraproject.org>
Subject: [389-users] Re: elapsed time gremlin

Thanks, that helps. ☺

Anthony

From: Noriko Hosoi [mailto:nho...@redhat.com]
Sent: Friday, February 17, 2017 10:43 AM
To: 389-users@lists.fedoraproject.org
Subject: [389-users] Re: elapsed time gremlin

On 02/17/2017 07:42 AM, Winstanley, Anthony wrote:
It was set to the default (4000). I set it to 70, and now both searches are 
fast (initial run time 3 seconds, then instant). And the server logs don't show 
any 'notes=?' for either search.

But… that's not what the docs say to do?
https://access.redhat.com/documentation/en-US/Red_Hat_Directory_Server/8.2/html/Performance_Tuning_Guide/improving-search-performance.html#setting-scan-limits
Glad to hear it improved the performance.

BTW, please refer a newer Doc.  There are lots of enhancements and bug fixes 
since the RHDS8.2 time frame...

RHDS 10 Performance Tuning Guide
https://access.redhat.com/documentation/en-US/Red_Hat_Directory_Server/10/html/Performance_Tuning_Guide/setting-scan-limits.html
5.3. Setting Index Scan Limits
In large directories, the search results list can get huge. A directory with a 
million inetorgperson entries would have a million entries that were returned 
with a filter like (objectclass=inetorgperson), and an index for the sn 
attribute would have at least a million entries in it.
Loading a long ID list from the database significantly reduces search 
performance. The configuration parameter, nsslapd-idlistscanlimit, sets a limit 
on the number of IDs that are read before a key is considered to match the 
entire primary index (meaning the search is treated as an unindexed search with 
a different set of resource limits).

So basically, setting nsslapd-idlistscanlimit big tells the server to NOT 
revert to unindexed search, and indexed search makes me happy in this case.
But why is unindexed search on ou=PEOPLE so much faster than on 
ou=STUDENTS,ou=RECORDS?
If you get the entire 7 ou=PEOPLE entries, it'd also take long time, I'd 
think.  If the search goes to the unindexed search, it scans all the entries in 
the database one by one and checks if it satisfies the search condition.  
Probably, OU=PEOPLE entries are put at the older part of the database (having 
lower entry id's) and OU=STUDENTS entries are placed after that (with higher 
entry id's)?  If that's the case, you cannot get the OU=STUDENTS entries unless 
you finish scanning OU=PEOPLE entries.

If indexed, you don't have to scan them but get the entry id list via indexes.

Hope it helps.
--noriko


Or should I not care what the answer to that question is and just set 
nsslapd-idlistscanlimit to a million and be happy?

Thanks for your help,
Anthony

From: Noriko Hosoi [mailto:nho...@redhat.com]
Sent: Thursday, February 16, 2017 5:16 PM
To: 389-users@lists.fedoraproject.org
Subject: [389-users] Re: elapsed time gremlin

Hello,

What is the value set to nsslapd-idlistscanlimit?

$ ldapsearch -x -LLL -H ldaps://ldap.example.com:636 -b  'cn=config,cn=ldbm 
database,cn=plugins,cn=config' -D cn=Directory\ Manager -w password -s base 
nsslapd-idlistscanlimit

If it is less than 6000, could you increase it to some large number, e.g., 
70, and retry the searches?

Thanks,
--noriko

On 02/16/2017 04:54 PM, Winstanley, Anthony wrote:
Hi,
I'm hoping somebody can tell me where I might look to find why this is 
happening…
We're running 389-Directory/1.2.11.15 B2014.300.2010

I have two ldapsearch queries that only vary in searchbase, one which is taking 
too long. (Times don't vary much with consecutive executions.)
ou=PEOPLE has just under 700,000 entries. Search takes 0-3 seconds.
ou=STUDENTS,ou=RECORDS has just under 6000 entries. Search takes 123-126 
seconds.
There are no attributes used in ou=STUDENTS,ou=RECORDS that aren't also used in 
ou=PEOPLE.

Two sample executions and log output:


[user@workstation ~]$ ldapsearch -x -LLL -H ldaps://ldap.example.com:636 -b 
ou=STUDENTS,ou=RECORDS,dc=example,dc=com -D cn=Directory\ Manager -w password 
-s one -z 5 'cn=*' dn

… 5 entries returned …

[user@workstation ~]$

[root@server slapd-ldap1]# grep conn=33505 access

[16/Feb/2017:16:31:37] conn=33505 fd=96 slot=96 SSL connection from IP1 to IP2

[16/Feb/2017:16:31:37] conn=33505 SSL 256-bit AES

[16/Feb/2017:16:31:37] conn=33505 op=0 BIND dn="cn=Directory Manager" 
method=128 version=3

[16/Feb/2017:16:31:37] conn=33505 op=0 RESULT err=0 tag=97 nentries=0 etime=0 
dn="cn=directory manager"

[16/Feb/2017:16:31:37] conn=33505 op=1 SRCH 
base="ou=STUDENTS,ou=RECORDS,dc=example,dc=com" scope=1 

[389-users] Re: ACI's on DB Linked Directories

[Fong, Trevor]

Hi William,

Thanks very much for your reply.




On 2016-03-29, 4:23 PM, "William Brown" <wibr...@redhat.com> wrote:

>On Tue, 2016-03-29 at 22:49 +, Fong, Trevor wrote:
>> Hi Everyone,
>> 
>> A question about how to best go about doing access control on db-linked
>> directories. Given the below 2-directory setup (Dir1 has a db link set up to
>> Dir2, and vice versa), is the shown ACI setup possible/advisable?  If not,
>> what’s the best way to make it work?:
>> 
>> Dir1:
>> dc=example,dc=com
>>ou=employees
>>   uid=alice
>>   uid=bob
>> 
>>cn=admins
>>   member:uid=alice,ou=employees,dc=example,dc=com
>> 
>> Dir2:
>> dc=example,dc=com
>>ou=projects
>>   aci:(targetattr ="*")(version 3.0;acl “Admins Group";allow (all) 
>> (groupdn
>> = "ldap:///cn=admins,dc=example,dc=com;);)
>> 
>> 
>> I ask because section 2.3.6. Database Links and Access Control Evaluation, of
>> the RHDS Admin Guide says:
>> "ACIs must be located with any groups they use. If the groups are dynamic, 
>> all
>> users in the group must be located with the ACI and the group. If the group 
>> is
>> static, it may refer to remote users."
>
>My interpretation is that dir2 has the aci, and only the data of ou=projects, 
>it
>is un-able to back query to dir1. In this case the aci will not work.
>
>You however, this might work:
>
>Dir1:
>dc=example,dc=com
>   cn=admins
>  member:uid=alice,ou=employees,dc=example,dc=com
>
>Dir2:
>dc=example,dc=com
>   ou=projects
>  aci:(targetattr ="*")(version 3.0;acl “Admins Group";allow (all) 
> (groupdn =
>"ldap:///cn=admins,dc=example,dc=com;);)
>
>   ou=employees
>  uid=alice
>  uid=bob

So the aci on Dir2 can refer to a remote group on ACI, but the members of the 
group have to be local? 


>You can easily check this with the get effective rights extension. 

Unfortunately, get effective rights only shows the rights if you can get to an 
entry.  If you can’t (e.g. aci not letting you/not working) then you don’t get 
anything.  I was hoping not to but guess I’ll have to get the ACL processing 
logged.


>
>> 
>> https://access.redhat.com/documentation/en-US/Red_Hat_Directory_Server/10/html/
>> Administration_Guide/Configuring_Directory_Databases-
>> Creating_and_Maintaining_Database_Links.html#Creating_and_Maintaining_Database_
>> Links-Database_Links_and_Access_Control_Evaluation
>> 
>> I’m afraid the phrasing is a little opaque to my understanding.  Does it mean
>> that “Admin Groups” act on Dir2 is not allowed to refer to
>> cn=admins,dc=example,dc=com on Dir1?
>> If so, then what is the best way of maintaining groups centrally but allowing
>> them to be used on remote directories?
>> 
>> *Bonus Question:
>> Say Alice only has access to Dir1, she can issue a search to ou=projects
>> because of the DB link from Dir1 —> Dir2.  When the aci on ou=projects is
>> processed, which user is used?  uid=alice or the proxy user of the db
>> link?  Will the aci work at all in this case?
>> 
>
>I believe that the db link uses the proxy control to impersonate alice on the
>remote server.
>
>Again, this can easily be validated by doing a search on dir1 as alice, then
>checking the access log of dir2 to see who was bound, whether the proxy control
>was used. 
>

Thanks again, I’ll up the errorlog level do a log crawl and try to work out 
what’s happening.

Just wondering if other users out there do a similar thing, where they have a 
central directory chaining to more specialized directories secured with acids?  
How do you write your ACI’s to avoid this kind of situation?

Thanks everyone,
Trev 




>
>-- 
>Sincerely,
>
>William Brown
>Software Engineer
>Red Hat, Brisbane
>
--
389 users mailing list
389-users@%(host_name)s
http://lists.fedoraproject.org/admin/lists/389-users@lists.fedoraproject.org

[389-users] ACI's on DB Linked Directories

[Fong, Trevor]

Hi Everyone,

A question about how to best go about doing access control on db-linked 
directories. Given the below 2-directory setup (Dir1 has a db link set up to 
Dir2, and vice versa), is the shown ACI setup possible/advisable?  If not, 
what’s the best way to make it work?:

Dir1:
dc=example,dc=com
   ou=employees
  uid=alice
  uid=bob

   cn=admins
  member:uid=alice,ou=employees,dc=example,dc=com

Dir2:
dc=example,dc=com
   ou=projects
  aci:(targetattr ="*")(version 3.0;acl “Admins Group";allow (all) (groupdn 
= "ldap:///cn=admins,dc=example,dc=com;);)


I ask because section 2.3.6. Database Links and Access Control Evaluation, of 
the RHDS Admin Guide says:
"ACIs must be located with any groups they use. If the groups are dynamic, all 
users in the group must be located with the ACI and the group. If the group is 
static, it may refer to remote users."

https://access.redhat.com/documentation/en-US/Red_Hat_Directory_Server/10/html/Administration_Guide/Configuring_Directory_Databases-Creating_and_Maintaining_Database_Links.html#Creating_and_Maintaining_Database_Links-Database_Links_and_Access_Control_Evaluation

I’m afraid the phrasing is a little opaque to my understanding.  Does it mean 
that “Admin Groups” act on Dir2 is not allowed to refer to 
cn=admins,dc=example,dc=com on Dir1?
If so, then what is the best way of maintaining groups centrally but allowing 
them to be used on remote directories?

*Bonus Question:
Say Alice only has access to Dir1, she can issue a search to ou=projects 
because of the DB link from Dir1 —> Dir2.  When the aci on ou=projects is 
processed, which user is used?  uid=alice or the proxy user of the db link?  
Will the aci work at all in this case?

Thanks a lot,
Trev
_
Trevor Fong
Senior Programmer Analyst
Information Technology | Engage. Envision. Enable.
The University of British Columbia
trevor.f...@ubc.ca | 
1-604-827-5247 | it.ubc.ca

--
389 users mailing list
389-users@%(host_name)s
http://lists.fedoraproject.org/admin/lists/389-users@lists.fedoraproject.org

Re: [389-users] DS crashed /killed by OS

[Fong, Trevor]

Hi German,

Thanks for your suggestion.  I’m happy to confirm that setting userRoot’s 
nsslapd-cachememsize: 429496730 (1/15th of previous value of 6 GB) has 
addressed the memory issue for now, and % Mem for the ns-slapd process seems to 
be at a manageable level.

Thanks very much,
Trev




On 2015-10-20, 11:07 AM, "389-users-boun...@lists.fedoraproject.org on behalf 
of German Parente" <389-users-boun...@lists.fedoraproject.org on behalf of 
gpare...@redhat.com> wrote:

>
>Hi Trevor,
>
>400Mb could be a more reasonable value. With a cache of 6gb, fragmentation 
>could very quickly provoke the OOM killer error.
>
>Regards,
>
>German.
>
>- Original Message -
>> From: "Trevor Fong" 
>> To: "General discussion list for the 389 Directory server project." 
>> <389-users@lists.fedoraproject.org>
>> Sent: Tuesday, October 20, 2015 7:44:06 PM
>> Subject: Re: [389-users] DS crashed /killed by OS
>> 
>> Hi German,
>> 
>> Thanks very much for your reply.
>> Just to make sure I have it straight, I’ve currently got userRoot’s
>> nsslapd-cachememsize = 6 GB on at 16GB machine.
>> I should change that to nsslapd-cachememsize = 6 GB / 15 = 429496730
>> Do I have that right?
>> 
>> Thanks again,
>> Trev
>> 
>> 
>> 
>> 
>> On 2015-10-20, 10:23 AM, "389-users-boun...@lists.fedoraproject.org on behalf
>> of German Parente" <389-users-boun...@lists.fedoraproject.org on behalf of
>> gpare...@redhat.com> wrote:
>> 
>> >Hi Trevor,
>> >
>> >no problem. In fact, this issue has been investigated by the experts and
>> >it's due to fragmentation. A fix is being tested right internally but not
>> >delivered yet, to use a different allocator.
>> >
>> >The official workaround is different to the one I have proposed. It's
>> >finally to define entry cache rather small since the fragmentation could be
>> >like
>> >
>> >15 * size of entry cache.
>> >
>> >So, we need something like (15 * size of entry cache )  <  Available memory.
>> >
>> >Thanks and regards,
>> >
>> >German.
>> >
>> >
>> >
>> >- Original Message -
>> >> From: "Trevor Fong" 
>> >> To: "General discussion list for the 389 Directory server project."
>> >> <389-users@lists.fedoraproject.org>
>> >> Sent: Tuesday, October 20, 2015 7:09:46 PM
>> >> Subject: Re: [389-users] DS crashed /killed by OS
>> >> 
>> >> Hi German,
>> >> 
>> >> Apologies for resurrecting an old thread.
>> >> We're also experiencing something similar.  We're currently running
>> >> 389-ds-base-1.2.11.15-48.el6_6.x86_64
>> >> 
>> >> I'm afraid I don't have login privileges in order to view the details of
>> >> the
>> >> bug you linked.
>> >> Could you please post details of how you defined an entry cache to include
>> >> the whole db, and why this works?
>> >> 
>> >> FYI - moves are afoot re upgrading DS on a set of new servers, but in the
>> >> meantime, we need to address this issue.
>> >> 
>> >> 
>> >> Thanks a lot,
>> >> Trev
>> >> 
>> >> 
>> >> 
>> >> 
>> >> 
>> >> On 2015-02-05, 1:57 AM, "389-users-boun...@lists.fedoraproject.org on
>> >> behalf
>> >> of German Parente" <389-users-boun...@lists.fedoraproject.org on behalf of
>> >> gpare...@redhat.com> wrote:
>> >> 
>> >> >
>> >> >Hi,
>> >> >
>> >> >we have had several customer cases showing this behavior. In one of these
>> >> >cases, we have confirmed it was due to memory fragmentation after
>> >> >cache-trashing.
>> >> >
>> >> >We have stopped seeing this behavior by defining an entry cache which
>> >> >includes the whole db (when possible, of course).
>> >> >
>> >> >Details can be found at:
>> >> >
>> >> >https://bugzilla.redhat.com/show_bug.cgi?id=1186512
>> >> >Apparent memory leak in ns-slapd; OOM-Killer invoked
>> >> >
>> >> >Regards,
>> >> >
>> >> >German
>> >> >
>> >> >- Original Message -
>> >> >> From: "David Boreham" 
>> >> >> To: 389-users@lists.fedoraproject.org
>> >> >> Sent: Wednesday, February 4, 2015 8:50:55 PM
>> >> >> Subject: Re: [389-users] DS crashed /killed by OS
>> >> >> 
>> >> >> On 2/4/2015 11:20 AM, ghiureai wrote:
>> >> >> 
>> >> >> 
>> >> >> 
>> >> >> Out of memory: Kill process 2090 (ns-slapd) score 954 or sacrifice
>> >> >> child
>> >> >> 
>> >> >> It wasn't clear to me from your post whether you already have a good
>> >> >> understanding of the OOM killer behavior in the kernel.
>> >> >> On the chance that you're not yet familiar with its ways, suggest
>> >> >> reading,
>> >> >> for example this article :
>> >> >> http://unix.stackexchange.com/questions/153585/how-oom-killer-decides-which-process-to-kill-first
>> >> >> I mention this because it may not be the DS that is the problem (not
>> >> >> saying
>> >> >> that it absolutely is not, but it might not be).
>> >> >> The OMM killer picks a process that is using a large amount of memory,
>> >> >> and
>> >> >> kills it in order to preserve system stability.
>> >> >> This does not necessarily imply that the process it kills is the
>> >> >> process
>> >> >> that
>> >> >> is causing the 

Re: [389-users] DS crashed /killed by OS

[Fong, Trevor]

Hi German,

Thanks very much for your reply.
Just to make sure I have it straight, I’ve currently got userRoot’s 
nsslapd-cachememsize = 6 GB on at 16GB machine.
I should change that to nsslapd-cachememsize = 6 GB / 15 = 429496730
Do I have that right?

Thanks again,
Trev




On 2015-10-20, 10:23 AM, "389-users-boun...@lists.fedoraproject.org on behalf 
of German Parente" <389-users-boun...@lists.fedoraproject.org on behalf of 
gpare...@redhat.com> wrote:

>Hi Trevor,
>
>no problem. In fact, this issue has been investigated by the experts and it's 
>due to fragmentation. A fix is being tested right internally but not delivered 
>yet, to use a different allocator.
>
>The official workaround is different to the one I have proposed. It's finally 
>to define entry cache rather small since the fragmentation could be like 
>
>15 * size of entry cache.
>
>So, we need something like (15 * size of entry cache )  <  Available memory.
>
>Thanks and regards,
>
>German.
>
>
>
>- Original Message -
>> From: "Trevor Fong" 
>> To: "General discussion list for the 389 Directory server project." 
>> <389-users@lists.fedoraproject.org>
>> Sent: Tuesday, October 20, 2015 7:09:46 PM
>> Subject: Re: [389-users] DS crashed /killed by OS
>> 
>> Hi German,
>> 
>> Apologies for resurrecting an old thread.
>> We're also experiencing something similar.  We're currently running
>> 389-ds-base-1.2.11.15-48.el6_6.x86_64
>> 
>> I'm afraid I don't have login privileges in order to view the details of the
>> bug you linked.
>> Could you please post details of how you defined an entry cache to include
>> the whole db, and why this works?
>> 
>> FYI - moves are afoot re upgrading DS on a set of new servers, but in the
>> meantime, we need to address this issue.
>> 
>> 
>> Thanks a lot,
>> Trev
>> 
>> 
>> 
>> 
>> 
>> On 2015-02-05, 1:57 AM, "389-users-boun...@lists.fedoraproject.org on behalf
>> of German Parente" <389-users-boun...@lists.fedoraproject.org on behalf of
>> gpare...@redhat.com> wrote:
>> 
>> >
>> >Hi,
>> >
>> >we have had several customer cases showing this behavior. In one of these
>> >cases, we have confirmed it was due to memory fragmentation after
>> >cache-trashing.
>> >
>> >We have stopped seeing this behavior by defining an entry cache which
>> >includes the whole db (when possible, of course).
>> >
>> >Details can be found at:
>> >
>> >https://bugzilla.redhat.com/show_bug.cgi?id=1186512
>> >Apparent memory leak in ns-slapd; OOM-Killer invoked
>> >
>> >Regards,
>> >
>> >German
>> >
>> >- Original Message -
>> >> From: "David Boreham" 
>> >> To: 389-users@lists.fedoraproject.org
>> >> Sent: Wednesday, February 4, 2015 8:50:55 PM
>> >> Subject: Re: [389-users] DS crashed /killed by OS
>> >> 
>> >> On 2/4/2015 11:20 AM, ghiureai wrote:
>> >> 
>> >> 
>> >> 
>> >> Out of memory: Kill process 2090 (ns-slapd) score 954 or sacrifice child
>> >> 
>> >> It wasn't clear to me from your post whether you already have a good
>> >> understanding of the OOM killer behavior in the kernel.
>> >> On the chance that you're not yet familiar with its ways, suggest reading,
>> >> for example this article :
>> >> http://unix.stackexchange.com/questions/153585/how-oom-killer-decides-which-process-to-kill-first
>> >> I mention this because it may not be the DS that is the problem (not
>> >> saying
>> >> that it absolutely is not, but it might not be).
>> >> The OMM killer picks a process that is using a large amount of memory, and
>> >> kills it in order to preserve system stability.
>> >> This does not necessarily imply that the process it kills is the process
>> >> that
>> >> is causing the system to run out of memory.
>> >> You said that the DS "crashed", but in fact the kernel killed it -- not
>> >> quite
>> >> the same thing!
>> >> 
>> >> It is also possible that the system has insufficient memory for the
>> >> processes
>> >> it is running, DS cache size and so on.
>> >> Certainly it is worthwhile checking that the DS hasn't been inadvertently
>> >> configured to use more peak memory than the machine has available.
>> >> 
>> >> Bottom line : there are a few potential explanations, including but not
>> >> limited to a memory leak in the DS process.
>> >> Some analysis will be needed to identify the cause. As a precaution, if
>> >> you
>> >> can -- configure more swap space on the box.
>> >> This will allow more runway before the kernel starts looking for processes
>> >> to
>> >> kill, and hence more time to figure out what's using memory and why.
>> >> 
>> >> 
>> >> 
>> >> 
>> >> 
>> >> 
>> >> --
>> >> 389 users mailing list
>> >> 389-users@lists.fedoraproject.org
>> >> https://admin.fedoraproject.org/mailman/listinfo/389-users
>> >--
>> >389 users mailing list
>> >389-users@lists.fedoraproject.org
>> >https://admin.fedoraproject.org/mailman/listinfo/389-users
>> --
>> 389 users mailing list
>> 389-users@lists.fedoraproject.org
>> 

Re: [389-users] DNA Plugin Causes 389-DS to Crash if Large Number of Candidates

[Fong, Trevor]

Hi Rich,

Thanks for your reply.  Nsslapd-idlistscanlimit is set to the default of 4000.
I’ll see if we can upgrade our install and see how that goes.  In the meantime, 
any further suggestions would be most welcome.

Thanks,
Trev

From: 
389-users-boun...@lists.fedoraproject.orgmailto:389-users-boun...@lists.fedoraproject.org
 on behalf of Rich Megginson rmegg...@redhat.commailto:rmegg...@redhat.com
Reply-To: 
389-users@lists.fedoraproject.orgmailto:389-users@lists.fedoraproject.org 
389-users@lists.fedoraproject.orgmailto:389-users@lists.fedoraproject.org
Date: Thursday, July 16, 2015 at 5:19 PM
To: 
389-users@lists.fedoraproject.orgmailto:389-users@lists.fedoraproject.org 
389-users@lists.fedoraproject.orgmailto:389-users@lists.fedoraproject.org
Subject: Re: [389-users] DNA Plugin Causes 389-DS to Crash if Large Number of 
Candidates

On 07/16/2015 05:47 PM, Fong, Trevor wrote:

Hi Guys,


We’re running 389-ds 1.2.11.29-1.el6

Can you upgrade to a newer version?  There have been several releases since 
then.


and are experimenting with the DNA plugin.  When trying to set an existing 
account’s  uidNumber to the magic regen number of 9, we get the error 
message in the errors log:


Allocation of a new value for range cn=uid numbers,cn=distributed numeric 
assignment plugin,cn=plugins,cn=config failed! Unable to proceed.

And then DS becomes unresponsive.

If the server is crashed: 
http://www.port389.org/docs/389ds/FAQ/faq.html#debugging-crashes

If the server is hung: 
http://www.port389.org/docs/389ds/FAQ/faq.html#debugging-hangs



Looking at the debug error logs, it seems that the number of candidates defined 
by the combination of dnaScope and dnaFilter is too large (1389387 ids) for the 
sort function?

Looks like it.


Is there a fix?  Or is there some error in our setup?

Looks like you are hitting the idlistscanlimit:

[16/Jul/2015:15:31:06 -0700] - = index_read( employeeNumber +  )
[16/Jul/2015:15:31:06 -0700] -indextype: pres indexmask: 0x3
[16/Jul/2015:15:31:06 -0700] - bulk fetch buffer nids=207
...
[16/Jul/2015:15:31:06 -0700] - bulk fetch buffer nids=4075
[16/Jul/2015:15:31:06 -0700] - idl_new_fetch + returns allids
[16/Jul/2015:15:31:06 -0700] - = index_read 1389387 candidates
...
[16/Jul/2015:15:31:06 -0700] - = index_read( objectClass = posixaccount )
[16/Jul/2015:15:31:06 -0700] -indextype: eq indexmask: 0x2
[16/Jul/2015:15:31:06 -0700] - bulk fetch buffer nids=207
...
[16/Jul/2015:15:31:06 -0700] - bulk fetch buffer nids=4075
[16/Jul/2015:15:31:06 -0700] - idl_new_fetch =posixaccount returns allids
[16/Jul/2015:15:31:06 -0700] - = index_read 1389387 candidates
[16/Jul/2015:15:31:06 -0700] -ival[0] = posixaccount = 1389387 IDs
...

[16/Jul/2015:15:31:06 -0700] - Asked to sort ALLIDS candidate list, refusing

[16/Jul/2015:15:31:06 -0700] - = send_ldap_result 12::Sort Response Control

You might try raising the nsslapd-idlistscanlimit to be greater than 1389387



Thanks,

Trev





Our setup is thus:


dn: cn=Distributed Numeric Assignment Plugin,cn=plugins,cn=config

objectClass: extensibleObject

objectClass: nsContainer

objectClass: nsSlapdPlugin

objectClass: top

cn: Distributed Numeric Assignment Plugin

nsslapd-pluginDescription: Distributed Numeric Assignment plugin

nsslapd-pluginEnabled: on

nsslapd-pluginId: Distributed Numeric Assignment

nsslapd-pluginInitfunc: dna_init

nsslapd-pluginPath: libdna-plugin

nsslapd-pluginType: bepreoperation

nsslapd-pluginVendor: 389 Project

nsslapd-pluginVersion: 1.2.11.29

nsslapd-plugin-depends-on-type: database



dn: cn=UID numbers,cn=Distributed Numeric Assignment Plugin,cn=plugins,cn=co

 nfig

objectClass: extensibleObject

objectClass: top

cn: UID numbers

dnaFilter: ((employeeNumber=*)(objectclass=posixAccount))

dnaMagicRegen: 9

dnaNextValue: 1002

dnaScope: ou=PEOPLE,ou=IDM,dc=dev,dc=example,dc=com

dnaType: uidNumber



Error log:

[16/Jul/2015:15:31:05 -0700] - add_pb

[16/Jul/2015:15:31:05 -0700] - = slapi_reslimit_get_integer_limit() 
conn=0x2907e68, handle=8

[16/Jul/2015:15:31:05 -0700] - = slapi_reslimit_get_integer_limit() returning 
NO VALUE

[16/Jul/2015:15:31:05 -0700] - = slapi_reslimit_get_integer_limit() 
conn=0x2907d30, handle=8

[16/Jul/2015:15:31:05 -0700] - = slapi_reslimit_get_integer_limit() returning 
NO VALUE

[16/Jul/2015:15:31:05 -0700] - -- pagedresults_is_timedout

[16/Jul/2015:15:31:05 -0700] - -- pagedresults_is_timedout: -

[16/Jul/2015:15:31:05 -0700] - -- pagedresults_is_timedout

[16/Jul/2015:15:31:05 -0700] - -- pagedresults_is_timedout: -

[16/Jul/2015:15:31:05 -0700] - get_pb

[16/Jul/2015:15:31:05 -0700] - -- pagedresults_in_use

[16/Jul/2015:15:31:05 -0700] - -- pagedresults_in_use: 0

[16/Jul/2015:15:31:05 -0700] - do_modify

[16/Jul/2015:15:31:05 -0700] - do_modify: dn 
(uid=myacct,ou=PEOPLE,ou=IDM,dc=dev,dc=example,dc=com)

[16/Jul/2015:15:31:05 -0700] - = get_ldapmessage_controls

[16/Jul/2015:15:31:05 -0700] - = get_ldapmessage_controls no controls

[16

Re: [389-users] DNA Plugin Causes 389-DS to Crash if Large Number of Candidates

[Fong, Trevor]

BTW, we were able to artificially make it work by changing the dnaFilter to the 
emplid of our test user:
dnaFilter: ((employeeNumber=12345)(objectclass=posixAccount))

Trev

From: Trevor Fong trevor.f...@ubc.camailto:trevor.f...@ubc.ca
Date: Thursday, July 16, 2015 at 4:47 PM
To: 
389-users@lists.fedoraproject.orgmailto:389-users@lists.fedoraproject.org 
389-users@lists.fedoraproject.orgmailto:389-users@lists.fedoraproject.org
Subject: DNA Plugin Causes 389-DS to Crash if Large Number of Candidates


Hi Guys,


We’re running 389-ds 1.2.11.29-1.el6 and are experimenting with the DNA plugin. 
 When trying to set an existing account’s  uidNumber to the magic regen number 
of 9, we get the error message in the errors log:


Allocation of a new value for range cn=uid numbers,cn=distributed numeric 
assignment plugin,cn=plugins,cn=config failed! Unable to proceed.

And then DS becomes unresponsive.


Looking at the debug error logs, it seems that the number of candidates defined 
by the combination of dnaScope and dnaFilter is too large (1389387 ids) for the 
sort function?  Is there a fix?  Or is there some error in our setup?


Thanks,

Trev





Our setup is thus:


dn: cn=Distributed Numeric Assignment Plugin,cn=plugins,cn=config

objectClass: extensibleObject

objectClass: nsContainer

objectClass: nsSlapdPlugin

objectClass: top

cn: Distributed Numeric Assignment Plugin

nsslapd-pluginDescription: Distributed Numeric Assignment plugin

nsslapd-pluginEnabled: on

nsslapd-pluginId: Distributed Numeric Assignment

nsslapd-pluginInitfunc: dna_init

nsslapd-pluginPath: libdna-plugin

nsslapd-pluginType: bepreoperation

nsslapd-pluginVendor: 389 Project

nsslapd-pluginVersion: 1.2.11.29

nsslapd-plugin-depends-on-type: database



dn: cn=UID numbers,cn=Distributed Numeric Assignment Plugin,cn=plugins,cn=co

 nfig

objectClass: extensibleObject

objectClass: top

cn: UID numbers

dnaFilter: ((employeeNumber=*)(objectclass=posixAccount))

dnaMagicRegen: 9

dnaNextValue: 1002

dnaScope: ou=PEOPLE,ou=IDM,dc=dev,dc=example,dc=com

dnaType: uidNumber



Error log:

[16/Jul/2015:15:31:05 -0700] - add_pb

[16/Jul/2015:15:31:05 -0700] - = slapi_reslimit_get_integer_limit() 
conn=0x2907e68, handle=8

[16/Jul/2015:15:31:05 -0700] - = slapi_reslimit_get_integer_limit() returning 
NO VALUE

[16/Jul/2015:15:31:05 -0700] - = slapi_reslimit_get_integer_limit() 
conn=0x2907d30, handle=8

[16/Jul/2015:15:31:05 -0700] - = slapi_reslimit_get_integer_limit() returning 
NO VALUE

[16/Jul/2015:15:31:05 -0700] - -- pagedresults_is_timedout

[16/Jul/2015:15:31:05 -0700] - -- pagedresults_is_timedout: -

[16/Jul/2015:15:31:05 -0700] - -- pagedresults_is_timedout

[16/Jul/2015:15:31:05 -0700] - -- pagedresults_is_timedout: -

[16/Jul/2015:15:31:05 -0700] - get_pb

[16/Jul/2015:15:31:05 -0700] - -- pagedresults_in_use

[16/Jul/2015:15:31:05 -0700] - -- pagedresults_in_use: 0

[16/Jul/2015:15:31:05 -0700] - do_modify

[16/Jul/2015:15:31:05 -0700] - do_modify: dn 
(uid=myacct,ou=PEOPLE,ou=IDM,dc=dev,dc=example,dc=com)

[16/Jul/2015:15:31:05 -0700] - = get_ldapmessage_controls

[16/Jul/2015:15:31:05 -0700] - = get_ldapmessage_controls no controls

[16/Jul/2015:15:31:05 -0700] - modifications:

[16/Jul/2015:15:31:05 -0700] -

replace: uidNumber

[16/Jul/2015:15:31:05 -0700] - = slapi_control_present (looking for 
2.16.840.1.113730.3.4.12)

[16/Jul/2015:15:31:05 -0700] - = slapi_control_present 0 (NO CONTROLS)

[16/Jul/2015:15:31:05 -0700] - = slapi_control_present (looking for 
2.16.840.1.113730.3.4.18)

[16/Jul/2015:15:31:05 -0700] - = slapi_control_present 0 (NO CONTROLS)

[16/Jul/2015:15:31:05 -0700] - mapping tree selected backend : userRoot

[16/Jul/2015:15:31:05 -0700] - = slapi_control_present (looking for 
2.16.840.1.113730.3.4.12)

[16/Jul/2015:15:31:05 -0700] - = slapi_control_present 0 (NO CONTROLS)

[16/Jul/2015:15:31:05 -0700] - = slapi_control_present (looking for 
2.16.840.1.113730.3.4.18)

[16/Jul/2015:15:31:05 -0700] - = slapi_control_present 0 (NO CONTROLS)

[16/Jul/2015:15:31:05 -0700] - mapping tree selected backend : userRoot

[16/Jul/2015:15:31:05 -0700] - = slapi_reslimit_get_integer_limit() conn=0x0, 
handle=6

[16/Jul/2015:15:31:05 -0700] - = slapi_reslimit_get_integer_limit() returning 
NO VALUE

[16/Jul/2015:15:31:05 -0700] - = slapi_reslimit_get_integer_limit() conn=0x0, 
handle=5

[16/Jul/2015:15:31:05 -0700] - = slapi_reslimit_get_integer_limit() returning 
NO VALUE

[16/Jul/2015:15:31:05 -0700] - = compute_limits: sizelimit=-1, timelimit=-1

[16/Jul/2015:15:31:05 -0700] - Calling plugin 'Account Usability Plugin' #1 
type 403

[16/Jul/2015:15:31:05 -0700] account-usability-plugin - -- auc_pre_search

[16/Jul/2015:15:31:05 -0700] account-usability-plugin - -- auc_pre_op

[16/Jul/2015:15:31:05 -0700] - Calling plugin 'ACL preoperation' #2 type 403

[16/Jul/2015:15:31:05 -0700] - Calling plugin 'deref' #4 type 403

[16/Jul/2015:15:31:05 -0700] deref-plugin - -- deref_pre_search

[389-users] DNA Plugin Causes 389-DS to Crash if Large Number of Candidates

[Fong, Trevor]

Hi Guys,


We’re running 389-ds 1.2.11.29-1.el6 and are experimenting with the DNA plugin. 
 When trying to set an existing account’s  uidNumber to the magic regen number 
of 9, we get the error message in the errors log:


Allocation of a new value for range cn=uid numbers,cn=distributed numeric 
assignment plugin,cn=plugins,cn=config failed! Unable to proceed.

And then DS becomes unresponsive.


Looking at the debug error logs, it seems that the number of candidates defined 
by the combination of dnaScope and dnaFilter is too large (1389387 ids) for the 
sort function?  Is there a fix?  Or is there some error in our setup?


Thanks,

Trev





Our setup is thus:


dn: cn=Distributed Numeric Assignment Plugin,cn=plugins,cn=config

objectClass: extensibleObject

objectClass: nsContainer

objectClass: nsSlapdPlugin

objectClass: top

cn: Distributed Numeric Assignment Plugin

nsslapd-pluginDescription: Distributed Numeric Assignment plugin

nsslapd-pluginEnabled: on

nsslapd-pluginId: Distributed Numeric Assignment

nsslapd-pluginInitfunc: dna_init

nsslapd-pluginPath: libdna-plugin

nsslapd-pluginType: bepreoperation

nsslapd-pluginVendor: 389 Project

nsslapd-pluginVersion: 1.2.11.29

nsslapd-plugin-depends-on-type: database



dn: cn=UID numbers,cn=Distributed Numeric Assignment Plugin,cn=plugins,cn=co

 nfig

objectClass: extensibleObject

objectClass: top

cn: UID numbers

dnaFilter: ((employeeNumber=*)(objectclass=posixAccount))

dnaMagicRegen: 9

dnaNextValue: 1002

dnaScope: ou=PEOPLE,ou=IDM,dc=dev,dc=example,dc=com

dnaType: uidNumber



Error log:

[16/Jul/2015:15:31:05 -0700] - add_pb

[16/Jul/2015:15:31:05 -0700] - = slapi_reslimit_get_integer_limit() 
conn=0x2907e68, handle=8

[16/Jul/2015:15:31:05 -0700] - = slapi_reslimit_get_integer_limit() returning 
NO VALUE

[16/Jul/2015:15:31:05 -0700] - = slapi_reslimit_get_integer_limit() 
conn=0x2907d30, handle=8

[16/Jul/2015:15:31:05 -0700] - = slapi_reslimit_get_integer_limit() returning 
NO VALUE

[16/Jul/2015:15:31:05 -0700] - -- pagedresults_is_timedout

[16/Jul/2015:15:31:05 -0700] - -- pagedresults_is_timedout: -

[16/Jul/2015:15:31:05 -0700] - -- pagedresults_is_timedout

[16/Jul/2015:15:31:05 -0700] - -- pagedresults_is_timedout: -

[16/Jul/2015:15:31:05 -0700] - get_pb

[16/Jul/2015:15:31:05 -0700] - -- pagedresults_in_use

[16/Jul/2015:15:31:05 -0700] - -- pagedresults_in_use: 0

[16/Jul/2015:15:31:05 -0700] - do_modify

[16/Jul/2015:15:31:05 -0700] - do_modify: dn 
(uid=myacct,ou=PEOPLE,ou=IDM,dc=dev,dc=example,dc=com)

[16/Jul/2015:15:31:05 -0700] - = get_ldapmessage_controls

[16/Jul/2015:15:31:05 -0700] - = get_ldapmessage_controls no controls

[16/Jul/2015:15:31:05 -0700] - modifications:

[16/Jul/2015:15:31:05 -0700] -

replace: uidNumber

[16/Jul/2015:15:31:05 -0700] - = slapi_control_present (looking for 
2.16.840.1.113730.3.4.12)

[16/Jul/2015:15:31:05 -0700] - = slapi_control_present 0 (NO CONTROLS)

[16/Jul/2015:15:31:05 -0700] - = slapi_control_present (looking for 
2.16.840.1.113730.3.4.18)

[16/Jul/2015:15:31:05 -0700] - = slapi_control_present 0 (NO CONTROLS)

[16/Jul/2015:15:31:05 -0700] - mapping tree selected backend : userRoot

[16/Jul/2015:15:31:05 -0700] - = slapi_control_present (looking for 
2.16.840.1.113730.3.4.12)

[16/Jul/2015:15:31:05 -0700] - = slapi_control_present 0 (NO CONTROLS)

[16/Jul/2015:15:31:05 -0700] - = slapi_control_present (looking for 
2.16.840.1.113730.3.4.18)

[16/Jul/2015:15:31:05 -0700] - = slapi_control_present 0 (NO CONTROLS)

[16/Jul/2015:15:31:05 -0700] - mapping tree selected backend : userRoot

[16/Jul/2015:15:31:05 -0700] - = slapi_reslimit_get_integer_limit() conn=0x0, 
handle=6

[16/Jul/2015:15:31:05 -0700] - = slapi_reslimit_get_integer_limit() returning 
NO VALUE

[16/Jul/2015:15:31:05 -0700] - = slapi_reslimit_get_integer_limit() conn=0x0, 
handle=5

[16/Jul/2015:15:31:05 -0700] - = slapi_reslimit_get_integer_limit() returning 
NO VALUE

[16/Jul/2015:15:31:05 -0700] - = compute_limits: sizelimit=-1, timelimit=-1

[16/Jul/2015:15:31:05 -0700] - Calling plugin 'Account Usability Plugin' #1 
type 403

[16/Jul/2015:15:31:05 -0700] account-usability-plugin - -- auc_pre_search

[16/Jul/2015:15:31:05 -0700] account-usability-plugin - -- auc_pre_op

[16/Jul/2015:15:31:05 -0700] - Calling plugin 'ACL preoperation' #2 type 403

[16/Jul/2015:15:31:05 -0700] - Calling plugin 'deref' #4 type 403

[16/Jul/2015:15:31:05 -0700] deref-plugin - -- deref_pre_search

[16/Jul/2015:15:31:05 -0700] deref-plugin - -- deref_pre_op

[16/Jul/2015:15:31:05 -0700] - Calling plugin 'Legacy replication preoperation 
plugin' #6 type 403

[16/Jul/2015:15:31:05 -0700] - Calling plugin 'Multimaster replication 
preoperation plugin' #9 type 403

[16/Jul/2015:15:31:05 -0700] - = slapi_reslimit_get_integer_limit() conn=0x0, 
handle=0

[16/Jul/2015:15:31:05 -0700] - = slapi_reslimit_get_integer_limit() returning 
NO VALUE

[16/Jul/2015:15:31:05 -0700] - = find_entry_internal 

Re: [389-users] 389-ds and Multi CPU's

[Fong, Trevor]

Last message got embargoed for being over-length.  Trimming this mail.

Trev

From: Fong, Trevor Fong trevor.f...@ubc.camailto:trevor.f...@ubc.ca
Date: Thursday, December 11, 2014 at 9:03 AM
To: mreyno...@redhat.commailto:mreyno...@redhat.com 
mreyno...@redhat.commailto:mreyno...@redhat.com, 
389-users@lists.fedoraproject.orgmailto:389-users@lists.fedoraproject.org 
389-users@lists.fedoraproject.orgmailto:389-users@lists.fedoraproject.org
Subject: Re: [389-users] 389-ds and Multi CPU's

Hi Mark,

Sorry for the lack of response – I was off yesterday.
There were no errors when I made the updates, but when I restated the dirsrv’s, 
I got the following error:

[10/Dec/2014:23:52:20 -0800] - 389-Directory/1.2.11.15 B2014.300.2010 starting 
up
[10/Dec/2014:23:52:20 -0800] - attr_index_config: from ldbm instance init: 
Failed to parse idscanlimit info: 53:attr_index_parse_idlistsize: value 
inetOrgPerson
 violates syntax for attribute objectClass

Trev

From: Mark Reynolds marey...@redhat.commailto:marey...@redhat.com
Reply-To: mreyno...@redhat.commailto:mreyno...@redhat.com 
mreyno...@redhat.commailto:mreyno...@redhat.com
Date: Tuesday, December 9, 2014 at 3:21 PM
To: Trevor Fong trevor.f...@ubc.camailto:trevor.f...@ubc.ca, 
mreyno...@redhat.commailto:mreyno...@redhat.com 
mreyno...@redhat.commailto:mreyno...@redhat.com, 
389-users@lists.fedoraproject.orgmailto:389-users@lists.fedoraproject.org 
389-users@lists.fedoraproject.orgmailto:389-users@lists.fedoraproject.org
Subject: Re: [389-users] 389-ds and Multi CPU's


On 12/09/2014 05:58 PM, Fong, Trevor wrote:
Hi Mark,

I’ve afraid adding nsIndexIDListScanLimit to 
cn=objectclass,cn=index,cn=userRoot,cn=ldbm database,cn=plugins,cn=config 
doesn’t seem to have any effect.  Did this:


dn: cn=objectclass,cn=index,cn=userRoot,cn=ldbm database,cn=plugins,cn=config

changetype: modify

add: nsIndexIDListScanLimit

nsIndexIDListScanLimit: limit=-1 type=eq values=inetOrgPerson

Hmm that's very strange, try this:

dn: cn=objectclass,cn=index,cn=userRoot,cn=ldbm database,cn=plugins,cn=config
changetype: modify
replace: nsIndexIDListScanLimit
nsIndexIDListScanLimit: limit=-1 type=eq values=inetOrgPerson
nsIndexIDListScanLimit: limit=-1 type=eq flags=AND values=inetOrgPerson

Also, is userRoot the mapping tree that refers to your dc=your dc?  I just 
want to make sure we are updating the correct objectclass index configuration 
entry.

This should get rid of those unindexed searches.  Is there anything in the 389 
DS errors log regarding these settings?

Thanks,
Mark




--
389 users mailing list
389-users@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/389-users

Re: [389-users] 389-ds and Multi CPU's

[Fong, Trevor]

Hi Mark,

Sorry for the lack of response – I was off yesterday.
There were no errors when I made the updates, but when I restated the dirsrv’s, 
I got the following error:

[10/Dec/2014:23:52:20 -0800] - 389-Directory/1.2.11.15 B2014.300.2010 starting 
up
[10/Dec/2014:23:52:20 -0800] - attr_index_config: from ldbm instance init: 
Failed to parse idscanlimit info: 53:attr_index_parse_idlistsize: value 
inetOrgPerson
 violates syntax for attribute objectClass

Trev

From: Mark Reynolds marey...@redhat.commailto:marey...@redhat.com
Reply-To: mreyno...@redhat.commailto:mreyno...@redhat.com 
mreyno...@redhat.commailto:mreyno...@redhat.com
Date: Tuesday, December 9, 2014 at 3:21 PM
To: Trevor Fong trevor.f...@ubc.camailto:trevor.f...@ubc.ca, 
mreyno...@redhat.commailto:mreyno...@redhat.com 
mreyno...@redhat.commailto:mreyno...@redhat.com, 
389-users@lists.fedoraproject.orgmailto:389-users@lists.fedoraproject.org 
389-users@lists.fedoraproject.orgmailto:389-users@lists.fedoraproject.org
Subject: Re: [389-users] 389-ds and Multi CPU's


On 12/09/2014 05:58 PM, Fong, Trevor wrote:
Hi Mark,

I’ve afraid adding nsIndexIDListScanLimit to 
cn=objectclass,cn=index,cn=userRoot,cn=ldbm database,cn=plugins,cn=config 
doesn’t seem to have any effect.  Did this:


dn: cn=objectclass,cn=index,cn=userRoot,cn=ldbm database,cn=plugins,cn=config

changetype: modify

add: nsIndexIDListScanLimit

nsIndexIDListScanLimit: limit=-1 type=eq values=inetOrgPerson

Hmm that's very strange, try this:

dn: cn=objectclass,cn=index,cn=userRoot,cn=ldbm database,cn=plugins,cn=config
changetype: modify
replace: nsIndexIDListScanLimit
nsIndexIDListScanLimit: limit=-1 type=eq values=inetOrgPerson
nsIndexIDListScanLimit: limit=-1 type=eq flags=AND values=inetOrgPerson

Also, is userRoot the mapping tree that refers to your dc=your dc?  I just 
want to make sure we are updating the correct objectclass index configuration 
entry.

This should get rid of those unindexed searches.  Is there anything in the 389 
DS errors log regarding these settings?

Thanks,
Mark



CPU was still in the 70’s.  Logconv.pl output looks much the same.  Follows.

Thanks a lot,
Trev


Total Log Lines Analysed:  424930


--- Access Log Output 

Start of Logs:09/Dec/2014:11:20:00
End of Logs:  09/Dec/2014:14:09:20

Processed Log Time:  2 Hours, 49 Minutes, 20 Seconds

Restarts: 0
Total Connections:3909
 - StartTLS Connections:  0
 - LDAPS Connections: 3534
 - LDAPI Connections: 0
Peak Concurrent Connections:  6
Total Operations: 61321
Total Results:61321
Overall Performance:  100.0%

Searches: 58178 (5.73/sec)  (343.57/min)
Modifications:266   (0.03/sec)  (1.57/min)
Adds: 93(0.01/sec)  (0.55/min)
Deletes:  0 (0.00/sec)  (0.00/min)
Mod RDNs: 0 (0.00/sec)  (0.00/min)
Compares: 0 (0.00/sec)  (0.00/min)
Binds:2570  (0.25/sec)  (15.18/min)

Proxied Auth Operations:  0
Persistent Searches:  0
Internal Operations:  0
Entry Operations: 0
Extended Operations:  214
Abandoned Requests:   0
Smart Referrals Received: 0

VLV Operations:   0
VLV Unindexed Searches:   0
VLV Unindexed Components: 0
SORT Operations:  1

Entire Search Base Queries:   0
Paged Searches:   546
Unindexed Searches:   1
Unindexed Components: 545

  Unindexed Search #1 (notes=A)
  -  Date/Time: 09/Dec/2014:13:19:05
  -  Connection Number: 348121
  -  Operation Number:  2
  -  Etime: 0
  -  Nentries:  100
  -  IP Address:10.7.8.105
  -  Search Base:   ou=employees,dc=our dc
  -  Search Scope:  2 (subtree)
  -  Search Filter: ((objectclass=inetorgperson))

  Unindexed Component #1 (notes=U)
  -  Date/Time: 09/Dec/2014:11:30:16
  -  Connection Number: 344719
  -  Operation Number:  101
  -  Etime: 0
  -  Nentries:  100
  -  IP Address:Unknown_Host
  -  Search Base:   ou=employees,dc=our dc
  -  Search Scope:  2 (subtree)
  -  Search Filter: ((objectclass=inetorgperson))

  Unindexed Component #2 (notes=U)
  -  Date/Time: 09/Dec/2014:11:31:47
  -  Connection Number: 344719
  -  Operation Number:  105
  -  Etime: 0
  -  Nentries:  100
  -  IP Address:Unknown_Host
  -  Search Base:   ou=employees,dc=our dc
  -  Search Scope:  2 (subtree)
  -  Search Filter: ((objectclass=inetorgperson))
…snip
 Unindexed Component #545 (notes=U)
  -  Date/Time: 09/Dec/2014:13:19:21
  -  Connection Number: 348121
  -  Operation Number:  9

Re: [389-users] 389-ds and Multi CPU's

[Fong, Trevor]

2.16.840.1.113730.3.5.5 End Replication Request (incremental 
update)


- Top 20 Most Requested Attributes -

78174   1.1
56799   o
28712   description
28712   cn
28712   nsAccountLock
28711   givenName
28711   ubcEduCwlPUID
28711   uid
28711   displayName
28711   employeeNumber
28711   mail
28711   objectClass
28711   sn
28400   ou
28400   passwordRetryCount
28400   distinguishedName
28399   businessCategory
28399   employeeType
28399   facsimileTelephoneNumber
28399   groups


- Recommendations -

 1.  You have unindexed searches, this can be caused from a search on an 
unindexed attribute, or your returned results exceeded the allidsthreshold.  
Unindexed searches are not recommended. To refuse unindexed searches, switch 
'nsslapd-require-index' to 'on' under your database entry (e.g. 
cn=UserRoot,cn=ldbm database,cn=plugins,cn=config).

 2.  You have unindexed components, this can be caused from a search on an 
unindexed attribute, or your returned results exceeded the allidsthreshold.  
Unindexed components are not recommended. To refuse unindexed searches, switch 
'nsslapd-require-index' to 'on' under your database entry (e.g. 
cn=UserRoot,cn=ldbm database,cn=plugins,cn=config).

Cleaning up temp files...
Done.







From: Mark Reynolds marey...@redhat.commailto:marey...@redhat.com
Reply-To: mreyno...@redhat.commailto:mreyno...@redhat.com 
mreyno...@redhat.commailto:mreyno...@redhat.com
Date: Tuesday, December 9, 2014 at 7:25 AM
To: 
389-users@lists.fedoraproject.orgmailto:389-users@lists.fedoraproject.org 
389-users@lists.fedoraproject.orgmailto:389-users@lists.fedoraproject.org, 
Trevor Fong trevor.f...@ubc.camailto:trevor.f...@ubc.ca
Subject: Re: [389-users] 389-ds and Multi CPU's


On 12/08/2014 05:43 PM, Fong, Trevor wrote:
Hi Mike,
It's Mark :-)  I get that a lot for some reason.

Thanks for the reply.  The typical result of the result is:

[08/Dec/2014:07:08:04 -0800] conn=130262 op=1 RESULT err=0 tag=101 nentries=5 
etime=0
Yeah this looks fine.

There are no notes=A/notes=U in the results.
Do you mean in the entire access log, or just for that search?

Can you run logconv.pl and post the results?   logconv.pl -V access logs

Thanks Trevor,
Mark
Objectclass and member are both indexed.
There were 30,000-odd searches on conn=130262, which took 34 mins.

Thanks,
Trev






From: Mark Reynolds marey...@redhat.commailto:marey...@redhat.com
Reply-To: mreyno...@redhat.commailto:mreyno...@redhat.com 
mreyno...@redhat.commailto:mreyno...@redhat.com
Date: Monday, December 8, 2014 at 11:29 AM
To: 
389-users@lists.fedoraproject.orgmailto:389-users@lists.fedoraproject.org 
389-users@lists.fedoraproject.orgmailto:389-users@lists.fedoraproject.org, 
Trevor Fong trevor.f...@ubc.camailto:trevor.f...@ubc.ca
Subject: Re: [389-users] 389-ds and Multi CPU's


On 12/08/2014 02:08 PM, Fong, Trevor wrote:
Hi Everyone,

We’ve inherited a 389-ds system (1.2.11.15-48.el6_6) that is running on a VM 
provisioned with a single CPU.  We have been experiencing high CPU with a 
client that connects with a single connection, and then runs large amounts of 
queries of the form:

SRCH base=ou=GROUPS,dc=our dc scope=2 
filter=((objectClass=groupOfNames)(member=uid=loginname,ou=EMPLOYEES,our 
dc)) attrs=“1.1
Trevor,

From the access log, what is the result of this search?  etime?  It there a 
notes=U/notes=A in the result?  It could be an unindexed search which would 
cause the high CPU.

Thanks,
Mark

We’re wondering if adding extra CPU’s to the vm will make things better.  The 
original engineer noted that at the time of implementation, he had come across 
some notes that indicated that the underlying process was single threaded and 
adding extra CPU’s would not make any improvement; in fact, on heavily loaded 
vm infrastructure like ours, may make things worse as the the vm would have to 
wait for the CPU to become available.  Is this still true?

Thanks a lot,
Trev



--
389 users mailing list
389-users@lists.fedoraproject.orgmailto:389-users@lists.fedoraproject.orghttps://admin.fedoraproject.org/mailman/listinfo/389-users




--
389 users mailing list
389-users@lists.fedoraproject.orgmailto:389-users@lists.fedoraproject.orghttps://admin.fedoraproject.org/mailman/listinfo/389-users

--
389 users mailing list
389-users@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/389-users

Re: [389-users] 389-ds and Multi CPU's

[Fong, Trevor]

Hi Mike,

Thanks for the reply.  The typical result of the result is:

[08/Dec/2014:07:08:04 -0800] conn=130262 op=1 RESULT err=0 tag=101 nentries=5 
etime=0

There are no notes=A/notes=U in the results.
Objectclass and member are both indexed.
There were 30,000-odd searches on conn=130262, which took 34 mins.

Thanks,
Trev






From: Mark Reynolds marey...@redhat.commailto:marey...@redhat.com
Reply-To: mreyno...@redhat.commailto:mreyno...@redhat.com 
mreyno...@redhat.commailto:mreyno...@redhat.com
Date: Monday, December 8, 2014 at 11:29 AM
To: 
389-users@lists.fedoraproject.orgmailto:389-users@lists.fedoraproject.org 
389-users@lists.fedoraproject.orgmailto:389-users@lists.fedoraproject.org, 
Trevor Fong trevor.f...@ubc.camailto:trevor.f...@ubc.ca
Subject: Re: [389-users] 389-ds and Multi CPU's


On 12/08/2014 02:08 PM, Fong, Trevor wrote:
Hi Everyone,

We’ve inherited a 389-ds system (1.2.11.15-48.el6_6) that is running on a VM 
provisioned with a single CPU.  We have been experiencing high CPU with a 
client that connects with a single connection, and then runs large amounts of 
queries of the form:

SRCH base=ou=GROUPS,dc=our dc scope=2 
filter=((objectClass=groupOfNames)(member=uid=loginname,ou=EMPLOYEES,our 
dc)) attrs=“1.1
Trevor,

From the access log, what is the result of this search?  etime?  It there a 
notes=U/notes=A in the result?  It could be an unindexed search which would 
cause the high CPU.

Thanks,
Mark

We’re wondering if adding extra CPU’s to the vm will make things better.  The 
original engineer noted that at the time of implementation, he had come across 
some notes that indicated that the underlying process was single threaded and 
adding extra CPU’s would not make any improvement; in fact, on heavily loaded 
vm infrastructure like ours, may make things worse as the the vm would have to 
wait for the CPU to become available.  Is this still true?

Thanks a lot,
Trev



--
389 users mailing list
389-users@lists.fedoraproject.orgmailto:389-users@lists.fedoraproject.orghttps://admin.fedoraproject.org/mailman/listinfo/389-users

--
389 users mailing list
389-users@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/389-users

Re: [389-users] Switching from 389-ds-base 1.2.11 to 1.3.3

[Fong, Trevor]

Hi Again Everyone,

Looking at other threads on this group, it looks like for EL6, only 1.2.11 is 
available.
The latest official version is 1.2.11.15-48.el6_6.  Does this contain all the 
changes from the COPR versions? From the Release Notes, it looks like the last 
available was 1.2.11.32?

Thanks,
Trev

From: Fong, Trevor Fong trevor.f...@ubc.camailto:trevor.f...@ubc.ca
Reply-To: 
389-users@lists.fedoraproject.orgmailto:389-users@lists.fedoraproject.org 
389-users@lists.fedoraproject.orgmailto:389-users@lists.fedoraproject.org
Date: Wednesday, November 26, 2014 at 3:07 PM
To: 
389-users@lists.fedoraproject.orgmailto:389-users@lists.fedoraproject.org 
389-users@lists.fedoraproject.orgmailto:389-users@lists.fedoraproject.org
Subject: [389-users] Switching from 389-ds-base 1.2.11 to 1.3.3

Hi Everyone,

I’m afraid this will be a total newb question.

I’m currently running 389 DS 1.2.11.29-1.el6 and have seen the notice that 
1.2.11.X will be discontinued soon (as of 24-Oct-2014).  [It probably has 
already been discontinued since the COPR repository is no longer there.]

I’d like to switch from 1.2.11 to 1.3.3.5 and have yum downgraded to 
1.2.11.15-48.el6_6.
However, when I try to yum upgrade 389-ds-base 389-ds-base-devel 
389-ds-base-libs” it tells me No Packages marked for Update”.  Am I doing 
something wrong?

I’ve followed the directions on http://www.port389.org/docs/389ds/download.html 
and epel-release-6-8.noarch.rpm is already installed.  [That doc is still 
telling me to install the temporary COPR, by-the-way.]

Thanks in advance,
Trev
--
389 users mailing list
389-users@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/389-users

Re: [389-users] Trouble with Replication - Initializing Consumers

[Fong, Trevor]

Hi Everyone,

I’m afraid this is still on-going.
Weird thing is that we have one other consumer that was successfully 
initialized (eldap3); the other two (eldap4, eldap5) fail on initialization 
with the error:

NSMMReplicationPlugin - replica_replace_ruv_tombstone: failed to update 
replication update vector for replica dc=stg,dc=id,dc=ubc,dc=ca: LDAP error - 1

I’ve tried importing both the master’s and the other consumer’s db’s into the 2 
failed consumers but then the master’s error logs say

NSMMReplicationPlugin - agmt=cn=eldap4 consumer (eldap4:636): Replica has 
a different generation ID than the local data.
NSMMReplicationPlugin - agmt=cn=eldap5 consumer (eldap5:636): Replica has 
a different generation ID than the local data.

Anyone have any pointers about this issue?

Thanks a lot,
Trev


From: 389-users-boun...@lists.fedoraproject.org 
[mailto:389-users-boun...@lists.fedoraproject.org] On Behalf Of Fong, Trevor
Sent: Wednesday, September 24, 2014 4:21 PM
To: 389-users@lists.fedoraproject.org
Subject: Re: [389-users] Trouble with Replication - Initializing Consumers

Hi Noriko,

Thanks for your response.  We’re running 389-Directory/1.2.11.29 
B2014.094.1833.  Which release would you advise we update to?
I’ve also made sure we’ve set the maxber and cache sizes to sufficiently large 
values and are not getting any errors or warnings regarding those.

Thanks,
Trev


From: 
389-users-boun...@lists.fedoraproject.orgmailto:389-users-boun...@lists.fedoraproject.org
 [mailto:389-users-boun...@lists.fedoraproject.org] On Behalf Of Noriko Hosoi
Sent: Wednesday, September 24, 2014 2:47 PM
To: 389-users@lists.fedoraproject.orgmailto:389-users@lists.fedoraproject.org
Subject: Re: [389-users] Trouble with Replication - Initializing Consumers

Hello Trevor,

What's the version of 389-ds-base?

Your server may not have this fix (https://fedorahosted.org/389/ticket/47606) 
yet.  This is in 389-ds-base-1.3.1 and newer.

Please check your maxbersize and nsslapd-cachememsize and if they are not large 
enough, try increase them.

1. maxbersize: If the size of an entry is larger than the consumer's

   maxbersize, the following error used to be logged:

 Incoming BER Element was too long, max allowable is ### bytes.

 Change the nsslapd-maxbersize attribute in cn=config to increase.

   This message does not indicate how large the maxbersize needs to be.

   This patch adds the code to retrieve the failed ber size.

   Revised message:

 Incoming BER Element was @@@ bytes, max allowable is ### bytes.

 Change the nsslapd-maxbersize attribute in cn=config to increase.

   Note: There is no lber API that returns the ber size if it fails to

   handle the ber.  This patch borrows the internal structure of ber

   and get the size.  This could be risky since the size or structure

   of the ber could be updated in the openldap/mozldap lber.

2. cache size: The bulk import depends upon the nsslapd-cachememsize

   value in the backend instance entry (e.g., cn=userRoot,cn=ldbm

   database,cn=plugins,cn=config).  If an entry size is larger than

   the cachememsize, the bulk import used to fail with this message:

 import userRoot: REASON: entry too large (@@@ bytes) for the

 import buffer size (### bytes).  Try increasing nsslapd-

 cachememsize.

   Also, the message follows the skipping entry message:

 import userRoot: WARNING: skipping entry DN

   but actually, it did NOT skip the entry and continue the bulk

   import, but it failed there and completely wiped out the backend

   database.

   This patch modifies the message as follows:

 import userRoot: REASON: entry too large (@@@ bytes) for the

 effective import buffer size (### bytes). Try increasing nsslapd-

 cachememsize for the backend instance userRoot.

   and as the message mentions, it just skips the failed entry and

   continues the bulk import.

Fong, Trevor wrote:
Hi Everyone,

I’m having trouble initializing consumers for replication.
On 2 of the 3 consumers, I get the following message after trying to 
re-initialize the consumers from the 389-console:
NSMMReplicationPlugin - replica_replace_ruv_tombstone: failed to update 
replication update vector for replica dc=stg,dc=id,dc=ubc,dc=ca: LDAP error - 1
I’ve tried to db2ldif dump the database and re-import everything again with 
ldif2db before re-initializing.
I’ve even tried to ldif2db an export from the master before re-initializing.
I always get the above error.

Does anyone know what’s going on or have any suggestions?
/var/log/dirsrv/slapd-instance/errors extract follows.

Thanks a lot,
Trev



[24/Sep/2014:14:13:13 -0700] - WARNING: Import is running with 
nsslapd-db-private-import-mem on; No other process is allowed to access the 
database
[24/Sep/2014:14:13:34 -0700] - import userRoot: Processed 19704 entries -- 
average rate 980.1/sec, recent rate 980.1/sec, hit ratio 0%
[24/Sep/2014:14:13:37 -0700] - ERROR

[389-users] Trouble with Replication - Initializing Consumers

[Fong, Trevor]

Hi Everyone,

I'm having trouble initializing consumers for replication.
On 2 of the 3 consumers, I get the following message after trying to 
re-initialize the consumers from the 389-console:
NSMMReplicationPlugin - replica_replace_ruv_tombstone: failed to update 
replication update vector for replica dc=stg,dc=id,dc=ubc,dc=ca: LDAP error - 1
I've tried to db2ldif dump the database and re-import everything again with 
ldif2db before re-initializing.
I've even tried to ldif2db an export from the master before re-initializing.
I always get the above error.

Does anyone know what's going on or have any suggestions?
/var/log/dirsrv/slapd-instance/errors extract follows.

Thanks a lot,
Trev



[24/Sep/2014:14:13:13 -0700] - WARNING: Import is running with 
nsslapd-db-private-import-mem on; No other process is allowed to access the 
database
[24/Sep/2014:14:13:34 -0700] - import userRoot: Processed 19704 entries -- 
average rate 980.1/sec, recent rate 980.1/sec, hit ratio 0%
[24/Sep/2014:14:13:37 -0700] - ERROR bulk import abandoned
[24/Sep/2014:14:13:37 -0700] - import userRoot: Aborting all Import threads...
[24/Sep/2014:14:13:44 -0700] - import userRoot: Import threads aborted.
[24/Sep/2014:14:13:44 -0700] - import userRoot: Closing files...
[24/Sep/2014:14:13:44 -0700] - libdb: userRoot/id2entry.db4: unable to flush: 
No such file or directory
[24/Sep/2014:14:13:44 -0700] - libdb: userRoot/cn.db4: unable to flush: No such 
file or directory
[24/Sep/2014:14:13:44 -0700] - libdb: userRoot/aci.db4: unable to flush: No 
such file or directory
[24/Sep/2014:14:13:44 -0700] - libdb: userRoot/departmentnumber.db4: unable to 
flush: No such file or directory
[24/Sep/2014:14:13:45 -0700] - libdb: userRoot/member.db4: unable to flush: No 
such file or directory
[24/Sep/2014:14:13:45 -0700] - libdb: userRoot/krbprincipalname.db4: unable to 
flush: No such file or directory
[24/Sep/2014:14:13:45 -0700] - libdb: userRoot/parentid.db4: unable to flush: 
No such file or directory
[24/Sep/2014:14:13:45 -0700] - libdb: userRoot/nsuniqueid.db4: unable to flush: 
No such file or directory
[24/Sep/2014:14:13:45 -0700] - libdb: userRoot/ou.db4: unable to flush: No such 
file or directory
[24/Sep/2014:14:13:45 -0700] - libdb: userRoot/ubceducwlpuid.db4: unable to 
flush: No such file or directory
[24/Sep/2014:14:13:45 -0700] - libdb: userRoot/entryrdn.db4: unable to flush: 
No such file or directory
[24/Sep/2014:14:13:45 -0700] - libdb: userRoot/sn.db4: unable to flush: No such 
file or directory
[24/Sep/2014:14:13:45 -0700] - libdb: userRoot/givenName.db4: unable to flush: 
No such file or directory
[24/Sep/2014:14:13:45 -0700] - libdb: userRoot/objectclass.db4: unable to 
flush: No such file or directory
[24/Sep/2014:14:13:45 -0700] - libdb: userRoot/uniquemember.db4: unable to 
flush: No such file or directory
[24/Sep/2014:14:13:45 -0700] - libdb: userRoot/uid.db4: unable to flush: No 
such file or directory
[24/Sep/2014:14:13:45 -0700] - libdb: userRoot/memberOf.db4: unable to flush: 
No such file or directory
[24/Sep/2014:14:13:45 -0700] - libdb: userRoot/mail.db4: unable to flush: No 
such file or directory
[24/Sep/2014:14:13:45 -0700] - libdb: userRoot/employeenumber.db4: unable to 
flush: No such file or directory
[24/Sep/2014:14:13:45 -0700] - import userRoot: Import failed.
[24/Sep/2014:14:13:45 -0700] - process_bulk_import_op: NULL target sdn
[24/Sep/2014:14:13:46 -0700] NSMMReplicationPlugin - 
replica_replace_ruv_tombstone: failed to update replication update vector for 
replica dc=stg,dc=id,dc=ubc,dc=ca: LDAP error - 1
--
389 users mailing list
389-users@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/389-users

Re: [389-users] Trouble with Replication - Initializing Consumers

[Fong, Trevor]

And nope, reindexing with
/usr/lib64/dirsrv/slapd-instance/db2index.pl -v -D cn=Directory Manager -w 
xxx -n userRoot
didn't help either.

Trev

From: 389-users-boun...@lists.fedoraproject.org 
[mailto:389-users-boun...@lists.fedoraproject.org] On Behalf Of Fong, Trevor
Sent: Wednesday, September 24, 2014 2:25 PM
To: 389-users@lists.fedoraproject.org
Subject: [389-users] Trouble with Replication - Initializing Consumers

Hi Everyone,

I'm having trouble initializing consumers for replication.
On 2 of the 3 consumers, I get the following message after trying to 
re-initialize the consumers from the 389-console:
NSMMReplicationPlugin - replica_replace_ruv_tombstone: failed to update 
replication update vector for replica dc=stg,dc=id,dc=ubc,dc=ca: LDAP error - 1
I've tried to db2ldif dump the database and re-import everything again with 
ldif2db before re-initializing.
I've even tried to ldif2db an export from the master before re-initializing.
I always get the above error.

Does anyone know what's going on or have any suggestions?
/var/log/dirsrv/slapd-instance/errors extract follows.

Thanks a lot,
Trev



[24/Sep/2014:14:13:13 -0700] - WARNING: Import is running with 
nsslapd-db-private-import-mem on; No other process is allowed to access the 
database
[24/Sep/2014:14:13:34 -0700] - import userRoot: Processed 19704 entries -- 
average rate 980.1/sec, recent rate 980.1/sec, hit ratio 0%
[24/Sep/2014:14:13:37 -0700] - ERROR bulk import abandoned
[24/Sep/2014:14:13:37 -0700] - import userRoot: Aborting all Import threads...
[24/Sep/2014:14:13:44 -0700] - import userRoot: Import threads aborted.
[24/Sep/2014:14:13:44 -0700] - import userRoot: Closing files...
[24/Sep/2014:14:13:44 -0700] - libdb: userRoot/id2entry.db4: unable to flush: 
No such file or directory
[24/Sep/2014:14:13:44 -0700] - libdb: userRoot/cn.db4: unable to flush: No such 
file or directory
[24/Sep/2014:14:13:44 -0700] - libdb: userRoot/aci.db4: unable to flush: No 
such file or directory
[24/Sep/2014:14:13:44 -0700] - libdb: userRoot/departmentnumber.db4: unable to 
flush: No such file or directory
[24/Sep/2014:14:13:45 -0700] - libdb: userRoot/member.db4: unable to flush: No 
such file or directory
[24/Sep/2014:14:13:45 -0700] - libdb: userRoot/krbprincipalname.db4: unable to 
flush: No such file or directory
[24/Sep/2014:14:13:45 -0700] - libdb: userRoot/parentid.db4: unable to flush: 
No such file or directory
[24/Sep/2014:14:13:45 -0700] - libdb: userRoot/nsuniqueid.db4: unable to flush: 
No such file or directory
[24/Sep/2014:14:13:45 -0700] - libdb: userRoot/ou.db4: unable to flush: No such 
file or directory
[24/Sep/2014:14:13:45 -0700] - libdb: userRoot/ubceducwlpuid.db4: unable to 
flush: No such file or directory
[24/Sep/2014:14:13:45 -0700] - libdb: userRoot/entryrdn.db4: unable to flush: 
No such file or directory
[24/Sep/2014:14:13:45 -0700] - libdb: userRoot/sn.db4: unable to flush: No such 
file or directory
[24/Sep/2014:14:13:45 -0700] - libdb: userRoot/givenName.db4: unable to flush: 
No such file or directory
[24/Sep/2014:14:13:45 -0700] - libdb: userRoot/objectclass.db4: unable to 
flush: No such file or directory
[24/Sep/2014:14:13:45 -0700] - libdb: userRoot/uniquemember.db4: unable to 
flush: No such file or directory
[24/Sep/2014:14:13:45 -0700] - libdb: userRoot/uid.db4: unable to flush: No 
such file or directory
[24/Sep/2014:14:13:45 -0700] - libdb: userRoot/memberOf.db4: unable to flush: 
No such file or directory
[24/Sep/2014:14:13:45 -0700] - libdb: userRoot/mail.db4: unable to flush: No 
such file or directory
[24/Sep/2014:14:13:45 -0700] - libdb: userRoot/employeenumber.db4: unable to 
flush: No such file or directory
[24/Sep/2014:14:13:45 -0700] - import userRoot: Import failed.
[24/Sep/2014:14:13:45 -0700] - process_bulk_import_op: NULL target sdn
[24/Sep/2014:14:13:46 -0700] NSMMReplicationPlugin - 
replica_replace_ruv_tombstone: failed to update replication update vector for 
replica dc=stg,dc=id,dc=ubc,dc=ca: LDAP error - 1
--
389 users mailing list
389-users@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/389-users

[389-users] db2index Not Stopping

[Fong, Trevor]

Hi Everyone,

Searches and any kind of access to a particular ou seem to be hanging (yep, 
even tried deleting it and it's children) so I ran db2index and left it running 
all night.  It's still running 20 hours later - is that normal?  If it's not, 
is it OK to kill the process?  What would you recommend?

Here's the kind of output I'm getting:

[30/May/2014:07:46:27 -0700] - WARNING: Import is running with 
nsslapd-db-private-import-mem on; No other process is allowed to access the 
database
[30/May/2014:07:46:27 -0700] - reindex userRoot: Sweep done.
[30/May/2014:07:46:27 -0700] - reindex userRoot: Beginning pass number 210
[30/May/2014:07:46:47 -0700] - reindex userRoot: Processed 678827 entries (pass 
210) -- average rate 214714423.4/sec, recent rate 0.0/sec, hit ratio 0%
[30/May/2014:07:47:07 -0700] - reindex userRoot: Processed 678827 entries (pass 
210) -- average rate 107357211.7/sec, recent rate 0.0/sec, hit ratio 0%
[30/May/2014:07:47:27 -0700] - reindex userRoot: Processed 678827 entries (pass 
210) -- average rate 71571474.5/sec, recent rate 0.0/sec, hit ratio 0%
[30/May/2014:07:47:47 -0700] - reindex userRoot: Processed 678827 entries (pass 
210) -- average rate 53678605.9/sec, recent rate 0.0/sec, hit ratio 0%
[30/May/2014:07:48:08 -0700] - reindex userRoot: Processed 678827 entries (pass 
210) -- average rate 42942884.7/sec, recent rate 0.0/sec, hit ratio 0%
[30/May/2014:07:48:28 -0700] - reindex userRoot: Processed 678827 entries (pass 
210) -- average rate 35489987.3/sec, recent rate 0.0/sec, hit ratio 0%
[30/May/2014:07:48:48 -0700] - reindex userRoot: Processed 678827 entries (pass 
210) -- average rate 30455946.6/sec, recent rate 0.0/sec, hit ratio 0%
[30/May/2014:07:49:08 -0700] - reindex userRoot: Processed 678827 entries (pass 
210) -- average rate 26672599.2/sec, recent rate 0.0/sec, hit ratio 0%
[30/May/2014:07:49:28 -0700] - reindex userRoot: Processed 678827 entries (pass 
210) -- average rate 23725350.7/sec, recent rate 0.0/sec, hit ratio 0%
[30/May/2014:07:49:48 -0700] - reindex userRoot: Processed 678827 entries (pass 
210) -- average rate 21364619.2/sec, recent rate 0.0/sec, hit ratio 0%
[30/May/2014:07:50:08 -0700] - reindex userRoot: Processed 678827 entries (pass 
210) -- average rate 19431169.5/sec, recent rate 0.0/sec, hit ratio 0%
[30/May/2014:07:50:28 -0700] - reindex userRoot: Processed 678827 entries (pass 
210) -- average rate 17818624.4/sec, recent rate 0.0/sec, hit ratio 0%
[30/May/2014:07:50:48 -0700] - reindex userRoot: Processed 678827 entries (pass 
210) -- average rate 16453212.5/sec, recent rate 0.0/sec, hit ratio 0%
[30/May/2014:07:51:08 -0700] - reindex userRoot: Processed 678827 entries (pass 
210) -- average rate 15282165.4/sec, recent rate 0.0/sec, hit ratio 0%
[30/May/2014:07:51:28 -0700] - reindex userRoot: Processed 678827 entries (pass 
210) -- average rate 14266739.1/sec, recent rate 0.0/sec, hit ratio 0%
[30/May/2014:07:51:28 -0700] - reindex userRoot: Decided to end this pass 
because the progress rate has dropped below the 50% threshold.
[30/May/2014:07:51:28 -0700] - reindex userRoot: Ending pass number 210 ...
[30/May/2014:07:51:28 -0700] - reindex userRoot: Foreman is done; waiting for 
workers to finish...
[30/May/2014:07:51:28 -0700] - reindex userRoot: Workers finished; cleaning 
up...
[30/May/2014:07:51:28 -0700] - reindex userRoot: Workers cleaned up.
[30/May/2014:07:51:28 -0700] - reindex userRoot: Sweeping files for merging 
later...

We're running 389-ds-base.x86_64 1.2.11.29-1.el6

Thanks a lot,
Trev

-
Trevor Fong - Senior Programmer Analyst
Identity and Access Management Group
University of British Columbia - Information Technology
6356 Agricultural Road, Vancouver, BC, V6T 1Z2, Canada
Ph: (604) 827-5247


--
389 users mailing list
389-users@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/389-users

Re: [389-users] Yum Update vs Yum Upgrade

[Fong, Trevor]

Hi Everyone,

Thanks for your messages.  Further to this, it looks like the default settings 
for RHEL6 does indeed make yum upgrade == yum update --obsoletes.
Looking at my /etc/yum.conf:
   obsoletes=1

Which means that obsoletes is true by default on update.

Thanks,
Trev

From: 389-users-boun...@lists.fedoraproject.org 
[mailto:389-users-boun...@lists.fedoraproject.org] On Behalf Of David Barr
Sent: May-20-14 10:41 AM
To: General discussion list for the 389 Directory server project.
Subject: Re: [389-users] Yum Update vs Yum Upgrade

Good Morning,

yum upgrade === yum update --obsoletes

where the -obsoletes flag says to remove packages that have been made 
obsolete by the upgrade (say you're upgrading from 5.9 to 6.5).

Cheers!
David

On May 20, 2014, at 10:00, Rich Megginson 
rmegg...@redhat.commailto:rmegg...@redhat.com wrote:


On 05/20/2014 10:58 AM, Fong, Trevor wrote:
Hi Everyone,

After taking over the LDAP service from a colleague, I updated the 389 DS 
service to the latest release by issuing a yum update   Then when 
searching around the 389-ds documentation, I came across the install page that 
said that I must use yum upgrade ... and not update.

My question is what's the difference between yum update and upgrade?

I think they are the same.  If someone knows any difference between yum 
update and yum upgrade I'd like to hear it.


Is it OK if I leave it like that or should I do the yum upgrade?

It is OK to leave it like that.


The service seems to be running OK after yum update, so far.

I went from 1.2.11.15-14.el6_4 -- 1.2.11.29-1.el6

Thanks,
Trev

-
Trevor Fong - Senior Programmer Analyst
Identity and Access Management Group
University of British Columbia - Information Technology
6356 Agricultural Road, Vancouver, BC, V6T 1Z2, Canada
Ph: (604) 827-5247







--

389 users mailing list

389-users@lists.fedoraproject.orgmailto:389-users@lists.fedoraproject.org

https://admin.fedoraproject.org/mailman/listinfo/389-users

--
389 users mailing list
389-users@lists.fedoraproject.orgmailto:389-users@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/389-users

--

David - Offbeat  http://dafydd.livejournal.com
dafydd - Online  http://pgp.mit.edu/
Battalion 4 - Black Rock City Emergency Services Department
Integrity*Commitment*Communication*Support

51525354555657--

Pavlov walks into a bar. The phone rings and he says,

Damn! I forgot to feed the dog!


--
389 users mailing list
389-users@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/389-users

[389-users] Yum Update vs Yum Upgrade

[Fong, Trevor]

Hi Everyone,

After taking over the LDAP service from a colleague, I updated the 389 DS 
service to the latest release by issuing a yum update   Then when 
searching around the 389-ds documentation, I came across the install page that 
said that I must use yum upgrade ... and not update.

My question is what's the difference between yum update and upgrade?  Is it OK 
if I leave it like that or should I do the yum upgrade?  The service seems to 
be running OK after yum update, so far.

I went from 1.2.11.15-14.el6_4 -- 1.2.11.29-1.el6

Thanks,
Trev

-
Trevor Fong - Senior Programmer Analyst
Identity and Access Management Group
University of British Columbia - Information Technology
6356 Agricultural Road, Vancouver, BC, V6T 1Z2, Canada
Ph: (604) 827-5247


--
389 users mailing list
389-users@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/389-users

Re: [389-users] Sync from RDBMS to LDAP

[Fong, Trevor]

Thanks a lot for the lead, Jeff - I'll check it out.
For the thread - there's also Ldap Sync Connector from lsc-project.org, which 
is open source, but commercial support is preferred.

Other suggestions welcome!
Trev

From: 389-users-boun...@lists.fedoraproject.org 
[mailto:389-users-boun...@lists.fedoraproject.org] On Behalf Of Clowser, Jeff
Sent: April-29-14 7:02 AM
To: 'General discussion list for the 389 Directory server project.'
Subject: Re: [389-users] Sync from RDBMS to LDAP

If you want a commercial solution, take a look at the Unbound ID 
Synchronization server.  Depending on how complex your needs are, that may work.


This e-mail and its attachments are confidential and solely for the intended 
addressee(s). Do not share or use them without Fannie Mae's approval. If 
received in error, contact the sender and delete them.



From: 
389-users-boun...@lists.fedoraproject.orgmailto:389-users-boun...@lists.fedoraproject.org
 [mailto:389-users-boun...@lists.fedoraproject.org] On Behalf Of Paul Robert 
Marino
Sent: Tuesday, April 29, 2014 8:55 AM
To: General discussion list for the 389 Directory server project.
Subject: Re: [389-users] Sync from RDBMS to LDAP
Sorry that kind of thing is always a custom scripting job.




-- Sent from my HP Pre3



On Apr 28, 2014 13:27, Fong, Trevor 
trevor.f...@ubc.camailto:trevor.f...@ubc.ca wrote:
*Bump*

Surely we can't be the only ones who want to this?

Trev

From: Fong, Trevor
Sent: April-22-14 3:33 PM
To: '389-users@lists.fedoraproject.org'
Subject: Sync from RDBMS to LDAP

Hi Everyone,

We are in the process of migrating from an old OpenLDAP service to 389-DS.
We currently synchronise users and attributes from an Oracle DB to OpenLDAP 
service using an aging set of custom scripts and DB triggers.
We would like to do something similar for 389-DS but using a 
commercial-off-the-shelf solution.
I was wondering what the good people on this list use or would recommend?

Thanks in advance,
Trev

-
Trevor Fong - Senior Programmer Analyst
Identity and Access Management Group
University of British Columbia - Information Technology
6356 Agricultural Road, Vancouver, BC, V6T 1Z2, Canada
Ph: (604) 827-5247

--
389 users mailing list
389-users@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/389-users

Re: [389-users] Sync from RDBMS to LDAP

[Fong, Trevor]

*Bump*

Surely we can't be the only ones who want to this?

Trev

From: Fong, Trevor
Sent: April-22-14 3:33 PM
To: '389-users@lists.fedoraproject.org'
Subject: Sync from RDBMS to LDAP

Hi Everyone,

We are in the process of migrating from an old OpenLDAP service to 389-DS.
We currently synchronise users and attributes from an Oracle DB to OpenLDAP 
service using an aging set of custom scripts and DB triggers.
We would like to do something similar for 389-DS but using a 
commercial-off-the-shelf solution.
I was wondering what the good people on this list use or would recommend?

Thanks in advance,
Trev

-
Trevor Fong - Senior Programmer Analyst
Identity and Access Management Group
University of British Columbia - Information Technology
6356 Agricultural Road, Vancouver, BC, V6T 1Z2, Canada
Ph: (604) 827-5247

--
389 users mailing list
389-users@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/389-users

[389-users] Sync from RDBMS to LDAP

[Fong, Trevor]

Hi Everyone,

We are in the process of migrating from an old OpenLDAP service to 389-DS.
We currently synchronise users and attributes from an Oracle DB to OpenLDAP 
service using an aging set of custom scripts and DB triggers.
We would like to do something similar for 389-DS but using a 
commercial-off-the-shelf solution.
I was wondering what the good people on this list use or would recommend?

Thanks in advance,
Trev

-
Trevor Fong - Senior Programmer Analyst
Identity and Access Management Group
University of British Columbia - Information Technology
6356 Agricultural Road, Vancouver, BC, V6T 1Z2, Canada
Ph: (604) 827-5247


--
389 users mailing list
389-users@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/389-users