issues with disabling filtered searches for memberURL groups

2023-03-07 Thread Kartik Subbarao
One of the changes from 2.4 to 2.5 is that dynlist groups are now 
returned with (member=memberDN) searches. This is potentially appealing, 
but even with the ITS#9929 performance improvements, given the number of 
dynlist groups we have, search times are significantly impacted.


We'd like to be able to cleanly disable this feature and exclude dynlist 
groups from (member=memberDN) filter consideration. The only way I've 
found so far is to patch the dynlist code itself. What I'm currently 
doing is adding a continue statement right above this line in 
dynlist_search():


https://git.openldap.org/openldap/openldap/-/blob/OPENLDAP_REL_ENG_2_5_14/servers/slapd/overlays/dynlist.c#L1830

That way the member searches are excluded, but dynlists otherwise work 
as expected.


Here is the dynlist config we're using, just basic support for 
groupOfURLs/memberURL:


overlay dynlist
dynlist-attrset groupOfURLs memberURL member

Is there some way to achieve my goal without having to patch the code? 
Or should I open an ITS feature request to add a configurable option to 
exclude dynlists from member searches?


Thanks,

    -Kartik


Any chance to include ITS#9990 fix in 2.5.14?

2023-01-26 Thread Kartik Subbarao
I ran into a passwd exop overlay problem this week when upgrading from 
2.4.57 to 2.5.13 and was able to track it down (ITS#9990). Fortunately 
the fix is very simple, just revert the changes to passwd.c made in 
ITS#8698. I noticed the testing call for 2.5.14 and wanted to ask if it 
might be possible to include this fix in that release.


Totally understood if you guys need to get this out the door as soon as 
possible. just figured I'd ask to see if we can get this fix included :-)


Thanks,

    -Kartik


Re: rwm bindDN context and ppolicy issues

2022-06-23 Thread Kartik Subbarao

On 6/23/22 4:29 PM, Quanah Gibson-Mount wrote:
If slapd segfaults, an ITS with clear reproduction steps should 
generally be filed. I would note that you have not specified the 
OpenLDAP release on which you encountered this problem.  There's been 
a lot of (relatively) recent work in fixing segfault items when using 
slapo-rwm, so it would be helpful to know what release you hit this on. 


I found a quick way to reproduce this on 2.5.12 (a small edit to 
tests/scripts/relay and tests/data/slapd-relay.conf) and reported this 
as ITS#9871.


Regards,

    -Kartik


Re: rwm bindDN context and ppolicy issues

2022-06-23 Thread Kartik Subbarao

On 6/23/22 4:29 PM, Quanah Gibson-Mount wrote:

[...]
I'm mulling over how much additional time to spend on this. rwm is a very
elegant solution to a current issue that could save me a bunch of 
time to

set up additional LDAP servers with the renamed data. If this is an
isolated bug for which a quick fix might be possible, I might 
investigate

further.

But if it's a thorny issue or just the tip of the iceberg of things 
where

rwm might break unexpectedly, then it may be better for me to consider
other options. OpenLDAP developers, what do your instincts say on this?


If slapd segfaults, an ITS with clear reproduction steps should 
generally be filed. I would note that you have not specified the 
OpenLDAP release on which you encountered this problem.  There's been 
a lot of (relatively) recent work in fixing segfault items when using 
slapo-rwm, so it would be helpful to know what release you hit this on.


Sorry -- I'm running 2.4.57+dfsg-2ubuntu1.

Regards,

    -Kartik


rwm bindDN context and ppolicy issues

2022-06-23 Thread Kartik Subbarao
I'm able to specify rwm bindDN rules without password-policy enabled 
just fine, like this one:


rwm-rewriteContext bindDN
rwm-rewriteRule 
"^([^=]+)=([^@]+)@olddomain.com(.+),dc=olddomain,dc=com$" 
"$1=$2...@newdomain.com$3,dc=newdomain,dc=com" ":@"


However, when I enable password policy (which also works fine on its 
own), slapd segfaults. From doing a backtrace and stepping through the 
code, it looks like the crux of the issue is that the mdb_info struct 
ends up with garbage data:


struct mdb_info *mdb = (struct mdb_info *) op->o_bd->be_private;

mi_dbenv_home and mi_monitor have random stuff in them.

I'm mulling over how much additional time to spend on this. rwm is a 
very elegant solution to a current issue that could save me a bunch of 
time to set up additional LDAP servers with the renamed data. If this is 
an isolated bug for which a quick fix might be possible, I might 
investigate further.


But if it's a thorny issue or just the tip of the iceberg of things 
where rwm might break unexpectedly, then it may be better for me to 
consider other options. OpenLDAP developers, what do your instincts say 
on this?


Regards,

    -Kartik


Re: rewriting non-DN attribute values with rwm?

2022-05-29 Thread Kartik Subbarao

On 5/29/22 7:04 PM, Howard Chu wrote:

Kartik Subbarao wrote:

[...]
But I can't figure out how to rewrite the *values* of the uid and mail 
attributes in the returned entry to u...@newdomain.com. What is the best way to 
achieve this?

Nothing in OpenLDAP rewrites non-DN attributes.

You could try using slapo-sock and put together an external process to do it.


Ah, bummer. Thanks in any case for the quick response Howard.

Regards,

    -Kartik


rewriting non-DN attribute values with rwm?

2022-05-29 Thread Kartik Subbarao

I'd like to rewrite the following entry:

dn: uid=u...@olddomain.com,dc=olddomain,dc=com
uid: u...@olddomain.com
mail: u...@olddomain.com

to appear and behave like this:

dn: uid=u...@newdomain.com,dc=newdomain,dc=com
uid: u...@newdomain.com
mail: u...@newdomain.com

I can get the DN rewritten with slapd-relay and rwm-suffixmassage, and 
can use rwm-rewriterule with the searchFilter and searchEntryDN contexts 
to return the entry when querying for mail=u...@newdomain.com. But I 
can't figure out how to rewrite the *values* of the uid and mail 
attributes in the returned entry to u...@newdomain.com. What is the best 
way to achieve this?


Thanks,

    -Kartik


issues configuring SASL EXTERNAL and proxyauth with chain used for ppolicy_forward_updates

2022-04-14 Thread Kartik Subbarao
I have a consumer server (2.4.57) that successfully forwards 
pwdFailureTime modifications to the master server using GSSAPI 
authentication. I want to replace GSSAPI with certificate-based (SASL 
EXTERNAL) authentication along with proxy authorization. Basically, I 
want to configure the equivalent of the following command line:


LDAPTLS_KEY=server.key LDAPTLS_CERT=server.crt \
ldapmodify -Z -Y EXTERNAL \
-e '!authzid=dn:cn=proxydn,dc=example,dc=com' -e relax \
-h ldap-master.example.com -f update_pwdfailuretime.ldif

The command line works as expected -- it authenticates successfully 
using the server certificate, and then does PROXYAUTHZ to 
cn=proxydn,dc=example,dc=com to perform the modify operation. The issue 
is when I try to configure this behavior with chain on the consumer 
server. I've tried various incantations along these lines:


chain-idassert-bind bindmethod=SASL saslmech=EXTERNAL 
tls_cert=server.crt tls_key=tls.key authzId=dn:cn=proxydn,dc=example,dc=com


The SASL EXTERNAL authentication works fine -- It binds to the master 
with the DN mapped from the certificate's subject. But it doesn't do the 
proxyauthz to cn=proxydn,dc=example,dc=com. I've read through the docs 
in detail and tried various modes, flags and other settings, but I can't 
get it to do the proxy authz.


Does anyone have a known working config for this kind of setup that they 
can share? Otherwise, any suggestions on the best way to troubleshoot 
this further would be great.


Thanks,

    -Kartik


Re: Incomplete members from dynlist group when Paged Results control is specified

2016-02-15 Thread Kartik Subbarao

On 02/12/2016 04:41 PM, Kartik Subbarao wrote:

[...]

I wanted to ask whether this is enough detail to file an ITS, or 
whether more information was desired. It could take some more time to 
put together a generically reproducible LDIF file, so I figured I 
would post this now in case someone familiar with the dynlist code 
might recognize this and see how the presence of the Paged Results 
control might be interfering with its behavior.


I was able to reproduce this with a simple LDIF file, will submit an ITS.

-Kartik



Incomplete members from dynlist group when Paged Results control is specified

2016-02-13 Thread Kartik Subbarao
I'm using Google Apps Directory Sync (GADS) to synchronize group 
memberships from an OpenLDAP (2.4.41) directory. GADS uses the Paged 
Results control (1.2.840.113556.1.4.319) to retrieve results in batches 
of 1000. One of the searches that it does is as follows:


base: ou=groups,dc=example,dc=com
filter: (mail=*@example.com)
attributes requested: member owner [and some others]

There are more than 1000 groups that match, so it returns the results in 
batches as expected. Some of these groups are dynlist groups that have 
memberURL attributes set, which dynamically expand the member attribute 
when retrieved. For dynlist groups that show up in the first page of 
results, their members are properly retrieved. But after the first page, 
the results for dynlist groups only return a subset of the proper 
membership. As a result, an incomplete membership is being synced to 
Google for those subsequent groups, and the missing members are causing 
problems.


To rule out GADS from being the root cause of the problem, I ran a 
standalone perl script to search with the Paged Results control, with 
the page size set to a small number of entries. Just as with GADS, after 
the first page of results, the entries started to return fewer members 
from dynlist groups than should be returned. With a page size of 1, the 
second dynlist group started to show fewer members (125 vs 127).


My guess is that the dynlist code is somehow getting confused by the 
presence of the Paged Results control in the top-level search, when it 
does its internal searches to retrieve the individual group memberships.


I wanted to ask whether this is enough detail to file an ITS, or whether 
more information was desired. It could take some more time to put 
together a generically reproducible LDIF file, so I figured I would post 
this now in case someone familiar with the dynlist code might recognize 
this and see how the presence of the Paged Results control might be 
interfering with its behavior.


Thanks,

-Kartik



Re: slapadd 4096-character LDIF line length limitation

2015-06-02 Thread Kartik Subbarao

On 06/01/2015 01:55 PM, Quanah Gibson-Mount wrote:

--On Sunday, May 31, 2015 9:37 PM -0400 Kartik Subbarao
subba...@computer.org wrote:


I wanted to ask if someone could shed some light on the 4096
character LDIF line length limitation, which seems to have been
introduced sometime after 2.4.25. [...]

[...]

Hi Kartik,

Can you reproduce with the current RE24 code?


Hi Quanah -- I just backported the latest Ubuntu 2.4.40 package to the 
system that I'm working on and the problem is no longer there! Glad that 
was an easy one :-)


Thanks,

-Kartik



slapadd 4096-character LDIF line length limitation

2015-06-01 Thread Kartik Subbarao
I wanted to ask if someone could shed some light on the 4096 character 
LDIF line length limitation, which seems to have been introduced 
sometime after 2.4.25. I learned about this the hard way, while trying 
to slapadd an LDIF file with long jpegPhoto attributes (e.g. 50K+) which 
loaded just fine on 2.4.25. Now, I get this error:


ldif_parse_line: jpegPhoto: invalid base64 encoding char (556b9f15 = 
str2entry: str2ad() slapadd: could not parse entry (line=14)


When I need to preprocess LDIF files before loading them with slapadd, I 
often remove line continuations in order to simplify pattern matching. 
So of course from my perspective, it would be great to have the previous 
behavior back again. But if there are compelling reasons for the current 
behavior, I'd like to suggest that the code print a better error message 
that explicitly mentions the line length limit.


Regards,

-Kartik



More efficient way to represent local+remote views?

2012-07-17 Thread Kartik Subbarao
I'm currently using the following configuration for an 
application-specific LDAP directory to present a unified view of its 
local data (ou=localapp) and remote data (ou=people), all under 
dc=example,dc=com:


database hdb
suffix ou=localapp,dc=example,dc=com
# ...

database meta
suffix dc=example,dc=com
uri ldapi:///ou=localapp,dc=example,dc=com
uri ldap://remoteldap.example.com/ou=people,dc=example,dc=com

The idea being that clients of this directory can simply set the base DN 
to dc=example,dc=com without needing to know which parts are local and 
which parts are pulled in remotely.


My understanding is that this approach incurs some overhead in going 
through the ldapi interface for each operation that ends up being 
performed on the local backend. Is it possible to eliminate that 
overhead through an alternate approach? I looked at back_relay, but 
couldn't get it to do what I wanted. I don't want to rewrite any of the 
suffixes -- I just want it to do exactly as above in the back-meta 
configuration, but replace ldapi:/// with internal-backend-api:///, 
as mentioned in this part of the back-relay docs: back-relay bypasses 
the real database frontend operations by short-circuiting operations 
through the internal backend API.


Thanks,

-Kartik



Re: More efficient way to represent local+remote views?

2012-07-17 Thread Kartik Subbarao

On 07/17/2012 11:13 AM, Aaron Richton wrote:

If everything will be strictly confined to the two ou you stated, try
http://www.openldap.org/lists/openldap-technical/201206/msg00166.html
... just change the first meta to hdb and tweak the suffix
directives (e.g. back-null would be dc=example,dc=com).


Thanks for the quick response. I had had some nebulous experiences with 
the glue functionality in the past (and didn't understand it well), but 
looking into it now, I see that the subordinate/glue functionality is 
well documented. Thanks for the pointer!


-Kartik



Re: Collabograte project

2012-03-03 Thread Kartik Subbarao

On 03/03/2012 08:11 AM, Daniel Pocock wrote:


 On 01/03/12 22:17, Kartik Subbarao wrote:
 I'm integrating OpenLDAP as an LDAP Directory component in the open
 source Collabograte project that I recently announced:

 http://kartiksubbarao.com/announcing-collabograte

 I think this is interesting and very worthwhile.  There are similar
 projects like FusionForge, some of them use OpenLDAP.  Can you
 elaborate on what makes Collabograte distinctive from those other
 alternatives?

The primary audience for Collabograte is IT integrators who want to 
assemble their own custom services from individual collaboration 
components that they choose. Collabograte helps them with the heavy 
lifting of integration -- connecting the dots between these different 
components. For example, in the role of LDAP Directory Component, 
Collabograte can provide configurations for a number of LDAP servers. 
OpenLDAP is the first one that it supports, but there is no restriction 
on the products that can be supported. The same is true for any other 
category of component. So down the line, Collabograte might support 5 
different OSes, 4 different LDAP servers, 3 different Wikis, etc.


Over time, someone could build a FusionForge out of Collabograte and 
make it generally available on the Internet (it's all open source, 
people can do whatever they want), but that's not the focal point of 
Collabograte itself. With Collabograte, I'm more interested in building 
a community of IT integrators than in providing services directly to end 
users. To put it in LDAP terms, Collabograte is more of an auxiliary 
class than a structural class :-)


 I'd be very interested in hearing any comments, ideas, suggestions, or
 questions you might have on this. I want to help Enterprise IT do a
 better job of integrating open source into their environments, as well
 as improve the collaboration between Enterprise IT and open source
 project communities.

 Other popular collaboration apps that come to mind are MoinMoin,
 Bugzilla, mailman

Thanks for the application suggestions.

 Do you plan to offer choice of app, e.g. someone can choose between
 Bugzilla, Mantis or RT for bug/service request handling?

 Or you anticipate a very tight integration with the apps you prefer?

The former, most definitely. Collabograte is all about freedom and 
choice, maximizing the options that integrators have to pick and choose 
the components they want. Basically, if someone is willing to contribute 
integration configuration for a new component, and the code meets a 
basic standard of quality, I'll lean towards accepting it. This may mean 
that not all combinations of all components will work with each other, 
but the ones people care about the most will. And I will be looking to 
build on existing work to make it more modular, get stuff pushed 
upstream where appropriate/possible, etc.


-Kartik



Re: Collabograte project

2012-03-03 Thread Kartik Subbarao

On 03/03/2012 05:40 PM, Daniel Pocock wrote:

It is interesting that you are posting about this on the OpenLDAP list,
I also came here just recently with similar aims in mind:

[...]

Daniel, can we take this discussion over to the Collabograte mailing 
list? The topic is drifting from OpenLDAP itself and I'm eager to get 
some discussions going on the Collabograte list:


http://groups.google.com/group/collabograte

Thanks,

-Kartik



Collabograte project

2012-03-01 Thread Kartik Subbarao
I'm integrating OpenLDAP as an LDAP Directory component in the open 
source Collabograte project that I recently announced:


http://kartiksubbarao.com/announcing-collabograte

Here are some areas of integration configured with OpenLDAP:
+ Username and password authentication to MediaWiki, WordPress 
Multisite, Cyrus IMAP, Sympa, and ejabberd

+ LDAP Groups used as Sympa Mailing Lists and ejabberd groups

If you want to see the implementation details, you can look at the 
puppet manifest and other files:


https://github.com/kartiksubbarao/collabograte/blob/master/puppet/modules/openldap/manifests/init.pp

https://github.com/kartiksubbarao/collabograte/tree/master/puppet/modules/openldap/templates

https://github.com/kartiksubbarao/collabograte/tree/master/puppet/modules/openldap/files

I'd be very interested in hearing any comments, ideas, suggestions, or 
questions you might have on this. I want to help Enterprise IT do a 
better job of integrating open source into their environments, as well 
as improve the collaboration between Enterprise IT and open source 
project communities.


Please feel free to respond here, on the Collabograte mailing list, or 
email me directly, whichever you prefer.


Thanks,

-Kartik



Re: Strange hang scenario, resumes after idletimeout, but plenty of FDs available

2011-06-03 Thread Kartik Subbarao

On 06/02/2011 01:36 PM, Howard Chu wrote:

Attached is the gdb stack trace from the hang state. It looks like the
the threads are stuck in pthread_cond_wait() from send_ldap_ber(). Are
there other relevant variables/structures to inspect for this scenario?


Also get a netstat -nA inet. Threads waiting in send_ldap_ber() means
their output buffers got full, clients didn't read the pending data.


(Un)fortunately that hang state hasn't occurred since yesterday (even 
with idletimeout set to 20 min).


I did run that netstat command periodically in any case, just to observe 
what happens during times of high traffic. The one thing I do notice 
sporadically is a bunch of connections in FIN_WAIT_1. But they do seem 
to clear up on their own within a minute or so. So it doesn't look like 
there's a long-term problem with accumulating TCP connections.


I'll try to capture the netstat during the hang time if/when it happens 
again.


-Kartik



Re: Strange hang scenario, resumes after idletimeout, but plenty of FDs available

2011-06-02 Thread Kartik Subbarao

On 06/01/2011 02:02 PM, Howard Chu wrote:

Kartik Subbarao wrote:

On 06/01/2011 09:08 AM, Kartik Subbarao wrote:

Update: It's not a locks issue.


Another update -- after I reduced the idletimeout to 60 seconds, the
problem seems to have gone away. It would still be useful to know what
might be causing this problem and to be able to support higher levels of
idletimeout, but at least I have another workable option now.


As usual, when investigating a hang, you should attach to slapd with gdb
and get a snapshot of what all the threads are doing. Working around it
before you know what caused it isn't all that useful in the long run.


Attached is the gdb stack trace from the hang state. It looks like the 
the threads are stuck in pthread_cond_wait() from send_ldap_ber(). Are 
there other relevant variables/structures to inspect for this scenario?


-Kartik
GNU gdb 6.8-debian
Copyright (C) 2008 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later http://gnu.org/licenses/gpl.html
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type show copying
and show warranty for details.
This GDB was configured as x86_64-linux-gnu...
(gdb) attach 28127
Attaching to program: /usr/sbin/slapd, process 28127
Reading symbols from /usr/lib/libldap_r-2.4.so.2...done.
Loaded symbols for /usr/lib/libldap_r-2.4.so.2
Reading symbols from /usr/lib/liblber-2.4.so.2...done.
Loaded symbols for /usr/lib/liblber-2.4.so.2
Reading symbols from /lib/libuuid.so.1...done.
Loaded symbols for /lib/libuuid.so.1
Reading symbols from /usr/lib/libdb-4.8.so...done.
Loaded symbols for /usr/lib/libdb-4.8.so
Reading symbols from /usr/lib/libodbc.so.1...done.
Loaded symbols for /usr/lib/libodbc.so.1
Reading symbols from /usr/lib/libslp.so.1...done.
Loaded symbols for /usr/lib/libslp.so.1
Reading symbols from /usr/lib/libsasl2.so.2...done.
Loaded symbols for /usr/lib/libsasl2.so.2
Reading symbols from /usr/lib/libssl.so.0.9.8...done.
Loaded symbols for /usr/lib/libssl.so.0.9.8
Reading symbols from /usr/lib/libcrypto.so.0.9.8...done.
Loaded symbols for /usr/lib/libcrypto.so.0.9.8
Reading symbols from /lib/libcrypt.so.1...done.
Loaded symbols for /lib/libcrypt.so.1
Reading symbols from /lib/libresolv.so.2...done.
Loaded symbols for /lib/libresolv.so.2
Reading symbols from /usr/lib/libltdl.so.3...done.
Loaded symbols for /usr/lib/libltdl.so.3
Reading symbols from /lib/libwrap.so.0...done.
Loaded symbols for /lib/libwrap.so.0
Reading symbols from /lib/libpthread.so.0...done.
[Thread debugging using libthread_db enabled]
[New Thread 0x7ff0899cb730 (LWP 28127)]
[New Thread 0x492ab950 (LWP 28155)]
[New Thread 0x48aaa950 (LWP 28154)]
[New Thread 0x482a9950 (LWP 28153)]
[New Thread 0x47aa8950 (LWP 28152)]
[New Thread 0x472a7950 (LWP 28151)]
[New Thread 0x46aa6950 (LWP 28150)]
[New Thread 0x462a5950 (LWP 28149)]
[New Thread 0x45aa4950 (LWP 28148)]
[New Thread 0x452a3950 (LWP 28147)]
[New Thread 0x44aa2950 (LWP 28146)]
[New Thread 0x442a1950 (LWP 28145)]
[New Thread 0x43aa0950 (LWP 28139)]
[New Thread 0x4329f950 (LWP 28138)]
[New Thread 0x42a9e950 (LWP 28137)]
[New Thread 0x40aea950 (LWP 28135)]
[New Thread 0x4154d950 (LWP 28134)]
[New Thread 0x4229d950 (LWP 28133)]
Loaded symbols for /lib/libpthread.so.0
Reading symbols from /lib/libc.so.6...done.
Loaded symbols for /lib/libc.so.6
Reading symbols from /lib/ld-linux-x86-64.so.2...done.
Loaded symbols for /lib64/ld-linux-x86-64.so.2
Reading symbols from /lib/libnsl.so.1...done.
Loaded symbols for /lib/libnsl.so.1
Reading symbols from /lib/libdl.so.2...done.
Loaded symbols for /lib/libdl.so.2
Reading symbols from /usr/lib/libz.so.1...done.
Loaded symbols for /usr/lib/libz.so.1
Reading symbols from /lib/libnss_files.so.2...done.
Loaded symbols for /lib/libnss_files.so.2
Reading symbols from /lib/libnss_ldap.so.2...done.
Loaded symbols for /lib/libnss_ldap.so.2
Reading symbols from /usr/lib/sasl2/libanonymous.so.2...done.
Loaded symbols for /usr/lib/sasl2/libanonymous.so.2
Reading symbols from /usr/lib/sasl2/libplain.so.2...done.
Loaded symbols for /usr/lib/sasl2/libplain.so.2
Reading symbols from /usr/lib/sasl2/libntlm.so.2...done.
Loaded symbols for /usr/lib/sasl2/libntlm.so.2
Reading symbols from /usr/lib/sasl2/libgssapiv2.so.2...done.
Loaded symbols for /usr/lib/sasl2/libgssapiv2.so.2
Reading symbols from /usr/lib/libgssapi.so.2...done.
Loaded symbols for /usr/lib/libgssapi.so.2
Reading symbols from /usr/lib/libkrb5.so.25...done.
Loaded symbols for /usr/lib/libkrb5.so.25
Reading symbols from /usr/lib/libasn1.so.8...done.
Loaded symbols for /usr/lib/libasn1.so.8
Reading symbols from /usr/lib/libroken.so.18...done.
Loaded symbols for /usr/lib/libroken.so.18
Reading symbols from /lib/libcom_err.so.2...done.
Loaded symbols for /lib/libcom_err.so.2
Reading symbols from /usr/lib/libheimntlm.so.0...done.
Loaded symbols for /usr/lib/libheimntlm.so.0
Reading symbols from /usr/lib/libhx509.so.3...done.
Loaded symbols

Re: Strange hang scenario, resumes after idletimeout, but plenty of FDs available

2011-06-01 Thread Kartik Subbarao

On 06/01/2011 08:43 AM, Kartik Subbarao wrote:

Are there any resources other than file descriptors that are freed up
during the idletimeout processing? Are there any other parameters that
can be tuned besides idletimeout here? Could it possibly be a case of
deadlock somewhere, something grabbing all the locks? Would things like
set_lk_max_locks be relevant to investigate here? Any log level settings
that might reveal more of what's happening here?


Update: It's not a locks issue. I ran db4.8_stat -C A while one of the 
servers was in this hang state and there are plenty of locks available. 
Also another clarification -- all of the operations during the 
bombardment times in question are read operations from clients, no writes.


-Kartik



Re: Strange hang scenario, resumes after idletimeout, but plenty of FDs available

2011-06-01 Thread Kartik Subbarao

On 06/01/2011 09:08 AM, Kartik Subbarao wrote:

Update: It's not a locks issue.


Another update -- after I reduced the idletimeout to 60 seconds, the 
problem seems to have gone away. It would still be useful to know what 
might be causing this problem and to be able to support higher levels of 
idletimeout, but at least I have another workable option now.


-Kartik



Re: Strange hang scenario, resumes after idletimeout, but plenty of FDs available

2011-06-01 Thread Kartik Subbarao

On 06/01/2011 02:02 PM, Howard Chu wrote:

Kartik Subbarao wrote:

On 06/01/2011 09:08 AM, Kartik Subbarao wrote:

Update: It's not a locks issue.


Another update -- after I reduced the idletimeout to 60 seconds, the
problem seems to have gone away. It would still be useful to know what
might be causing this problem and to be able to support higher levels of
idletimeout, but at least I have another workable option now.


As usual, when investigating a hang, you should attach to slapd with gdb
and get a snapshot of what all the threads are doing. Working around it
before you know what caused it isn't all that useful in the long run.


Unfortunately this instance of slapd doesn't have debugging symbols 
enabled (it's a stock Debian 2.4.25 slapd package), so I didn't think 
gdb would be that helpful...But now that you mention it, I could 
probably get some basic stack trace info that might yield some clues.


Thanks for the reminder :-)

-Kartik



Re: Strange hang scenario, resumes after idletimeout, but plenty of FDs available

2011-06-01 Thread Kartik Subbarao

On 06/01/2011 02:49 PM, Quanah Gibson-Mount wrote:

Unfortunately this instance of slapd doesn't have debugging symbols
enabled (it's a stock Debian 2.4.25 slapd package), so I didn't think gdb
would be that helpful...But now that you mention it, I could probably get
some basic stack trace info that might yield some clues.

Thanks for the reminder :-)


Debian ships the debugging symbols as a separate package. All you need
to do is install that package, and you can get a full backtrace.

p slapd-dbg - Debugging information for the OpenLDAP server (slapd)


Even better! Thanks for the info.

-Kartik



Problems with ppolicy_forward_updates and starttls with certificate-based auth

2010-11-19 Thread Kartik Subbarao
I'm trying to get a consumer server to forward ppolicy-related updates 
to its provider server, and to use certificate-based authentication 
(SASL EXTERNAL) over STARTTLS when authenticating to the provider. This 
is with 2.4.23 on a Debian 5.0.5 system (I've seen similar issues 
reported elsewhere so I doubt this is platform specific).


I'm running into multiple problems here. The core problem seems to be 
that enabling ppolicy_forward_updates breaks the chaining overlay such 
that it binds anonymously instead of with SASL EXTERNAL. Another problem 
is that bind operations to the consumer server start to return two 
result messages -- one with the error code of the chained operation, and 
one with the error code of the bind operation. This latter problem seems 
to the cause of the (still unresolved?) errors from this message thread 
earlier this year:


http://www.mail-archive.com/openldap-technical@openldap.org/msg01215.html

To simplify reproducing the problem, I've worked with test022-ppolicy in 
the openldap test framework. I've submitted ITS 6711 based on this. 
Here, I ran into another issue. I can't seem to be able to configure 
sasl external/starttls chaining properly with the cn=config style 
configuration that test022-ppolicy applies. The self-signed cert that 
I'm using works fine with replication, but it doesn't seem to work with 
chaining. This may or may not be another issue that needs to be resolved.


In any case, with the attached files in the ITS, I hope that what I'm 
trying to do and the results that I'm getting should be as clear and 
unambiguous as possible. I'd appreciate any feedback on whether there is 
something else I need to configure or if there are bugs here that need 
to be fixed.


Thanks,

-Kartik