Re: Performance issues and strange logs
Assuming that the errors and warnings are not a big deal, can you suggest any reasons that we would have connections stacking up and timing out? The use case is likely heavier on write than would be standard for LDAP, but it seems the failure threshold (number of connections) is very low. This is hosted on a multi-core machine (virtual machine) and when it gets bad, our operations people say that a single core of the machine is pegged at 100% CPU while others are essentially idle. Are there parts of Apache DS that have thread affinity and would be stuck to a single processor? Any thoughts would be appreciated, Craig On Wednesday, October 12, 2011, Emmanuel Lecharny wrote: On 10/12/11 10:24 PM, Craig Setera wrote: Hello, Hi, which ADS version are you using ? Is it still 1.5.5 ? Last week I was asking about indexing and performance gains from those indexes. The question stemmed from some performance problems that we are currently having in our environment. In that environment, we are seeing extremely poor performance and Apache DS getting bogged down after a relatively small number of connections (under 100). When that happens, the connections start to stack up and eventually time out. I was surprised by the performance, but was looking at things like indexing and disabling sync on write as ways to improve the situation. Now, I've received some logs from testing and I'm beginning to wonder if there is something else going on. I'm seeing many entries in the logs that look like: [18:27:52] WARN [org.apache.directory.server.**ldap.LdapSession] - AbandonableRequest with messageId 2 not found in outstandingRequests. [18:27:53] WARN [org.apache.directory.shared.**asn1.ber.Asn1Decoder] - The PDU has been fully decoded but there are still bytes in the buffer. [18:27:53] WARN [org.apache.directory.shared.**asn1.ber.Asn1Decoder] - The PDU has been fully decoded but there are still bytes in the buffer. [18:27:53] WARN [org.apache.directory.server.**ldap.LdapSession] - AbandonableRequest with messageId 2 not found in outstandingRequests. [18:27:53] WARN [org.apache.directory.shared.**asn1.ber.Asn1Decoder] - The PDU has been fully decoded but there are still bytes in the buffer. [18:27:53] WARN [org.apache.directory.shared.**asn1.ber.Asn1Decoder] - The PDU has been fully decoded but there are still bytes in the buffer. This message just means that some data has been received, decoded correctly, but we have still some remaining bytes in the buffer. It's a warning, not an error. The remaining bytes will be decoded later. Nothing to be scared about. -- Regards, Cordialement, Emmanuel Lécharny www.iktek.com -- Craig Setera Director, Product Engineering mFoundry p 415.324.5801 cr...@mfoundry.com
Re: Performance issues and strange logs
On 10/13/11 5:31 PM, Craig Setera wrote: Assuming that the errors and warnings are not a big deal, can you suggest any reasons that we would have connections stacking up and timing out? The use case is likely heavier on write than would be standard for LDAP, but it seems the failure threshold (number of connections) is very low. This is hosted on a multi-core machine (virtual machine) and when it gets bad, our operations people say that a single core of the machine is pegged at 100% CPU while others are essentially idle. Are there parts of Apache DS that have thread affinity and would be stuck to a single processor? Ok, 1.5.5 is pretty old, and depends on MINA-2.0.0-M6. This version of the NIO framework sometime gets stuck with 100% CPU (it's a JVM bug) when some session are closed just after having been opened (basically, the selector loop get crazy and eats 100% CPU looping). It might be the reason why you see this 100% CPU for some sessions (as we have one selector per CPU, the other sessions might be ok). I would suggest you switch to at least 1.5.7, but even better to 2.0.0-M3. -- Regards, Cordialement, Emmanuel Lécharny www.iktek.com
Re: Performance issues and strange logs
Is it possible to switch out just MINA or would that break something? On Thursday, October 13, 2011, Emmanuel Lecharny wrote: On 10/13/11 5:31 PM, Craig Setera wrote: Assuming that the errors and warnings are not a big deal, can you suggest any reasons that we would have connections stacking up and timing out? The use case is likely heavier on write than would be standard for LDAP, but it seems the failure threshold (number of connections) is very low. This is hosted on a multi-core machine (virtual machine) and when it gets bad, our operations people say that a single core of the machine is pegged at 100% CPU while others are essentially idle. Are there parts of Apache DS that have thread affinity and would be stuck to a single processor? Ok, 1.5.5 is pretty old, and depends on MINA-2.0.0-M6. This version of the NIO framework sometime gets stuck with 100% CPU (it's a JVM bug) when some session are closed just after having been opened (basically, the selector loop get crazy and eats 100% CPU looping). It might be the reason why you see this 100% CPU for some sessions (as we have one selector per CPU, the other sessions might be ok). I would suggest you switch to at least 1.5.7, but even better to 2.0.0-M3. -- Regards, Cordialement, Emmanuel Lécharny www.iktek.com -- Craig Setera Director, Product Engineering mFoundry p 415.324.5801 cr...@mfoundry.com
Re: Performance issues and strange logs
On 10/13/11 5:53 PM, Craig Setera wrote: Is it possible to switch out just MINA or would that break something? It *should* be possible, as the MINA API has been frozen in 2.0.0-RC1. -- Regards, Cordialement, Emmanuel Lécharny www.iktek.com
Re: Performance issues and strange logs
We have a custom interceptor plugged in to 1.5.5. Did anything change in 1.5.7 that would break that? If not, we may be best off to just move to 1.5.7. Thanks again, craig On Thursday, October 13, 2011, Emmanuel Lécharny wrote: On 10/13/11 5:53 PM, Craig Setera wrote: Is it possible to switch out just MINA or would that break something? It *should* be possible, as the MINA API has been frozen in 2.0.0-RC1. -- Regards, Cordialement, Emmanuel Lécharny www.iktek.com -- Craig Setera Director, Product Engineering mFoundry p 415.324.5801 cr...@mfoundry.com
Re: Performance issues and strange logs
On Thu, Oct 13, 2011 at 12:52 PM, Craig Setera cr...@mfoundry.com wrote: We have a custom interceptor plugged in to 1.5.5. Did anything change in 1.5.7 that would break that? If not, we may be best off to just move to 1.5.7. hard to tell without knowing(at least in a very high level view) what this interceptor does Thanks again, craig On Thursday, October 13, 2011, Emmanuel Lécharny wrote: On 10/13/11 5:53 PM, Craig Setera wrote: Is it possible to switch out just MINA or would that break something? It *should* be possible, as the MINA API has been frozen in 2.0.0-RC1. -- Regards, Cordialement, Emmanuel Lécharny www.iktek.com -- Craig Setera Director, Product Engineering mFoundry p 415.324.5801 cr...@mfoundry.com -- Kiran Ayyagari
Re: Performance issues and strange logs
High-level... does alteration of the values of some of our schema attributes. So, it sits in the flow and alters the data for those particular attributes as they flow through. On Thursday, October 13, 2011, Kiran Ayyagari wrote: On Thu, Oct 13, 2011 at 12:52 PM, Craig Setera cr...@mfoundry.comjavascript:; wrote: We have a custom interceptor plugged in to 1.5.5. Did anything change in 1.5.7 that would break that? If not, we may be best off to just move to 1.5.7. hard to tell without knowing(at least in a very high level view) what this interceptor does Thanks again, craig On Thursday, October 13, 2011, Emmanuel Lécharny wrote: On 10/13/11 5:53 PM, Craig Setera wrote: Is it possible to switch out just MINA or would that break something? It *should* be possible, as the MINA API has been frozen in 2.0.0-RC1. -- Regards, Cordialement, Emmanuel Lécharny www.iktek.com -- Craig Setera Director, Product Engineering mFoundry p 415.324.5801 cr...@mfoundry.com javascript:; -- Kiran Ayyagari -- Craig Setera Director, Product Engineering mFoundry p 415.324.5801 cr...@mfoundry.com
Re: Performance issues and strange logs
On Thu, Oct 13, 2011 at 2:10 PM, Craig Setera cr...@mfoundry.com wrote: High-level... does alteration of the values of some of our schema attributes. So, it sits in the flow and alters the data for those particular attributes as they flow through. ok, should work in the latest code without any issues (might require fixing some imports) On Thursday, October 13, 2011, Kiran Ayyagari wrote: On Thu, Oct 13, 2011 at 12:52 PM, Craig Setera cr...@mfoundry.comjavascript:; wrote: We have a custom interceptor plugged in to 1.5.5. Did anything change in 1.5.7 that would break that? If not, we may be best off to just move to 1.5.7. hard to tell without knowing(at least in a very high level view) what this interceptor does Thanks again, craig On Thursday, October 13, 2011, Emmanuel Lécharny wrote: On 10/13/11 5:53 PM, Craig Setera wrote: Is it possible to switch out just MINA or would that break something? It *should* be possible, as the MINA API has been frozen in 2.0.0-RC1. -- Regards, Cordialement, Emmanuel Lécharny www.iktek.com -- Craig Setera Director, Product Engineering mFoundry p 415.324.5801 cr...@mfoundry.com javascript:; -- Kiran Ayyagari -- Craig Setera Director, Product Engineering mFoundry p 415.324.5801 cr...@mfoundry.com -- Kiran Ayyagari
Re: Performance issues and strange logs
Oh... Hopefully it works without recompile or this gets trickier. I guess we will find out soon enough. Thanks, Craig On Thursday, October 13, 2011, Kiran Ayyagari wrote: On Thu, Oct 13, 2011 at 2:10 PM, Craig Setera cr...@mfoundry.comjavascript:; wrote: High-level... does alteration of the values of some of our schema attributes. So, it sits in the flow and alters the data for those particular attributes as they flow through. ok, should work in the latest code without any issues (might require fixing some imports) -- Craig Setera Director, Product Engineering mFoundry p 415.324.5801 cr...@mfoundry.com
Re: Replication configuration
I built 1180264 (after much difficulty, getting deb packages to build, so I could deploy it to my admin nodes), but can't get it to start because things are missing, and unfortunately the documentation for 2.0 is, well, nonexistent. Right now it's vomiting on the lack of any schema ldif/structure being present. I am trying to not build locally on the admin nodes, they need to remain clean and reproducible, deployable binary packages are required (I need to build, at last count, approx 13 replicas world wide). I will svn up and rebuild, but I suspect that will not fix my schema problems. Is there any 2.0 documentation in any functional state? -j On Oct 11, 2011, at 1:24 PM, Kiran Ayyagari wrote: the trunk is back to stable condition, you can svn up your local copy or checkout the head directly On Mon, Oct 10, 2011 at 6:32 PM, Kiran Ayyagari kayyag...@apache.org wrote: http://svn.apache.org/repos/asf/directory/apacheds/trunk-with-dependencies please checkout the revision 1180264 cause currently there are few changes that destabilized the trunk (should be back to normal in couple days) Please let us know if you have any issues. On Mon, Oct 10, 2011 at 6:06 PM, Jonathan Disher jdis...@parad.net wrote: So, I checked out trunk and appear to have gotten ApacheDS 1.5. Where do I check out the 2.0 tree? -j On Oct 6, 2011, at 5:30 PM, Kiran Ayyagari wrote: This has been fixed, please see https://issues.apache.org/jira/browse/DIRSERVER-1666 for the details. Appreciate if you can verify the fix and report your findings. On Thu, Oct 6, 2011 at 3:33 PM, Kiran Ayyagari kayyag...@apache.org wrote: I confirm the presence of this issue, (which seems to be introduced during some code cleanup) will commit a fix in another 7 hours and let you know. Thanks for the detailed trace. On Thu, Oct 6, 2011 at 2:57 PM, Jonathan Disher jdis...@parad.net wrote: Yes, I did. Actually, I am getting this: [18:55:46] ERROR [org.apache.directory.server.ldap.replication.provider.SyncReplRequestHandler] - Failed to initialize the log files required by the syncrepl provider java.lang.NullPointerException at org.apache.directory.server.ldap.replication.provider.SyncReplSearchListener.init(SyncReplSearchListener.java:98) at org.apache.directory.server.ldap.replication.provider.SyncReplRequestHandler.registerPersistentSearches(SyncReplRequestHandler.java:929) at org.apache.directory.server.ldap.replication.provider.SyncReplRequestHandler.start(SyncReplRequestHandler.java:171) at org.apache.directory.server.ldap.LdapServer.startReplicationProducer(LdapServer.java:506) at org.apache.directory.server.ldap.LdapServer.start(LdapServer.java:488) at org.apache.directory.server.ApacheDsService.startLdap(ApacheDsService.java:374) at org.apache.directory.server.ApacheDsService.start(ApacheDsService.java:174) at org.apache.directory.server.ApacheDsTanukiWrapper.start(ApacheDsTanukiWrapper.java:71) at org.tanukisoftware.wrapper.WrapperManager$12.run(WrapperManager.java:2788) [18:55:46] ERROR [PROVIDER_LOG] - Failed to initialize the log files required by the syncrepl provider java.lang.NullPointerException at org.apache.directory.server.ldap.replication.provider.SyncReplSearchListener.init(SyncReplSearchListener.java:98) at org.apache.directory.server.ldap.replication.provider.SyncReplRequestHandler.registerPersistentSearches(SyncReplRequestHandler.java:929) at org.apache.directory.server.ldap.replication.provider.SyncReplRequestHandler.start(SyncReplRequestHandler.java:171) at org.apache.directory.server.ldap.LdapServer.startReplicationProducer(LdapServer.java:506) at org.apache.directory.server.ldap.LdapServer.start(LdapServer.java:488) at org.apache.directory.server.ApacheDsService.startLdap(ApacheDsService.java:374) at org.apache.directory.server.ApacheDsService.start(ApacheDsService.java:174) at org.apache.directory.server.ApacheDsTanukiWrapper.start(ApacheDsTanukiWrapper.java:71) at org.tanukisoftware.wrapper.WrapperManager$12.run(WrapperManager.java:2788) [18:55:46] ERROR [org.apache.directory.server.ApacheDsService] - Cannot start the server : java.lang.NullPointerException [18:56:42] WARN [org.apache.directory.server.ldap.handlers.SearchHandler] - This server does not allow replication Permissions should be fine (it runs as root, I need to listen on 389), so creating files shouldn't be a problem. -j On Oct 6, 2011, at 11:48 AM, Kiran Ayyagari wrote: did you add the attribute 'ads-replReqHandler' and restarted the server? On Thu, Oct 6, 2011 at 2:26 PM, Jonathan Disher jdis...@parad.net wrote: So, I did, and I followed the directions. I get this on my slave: [18:37:13] WARN [org.apache.directory.server.ldap.handlers.SearchHandler] - This server does not allow
Re: Replication configuration
On Thu, Oct 13, 2011 at 2:54 PM, Jonathan Disher jdis...@parad.net wrote: I built 1180264 (after much difficulty, getting deb packages to build, so I could deploy it to my admin nodes), but can't get it to start because things are missing, and unfortunately the documentation for 2.0 is, well, nonexistent. Right now it's vomiting on the lack of any schema ldif/structure being present. I am trying to not build locally on the admin nodes, they need to remain clean and reproducible, deployable binary packages are required (I need to build, at last count, approx 13 replicas world wide). I will svn up and rebuild, but I suspect that will not fix my schema problems. Is there any 2.0 documentation in any functional state? please give a try after doing svn up (I confirm that server starts up fine, tested with the archive installer) Note that you can safely ignore even if there are kerberos test failures which maven reports like ApacheDS Protocol Kerberos Test ... FAILURE let us know if you have any issues -j On Oct 11, 2011, at 1:24 PM, Kiran Ayyagari wrote: the trunk is back to stable condition, you can svn up your local copy or checkout the head directly On Mon, Oct 10, 2011 at 6:32 PM, Kiran Ayyagari kayyag...@apache.org wrote: http://svn.apache.org/repos/asf/directory/apacheds/trunk-with-dependencies please checkout the revision 1180264 cause currently there are few changes that destabilized the trunk (should be back to normal in couple days) Please let us know if you have any issues. On Mon, Oct 10, 2011 at 6:06 PM, Jonathan Disher jdis...@parad.net wrote: So, I checked out trunk and appear to have gotten ApacheDS 1.5. Where do I check out the 2.0 tree? -j On Oct 6, 2011, at 5:30 PM, Kiran Ayyagari wrote: This has been fixed, please see https://issues.apache.org/jira/browse/DIRSERVER-1666 for the details. Appreciate if you can verify the fix and report your findings. On Thu, Oct 6, 2011 at 3:33 PM, Kiran Ayyagari kayyag...@apache.org wrote: I confirm the presence of this issue, (which seems to be introduced during some code cleanup) will commit a fix in another 7 hours and let you know. Thanks for the detailed trace. On Thu, Oct 6, 2011 at 2:57 PM, Jonathan Disher jdis...@parad.net wrote: Yes, I did. Actually, I am getting this: [18:55:46] ERROR [org.apache.directory.server.ldap.replication.provider.SyncReplRequestHandler] - Failed to initialize the log files required by the syncrepl provider java.lang.NullPointerException at org.apache.directory.server.ldap.replication.provider.SyncReplSearchListener.init(SyncReplSearchListener.java:98) at org.apache.directory.server.ldap.replication.provider.SyncReplRequestHandler.registerPersistentSearches(SyncReplRequestHandler.java:929) at org.apache.directory.server.ldap.replication.provider.SyncReplRequestHandler.start(SyncReplRequestHandler.java:171) at org.apache.directory.server.ldap.LdapServer.startReplicationProducer(LdapServer.java:506) at org.apache.directory.server.ldap.LdapServer.start(LdapServer.java:488) at org.apache.directory.server.ApacheDsService.startLdap(ApacheDsService.java:374) at org.apache.directory.server.ApacheDsService.start(ApacheDsService.java:174) at org.apache.directory.server.ApacheDsTanukiWrapper.start(ApacheDsTanukiWrapper.java:71) at org.tanukisoftware.wrapper.WrapperManager$12.run(WrapperManager.java:2788) [18:55:46] ERROR [PROVIDER_LOG] - Failed to initialize the log files required by the syncrepl provider java.lang.NullPointerException at org.apache.directory.server.ldap.replication.provider.SyncReplSearchListener.init(SyncReplSearchListener.java:98) at org.apache.directory.server.ldap.replication.provider.SyncReplRequestHandler.registerPersistentSearches(SyncReplRequestHandler.java:929) at org.apache.directory.server.ldap.replication.provider.SyncReplRequestHandler.start(SyncReplRequestHandler.java:171) at org.apache.directory.server.ldap.LdapServer.startReplicationProducer(LdapServer.java:506) at org.apache.directory.server.ldap.LdapServer.start(LdapServer.java:488) at org.apache.directory.server.ApacheDsService.startLdap(ApacheDsService.java:374) at org.apache.directory.server.ApacheDsService.start(ApacheDsService.java:174) at org.apache.directory.server.ApacheDsTanukiWrapper.start(ApacheDsTanukiWrapper.java:71) at org.tanukisoftware.wrapper.WrapperManager$12.run(WrapperManager.java:2788) [18:55:46] ERROR [org.apache.directory.server.ApacheDsService] - Cannot start the server : java.lang.NullPointerException [18:56:42] WARN [org.apache.directory.server.ldap.handlers.SearchHandler] - This server does not allow replication Permissions should be fine (it runs as root, I need to listen on 389), so creating files shouldn't be a problem. -j On Oct 6,
Re: ApacheDS Password policy issues
On Tue, Oct 11, 2011 at 3:11 PM, carlo.acco...@ibs-ag.com wrote: Hi, I've been working with the password policy functionality this week and have encountered a few issues I'm hoping you can help clarify. These attributes are on the policy itself unless otherwise specified. 1. ads-pwdminlength (minimum # of chars require for a password) having a non-zero value accepts passwords that are any length. a. I didn't test ads-pwdmaxlength but might check that while you're there. 2. The value ads-pwmaxage is supposed to be how long a password is valid (in seconds). a. Setting this to a non-zero value causes a pwdChangedTime attribute to be set on the user when their password changes (ok) b. However it never enforces the expiry i. The ads-pwdgraceauthnlimit ( # of grace logins after expiration) doesn't seem to have any effect ii. Also setting ads-pwdexpirewarning above and below the max age doesn't seem to matter either c. If it did expire, how is this indicated on the user object ? have fixed this issue. Server indicates the user about expiry by sending the ppolicy response control after setting the value for timeBeforeExpiration property to the time left before the password expires. Note that this only happens if the user sent a request with ppolicy control (OID - 1.3.6.1.4.1.42.2.27.8.5.1) 3. When ads-pwdmaxfailure (number of times failed bind is permitted) is set to 5 , it allows 11 login failures before locking the account. a. Each login failure creates an additional pwdFailureTime attribute for the user (ok) b. pwdAccountLockedTime attribute is created after the 11th failed bind. (Also what we want, but after 5 failures) c. This might be some caching issue because I think once it took 13 failed attempts before it locked. this is a bit strange, do you have some custom caching mechanism in place? OR some custom authenticator implementation that doesn't inherit the AbstractAuthenticator? 4. When ads-pwdinhistory (# of old passwords kept so they're not reused) is set to 5 . a. Users initially have no pwdHistory attribute (ok) b. Each of the first 5 password changes happens successfully. Each time adding new pwdHistory attribute to the user. (ok) c. On the 6th change, the exception below occurs. It's like it needs to reuse the first pwdHistory attribute but cannot. have fixed this issue, please verify with the latest trunk and let us know. #!RESULT ERROR #!CONNECTION ldap://localhost:10389 #!DATE 2011-10-11T14:32:58.205 #!ERROR [LDAP: error code 20 - ATTRIBUTE_OR_VALUE_EXISTS: failed for MessageType : MODIFY_REQUEST Message ID : 29 Modify Request Object : 'uid=1286309809116,ou=users,ou=int,o=cpro' Modification[0] Operation : replace Modification userPassword: '0x7B 0x53 0x48 0x41 0x7D 0x79 0x59 0x53 0x75 0x30 0x42 0x53 0x75 0x78 0x32 0x49 ...' org.apache.directory.shared.ldap.model.message.ModifyRequestImpl@3d1acad9: ERR_54 Cannot add a value which is already present : '0x32 0x30 0x31 0x31 0x31 0x30 0x31 0x31 0x31 0x38 0x33 0x32 0x30 0x34 0x5A 0x23 ...'] dn: uid=1286309809117,ou=users,ou=int,o=cpro changetype: modify replace: userPassword userPassword:: e1NIQX15VVN1MEJTdXgySTZWUEJaSGFCNmhmMUxkaTA9 I'll keep testing and thank you in advance!! Carlo Accorsi -- Kiran Ayyagari
Re: Performance issues and strange logs
On 10/13/11 6:52 PM, Craig Setera wrote: We have a custom interceptor plugged in to 1.5.5. Did anything change in 1.5.7 that would break that? If not, we may be best off to just move to 1.5.7. I don't know. Can you post the interceptor method's signatures? -- Regards, Cordialement, Emmanuel Lécharny www.iktek.com
Re: Performance issues and strange logs
Unfortunately, it appears there was a lot of refactoring between 1.5.5 and
1.5.7. It looks like MINA was updated in 1.5.6... do you have any idea if
the refactoring in that case would be less than what I'm seeing?
Thanks again,
Craig
On Thursday, October 13, 2011, Craig Setera wrote:
Oh... Hopefully it works without recompile or this gets trickier. I guess
we will find out soon enough.
Thanks,
Craig
--
Craig Setera
Director, Product Engineering
mFoundry
p 415.324.5801
cr...@mfoundry.com javascript:_e({}, 'cvml', 'cr...@mfoundry.com');
--
Craig Setera
Director, Product Engineering
mFoundry
p 415.324.5801
cr...@mfoundry.com
Re: Performance issues and strange logs
Looks like our emails crossed along the way. The breakage is pretty extensive. I can work through it, but wondering if 1.5.6 would be any better? On Thursday, October 13, 2011, Emmanuel Lécharny wrote: On 10/13/11 6:52 PM, Craig Setera wrote: We have a custom interceptor plugged in to 1.5.5. Did anything change in 1.5.7 that would break that? If not, we may be best off to just move to 1.5.7. I don't know. Can you post the interceptor method's signatures? -- Regards, Cordialement, Emmanuel Lécharny www.iktek.com -- Craig Setera Director, Product Engineering mFoundry p 415.324.5801 cr...@mfoundry.com
Re: Performance issues and strange logs
On 10/13/11 10:43 PM, Craig Setera wrote: Unfortunately, it appears there was a lot of refactoring between 1.5.5 and 1.5.7. It looks like MINA was updated in 1.5.6... do you have any idea if the refactoring in that case would be less than what I'm seeing? Again, it's hard to tell. If you provide your interceptor's method signatures, I can check. -- Regards, Cordialement, Emmanuel Lécharny www.iktek.com
Re: Performance issues and strange logs
I appreciate the offer. Unfortunately, it is quite a lot of code. I'm going to try a new version of MINA inside of 1.5.5 and see what happens. On Thursday, October 13, 2011, Emmanuel Lecharny wrote: On 10/13/11 10:43 PM, Craig Setera wrote: Unfortunately, it appears there was a lot of refactoring between 1.5.5 and 1.5.7. It looks like MINA was updated in 1.5.6... do you have any idea if the refactoring in that case would be less than what I'm seeing? Again, it's hard to tell. If you provide your interceptor's method signatures, I can check. -- Regards, Cordialement, Emmanuel Lécharny www.iktek.com -- Craig Setera Director, Product Engineering mFoundry p 415.324.5801 cr...@mfoundry.com
Re: Performance issues and strange logs
On 10/13/11 11:01 PM, Craig Setera wrote: I appreciate the offer. Unfortunately, it is quite a lot of code. I'm going to try a new version of MINA inside of 1.5.5 and see what happens. Just the method's signature, not the code. If there is some issue with 1.5.7, it will be around the signatures. -- Regards, Cordialement, Emmanuel Lécharny www.iktek.com
Re: Performance issues and strange logs
Having connected my code to the 1.5.7 code it is more than that. I think I may have reached into more internals for my implementation than I should have and that that is the primary problem. With that said, my initial tests with Mina 2.0.4 underneath 1.5.5 seem to be working. Is there anything I would want to look for that might not be obvious in that scenario? Thanks yet again, Craig On Thursday, October 13, 2011, Emmanuel Lécharny wrote: On 10/13/11 11:01 PM, Craig Setera wrote: I appreciate the offer. Unfortunately, it is quite a lot of code. I'm going to try a new version of MINA inside of 1.5.5 and see what happens. Just the method's signature, not the code. If there is some issue with 1.5.7, it will be around the signatures. -- Regards, Cordialement, Emmanuel Lécharny www.iktek.com -- Craig Setera Director, Product Engineering mFoundry p 415.324.5801 cr...@mfoundry.com
Re: Replication configuration
Yeah, kerberos failed, I commented it out because I don't run it. I will let you know -j On Oct 13, 2011, at 11:54 AM, Jonathan Disher wrote: I built 1180264 (after much difficulty, getting deb packages to build, so I could deploy it to my admin nodes), but can't get it to start because things are missing, and unfortunately the documentation for 2.0 is, well, nonexistent. Right now it's vomiting on the lack of any schema ldif/structure being present. I am trying to not build locally on the admin nodes, they need to remain clean and reproducible, deployable binary packages are required (I need to build, at last count, approx 13 replicas world wide). I will svn up and rebuild, but I suspect that will not fix my schema problems. Is there any 2.0 documentation in any functional state? -j On Oct 11, 2011, at 1:24 PM, Kiran Ayyagari wrote: the trunk is back to stable condition, you can svn up your local copy or checkout the head directly On Mon, Oct 10, 2011 at 6:32 PM, Kiran Ayyagari kayyag...@apache.org wrote: http://svn.apache.org/repos/asf/directory/apacheds/trunk-with-dependencies please checkout the revision 1180264 cause currently there are few changes that destabilized the trunk (should be back to normal in couple days) Please let us know if you have any issues. On Mon, Oct 10, 2011 at 6:06 PM, Jonathan Disher jdis...@parad.net wrote: So, I checked out trunk and appear to have gotten ApacheDS 1.5. Where do I check out the 2.0 tree? -j On Oct 6, 2011, at 5:30 PM, Kiran Ayyagari wrote: This has been fixed, please see https://issues.apache.org/jira/browse/DIRSERVER-1666 for the details. Appreciate if you can verify the fix and report your findings. On Thu, Oct 6, 2011 at 3:33 PM, Kiran Ayyagari kayyag...@apache.org wrote: I confirm the presence of this issue, (which seems to be introduced during some code cleanup) will commit a fix in another 7 hours and let you know. Thanks for the detailed trace. On Thu, Oct 6, 2011 at 2:57 PM, Jonathan Disher jdis...@parad.net wrote: Yes, I did. Actually, I am getting this: [18:55:46] ERROR [org.apache.directory.server.ldap.replication.provider.SyncReplRequestHandler] - Failed to initialize the log files required by the syncrepl provider java.lang.NullPointerException at org.apache.directory.server.ldap.replication.provider.SyncReplSearchListener.init(SyncReplSearchListener.java:98) at org.apache.directory.server.ldap.replication.provider.SyncReplRequestHandler.registerPersistentSearches(SyncReplRequestHandler.java:929) at org.apache.directory.server.ldap.replication.provider.SyncReplRequestHandler.start(SyncReplRequestHandler.java:171) at org.apache.directory.server.ldap.LdapServer.startReplicationProducer(LdapServer.java:506) at org.apache.directory.server.ldap.LdapServer.start(LdapServer.java:488) at org.apache.directory.server.ApacheDsService.startLdap(ApacheDsService.java:374) at org.apache.directory.server.ApacheDsService.start(ApacheDsService.java:174) at org.apache.directory.server.ApacheDsTanukiWrapper.start(ApacheDsTanukiWrapper.java:71) at org.tanukisoftware.wrapper.WrapperManager$12.run(WrapperManager.java:2788) [18:55:46] ERROR [PROVIDER_LOG] - Failed to initialize the log files required by the syncrepl provider java.lang.NullPointerException at org.apache.directory.server.ldap.replication.provider.SyncReplSearchListener.init(SyncReplSearchListener.java:98) at org.apache.directory.server.ldap.replication.provider.SyncReplRequestHandler.registerPersistentSearches(SyncReplRequestHandler.java:929) at org.apache.directory.server.ldap.replication.provider.SyncReplRequestHandler.start(SyncReplRequestHandler.java:171) at org.apache.directory.server.ldap.LdapServer.startReplicationProducer(LdapServer.java:506) at org.apache.directory.server.ldap.LdapServer.start(LdapServer.java:488) at org.apache.directory.server.ApacheDsService.startLdap(ApacheDsService.java:374) at org.apache.directory.server.ApacheDsService.start(ApacheDsService.java:174) at org.apache.directory.server.ApacheDsTanukiWrapper.start(ApacheDsTanukiWrapper.java:71) at org.tanukisoftware.wrapper.WrapperManager$12.run(WrapperManager.java:2788) [18:55:46] ERROR [org.apache.directory.server.ApacheDsService] - Cannot start the server : java.lang.NullPointerException [18:56:42] WARN [org.apache.directory.server.ldap.handlers.SearchHandler] - This server does not allow replication Permissions should be fine (it runs as root, I need to listen on 389), so creating files shouldn't be a problem. -j On Oct 6, 2011, at 11:48 AM, Kiran Ayyagari wrote: did you add the attribute 'ads-replReqHandler' and restarted the server? On Thu, Oct 6, 2011 at 2:26 PM, Jonathan Disher jdis...@parad.net wrote: So, I did, and I followed the
RE: ApacheDS Password policy issues
Thank you so much. I won't be able to test all you fixed until Monday but I'll
let you know.
I meant to get back to you about the ads-pwcheckquality. When I have it set to
2, I get exceptions for length. (ok, thanks)
We set the userPassword attribute using the code snip below.
String strPassword = foo;
MessageDigest oMsgDigest = MessageDigest.getInstance(SHA);
oMsgDigest.update(strPassword.getBytes());
byte[] b = oMsgDigest.digest();
String strResult = {SHA}+getEncodeBase64(b);
When we try and do this (with ads-pwcheckquality=2)
javax.naming.directory.InvalidAttributeValueException is thrown .. cannot
verify the quality of the non-cleartext passwords
OK fine. I'd be happy to just set the clear text value but how does it know
the pw algorithm to store it with? Or does it not matter anymore? Thanks!!
-Original Message-
From: ayyagariki...@gmail.com [mailto:ayyagariki...@gmail.com] On Behalf Of
Kiran Ayyagari
Sent: Thursday, October 13, 2011 4:17 PM
To: users@directory.apache.org
Subject: Re: ApacheDS Password policy issues
On Tue, Oct 11, 2011 at 3:11 PM, carlo.acco...@ibs-ag.com wrote:
Hi, I've been working with the password policy functionality this week and
have encountered a few issues I'm hoping you can help clarify.
These attributes are on the policy itself unless otherwise specified.
1. ads-pwdminlength (minimum # of chars require for a password) having
a non-zero value accepts passwords that are any length.
a. I didn't test ads-pwdmaxlength but might check that while you're
there.
2. The value ads-pwmaxage is supposed to be how long a password is
valid (in seconds).
a. Setting this to a non-zero value causes a pwdChangedTime
attribute to be set on the user when their password changes (ok)
b. However it never enforces the expiry
i.
The ads-pwdgraceauthnlimit ( # of grace logins after expiration)
doesn't seem to have any effect
ii.
Also setting ads-pwdexpirewarning above and below the max age
doesn't seem to matter either
c. If it did expire, how is this indicated on the user object ?
have fixed this issue. Server indicates the user about expiry by sending the
ppolicy response control after setting the value for timeBeforeExpiration
property to the time left before the password expires.
Note that this only happens if the user sent a request with ppolicy control
(OID - 1.3.6.1.4.1.42.2.27.8.5.1)
3. When ads-pwdmaxfailure (number of times failed bind is permitted) is
set to 5 , it allows 11 login failures before locking the account.
a. Each login failure creates an additional pwdFailureTime
attribute for the user (ok)
b. pwdAccountLockedTime attribute is created after the 11th
failed bind. (Also what we want, but after 5 failures)
c. This might be some caching issue because I think once it took 13
failed attempts before it locked.
this is a bit strange, do you have some custom caching mechanism in place? OR
some custom authenticator implementation that doesn't inherit the
AbstractAuthenticator?
4. When ads-pwdinhistory (# of old passwords kept so they're not
reused) is set to 5 .
a. Users initially have no pwdHistory attribute (ok)
b. Each of the first 5 password changes happens successfully.
Each time adding new pwdHistory attribute to the user. (ok)
c. On the 6th change, the exception below occurs. It's like it needs
to reuse the first pwdHistory attribute but cannot.
have fixed this issue, please verify with the latest trunk and let us know.
#!RESULT ERROR
#!CONNECTION ldap://localhost:10389
#!DATE 2011-10-11T14:32:58.205
#!ERROR [LDAP: error code 20 - ATTRIBUTE_OR_VALUE_EXISTS: failed for
MessageType : MODIFY_REQUEST Message ID : 29 Modify Request
Object : 'uid=1286309809116,ou=users,ou=int,o=cpro'
Modification[0] Operation : replace
Modification userPassword: '0x7B 0x53 0x48 0x41 0x7D 0x79 0x59
0x53 0x75 0x30 0x42 0x53 0x75 0x78 0x32 0x49 ...'
org.apache.directory.shared.ldap.model.message.ModifyRequestImpl@3d1ac
ad9: ERR_54 Cannot add a value which is already present : '0x32 0x30
0x31 0x31 0x31 0x30 0x31 0x31 0x31 0x38 0x33 0x32 0x30 0x34 0x5A 0x23
...']
dn: uid=1286309809117,ou=users,ou=int,o=cpro
changetype: modify
replace: userPassword
userPassword:: e1NIQX15VVN1MEJTdXgySTZWUEJaSGFCNmhmMUxkaTA9
I'll keep testing and thank you in advance!!
Carlo Accorsi
--
Kiran Ayyagari
Re: Performance issues and strange logs
On 10/13/11 11:22 PM, Craig Setera wrote: Having connected my code to the 1.5.7 code it is more than that. I think I may have reached into more internals for my implementation than I should have and that that is the primary problem. With that said, my initial tests with Mina 2.0.4 underneath 1.5.5 seem to be working. Is there anything I would want to look for that might not be obvious in that scenario? Not that much. May be compare the LdapServer class, we may have changed the way we have initialized the MINA layer there (the startNetwork method, from the top of my head). -- Regards, Cordialement, Emmanuel Lécharny www.iktek.com
Re: ApacheDS Password policy issues
On Thu, Oct 13, 2011 at 5:45 PM, carlo.acco...@ibs-ag.com wrote:
Thank you so much. I won't be able to test all you fixed until Monday but
I'll let you know.
I meant to get back to you about the ads-pwcheckquality. When I have it set
to 2, I get exceptions for length. (ok, thanks)
We set the userPassword attribute using the code snip below.
String strPassword = foo;
MessageDigest oMsgDigest = MessageDigest.getInstance(SHA);
oMsgDigest.update(strPassword.getBytes());
byte[] b = oMsgDigest.digest();
String strResult = {SHA}+getEncodeBase64(b);
When we try and do this (with ads-pwcheckquality=2)
javax.naming.directory.InvalidAttributeValueException is thrown .. cannot
verify the quality of the non-cleartext passwords
OK fine. I'd be happy to just set the clear text value but how does it know
the pw algorithm to store it with? Or does it not matter anymore? Thanks!!
there is an interceptor enabled by default for hashing the passwords
using hash method 'SSHA', so clear text passwords will be checked
for ppolicy conformance and hashed before storing into server
-Original Message-
From: ayyagariki...@gmail.com [mailto:ayyagariki...@gmail.com] On Behalf Of
Kiran Ayyagari
Sent: Thursday, October 13, 2011 4:17 PM
To: users@directory.apache.org
Subject: Re: ApacheDS Password policy issues
On Tue, Oct 11, 2011 at 3:11 PM, carlo.acco...@ibs-ag.com wrote:
Hi, I've been working with the password policy functionality this week and
have encountered a few issues I'm hoping you can help clarify.
These attributes are on the policy itself unless otherwise specified.
1. ads-pwdminlength (minimum # of chars require for a password) having
a non-zero value accepts passwords that are any length.
a. I didn't test ads-pwdmaxlength but might check that while you're
there.
2. The value ads-pwmaxage is supposed to be how long a password is
valid (in seconds).
a. Setting this to a non-zero value causes a pwdChangedTime
attribute to be set on the user when their password changes (ok)
b. However it never enforces the expiry
i.
The ads-pwdgraceauthnlimit ( # of grace logins after expiration)
doesn't seem to have any effect
ii.
Also setting ads-pwdexpirewarning above and below the max age
doesn't seem to matter either
c. If it did expire, how is this indicated on the user object ?
have fixed this issue. Server indicates the user about expiry by sending the
ppolicy response control after setting the value for timeBeforeExpiration
property to the time left before the password expires.
Note that this only happens if the user sent a request with ppolicy control
(OID - 1.3.6.1.4.1.42.2.27.8.5.1)
3. When ads-pwdmaxfailure (number of times failed bind is permitted)
is set to 5 , it allows 11 login failures before locking the account.
a. Each login failure creates an additional pwdFailureTime
attribute for the user (ok)
b. pwdAccountLockedTime attribute is created after the 11th
failed bind. (Also what we want, but after 5 failures)
c. This might be some caching issue because I think once it took 13
failed attempts before it locked.
this is a bit strange, do you have some custom caching mechanism in place? OR
some custom authenticator implementation that doesn't inherit the
AbstractAuthenticator?
4. When ads-pwdinhistory (# of old passwords kept so they're not
reused) is set to 5 .
a. Users initially have no pwdHistory attribute (ok)
b. Each of the first 5 password changes happens successfully.
Each time adding new pwdHistory attribute to the user. (ok)
c. On the 6th change, the exception below occurs. It's like it needs
to reuse the first pwdHistory attribute but cannot.
have fixed this issue, please verify with the latest trunk and let us know.
#!RESULT ERROR
#!CONNECTION ldap://localhost:10389
#!DATE 2011-10-11T14:32:58.205
#!ERROR [LDAP: error code 20 - ATTRIBUTE_OR_VALUE_EXISTS: failed for
MessageType : MODIFY_REQUEST Message ID : 29 Modify Request
Object : 'uid=1286309809116,ou=users,ou=int,o=cpro'
Modification[0] Operation : replace
Modification userPassword: '0x7B 0x53 0x48 0x41 0x7D 0x79 0x59
0x53 0x75 0x30 0x42 0x53 0x75 0x78 0x32 0x49 ...'
org.apache.directory.shared.ldap.model.message.ModifyRequestImpl@3d1ac
ad9: ERR_54 Cannot add a value which is already present : '0x32 0x30
0x31 0x31 0x31 0x30 0x31 0x31 0x31 0x38 0x33 0x32 0x30 0x34 0x5A 0x23
...']
dn: uid=1286309809117,ou=users,ou=int,o=cpro
changetype: modify
replace: userPassword
userPassword:: e1NIQX15VVN1MEJTdXgySTZWUEJaSGFCNmhmMUxkaTA9
I'll keep testing and thank you in advance!!
Carlo Accorsi
--
Kiran Ayyagari
--
Kiran Ayyagari
Re: Apache Directory LDAP client API persistent search
On Thu, Oct 6, 2011 at 11:05 PM, Emmanuel Lecharny elecha...@gmail.com wrote: On 10/6/11 10:44 PM, Charles FENDT wrote: Hi, I'm trying to work with Apache DS for a central repository and Apache Directory LDAP Client API on Java servers to request in the repository. It work pretty good... except one thing : I need to make persistent sear in the LDAP server... and i can't make it work... I didn't found any example or howto on which base my code... So I'm asking for some help ! You may check this test : http://svn.apache.org/viewvc/directory/apacheds/trunk/server-integ/src/test/java/org/apache/directory/server/operations/search/PersistentSearchIT.java?revision=1124675view=markup It implements a persistent search. Be aware that this test is a bit complex, but feel free to post your question on this mailing list. Hm, the linked test uses JNDI to perform persistent search. Does the LDAP API also support persistent search and if yes are there examples?
Re: Apache Directory LDAP client API persistent search
On 10/14/11 12:33 AM, Stefan Seelmann wrote:
On Thu, Oct 6, 2011 at 11:05 PM, Emmanuel Lecharnyelecha...@gmail.com wrote:
On 10/6/11 10:44 PM, Charles FENDT wrote:
Hi,
I'm trying to work with Apache DS for a central repository and Apache
Directory LDAP Client API on Java servers to request in the repository.
It work pretty good... except one thing : I need to make persistent sear
in the LDAP server... and i can't make it work...
I didn't found any example or howto on which base my code...
So I'm asking for some help !
You may check this test :
http://svn.apache.org/viewvc/directory/apacheds/trunk/server-integ/src/test/java/org/apache/directory/server/operations/search/PersistentSearchIT.java?revision=1124675view=markup
It implements a persistent search.
Be aware that this test is a bit complex, but feel free to post your
question on this mailing list.
Hm, the linked test uses JNDI to perform persistent search. Does the
LDAP API also support persistent search and if yes are there examples?
Well, we have to inject the PersistentSearch control this way :
@Test
public void testSimpleSearchWithControl() throws Exception
{
SearchRequest searchRequest = new SearchRequestImpl().setBase(
new Dn( ou=system ) ).setFilter( (objectclass=*) )
.setScope( SearchScope.ONELEVEL ).addControl( new
PersistentSearchImpl() );
EntryCursor cursor = connection.search( ou=system,
(objectclass=*), SearchScope.ONELEVEL );
Now, we have to deal with the result. It's a bit late for me to add a
test that does it.
--
Regards,
Cordialement,
Emmanuel Lécharny
www.iktek.com
