RE: Update 2.0.0-M7 and indexes
Emanuel.. values are there in the debug , just after GracefulDisconnectFactory.. I just arrived in the office but I don't have my 64bit setup done. Are you still able to meet online today? I will have my configuration done in ~20 mins. Then we can speak? I'll send a webex invitation in a few mins. Regards, Carlo Accorsi Carlo Accorsi | Technical Services IBSAmerica, Inc. | www.ibs-us.com | Visit IBS on LinkedIn 24 Hartwell Ave | Lexington | Massachusetts | 02421 +1-781-862-9002 (Office) | +1-781-676-8129 (Direct) | +1-781-862-9003 (Fax) Sign up for the IBS Blog today! IBS provides integrated management solutions that help companies improve quality and lower costs. -Original Message- From: Emmanuel Lécharny [mailto:elecha...@gmail.com] Sent: Tuesday, April 17, 2012 6:17 PM To: users@directory.apache.org Subject: Re: Update 2.0.0-M7 and indexes Le 4/17/12 11:18 PM, carlo.acco...@ibs-ag.com a écrit : Was away all day today. From the debug below, i'm using these memory settings. -Xms512m -Xmx1024m However i've tried 1024 / 1300 and the same thing occurs. I'll set up a 64bit wrapper later tonight and will assign 2GB. I'll let you know if this makes any difference. thanks. It's strange that the command line does not show the -XmxNNN value you have set. Injecting 80K entris should not eat more than 300Mo, so 512M should be just plain perfect. I'm fighting with verious versions of Linux on Virtual box to set up a configuration close to what you have. Wasted 3 hours at least with Ubuntu (man, those guys have lost their grinta...) and Fedora. Trying Mint now... -- Regards, Cordialement, Emmanuel Lécharny www.iktek.com
Re: Update 2.0.0-M7 and indexes
Le 4/18/12 3:01 PM, Emmanuel Lécharny a écrit : Le 4/18/12 12:16 AM, Emmanuel Lécharny a écrit : Le 4/17/12 11:18 PM, carlo.acco...@ibs-ag.com a écrit : Was away all day today. From the debug below, i'm using these memory settings. -Xms512m -Xmx1024m However i've tried 1024 / 1300 and the same thing occurs. I'll set up a 64bit wrapper later tonight and will assign 2GB. I'll let you know if this makes any difference. thanks. It's strange that the command line does not show the -XmxNNN value you have set. Injecting 80K entris should not eat more than 300Mo, so 512M should be just plain perfect. I'm fighting with verious versions of Linux on Virtual box to set up a configuration close to what you have. Wasted 3 hours at least with Ubuntu (man, those guys have lost their grinta...) and Fedora. Trying Mint now... Ok, making progress... First, there is a missing option in the wrapper.conf file : wrapper.startup.timeout=0 It will not exit after 30 seconds (the default value) if the server is not started. Second, there is still a problem when you restart the server (even if you have created the index, stopped, restarted the server, injected the 80K entries, stopped and restarted the server) : the index are fully rebuilt, which is just plain wrong. I'll investigate this point and I'll probably be able to get a fix working for this issue today. Ok, the issue has been fixed with http://svn.apache.org/viewvc?rev=1327580view=rev I have tested the server with this scenario : - create a brand new server with nothing in it - added a context entry for dc=example,dc=com - added index for sn, cn and displayName - stopped the server - re-started the server, index are now present, but empty - injected 80 000 entries - the index are full of data - stopped the server - re-started the server the server was up and running in 15 seconds, with all the data present, and index working. I think we are done with those nasty bugs... pfewww... :) -- Regards, Cordialement, Emmanuel Lécharny www.iktek.com
RE: Update 2.0.0-M7 and indexes
Kool!! Ok, will try it out and get back to the group. Thanks for your help. Regards, Carlo Accorsi -Original Message- From: Emmanuel Lécharny [mailto:elecha...@gmail.com] Sent: Wednesday, April 18, 2012 1:14 PM To: users@directory.apache.org Subject: Re: Update 2.0.0-M7 and indexes Le 4/18/12 3:01 PM, Emmanuel Lécharny a écrit : Le 4/18/12 12:16 AM, Emmanuel Lécharny a écrit : Le 4/17/12 11:18 PM, carlo.acco...@ibs-ag.com a écrit : Was away all day today. From the debug below, i'm using these memory settings. -Xms512m -Xmx1024m However i've tried 1024 / 1300 and the same thing occurs. I'll set up a 64bit wrapper later tonight and will assign 2GB. I'll let you know if this makes any difference. thanks. It's strange that the command line does not show the -XmxNNN value you have set. Injecting 80K entris should not eat more than 300Mo, so 512M should be just plain perfect. I'm fighting with verious versions of Linux on Virtual box to set up a configuration close to what you have. Wasted 3 hours at least with Ubuntu (man, those guys have lost their grinta...) and Fedora. Trying Mint now... Ok, making progress... First, there is a missing option in the wrapper.conf file : wrapper.startup.timeout=0 It will not exit after 30 seconds (the default value) if the server is not started. Second, there is still a problem when you restart the server (even if you have created the index, stopped, restarted the server, injected the 80K entries, stopped and restarted the server) : the index are fully rebuilt, which is just plain wrong. I'll investigate this point and I'll probably be able to get a fix working for this issue today. Ok, the issue has been fixed with http://svn.apache.org/viewvc?rev=1327580view=rev I have tested the server with this scenario : - create a brand new server with nothing in it - added a context entry for dc=example,dc=com - added index for sn, cn and displayName - stopped the server - re-started the server, index are now present, but empty - injected 80 000 entries - the index are full of data - stopped the server - re-started the server the server was up and running in 15 seconds, with all the data present, and index working. I think we are done with those nasty bugs... pfewww... :) -- Regards, Cordialement, Emmanuel Lécharny www.iktek.com
RE: Update 2.0.0-M7 and indexes
Checked out and built revision 1327587 and it's working like a charm.. Server starts up immediately. No issues searching and is super-fast.. Thanks Emmanuel !! Regards, Carlo Accorsi -Original Message- From: Emmanuel Lécharny [mailto:elecha...@gmail.com] Sent: Wednesday, April 18, 2012 1:14 PM To: users@directory.apache.org Subject: Re: Update 2.0.0-M7 and indexes Le 4/18/12 3:01 PM, Emmanuel Lécharny a écrit : Le 4/18/12 12:16 AM, Emmanuel Lécharny a écrit : Le 4/17/12 11:18 PM, carlo.acco...@ibs-ag.com a écrit : Was away all day today. From the debug below, i'm using these memory settings. -Xms512m -Xmx1024m However i've tried 1024 / 1300 and the same thing occurs. I'll set up a 64bit wrapper later tonight and will assign 2GB. I'll let you know if this makes any difference. thanks. It's strange that the command line does not show the -XmxNNN value you have set. Injecting 80K entris should not eat more than 300Mo, so 512M should be just plain perfect. I'm fighting with verious versions of Linux on Virtual box to set up a configuration close to what you have. Wasted 3 hours at least with Ubuntu (man, those guys have lost their grinta...) and Fedora. Trying Mint now... Ok, making progress... First, there is a missing option in the wrapper.conf file : wrapper.startup.timeout=0 It will not exit after 30 seconds (the default value) if the server is not started. Second, there is still a problem when you restart the server (even if you have created the index, stopped, restarted the server, injected the 80K entries, stopped and restarted the server) : the index are fully rebuilt, which is just plain wrong. I'll investigate this point and I'll probably be able to get a fix working for this issue today. Ok, the issue has been fixed with http://svn.apache.org/viewvc?rev=1327580view=rev I have tested the server with this scenario : - create a brand new server with nothing in it - added a context entry for dc=example,dc=com - added index for sn, cn and displayName - stopped the server - re-started the server, index are now present, but empty - injected 80 000 entries - the index are full of data - stopped the server - re-started the server the server was up and running in 15 seconds, with all the data present, and index working. I think we are done with those nasty bugs... pfewww... :) -- Regards, Cordialement, Emmanuel Lécharny www.iktek.com
Re: Update 2.0.0-M7 and indexes
,localport=31001]) wrapperp | accepted a socket from 127.0.0.1 on port 31001 wrapperp | read a packet KEY : 9v_49BY1PMOWCwTs wrapper | Got key from JVM: 9v_49BY1PMOWCwTs wrapperp | send a packet LOW_LOG_LEVEL : 1 wrapperp | send a packet PING_TIMEOUT : 24000 wrapperp | send a packet PROPERTIES : (Property Values) wrapper | Start Application. wrapperp | send a packet START : start jvm 1| Received a packet LOW_LOG_LEVEL : 1 jvm 1| Wrapper Manager: LowLogLevel from Wrapper is 1 jvm 1| Received a packet PING_TIMEOUT : 24000 jvm 1| PingTimeout from Wrapper is 2400 jvm 1| Received a packet PROPERTIES : (Property Values) jvm 1| Received a packet START : start jvm 1| calling WrapperListener.start() jvm 1| Waiting for WrapperListener.start runner thread to complete. jvm 1| WrapperListener.start runner thread started. wrapper | Startup failed: Timed out waiting for signal from JVM. wrapper | JVM did not exit on request, terminated wrapperp | server listening on port 32000. wrapper | JVM was only running for 30 seconds leading to a failed restart count of 1. wrapper | There were 1 failed launches in a row, each lasting less than 300 seconds. Giving up. wrapper | There may be a configuration problem: please check the logs. wrapper |-- Wrapper Stopped D:\Program Files\ApacheDSM7\binpause Regards, Carlo Accorsi Sign up for the IBS Blog today! IBS provides integrated management solutions that help companies improve quality and lower costs. -Original Message- From: Emmanuel Lécharny [mailto:elecha...@gmail.com] Sent: Sunday, April 15, 2012 6:33 AM To: users@directory.apache.org Subject: Re: Update 2.0.0-M7 and indexes Le 4/14/12 5:12 AM, carlo.acco...@ibs-ag.com a écrit : Hi, regarding the server startup issue I'm having after loading a custom partition with 80k users, the problem seems to be the size of master.db For my partition. The file is 450 MB. If I delete my partition directory (and let to recreate on startup) the server starts fine. Here are some other things we tried, none of which worked. Don't waste your time doing experiments, tehre is a clear problem in the index construction when the server is restarted. I'm on it this afternoon. Seems like the index creation actually works, when we have a few hundreds of entries, but we have serious performance issues with 80 000 entries. I'm not sure it's a bug, but certainly some sub-optimal process. Keep tuned... -- Regards, Cordialement, Emmanuel Lécharny www.iktek.com -- Regards, Cordialement, Emmanuel Lécharny www.iktek.com
RE: Update 2.0.0-M7 and indexes
Was away all day today. From the debug below, i'm using these memory settings. -Xms512m -Xmx1024m However i've tried 1024 / 1300 and the same thing occurs. I'll set up a 64bit wrapper later tonight and will assign 2GB. I'll let you know if this makes any difference. thanks. From: Emmanuel Lécharny [elecha...@gmail.com] Sent: Tuesday, April 17, 2012 6:32 AM To: users@directory.apache.org Subject: Re: Update 2.0.0-M7 and indexes Le 4/17/12 12:05 AM, carlo.acco...@ibs-ag.com a écrit : Hi, I just updated the JIRA. https://issues.apache.org/jira/browse/DIRSERVER-1711?focusedCommentId=13255111#comment-13255111 Thanks for all you've done but I'm experiencing the same issue. I've rebuilt the server from scratch from the trunk today at noon eastern time. Again, I create a partition with indexes, restart. Import my o/ou branches, restart. Import 80k users. Once it's complete, searching of users is fast. Once I restart the server, the issue below occurs. I'll investigate the issue. It's pretty strange ... What if you set up the JVM tu use 512M ? D:\Program Files\ApacheDSM7\binwrapper.exe ..\conf\wrapper.conf set.INSTANCE_DIRECTORY=D:\Program files\apachedsm7\in ances\default set.INSTANCE=default wrapper | -- Wrapper Started as Console wrapper | Using tick timer. wrapperp | server listening on port 32000. wrapper | Launching a JVM... wrapper | command: C:\Program Files (x86)\Java\jdk1.6.0_31\bin\java.exe -Dlog4j.configuration=file:///D:\Program fi s\apachedsm7\instances\default/conf/log4j.properties -Dapacheds.var.dir=D:\Program files\apachedsm7\instances\default -Dapacheds.log.dir=D:\Program files\apachedsm7\instances\default/log -Dapacheds.run.dir=D:\Program files\apachedsm7\ stances\default/run -Dapacheds.instance=default -Ddefault.controls=org.apache.directory.shared.ldap.codec.controls. scade.CascadeFactory,org.apache.directory.shared.ldap.codec.controls.manageDsaIT.ManageDsaITFactory,org.apache.director shared.ldap.codec.controls.search.entryChange.EntryChangeFactory,org.apache.directory.shared.ldap.codec.controls.search agedSearch.PagedResultsFactory,org.apache.directory.shared.ldap.codec.controls.search.persistentSearch.PersistentSearch ctory,org.apache.directory.shared.ldap.codec.controls.search.subentries.SubentriesFactory -Dextra.controls=org.apache irectory.shared.ldap.extras.controls.ppolicy_impl.PasswordPolicyFactory,org.apache.directory.shared.ldap.extras.control syncrepl_impl.SyncDoneValueFactory,org.apache.directory.shared.ldap.extras.controls.syncrepl_impl.SyncInfoValueFactory, g.apache.directory.shared.ldap.extras.controls.syncrepl_impl.SyncRequestValueFactory,org.apache.directory.shared.ldap.e ras.controls.syncrepl_impl.SyncStateValueFactory -Ddefault.extendedOperation.requests=org.apache.directory.shared.lda extras.extended.ads_impl.cancel.CancelFactory,org.apache.directory.shared.ldap.extras.extended.ads_impl.certGeneration. rtGenerationFactory,org.apache.directory.shared.ldap.extras.extended.ads_impl.gracefulShutdown.GracefulShutdownFactory, g.apache.directory.shared.ldap.extras.extended.ads_impl.storedProcedure.StoredProcedureFactory -Ddefault.extendedOpera on.responses=org.apache.directory.shared.ldap.extras.extended.ads_impl.gracefulDisconnect.GracefulDisconnectFactory - erver -Xms512m -Xmx1024m -Djava.library.path=../lib -classpath ../lib/apacheds-all-2.0.0-M7-SNAPSHOT.jar;../lib/apac ds-service-2.0.0-M7-SNAPSHOT.jar;../lib/apacheds-wrapper-2.0.0-M7-SNAPSHOT.jar;../lib/wrapper-3.2.3.jar -Dwrapper.key= v_49BY1PMOWCwTs -Dwrapper.port=32000 -Dwrapper.jvm.port.min=31000 -Dwrapper.jvm.port.max=31999 -Dwrapper.debug=TRUE wrapper.pid=7280 -Dwrapper.version=3.2.3 -Dwrapper.native_library=wrapper -Dwrapper.cpu.timeout=10 -Dwrapper.jvmi 1 org.apache.directory.server.wrapper.ApacheDsTanukiWrapper D:\Program files\apachedsm7\instances\default wrapper | JVM started (PID=3132) jvm 1| WrapperManager class initialized by thread: main Using classloader: sun.misc.Launcher$AppClassLoader@f4a24a jvm 1| Wrapper (Version 3.2.3) http://wrapper.tanukisoftware.org jvm 1| Copyright 1999-2006 Tanuki Software, Inc. All Rights Reserved. jvm 1| jvm 1| Wrapper Manager: JVM #1 jvm 1| Running a 32-bit JVM. jvm 1| Wrapper Manager: Registering shutdown hook jvm 1| Wrapper Manager: Using wrapper jvm 1| Load native library. One or more attempts may fail if platform specific libraries do not exist. jvm 1| Loaded native library: wrapper-windows-x86-32.dll jvm 1| Calling native initialization method. jvm 1| Initializing WrapperManager native library. jvm 1| Java Executable: C:\Program Files (x86)\Java\jdk1.6.0_31\bin\java.exe jvm 1| Windows version: 6.1.7601 jvm 1| Java Version : 1.6.0_31-b05 Java HotSpot(TM) Client VM jvm 1| Java VM Vendor : Sun Microsystems Inc. jvm 1| jvm 1| Control event monitor thread
Re: Update 2.0.0-M7 and indexes
Le 4/17/12 11:18 PM, carlo.acco...@ibs-ag.com a écrit : Was away all day today. From the debug below, i'm using these memory settings. -Xms512m -Xmx1024m However i've tried 1024 / 1300 and the same thing occurs. I'll set up a 64bit wrapper later tonight and will assign 2GB. I'll let you know if this makes any difference. thanks. It's strange that the command line does not show the -XmxNNN value you have set. Injecting 80K entris should not eat more than 300Mo, so 512M should be just plain perfect. I'm fighting with verious versions of Linux on Virtual box to set up a configuration close to what you have. Wasted 3 hours at least with Ubuntu (man, those guys have lost their grinta...) and Fedora. Trying Mint now... -- Regards, Cordialement, Emmanuel Lécharny www.iktek.com
Re: Update 2.0.0-M7 and indexes
Making progress... The index were deleted due to some regression introduced last year when using alias and not OIDs when creating index. Another issue is that the master tabl, containing all the entries, was read fully for each index to create, instead of reading it only once, and adding the read entries in all the indexes in one operation. I'm running some more tests before committing the changes. One suggestion : declare the index, stop the server, restart the server, then inject the entries. The index will be present and the server won't try to recreate them when starting, as they will already be present when you will start to inject the entries. You'll pay the price of injecting the entries and creating the indexes only once. More to come... Le 4/15/12 1:42 PM, carlo.acco...@ibs-ag.com a écrit : ok much appreciaed. Thank you! From: Emmanuel Lécharny [elecha...@gmail.com] Sent: Sunday, April 15, 2012 6:32 AM To: users@directory.apache.org Subject: Re: Update 2.0.0-M7 and indexes Le 4/14/12 5:12 AM, carlo.acco...@ibs-ag.com a écrit : Hi, regarding the server startup issue I'm having after loading a custom partition with 80k users, the problem seems to be the size of master.db For my partition. The file is 450 MB. If I delete my partition directory (and let to recreate on startup) the server starts fine. Here are some other things we tried, none of which worked. Don't waste your time doing experiments, tehre is a clear problem in the index construction when the server is restarted. I'm on it this afternoon. Seems like the index creation actually works, when we have a few hundreds of entries, but we have serious performance issues with 80 000 entries. I'm not sure it's a bug, but certainly some sub-optimal process. Keep tuned... -- Regards, Cordialement, Emmanuel Lécharny www.iktek.com -- Regards, Cordialement, Emmanuel Lécharny www.iktek.com
RE: Update 2.0.0-M7 and indexes
: 9v_49BY1PMOWCwTs wrapperp | send a packet LOW_LOG_LEVEL : 1 wrapperp | send a packet PING_TIMEOUT : 24000 wrapperp | send a packet PROPERTIES : (Property Values) wrapper | Start Application. wrapperp | send a packet START : start jvm 1| Received a packet LOW_LOG_LEVEL : 1 jvm 1| Wrapper Manager: LowLogLevel from Wrapper is 1 jvm 1| Received a packet PING_TIMEOUT : 24000 jvm 1| PingTimeout from Wrapper is 2400 jvm 1| Received a packet PROPERTIES : (Property Values) jvm 1| Received a packet START : start jvm 1| calling WrapperListener.start() jvm 1| Waiting for WrapperListener.start runner thread to complete. jvm 1| WrapperListener.start runner thread started. wrapper | Startup failed: Timed out waiting for signal from JVM. wrapper | JVM did not exit on request, terminated wrapperp | server listening on port 32000. wrapper | JVM was only running for 30 seconds leading to a failed restart count of 1. wrapper | There were 1 failed launches in a row, each lasting less than 300 seconds. Giving up. wrapper | There may be a configuration problem: please check the logs. wrapper | -- Wrapper Stopped D:\Program Files\ApacheDSM7\binpause Regards, Carlo Accorsi Sign up for the IBS Blog today! IBS provides integrated management solutions that help companies improve quality and lower costs. -Original Message- From: Emmanuel Lécharny [mailto:elecha...@gmail.com] Sent: Sunday, April 15, 2012 6:33 AM To: users@directory.apache.org Subject: Re: Update 2.0.0-M7 and indexes Le 4/14/12 5:12 AM, carlo.acco...@ibs-ag.com a écrit : Hi, regarding the server startup issue I'm having after loading a custom partition with 80k users, the problem seems to be the size of master.db For my partition. The file is 450 MB. If I delete my partition directory (and let to recreate on startup) the server starts fine. Here are some other things we tried, none of which worked. Don't waste your time doing experiments, tehre is a clear problem in the index construction when the server is restarted. I'm on it this afternoon. Seems like the index creation actually works, when we have a few hundreds of entries, but we have serious performance issues with 80 000 entries. I'm not sure it's a bug, but certainly some sub-optimal process. Keep tuned... -- Regards, Cordialement, Emmanuel Lécharny www.iktek.com
Re: Update 2.0.0-M7 and indexes
Le 4/14/12 5:12 AM, carlo.acco...@ibs-ag.com a écrit : Hi, regarding the server startup issue I'm having after loading a custom partition with 80k users, the problem seems to be the size of master.db For my partition. The file is 450 MB. If I delete my partition directory (and let to recreate on startup) the server starts fine. Here are some other things we tried, none of which worked. Don't waste your time doing experiments, tehre is a clear problem in the index construction when the server is restarted. I'm on it this afternoon. Seems like the index creation actually works, when we have a few hundreds of entries, but we have serious performance issues with 80 000 entries. I'm not sure it's a bug, but certainly some sub-optimal process. Keep tuned... -- Regards, Cordialement, Emmanuel Lécharny www.iktek.com
RE: Update 2.0.0-M7 and indexes
ok much appreciaed. Thank you! From: Emmanuel Lécharny [elecha...@gmail.com] Sent: Sunday, April 15, 2012 6:32 AM To: users@directory.apache.org Subject: Re: Update 2.0.0-M7 and indexes Le 4/14/12 5:12 AM, carlo.acco...@ibs-ag.com a écrit : Hi, regarding the server startup issue I'm having after loading a custom partition with 80k users, the problem seems to be the size of master.db For my partition. The file is 450 MB. If I delete my partition directory (and let to recreate on startup) the server starts fine. Here are some other things we tried, none of which worked. Don't waste your time doing experiments, tehre is a clear problem in the index construction when the server is restarted. I'm on it this afternoon. Seems like the index creation actually works, when we have a few hundreds of entries, but we have serious performance issues with 80 000 entries. I'm not sure it's a bug, but certainly some sub-optimal process. Keep tuned... -- Regards, Cordialement, Emmanuel Lécharny www.iktek.com
Update 2.0.0-M7 and indexes
Hi Folks, we've made a lot of progress but it wouldn't be fun if there wasn't one more gotcha. We Built 2.0.0-M7 SHAPSHOT from the trunk and it installs and runs fine with 1GB memory / 32 bit JVM (Thanks Kiran for pointers last night) I create a partition and index several attributes. When the server restarts the user attribute DBM databases appear ! (Thanks Emmanuel) I load my ldif file with 80 k users. When the ldif is loaded, I can see all 80k users, everything seems fine, the indexes are populated I can search and it all works. Once I shut down the server, it will not restart. The wrapper fails with the error below. The size of the JVM goes from zero to 1.3GB, pauses and exists. I'm sure there's an OutOfMemory exception occurring but even with Debug it's not appearing in the logs. The question I have is with a partition this size, with this number of users, what can I do to make the jvm as small as possible to start? I've reset all my attribute indexes to 100, partition cache is set to 1000 BEFORE I loaded all the users. Still having same result. Any ideas? Many thanks. BTW, those index delete warnings were there before I put any users in the system. D:\Program Files\ApacheDSm7\binwrapper.exe ..\conf\wrapper.conf set.INSTANCE_DIRECTORY=D:\Program files\apachedsm7\inst ances\default set.INSTANCE=default wrapper | -- Wrapper Started as Console wrapper | Using tick timer. wrapperp | server listening on port 32000. wrapper | Launching a JVM... wrapper | command: C:\Program Files (x86)\Java\jdk1.6.0_31\bin\java.exe -Dlog4j.configuration=file:///D:\Program file s\apachedsm7\instances\default/conf/log4j.properties -Dapacheds.var.dir=D:\Program files\apachedsm7\instances\default -Dapacheds.log.dir=D:\Program files\apachedsm7\instances\default/log -Dapacheds.run.dir=D:\Program files\apachedsm7\in stances\default/run -Dapacheds.instance=default -Ddefault.controls=org.apache.directory.shared.ldap.codec.controls.ca scade.CascadeFactory,org.apache.directory.shared.ldap.codec.controls.manageDsaIT.ManageDsaITFactory,org.apache.directory. shared.ldap.codec.controls.search.entryChange.EntryChangeFactory,org.apache.directory.shared.ldap.codec.controls.search.p agedSearch.PagedResultsFactory,org.apache.directory.shared.ldap.codec.controls.search.persistentSearch.PersistentSearchFa ctory,org.apache.directory.shared.ldap.codec.controls.search.subentries.SubentriesFactory -Dextra.controls=org.apache.d irectory.shared.ldap.extras.controls.ppolicy_impl.PasswordPolicyFactory,org.apache.directory.shared.ldap.extras.controls. syncrepl_impl.SyncDoneValueFactory,org.apache.directory.shared.ldap.extras.controls.syncrepl_impl.SyncInfoValueFactory,or g.apache.directory.shared.ldap.extras.controls.syncrepl_impl.SyncRequestValueFactory,org.apache.directory.shared.ldap.ext ras.controls.syncrepl_impl.SyncStateValueFactory -Ddefault.extendedOperation.requests=org.apache.directory.shared.ldap. extras.extended.ads_impl.cancel.CancelFactory,org.apache.directory.shared.ldap.extras.extended.ads_impl.certGeneration.Ce rtGenerationFactory,org.apache.directory.shared.ldap.extras.extended.ads_impl.gracefulShutdown.GracefulShutdownFactory,or g.apache.directory.shared.ldap.extras.extended.ads_impl.storedProcedure.StoredProcedureFactory -Ddefault.extendedOperati on.responses=org.apache.directory.shared.ldap.extras.extended.ads_impl.gracefulDisconnect.GracefulDisconnectFactory -Xm s1024m -Xmx1300m -Djava.library.path=../lib -classpath ../lib/apacheds-all-2.0.0-M7-SNAPSHOT.jar;../lib/apacheds-servi ce-2.0.0-M7-SNAPSHOT.jar;../lib/apacheds-wrapper-2.0.0-M7-SNAPSHOT.jar;../lib/wrapper-3.2.3.jar -Dwrapper.key=idVWuG6KW XIDWvWZ -Dwrapper.port=32000 -Dwrapper.jvm.port.min=31000 -Dwrapper.jvm.port.max=31999 -Dwrapper.debug=TRUE -Dwrapper. pid=2056 -Dwrapper.version=3.2.3 -Dwrapper.native_library=wrapper -Dwrapper.cpu.timeout=10 -Dwrapper.jvmid=1 org.ap ache.directory.server.wrapper.ApacheDsTanukiWrapper D:\Program files\apachedsm7\instances\default wrapper | JVM started (PID=8520) jvm 1| WrapperManager class initialized by thread: main Using classloader: sun.misc.Launcher$AppClassLoader@f4a24a jvm 1| Wrapper (Version 3.2.3) http://wrapper.tanukisoftware.org jvm 1| Copyright 1999-2006 Tanuki Software, Inc. All Rights Reserved. jvm 1| jvm 1| Wrapper Manager: JVM #1 jvm 1| Running a 32-bit JVM. jvm 1| Wrapper Manager: Registering shutdown hook jvm 1| Wrapper Manager: Using wrapper jvm 1| Load native library. One or more attempts may fail if platform specific libraries do not exist. jvm 1| Loaded native library: wrapper-windows-x86-32.dll jvm 1| Calling native initialization method. jvm 1| Initializing WrapperManager native library. jvm 1| Java Executable: C:\Program Files (x86)\Java\jdk1.6.0_31\bin\java.exe jvm 1| Windows version: 6.1.7601 jvm 1| Java Version : 1.6.0_31-b05 Java HotSpot(TM) Client VM jvm 1| Java VM Vendor : Sun Microsystems Inc. jvm
Re: Update 2.0.0-M7 and indexes
Le 4/13/12 10:03 PM, carlo.acco...@ibs-ag.com a écrit : Hi Folks, we've made a lot of progress but it wouldn't be fun if there wasn't one more gotcha. :) We Built 2.0.0-M7 SHAPSHOT from the trunk and it installs and runs fine with 1GB memory / 32 bit JVM (Thanks Kiran for pointers last night) I create a partition and index several attributes. When the server restarts the user attribute DBM databases appear ! (Thanks Emmanuel) I load my ldif file with 80 k users. When the ldif is loaded, I can see all 80k users, everything seems fine, the indexes are populated I can search and it all works. Performances are good, I hope... Once I shut down the server, it will not restart. The wrapper fails with the error below. The size of the JVM goes from zero to 1.3GB, pauses and exists. I'm sure there's an OutOfMemory exception occurring but even with Debug it's not appearing in the logs. The question I have is with a partition this size, with this number of users, what can I do to make the jvm as small as possible to start? I've reset all my attribute indexes to 100, partition cache is set to 1000 BEFORE I loaded all the users. Still having same result. Any ideas? Many thanks. grmblgtmbl :/ I need to check what's going on when the serve ris stopped and restarted. This is clearly a bug, and it should be easy to fix. The thing is that this is a work in progress, and we have modified many things in the index area recently. This should not too much time to get a clue about what's going on and to fix the issue. I'l give it a try this week-end. At least, we are making progress ! Thanks for the info and the follow-up, this is really helpful for us. -- Regards, Cordialement, Emmanuel Lécharny www.iktek.com
RE: Update 2.0.0-M7 and indexes
OK, thanks for all your effort! Understood this a work in progress. Yes, performance is very good, until we restart :) If I could just get the server to restart I'd call it success! Thanks! Regards, Carlo Accorsi -Original Message- From: Emmanuel Lécharny [mailto:elecha...@gmail.com] Sent: Friday, April 13, 2012 5:06 PM To: users@directory.apache.org Subject: Re: Update 2.0.0-M7 and indexes Le 4/13/12 10:03 PM, carlo.acco...@ibs-ag.com a écrit : Hi Folks, we've made a lot of progress but it wouldn't be fun if there wasn't one more gotcha. :) We Built 2.0.0-M7 SHAPSHOT from the trunk and it installs and runs fine with 1GB memory / 32 bit JVM (Thanks Kiran for pointers last night) I create a partition and index several attributes. When the server restarts the user attribute DBM databases appear ! (Thanks Emmanuel) I load my ldif file with 80 k users. When the ldif is loaded, I can see all 80k users, everything seems fine, the indexes are populated I can search and it all works. Performances are good, I hope... Once I shut down the server, it will not restart. The wrapper fails with the error below. The size of the JVM goes from zero to 1.3GB, pauses and exists. I'm sure there's an OutOfMemory exception occurring but even with Debug it's not appearing in the logs. The question I have is with a partition this size, with this number of users, what can I do to make the jvm as small as possible to start? I've reset all my attribute indexes to 100, partition cache is set to 1000 BEFORE I loaded all the users. Still having same result. Any ideas? Many thanks. grmblgtmbl :/ I need to check what's going on when the serve ris stopped and restarted. This is clearly a bug, and it should be easy to fix. The thing is that this is a work in progress, and we have modified many things in the index area recently. This should not too much time to get a clue about what's going on and to fix the issue. I'l give it a try this week-end. At least, we are making progress ! Thanks for the info and the follow-up, this is really helpful for us. -- Regards, Cordialement, Emmanuel Lécharny www.iktek.com
