Ok, Well, we are running inside VM on HDD for server usage (Guess 10000 spins/s), but I believe, there are not SSD drives. In fact, we are using JMS to "buffer" writes, and simulate transactional behaviour (in case user is not added due to any reason, JMS container rolls back operation, and tries next time)
As I told we need to add accounts in normal speed. So really, tens of writes per sec. are just fine, and I believe, hardware is more than sufficient... Target is like 6000 accounts within few hours. AS this addition is buffered by JMS, there is no HUGE load... Today we will move those LDAP servers to dedicated VM machines (this time ubuntu), and will see... I'll post here, if I get some results... We will also review our JMS config... Just last question for now. Do you have some syntetic tests with some parallel writes/reads? On Jul 9, 2013, at 15:03 , Emmanuel Lécharny <[email protected]> wrote: > Le 7/9/13 2:17 PM, Slavomir Kocka a écrit : >> Thanks for response. >> >> I'll write into your answers... >> >> >> On Jul 9, 2013, at 13:33 , Emmanuel Lécharny <[email protected]> wrote: >> >>> Le 7/9/13 12:35 PM, Slavomir Kocka a écrit : >>>> The interesting point is, that we have two servers in mirror. >>>> Not using replication from 2.0, but doing writes separately. >>>> and the second server is just fine... In identical JVM and same hardware... >>>> The difference is, that websphere uses first one as main source, and >>>> second, as spare... >>>> >>>> Few questions anyway. >>>> 1. Where I set memory limits? I didn't change it, so currently "top" >>>> command shows up to 2 GB RAM occupied by apacheds. Can you please point me >>>> out to doc, or just leave default? We are not using swap. >>> There is no configuration in ApacheDS per se, it's about the JVM memory >> I understand that, I just found some commented settings in wrapper.conf. I'm >> running application using command "/etc/init.d/apacheds-2.0.0_M14-default >> start". >> So I'm not sure, where can I set JVM params, and if defaults are not >> sufficient... > > Sorry, I thought you were starting the server in a different way. You > can modify the wrapper.conf file to increase the default memory used by > the server (the current default is 384Mb). >> >>> you set when you start the server (or websphere). >>>> 2. I cannot give you exact numbers, only if I enable logs. >>> Rough estimation is good enough. There is a huge difference between 2 >>> additions per second and 200... >> I believe, now we have like 10 additions per second, if everything goes >> fine. If not, there is like one addition per 30 seconds... :( > Damn... Are you using spinning disks or SSD ? > > Also what is the underlying FS ? (and if it's a spinning disk, what is > the disk speed) >> What we do is: We run 4 servers in parallel. Each of these servers is doing >> write sequentially. > Are all the servers on the same physical machine ? > > >> So in general, there are 4 threads trying to write... As these writes are >> synchronous (we do not start new write until old one finishes) amount of >> writes per second depends on apacheds server response... We need at least 10 >> writes/s. > > Ok, let me give you some insight about the performances I get on my > machine : > - I have a SSD (256Gb), a OCX Vertex 3 (so it's not top-notch, but > pretty decent). > - when I run the AddPerf test we have, I can inject up to 73 entries per > second (15 000 entries are injected) > - the number of indexes are critical : the more index you have, the > slower the server will be. The default server already uses 7 indexes. > > So it should be enough, as soon as you uses a SSD. > > On a spinning disk, the performances will be drastically worse (probably > by an order of magnitude). > > The reason is that we have to flush on disk every time we update the > backend, in order not to lose any information. Ok, it's slightly more > complicated than that, as we use a journal and a transction manager to > guareantee that no data will be lost, even if the server crashes. > > Bottom line the backend we use (JDBM) is nothing but the fastest backend > on earth. We are currently working on a new backend (mavibot, see > https://cwiki.apache.org/confluence/display/labs/Mavibot) which is at > least 2.5 times faster (I re-ran the addPerf test and got 193 add/s, a > 2.6x speed up), but it's not production ready yet. > > We also did some perf tests with Berkeley DB JE, but the performances we > get are pretty much similar with JDBM, for a li which licence is not > compatible. > >> >>>> What I can say, that there are four parallel servers doing writes... And >>>> Whole cell doing reads. So Writes up to 4 servers, reads up to 10 servers >>>> (4 as app reads, 10 as infrastructure reads) >>> FYI, you have to know that while a server is writing, nothing else can >>> be done (including reads). If you have clients poudning the server with >>> writes, then it's likely that the other clients will be blocked, and in >>> this case, it might have some impact on the performances. >>> >> This is not completely clear to me. You say, that if I constantly write to >> LDAP, I cannot read? > No, in fact, between too writes, if some read requests have been > received, they will be executed. It's just that while a write *is* > processed, no other operation can run, and while a read is processed, no > write can be executed (they will just be queued) > > > > -- > Regards, > Cordialement, > Emmanuel Lécharny > www.iktek.com >
