Yo Yo! More Inline...
--- Alex Karasulu <[EMAIL PROTECTED]> wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Ole Ersoy wrote: > > Hey Alex, > > Hey Ole good to hear from you again. > Yeah - Sorry - been a bit busy proving to the wife that I'm really not just playing video games on the computer all day - http://www.mydemandchain.com ... ok - enough of the shameless self promotion > > Welcome back from vacation! > > Thanks. It was too short. > Thank God 4th of July is coming up! > > - Created maven archetypes specific to performance > > testing directory servers, with a set of > directories > > for DS configuration files. > > Hmmm I see where you're going. You want to generate > projects that have > a server.xml that can be customized for the test > case. What about > having test cases for embedded operation where the > server.xml does not > come into play? There can be 2 separate archetypes for these cases. Just a quick review of the point of the archetype and some general review. BTW - I always go for as simple as possible, just for everyone's benefit...including myself...so please forgive me if I'm getting to trivial. The purpose of a project archetype is to provide a starting point for a project that produces a specific result. So there is one archetype for a webapp, one for writing java code, etc. Most of us have run mvn archetype:create .... And then maybe cd'd into the project directory just created and run mvn eclipse:eclipse. Now the project can be imported into eclipse, since the maven eclipse plugin produced all the additional project files eclipse needs to understand classpath setting, etc. So the 2 maven commands produced a project that we can use as a starting point in eclipse, with default directories for application source code, test code, resources, etc. But, what if it did'nt have everything we wanted in the starting point for this project. We just create a maven project using the archetype plugin, which is passed as a parameter to the archetype:create command and this project creates a project that is the starting point for creating other projects of a specific type, just like a .java file is a means of creating an object of a specific type. So now we can go ahead and create a project starting point (directories and files) that we will create over and over again whenever that specific starting point is needed. So sometimes we want to manipulate the server.xml file, I'm just going to interrupt myself here for a minute...there should be an outuput directory that captures the diff/patch to the server.xml from the starting server.xml...ok...continuing so there's one specific archetype for that type of project, and sometimes we just want to configure ADS programatically, so we don't need the server.xml, so we could make a different archetype for that type of project, although since having a directory with server.xml in it does not inter with programmtic configuration, it might be easier to just explain in the documentation that the server.xml is there...and that it can be used, or .... can be used to configure ADS. So when testing ADS the developer would run mvn archetype:create .... followed by the specific parameters for creating an ADS testing project. This ensures that everyone testing ADS starts with the same inputs. Someone mentioned generating LDIF files. To do this we could create another input directory in the archetype that contained a file with some configuration parameters for generating the LDIF. That way everyone just has to specify that file for apples to apples comparisons. A mojo (which is a maven pojo - and a group of mojos make a maven plugin) would then run and generate the test data from the ldif input file. Similary, there could be a distributed client configuration input file that coordinates multiple clients that are hitting ADS, along with a corresponding mojo for executing that. That file would store things like network configuration params, hardware on client(s), hardware on ADS server, thus also providing the apple for comparison. > > Thus there should be one > > archetype per DS. So apache would have one, and > > OpenLDAP would have another. > > Ok I see now. The OpenLDAP archetype is a bit more > difficult to pull > off tho since it is C-code and not embeddable or > launchable via test > cases. We might have to fork an instance of it. Yeah - I just through this into the mix, since one of the goals was to compare results with other servers. If I had to do it right now I would just run OpenLDAP on a "Sterilized" server that I'm also running apache ds on and then point OpenLDAP to the configuration files in the test directory...which I'm just assuming is possible. Then code that archetype's mojo to perform testing in a way that is as close to the ADS mojo as possible, or precisely the same as the ADS mojo, by making perhaps using the same mojo executing the same JNDI code on both servers. Thus the only difference would be / could be configuration server configuration. > > The configuration files > > directory should mirror the DS's configuration > file > > directory structure. This isolates the DS's > > configuration settings per performance test, and > > provides the initial baseline for everyone to use. > > So by using an archetype you plan to isolate each > performance test case > into a separate maven module? Isolate each performance test case per performance testing project. So there could be 3 separate projects each created using the same starting point, but maybe different hardware, number of clients hitting ADS, different ADS configuration, etc. We're probably saying the same thing here though... > > > - Create another Archetype input directory for the > > LDIF files. > > Ok starting to loose you here. Perhaps my knowledge > of maven archetypes > are breaking down now. Can you elaborate? This is just so that everyone is loading the same LDIF file(s) for the test. So we create the starting point project, and it already comes with the LDIF files, or a configuration file for generating an LDIF file(s). Thus we (and each performance testing project that are being compared) are working with the same input data. > > > - Write a mojo that embeds ApacheDS, loads the > > configuration files, and then runs all the tests > on > > the specific server instance, and then generates > > reports. > > Yeah this I understand. We can do this pretty > easily. > > This way each instance of this archetype is > > running the DS in an isolated manner. > > Hmmm again loosing you here on the meaning or use of > the m2 archetype. > So we are starting with the same inputs, we can have many projects that only do performance testing, but change the inputs in various ways, and this is clean and easy to see, because all the projects have the same structure. For instance we might have a single performance testing project that gets updated and rechecked into subversion, but now someone would have to go back in time through subversion to see how inputs have changed over time, rather than just comparing each project. So we would never check out a performance testing project, change something, run tests, and check it back in, unless we wanted to have that project as a starting point for comparison to itself. > Would also be > > cool to integrate this stuff with Eclipse TPTP, > using > > something similar to the maven eclipse > plugin...that > > way all the results are cross compatible with TPTP > and > > can be opened up in an eclipse TPTP > project...maybe in > > version 2 of the mojo...or mojo plugin set. > > Is this the Eclipse profiling stuff? Yup This > integration would be pretty > tight. Perhaps as you suggest a bit overkill for a > first round. Definitely...although very nice to have for the round where distributed client testing starts happening, or for the round where someone wants to be able to experiment with the tesing in eclipse....doing on the fly type testing. > > What I'm looking to do is to work with a couple guys > here over the > course of the next 1-3 weeks to erect a solid > framework. If we can add > some of this profiler spice it would be really cool > but not a > requirement. FYI I'm a big fan of Yourkit but free > tools are much more > appealing. > I played with TPTP for a day and was very impressed by it's ability to coordinate agents that could hit ADS from multiple clients. I think the integration would be cool in terms of having a TPTP configuration file that both maven and eclipse understand, thus a mojo could coordinate TPTP and it could be done through eclipse as well, when hands on experimenting is needed. > > - Something similar for OpenLDAP... > > - Bind this mojo to the maven lifecycle in each > > archetype. > > - Now anyone can create a quick maven DS > performance > > testing project, make a few configuration file > changes > > if that's what's being tested, add unit tests if > > necessary, and go! > > Sounds really fun to do: it's a nice little project > and the utility will > be high. I like the idea of having different mojos > for each LDAP server > that needs to be setup and leaving setup tasks to > the mojo/archetype > pair. > > > - The mojo should also produce put the reports in > an > > output directory somewhere within the site > directory, > > and this directory should be the same for all DSs. > > This will be nice to have ... an easily published > set of results. > > > I'll be glad to throw together a sample archetype > for > > ApacheDS if this sounds interesting. > > I'm interested in the idea. I'd love to take a look > at it. My Maven 2 > archetype skills are non-existent tho. I need to > brush them up. I'll throw together a starting baseline asap. Here's the quickee on maven archetype development just for future reference. http://maven.apache.org/guides/mini/guide-creating-archetypes.html > > > Coding the mojo should also be fairly > straightforward. > > Yeah I agree plugins are a cake walk in m2. > > SOME NOTES > ========== > > o [Observation] As Quanah suggested in this thread, > we need more than > one client to really stress test an LDAP server for > total throughput > under high concurrency. This however is one kind of > test we will > perform: there are also capacity tests as well. We > can still use the > recommended tools (SLAMD or another), the partial > goal of this effort is > to reduce the setup, and running of LDAP servers to > be tested in various > ways. We want to automate the setup process with an > easy way to > create/setup a harness for various LDAP servers with > similar testing > conditions. > > o Why Maven? It is important to figure out why we > (me too btw) want to > use Maven. OK PLEASE DON'T FLAME ME ON THIS GUYS > :D. First off it's > our build tool of choice for this project. > Secondly it can run test > cases, and fork them if need be with new JVM > settings. It's got a nice > plugin architecture and has these things for > creating project templates > called archetypes. > > BTW Ole, you caught me by surprise when you > suggested using archetypes > and plugins (mojos) for this effort. The idea gives > me a good feeling > even though I don't know why yet ... there must be > many good advantages > to this. > > Alex Well - my goal when I started 4 years ago was to be able to manufacture software while automating everything that can be automated, which I initially started doing a ant prototype for, and then said Nuuu way - and started looking at Maven 2, which was like going from cutting the lawn with scissors to Gasoline power Baby! So yeah - working with a standardized structure and standardized ways, that don't limit flexibility, in working on that structure is fantastic, especially since it's becoming a standard build tool/software manufacturing tool of choice in open source and closed. For instance right now I'm creating archetypes that automate most of the work involved in creating Java Server Faces components, thus in the end all I have to do in provide a single configuration file, and it automatically generates all the JSF files, including configuration files, component java files, renderer files, projects for deploying the testing of the component to Tomcat, testing, packaging, etc. etc. And all this is really easy to do in an iterative way with Maven's multi project capability. Which reminds me - this article is really great in case anyone wants to read up on the type of stuff I just mentioned: http://www.devx.com/Java/Article/28801/0/page/1 OK - I'll send out a zip of a sample archetype creation file along with install instructions for it, so that anyone who wants to can try it out. Cheers, - Ole > > > > > > - Write a maven mojo that > > > > --- Alex Karasulu <[EMAIL PROTECTED]> wrote: > > > > Need Benchmarking/Profiling Perf Suite (BPPS) for > > ApacheDS > > > >> > ========================================================== > > I just sat down in front of my machine and wanted > to > > run some > > performance tests on the server. These questions > > came up: > > > > o What tests do I run? > > o What LDIF should I import first before running > > those tests? > > o Which version of the server should I run these > > tests against? > > o Can I run these tests against other LDAP > servers? > === message truncated === __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com
