Hi. As you may know, I've lately spent quite a bit of time trying out and learning how to use JMeter, with the intent of replacing the tool I've been using so far (which is very expensive but totally innapropriate for the job I usually do).
This week I've had my first opportunity to use JMeter in the real world, to help a customer load-test their application. Frankly, it's been a real nightmare, and it's not yet over. To start with, the application is pretty complex. It uses all sorts of tricks to do its function: multiple forms chained up which carry data from one to the next, POST requests with additional parameters in the URI, fancy parameter encodings, lots of redirects, most of them with parameters, forms with dynamically-named fields,... Also, probably wasn't helping that I have never worked with the development team, and so I just didn't know about the internals -- I had to go guessing as I progressed. I've not yet had time to isolate and rationalize the problems I've run into, so I can post them as specific bugs and enhancement proposals, but here's a quick description. Please don't take it for what it's not: it's intended to be constructive criticism: I appreciate the speed at which JMeter is improving and I hope knowledge of these limitations will help working in the right (or at least in the most urgent) direction: - The HTTPSampler and the HTTP Proxy don't allow POST requests with additional parameters in the query string (in the URI). - The HTTPSampler and the HTTP Proxy fail on redirects which carry parameters in the query string (they do the redirect but don't pass the parameters on). - The HTTPSampler generates one single Sample even if it does a redirect. This makes debugging the scripts really difficult, because the information on the original request is lost (you only get the page at the end of the redirect). - It's not possible to use any unusual characters in request parameters: the HTTPSampler will always URLEncode them -- even if they were not encoded when the HTTP Proxy received them. - I would summarize all of the above in that they get too much in the way -- mainly for the HTTP Proxy. - When trying to correct the above, I ran into lots of trouble because the protocol.http packages are too convoluted: looks like each class is tightly coupled to each other class there. I think those packages call for refactoring. - The interface to edit the request parameters needs much improvement. I had lots of parameter names 30 or 40 characters long which was really hard to edit in that very small field which only accepts deleting and writing at the end. Add a __regexp function to cope with variable form field names and it's madness. - Do we really need embedded functions? I have a feeling that just having placeholders (aka variables) and modifiers to define them would do the job just as well, and would be much simpler. - The View Results Tree Listener is a very useful debugging tool. I was just missing the information sent in POST requests and a "browse" button. Would also be cool if it labelled events as per the originating sampler, instead of the URL. A "clean" button would also be extremely handy. - I also saw the agregate report throwing lots of ConcurrentUpdateExceptions ... don't yet know if they are harmful. - The counters in the User Parameter Modifier are not "thread safe": each counter goes its own way. To put a simpler equivalent example: I had registered users named "user1" to "user200" with passwords "pwd1" to "pwd200", and I found no way to ensure each thread would properly log in. My case was even worse because I needed to use the same number in two consecutive pages. We need variables (not only constants) real soon (in case they are not already there). - The same goes for the need to use the same regexps again and again -- which very badly affected the performance of my scripts -- variables and modifiers would probably cure this. Finally, less urgent but probably most important, I feel the code is generally too convoluted. I can't tell if it's only in the http packages as I mentioned before or it's a more general problem, but the result is that it makes the threshold for new contributors very high. I will make this the subject of a later e-mail when the dust settles down. Faithfully, Jordi Salvat i Alabart -- To unsubscribe, e-mail: <mailto:[EMAIL PROTECTED]> For additional commands, e-mail: <mailto:[EMAIL PROTECTED]>
