RE: Code Style Guidelines
> -Original Message-
> From: Vladimir Sitnikov [mailto:sitnikov.vladi...@gmail.com]
> Sent: Monday, February 13, 2017 10:20 AM
> To: dev@jmeter.apache.org
> Subject: Re: Code Style Guidelines
>
>> It kind of ruins git-blame output and tends to make understanding the
>> scope of work in any file or tree very unclear.
>
> Please provide at least one example.
>> I recently wanted to see when and why a specific function was added to a
>> project and ran afoul of exactly this issue
The project was internal and I cannot show it publicly. That doesn't make
it less valid an experience or less valuable as a point of perspective.
It's not like this sort of thing is a unique (or even especially rare)
occurrence when you're doing archaeology on legacy code. Avoid
repeating things that cause pain.
> I've provided two examples
Have you? From my perspective, you haven't shown much of anything because
we don't see your images. Regardless, you cannot possibly expect everyone
to use your exact tooling just to browse project history?
Cheers,
Wyatt
Confidentiality Notice: This electronic message transmission, including any
attachment(s), may contain confidential, proprietary, or privileged information
from Chemical Abstracts Service ("CAS"), a division of the American Chemical
Society ("ACS"). If you have received this transmission in error, be advised
that any disclosure, copying, distribution, or use of the contents of this
information is strictly prohibited. Please destroy all copies of the message
and contact the sender immediately by either replying to this message or
calling 614-447-3600.
RE: Code Style Guidelines
> From: Vladimir Sitnikov [mailto:sitnikov.vladi...@gmail.com]
> Sent: Monday, February 13, 2017 4:00 AM
> To: dev@jmeter.apache.org
> Subject: Re: Code Style Guidelines
>
>> My experience is that code reformatting makes history browsing very much
>> harder.
>
> Could you give any example here? What particular use case breaks with
> single-commit reformatting?
It kind of ruins git-blame output and tends to make understanding the scope
of work in any file or tree very unclear.
I recently wanted to see when and why a specific function was added to a
project and ran afoul of exactly this issue: between two moves and a
reformat, the task went from trivial to annoying very fast.
Cheers,
Wyatt
Confidentiality Notice: This electronic message transmission, including any
attachment(s), may contain confidential, proprietary, or privileged information
from Chemical Abstracts Service ("CAS"), a division of the American Chemical
Society ("ACS"). If you have received this transmission in error, be advised
that any disclosure, copying, distribution, or use of the contents of this
information is strictly prohibited. Please destroy all copies of the message
and contact the sender immediately by either replying to this message or
calling 614-447-3600.
RE: How should no-op setter methods be filed in Bugzilla?
> -Original Message-
> From: Felix Schumacher [mailto:felix.schumac...@internetallee.de]
> Sent: Monday, November 28, 2016 2:51 PM
> To: dev@jmeter.apache.org
> Subject: RE: How should no-op setter methods be filed in Bugzilla?
>
> The class you mentioned uses an instance variable instead of using
> this.setProperty(...), so I thought that this might be your problem and
> wanted to ask, of the other examples you found, have the same property
> (that is, not using setProperty).
You know, I hadn't thought of that! That could be a promising lead; I'll
take a closer look and see if I can confirm/deny.
If that _is_ the case, what do you suppose would be the correct approach to
fixing the problem? IMO, the "obvious" solution is to prefer setProperty()
in setter methods. But that's both ugly to look at and not enforceable in
code, so it only exists by convention (i.e. still really easy to mess it
up). I believe it would be more robust and sensible to consider this a bug
in SaveService failing to lower user-facing properties so they can be
included in the output.
>> I've tried. XStream is kind of obtuse, and I'm still trying to figure
>> out the point where the data structure is lowered to a HashTree that
>> SaveService can work with. If I knew that, I might be able to _fix_ it.
>
> I would love to hear more about this, when you have something to report
> here.
I wonder if someone else here can point me in the right direction? Sebb?
Cheers,
Wyatt
Confidentiality Notice: This electronic message transmission, including any
attachment(s), may contain confidential, proprietary, or privileged information
from Chemical Abstracts Service ("CAS"), a division of the American Chemical
Society ("ACS"). If you have received this transmission in error, be advised
that any disclosure, copying, distribution, or use of the contents of this
information is strictly prohibited. Please destroy all copies of the message
and contact the sender immediately by either replying to this message or
calling 614-447-3600.
RE: How should no-op setter methods be filed in Bugzilla?
Whoops, forgot to address this after my machine was rebooted by a power
outage and then work got in the way. Sorry about that.
> -Original Message-
> From: Felix Schumacher [mailto:felix.schumac...@internetallee.de]
> Sent: Tuesday, November 15, 2016 2:51 PM
> To: dev@jmeter.apache.org
> Subject: Re: How should no-op setter methods be filed in Bugzilla?
>
> Am 14.11.2016 um 20:20 schrieb Epp, Jeremiah W (Contractor):
>>> -Original Message-
>>> From: Felix Schumacher [mailto:felix.schumac...@internetallee.de]
>>>
>>> I have attached a test case that models this, with the only downside,
>>> that it works with current trunk.
>>
>> Thank you, though I have to admit I'm not sure how to correctly run that.
>> Unfortunately, nor do I really have time to get deep in this problem right
>> now. The example was really just to show the _class_ of problem I've had to
>> work around. There have been a slew of these.
>
> Is this class of problems, those classes, that use instance variables for
> state instead of jmeter-properties?
Beg your pardon? I was saying its representative of a category of similar
examples that only differ in the class and method name.
>> Let's try this: if you were going to export those postProcessors in your
>> test to a JMX file on the disk, what would the code look like?
>
> Well, if I remember correctly, you are using JMeter in quite some
> interesting ways, that are not the usual ones. So it may very well be,
> that you find interesting behaviour.
I'd say it's a moderately interesting application at most. I've only had to
extend one class; 99% of what I have is pretty obvious use of the APIs.
> The only way I save elements is to let JMeter save them and I haven't
> looked to deeply into that, yet.
I've tried. XStream is kind of obtuse, and I'm still trying to figure out
the point where the data structure is lowered to a HashTree that SaveService
can work with. If I knew that, I might be able to _fix_ it.
Regards,
Wyatt
Confidentiality Notice: This electronic message transmission, including any
attachment(s), may contain confidential, proprietary, or privileged information
from Chemical Abstracts Service ("CAS"), a division of the American Chemical
Society ("ACS"). If you have received this transmission in error, be advised
that any disclosure, copying, distribution, or use of the contents of this
information is strictly prohibited. Please destroy all copies of the message
and contact the sender immediately by either replying to this message or
calling 614-447-3600.
RE: Ideas for 3.2
> -Original Message-
> From: Felix Schumacher [mailto:felix.schumac...@internetallee.de]
> Sent: Sunday, November 27, 2016 5:58 AM
> To: dev@jmeter.apache.org
> Subject: Re: Ideas for 3.2
>
> Am 23.11.2016 um 23:04 schrieb Philippe Mouawad:
>> - Undo/Redo : I sent a mail on this one and a possible way to
>> implement it
>
> It would be nice to have it working, but it seems like an awful lot to do,
> to get it working.
I'd suggest waiting until you have a better idea of how the internals will
look after refactoring. That'll inform the undo API and might even make
things easier if you play it right.
>> - Your idea: possible rework of core architecture to at least
>> introduce a pool of threads or switch to async model allowing us to
>> take advantage of async io
>
> I still think we should experiment in that direction, but it seems to be a
> really big step and shouldn't be done in a hurry.
IMHO, the first step of any major internals work is divorcing the test plan
export format from the straight serialisation of the ListedHashTree.
>> In a previous discussion we had a LONNNG discussion on DSL.
>
> The demo looked really intriguing, but I have no knowledge in that
> direction to help out.
Since we're (finally!) nearing the end of this product cycle, I may be able
to argue in favour of allocating some of my time to assist with this. I do
think this is something that's worth committing to.
Regards,
Wyatt
Confidentiality Notice: This electronic message transmission, including any
attachment(s), may contain confidential, proprietary, or privileged information
from Chemical Abstracts Service ("CAS"), a division of the American Chemical
Society ("ACS"). If you have received this transmission in error, be advised
that any disclosure, copying, distribution, or use of the contents of this
information is strictly prohibited. Please destroy all copies of the message
and contact the sender immediately by either replying to this message or
calling 614-447-3600.
RE: Ideas for 3.2
> -Original Message-
> From: Philippe Mouawad [mailto:philippe.moua...@gmail.com]
> Sent: Sunday, November 27, 2016 7:54 AM
> To: dev@jmeter.apache.org
> Subject: Re: Ideas for 3.2
>
> On Sun, Nov 27, 2016 at 11:57 AM, Felix Schumacher <
> felix.schumac...@internetallee.de> wrote:
>
>>> Couldn't we start simply by trying to replace XML by JSON as an
>>> output/input format ? XStream has an implementation for this.
>>>
>> I JSON really that much more readable compared to XML?
>>
> JSON is more readable but the XStream one might not be, I can investigate
> this more
More readable? Eeh, sorta. It's certainly (a lot) lighter-weight to
parse and it's a decent bog-standard textual data format. But readability
(for humans) can become awfully spotty when you start nesting data
structures. If the XStream JSON output is as neurotic as the XML, it'll be
much, much worse.
Honestly, rather than simply abandoning XML, JMeter could do substantially
better just by using it as a proper format instead of a serialisation.
>>> We wouldn't drop XML yet.
>>>
>> We shouldn't.
>>
>I agree
Though it'll probably end up being painful to maintain, especially never
remove the ability to load it. Keeping backward compatible reading is much
more important than forward compatible export.
Regards,
Wyatt
Confidentiality Notice: This electronic message transmission, including any
attachment(s), may contain confidential, proprietary, or privileged information
from Chemical Abstracts Service ("CAS"), a division of the American Chemical
Society ("ACS"). If you have received this transmission in error, be advised
that any disclosure, copying, distribution, or use of the contents of this
information is strictly prohibited. Please destroy all copies of the message
and contact the sender immediately by either replying to this message or
calling 614-447-3600.
RE: How should no-op setter methods be filed in Bugzilla?
> -Original Message-
> From: Felix Schumacher [mailto:felix.schumac...@internetallee.de]
> Sent: Monday, November 14, 2016 10:11 AM
> To: dev@jmeter.apache.org
> Subject: Re: How should no-op setter methods be filed in Bugzilla?
>
> I have attached a test case that models this, with the only downside,
> that it works with current trunk.
Thank you, though I have to admit I'm not sure how to correctly run that.
Unfortunately, nor do I really have time to get deep in this problem right
now. The example was really just to show the _class_ of problem I've had to
work around. There have been a slew of these.
But, looking at the code real quick gives me an idea...
Okay, that does help narrow it down. Using the getter and setter methods
works fine... at runtime. But there's something janky in how JMX export is
working. Whether it's in JMeter or my own code, I'm not really sure, but if
we don't set the actual properties, they don't get saved in the output.
Let's try this: if you were going to export those postProcessors in your
test to a JMX file on the disk, what would the code look like?
> Which version of JMeter do you use?
Just vanilla 3.0 (previously, 2.13). 3.1 once it comes out. Really, there's
nothing special going on here.
> If you think the api doesn't work as it should, you could first try to
> discuss it here on the mailing list, or if it is really a no-brainer,
> submit a bug. It would be superb, if you could provide a test case showing
> the error.
I was thinking this was pretty open-and-shut. It may still be, but that's
less clear. I guess it really depends on what we settle on as the real bug
in this case. That'll inform how I file it.
Cheers,
Wyatt
Confidentiality Notice: This electronic message transmission, including any
attachment(s), may contain confidential, proprietary, or privileged information
from Chemical Abstracts Service ("CAS"), a division of the American Chemical
Society ("ACS"). If you have received this transmission in error, be advised
that any disclosure, copying, distribution, or use of the contents of this
information is strictly prohibited. Please destroy all copies of the message
and contact the sender immediately by either replying to this message or
calling 614-447-3600.
How should no-op setter methods be filed in Bugzilla?
While working on our load automation tooling, I've had to figure out a lot
of obscure and relatively undocumented things about the JMeter internals.
One of the more bizarre things I've run into is a bunch of setter methods
that apparently just don't work, forcing you to use setProperty() in a bunch
of places instead. It's really annoying.
Example:
JSR223PostProcessor jsr = new JSR223PostProcessor();
jsr. setScriptLanguage("beanshell"); // This doesn't actually do anything.
jsr. setProperty("scriptLanguage", "beanshell"); // We have to use this.
I realised last Friday (after hitting two more) that I should probably be
reporting these, but I wanted to check on the list first about how we want
them in Bugzilla. Should each method have a bug? Each class? Should there
be a tracking bug for them? A bz Keyword? Please advise.
Regards,
Wyatt
Confidentiality Notice: This electronic message transmission, including any
attachment(s), may contain confidential, proprietary, or privileged information
from Chemical Abstracts Service ("CAS"), a division of the American Chemical
Society ("ACS"). If you have received this transmission in error, be advised
that any disclosure, copying, distribution, or use of the contents of this
information is strictly prohibited. Please destroy all copies of the message
and contact the sender immediately by either replying to this message or
calling 614-447-3600.
RE: Throughput timer that generates Poisson process
> -Original Message-
> From: Vladimir Sitnikov [mailto:sitnikov.vladi...@gmail.com]
> Sent: Tuesday, September 20, 2016 10:37 AM
> To: Philippe Mouawad
> Subject: Throughput timer that generates Poisson process
>
> with constant throughput in a range of 0..3600 requests/hour (I'm not sure
> what is the upper bound exactly, however it does exist)
Why is there a limit? Can it be removed?
Also, if getting it merged into JMeter core is a problem, would you be
amenable to releasing it as a plugin instead?
As for name, maybe something like "Poisson Arrivals Timer"?
Regards,
Wyatt
Confidentiality Notice: This electronic message transmission, including any
attachment(s), may contain confidential, proprietary, or privileged information
from Chemical Abstracts Service ("CAS"), a division of the American Chemical
Society ("ACS"). If you have received this transmission in error, be advised
that any disclosure, copying, distribution, or use of the contents of this
information is strictly prohibited. Please destroy all copies of the message
and contact the sender immediately by either replying to this message or
calling 614-447-3600.
RE: Test plan warnings
> -Original Message-
> From: Vladimir Sitnikov [mailto:sitnikov.vladi...@gmail.com]
> Sent: Wednesday, August 31, 2016 4:24 AM
> To: dev@jmeter.apache.org
> Subject: Test plan warnings
>
> 1) A proper way to place inter-page timers is to wrap the timer into some
> kind of "test action". So, if user has "page, timer, page, timer" kind of
> test plan, that deserves a warning "you might probably want to wrap
> timers..."
Having had to explain this to people repeatedly, I can confirm the scoping
rules for timers are not easy for many users to understand.
But is this really the solution?
Currently, the TestAction lets one set a constant pause using an internal
implementation, but the comments in TestAction.java seem to indicate that
inserting a pause isn't really its "intended" purpose, regardless of how it
tends to be used in practice.
My view is there should be an obvious "Think Time Sampler" that removes the
need for end users to set up the wrapper manually (which is really silly,
when you think about it). Set it up so you can bind any timer to it
directly; in the GUI make it a dropdown selector with every available
AbstractTestElement-implementing Timer.
(Also, I do still think there's value in timers discriminating on whether
they apply to Samplers, Controllers, or any TestElement.)
Regards,
Wyatt
Confidentiality Notice: This electronic message transmission, including any
attachment(s), may contain confidential, proprietary, or privileged information
from Chemical Abstracts Service ("CAS"), a division of the American Chemical
Society ("ACS"). If you have received this transmission in error, be advised
that any disclosure, copying, distribution, or use of the contents of this
information is strictly prohibited. Please destroy all copies of the message
and contact the sender immediately by either replying to this message or
calling 614-447-3600.
RE: Thoughts on 2 proposals
> -Original Message-
> From: Vladimir Sitnikov [mailto:sitnikov.vladi...@gmail.com]
> Sent: Friday, August 19, 2016 8:22 AM
> To: dev@jmeter.apache.org
> Cc: supp...@ubikloadpack.com
> Subject: Re: Thoughts on 2 proposals
>
> Once upon a time we've discussed "macro node" feature.
> Basically, JMeter UI don't have to be a one-to-one match of the test plan.
> For instance, JMeter UI might understand "TestAction/Timer" pattern, and
> show that via single node in the tree.
> So it would simplify the particular use-case (I would admit it is an often
> one), while keeping executor logic untouched.
The actual test plan already isn't a full structural match, but I'm not much
a fan of the idea of making things even less clear. My suggestion would be
to include shortcuts for some common/common-sense fragments in the UI.
e.g. you'd go to the context menu->add->fragment templates->"think delay"
...and it would insert the TestAction and Timer for you.
But if we consider the timers, I think a more useful semantic would involve
the ability to specify that the timer only fires on certain node types as a
standard feature.
e.g. It would be a property of timers and probably look something like so:
transaction
With this, you could drop two timers into your test plan root to set constant
throughput and the delay between transaction groups and that's all, folks.
Cheers,
Wyatt
Confidentiality Notice: This electronic message transmission, including any
attachment(s), may contain confidential, proprietary, or privileged information
from Chemical Abstracts Service ("CAS"), a division of the American Chemical
Society ("ACS"). If you have received this transmission in error, be advised
that any disclosure, copying, distribution, or use of the contents of this
information is strictly prohibited. Please destroy all copies of the message
and contact the sender immediately by either replying to this message or
calling 614-447-3600.
RE: JMeter JMX file format
> -Original Message-
> From: sebb [mailto:seb...@gmail.com]
> Sent: Friday, August 12, 2016 4:01 PM
> To: dev@jmeter.apache.org
> Subject: Re: JMeter JMX file format
>
> On 12 August 2016 at 20:33, Epp, Jeremiah W (Contractor) <j...@cas.org>
> wrote:
>> Though it's true that it NEEDS to be changed when the format changes or
>> it's somewhat meaningless. (For example it probably should have changed
>> some time between 2.8 and 2.13 because I'm pretty sure we found a break
>> in there.)
>
> Is there a bug report?
>
> AFAIK a JMX created in 2.8 will still run in 2.13 unless it uses a test
> element that has been retired.
Other way around. JMX created in 2.13 opened in 2.8 but blew up at runtime
somehow. I don't remember the details beyond being peeved at Ubuntu RelEng
for packaging a prehistoric JMeter.
I'm not sure what the bug report for that would even be?
>> Oh god, it really does work exactly like I thought it did. It is quite
>> literally implementation defined.
>
> Of course: that's what serialisation is designed to do. It is however
> much more portable and readable than Java serialisation.
Doesn't it strike you as at least a little bit of a bad idea to use an
undocumented serialisation of a complex recursive data structure as the
"human readable" format, though?
We've definitely found it a huge pain in the butt to work with. :/
>> No wonder it's such a horrible mess.
>
> It works perfectly well for the purpose for which it was designed.
> Calling it a horrible mess does not help.
I went into a little more detail on this topic in the DSL discussion thread.
In brief: the structure of a JMX document _partially_ lacks correspondence
to the structure of the test plan it represents, confounding correct and
safe manipulation. That's a bad thing.
> [Human languages are a 'horrible mess' if considered from the perspective
> of automatically parsing them.]
Haha, this is very true, if a non sequitur.
>> Interesting. Doc link?
>
> https://svn.apache.org/repos/asf/jmeter/trunk/bin/saveservice.properties
>
Thanks. Not what I expected, but enlightening in its own way.
>> What? I think implementing a DSL sort of implies there will be an
>> interpreter for it...
>
> Well yes, but if JMeter also continues to support reading and writing the
> current JMX format how can one have a clean break?
JMX import and export is legacy code. It works. It's content. Leave it
alone. DSL import and export can exist concurrently-- philosophically, it's
no different from how JTL supports CSV and XML.
Users can save to either format unless a test plan uses new features (such
as those mentioned in the DSL thread) that exceed the semantic capabilities
of the JMX format. In that case, error if users attempt to export JMX with
a notification that they have to use the new format (and why).
All right, I'm out. Have a good weekend!
Cheers,
Wyatt
Confidentiality Notice: This electronic message transmission, including any
attachment(s), may contain confidential, proprietary, or privileged information
from Chemical Abstracts Service ("CAS"), a division of the American Chemical
Society ("ACS"). If you have received this transmission in error, be advised
that any disclosure, copying, distribution, or use of the contents of this
information is strictly prohibited. Please destroy all copies of the message
and contact the sender immediately by either replying to this message or
calling 614-447-3600.
RE: JMeter DSL discussion
> -Original Message-
> From: Philippe Mouawad [mailto:philippe.moua...@gmail.com]
> Sent: Friday, August 12, 2016 3:19 PM
> To: dev@jmeter.apache.org
> Subject: Re: JMeter DSL discussion
>
> On Fri, Aug 12, 2016 at 5:10 PM, Epp, Jeremiah W (Contractor)
> <j...@cas.org>
> wrote:
>> There's also the ability to use a remote machine as the recording proxy,
>> though it doesn't work without some sort of display server because of the
>> ProxyControl's dependency on GuiPackage [0] -- we've been xvfb-run to
>> give it the illusion of a screen, but that's an ugly hack no one likes.
>>
>
> Why not provide a patch then ? I'll be happy to review it and commit it
> Did you test the patch that was provided ? Does it work ?
Not yet. I was going to test that patch after I move us over to JMeter 3.x.
That'll hopefully come after we finish bringup on this next product.
>> Interesting! I'd actually be curious what that looks like. We've been
>> toying with the idea of writing a delta-based auto-correlation tool to
>> reduce the special case tools in new products, but that takes time.
>>
> And is complex. But it would be great in JMeter. I'd be happy to help
> you on this.
Regrettably, I can't work on public code without clearing it with legal. If
I can get my current work opened up, I may be able to get my boss to find me
some time to work on this. But not until then. :(
>> For what it's worth, I strongly recommend _against_ using a full
>> programming language as your DSL. That's sort of the opposite of "domain
>> specific". *g*
>
> Did you look at groovy dsls ?
If you want to implement your DSL _in_ Groovy, okay, they have moderately
decent tools for that. But you still pull the Groovy toolchain into the
JMeter build process and you still have to deal with the type system when
you hook into the Java code, so you haven't saved yourself any real effort
from my perspective-- IMHO, you'd be better off working with an ordinary
parser generator lib.
If you want to use Groovy _as_ your DSL, well, okay. It's not very Domain
Specific at that point, but if that's what you really want, I guess you
should do it... The JMeter side is probably going to need a whole pile of
glue so you can properly work with it, though.
Or maybe you have some other vision and I'm missing out (TIMTOWTDI, after
all!). Do you have a sketch of what you think the DSL might look like and
the pipeline for reading it into JMeter, writing it out, etc?
Regards,
Wyatt
Confidentiality Notice: This electronic message transmission, including any
attachment(s), may contain confidential, proprietary, or privileged information
from Chemical Abstracts Service ("CAS"), a division of the American Chemical
Society ("ACS"). If you have received this transmission in error, be advised
that any disclosure, copying, distribution, or use of the contents of this
information is strictly prohibited. Please destroy all copies of the message
and contact the sender immediately by either replying to this message or
calling 614-447-3600.
RE: JMeter JMX file format
> -Original Message-
> From: sebb [mailto:seb...@gmail.com]
> Sent: Friday, August 12, 2016 1:02 PM
> To: dev@jmeter.apache.org
> Subject: Re: JMeter JMX file format
>
>On 12 August 2016 at 15:38, Epp, Jeremiah W (Contractor) <j...@cas.org>
>wrote:
>>
>> Part of the problem there is JMX isn't versioned.
>
> There *is* some versioning info; it's perhaps not changed as frequently as
> it should be. But any changes are upwards compatible.
Okay, egg on my face. Somehow I never actually noticed the version="1.2"
and properties="1.8" attributes on the .
Though it's true that it NEEDS to be changed when the format changes or it's
somewhat meaningless. (For example it probably should have changed some time
between 2.8 and 2.13 because I'm pretty sure we found a break in there.)
> However JMeter uses XStream to serialise the classes that form the nodes,
> so the format is defined by that.
Oh god, it really does work exactly like I thought it did. It is quite
literally implementation defined. No wonder it's such a horrible mess.
> config files to define shorthand aliases to reduce the file size.
Interesting. Doc link?
> Otherwise of course the source code is available.
Source code is not documentation.
Especially _that_ code. I actually _tried_ to figure out what the hell was
happening in SaveService at a few points and it's not easy reading.
>> In my mind, this is the major motivation for a test plan DSL: retain
>> support for the "legacy" JMX as much as possible (read and write), but
>> otherwise have a clean break that lets the project move forward sanely.
>
> I'm not sure how you can do that unless the DSL generates JMX.
What? I think implementing a DSL sort of implies there will be an
interpreter for it...
Cheers,
Wyatt
Confidentiality Notice: This electronic message transmission, including any
attachment(s), may contain confidential, proprietary, or privileged information
from Chemical Abstracts Service ("CAS"), a division of the American Chemical
Society ("ACS"). If you have received this transmission in error, be advised
that any disclosure, copying, distribution, or use of the contents of this
information is strictly prohibited. Please destroy all copies of the message
and contact the sender immediately by either replying to this message or
calling 614-447-3600.
RE: JMeter DSL discussion
> -Original Message-
> From: Andrey Pokhilko [mailto:a...@ya.ru]
> Sent: Wednesday, August 10, 2016 3:46 PM
> To: dev@jmeter.apache.org
> Subject: Re: JMeter DSL discussion
>
> You sound like very sane person
Thanks, I think? Doesn't feel like it, some days... :P
> so I'd be happy to hear your suggestions on Taurus
I'm sure between my colleagues and I we could come up with a laundry list of
small grievances and limitations, though I suspect a lot of it will fall out
of your project scope because you're not trying to enable everything that
JMeter does from Taurus (or so I've been told).
> how to help JMeter users more with lightweight format, without the need to
> rework underlying JMX used by JMeter.
Like I mentioned to sebb in the file format thread, I think the best way to
"handle" JMX is by basically ignoring it.
It can't be changed or it breaks backwards compatibility, and I can't really
see it as a good basis for a new format for reasons I mentioned above (and
that's setting aside my distaste for XML).
What might a new "Test Plan Language" look like? I don't know. I
personally have some vague ideas on a general grammar, but it's a discussion
that needs to happen if and only if people are on board with doing it.
Regards,
Wyatt
Confidentiality Notice: This electronic message transmission, including any
attachment(s), may contain confidential, proprietary, or privileged information
from Chemical Abstracts Service ("CAS"), a division of the American Chemical
Society ("ACS"). If you have received this transmission in error, be advised
that any disclosure, copying, distribution, or use of the contents of this
information is strictly prohibited. Please destroy all copies of the message
and contact the sender immediately by either replying to this message or
calling 614-447-3600.
RE: JMeter DSL discussion
> -Original Message-
> From: Philippe Mouawad [mailto:philippe.moua...@gmail.com]
> Sent: Wednesday, August 10, 2016 4:00 PM
> To: dev@jmeter.apache.org
> Subject: Re: JMeter DSL discussion
>
> First thanks for having in mind to contribute a non gui recorder,
Don't thank me yet; legal's giving me hell over this. :(
> by the way what is the use case ?
What is the use case for a recorder that can be started without a GUI?
Well, recording from continuous integration is ours, and it's only because
of that that we can use JMeter at all.
There's also the ability to use a remote machine as the recording proxy,
though it doesn't work without some sort of display server because of the
ProxyControl's dependency on GuiPackage [0] -- we've been xvfb-run to give
it the illusion of a screen, but that's an ugly hack no one likes.
> Since I started writing JMeter plans, I have always had to write some part
> of it (correlation part) in JSR223+Groovy.
Interesting! I'd actually be curious what that looks like. We've been
toying with the idea of writing a delta-based auto-correlation tool to
reduce the special case tools in new products, but that takes time.
You know how it is.
> Although over the years the volume of custom code tends to drop, I doubt
> it will be ever possible to express in a DSL all the possibilities.
Maybe it won't be able to get _everything_-- needs change over time, after
all. But I think it could get close to 99.9%. And there's nothing keeping
us from adding features if they happen to be needed later.
> But what Vladimir has in mind is maybe to use a language (Groovy or other
> for example) and write in it a DSL for JMeter, in that case what I used to
> do with JSR223 might be possible with this language.
Thinking about this a bit more, I can see how there may be some merit to a
TestPlan API that lets you write test plans entirely in code. But that's a
separate idea, really. For what it's worth, I strongly recommend _against_
using a full programming language as your DSL. That's sort of the opposite
of "domain specific". *g*
> But I am afraid it's a breaking change as he wrote it, I don't know how we
> would be able to handle backward compatibility and Gui compat. And using
> the GUI a lot (for example currently in a load testing mission) , I am
> happy to see that my productivity has even more increased with the new
> features. I am now definitely convinced that it is very useful.
IMO, one aspect of this work should be some refactoring to decouple "JMeter,
the graphical test IDE" from "JMeter, the load execution engine".
This isn't to say the GUI is bad or should be abandoned; far from it! It's
very important!
Tooling matters!
...But I've hit a number of snags when trying to write my proxy that where
the underlying model depends unneccessarily on GUI components.
There are abstraction leaks.
Regards,
Wyatt
[0] https://bz.apache.org/bugzilla/show_bug.cgi?id=57305
Confidentiality Notice: This electronic message transmission, including any
attachment(s), may contain confidential, proprietary, or privileged information
from Chemical Abstracts Service ("CAS"), a division of the American Chemical
Society ("ACS"). If you have received this transmission in error, be advised
that any disclosure, copying, distribution, or use of the contents of this
information is strictly prohibited. Please destroy all copies of the message
and contact the sender immediately by either replying to this message or
calling 614-447-3600.
RE: JMeter JMX file format
> -Original Message-
> From: sebb [mailto:seb...@gmail.com]
> Sent: Friday, August 12, 2016 7:36 AM
> To: dev@jmeter.apache.org
> Subject: Re: JMeter JMX file format
>
> Note that changing the JMX file format has major implications for
> upwards compatibility.
Part of the problem there is JMX isn't versioned. Or really specified at all.
It's a de facto format that arises from the [voodoo] serialisation of the test
plan's root ListedHashTree of JMeterTreeNodes. (Please, DO correct me if I'm
wrong; even a bad spec would have been useful months ago.)
> That's not to say it cannot be done, but the additional work must be
> considered before deciding to make a change.
In my mind, this is the major motivation for a test plan DSL: retain support
for the "legacy" JMX as much as possible (read and write), but otherwise have a
clean break that lets the project move forward sanely.
Regards,
Wyatt
Confidentiality Notice: This electronic message transmission, including any
attachment(s), may contain confidential, proprietary, or privileged information
from Chemical Abstracts Service ("CAS"), a division of the American Chemical
Society ("ACS"). If you have received this transmission in error, be advised
that any disclosure, copying, distribution, or use of the contents of this
information is strictly prohibited. Please destroy all copies of the message
and contact the sender immediately by either replying to this message or
calling 614-447-3600.
RE: JMeter DSL discussion
> -Original Message-
> From: Andrey Pokhilko [mailto:a...@ya.ru]
> Sent: Wednesday, August 10, 2016 2:38 PM
> To: dev@jmeter.apache.org
> Subject: Re: JMeter DSL discussion
>
> As a side note, did you see Taurus project (http://gettaurus.org/),
> which tries to bring some YAML-based way to express JMeter test plans?
Funny enough, you've responded to some of my posts on codename-taurus.
How's it going? ;)
I didn't detail it before, but we _are_ actually using Taurus, though
probably not how you expect, and not as much as I had hoped.
Right now, we use it only to massage the final JMX test plan for adjusting
ramps, thread counts, and slaves. The config is very bare-bones, with just a
single execution. We experimented with the config merging and jmx2yaml, but
ended up spawning a separate JMeter instance for each test. From what you
said before, a slave can only be bound by one instance at a time, so we're
sort of forced to run from a single JMX source file with many thread groups.
Cheers,
Wyatt
Confidentiality Notice: This electronic message transmission, including any
attachment(s), may contain confidential, proprietary, or privileged information
from Chemical Abstracts Service ("CAS"), a division of the American Chemical
Society ("ACS"). If you have received this transmission in error, be advised
that any disclosure, copying, distribution, or use of the contents of this
information is strictly prohibited. Please destroy all copies of the message
and contact the sender immediately by either replying to this message or
calling 614-447-3600.
RE: JMeter DSL discussion
> -Original Message-
> From: Vladimir Sitnikov [mailto:sitnikov.vladi...@gmail.com]
> Sent: Wednesday, August 10, 2016 2:10 PM
> To: dev@jmeter.apache.org
> Subject: Re: JMeter DSL discussion
>
>> Hang on, why would you develop your DSL in a different language from what
>> JMeter itself is actually written in? This wold seem to add an entire
>> new toolchain to the build process.
>
> Can you clarify what you mean by " different language from what JMeter
> itself is actually written in"?
>
> Do you suggest we should use java as a language to store JMeter test
> plans? I'm afraid that won't fly high.
I thought he was suggesting implementing the language with Groovy, not using
Groovy _as_ the language. And no, I'm not suggesting Java would work
better. Quite the opposite, in fact.
The entire point of a Domain Specific Langauge is to have precise control
over what you can and cannot do (semantics) and how it looks when written
out (syntax). It's a form of abstraction.
>>> - how to express what today we can do through JSR223 languages which
>>> might be the most complex think to translate
>>
>> Is this really an issue? This seems orthogonal to the goal of creating a
>> language to describe _test plans_ themselves. As long as this DSL has a
>> way to express the JSR223 sampler, saving the actual code that goes in it
>> should go without saying.
>
> Jeremiah, you are missing a point here. Philippe means: "currently we
> have to use JDS232 samples to workaround limitations of JMeter test plan
> language". For instance, there is no support for "arrays" in the
> samplers, so one has to use JSR232 kind of scripts to work with arrays.
>
> So, the quesion is "how to introduce control flow and data processing into
> the DSL itself". Does it make things more clear?
That does clarify, thank you.
So the issue here is the current test plan implementation lacks important
syntax and semantics for end users. There are workarounds, but they are
kludgy and nontrivial. This is important information for designing a DSL.
With that understanding, though, I'm somewhat worried that the actual
internal representation of a test plan is in question. I suspect changing
_that_ would be a considerable engineering effort.
Before we take this line of thought any further, I think the actual desired
semantics of this DSL need to be hammered out. That is, what do people
explicitly _need_ to be able to do?
Regards,
Wyatt
Confidentiality Notice: This electronic message transmission, including any
attachment(s), may contain confidential, proprietary, or privileged information
from Chemical Abstracts Service ("CAS"), a division of the American Chemical
Society ("ACS"). If you have received this transmission in error, be advised
that any disclosure, copying, distribution, or use of the contents of this
information is strictly prohibited. Please destroy all copies of the message
and contact the sender immediately by either replying to this message or
calling 614-447-3600.
RE: JMeter DSL discussion
> -Original Message-
> From: Philippe Mouawad [mailto:philippe.moua...@gmail.com]
> Sent: Thursday, July 28, 2016 4:37 PM
> To: dev@jmeter.apache.org
> Subject: Re: JMeter DSL discussion
>
> I think DSL has 2 important advantages:
> - It is more readable in source repositories.
> - it is better for developers who load test their application in CI/CD.
> This is clearly an important move JMeter must follow.
If I might chime in on this conversation? I'd like to offer the
perspective of an organisation actually using JMeter with CI right now.
To preface, we're doing something rather unusual in that we're actually
generating test plans in CI from our functional tests using a command-line
recorder daemon I wrote (based on JMeter; I'm still trying to get permission
to release the source) and then correlating them with post-processing
scripts.
In this capacity, we would be VERY interested in the ability to record to
and operate on a format that is not JMX. From our perspective, a robust and
well-considered test plan DSL brings two major advantages to the table:
determinism and structural soundness.
By determinism, I mean that every node of the test plan tree is consistently
represented. By structural soundness, I mean that every relationship
between nodes arises as a result of the structure expressed in the test plan
file itself (i.e. implementation details don't create implicit structure).
To elaborate further, currently, test plans are written to JMX files. JMX
is an XML-based format which, if you look at its actual structure, is
moderately insane, horrendously verbose, and basically undocumented. After
several months of working with it, I believe it's better to consider it a
serialisation of a Java data structure that happens to be in text form more
than an actual file format. I would personally love nothing more than to
see it deprecated.
If you haven't looked recently, I'll recap: most testElements use stringProp
and friends with a name property to configure their nodes, and the
parent/child relationship between most elements is implicit, relying on the
hashTree immediately following it in the file. The actual tree-like nature
of an XML document is largely ignored in many cases within the format.
(Also, again, it's XML, so there are invalid characters, which is bad.)
For example, this is the structure of a single HTTP request with one
Argument (and some of my own comments):
false
state
${state009951}
=
true
Unaltered, this single request description is 107 lines and 7208 bytes.
Note the after the (and many -- but not all!
-- other elements) is understood by JMeter as containing the _children_ of
the . For elements with no children, it's still necessary
that they be followed by an empty if there are more elements at
the same level of the tree. (This is, I believe, a consequence of a
TestPlan's internal representation as a ListedHashTree of ListedHashTrees.)
As far as we've been able to determine, this format is entirely
undocumented.
The consequence of all this is it is EXTREMELY difficult to reliably operate
on a JMX file with external tools, and much of it has been trial-and-error.
Really, it's been a nightmare. A DSL should have clean and clear semantics
that prevent parsing ambiguity and speed development time. Anything that
does not is a wasted effort.
So bearing all that in mind, I would strongly recommend this additional
requirement: a new DSL MUST decouple the test plan format from
implementation details. A good example of how this is bad in JMX is all the
attribute noise on any sampler-- the test plan shouldn't have to know about
the guiclass, testclass, elementType, etc. It's all obvious from the
structure and the type of the node.
> - the language it should be developed in. My personal preference would go
> for Groovy but I am not an expert in DSLs.
Hang on, why would you develop your DSL in a different language from what
JMeter itself is actually written in? This wold seem to add an entire new
toolchain to the build process.
> - how to express what today we can do through JSR223 languages which might
> be the most complex think to translate
Is this really an issue? This seems orthogonal to the goal of creating a
language to describe _test plans_ themselves. As long as this DSL has a way
to express the JSR223 sampler, saving the actual code that goes in it should
go without saying.
Regards,
Wyatt
Confidentiality Notice: This electronic message transmission, including any
attachment(s), may contain confidential, proprietary, or privileged information
RE: Proposal for some changes in Menus
> -Original Message-
> From: Philippe Mouawad [mailto:philippe.moua...@gmail.com]
> Sent: Wednesday, April 27, 2016 16:47
> To: dev@jmeter.apache.org
> Subject: Re: Proposal for some changes in Menus
>
> On Wed, Apr 27, 2016 at 10:40 PM, Epp, Jeremiah W (Contractor)
> <j...@cas.org
>> wrote:
>
>>> -Original Message-
>>> From: Philippe Mouawad [mailto:philippe.moua...@gmail.com]
>>> Sent: Wednesday, April 27, 2016 15:27
>>> To: dev@jmeter.apache.org
>>> Subject: Re: Proposal for some changes in Menus
>>>
>>> Either we spend time on fixing it but IMO it is not worth the investment.
>>>
>> Is the issue really that bad?
>>
>
> No but I don't like "buggy features" :-)
I was thinking more about the actual bug that causes the deferred/partial
update of the UI. That is, are you sure it would be a major time sink that
touches a lot of code? Or could it be a relatively simple fix? (I could see
it going either way depending on how it's all set up, but I haven't looked at
all.)
>> It's far better to have something janky that users might have a chance of
>> finding when they need it.
>>
> I think a bug in a feature does not help reputation of a product.
More than the apparent lack of the feature? ;)
>> Please, no, properties seriously suck. They're about as opaque as you can
>> possibly get for configuring a GUI application. They're basically a
>> non-mechanism from a user standpoint.
>>
> They will soon be fully documented.
How will that be exposed/communicated to users, though? That's possibly even
more important.
Most users have the mindset that nothing can change and they suffer through
whatever the defaults are. It's much worse when the option is all but
invisible.
> But I agree properties are not optimal
I think the biggest issue they have is that you can't actually set them from
within the application itself. Even if it's an option that "(Requires
Restart)", that's something people understand.
> Happy to know I am not the only user to face it.
> In the future, please report any issue you face don't live with it .
Yeah, I figured it was already known (or maybe fixed seeing as we're in the 3.0
RC phase) and there was a lot of other stuff going on. I'll try to get some
time set aside for reporting a bunch of bugs, though.
>> It's just, burying things in properties is a sure way to ensure the
>> options never even get used at all.
>>
> What do you think about my proposal to at least display a popup message to
> tell user to restart JMeter ?
Yeah, a warning that things might be weird until they restart would be a good
call.
Regards,
Wyatt
Confidentiality Notice: This electronic message transmission, including any
attachment(s), may contain confidential, proprietary, or privileged information
from Chemical Abstracts Service ("CAS"), a division of the American Chemical
Society ("ACS"). If you have received this transmission in error, be advised
that any disclosure, copying, distribution, or use of the contents of this
information is strictly prohibited. Please destroy all copies of the message
and contact the sender immediately by either replying to this message or
calling 614-447-3600.
RE: Proposal for some changes in Menus
> -Original Message-
> From: Philippe Mouawad [mailto:philippe.moua...@gmail.com]
> Sent: Wednesday, April 27, 2016 15:27
> To: dev@jmeter.apache.org
> Subject: Re: Proposal for some changes in Menus
>
> Either we spend time on fixing it but IMO it is not worth the investment.
>
Is the issue really that bad?
> I think having features that "partially work" in JMeter is not good for the
> product reputation.
>
It's far better to have something janky that users might have a chance of
finding when they need it.
And "need" isn't an exaggeration: Just yesterday, I was helping a colleague
debug something with JMeter and the font rendering was so bad, it was basically
unreadable; we couldn't even distinguish between lower- and upper-case "i"
until I switched LAF to something else.
...granted, X forwarding to a Windows workstation isn't a COMMON scenario, but
it was a real problem that I would NOT have solved if I had to dig into a
config file and change some option that may or may not even be documented then
restart the application.
> You will still be able to through properties.
>
Please, no, properties seriously suck. They're about as opaque as you can
possibly get for configuring a GUI application. They're basically a
non-mechanism from a user standpoint.
This isn't to say it's not a problem-- I kept having to change the LAF when we
went to a different sampler yesterday, because it doesn't update everything, so
I know the problems you mentioned and they really don't lend a polished feel to
the option.
It's just, burying things in properties is a sure way to ensure the options
never even get used at all.
Regards,
Wyatt
Confidentiality Notice: This electronic message transmission, including any
attachment(s), may contain confidential, proprietary, or privileged information
from Chemical Abstracts Service ("CAS"), a division of the American Chemical
Society ("ACS"). If you have received this transmission in error, be advised
that any disclosure, copying, distribution, or use of the contents of this
information is strictly prohibited. Please destroy all copies of the message
and contact the sender immediately by either replying to this message or
calling 614-447-3600.
RE: Saving responses while recording (from code)
> -Original Message-
> From: Philippe Mouawad [mailto:philippe.moua...@gmail.com]
> Sent: Tuesday, April 12, 2016 08:45
> To: dev@jmeter.apache.org
> Subject: Re: Saving responses while recording (from code)
>
> If you mean that you need the response from api of Proxy then you will have
> to contribute a PR.
That would be cool, but I already noted it's a TODO for
ProxyControl.deliverSampler(). I considered doing that, but I'd be blocked on
the same thing I'm blocked on already: getting the response from the proxy at
all.
> If not, then just put a View Results Tree under Proxy where filename is
> setup and request/response will be saved in xml or csv depending on your
> choice.
>
This is basically what I'm trying to do for now. Except without the GUI.
What does " put a View Results Tree under Proxy " look like from a code
standpoint?
I've been digging at the JMeter code for days, now, and I cannot for the life
of me figure out the underlying mechanism for associating an AbstractVisualizer
(presumably, it'd be this because SimpleDataWriter is basically a no-op) with a
ProxyControl object such that I can then save the output to an XML file. It's
obviously possible because I can get it to work in the GUI.
Right now, I have something like this:
SimpleDataWriter writer = new SimpleDataWriter();
writer.setEnabled(true);
ResultCollector collector = (ResultCollector)
writer.createTestElement(); //This SHOULD set up the default ResultCollector
collector.setFilename("/tmp/response.xml");
proxyControl.addTestElement(collector); //This is probably where I've
gotten it wrong
But when I hit the breakpoint after I stop the proxy and look at the
ResultCollector, nothing's been saved. That last line is probably wrong, but
it's not clear what is right. I can't add the writer itself because it's not a
TestElement, but the other add* methods for ProxyControl make even less sense.
ResultCollector implements TestStateListener, so notifyTestListenersOfStop()
should work on it, but since it's not getting populated, something is clearly
not set up right. But what?
Regards,
Wyatt
Confidentiality Notice: This electronic message transmission, including any
attachment(s), may contain confidential, proprietary, or privileged information
from Chemical Abstracts Service ("CAS"), a division of the American Chemical
Society ("ACS"). If you have received this transmission in error, be advised
that any disclosure, copying, distribution, or use of the contents of this
information is strictly prohibited. Please destroy all copies of the message
and contact the sender immediately by either replying to this message or
calling 614-447-3600.
Saving responses while recording (from code)
I've been working on automating captures of e2e test cases for use in load
testing of our upcoming product line. Getting a recording without the use of
GUI has been pretty awful, but I've managed to reach parity with the manual
recordings one of our QAs has been getting (mostly).
Unfortunately, we need some information from responses at the time of recording.
Through a lot of mucking about in the GUI, I discovered I can add a
SimpleDataWriter as the child of a ProxyControl and capture the responses of a
recording [0] in an XML file. But how do I represent this relationship in
code? The API is rather opaque, and the relationship between
AbstractVisualizer, Result(Collector|Saver|Renderer), et al is unclear even
after digging into the source for a while. I can:
SimpleDataWriter writer = new SimpleDataWriter();
writer.setEnabled(true);
writer.setFile("/tmp/response.xml");
...easily enough, but I'm lost on how to properly bind that to my ProxyControl
and how to make it spit a file out. (Preferably without having to stop the
recorder like with the GUI.)
Alternatively, is there a better way to get the response header and data out of
the ProxyControl? It seems like there's some intent to be able to get the
response while doing traversal of the ProxyControl.target to build the final
test plan: ProxyControl.deliverSampler() docs have a TODO about adding a
serverResponse parameter. But that's been there for a while.
Cheers,
Wyatt
PS: Sorry about the overly-large footer our MTA adds. I can't stop it. orz
[0] Found a funny bug where this only works once per writer; have to make a new
one if you want to do more.
Confidentiality Notice: This electronic message transmission, including any
attachment(s), may contain confidential, proprietary, or privileged information
from Chemical Abstracts Service ("CAS"), a division of the American Chemical
Society ("ACS"). If you have received this transmission in error, be advised
that any disclosure, copying, distribution, or use of the contents of this
information is strictly prohibited. Please destroy all copies of the message
and contact the sender immediately by either replying to this message or
calling 614-447-3600.
