> -----Original Message-----
> From: Philippe Mouawad [mailto:philippe.moua...@gmail.com]
> Sent: Thursday, July 28, 2016 4:37 PM
> To: dev@jmeter.apache.org
> Subject: Re: JMeter DSL discussion
>
> I think DSL has 2 important advantages:
> - It is more readable in source repositories.
> - it is better for developers who load test their application in CI/CD.
> This is clearly an important move JMeter must follow.
If I might chime in on this conversation? I'd like to offer the
perspective of an organisation actually using JMeter with CI right now.
To preface, we're doing something rather unusual in that we're actually
generating test plans in CI from our functional tests using a command-line
recorder daemon I wrote (based on JMeter; I'm still trying to get permission
to release the source) and then correlating them with post-processing
scripts.
In this capacity, we would be VERY interested in the ability to record to
and operate on a format that is not JMX. From our perspective, a robust and
well-considered test plan DSL brings two major advantages to the table:
determinism and structural soundness.
By determinism, I mean that every node of the test plan tree is consistently
represented. By structural soundness, I mean that every relationship
between nodes arises as a result of the structure expressed in the test plan
file itself (i.e. implementation details don't create implicit structure).
To elaborate further, currently, test plans are written to JMX files. JMX
is an XML-based format which, if you look at its actual structure, is
moderately insane, horrendously verbose, and basically undocumented. After
several months of working with it, I believe it's better to consider it a
serialisation of a Java data structure that happens to be in text form more
than an actual file format. I would personally love nothing more than to
see it deprecated.
If you haven't looked recently, I'll recap: most testElements use stringProp
and friends with a name property to configure their nodes, and the
parent/child relationship between most elements is implicit, relying on the
hashTree immediately following it in the file. The actual tree-like nature
of an XML document is largely ignored in many cases within the format.
(Also, again, it's XML, so there are invalid characters, which is bad.)
For example, this is the structure of a single HTTP request with one
Argument (and some of my own comments):
<HTTPSamplerProxy guiclass="HttpTestSampleGui"
testclass="HTTPSamplerProxy" testname="9952 /as/authorization.oauth2"
enabled="true">
<elementProp name="HTTPsampler.Arguments" elementType="Arguments"
guiclass="HTTPArgumentsPanel" testclass="Arguments" enabled="true">
<collectionProp name="Arguments.arguments">
<elementProp name="state" elementType="HTTPArgument"> <!-- Note
the redundancies, particularly the names -->
<boolProp name="HTTPArgument.always_encode">false</boolProp>
<stringProp name="Argument.name">state</stringProp>
<stringProp name="Argument.value">${state009951}</stringProp>
<stringProp name="Argument.metadata">=</stringProp>
<boolProp name="HTTPArgument.use_equals">true</boolProp>
</elementProp>
</collectionProp>
</elementProp>
<stringProp /> <!-- I'll omit all the attributes and values from here
onward; I think you get the idea -->
<!-- Nine more stringProp here -->
<boolProp />
<!-- Four more boolProp here -->
</HTTPSamplerProxy>
<hashTree>
<HeaderManager>
<collectionProp>
<elementProp>
<stringProp />
<stringProp />
</elementProp>
<!-- Four more elementProp with two stringProp each here -->
</collectionProp>
</HeaderManager>
<hashTree/> <!-- This empty hashTree is NOT optional -->
<RegexExtractor>
<stringProp />
<!-- Six more stringProp here -->
</RegexExtractor>
<hashTree/> <!-- This empty hashTree at the end IS optional -->
</hashTree>
Unaltered, this single request description is 107 lines and 7208 bytes.
Note the <hashTree> after the </HTTPSamplerProxy> (and many -- but not all!
-- other elements) is understood by JMeter as containing the _children_ of
the <HTTPSamplerProxy>. For elements with no children, it's still necessary
that they be followed by an empty <hashTree/> if there are more elements at
the same level of the tree. (This is, I believe, a consequence of a
TestPlan's internal representation as a ListedHashTree of ListedHashTrees.)
As far as we've been able to determine, this format is entirely
undocumented.
The consequence of all this is it is EXTREMELY difficult to reliably operate
on a JMX file with external tools, and much of it has been trial-and-error.
Really, it's been a nightmare. A DSL should have clean and clear semantics
that prevent parsing ambiguity and speed development time. Anything that
does not is a wasted effort.
So bearing all that in mind, I would strongly recommend this additional
requirement: a new DSL MUST decouple the test plan format from
implementation details. A good example of how this is bad in JMX is all the
attribute noise on any sampler-- the test plan shouldn't have to know about
the guiclass, testclass, elementType, etc. It's all obvious from the
structure and the type of the node.
> - the language it should be developed in. My personal preference would go
> for Groovy but I am not an expert in DSLs.
Hang on, why would you develop your DSL in a different language from what
JMeter itself is actually written in? This wold seem to add an entire new
toolchain to the build process.
> - how to express what today we can do through JSR223 languages which might
> be the most complex think to translate
Is this really an issue? This seems orthogonal to the goal of creating a
language to describe _test plans_ themselves. As long as this DSL has a way
to express the JSR223 sampler, saving the actual code that goes in it should
go without saying.
Regards,
Wyatt
Confidentiality Notice: This electronic message transmission, including any
attachment(s), may contain confidential, proprietary, or privileged information
from Chemical Abstracts Service ("CAS"), a division of the American Chemical
Society ("ACS"). If you have received this transmission in error, be advised
that any disclosure, copying, distribution, or use of the contents of this
information is strictly prohibited. Please destroy all copies of the message
and contact the sender immediately by either replying to this message or
calling 614-447-3600.
