Thanks Dave: my tests need to assert against XML, HTML and EDI flat files.
But even for XML, a schema often isn't available (e.g.: when a standard is
only defined by a DTD), or may not be useful. Let me illustrate...
We use the webMethods platform at work. Once, after installing a patch, the
platform subtly changed how it built XML tags containing empty strings.
Instead of building tags together like this:
<a></a>
...it started building them with a carriage return in between:
<a>
</a>
Ariba provides a DTD for the cXML standard. cXML documents built in both,
the old and new fashions, validated against Ariba's DTD in XMLSpy, but Ariba
itself rejected the new documents as invalid. Eventually, webMethods
provided a fix that reverted the platform to the earlier behavior.
I think there is an XML parser setting (ignore whitespace or not) but this
is an example where a simple full-text response checker (with regexs
accounting for permitted variants) would have caught the different behavior
in testing.
Regards,
Sonam Chauhan
--
Electronic Commerce, Corporate Express Australia Ltd.
Phone: +61-2-9335-0725, Email: [EMAIL PROTECTED]
> -----Original Message-----
> From: Dave Maung [mailto:[EMAIL PROTECTED]
> Sent: Wednesday, 27 April 2005 12:10 PM
> To: JMeter Users List
> Subject: RE: Regression Testing w/ JMeter
>
> at work, I wrote XML Schema and validate against xml response using XML
> scehma assertion. It is best suite, if your application is xml based.
> However, If you are using HTML, it is very hard. Maybe XHTML validation
> may work.
>
>
> Dave
>
> Sonam Chauhan <[EMAIL PROTECTED]> wrote:
> Hi Mike, Sebb -
>
> > It would be hard - there could be dozens or even hundreds of characters
> > that need to be escaped else they'll be treated as funky regular
> > expression signifiers.
>
> I played around on this a while back... I recall I used a Perl script to
> process the text to escape metacharacters. (I used the quotemeta builtin
> Perl function that escapes just about everything not alphanumeric with a
> backslash - even spaces!). I could not get it to work though - even with
> modifiers. I _suspect_ it could be a line terminator issue - I suspect
> Java
> does something strange with line terminations of text pasted into a
> textbox.
>
>
> Obviously, even if it works, a 4-step process involving external tools
> isn't
> practical for everyday use. Ideally, an 'Enhanced Response Assertion'
> element would allow me to simply paste in a full document, mark certain of
> text, and edit them as regular expressions (these would then display in a
> different font or color).
>
> Seb, you also mentioned:
> > > The Regex Tester might help here.
>
> I haven't used one so far -- is there a specific one you had in mind?
>
> Regards,
> Sonam Chauhan
> --
> Electronic Commerce, Corporate Express Australia Ltd.
> Phone: +61-2-9335-0725, Email: [EMAIL PROTECTED]
>
>
> > -----Original Message-----
> > From: Michael Stover [mailto:[EMAIL PROTECTED]
> > Sent: Tuesday, 26 April 2005 11:25 PM
> > To: [email protected]
> > Subject: Re: Regression Testing w/ JMeter
> >
> > It would be hard - there could be dozens or even hundreds of characters
> > that need to be escaped else they'll be treated as funky regular
> > expression signifiers.
> >
> > -Mike
> >
> > On Tue, 2005-04-26 at 11:18 +0100, sebb wrote:
> > > Clever idea - should work.
> > > But it is harder to make work than removing/replacing the variable
> > > bits as you are finding!
> > >
> > > The modifiers such as (?m) and (?s) do work in JMeter.
> > >
> > > Note that these can change the meaning of ".", ^ $ etc
> > >
> > > The Regex Tester might help here.
> > >
> > > S.
> > > On 4/26/05, Sonam Chauhan wrote:
> > > > > Or one could develop a new Assertion.
> > > > >
> > > > > You might still need to add a way to edit the responses to remove
> > the
> > > > > variable data.
> > > >
> > > > Sebb and Keith - I tried something similar with plain response
> > assertions
> > > > and regexs... my intention was to get JMeter to act as a rudimentary
> > diff
> > > > facility (it just signals if something is different), but I may have
> > run
> > > > into a limitation the regex support.
> > > >
> > > > The usual way to use Response Assertions is setting snippets of text
> > (which
> > > > may have regexs) in an assertion. JMeter then tries to match the
> > response
> > > > body (or headers) against it.
> > > >
> > > > Is it possible to assert a suitably processed multiline document? I
> > was
> > > > thinking of something like this:
> > > >
> > > > 1. Take the _entire_ body response from a server
> > > > 2. Replace the variable bits with suitable regular expressions
> > > > 3. Quote any other regex metacharacters
> > > > 4. Take the text processed this way, and set it as a JMeter response
> > > > assertion
> > > >
> > > > I tried doing this - JMeter lets me paste in this multi-line text
> into
> > the
> > > > assertion textbox with no problems, but try as I might, I could not
> it
> > to
> > > > work. I also tried using Perl5 extended regular expressions of
> > multiline
> > > > matching - i.e., instead of /abc/m, I set the assertion to (?m)abc -
> > but no
> > > > go.
> > > >
> > > > I'd like your thoughts on this issue.
> > > >
> > > > Regards,
> > > > Sonam Chauhan
> > > > --
> > > > Electronic Commerce, Corporate Express Australia Ltd.
> > > > Phone: +61-2-9335-0725, Email: [EMAIL PROTECTED]
> > > >
> > > >
> > > > > -----Original Message-----
> > > > > From: sebb [mailto:[EMAIL PROTECTED]
> > > > > Sent: Monday, 25 April 2005 10:51 PM
> > > > > To: JMeter Users List
> > > > > Subject: Re: Regression Testing w/ JMeter
> > > > >
> > > > > The Post-Processor Save Responses to a File was partly introduced
> > with
> > > > > this in mind.
> > > > >
> > > > > This takes a name prefix (which I think can be a variable) so you
> > can
> > > > > run the test twice, saving the results to two sets of files, and
> > then
> > > > > use some other tool to do comparisons of the files.
> > > > >
> > > > > You may need to pre-process the files to remove some dynamic data
> > > > > (e.g. dates/times) before doing the diffs.
> > > > >
> > > > > There's no differencing capability built into JMeter at present.
> > > > >
> > > > > However, I guess the Save Responses test element could be extended
> > to
> > > > > include a second filename prefix, and could save the new data and
> > then
> > > > > compare it. Not sure offhand how this should report differences -
> I
> > > > > think it may run too late to change the failure status - but it
> > could
> > > > > certainly log a message.
> > > > >
> > > > > Or one could develop a new Assertion.
> > > > >
> > > > > You might still need to add a way to edit the responses to remove
> > the
> > > > > variable data.
> > > > > I think this could be done using another Post-Processor (BeanShell
> > > > > Assertion should be usable here) that gets run before the Saver
> gets
> > > > > the result. Or this could be added to the Saver itself (easier to
> > > > > understand).
> > > > >
> > > > > I suggest you try using external comparisons first, as this will
> > give
> > > > > a better idea of the sort of text that might need to be removed
> from
> > > > > the responses before diffing.
> > > > >
> > > > > S.
> > > > > On 4/25/05, Keith Weicksel wrote:
> > > > > > I was wondering if anyone used JMeter to regression test a site.
> > > > > > Particularily, what I am looking for is the ability to:
> > > > > >
> > > > > > 1 - create a baseline tests for some path in our site (storing
> the
> > > > > > HTML/DOM response somewhere)
> > > > > > 2 - run the same test again after changes were made to the site
> > and be
> > > > > > able to DIFF the responses (showing the differences somehow)
> > > > > > 3 - if there are changes (and they are OK), replace the baseline
> > > > > > responses with the new valid ones
> > > > > >
> > > > > > I have used JMeter for about a month now, and do not believe it
> > has this
> > > > > > functionality built right into it at this point. My question is
> > what is
> > > > > > the extent of the regression testing ability buiilt into JMeter?
> > And
> > > > > > how difficult would it be to add this functionaility to it
> > myselft?
> > > > > >
> > > > > > Thanks,
> > > > > >
> > > > > > Keith
> > > > > >
> > > > > > ----------------------------------------------------------------
> --
> > ---
> > > > > > To unsubscribe, e-mail: jmeter-user-
> [EMAIL PROTECTED]
> > > > > > For additional commands, e-mail: jmeter-user-
> > [EMAIL PROTECTED]
> > > > > >
> > > > > >
> > > > >
> > > > > ------------------------------------------------------------------
> --
> > -
> > > > > To unsubscribe, e-mail: [EMAIL PROTECTED]
> > > > > For additional commands, e-mail: jmeter-user-
> [EMAIL PROTECTED]
> > > >
> > > > --------------------------------------------------------------------
> -
> > > > To unsubscribe, e-mail: [EMAIL PROTECTED]
> > > > For additional commands, e-mail: [EMAIL PROTECTED]
> > > >
> > > >
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: [EMAIL PROTECTED]
> > > For additional commands, e-mail: [EMAIL PROTECTED]
> >
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: [EMAIL PROTECTED]
> > For additional commands, e-mail: [EMAIL PROTECTED]
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [EMAIL PROTECTED]
> For additional commands, e-mail: [EMAIL PROTECTED]
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
