On Tue, Jan 10, 2012 at 08:25:21PM +0100, Sjoerd Mullender wrote:
> On 2012-01-10 19:54, Charalampos Nikolaou wrote:
> > Hi,
> >
> > I have written some code and now I would like to make changes and
> > still be sure that the new changes are correct. So, I have written
> > some scenarios in mal and put it in a Tests directory. I have read the
> > README file under testing/ directory, but still haven't understood how
> > I could generate the <scenario>.stable.out file. How could that file
> > be generated? Even if I run Mtest.py <mytest>, then Mapprove.py the
> > previously "wrong" output, and then again Mtest.py the test fails.
.. and why / how does it fail?
from the README:
- A tool Mapprove.py is provided to copy the recent output from
$TSTTRGDIR/TST.test.{out,err} to $TSTSRCDIR/TST.stable.{out,err}. Lines
starting with a '!' (i.e. error messages) are eliminated during copying.
Of course, $TSTTRGDIR/TST.test.{out,err} have to be checked and claimed
correct before being approved!
Well, that's about it. After running Mtest.py, run Mapprove.py to copy the
just generated test output from $TSTTRGDIR/TST.test.{out,err} to
$TSTSRCDIR/TST.stable.{out,err}. As Sjoerd pointed out, by default lines
starting with a '!', (MonetDB error messages are usually tagged like this)
are eliminated during copying, i.e., do not end up in
$TSTTRGDIR/TST.test.{out,err}. If your test does generate expected errors
(or other lines starting with a '!') that need to be approved as correct
output, use Mapprove.py's "-f" option (see also `Mapprove.py --help` for
details) to keep these lines. Then, check (by hand --- there is now way to
automize this) whether the just generated and approved output in
$TSTSRCDIR/TST.stable.{out,err} is indeed correct, and fix (by hand)
otherwise. Then, run Mtest.py, again. If Mtest.py reportes a filing test,
check the generated html pages to the the differences in detail.
Please be aware that Mtest.py basically uses a simple "diff" to compare
stable (approved) output and new output, i.e., you test need to produce
exactly the same output in a deterministic way with each run to be suitable
for Mtest.py.
Hope this helps you further.
Stefan
> > Thanks,
> > Babis
>
> In principle that's how it works. However, if there are expected errors
> in the output, you have to supply the -f (force) option to Mapprove.py.
> It will normally filter error messages out.
>
>
> --
> Sjoerd Mullender
>
> ------------------------------------------------------------------------------
> Write once. Port to many.
> Get the SDK and tools to simplify cross-platform app development. Create
> new or port existing apps to sell to consumers worldwide. Explore the
> Intel AppUpSM program developer opportunity. appdeveloper.intel.com/join
> http://p.sf.net/sfu/intel-appdev
> _______________________________________________
> Monetdb-developers mailing list
> Monetdb-developers@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/monetdb-developers
--
| Stefan.Manegold @ CWI.nl | DB Architectures (INS1) |
| http://CWI.nl/~manegold/ | Science Park 123 (L321) |
| Tel.: +31 (0)20 592-4212 | 1098 XG Amsterdam (NL) |
------------------------------------------------------------------------------
Write once. Port to many.
Get the SDK and tools to simplify cross-platform app development. Create
new or port existing apps to sell to consumers worldwide. Explore the
Intel AppUpSM program developer opportunity. appdeveloper.intel.com/join
http://p.sf.net/sfu/intel-appdev
_______________________________________________
Monetdb-developers mailing list
Monetdb-developers@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/monetdb-developers
