Thanks for the feedback. It appeared that I was running Mtest.py and
mserver5 from different builds which caused the different output. The
html file produced by Mdiff contained only differences in the number
of lines starting with # or =, etc.
Babis
On Tue, Jan 10, 2012 at 10:00 PM, Stefan Manegold
<stefan.maneg...@cwi.nl> wrote:
> On Tue, Jan 10, 2012 at 08:25:21PM +0100, Sjoerd Mullender wrote:
>> On 2012-01-10 19:54, Charalampos Nikolaou wrote:
>> > Hi,
>> >
>> > I have written some code and now I would like to make changes and
>> > still be sure that the new changes are correct. So, I have written
>> > some scenarios in mal and put it in a Tests directory. I have read the
>> > README file under testing/ directory, but still haven't understood how
>> > I could generate the <scenario>.stable.out file. How could that file
>> > be generated? Even if I run Mtest.py <mytest>, then Mapprove.py the
>> > previously "wrong" output, and then again Mtest.py the test fails.
>
> .. and why / how does it fail?
>
>
> from the README:
>
> - A tool Mapprove.py is provided to copy the recent output from
> $TSTTRGDIR/TST.test.{out,err} to $TSTSRCDIR/TST.stable.{out,err}. Lines
> starting with a '!' (i.e. error messages) are eliminated during copying.
> Of course, $TSTTRGDIR/TST.test.{out,err} have to be checked and claimed
> correct before being approved!
>
> Well, that's about it. After running Mtest.py, run Mapprove.py to copy the
> just generated test output from $TSTTRGDIR/TST.test.{out,err} to
> $TSTSRCDIR/TST.stable.{out,err}. As Sjoerd pointed out, by default lines
> starting with a '!', (MonetDB error messages are usually tagged like this)
> are eliminated during copying, i.e., do not end up in
> $TSTTRGDIR/TST.test.{out,err}. If your test does generate expected errors
> (or other lines starting with a '!') that need to be approved as correct
> output, use Mapprove.py's "-f" option (see also `Mapprove.py --help` for
> details) to keep these lines. Then, check (by hand --- there is now way to
> automize this) whether the just generated and approved output in
> $TSTSRCDIR/TST.stable.{out,err} is indeed correct, and fix (by hand)
> otherwise. Then, run Mtest.py, again. If Mtest.py reportes a filing test,
> check the generated html pages to the the differences in detail.
> Please be aware that Mtest.py basically uses a simple "diff" to compare
> stable (approved) output and new output, i.e., you test need to produce
> exactly the same output in a deterministic way with each run to be suitable
> for Mtest.py.
>
> Hope this helps you further.
>
> Stefan
>
>> > Thanks,
>> > Babis
>>
>> In principle that's how it works. However, if there are expected errors
>> in the output, you have to supply the -f (force) option to Mapprove.py.
>> It will normally filter error messages out.
>>
>>
>> --
>> Sjoerd Mullender
>>
>
>
>
>> ------------------------------------------------------------------------------
>> Write once. Port to many.
>> Get the SDK and tools to simplify cross-platform app development. Create
>> new or port existing apps to sell to consumers worldwide. Explore the
>> Intel AppUpSM program developer opportunity. appdeveloper.intel.com/join
>> http://p.sf.net/sfu/intel-appdev
>
>> _______________________________________________
>> Monetdb-developers mailing list
>> Monetdb-developers@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/monetdb-developers
>
>
> --
> | Stefan.Manegold @ CWI.nl | DB Architectures (INS1) |
> | http://CWI.nl/~manegold/ | Science Park 123 (L321) |
> | Tel.: +31 (0)20 592-4212 | 1098 XG Amsterdam (NL) |
>
> ------------------------------------------------------------------------------
> Write once. Port to many.
> Get the SDK and tools to simplify cross-platform app development. Create
> new or port existing apps to sell to consumers worldwide. Explore the
> Intel AppUpSM program developer opportunity. appdeveloper.intel.com/join
> http://p.sf.net/sfu/intel-appdev
> _______________________________________________
> Monetdb-developers mailing list
> Monetdb-developers@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/monetdb-developers
------------------------------------------------------------------------------
Write once. Port to many.
Get the SDK and tools to simplify cross-platform app development. Create
new or port existing apps to sell to consumers worldwide. Explore the
Intel AppUpSM program developer opportunity. appdeveloper.intel.com/join
http://p.sf.net/sfu/intel-appdev
_______________________________________________
Monetdb-developers mailing list
Monetdb-developers@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/monetdb-developers
