RE: Upporting status

2017-07-03 Thread rleigh

On 2017-07-03 14:32, Cantor, Scott wrote:

You might find https://issues.apache.org/jira/browse/XERCESC-2104 of
interest.  This replaces sanityTest.pl with separate automake checks.
You can still run "make check", but it now shows you each individual
test being run and stores the logs in separate files.  This makes it
much clearer what's breaking.


I don't know what XFAIL means but the tests all pass for me on Windows
and Linux at the moment.


XFAIL is "expected fail".  Some of the tests expect a nonzero exit 
status; mainly just tests which display usage information when run with 
no options.  It's treated the same as PASS in that it's not counted as 
an error.  If they ever pass unexpectedly then you'll get an XPASS which 
is treated as an error.  These are XFAIL_TESTS in Makefile.am, and 
EXPECT_FAIL in CMakeLists.txt.



Regards,
Roger


-
To unsubscribe, e-mail: c-dev-unsubscr...@xerces.apache.org
For additional commands, e-mail: c-dev-h...@xerces.apache.org



RE: Upporting status

2017-07-03 Thread rleigh

On 2017-07-03 14:55, Cantor, Scott wrote:

Roger, is that separate fork updated with the master copy? It looks
like maybe it's missing the bug fixes I checked in Friday after you
let me know they were failing. That would certainly explain it.

The link issue was just a dangling reference to 3_1 in the ICU build
from the version change, which I probably should see if we can get
indirected into a macro.

I'm not familiar with Travis yet but I interpreted the output here [1]
as successful (the #9 link).


Yes, thanks.  After that fix went in, everything passed across the 
board.


-
To unsubscribe, e-mail: c-dev-unsubscr...@xerces.apache.org
For additional commands, e-mail: c-dev-h...@xerces.apache.org



RE: Upporting status

2017-07-03 Thread Cantor, Scott
Roger, is that separate fork updated with the master copy? It looks like maybe 
it's missing the bug fixes I checked in Friday after you let me know they were 
failing. That would certainly explain it.

The link issue was just a dangling reference to 3_1 in the ICU build from the 
version change, which I probably should see if we can get indirected into a 
macro.

I'm not familiar with Travis yet but I interpreted the output here [1] as 
successful (the #9 link).

-- Scott

[1] https://travis-ci.org/apache/xerces-c/builds

-
To unsubscribe, e-mail: c-dev-unsubscr...@xerces.apache.org
For additional commands, e-mail: c-dev-h...@xerces.apache.org



RE: Upporting status

2017-07-03 Thread Cantor, Scott
> You might find https://issues.apache.org/jira/browse/XERCESC-2104 of
> interest.  This replaces sanityTest.pl with separate automake checks.
> You can still run "make check", but it now shows you each individual
> test being run and stores the logs in separate files.  This makes it
> much clearer what's breaking.

I don't know what XFAIL means but the tests all pass for me on Windows and 
Linux at the moment.

-- Scott



-
To unsubscribe, e-mail: c-dev-unsubscr...@xerces.apache.org
For additional commands, e-mail: c-dev-h...@xerces.apache.org



Re: Upporting status

2017-07-03 Thread rleigh

On 2017-06-29 20:25, Roger Leigh wrote:

On 29/06/17 20:17, Cantor, Scott wrote:

On 6/29/17, 3:02 PM, "Roger Leigh"  wrote:


The recent trunk changes broke a few of the unit tests.


I don't understand how, other than the ones that are for some reason 
depending on the output of the parameter options for the DOMCount 
sample. That seems like an odd test, but it certainly would have 
broken.


Since nothing should be actually invoking the sample with the new 
option, I don't know why a new error would be raised in the tests, it 
defaults to behaving the same as before, allowing DTDs.



Do you need a hand looking at fixing any of these bits?


I don't know any of the tests or even how to run them, so probably.

If the tests can be run locally with an autotools build, I haven't 
found that trick yet.


It's "scripts/sanityTest.pl", a Perl script which runs all the tests,
concatenates their output, and then diffs it with the expected output.
It fails if the output differs or the tests fail prematurely.


You might find https://issues.apache.org/jira/browse/XERCESC-2104 of 
interest.  This replaces sanityTest.pl with separate automake checks.  
You can still run "make check", but it now shows you each individual 
test being run and stores the logs in separate files.  This makes it 
much clearer what's breaking.


You can see this working in 
https://travis-ci.org/rleigh-codelibre/xerces-c/builds/249590170 #1 #5 
and #7.



Regards,
Roger


-
To unsubscribe, e-mail: c-dev-unsubscr...@xerces.apache.org
For additional commands, e-mail: c-dev-h...@xerces.apache.org



Re: Upporting status

2017-06-29 Thread Roger Leigh

On 29/06/17 20:47, Cantor, Scott wrote:

On 6/29/17, 3:25 PM, "Roger Leigh"  wrote:


It's "scripts/sanityTest.pl", a Perl script which runs all the tests,
concatenates their output, and then diffs it with the expected output.
It fails if the output differs or the tests fail prematurely.


Well, I've run that before, and now, and it never works, so I'm still missing 
the trick, that's why I never bothered with them, assumed they were all broken. 
Is there some special directory to be in? Files in place?

Doesn't work on a Mac maybe?


It's broken on (IIRC) FreeBSD and MacOS X.

The reason is a bit odd.  These platforms are inserting an extra newline 
at the end of the output when writing out an XML document (again, IIRC, 
it's a year since I looked in detail).  I think the CMake normalisation 
of the output to make diff work on Windows (stripping CRLFs) also strips 
off that last newline, but it would be nice to figure out why it's 
different on these platforms since it's likely an oversight somewhere, 
but I've not yet figured out where or why it's happening.



Regards,
Roger

-
To unsubscribe, e-mail: c-dev-unsubscr...@xerces.apache.org
For additional commands, e-mail: c-dev-h...@xerces.apache.org



Re: Upporting status

2017-06-29 Thread Cantor, Scott
Ack, never mind, PEBKAC, they're running now.

-- Scott

On 6/29/17, 3:49 PM, "Cantor, Scott"  wrote:

On 6/29/17, 3:46 PM, "Roger Leigh"  wrote:

> Actually, just run "make check" which builds the tests and runs them foryou.

Not for me unfortunately.

export 
PATH=/Users/scantor/Documents/workspace/xerces-3.2/samples:/Users/scantor/Documents/workspace/xerces-3.2/tests:"/opt/local/bin:/opt/local/sbin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/X11/bin:/usr/local/MacGPG2/bin:/Library/Frameworks/Mono.framework/Versions/Current/Commands:/opt/local/bin"
 && export XERCESC_NLS_HOME=/Users/scantor/Documents/workspace/xerces-3.2/src/ 
&& cd . && perl scripts/sanityTest.pl 2>&1 | /usr/bin/sed 's/ *[0-9][0-9]*  *ms 
*/{timing removed}/' 1> 
/Users/scantor/Documents/workspace/xerces-3.2/test-results.log
/bin/sh: /Users/scantor/Documents/workspace/xerces-3.2/test-results.log: No 
such file or directory
make: *** [check] Error 1

-- Scott







Re: Upporting status

2017-06-29 Thread Cantor, Scott
On 6/29/17, 3:46 PM, "Roger Leigh"  wrote:

> Actually, just run "make check" which builds the tests and runs them foryou.

Not for me unfortunately.

export 
PATH=/Users/scantor/Documents/workspace/xerces-3.2/samples:/Users/scantor/Documents/workspace/xerces-3.2/tests:"/opt/local/bin:/opt/local/sbin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/X11/bin:/usr/local/MacGPG2/bin:/Library/Frameworks/Mono.framework/Versions/Current/Commands:/opt/local/bin"
 && export XERCESC_NLS_HOME=/Users/scantor/Documents/workspace/xerces-3.2/src/ 
&& cd . && perl scripts/sanityTest.pl 2>&1 | /usr/bin/sed 's/ *[0-9][0-9]*  *ms 
*/{timing removed}/' 1> 
/Users/scantor/Documents/workspace/xerces-3.2/test-results.log
/bin/sh: /Users/scantor/Documents/workspace/xerces-3.2/test-results.log: No 
such file or directory
make: *** [check] Error 1

-- Scott





Re: Upporting status

2017-06-29 Thread Cantor, Scott
On 6/29/17, 3:25 PM, "Roger Leigh"  wrote:

> It's "scripts/sanityTest.pl", a Perl script which runs all the tests, 
> concatenates their output, and then diffs it with the expected output. 
> It fails if the output differs or the tests fail prematurely.

Well, I've run that before, and now, and it never works, so I'm still missing 
the trick, that's why I never bothered with them, assumed they were all broken. 
Is there some special directory to be in? Files in place?

Doesn't work on a Mac maybe?

I'll keep playing.

-- Scott




Re: Upporting status

2017-06-29 Thread Roger Leigh



On 29/06/17 20:25, Roger Leigh wrote:

On 29/06/17 20:17, Cantor, Scott wrote:

On 6/29/17, 3:02 PM, "Roger Leigh"  wrote:


The recent trunk changes broke a few of the unit tests.


I don't understand how, other than the ones that are for some reason 
depending on the output of the parameter options for the DOMCount 
sample. That seems like an odd test, but it certainly would have broken.


Since nothing should be actually invoking the sample with the new 
option, I don't know why a new error would be raised in the tests, it 
defaults to behaving the same as before, allowing DTDs.



Do you need a hand looking at fixing any of these bits?


I don't know any of the tests or even how to run them, so probably.

If the tests can be run locally with an autotools build, I haven't 
found that trick yet.


It's "scripts/sanityTest.pl", a Perl script which runs all the tests, 
concatenates their output, and then diffs it with the expected output. 
It fails if the output differs or the tests fail prematurely.


Actually, just run "make check" which builds the tests and runs them for 
you.


-
To unsubscribe, e-mail: c-dev-unsubscr...@xerces.apache.org
For additional commands, e-mail: c-dev-h...@xerces.apache.org



Re: Upporting status

2017-06-29 Thread Cantor, Scott
On 6/29/17, 3:31 PM, "Roger Leigh"  wrote:

> This is because the unit test is comparing the tool help output line by 
> line and it's simply due to an extra line being added to the help 
> output.  It's not a fault of the change, it's just that the test data 
> needs updating to keep in sync with the change.

I get that one, but the invalid document structure message I think one of them 
mentioned is what the new option reports if you have a DTD. Since the test 
should still be allowing DTDs, I don't know why that would be getting raised. 
Could be one of the tests actually sets the DOM option that before wasn't 
implemented, I'll have to see.

Thanks for the tips on running them.

-- Scott



-
To unsubscribe, e-mail: c-dev-unsubscr...@xerces.apache.org
For additional commands, e-mail: c-dev-h...@xerces.apache.org



Re: Upporting status

2017-06-29 Thread Roger Leigh



On 29/06/17 20:17, Cantor, Scott wrote:

On 6/29/17, 3:02 PM, "Roger Leigh"  wrote:


The recent trunk changes broke a few of the unit tests.


I don't understand how, other than the ones that are for some reason depending 
on the output of the parameter options for the DOMCount sample. That seems like 
an odd test, but it certainly would have broken.

Since nothing should be actually invoking the sample with the new option, I 
don't know why a new error would be raised in the tests, it defaults to 
behaving the same as before, allowing DTDs.


This is because the unit test is comparing the tool help output line by 
line and it's simply due to an extra line being added to the help 
output.  It's not a fault of the change, it's just that the test data 
needs updating to keep in sync with the change.



-
To unsubscribe, e-mail: c-dev-unsubscr...@xerces.apache.org
For additional commands, e-mail: c-dev-h...@xerces.apache.org



Re: Upporting status

2017-06-29 Thread Roger Leigh

On 29/06/17 20:17, Cantor, Scott wrote:

On 6/29/17, 3:02 PM, "Roger Leigh"  wrote:


The recent trunk changes broke a few of the unit tests.


I don't understand how, other than the ones that are for some reason depending 
on the output of the parameter options for the DOMCount sample. That seems like 
an odd test, but it certainly would have broken.

Since nothing should be actually invoking the sample with the new option, I 
don't know why a new error would be raised in the tests, it defaults to 
behaving the same as before, allowing DTDs.


Do you need a hand looking at fixing any of these bits?


I don't know any of the tests or even how to run them, so probably.

If the tests can be run locally with an autotools build, I haven't found that 
trick yet.


It's "scripts/sanityTest.pl", a Perl script which runs all the tests, 
concatenates their output, and then diffs it with the expected output. 
It fails if the output differs or the tests fail prematurely.


or "ctest [-V]" for CMake, which is exactly the same, but runs the tests 
individually without concatenating the output (so you can run them 
individually).


You can run either.  CTest is more flexible (you can run individual 
tests) and it tells you which specific tests failed.  The Perl script 
just gives you a diff and you have to work out what failed yourself.


The DOMCount test is definitely just an output difference issue.  But 
the others are exiting nonzero or segfaulting.




Regards,
Roger

-
To unsubscribe, e-mail: c-dev-unsubscr...@xerces.apache.org
For additional commands, e-mail: c-dev-h...@xerces.apache.org



Re: Upporting status

2017-06-29 Thread Cantor, Scott
On 6/29/17, 3:02 PM, "Roger Leigh"  wrote:

> The recent trunk changes broke a few of the unit tests.

I don't understand how, other than the ones that are for some reason depending 
on the output of the parameter options for the DOMCount sample. That seems like 
an odd test, but it certainly would have broken.

Since nothing should be actually invoking the sample with the new option, I 
don't know why a new error would be raised in the tests, it defaults to 
behaving the same as before, allowing DTDs.

>Do you need a hand looking at fixing any of these bits?

I don't know any of the tests or even how to run them, so probably.

If the tests can be run locally with an autotools build, I haven't found that 
trick yet.

-- Scott




Re: Upporting status

2017-06-29 Thread Roger Leigh

On 22/06/17 19:23, Cantor, Scott wrote:

I've ported essentially all code-related changes and a decent amount of the web 
site changes from the 3.1 branch back up to trunk.

At least one of the original security fixes to the branch apparently caused a regression, 
which I wasn't surprised by. I think there's a separate bug open on that, plus the 
additional security concerns with the bad casts that needs fixing, so the rest is 
"new work" to get a release done, please at least one new feature setting I 
will be adding (the disable DTDs option).

Hope to get most of this done over the next few weeks at the latest.

I haven't pulled any of the old Windows solutions yet, but that's TBD. I'm 
using the cmake-generated version to actually build and test anything.


Hi Scott,

The recent trunk changes broke a few of the unit tests.

https://travis-ci.org/apache/xerces-c/builds/247945962?utm_source=github_status_medium=notification
https://travis-ci.org/apache/xerces-c/builds/247945962?utm_source=github_status_medium=notification

Test failure summary:

- DOMCount (all platforms)
- InitTermTest1 (linux, mingw)
- PParse1 (macos)
- ThreadTest1-9 (cygwin, mingw, msvc)
- ThreadTest2 (macos, cygwin)
- Link error (looks ICU-related)
(and others on Windows with VS2015: DOMTest, XSerializerTest[12], 
MemHandlerTest1, DOMTypeInfoTest, SAX2?Count[12], SAX2?Print[12], 
MemParse1?, Redirect1, DOMPrint[1235], StdInParse[12], EnumVal1)


DOMCount looks like it simply needs the added option adding to the 
expected output for both the perl test script and the cmake expected 
text file


The link error looks like it's missing a symbol.

The thread test shows up on all Windows compilers, but also shows up on 
MacOS X.


Not yet looked in detail at the other errors.

Do you need a hand looking at fixing any of these bits?


Regards,
Roger


A summary of the failures follows:

Autotools/Linux:

XERCESC_NLS_HOME=/home/travis/build/apache/xerces-c/autoconf-build/src/ 
&& cd .. && perl scripts/sanityTest.pl 2>&1 | /bin/sed 's/ *[0-9][0-9]* 
*ms */{timing removed}/' 1> 
/home/travis/build/apache/xerces-c/autoconf-build/test-results.log

diff test-results.log ../scripts/sanityTest_ExpectedResult.log
623d622
< -d  Disallow DOCTYPE. Defaults to false.
make: *** [check] Error 1

If the option was added to the output, the expected test data likely 
needs updating.


and

XERCESC_NLS_HOME=/home/travis/build/apache/xerces-c/autoconf-build/src/ 
&& cd .. && perl scripts/sanityTest.pl 2>&1 | /bin/sed 's/ *[0-9][0-9]* 
*ms */{timing removed}/' 1> 
/home/travis/build/apache/xerces-c/autoconf-build/test-results.log

diff test-results.log ../scripts/sanityTest_ExpectedResult.log
623d622
< -d  Disallow DOCTYPE. Defaults to false.
1147,1153c1146
< 1
< Fatal Error at file 
/home/travis/build/apache/xerces-c/samples/data/personal.xml, line 2, 
char 10

<   Message:  invalid document structure
<
< Errors occurred, no output available
<
< Test Failed
---
> 1Test Run Successfully

This one looks like an error?

Autotools/MacOS X:

XERCESC_NLS_HOME=/Users/travis/build/apache/xerces-c/autoconf-build/src/ 
&& cd .. && perl scripts/sanityTest.pl 2>&1 | /usr/bin/sed 's/ 
*[0-9][0-9]*  *ms */{timing removed}/' 1> 
/Users/travis/build/apache/xerces-c/autoconf-build/test-results.log

diff test-results.log ../scripts/sanityTest_ExpectedResult.log
623d622
< -d  Disallow DOCTYPE. Defaults to false.
1172,1177c1171
< 3 during parsing: personal.xml
<  Exception message is: invalid document structure
<
< Thread 2: Parse Check sum error on file  "personal.xml" for parse # 2. 
 Expected 7352cd96,  got 0

< Total number of parses completed is 22.00.
 3Test Run Successfully
1183,1185c1177
< 9 during parsing: personal.xml
<  Exception message is: invalid document structure
< Test Run Successfully
---
> 9Test Run Successfully
1220a1213
> Test Run Successfully

There are also some stray "9" symbols in here.  Is that intentional?

CMake/Linux:

Linking CXX executable XSTSHarness
../src/libxerces-c-3.2.so: undefined reference to `xercesc_messages_3_1_dat'
collect2: error: ld returned 1 exit status

Build failure.

58: Test command: /home/travis/build/apache/xerces-c/tools/bin/cmake 
"-DNAME=DOMCount" 
"-DPROGRAM=/home/travis/build/apache/xerces-c/cmake-build/samples/DOMCount" 
"-DARGS=" 
"-DLIBXERCES_C=/home/travis/build/apache/xerces-c/cmake-build/src/libxerces-c-3.2.so" 
"-DWORKDIR=/home/travis/build/apache/xerces-c/samples/data" "-DSTDIN=" 
"-DEXPECT_FAIL=TRUE" 
"-DOBSERVED_DIR=/home/travis/build/apache/xerces-c/cmake-build/samples/observed" 
"-DEXPECTED_DIR=/home/travis/build/apache/xerces-c/samples/expected" 
"-DDIFF=/usr/bin/diff" 
"-DNLS_HOME=/home/travis/build/apache/xerces-c/cmake-build/src" "-P" 
"/home/travis/build/apache/xerces-c/cmake/RunTest.cmake"

58: Test timeout computed to be: 9.99988e+06
58: -- Running 
/home/travis/build/apache/xerces-c/cmake-build/samples/DOMCount
58: ---