On 2/23/20 10:46 AM, Tadeus Prastowo via lfs-dev wrote:
On Sun, Feb 23, 2020 at 5:20 PM Xi Ruoyao via lfs-dev
<lfs-dev@lists.linuxfromscratch.org> wrote:
We don't consider XPASS a failure. There are many XPASS in LFS packages (>20 in
total, I think) and it does not make sense to document them one by one.
I'd never seen a XPASS which is a "bad thing". Once a grep maintainer believed
"XPASS is bad" so he made the entire test suite to FAIL if there was any XPASS.
Then after a Glibc upgrading, a XPASS in grep forced us to use "-k" for grep
"make check". It's stupid IMO - a test suite should not be a fragile "status
change detector".
Very interesting. So, XPASS here does not mean something that is
really expected to fail as in the unit test "test to open a lock with
the wrong key", which is expected to fail and with an XPASS really
means something goes wrong?
If they expected it to do one thing and it didn't it would FAIL. If
something normally fails, that is XFAIL. If it actually passes due to
an update to a library or the kernel, then it would be an XPASS.
Really the only thing to be concened about is a lot of FAILs. Quite a
few packages have some small number of failures.
-- Bruce
We can add a discussion about the meaning of PASS/FAIL/XPASS/XFAIL in "4.6 About
the Test Suites", though.
Seconded, and please include the case of "test to open a lock with the
wrong key" to define the semantics of XPASS. Thank you.
--
Xi Ruoyao <xry...@mengyan1223.wang>
School of Aerospace Science and Technology, Xidian University
--
http://lists.linuxfromscratch.org/listinfo/lfs-dev
FAQ: http://www.linuxfromscratch.org/faq/
Unsubscribe: See the above information page
