Re: [gentoo-dev] [PATCH] New pre-GLEP: TEST_SUITE_PRESENT variable

2023-02-24 Thread Michał Górny
On Sun, 2023-02-19 at 18:32 +0100, Michał Górny wrote:
> +Abstract
> +
> +
> +A new ``TEST_SUITE_PRESENT`` variable is introduced to indicate whether
> +the package features a test suite.  It can be set either by the ebuild,
> +the eclass or the default ``src_test`` implementation, and afterwards
> +included in the package manager logs.  This can aid in analyzing
> +the results of automated package testing.
> 

I've decided to withdraw this proposal.  After giving it more thought,
it doesn't really solve the problem it was intended to solve, and it is
unnecessarily complex for a non-solution.

Long story short, a package may use Python AND have a test suite, yet
not test anything actually written in Python.  The proposal would
qualify these packages as having the test suite, yet in the end it
wouldn't provide useful input for PYTHON_COMPAT testing.

I think a domain-specific solution such as checking for python_test() or
distutils_enable_tests will work better here, and other packages will
still require manual verification.

-- 
Best regards,
Michał Górny




Re: [gentoo-dev] [PATCH] New pre-GLEP: TEST_SUITE_PRESENT variable

2023-02-20 Thread Alec Warner
On Sun, Feb 19, 2023 at 3:11 PM Maciej Barć  wrote:
>
> What if developer configured an ebuild in a way that it downloads the
> test suite/files/data with USE=test?
>
> IMO it should be added to the GLEP that then TEST_SUITE_PRESENT should
> be true (exists).

This is what I was afraid of with the name; it's not only about a
testsuite being there or not, it's about trusting that:
 - There is a test suite.
 - It was run.
 - It passed, and passing is valuable.

Just existing is not sufficient, because we cannot gain any
information from an src_test that just downloaded but didn't run the
suite. That is not valuable information.

-A

>
> W dniu 19.02.2023 o 18:32, Michał Górny pisze:
> > Signed-off-by: Michał Górny 
> > ---
> >   glep-.ebuild | 132 +++
> >   1 file changed, 132 insertions(+)
> >   create mode 100644 glep-.ebuild
> >
> > diff --git a/glep-.ebuild b/glep-.ebuild
> > new file mode 100644
> > index 000..9ee18ca
> > --- /dev/null
> > +++ b/glep-.ebuild
> > @@ -0,0 +1,132 @@
> > +---
> > +GLEP: 
> > +Title: TEST_SUITE_PRESENT variable
> > +Author: Michał Górny 
> > +Type: Standards Track
> > +Status: Draft
> > +Version: 1
> > +Created: 2023-02-19
> > +Last-Modified: 2023-02-19
> > +Post-History: 2023-02-19
> > +Content-Type: text/x-rst
> > +---
> > +
> > +
> > +Abstract
> > +
> > +
> > +A new ``TEST_SUITE_PRESENT`` variable is introduced to indicate whether
> > +the package features a test suite.  It can be set either by the ebuild,
> > +the eclass or the default ``src_test`` implementation, and afterwards
> > +included in the package manager logs.  This can aid in analyzing
> > +the results of automated package testing.
> > +
> > +
> > +Motivation
> > +==
> > +
> > +The deployment of new Python targets in Gentoo currently involves
> > +testing of a large number of Gentoo packages against the said target.
> > +This is currently done manually for the most part.  It can be
> > +particularly time consuming if multiple individuals repeatedly test
> > +the same package only to determine that it remains incompatible with
> > +the new interpreter.
> > +
> > +The Python team wanted to explore the use of automation to aid this
> > +testing.  Unfortunately, this faces a major problem: for the vast
> > +of majority of packages, the incompatibilities with new Python versions
> > +do not exhibit during the installation and can only be detected through
> > +running the test suite.  The results of automated testing are therefore
> > +only meaningful if the package features a test phase.
> > +
> > +For packages using ``distutils-r1`` eclass, the presence of test suite
> > +can usually be easily determined through grepping for
> > +``distutils_enable_tests`` call or an explicit ``python_test()``
> > +function.  Even then, it seems sensible to work towards a more generic
> > +approach to tell whether a package had a test suite or not,
> > +and therefore whether a particular successful automated testing result
> > +means that the package actually passed tests or only confirmed that
> > +the Python files were copied successfully.
> > +
> > +An explicit indication whether a test suite was present can be presented
> > +by the package manager as part of logs, along with the result of running
> > +the test phase.  Afterwards, these logs can be used to determine which
> > +packages were actually tested.
> > +
> > +
> > +Specification
> > +=
> > +
> > +A new ``TEST_SUITE_PRESENT`` variable is introduced that can be set
> > +by a ``src_test()`` implementation to indicate whether the package
> > +featured a test suite.  It can take three values:
> > +
> > +- ``yes`` indicating that a test suite was run
> > +- ``indeterminate`` indicating that it was not possible to clearly
> > +  determine whether the test suite was present or not (this could be
> > +  a case e.g. when a generic test command is run and it does not
> > +  indicate whether any tests were found)
> > +- ``no`` indicating that no test suite was run
> > +
> > +This variable *should* be set by eclasses defining the ``src_test()``
> > +phase.  If the package in question is using ``src_test()`` defined
> > +by an eclass that does not declare it explicitly, the PM must assume
> > +``indeterminate``.
> > +
> > +The variable *may* be set by an ebuild defining the ``src_test()``
> > +phase.  If the ebuild does not define it explicitly, the PM must assume
> > +``yes``.
> > +
> > +The default ``src_test()`` implementation as defined by the PMS sets
> > +the value to ``indeterminate`` if it runs a ``check`` or ``test``
> > +target, and to ``no`` if neither of the targets is found.
> > +
> > +
> > +Rationale
> > +=
> > +
> > +The use of ternary flag makes it possible to clearly represent all three
> > +possible outcomes while navigating the defaults defined in the GLEP.
> > +The flag is set in ``src_test()``, so that runtime conditions (such
> > +as the results 

Re: [gentoo-dev] [PATCH] New pre-GLEP: TEST_SUITE_PRESENT variable

2023-02-20 Thread Michał Górny
On Mon, 2023-02-20 at 12:42 +0100, Ulrich Mueller wrote:
> How about using a new token in PROPERTIES instead?

I don't see how you could alter PROPERTIES at runtime, or how you'd
express three possible outcomes with a binary flag.

-- 
Best regards,
Michał Górny




Re: [gentoo-dev] [PATCH] New pre-GLEP: TEST_SUITE_PRESENT variable

2023-02-20 Thread Ulrich Mueller
How about using a new token in PROPERTIES instead?


signature.asc
Description: PGP signature


Re: [gentoo-dev] [PATCH] New pre-GLEP: TEST_SUITE_PRESENT variable

2023-02-20 Thread Sam James


> On 20 Feb 2023, at 08:37, Florian Schmaus  wrote:
> 
> On 19/02/2023 18.32, Michał Górny wrote:
>> +Abstract
>> +
>> +
>> +A new ``TEST_SUITE_PRESENT`` variable is introduced to indicate whether
>> +the package features a test suite.
> I wonder if we could not simply use IUSE="test" for this purpose? That is, 
> allow declaring the 'test' use flag, even though it is not used in a USE 
> conditional, to indicate that the package has a test suite.
> 
This might make the confusion wrt "USE=test should enable running tests" worse, 
as users will see every package with tests has it.


signature.asc
Description: Message signed with OpenPGP


Re: [gentoo-dev] [PATCH] New pre-GLEP: TEST_SUITE_PRESENT variable

2023-02-20 Thread Florian Schmaus

On 19/02/2023 18.32, Michał Górny wrote:

+Abstract
+
+
+A new ``TEST_SUITE_PRESENT`` variable is introduced to indicate whether
+the package features a test suite.
I wonder if we could not simply use IUSE="test" for this purpose? That 
is, allow declaring the 'test' use flag, even though it is not used in a 
USE conditional, to indicate that the package has a test suite.


With that said, I don't have any strong objections against this 
proposal. But I'd like to mention that with RESTRICT="test", IUSE="test" 
and then TEST_SUITE_PRESENT="true", we end up with an awful lot of test 
related metadata and complexity, which potentially contributes to confusion.


- Flow



Re: [gentoo-dev] [PATCH] New pre-GLEP: TEST_SUITE_PRESENT variable

2023-02-19 Thread Michał Górny
On Mon, 2023-02-20 at 00:10 +0100, Maciej Barć wrote:
> What if developer configured an ebuild in a way that it downloads the 
> test suite/files/data with USE=test?
> 
> IMO it should be added to the GLEP that then TEST_SUITE_PRESENT should 
> be true (exists).
> 

How does that change anything?  If tests were run, it's a "yes". 
Otherwise, it's "no".

-- 
Best regards,
Michał Górny




Re: [gentoo-dev] [PATCH] New pre-GLEP: TEST_SUITE_PRESENT variable

2023-02-19 Thread Maciej Barć
What if developer configured an ebuild in a way that it downloads the 
test suite/files/data with USE=test?


IMO it should be added to the GLEP that then TEST_SUITE_PRESENT should 
be true (exists).


W dniu 19.02.2023 o 18:32, Michał Górny pisze:

Signed-off-by: Michał Górny 
---
  glep-.ebuild | 132 +++
  1 file changed, 132 insertions(+)
  create mode 100644 glep-.ebuild

diff --git a/glep-.ebuild b/glep-.ebuild
new file mode 100644
index 000..9ee18ca
--- /dev/null
+++ b/glep-.ebuild
@@ -0,0 +1,132 @@
+---
+GLEP: 
+Title: TEST_SUITE_PRESENT variable
+Author: Michał Górny 
+Type: Standards Track
+Status: Draft
+Version: 1
+Created: 2023-02-19
+Last-Modified: 2023-02-19
+Post-History: 2023-02-19
+Content-Type: text/x-rst
+---
+
+
+Abstract
+
+
+A new ``TEST_SUITE_PRESENT`` variable is introduced to indicate whether
+the package features a test suite.  It can be set either by the ebuild,
+the eclass or the default ``src_test`` implementation, and afterwards
+included in the package manager logs.  This can aid in analyzing
+the results of automated package testing.
+
+
+Motivation
+==
+
+The deployment of new Python targets in Gentoo currently involves
+testing of a large number of Gentoo packages against the said target.
+This is currently done manually for the most part.  It can be
+particularly time consuming if multiple individuals repeatedly test
+the same package only to determine that it remains incompatible with
+the new interpreter.
+
+The Python team wanted to explore the use of automation to aid this
+testing.  Unfortunately, this faces a major problem: for the vast
+of majority of packages, the incompatibilities with new Python versions
+do not exhibit during the installation and can only be detected through
+running the test suite.  The results of automated testing are therefore
+only meaningful if the package features a test phase.
+
+For packages using ``distutils-r1`` eclass, the presence of test suite
+can usually be easily determined through grepping for
+``distutils_enable_tests`` call or an explicit ``python_test()``
+function.  Even then, it seems sensible to work towards a more generic
+approach to tell whether a package had a test suite or not,
+and therefore whether a particular successful automated testing result
+means that the package actually passed tests or only confirmed that
+the Python files were copied successfully.
+
+An explicit indication whether a test suite was present can be presented
+by the package manager as part of logs, along with the result of running
+the test phase.  Afterwards, these logs can be used to determine which
+packages were actually tested.
+
+
+Specification
+=
+
+A new ``TEST_SUITE_PRESENT`` variable is introduced that can be set
+by a ``src_test()`` implementation to indicate whether the package
+featured a test suite.  It can take three values:
+
+- ``yes`` indicating that a test suite was run
+- ``indeterminate`` indicating that it was not possible to clearly
+  determine whether the test suite was present or not (this could be
+  a case e.g. when a generic test command is run and it does not
+  indicate whether any tests were found)
+- ``no`` indicating that no test suite was run
+
+This variable *should* be set by eclasses defining the ``src_test()``
+phase.  If the package in question is using ``src_test()`` defined
+by an eclass that does not declare it explicitly, the PM must assume
+``indeterminate``.
+
+The variable *may* be set by an ebuild defining the ``src_test()``
+phase.  If the ebuild does not define it explicitly, the PM must assume
+``yes``.
+
+The default ``src_test()`` implementation as defined by the PMS sets
+the value to ``indeterminate`` if it runs a ``check`` or ``test``
+target, and to ``no`` if neither of the targets is found.
+
+
+Rationale
+=
+
+The use of ternary flag makes it possible to clearly represent all three
+possible outcomes while navigating the defaults defined in the GLEP.
+The flag is set in ``src_test()``, so that runtime conditions (such
+as the results obtained from the actual test runner) can be used to
+determine the actual value.
+
+The defaults were defined based on the following assumptions:
+
+1. The presence of ``check`` target is common in autotools projects but
+   it does not guarantee that the target actually does anything, let
+   alone run a proper test suite.  However, the lack of any test target
+   clearly indicates that no tests were run.
+
+2. Eclass ``src_test`` implementations can be very generic and succeed
+   without actually performing any testing.  It is therefore reasonable
+   to default to ``indeterminate`` result when they are used,
+   and recommend them to explicitly override the variable.
+
+3. Explicit ``src_test`` declared in ebuild can generally be assumed
+   to actually run tests, with the exception of declaring the function
+   to prevent 

Re: [gentoo-dev] [PATCH] New pre-GLEP: TEST_SUITE_PRESENT variable

2023-02-19 Thread Alec Warner
On Sun, Feb 19, 2023 at 9:32 AM Michał Górny  wrote:
>
> Signed-off-by: Michał Górny 
> ---
>  glep-.ebuild | 132 +++
>  1 file changed, 132 insertions(+)
>  create mode 100644 glep-.ebuild
>
> diff --git a/glep-.ebuild b/glep-.ebuild
> new file mode 100644
> index 000..9ee18ca
> --- /dev/null
> +++ b/glep-.ebuild
> @@ -0,0 +1,132 @@
> +---
> +GLEP: 
> +Title: TEST_SUITE_PRESENT variable
> +Author: Michał Górny 
> +Type: Standards Track
> +Status: Draft
> +Version: 1
> +Created: 2023-02-19
> +Last-Modified: 2023-02-19
> +Post-History: 2023-02-19
> +Content-Type: text/x-rst
> +---
> +
> +
> +Abstract
> +
> +
> +A new ``TEST_SUITE_PRESENT`` variable is introduced to indicate whether
> +the package features a test suite.  It can be set either by the ebuild,
> +the eclass or the default ``src_test`` implementation, and afterwards
> +included in the package manager logs.  This can aid in analyzing
> +the results of automated package testing.
> +
> +
> +Motivation
> +==
> +
> +The deployment of new Python targets in Gentoo currently involves
> +testing of a large number of Gentoo packages against the said target.
> +This is currently done manually for the most part.  It can be
> +particularly time consuming if multiple individuals repeatedly test
> +the same package only to determine that it remains incompatible with
> +the new interpreter.
> +
> +The Python team wanted to explore the use of automation to aid this
> +testing.  Unfortunately, this faces a major problem: for the vast
> +of majority of packages, the incompatibilities with new Python versions
> +do not exhibit during the installation and can only be detected through
> +running the test suite.  The results of automated testing are therefore
> +only meaningful if the package features a test phase.
> +
> +For packages using ``distutils-r1`` eclass, the presence of test suite
> +can usually be easily determined through grepping for
> +``distutils_enable_tests`` call or an explicit ``python_test()``
> +function.  Even then, it seems sensible to work towards a more generic
> +approach to tell whether a package had a test suite or not,
> +and therefore whether a particular successful automated testing result
> +means that the package actually passed tests or only confirmed that
> +the Python files were copied successfully.
> +
> +An explicit indication whether a test suite was present can be presented
> +by the package manager as part of logs, along with the result of running
> +the test phase.  Afterwards, these logs can be used to determine which
> +packages were actually tested.

So for my own edification:
 A package has no test suite: src_test always returns true; and the
output is useless for automation purposes.
 A package has a test suite: src_test may return true; and the output
is useless because it's not distinguishable from the first case.
 A package has a test suite: src_test returns false (because the tests
failed.) This is currently the case where all the value out of
src_test is gained for automation.

Here this proposal intends to distinguish between the first two cases;
by basically annotating packages so that if src_test passes and
TEST_SUITE_DETECTED=true, we can take a positive inference; and if
TEST_SUITE_DETECTED=no or indeterminate, we cannot take an inference
either way.

If so +1 to this.

-A

> +
> +
> +Specification
> +=
> +
> +A new ``TEST_SUITE_PRESENT`` variable is introduced that can be set
> +by a ``src_test()`` implementation to indicate whether the package
> +featured a test suite.  It can take three values:
> +
> +- ``yes`` indicating that a test suite was run
> +- ``indeterminate`` indicating that it was not possible to clearly
> +  determine whether the test suite was present or not (this could be
> +  a case e.g. when a generic test command is run and it does not
> +  indicate whether any tests were found)
> +- ``no`` indicating that no test suite was run
> +
> +This variable *should* be set by eclasses defining the ``src_test()``
> +phase.  If the package in question is using ``src_test()`` defined
> +by an eclass that does not declare it explicitly, the PM must assume
> +``indeterminate``.
> +
> +The variable *may* be set by an ebuild defining the ``src_test()``
> +phase.  If the ebuild does not define it explicitly, the PM must assume
> +``yes``.
> +
> +The default ``src_test()`` implementation as defined by the PMS sets
> +the value to ``indeterminate`` if it runs a ``check`` or ``test``
> +target, and to ``no`` if neither of the targets is found.
> +
> +
> +Rationale
> +=
> +
> +The use of ternary flag makes it possible to clearly represent all three
> +possible outcomes while navigating the defaults defined in the GLEP.
> +The flag is set in ``src_test()``, so that runtime conditions (such
> +as the results obtained from the actual test runner) can be used to
> +determine the actual value.
> +
> +The 

Re: [gentoo-dev] [PATCH] New pre-GLEP: TEST_SUITE_PRESENT variable

2023-02-19 Thread Arsen Arsenović
Hi,

Michał Górny  writes:

> Signed-off-by: Michał Górny 
> ---
>  glep-.ebuild | 132 +++
>  1 file changed, 132 insertions(+)
>  create mode 100644 glep-.ebuild
>
> diff --git a/glep-.ebuild b/glep-.ebuild
> new file mode 100644
> index 000..9ee18ca
> --- /dev/null
> +++ b/glep-.ebuild
> @@ -0,0 +1,132 @@
> +---
> +GLEP: 
> +Title: TEST_SUITE_PRESENT variable
> +Author: Michał Górny 
> +Type: Standards Track
> +Status: Draft
> +Version: 1
> +Created: 2023-02-19
> +Last-Modified: 2023-02-19
> +Post-History: 2023-02-19
> +Content-Type: text/x-rst
> +---
> +
> +
> +Abstract
> +
> +
> +A new ``TEST_SUITE_PRESENT`` variable is introduced to indicate whether
> +the package features a test suite.  It can be set either by the ebuild,
> +the eclass or the default ``src_test`` implementation, and afterwards
> +included in the package manager logs.  This can aid in analyzing
> +the results of automated package testing.
> +
> +
> +Motivation
> +==
> +
> +The deployment of new Python targets in Gentoo currently involves
> +testing of a large number of Gentoo packages against the said target.
> +This is currently done manually for the most part.  It can be
> +particularly time consuming if multiple individuals repeatedly test
> +the same package only to determine that it remains incompatible with
> +the new interpreter.
> +
> +The Python team wanted to explore the use of automation to aid this
> +testing.  Unfortunately, this faces a major problem: for the vast
> +of majority of packages, the incompatibilities with new Python versions
> +do not exhibit during the installation and can only be detected through
> +running the test suite.  The results of automated testing are therefore
> +only meaningful if the package features a test phase.
> +
> +For packages using ``distutils-r1`` eclass, the presence of test suite
> +can usually be easily determined through grepping for
> +``distutils_enable_tests`` call or an explicit ``python_test()``
> +function.  Even then, it seems sensible to work towards a more generic
> +approach to tell whether a package had a test suite or not,
> +and therefore whether a particular successful automated testing result
> +means that the package actually passed tests or only confirmed that
> +the Python files were copied successfully.
> +
> +An explicit indication whether a test suite was present can be presented
> +by the package manager as part of logs, along with the result of running
> +the test phase.  Afterwards, these logs can be used to determine which
> +packages were actually tested.
> +

+1.  I have had similar aspirations lately, so I'm glad someone else
beat me to it :)

> +
> +Specification
> +=
> +
> +A new ``TEST_SUITE_PRESENT`` variable is introduced that can be set
> +by a ``src_test()`` implementation to indicate whether the package
> +featured a test suite.  It can take three values:
> +
> +- ``yes`` indicating that a test suite was run
> +- ``indeterminate`` indicating that it was not possible to clearly
> +  determine whether the test suite was present or not (this could be
> +  a case e.g. when a generic test command is run and it does not
> +  indicate whether any tests were found)
> +- ``no`` indicating that no test suite was run
> +
> +This variable *should* be set by eclasses defining the ``src_test()``
> +phase.  If the package in question is using ``src_test()`` defined
> +by an eclass that does not declare it explicitly, the PM must assume
> +``indeterminate``.
> +
> +The variable *may* be set by an ebuild defining the ``src_test()``
> +phase.  If the ebuild does not define it explicitly, the PM must assume
> +``yes``.
> +
> +The default ``src_test()`` implementation as defined by the PMS sets
> +the value to ``indeterminate`` if it runs a ``check`` or ``test``
> +target, and to ``no`` if neither of the targets is found.
> +
> +
> +Rationale
> +=
> +
> +The use of ternary flag makes it possible to clearly represent all three
> +possible outcomes while navigating the defaults defined in the GLEP.
> +The flag is set in ``src_test()``, so that runtime conditions (such
> +as the results obtained from the actual test runner) can be used to
> +determine the actual value.
> +
> +The defaults were defined based on the following assumptions:
> +
> +1. The presence of ``check`` target is common in autotools projects but
> +   it does not guarantee that the target actually does anything, let
> +   alone run a proper test suite.  However, the lack of any test target
> +   clearly indicates that no tests were run.
> +
> +2. Eclass ``src_test`` implementations can be very generic and succeed
> +   without actually performing any testing.  It is therefore reasonable
> +   to default to ``indeterminate`` result when they are used,
> +   and recommend them to explicitly override the variable.
> +
> +3. Explicit ``src_test`` declared in ebuild can generally be assumed
> +   

Re: [gentoo-dev] [PATCH] New pre-GLEP: TEST_SUITE_PRESENT variable

2023-02-19 Thread Michał Górny
On Sun, 2023-02-19 at 22:35 +0500, Anna (cybertailor) Vyalkova wrote:
> Is it better than
> 
>   RESTRICT="test"
> 
> ?

Yes.  RESTRICT=test is only workable if everyone reliably set it on all
ebuilds not having any tests.

-- 
Best regards,
Michał Górny




Re: [gentoo-dev] [PATCH] New pre-GLEP: TEST_SUITE_PRESENT variable

2023-02-19 Thread Anna (cybertailor) Vyalkova
Is it better than

RESTRICT="test"

?



[gentoo-dev] [PATCH] New pre-GLEP: TEST_SUITE_PRESENT variable

2023-02-19 Thread Michał Górny
Signed-off-by: Michał Górny 
---
 glep-.ebuild | 132 +++
 1 file changed, 132 insertions(+)
 create mode 100644 glep-.ebuild

diff --git a/glep-.ebuild b/glep-.ebuild
new file mode 100644
index 000..9ee18ca
--- /dev/null
+++ b/glep-.ebuild
@@ -0,0 +1,132 @@
+---
+GLEP: 
+Title: TEST_SUITE_PRESENT variable
+Author: Michał Górny 
+Type: Standards Track
+Status: Draft
+Version: 1
+Created: 2023-02-19
+Last-Modified: 2023-02-19
+Post-History: 2023-02-19
+Content-Type: text/x-rst
+---
+
+
+Abstract
+
+
+A new ``TEST_SUITE_PRESENT`` variable is introduced to indicate whether
+the package features a test suite.  It can be set either by the ebuild,
+the eclass or the default ``src_test`` implementation, and afterwards
+included in the package manager logs.  This can aid in analyzing
+the results of automated package testing.
+
+
+Motivation
+==
+
+The deployment of new Python targets in Gentoo currently involves
+testing of a large number of Gentoo packages against the said target.
+This is currently done manually for the most part.  It can be
+particularly time consuming if multiple individuals repeatedly test
+the same package only to determine that it remains incompatible with
+the new interpreter.
+
+The Python team wanted to explore the use of automation to aid this
+testing.  Unfortunately, this faces a major problem: for the vast
+of majority of packages, the incompatibilities with new Python versions
+do not exhibit during the installation and can only be detected through
+running the test suite.  The results of automated testing are therefore
+only meaningful if the package features a test phase.
+
+For packages using ``distutils-r1`` eclass, the presence of test suite
+can usually be easily determined through grepping for
+``distutils_enable_tests`` call or an explicit ``python_test()``
+function.  Even then, it seems sensible to work towards a more generic
+approach to tell whether a package had a test suite or not,
+and therefore whether a particular successful automated testing result
+means that the package actually passed tests or only confirmed that
+the Python files were copied successfully.
+
+An explicit indication whether a test suite was present can be presented
+by the package manager as part of logs, along with the result of running
+the test phase.  Afterwards, these logs can be used to determine which
+packages were actually tested.
+
+
+Specification
+=
+
+A new ``TEST_SUITE_PRESENT`` variable is introduced that can be set
+by a ``src_test()`` implementation to indicate whether the package
+featured a test suite.  It can take three values:
+
+- ``yes`` indicating that a test suite was run
+- ``indeterminate`` indicating that it was not possible to clearly
+  determine whether the test suite was present or not (this could be
+  a case e.g. when a generic test command is run and it does not
+  indicate whether any tests were found)
+- ``no`` indicating that no test suite was run
+
+This variable *should* be set by eclasses defining the ``src_test()``
+phase.  If the package in question is using ``src_test()`` defined
+by an eclass that does not declare it explicitly, the PM must assume
+``indeterminate``.
+
+The variable *may* be set by an ebuild defining the ``src_test()``
+phase.  If the ebuild does not define it explicitly, the PM must assume
+``yes``.
+
+The default ``src_test()`` implementation as defined by the PMS sets
+the value to ``indeterminate`` if it runs a ``check`` or ``test``
+target, and to ``no`` if neither of the targets is found.
+
+
+Rationale
+=
+
+The use of ternary flag makes it possible to clearly represent all three
+possible outcomes while navigating the defaults defined in the GLEP.
+The flag is set in ``src_test()``, so that runtime conditions (such
+as the results obtained from the actual test runner) can be used to
+determine the actual value.
+
+The defaults were defined based on the following assumptions:
+
+1. The presence of ``check`` target is common in autotools projects but
+   it does not guarantee that the target actually does anything, let
+   alone run a proper test suite.  However, the lack of any test target
+   clearly indicates that no tests were run.
+
+2. Eclass ``src_test`` implementations can be very generic and succeed
+   without actually performing any testing.  It is therefore reasonable
+   to default to ``indeterminate`` result when they are used,
+   and recommend them to explicitly override the variable.
+
+3. Explicit ``src_test`` declared in ebuild can generally be assumed
+   to actually run tests, with the exception of declaring the function
+   to prevent ``default_src_test`` from running.  It therefore makes
+   sense to default to ``yes`` but allow ebuilds to override the value
+   in the latter case.
+
+
+Backwards Compatibility
+===
+
+This GLEP is entirely optional.  The package managers not