On 28 Sep 2024, at 15:03, Aaron Conole wrote:
> The 0-day robot has been testing patches for 6 years, and we've
> had support for other labs to integrate for 3. However, this
> isn't well documented, and has made it difficult for others to
> know how they can contribute to testing. To that end, this
> patch introduces some documentation for the 0-day robot and
> how to integrate into the patch reporting process.
Thanks for adding this documentation, some small comments below.
Cheers,
Eelco
> Signed-off-by: Aaron Conole <[email protected]>
> ---
> .../contributing/submitting-patches.rst | 3 +-
> Documentation/topics/testing.rst | 105 ++++++++++++++++++
> 2 files changed, 107 insertions(+), 1 deletion(-)
>
> diff --git a/Documentation/internals/contributing/submitting-patches.rst
> b/Documentation/internals/contributing/submitting-patches.rst
> index 8a8bc11b0a..c01ac7bbdc 100644
> --- a/Documentation/internals/contributing/submitting-patches.rst
> +++ b/Documentation/internals/contributing/submitting-patches.rst
> @@ -70,7 +70,8 @@ Testing is also important:
>
> If you are using GitHub, then you may utilize the GitHub Actions CI build
> systems. They will run some of the above tests automatically
> -when you push changes to your repository.
> +when you push changes to your repository. See the "Continuous Integration"
> +section in :doc:`/topics/testing` for details on continuous integration.
>
> Email Subject
> -------------
> diff --git a/Documentation/topics/testing.rst
> b/Documentation/topics/testing.rst
> index dcf10a4db2..49ee7d7ffa 100644
> --- a/Documentation/topics/testing.rst
> +++ b/Documentation/topics/testing.rst
> @@ -545,3 +545,108 @@ Once you are done with experimenting you can tear down
> setup with::
>
> Sometimes deployment of Proof of Concept may fail, if, for example, VMs
> don't have network reachability to the Internet.
> +
> +
> +Continuous Integration
> +----------------------
> +
> +The Open vSwitch project can make use of multiple public and hosted
> +CI services to help developers ensure their patches don't introduce
> +additional regressions. Each service requires different forms of
> +configuration, and for the supported services the configuration
> +file(s) to automatically build Open vSwitch with various build
> +configurations and run the testsuites is/are provided in the
I think test suites are two words.
> +repostiory. For example, the GitHub Actions builds will be performed
> +with gcc, clang, sparse, including DPDK, etc. with the -Werror
> +compiler flag included. Therefore, the build will fail if a new
> +warning gets introduced by a change.
> +
> +Each ci system uses a different method of enablement, but most are available
CI with captials as in the rest of the text?
> +from the GitHub settings page. By default, Open vSwitch includes a GitHub
> +actions running configuration, and this will automatically email the
> repository
> +owner.
> +
> +Currently, Open vSwitch project enables the following public CI services
> along
> +with the appropriate configuration settings::
> +
> + - AppVeyor: appveyor.yml
> + - Cirrus-CI: .cirrus.yml
> + - GitHub Actions: .github/workflows
> +
> +GitHub Actions is available without any additional configuration.
> +
> +Additionally, as each patch is posted to the mailing list, the public CI
> +machinery will run additional tests on the patches and report results.
> +
> +Public report / Private lab hybrid testing
> +------------------------------------------
> +
> +The Open vSwitch project makes use of the ozlabs patchwork instance
Maybe put in a link to the instance?
> +to track patch status and management. This patch tracking tool
> +provides information to maintainers on the state of patches proposed
> +for Open vSwitch. The CI process for Open vSwitch makes use of the
> +checks feature of the ozlabs patchwork instance. These allow developers
OzLabs
> +and maintainers to see what tests have been run, and report pass / fail
> +criteria.
> +
> +In order to know that a patch or series is ready for testing, the
> +Open vSwitch project makes use of the "0-day Robot", which is a hosted
> +jenkins instance running the pw-ci_ scripts. These can monitor a
Jenkins
> +running patchwork instance for new patches and submit the patch details
> +to other build systems, like jenkins.
> +
> +Once a patch is tested, it would be good to report the results. To this
> +end, the Open vSwitch "0-day Robot" will accept emails sent to
> [email protected] formatted in the correct way to be reflected
> +on this page. This allows any lab to contribute to the testing and
> +validation of patches. Note that the ovs-build list participation
> +requires subscribing the reporting email account to the list.
> +
> +To report a test status to a particular patch, send exactly one email to
> +the mailing formatted as such::
> +
> + From: your email <[email protected]>
> + To: [email protected]
> + Date: Mon, 28 Jun 2021 00:00:00 +0000
> + Subject: |STATUS| pwPATCHID commit subject
> +
> + Test-Label: your-robot-or-test-name
> + Test-Status: STATUS
> + http://patchwork.ozlabs.org/api/patches/PATCHID/
> +
> + _ONE LINE DESCRIPTION_
> +
> + Addtional details
> +
> +In the above example, the STATUS is one of "success" "warning" "fail"
> depending
> +on the outcome of the test. This value is case-sensitive. PATCHID should be
> +the patch for which the test was executed, and ONE LINE DESCRIPTION should
> be a
> +simple one line result description, ex: "robot-test: success"
> +
> +It is strongly recommended that if the patch is reporting a failure case, one
> +of the patch authors is cc'd as well. Please try to keep the details of a
> +pass or fail to smallest amount that includes useful data. For example, if
> +a build log can be hosted in a publicly viewable way, then please include a
> +URL rather than the entire build log. If that isn't possible, try to parse
> +the test logs and include a small snippet with the relevent details. Always
relevant details.
> +also try to publish what the test is running - that can help developers to
> +recreate failures themselves.
> +
> +Example for success::
> +
> + From: 0-Day Robot <[email protected]>
> + To: [email protected]
> + Date: Fri, 25 Jun 2021 14:30:37 +0400
> + Subject: |success| pw1497375 [ovs-dev] [PATCH v2] checkpatch: Ignore macro
> definitions of FOR_EACH
> +
> + Test-Label: github-robot
> + Test-Status: success
> + http://patchwork.ozlabs.org/api/patches/1497375/
> +
> + _github build: passed_
> + Build URL: https://github.com/ovsrobot/ovs/actions/runs/972188336
> +
> +If possible, please also try to support the ``recheck-request`` feature that
> +the "0-day Robot" currently supports.
> +
> +.. _pw-ci: https://github.com/ovsrobot/pw-ci
Should we explain the recheck-request feature earlier in the documentation?
> --
> 2.46.1
>
> _______________________________________________
> dev mailing list
> [email protected]
> https://mail.openvswitch.org/mailman/listinfo/ovs-dev
_______________________________________________
dev mailing list
[email protected]
https://mail.openvswitch.org/mailman/listinfo/ovs-dev
