Eelco Chaudron <[email protected]> writes: > On 28 Sep 2024, at 15:03, Aaron Conole wrote: > >> The 0-day robot has been testing patches for 6 years, and we've >> had support for other labs to integrate for 3. However, this >> isn't well documented, and has made it difficult for others to >> know how they can contribute to testing. To that end, this >> patch introduces some documentation for the 0-day robot and >> how to integrate into the patch reporting process. > > Thanks for adding this documentation, some small comments below. > > Cheers, > > Eelco > >> Signed-off-by: Aaron Conole <[email protected]> >> --- >> .../contributing/submitting-patches.rst | 3 +- >> Documentation/topics/testing.rst | 105 ++++++++++++++++++ >> 2 files changed, 107 insertions(+), 1 deletion(-) >> >> diff --git >> a/Documentation/internals/contributing/submitting-patches.rst >> b/Documentation/internals/contributing/submitting-patches.rst >> index 8a8bc11b0a..c01ac7bbdc 100644 >> --- a/Documentation/internals/contributing/submitting-patches.rst >> +++ b/Documentation/internals/contributing/submitting-patches.rst >> @@ -70,7 +70,8 @@ Testing is also important: >> >> If you are using GitHub, then you may utilize the GitHub Actions CI build >> systems. They will run some of the above tests automatically >> -when you push changes to your repository. >> +when you push changes to your repository. See the "Continuous Integration" >> +section in :doc:`/topics/testing` for details on continuous integration. >> >> Email Subject >> ------------- >> diff --git a/Documentation/topics/testing.rst >> b/Documentation/topics/testing.rst >> index dcf10a4db2..49ee7d7ffa 100644 >> --- a/Documentation/topics/testing.rst >> +++ b/Documentation/topics/testing.rst >> @@ -545,3 +545,108 @@ Once you are done with experimenting you can >> tear down setup with:: >> >> Sometimes deployment of Proof of Concept may fail, if, for example, VMs >> don't have network reachability to the Internet. >> + >> + >> +Continuous Integration >> +---------------------- >> + >> +The Open vSwitch project can make use of multiple public and hosted >> +CI services to help developers ensure their patches don't introduce >> +additional regressions. Each service requires different forms of >> +configuration, and for the supported services the configuration >> +file(s) to automatically build Open vSwitch with various build >> +configurations and run the testsuites is/are provided in the > > I think test suites are two words.
I guess it could be either. From what I see in the documentation / comments, we use 'testsuite' more often than 'test suite'. I can switch it in v2. >> +repostiory. For example, the GitHub Actions builds will be performed >> +with gcc, clang, sparse, including DPDK, etc. with the -Werror >> +compiler flag included. Therefore, the build will fail if a new >> +warning gets introduced by a change. >> + >> +Each ci system uses a different method of enablement, but most are available > > CI with captials as in the rest of the text? Ack. >> +from the GitHub settings page. By default, Open vSwitch includes a GitHub >> +actions running configuration, and this will automatically email >> the repository >> +owner. >> + >> +Currently, Open vSwitch project enables the following public CI >> services along >> +with the appropriate configuration settings:: >> + >> + - AppVeyor: appveyor.yml >> + - Cirrus-CI: .cirrus.yml >> + - GitHub Actions: .github/workflows >> + >> +GitHub Actions is available without any additional configuration. >> + >> +Additionally, as each patch is posted to the mailing list, the public CI >> +machinery will run additional tests on the patches and report results. >> + >> +Public report / Private lab hybrid testing >> +------------------------------------------ >> + >> +The Open vSwitch project makes use of the ozlabs patchwork instance > > Maybe put in a link to the instance? Good idea. >> +to track patch status and management. This patch tracking tool >> +provides information to maintainers on the state of patches proposed >> +for Open vSwitch. The CI process for Open vSwitch makes use of the >> +checks feature of the ozlabs patchwork instance. These allow developers > > OzLabs > >> +and maintainers to see what tests have been run, and report pass / fail >> +criteria. >> + >> +In order to know that a patch or series is ready for testing, the >> +Open vSwitch project makes use of the "0-day Robot", which is a hosted >> +jenkins instance running the pw-ci_ scripts. These can monitor a > > Jenkins > >> +running patchwork instance for new patches and submit the patch details >> +to other build systems, like jenkins. >> + >> +Once a patch is tested, it would be good to report the results. To this >> +end, the Open vSwitch "0-day Robot" will accept emails sent to >> [email protected] formatted in the correct way to be reflected >> +on this page. This allows any lab to contribute to the testing and >> +validation of patches. Note that the ovs-build list participation >> +requires subscribing the reporting email account to the list. >> + >> +To report a test status to a particular patch, send exactly one email to >> +the mailing formatted as such:: >> + >> + From: your email <[email protected]> >> + To: [email protected] >> + Date: Mon, 28 Jun 2021 00:00:00 +0000 >> + Subject: |STATUS| pwPATCHID commit subject >> + >> + Test-Label: your-robot-or-test-name >> + Test-Status: STATUS >> + http://patchwork.ozlabs.org/api/patches/PATCHID/ >> + >> + _ONE LINE DESCRIPTION_ >> + >> + Addtional details >> + >> +In the above example, the STATUS is one of "success" "warning" >> "fail" depending >> +on the outcome of the test. This value is case-sensitive. PATCHID should >> be >> +the patch for which the test was executed, and ONE LINE DESCRIPTION >> should be a >> +simple one line result description, ex: "robot-test: success" >> + >> +It is strongly recommended that if the patch is reporting a failure case, >> one >> +of the patch authors is cc'd as well. Please try to keep the details of a >> +pass or fail to smallest amount that includes useful data. For example, if >> +a build log can be hosted in a publicly viewable way, then please include a >> +URL rather than the entire build log. If that isn't possible, try to parse >> +the test logs and include a small snippet with the relevent details. Always > > relevant details. > >> +also try to publish what the test is running - that can help developers to >> +recreate failures themselves. >> + >> +Example for success:: >> + >> + From: 0-Day Robot <[email protected]> >> + To: [email protected] >> + Date: Fri, 25 Jun 2021 14:30:37 +0400 >> + Subject: |success| pw1497375 [ovs-dev] [PATCH v2] checkpatch: >> Ignore macro definitions of FOR_EACH >> + >> + Test-Label: github-robot >> + Test-Status: success >> + http://patchwork.ozlabs.org/api/patches/1497375/ >> + >> + _github build: passed_ >> + Build URL: https://github.com/ovsrobot/ovs/actions/runs/972188336 >> + >> +If possible, please also try to support the ``recheck-request`` feature that >> +the "0-day Robot" currently supports. >> + >> +.. _pw-ci: https://github.com/ovsrobot/pw-ci > > > Should we explain the recheck-request feature earlier in the documentation? I can move it earlier, makes sense to me. > >> -- >> 2.46.1 >> >> _______________________________________________ >> dev mailing list >> [email protected] >> https://mail.openvswitch.org/mailman/listinfo/ovs-dev _______________________________________________ dev mailing list [email protected] https://mail.openvswitch.org/mailman/listinfo/ovs-dev
