This is an automated email from the ASF dual-hosted git repository. akm pushed a commit to branch improve-documentation-486 in repository https://gitbox.apache.org/repos/asf/tooling-trusted-releases.git
commit 559986ef36bd22d49d9bbff9f6a8b1b3ac0e2e6d Author: Andrew K. Musselman <[email protected]> AuthorDate: Wed Jan 21 18:40:54 2026 -0800 Fixes #486 --- CONTRIBUTING.md | 46 ++++++++++++---- DEVELOPMENT.md | 2 +- atr/docs/running-and-creating-tests.md | 96 ++++++++++++++++++++++++++++++++++ 3 files changed, 132 insertions(+), 12 deletions(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 1d0ae8b..2b8bcdb 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -22,8 +22,12 @@ For detailed ASF policies, commit message guidelines, and security consideration ```shell git clone https://github.com/YOUR_USERNAME/tooling-trusted-releases.git cd tooling-trusted-releases + git remote add upstream https://github.com/apache/tooling-trusted-releases.git + git config pull.rebase true ``` + This configures `upstream` to point to the Apache repository and sets git to rebase (rather than merge) when pulling, which keeps your commit history clean. + 2. **Install dependencies** (includes pre-commit, dev tools, and test dependencies): ```shell @@ -52,27 +56,41 @@ For detailed ASF policies, commit message guidelines, and security consideration 2. **Make your changes** following our [code conventions](https://release-test.apache.org/docs/code-conventions) -3. **Run checks** before committing: +3. **Run checks and tests** before committing: ```shell - make check + make check # Required: lints and type checks + sh tests/run-e2e.sh # Required: end-to-end tests + sh tests/run-unit.sh # Required: unit tests ``` + All checks and tests must pass locally before submitting. + 4. **Commit** with a clear message (see [commit style](#commit-message-style) below) -5. **Push** your branch: +5. **Rebase on main** before pushing: + + ```shell + git fetch upstream + git rebase upstream/main + ``` + +6. **Push** your branch: ```shell git push origin your-branch-name ``` -6. **Open a pull request** to the `main` branch - - Explain what your changes do and why +7. **Open a pull request** to the `main` branch + - Fill out the PR template completely, confirming all required acknowledgements - Reference any related issues (e.g., "Fixes #123") - - Mark as draft until ready for review + - **Open as Draft** until all checks pass and you have confirmed local testing - **Enable "Allow maintainer edits"** (strongly recommended) + - Convert from Draft to ready for review only after all acknowledgements are confirmed + +8. **Participate in review** - we may request changes -7. **Participate in review** - we may request changes +PRs that fail to demonstrate proper local testing or do not complete the PR template may be closed. ## Commit message style @@ -114,13 +132,19 @@ See the [full code conventions](https://release-test.apache.org/docs/code-conven ## Running tests ```shell -# Browser tests (requires Docker) -sh tests/run-playwright.sh +# Full pre-commit checks (required before submitting PR) +make check -# End-to-end tests +# End-to-end tests (required before submitting PR) sh tests/run-e2e.sh -# Quick pre-commit checks +# Unit tests (required before submitting PR) +sh tests/run-unit.sh + +# Browser tests (requires Docker) +sh tests/run-playwright.sh + +# Quick pre-commit checks (for rapid iteration) make check-light ``` diff --git a/DEVELOPMENT.md b/DEVELOPMENT.md index 7bf4946..b8297dd 100644 --- a/DEVELOPMENT.md +++ b/DEVELOPMENT.md @@ -4,7 +4,7 @@ This guide will help you get started with developing ATR. For detailed technical ## Prerequisites -ATR can be developed on **Linux** or **macOS**. Windows and other platforms are not supported. +ATR can be developed on **Linux** or **macOS**. Windows and other platforms are not supported. This is because ATR's build scripts and test harnesses use POSIX shell, Docker volume mounts behave differently on Windows, and several tools (mkcert, cmark) have platform-specific considerations. **Required (install manually):** diff --git a/atr/docs/running-and-creating-tests.md b/atr/docs/running-and-creating-tests.md index 6b9956c..f50abd2 100644 --- a/atr/docs/running-and-creating-tests.md +++ b/atr/docs/running-and-creating-tests.md @@ -11,6 +11,7 @@ * [Running Playwright tests](#running-playwright-tests) * [Creating Playwright tests](#creating-playwright-tests) * [Running end-to-end tests](#running-end-to-end-tests) +* [Creating end-to-end tests](#creating-end-to-end-tests) ## Running Playwright tests @@ -69,6 +70,16 @@ The actual test cases themselves tend to use helpers such as [`go_to_path`](/ref To run ATR end-to-end (e2e) tests, you must first have an OCI container runtime with Compose functionality, such as Docker or Podman, installed. You will also need a POSIX shell. You can then run `tests/run-e2e.sh` to run the entire e2e test suite. +### Running unit tests + +Unit tests can be run separately from e2e tests: + +```shell +sh tests/run-unit.sh +``` + +Unit tests are located in `tests/unit/` and test individual functions without requiring a running ATR instance. + ### Debugging e2e test failures When e2e tests fail, the test script will display suggestions for debugging. You can also use the following techniques: @@ -139,3 +150,88 @@ cd tests && docker compose down -v --rmi all ``` You probably only need to do this if you're running out of disk space. + +## Creating end-to-end tests + +The e2e tests use [pytest](https://docs.pytest.org/) with the [pytest-playwright](https://playwright.dev/python/docs/pytest) plugin. Tests are organized in `tests/e2e/` by feature area. + +### Test structure + +Each feature area has its own directory containing: + +* `conftest.py` - Pytest fixtures for test setup +* `test_*.py` - Test files containing test functions + +For example, the `tests/e2e/root/` directory contains tests for unauthenticated pages: + +```text +tests/e2e/root/ +├── conftest.py # Fixtures like page_index, page_policies +└── test_get.py # Tests using those fixtures +``` + +### Writing fixtures + +Fixtures set up the state needed for tests. They are defined in `conftest.py` files using the `@pytest.fixture` decorator. Fixtures can be scoped to control how often they run: + +* `scope="function"` (default) - Runs for each test function +* `scope="module"` - Runs once per test module +* `scope="session"` - Runs once per test session + +Here is an example fixture that creates a page navigated to the index: + +```python +import pytest +from playwright.sync_api import Page +import e2e.helpers as helpers + [email protected] +def page_index(page: Page): + helpers.visit(page, "/") + yield page +``` + +For tests that require authenticated state or complex setup, use module-scoped fixtures to avoid repeating expensive operations: + +```python [email protected](scope="module") +def compose_context(browser: Browser): + context = browser.new_context(ignore_https_errors=True) + page = context.new_page() + helpers.log_in(page) + # ... set up state ... + yield context + context.close() +``` + +### Helper functions + +The `tests/e2e/helpers.py` module provides common utilities: + +* `visit(page, path)` - Navigate to a path and wait for load +* `log_in(page)` - Log in using the test login endpoint +* `delete_release_if_exists(page, project, version)` - Clean up test releases +* `api_get(request, path)` - Make API requests + +### Writing tests + +Test functions receive fixtures as arguments. Use Playwright's `expect` for assertions: + +```python +from playwright.sync_api import Page, expect + +def test_index_has_login_button(page_index: Page): + login_button = page_index.get_by_role("button", name="Log in") + expect(login_button).to_be_visible() +``` + +### Adding a new test area + +To add tests for a new feature: + +1. Create a directory: `tests/e2e/myfeature/` +2. Add `__init__.py` (can be empty, but include the license header) +3. Add `conftest.py` with fixtures for your feature +4. Add `test_*.py` files with your tests + +Tests run in the order pytest discovers them. If your tests depend on state created by other tests, consider using module-scoped fixtures to manage that state explicitly. --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
