[ 
https://issues.apache.org/jira/browse/HDDS-17840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

JyunSheng Kao updated HDDS-17840:
---------------------------------
    Description: 
Follow-up to HDDS-17463, suggested by [~adoroszlai] during review 
[https://github.com/apache/ozone/pull/12054#discussion_r4580339685]
h2. Problem (Improvement)

Python helpers in {{dev-support/ci}} ({{{}junit_summary.py{}}}, 
{{{}coverage_summary.py{}}}) have unit tests, but CI runs them through 
{{.bats}} wrappers. ({{{}junit_summary.bats{}}}, {{{}coverage_summary.bats{}}})

This introduces some problems
 - Domain leak
 -- Every new Python helper needs its own {{.bats}} file, and a contributor has 
to know that
 - Swallowed test errors
 -- When a Python test fails, bats {{run}} captures the output, so the job 
summary shows only {{{}not ok 1 coverage_summary.py unit tests{}}}. The failing 
test name and the assertion diff are not visible anywhere in CI

h2. What value it brings
 - Reduce cognitive load
 -- A new Python helper only needs its {{test_*.py}} file
 -- The bats check is back to testing shell scripts only, which is what its 
name says
 - Make test errors transparent to reduce debug effort
 -- A failing Python test shows the test name and assertion diff

h2. Proposal

(Not final, better ideas welcome)
 - Introduce new {{hadoop-ozone/dev-support/checks/pyunit.sh}}
 -- Contains {{{}#checks:basic{}}}, to be triggered by 
{{set_outputs_run_everything_and_exit}}
 -- Runs 
{code:java}
python3 -m unittest discover -s dev-support/ci -p 'test_*.py'{code}
 New Python test files are picked up automatically.

 -- Writes {{summary.txt}} / {{failures}} like the other checks

 - Add {{check_needs_pyunit}} to {{{}selective_ci_checks.sh{}}}.
 - Remove {{junit_summary.bats}} and {{coverage_summary.bats}}

 
h2. Reference
 # 
[https://github.com/apache/ozone/blob/master/dev-support/ci/selective_ci_checks.sh#L87]
 # 
[https://github.com/apache/ozone/blob/master/dev-support/ci/selective_ci_checks.sh#L632L642]

  was:
Follow-up to HDDS-17463, suggested by [~adoroszlai] during review 
[https://github.com/apache/ozone/pull/12054#discussion_r4580339685]
h2. Problem (Improvement)

Python helpers in {{dev-support/ci}} ({{{}junit_summary.py{}}}, 
{{{}coverage_summary.py{}}}) have unit tests, but CI runs them through 
{{.bats}} wrappers. ({{{}junit_summary.bats{}}}, {{{}coverage_summary.bats{}}})

This introduces some problems
 - Domain leak
 -- Every new Python helper needs its own {{.bats}} file, and a contributor has 
to know that
 - Swallowed test errors
 -- When a Python test fails, bats {{run}} captures the output, so the job 
summary shows only {{{}not ok 1 coverage_summary.py unit tests{}}}. The failing 
test name and the assertion diff are not visible anywhere in CI

h2. What value it brings
 - Reduce cognitive load
 -- A new Python helper only needs its {{test_*.py}} file
 -- The bats check is back to testing shell scripts only, which is what its 
name says
 - Make test errors transparent to reduce debug effort
 -- A failing Python test shows the test name and assertion diff

h2. Proposal

(Not final, better ideas welcome)
 - Introduce new {{hadoop-ozone/dev-support/checks/pyunit.sh}}
 -- Contains {{{}#checks:basic{}}}, to be triggered by 
{{set_outputs_run_everything_and_exit}}
 -- Runs 
{code:java}
python3 -m unittest discover -s dev-support/ci -p 'test_*.py'{code}
 New Python test files are picked up automatically.

 -- Writes {{summary.txt}} / {{failures}} like the other checks
 - Add {{check_needs_pyunit}} to {{{}selective_ci_checks.sh{}}}.
 - Remove {{junit_summary.bats}} and {{coverage_summary.bats}}

 
h2. Reference
 # 
[https://github.com/apache/ozone/blob/master/dev-support/ci/selective_ci_checks.sh#L87]
 # 
[https://github.com/apache/ozone/blob/master/dev-support/ci/selective_ci_checks.sh#L632L642]


> Run Python unit tests in a dedicated CI check instead of bats
> -------------------------------------------------------------
>
>                 Key: HDDS-17840
>                 URL: https://issues.apache.org/jira/browse/HDDS-17840
>             Project: Apache Ozone
>          Issue Type: Sub-task
>          Components: CI
>            Reporter: JyunSheng Kao
>            Assignee: JyunSheng Kao
>            Priority: Minor
>              Labels: GoodForNewContributors, good-first-issue
>
> Follow-up to HDDS-17463, suggested by [~adoroszlai] during review 
> [https://github.com/apache/ozone/pull/12054#discussion_r4580339685]
> h2. Problem (Improvement)
> Python helpers in {{dev-support/ci}} ({{{}junit_summary.py{}}}, 
> {{{}coverage_summary.py{}}}) have unit tests, but CI runs them through 
> {{.bats}} wrappers. ({{{}junit_summary.bats{}}}, 
> {{{}coverage_summary.bats{}}})
> This introduces some problems
>  - Domain leak
>  -- Every new Python helper needs its own {{.bats}} file, and a contributor 
> has to know that
>  - Swallowed test errors
>  -- When a Python test fails, bats {{run}} captures the output, so the job 
> summary shows only {{{}not ok 1 coverage_summary.py unit tests{}}}. The 
> failing test name and the assertion diff are not visible anywhere in CI
> h2. What value it brings
>  - Reduce cognitive load
>  -- A new Python helper only needs its {{test_*.py}} file
>  -- The bats check is back to testing shell scripts only, which is what its 
> name says
>  - Make test errors transparent to reduce debug effort
>  -- A failing Python test shows the test name and assertion diff
> h2. Proposal
> (Not final, better ideas welcome)
>  - Introduce new {{hadoop-ozone/dev-support/checks/pyunit.sh}}
>  -- Contains {{{}#checks:basic{}}}, to be triggered by 
> {{set_outputs_run_everything_and_exit}}
>  -- Runs 
> {code:java}
> python3 -m unittest discover -s dev-support/ci -p 'test_*.py'{code}
>  New Python test files are picked up automatically.
>  -- Writes {{summary.txt}} / {{failures}} like the other checks
>  - Add {{check_needs_pyunit}} to {{{}selective_ci_checks.sh{}}}.
>  - Remove {{junit_summary.bats}} and {{coverage_summary.bats}}
>  
> h2. Reference
>  # 
> [https://github.com/apache/ozone/blob/master/dev-support/ci/selective_ci_checks.sh#L87]
>  # 
> [https://github.com/apache/ozone/blob/master/dev-support/ci/selective_ci_checks.sh#L632L642]



--
This message was sent by Atlassian Jira
(v10.3.26#10030026)

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to