[
https://issues.apache.org/jira/browse/BEAM-14334?focusedWorklogId=764869&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-764869
]
ASF GitHub Bot logged work on BEAM-14334:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 02/May/22 07:41
Start Date: 02/May/22 07:41
Worklog Time Spent: 10m
Work Description: echauchot commented on PR #17406:
URL: https://github.com/apache/beam/pull/17406#issuecomment-1114582816
> Usage of `forkEvery 1` is typically a strong sign of poor quality / bad
code and should be avoided:
>
> * It often hides resource leaks, either in code or worse, potentially in
the runner itself.
> * It significantly impacts performance when running tests.
>
> This PR introduces new JUnit rules to better manage the lifecyle of
`SparkContext` or `SparkContextOptions` and allows to safely reuse those for
all tests of a suite where applicable. The previous `ReuseSparkContextRule`
leaks the SparkContext and was removed. Besides, with `forkEvery 1` trying to
reuse the `SparkContext` is absolutely pointless ...
>
> There's might be more issues hidden I didn't discover locally ... I'll
keep a look.
>
> This also reduce the runtime of tests to ~30 secs from 1:30+ mins.
>
> Thank you for your contribution! Follow this checklist to help us
incorporate your contribution quickly and easily:
>
> * [ ] [**Choose
reviewer(s)**](https://beam.apache.org/contribute/#make-your-change) and
mention them in a comment (`R: @username`).
> * [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in
ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA
issue, if applicable. This will automatically link the pull request to the
issue.
> * [ ] Update `CHANGES.md` with noteworthy changes.
> * [ ] If this contribution is large, please file an Apache [Individual
Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
>
> See the [Contributor Guide](https://beam.apache.org/contribute) for more
tips on [how to make review process
smoother](https://beam.apache.org/contribute/#make-reviewers-job-easier).
>
> To check the build health, please visit
https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md
>
> ## GitHub Actions Tests Status (on master branch)
> [](https://github.com/apache/beam/actions?query=workflow%3A%22Build+python+source+distribution+and+wheels%22+branch%3Amaster+event%3Aschedule)
[](https://github.com/apache/beam/actions?query=workflow%3A%22Python+Tests%22+branch%3Amaster+event%3Aschedule)
[](https://github.com/apache/beam/actions?query=workflow%3A%22Java+Tests%22+branch%3Amaster+event%3Aschedule)
>
> See [CI.md](https://github.com/apache/beam/blob/master/CI.md) for more
information about GitHub Actions CI.
Bad smell indeed ! Thanks for catching this @mosche and reducing the test
time
Issue Time Tracking
-------------------
Worklog Id: (was: 764869)
Time Spent: 1.5h (was: 1h 20m)
> Avoid using forkEvery in Spark runner tests
> -------------------------------------------
>
> Key: BEAM-14334
> URL: https://issues.apache.org/jira/browse/BEAM-14334
> Project: Beam
> Issue Type: Improvement
> Components: runner-spark, testing
> Reporter: Moritz Mack
> Assignee: Moritz Mack
> Priority: P2
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> Usage of *{color:#FF0000}forkEvery 1{color}* is typically a strong sign of
> poor quality / bad code and should be avoided:
> * It significantly impacts performance when running tests.
> * And it often hides resource leaks, either in code or worse in the runner
> itself.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
