[
https://issues.apache.org/jira/browse/BEAM-7772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Beam JIRA Bot updated BEAM-7772:
--------------------------------
Labels: stale-P2 (was: )
> Stop using Perfkit Benchmarker tool in all tests
> ------------------------------------------------
>
> Key: BEAM-7772
> URL: https://issues.apache.org/jira/browse/BEAM-7772
> Project: Beam
> Issue Type: Improvement
> Components: testing
> Reporter: Lukasz Gajowy
> Priority: P2
> Labels: stale-P2
> Time Spent: 6h 50m
> Remaining Estimate: 0h
>
> [Devlist thread
> link|https://lists.apache.org/thread.html/dab1c093799248787e8b75e63b66d7389b594b649a4d9a4a5db1cfbb@%3Cdev.beam.apache.org%3E]
>
> Currently Python, IOIT and some Dataflow and Spark performance tests are
> relying on Perfkit Benchmarker tool. Due to the reasons discussed on the
> devlist it was decided to remove it from Beam's tests.
> Problems that we face currently:
> # Changes to Gradle tasks/build configuration in the Beam codebase have to
> be reflected in Perfkit code. This required PRs to Perfkit which can last and
> the tests break due to this sometimes (no change in Perfkit + change already
> there in beam = incompatibility). This is what happened in PR 8919 (above),
> # Can't run in Python3 (depends on python 2 only library like functools32),
> # Black box testing which hard to collect pipeline related metrics,
> # Measurement of run time is inaccurate,
> # It offers relatively small elasticity in comparison with eg. Jenkins tasks
> in terms of setting up the testing infrastructure (runners, databases). For
> example, if we'd like to setup Flink runner, and reuse it in consequent tests
> in one go, that would be impossible. We can easily do this in Jenkins.
> Tests that use Perfkit:
> # IO integration tests,
> # Python performance tests,
> # beam_PerformanceTests_Dataflow (disabled),
> # beam_PerformanceTests_Spark (failing constantly - looks not maintained).
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
