[
https://issues.apache.org/jira/browse/BEAM-7772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lukasz Gajowy updated BEAM-7772:
--------------------------------
Issue Type: Improvement (was: Bug)
> Stop using Perfkit Benchmarker tool in all tests
> ------------------------------------------------
>
> Key: BEAM-7772
> URL: https://issues.apache.org/jira/browse/BEAM-7772
> Project: Beam
> Issue Type: Improvement
> Components: testing
> Reporter: Lukasz Gajowy
> Assignee: Lukasz Gajowy
> Priority: Major
>
> [Devlist thread
> link|https://lists.apache.org/thread.html/dab1c093799248787e8b75e63b66d7389b594b649a4d9a4a5db1cfbb@%3Cdev.beam.apache.org%3E]
>
> Currently Python, IOIT and some Dataflow and Spark performance tests are
> relying on Perfkit Benchmarker tool. Due to the reasons discussed on the
> devlist it was decided to remove it from Beam's tests.
> Problems that we face currently:
> # Changes to Gradle tasks/build configuration in the Beam codebase have to
> be reflected in Perfkit code. This required PRs to Perfkit which can last and
> the tests break due to this sometimes (no change in Perfkit + change already
> there in beam = incompatibility). This is what happened in PR 8919 (above),
> # Can't run in Python3 (depends on python 2 only library like functools32),
> # Black box testing which hard to collect pipeline related metrics,
> # Measurement of run time is inaccurate,
> # It offers relatively small elasticity in comparison with eg. Jenkins tasks
> in terms of setting up the testing infrastructure (runners, databases). For
> example, if we'd like to setup Flink runner, and reuse it in consequent tests
> in one go, that would be impossible. We can easily do this in Jenkins.
> Tests that use Perfkit:
> # IO integration tests,
> # Python performance tests,
> # beam_PerformanceTests_Dataflow (disabled),
> # beam_PerformanceTests_Spark (failing constantly - looks not maintained).
>
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
