I came out a PR <https://github.com/apache/beam/pull/6392> which is the first step to move benchmark execution on Gradle task. Corresponding Perfkit changes <https://github.com/markflyhigh/PerfKitBenchmarker/pull/2/files> are also required.
Changes include: - new `run_integration_test.sh` script that provide better command line flags and extend more use cases. - new Gradle task `beam-sdks-python:integrationTest` that uses the new script and can be used by Perfkit to execute any integration test without worry about environment setup. Hopefully those changes can fix existing benchmark failure, but also it targets on improving user experience for running python integration tests and simplify test infrastructure. Ideas and suggestions are welcome. Mark On Sat, Sep 8, 2018 at 1:37 AM Łukasz Gajowy <[email protected]> wrote: > Oh, ok. Sorry. Thank you for the clarification! > > > pt., 7 wrz 2018 o 21:03 Mark Liu <[email protected]> napisał(a): > >> PostCommit_Python_Verify do run this command which is configured in >> sdks/python/build.gradle >> <https://github.com/apache/beam/blob/master/sdks/python/build.gradle#L192> >> and >> is a dependency of `:beam-sdks-python:postCommit` >> <https://github.com/apache/beam/blob/master/sdks/python/build.gradle#L320> >> which >> is called by Jenkins in here >> <https://github.com/apache/beam/blob/master/.test-infra/jenkins/job_PostCommit_Python_Verify.groovy#L36> >> . >> >> On Fri, Sep 7, 2018 at 7:53 AM Łukasz Gajowy <[email protected]> >> wrote: >> >>> >>>> We can also think about moving performance tests to Gradle which seems >>>> provide a stable way to setup python environment >>>> <https://github.com/apache/beam/blob/932858ece39fb2477baa19638f0a6b36fb5dee9d/sdks/python/build.gradle#L31> >>>> since recent beam_PostCommit_Python_Verify >>>> <https://builds.apache.org/view/A-D/view/Beam/job/beam_PostCommit_Python_Verify/> >>>> keeps green. >>>> >>>> >>> Correct me if I'm wrong but PostCommit_Python_Verify doesn't run "pip >>> install -e sdks/python/[gcp,test]". As this is the line that breaks the >>> build in other jobs, it may be the reason why the job doesnt fail. >>> >>
