Pip has a --cache-dir which should be safe with concurrent writes. On Fri, Apr 26, 2019 at 3:59 PM Ahmet Altay <al...@google.com> wrote:
> It is possible to download dependencies with pip to a local directory and > install from there [1]. As a side benefit this is supposed to speed up the > installation process. Since we setup virtualenv multiple times, this could > actually help us in a single run. And if we can keep this cache across test > runs we can reduce flakiness. > > [1] > https://pip.pypa.io/en/latest/user_guide/#installing-from-local-packages > > On Fri, Apr 26, 2019 at 3:42 PM Valentyn Tymofieiev <valen...@google.com> > wrote: > >> We do retry certain inherently flaky tests, for example, see[1]. This >> practice should be used with caution, see discussion [2]. >> >> However retrying an individual test would not avoid the flake that Alex >> brought up in this thread, we'd have to retry setupVirtualEnv task that is >> executed once per suite of tests. Retrying just that task is different from >> retrying the whole suite. >> >> [1] >> https://github.com/apache/beam/blob/516cdb6401d9fb7adb004de472771fb1fb3a92af/sdks/python/apache_beam/runners/worker/statesampler_test.py#L41, >> this was discussed >> [2] >> https://lists.apache.org/thread.html/16060fd7f4d408857a5e4a2598cc96ebac0f744b65bf4699001350af@%3Cdev.beam.apache.org%3E >> discussed >> >> On Fri, Apr 26, 2019 at 3:30 PM Udi Meiri <eh...@google.com> wrote: >> >>> Alex, I changed my mind: I'm okay retrying single tests, just not entire >>> suites of tests (e.g. if precommits take an hour, retrying the run takes up >>> an additional hour on the Jenkins machine). >>> This is more of an issue in Python, where gradle does not (currently) >>> have insight into which tests failed and how to retry just them. >>> >>> >>> >>> On Fri, Apr 26, 2019 at 2:17 PM Alex Amato <ajam...@google.com> wrote: >>> >>>> @Udi Meiri <eh...@google.com>, Is this true if the specific tests are >>>> rerun? I don't think we should rerun all tests. >>>> >>>> On Fri, Apr 26, 2019 at 12:11 PM Valentyn Tymofieiev < >>>> valen...@google.com> wrote: >>>> >>>>> Preinstalling dependencies may affect the dependency resolution, and >>>>> we may end up testing a different configuration than a user would have >>>>> after installing beam into a clean environment. >>>>> >>>>> I do think pip uses cache, unless one specifies "--no-cache-dir". By >>>>> default the cache is ~/.cache/pip. Looking up the log message in OP, we >>>>> can >>>>> see several "Using cached..." log entries. Not sure why futures was not >>>>> fetched from cache or PyPi. Perhaps it is also a pip flake. >>>>> >>>>> I would be against wiping flakes under the rug by rerunning the whole >>>>> suite after an error, but re-rerunning parts of the test environment set >>>>> up, that are prone to environmental flakes, such as setupVirtualEnv seems >>>>> reasonable. I agree with Udi that care should be taken to not overload >>>>> Jenkins (e.g. retries should be limited) >>>>> >>>>
smime.p7s
Description: S/MIME Cryptographic Signature
