> On Jun 12, 2017, at 4:45 PM, Pavol Vaskovic <p...@pali.sk> wrote:
>
> We really have two problems:
> 1. spurious results
> 2. the turnaround time for the entire benchmark suite
>
>
> I don't think we can get more consistent test results just from re-running
> tests that were detected as changes in the first pass, as described inSR-4669
> <https://bugs.swift.org/browse/SR-4669>, because that improves accuracy only
> for one side of the comparison - the branch. When the measurement error is
> with the baseline from the master, re-running the branch would not help.
My understanding of this feature is that it would rerun both branches (or
possibly whichever is slower or more jittery, but that’s probably over
complicating it).
-Andy
_______________________________________________
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev
