Re: [swift-corelibs-dev] Performance testing via SwiftPM and XCTest

2016-07-29 Thread Brian Gesiak via swift-corelibs-dev
I received some feedback from Paulo Faria that I’d like to share here: Is there a way to change the performance test baselines by code? I can’t find a way to do it today. So my question would be, why can’t we set the baselines by code? Is it because we need a different baseline for each different

Re: [swift-corelibs-dev] Performance testing via SwiftPM and XCTest

2016-07-26 Thread Brian Gesiak via swift-corelibs-dev
Hi, it’s me again. :) I figured out how we can do this on Darwin: by using .xctestconfiguration files. SwiftPM performance tests on Darwin, using XCTestConfiguration files Xcode passes all sorts of variables to XCTest by specifying the XCTestConfigurationFilePath=/path/to/an.xctestconfiguration p

Re: [swift-corelibs-dev] Performance testing via SwiftPM and XCTest

2016-07-25 Thread Brian Gesiak via swift-corelibs-dev
I received some feedback on this proposal from Ankit Aggarwal, which centered on how developers would edit and update their baseline metrics. Here’s what I’m envisioning specifically: Two new command-line options for swift test 1. swift test --performance-metrics . This is a path to a direct