> On Oct 25, 2017, at 7:45 PM, Mojca Miklavec <mo...@macports.org> wrote: > > On 25 October 2017 at 16:23, Ryan Schmidt wrote: >> On Oct 25, 2017, at 03:28, db wrote: >>> On 25 Oct 2017, at 07:35, Mojca Miklavec wrote: >>>> On 23 October 2017 at 10:40, db wrote: >>>>> On 9 Oct 2017, at 12:46, Mojca Miklavec wrote: >>>>>> That info is easy to read and collect from the buildbot. The keyword is: >>>>>> "to collect". >>>>> I checked https://build.macports.org/json/help, but couldn't find it. >>>> For example: >>>> - >>>> https://build.macports.org/json/builders/ports-10.13_x86_64-builder/builds/10627?as_text=1 >>>> which corresponds to >>>> - >>>> https://build.macports.org/builders/ports-10.13_x86_64-builder/builds/10627 >>>> >>>> The idea would be to write a script to iterate through all numbers >>>> from 1 on and store results somewhere. >>>> >>>> If you want the time estimate specifically, here might be the relevant >>>> information: >>>> >>>> "name": "install-port", >>>> ... >>>> "times": [ >>>> 1508901749.648058, >>>> 1508901850.387383 >>>> ] >>>> >>>> I guess the difference represents the seconds spent building the port. >>> >>> Thanks. I was searching for "to collect" there and in base, to no avail. > > "To collect" was not meant as a literate strings to search for, but as > the main action that needs to be done. > >>> Those are the start/end epochs that I need to subtract, for a single port. >>> >>> How can I correlate portname to buildnumber? > > You cannot at the moment. > > What I envision is a script that would run once from number 1 on and > then either daily or hourly to update for new builds. The script would > then construct a database of builds and do some simple visualisation. > The most important part would be to ask some simple app to return data > about one particular port. I would expect it to return the list of all > builds (including URLs) for that particular port, but most > importantly, for each buildslave: > - whether the last build succeeded or failed > - (if failed, whether it was a dependency or the port installation > that failed) > - commit timestamp / shasum & version of the port > - whatever else seems relevant, potentially including the build time > and size of the package > >> I don't think scraping information from buildbot json files is the right way >> to collect this data. If we want to collect it, the buildbot scripts could >> be modified to do so. > > I don't see any big difference. The important part would be to do it > in the first place. How to do it most efficiently can be a subject for > discussion, but parsing json files with a library seems the easiest > approach to me, in particular because we are currently unable to > modify the buildbot script to return us data about packages built in > the past. We can always modify the build scripts later. > >> But as I said before, I don't think the information is useful to collect. > > Build time doesn't seem *the most* interesting part of information, > but it is useful to know in any case. > >> Besides the fact that the buildbot worker machines have different hardware >> specs than user hardware, not all of the buildbot workers run on the same >> hardware. A build will go faster on our High Sierra worker than on the >> Sierra worker, because the High Sierra worker has a faster processor. In >> addition, the buildbot workers are virtual machines, with multiple VMs >> running on a single host. So a build may be faster or slower depending on >> whether the other VMs that run on the same host are busy or not. >> >> If all you want to do is display on a web site somewhere that a particular >> build of a port took a particular amount of time on our buildbot workers, >> ok. But I wouldn't try to use this information to, for example, modify >> MacPorts base to have it offer the user a prediction of how long the port >> will take to build on their system, because I don't think we can accurately >> predict that. > > I would definitely not use it for that purpose. But having *some > statistics* about which ports built and failed on which builder would > be super super useful. What I miss most is quickly finding the build > where a particular port failed to build. At the moment it's nearly > impossible to find it.
If we’re going to do this, can we be a little more precise about “failed” builds? 1) Any port that relies on a non-default variant “fails” on the buildbots. All of my myth* ports will ALWAYS show a “failed” status on ALL of the buildbots for this reason. I would rather that users were told that the build was “not attempted”. Telling them that build “failed" isn’t helpful. 2) Any port that needs C++11 “fails” on the existing 10.7 and earlier buildbots. 3) Various ports have a pre-fetch block that checks the OS version and “fails” if it is not supported. Assuming the above can be fixed, is the following a mock-up of the data we’d collect? Each commit to the ports tree triggers builds on the then-current fleet of builders: Port: blurfl Verson: 2.54.1_0 Commmit: https://github.com/macports/macports-ports/commit/6b120d678245ba6f14a5a364f83c3c5970f762af Buildbots: Builder Build ID Build (minutes.seconds) X86-10.13 nnnnnnnn 1.20 X86-10.12 nnnnnnnn 2.20 X86-10.11 nnnnnnnn 2.00 X86-10.10 nnnnnnnn 4.50 X86-10.9 nnnnnnnn 2.40 X86-10.8 nnnnnnnn fail X86-10.7 nnnnnnnn not supported X86-10.6 nnnnnnnn not supported PPC-10.5 nnnnnnnn not supported Craig