On Wed, May 29, 2019 at 11:59 PM Mojca Miklavec <mo...@macports.org> wrote:
> (CC-ing Arjun as parts of this might be of interest for the app as well, > and the general mailing list as they might provide further feedback.) > > On Wed, 29 May 2019 at 08:57, Rajdeep Bharati <rajdeepbharat...@gmail.com> > wrote: > >> Hi, >> I wanted to be sure about one thing: >> In view #2 >> Screenshot 2019-05-29 at 12.21.13 PM.png >> <https://drive.google.com/file/d/118N91qeOPFXpBHuQs5Bo085uKuOF0i4_/view?usp=drive_web> >> do you want me to put information about >> 1) only the builds >> 2) builds as well as commits >> I assume that the table (as shown in the screenshot) would have info of >> the >> builds and their results on every port and OS version. >> > > For anyone reading the mailing list, we are talking about > https://trac.macports.org/ticket/55978 > > I would primarily like to see the commits (as well as potentially forced > builds, but those with a somewhat lower priority; forced builds could be > discussed later; as I understand one cannot even trigger builds at the same > time on multiple builders without modifying configuration). > > The builds by themselves are hardly useful. My primary goal for this is as > follows. Let's say that I know that I changed the python37 port in commit > X. Two months later I want to know on which workers the build failed and on > which ones it succeeded. With the existing build.macports.org this info > is literally impossible to find unless you write a bot that fetches json > data from the builds as far back as you can get (maybe doing some bisection > or so). And once you find the relevant build 13584 for Mac OS X 10.6, you > need to repeat the exercise for 10.7, 10.8, ... 10.14 all from scratch. > > With the new buildbot the console view partially does what I would like to > see. Take a look at > https://nine.buildbot.net/#/console > If you ignore the bug with "unknow revision x-y" and scroll down to > something like a pull request or an actual commit, you can extend the view > and get something like this: > > [image: image.png] > > We would like to be able to browse commits (that's why we talked about > "calendar widget"), click on a commit of interest and get to a page with > information for that commit (group) only. Mostly because there will be so > much information that a table like this might be simply too crowdy, but > also because it would be nice to have a link that can be shared, I'm not > sure if you can share a link to the screenshot I sent above. Having a > single page is not a strict requirement though. With some folding it could > be ok to have it implemented in a similar way as on the above screenshot. > > The screenshot above already contains a lot of information that we want to > have: > > - author name & timestamp are already there > - (committer name & timestamp are already on your short-term TODO list) > - commit message is already there > - link to github is already there > - (link to trac tickets could be extracted from commit message; it's > already in URL form if present, it's just a matter of making it active, > i.e. adding "a href" around it) > - (link to the PR that resulted in that commit might need to be added; > or maybe it's there already, but I cannot see it on this view anyway) > - (I forgot to mention, but probably the list of files modified would > be useful as well, and maybe it's there already, at least it says "No > files" in the view above.) > > The console also shows a nice view of the results across different > builders. > > What's special about the way ports are built for MacPorts is that one > commit (or rather: one build potentially encompassing multiple subsequent > commits) triggers the builds of an arbitrary number of ports. And the > information about whether that build in total (one single job on port > watcher) passed or not is useless if it is not split into multiple ports. > We want to make some kind of equivant of the above console view, but the > view needs to contain a table of all the ports that were built as part of > that "port watcher" job or "port builders". > > How do we decide which ports are supposed to be built? Is it only the ports (files) that are affected by a particular `change`? Also, I guess it would be better to have a separate view for each `change`. One option would be implementing it as a modal in view #1 (for each change). (I'll check if there's a way to create link to modal in Vue). Otherwise it can be a different route. As an intermezzo I want to explain one thing. Initially we had one single > job (which could run for days), and it built all the ports in a single job. > However one could not infer any information about what worked and what > failed without downloading logs first, which could have reached hundreds of > megabytes each. We wanted to rewrite it with a custom number of steps, so > that a build of 100 ports would contain 100 steps (probably more like a few > times hundred; with "clean", "build dependencies", "build port", "upload > port", ... for each of the ports individually). The reason for creating > port watcher and port builder as separate entities was used as a workaround > for lack of functionality in buildbot 0.8, which is no longer the case in > the latest buildbot. So maybe it's even time to revise this decision and > figure out if there's a better way to organize port watcher/builder into a > single builder. > > Assume that I made a change to py-django port. Which ports should be built in the following situations: 1. Version of py-django is updated (no other changes) 2. Version of py-django is updated, new dependency added 3. Version of py-django is updated, dependencies' ports also updated Does any port *depending on* py-django also be built in any of these situations? Which dependencies are built along with py-django? One problem with current setup is that it may happen that one presses the > "rebuild" button and starts building a port on the portbuilder at the same > time as portindex is being regenerated, or macports is being updated in the > portwatcher. These two actions should not happen at the same time. Another > problem is that, say, a perl version increase may result in rebuilding a > few thousand ports. Buildbot 0.8 usually choked and became uselessly slow > (with builders idling for as long as an hour just to get the next job > assigned). Before that rewrite, when everything ran as a single job, we > would sometimes schedule builds of all ports. With current setup it's more > of a mission impossible to imagine that. I have no idea how buildbot 2.x > can handle builds with tens of thousands of steps. > > If you look at a random build in current setup, like > > https://build.macports.org/builders/ports-10.12_x86_64-builder/builds/92478 > for example, it tells you which commit(s) it belongs to, but you cannot > ask the buildbot to show you all builds belonging to those commits. Also, > if you then go to the portwatcher, say, > > https://build.macports.org/builders/ports-10.12_x86_64-watcher/builds/23943 > you don't even know on link in the list to find a particular port. > > Mojca > >