Re: Inconsistent criteria of success on the buildbot

[Ryan Schmidt]

> On Feb 25, 2016, at 5:50 AM, Mojca Miklavec  wrote:
> 
> On 25 February 2016 at 11:22, Joshua Root wrote:
>> On 2016-2-25 18:18 , Mojca Miklavec wrote:
>>> 
>>> Hi,
>>> 
>>> A while ago I committed a new subport for geant4.10.2 which is
>>> currently broken (in the meantime I lacked time/motivation to look
>>> into it; and nobody else complained so far either):
>>> https://trac.macports.org/ticket/50426
>>> 
>>> The initial builds failed, see for example:
>>> 
>>> https://build.macports.org/builders/buildports-mtln-x86_64/builds/27442
>>> but judging from
>>> http://packages.macports.org/geant4.10.2/
>>> it seems as if the "build all ports" picked up and uploaded the
>>> "broken" package with "no questions asked".
>>> 
>>> Shouldn't this be fixed somehow? (Not just the port, but also the
>>> behaviour.) Or did the package automagically start to work properly?
>>> 
>>> I don't dare to even start parsing the logs from the all-ports build.
>>> The progress log from the same buildslave
>>> 
>>> https://build.macports.org/builders/buildports-mtln-x86_64/builds/28037/steps/compile/logs/progress
>>> says "package found, not building again".
>> 
>> That happens if installation succeeded but activation failed, or rev-upgrade
>> failed afterwards. What behaviour are you wanting?
> 
> Good question.
> 
> I would expect:
> - from the buildbot to send me an email about the failure (so that I
> know that I have to look at it and fix it)

The buildbot does send emails on build failures, to all the maintainers of the 
ports that were built.

> - consistency in uploading: either an upload or no upload of the
> package in both build attempts
> 
> But I do realize that implementing the second might be tricky and
> might add a lot of overhead to the process.

That does sound like a reasonable request. I have not read through the code to 
see how the current buildbot configuration is implemented or how we might 
achieve this. But it seems as though the buildbot should just be told to upload 
a package if it finds it, regardless of whether activation succeeded or not. 
That would at least be consistent. I would also like for packages to be 
uploaded immediately after they're created, and not in a separate phase after 
all ports are built. That would help packages get into users' hands quicker.


>> For it to uninstall the
>> port in this case?
>> 
>> I agree it's not really ideal, but it does speed things up for users as
>> archives are intended to do, even though in this case what they end up with
>> is broken either way. And you'll need a version or revision bump to fix it,
>> so uninstalling and attempting the build again in future doesn't really
>> help.
> 
> The only thing that comes to my mind would be to flag a package when
> it fails to build for one reason or another

The buildbot does flag ports as failed or succeeded, and skips them on 
subsequent builds if they haven't changed since the prior success or failure. 
If failure, buildbot also skips ports that depend on the failed port.

> In case a port is
> flagged, the buildbot should try to at least activate the port and run
> some sanity checks (if not rebuilding the port completely) to make
> sure that the port is flawless.

The buildbot should not behave differently from user systems, since the 
buildbot is meant to be our "ideal user". If a port installs but fails to 
activate, a revbump is needed to correct the problem on user systems, so it is 
reasonable to require that to correct the problem on the buildbot as well.

> On the other hand a package might become broken after a while if one
> of its dependencies gets upgraded without revbumping the package. So
> the situation is not much better in that case.

A revbump is required to fix broken packages too, so we needn't consider that 
scenario for the buildbot.


___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: Inconsistent criteria of success on the buildbot

[Mojca Miklavec]

On 25 February 2016 at 11:22, Joshua Root wrote:
> On 2016-2-25 18:18 , Mojca Miklavec wrote:
>>
>> Hi,
>>
>> A while ago I committed a new subport for geant4.10.2 which is
>> currently broken (in the meantime I lacked time/motivation to look
>> into it; and nobody else complained so far either):
>>  https://trac.macports.org/ticket/50426
>>
>> The initial builds failed, see for example:
>>
>> https://build.macports.org/builders/buildports-mtln-x86_64/builds/27442
>> but judging from
>>  http://packages.macports.org/geant4.10.2/
>> it seems as if the "build all ports" picked up and uploaded the
>> "broken" package with "no questions asked".
>>
>> Shouldn't this be fixed somehow? (Not just the port, but also the
>> behaviour.) Or did the package automagically start to work properly?
>>
>> I don't dare to even start parsing the logs from the all-ports build.
>> The progress log from the same buildslave
>>
>> https://build.macports.org/builders/buildports-mtln-x86_64/builds/28037/steps/compile/logs/progress
>> says "package found, not building again".
>
> That happens if installation succeeded but activation failed, or rev-upgrade
> failed afterwards. What behaviour are you wanting?

Good question.

I would expect:
- from the buildbot to send me an email about the failure (so that I
know that I have to look at it and fix it)
- consistency in uploading: either an upload or no upload of the
package in both build attempts

But I do realize that implementing the second might be tricky and
might add a lot of overhead to the process.

> For it to uninstall the
> port in this case?
>
> I agree it's not really ideal, but it does speed things up for users as
> archives are intended to do, even though in this case what they end up with
> is broken either way. And you'll need a version or revision bump to fix it,
> so uninstalling and attempting the build again in future doesn't really
> help.

The only thing that comes to my mind would be to flag a package when
it fails to build for one reason or another. In case a port is
flagged, the buildbot should try to at least activate the port and run
some sanity checks (if not rebuilding the port completely) to make
sure that the port is flawless.

On the other hand a package might become broken after a while if one
of its dependencies gets upgraded without revbumping the package. So
the situation is not much better in that case.

Mojca
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev

Re: Inconsistent criteria of success on the buildbot

[Joshua Root]

On 2016-2-25 18:18 , Mojca Miklavec wrote:

Hi,

A while ago I committed a new subport for geant4.10.2 which is
currently broken (in the meantime I lacked time/motivation to look
into it; and nobody else complained so far either):
 https://trac.macports.org/ticket/50426

The initial builds failed, see for example:
 https://build.macports.org/builders/buildports-mtln-x86_64/builds/27442
but judging from
 http://packages.macports.org/geant4.10.2/
it seems as if the "build all ports" picked up and uploaded the
"broken" package with "no questions asked".

Shouldn't this be fixed somehow? (Not just the port, but also the
behaviour.) Or did the package automagically start to work properly?

I don't dare to even start parsing the logs from the all-ports build.
The progress log from the same buildslave
 
https://build.macports.org/builders/buildports-mtln-x86_64/builds/28037/steps/compile/logs/progress
says "package found, not building again".


That happens if installation succeeded but activation failed, or 
rev-upgrade failed afterwards. What behaviour are you wanting? For it to 
uninstall the port in this case?


I agree it's not really ideal, but it does speed things up for users as 
archives are intended to do, even though in this case what they end up 
with is broken either way. And you'll need a version or revision bump to 
fix it, so uninstalling and attempting the build again in future doesn't 
really help.


- Josh
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev