Re: Build Failure on ports-10.6_x86_64_legacy: libpsl, mosquitto

[Mojca Miklavec]

On 5 April 2018 at 04:25, Andrew Moore wrote:
>
> Not being familiar with buildbot, I’m curious if there’s an easy way to 
> confirm a successful build?  If I go to, for instance, 
> https://build.macports.org/console, the format looks promising, but the 
> content is limited.  Google doesn’t seem to help either, e.g., searching 
> `mosquitto site:build.macports.org’ produces no results, which begs another 
> question:  would it be possible/worthwhile to make build.macports.org 
> “search-engine optimized”?

There are three orthogonal approaches to this problem:

(a) The only one working right now: check for green checkmarks or red
crosses in commit log, for example

https://github.com/macports/macports-ports/commits/master/net/mosquitto/Portfile
The list does not get corrections for forced builds (or maybe it would
if I clicked "rebuild" instead of "force build"). It might be possible
to hack buildbot configuration to find the correct commit (maybe we
should do this anyway). If multiple ports are built in the same
commit, you'll get just the result of the latest build, I think.
There's some room for improvements here, but it's a killer feature
already :)

(b) We are looking for a front-end developer willing to look into some
react.js/vue.js black magic to talk to buildbot
https://trac.macports.org/ticket/55978

(c) As Ryan mentioned, there was a student proposal for GSOC this year
to address this exact issue in a standalone website.
(Accepted projects will be announced on 23rd April. If selected, the
work should be done by the end of the summer.)

Mojca

Re: Build Failure on ports-10.6_x86_64_legacy: libpsl, mosquitto

[Ryan Schmidt]

On Apr 4, 2018, at 21:25, Andrew Moore wrote:

> On Apr 4, 2018, at 12:18 PM, Mojca Miklavec wrote
>> It also failed on 10.7 and 10.8.
> 
> That’s more a problem with my eyes :). 
> 
>> I triggered new builds of mosquitto now that libpsl has been built on
>> all systems.
> 
> Okay, so no buildbot notifications since then.  Not being familiar with 
> buildbot, I’m curious if there’s an easy way to confirm a successful build?  
> If I go to, for instance, https://build.macports.org/console, the format 
> looks promising, but the content is limited.  Google doesn’t seem to help 
> either, e.g., searching `mosquitto site:build.macports.org’ produces no 
> results, which begs another question:  would it be possible/worthwhile to 
> make build.macports.org “search-engine optimized”?

It is a proposed Google Summer of Code project this year to create a new 
MacPorts web site with a better view onto this data.

https://trac.macports.org/wiki/SummerOfCode#build-stats

Re: Build Failure on ports-10.6_x86_64_legacy: libpsl, mosquitto

[Andrew Moore]

> On Apr 4, 2018, at 12:18 PM, Mojca Miklavec  wrote
> It also failed on 10.7 and 10.8.

That’s more a problem with my eyes :). 

> I triggered new builds of mosquitto now that libpsl has been built on
> all systems.

Okay, so no buildbot notifications since then.  Not being familiar with 
buildbot, I’m curious if there’s an easy way to confirm a successful build?  If 
I go to, for instance, https://build.macports.org/console, the format looks 
promising, but the content is limited.  Google doesn’t seem to help either, 
e.g., searching `mosquitto site:build.macports.org’ produces no results, which 
begs another question:  would it be possible/worthwhile to make 
build.macports.org “search-engine optimized”?
-AM

Re: Build Failure on ports-10.6_x86_64_legacy: libpsl, mosquitto

[Ryan Schmidt]

On Apr 4, 2018, at 11:59, Rainer Müller wrote:

> On 2018-04-04 17:19, Rainer Müller wrote:
>> Now that we execute the mirror step before attempting a build, we should
>> make the buildbot use a master_site_local pointing to a local mirror, as
>> we cannot expect the upload to be completed at the time the build runs.
>> Right now we are downloading all files twice.
> 
> I split the fetch phase into a separate command. This should make it
> easier to see that the dependency failed because it was unable to fetch
> the distfiles.
> 
> https://github.com/macports/mpbb/commit/639def5ad990ecf1e227e377e5a46124c447ca94
> 
> This will also no longer update the failcache, as fetching could be a
> temporary problem.

Thanks!


> This should also remedy the situation on legacy systems for now.
> Installation as a dependency will be retried even if a previous run
> failed due to fetching. By that time, the distfile was hopefully
> uploaded to the mirror.
> 
> We should still investigate in adding a master_site_local, as this
> involves more fetches from the internet than necessary at the moment.

The buildbot has always fetched from the local private distfiles mirror. I 
didn't know about master_site_local, so I accomplished this by having 
distfiles.macports.org, packages.macports.org and rsync.macports.org resolve to 
the local mirror's IP on the network that the build machines are on.

Re: Build Failure on ports-10.6_x86_64_legacy: libpsl, mosquitto

[Rainer Müller]

On 2018-04-04 17:19, Rainer Müller wrote:
> Now that we execute the mirror step before attempting a build, we should
> make the buildbot use a master_site_local pointing to a local mirror, as
> we cannot expect the upload to be completed at the time the build runs.
> Right now we are downloading all files twice.

I split the fetch phase into a separate command. This should make it
easier to see that the dependency failed because it was unable to fetch
the distfiles.

https://github.com/macports/mpbb/commit/639def5ad990ecf1e227e377e5a46124c447ca94

This will also no longer update the failcache, as fetching could be a
temporary problem.

This should also remedy the situation on legacy systems for now.
Installation as a dependency will be retried even if a previous run
failed due to fetching. By that time, the distfile was hopefully
uploaded to the mirror.

We should still investigate in adding a master_site_local, as this
involves more fetches from the internet than necessary at the moment.

Rainer

Re: Build Failure on ports-10.6_x86_64_legacy: libpsl, mosquitto

[Mojca Miklavec]

On 4 April 2018 at 17:19, Rainer Müller wrote:
> On 2018-04-04 16:36, Andrew Moore wrote:
>> If I’m reading correctly the attached report from buildbot
>> port net/mosquitto fails for OS X <=
>> 10.6 due to indirect dependency: libpsl.  If so, this is presumably a
>> known issue, since many ports depend upon libpsl?

It also failed on 10.7 and 10.8.

> The failed libpsl build is this one (you can find it in the log of the
> install-dependencies step). The build failed because libpsl cannot be
> fetched on systems that do not support modern TLS protocol versions.
> This is a well known problem.

I triggered new builds of mosquitto now that libpsl has been built on
all systems.

Mojca

Re: Build Failure on ports-10.6_x86_64_legacy: libpsl, mosquitto

[Rainer Müller]

On 2018-04-04 16:36, Andrew Moore wrote:
> If I’m reading correctly the attached report from build...@macports.org
> , port net/mosquitto fails for OS X <=
> 10.6 due to indirect dependency: libpsl.  If so, this is presumably a
> known issue, since many ports depend upon libpsl?

The failed libpsl build is this one (you can find it in the log of the
install-dependencies step). The build failed because libpsl cannot be
fetched on systems that do not support modern TLS protocol versions.
This is a well known problem.

https://build.macports.org/builders/ports-10.6_x86_64_legacy-builder/builds/61220

Now that we execute the mirror step before attempting a build, we should
make the buildbot use a master_site_local pointing to a local mirror, as
we cannot expect the upload to be completed at the time the build runs.
Right now we are downloading all files twice.

Rainer

Re: Build Failure on ports-10.6_x86_64_legacy: libpsl, mosquitto

[Ken Cunningham]

You're not expected to fix it.

It would be nice perhaps to make a ticket about it if you like, so someone who 
uses that OS cersion might dig in.

Best, Ken

> On Apr 4, 2018, at 07:36, Andrew Moore <slew...@gmail.com> wrote:
> 
> If I’m reading correctly the attached report from build...@macports.org, port 
> net/mosquitto fails for OS X <= 10.6 due to indirect dependency: libpsl.  If 
> so, this is presumably a known issue, since many ports depend upon libpsl?
> 
> I don’t have the time or resources to debug the issue on legacy systems at 
> the moment.  What to do?  Thank you!
> -AM
> 
>> Begin forwarded message:
>> 
>> From: build...@macports.org
>> Subject: Build Failure on ports-10.6_x86_64_legacy: libpsl, mosquitto
>> Date: April 3, 2018 at 10:05:42 AM EDT
>> To: slew...@gmail.com
>> Cc: macports-bui...@lists.macports.org
>> Reply-To: nore...@macports.org
>> 
>> Status:   Failure
>> Build slave:  ports-10.6_x86_64_legacy
>> Full logs:
>> https://build.macports.org/builders/ports-10.6_x86_64_legacy-watcher/builds/13869
>> Build reason: The SingleBranchScheduler scheduler named 'ports' triggered 
>> this build
>> Port list:mosquitto
>> Subport list:
>>  - mosquitto
>> Variants: None
>> Revision: 7099171b51ebc15193f167486955d9f60e13baa4
>> Build time:   0:04:24
>> Author:   Andrew L. Moore <slew...@gmail.com>
>> 
>> Log from failed builds:
>>  Building 'mosquitto' ... [ERROR] (failed to install dependency 'libpsl')
>>  > maintainers: slew...@gmail.com.
>> 
>> Broken ports:
>>  - libpsl
>>  - mosquitto
>> 
>> Responsible maintainers:
>>  - slew...@gmail.com
>> 
>> Links to individual build jobs:
>> - jobs-mirror #6592
>>  https://build.macports.org/builders/jobs-mirror/builds/6592
>> 
>> -- 
>> Best regards,
>> MacPorts Buildbot
>> https://build.macports.org/
>> 
> 

Re: Build Failure on ports-10.6_x86_64_legacy: libpsl, mosquitto

[Andrew Moore]

If I’m reading correctly the attached report from build...@macports.org, port 
net/mosquitto fails for OS X <= 10.6 due to indirect dependency: libpsl.  If 
so, this is presumably a known issue, since many ports depend upon libpsl?

I don’t have the time or resources to debug the issue on legacy systems at the 
moment.  What to do?  Thank you!
-AM

> Begin forwarded message:
> 
> From: build...@macports.org
> Subject: Build Failure on ports-10.6_x86_64_legacy: libpsl, mosquitto
> Date: April 3, 2018 at 10:05:42 AM EDT
> To: slew...@gmail.com
> Cc: macports-bui...@lists.macports.org
> Reply-To: nore...@macports.org
> 
> Status:   Failure
> Build slave:  ports-10.6_x86_64_legacy
> Full logs:
> https://build.macports.org/builders/ports-10.6_x86_64_legacy-watcher/builds/13869
> Build reason: The SingleBranchScheduler scheduler named 'ports' triggered 
> this build
> Port list:mosquitto
> Subport list:
>   - mosquitto
> Variants: None
> Revision: 7099171b51ebc15193f167486955d9f60e13baa4
> Build time:   0:04:24
> Author:   Andrew L. Moore <slew...@gmail.com>
> 
> Log from failed builds:
>   Building 'mosquitto' ... [ERROR] (failed to install dependency 'libpsl')
>   > maintainers: slew...@gmail.com.
> 
> Broken ports:
>   - libpsl
>   - mosquitto
> 
> Responsible maintainers:
>   - slew...@gmail.com
> 
> Links to individual build jobs:
> - jobs-mirror #6592
>  https://build.macports.org/builders/jobs-mirror/builds/6592
> 
> -- 
> Best regards,
> MacPorts Buildbot
> https://build.macports.org/
>