Re: What is up with perl5.26 on Snow Leo?

[Uli Wienands]

Well, I wonder whether someone did something within the last day or two. 
I just tried it again, and lo-and-behold, perl5.26 installed just fine & 
now MacPorts is getting to work on the backlog of things that I could 
not update because of this problem.


This after I just found out (in debug mode) that the perl5.26 port 
actually was not able to get its files; the fetch errored out with a 404 
error for all attempts. What is odd is that it proceeded to do a bunch 
of things after that & even went to the destroot phase. I would love to 
send the log, but the renewed attempt was to generate the log so I could 
pack it up & send it, only for now the install succeeded making this moot.


So, my thanks go out to whomever fixed this.

Uli


On 11/29/21 9:04 AM, Bjarne D Mathiesen wrote:

This looks as the same problem we have with MySQL57, where it's using
the system cctools instaead of the more modern cctools from macports.

The fix is described here :
https://trac.macports.org/ticket/59072#comment:1

declare usrBin='/usr/bin'
declare devBin="/Developer${usrBin}"

while read -u 9 cctool
do
 theTool=$( basename ${cctool} )
 echo ${theTool}
 for binDir in "${usrBin}" "${devBin}"
 do
 mv -n "${binDir}/${theTool}" "${binDir}/${theTool}.orig"
 rm "${binDir}/${theTool}"
 ln -s "${cctool}" "${binDir}/${theTool}"
 done
done 9< <( port contents cctools | fgrep /bin/ )

On my 2 x 10.6.8 I've got :
#=> port installed perl*
The following ports are currently installed:
   perl5 @5.28.3_0+perl5_30 (active)
   perl5.30 @5.30.3_3 (active)

Re: is macports getting rusty?

[Chris Jones]

To first order builds are done on a first-committed-first-built basis. 
The time a build takes is not something known up front so prioritizing 
basis on this would anyway not really be feasible.


Yes, the builders must install the dependencies of a port before a given 
port can be built. If one of those deps also has an update in the queue, 
later on than the current build, but needed for the build then it will 
be done 'early'. so in a sense it is correct that commonly used 
dependencies will tend to get built a bit faster than they might 
otherwise get done.


'Normally' though the builders keep up with the pace of commits, so the 
build queue is reasonable short. It can happen that some commit might 
happen that requires a lot of ports to get built, or a big build (like 
gcc, clang or rust) comes along that causes a small backlog to build up. 
These are usually cleared quite quickly (by which I mean a few days).


Another cause of a backlog is when a new builder is added, like was done 
recently for macOS12. The builders in this case have to work their way 
through all the various deps, so a backlog can build up that takes a 
while to clear.


Chris

On 29/11/2021 2:32 pm, Richard L. Hamilton wrote:
I would expect that the buildbots also need to satisfy dependencies, and 
thus, heavily-depended on builds would tend to be done earlier, 
regardless of whether or not they were large and slow; and if they are 
large and slow, they're arguably delaying the building of even more 
smaller ports that don't depend on them, which is not necessarily a gain.


If I'm right about ports being built also by the buildbots in dependency 
order, any further tweak to build order would likely have minimal benefits.


The real improvement would be more buildbots, but that would take either 
a way of trusting volunteer buildbots (and that they were managed 
properly to produce fully correct results, and quite possibly dedicated 
to the purpose to avoid anything that would conflict or interfere), or 
an actual budget, or donated servers and server farm space, power, 
cooling, etc. Another improvement might be more people qualified, 
trusted, and authorized to babysit the buildbots, and/or automatic 
tickets on failed builds, since if a build on a buildbot fails, it's 
either a problem with the buildbot or with the port itself; and in the 
latter case, those building themselves will also have the problem.


On Nov 29, 2021, at 09:15, Bill Cole 
> wrote:


On 2021-11-29 at 08:32:46 UTC-0500 (Mon, 29 Nov 2021 13:32:46 +)
Chris Jones mailto:jon...@hep.phy.cam.ac.uk>>
is rumored to have said:

If you find yourself during an port update building rust from source, 
either just let it finish, or if you don't want your machine tied up 
building rust for some period, just cancel the build and try again 
later on, as as others have pointed out eventually the buildbots will 
provide the binary tarball for it and thus you will just pick that 
once its available.


This raises a question that I cannot find a clear answer to:

How are builds prioritized on the buildbots?

I would hope that in an ideal world, some priority is given to large, 
slow, and heavily-depended-upon builds. It's also easier to say that 
than to actually implement it, but it would help address a real pain 
point in using MacPorts.



--
Bill Cole
b...@scconsult.com  or billc...@apache.org 


(AKA @grumpybozo and many *@billmail.scconsult.com addresses)
Not Currently Available For Hire



--
eMail:mailto:rlha...@smart.net 

Re: What is up with perl5.26 on Snow Leo?

[Bjarne D Mathiesen]

This looks as the same problem we have with MySQL57, where it's using
the system cctools instaead of the more modern cctools from macports.

The fix is described here :
https://trac.macports.org/ticket/59072#comment:1

declare usrBin='/usr/bin'
declare devBin="/Developer${usrBin}"

while read -u 9 cctool
do
theTool=$( basename ${cctool} )
echo ${theTool}
for binDir in "${usrBin}" "${devBin}"
do
mv -n "${binDir}/${theTool}" "${binDir}/${theTool}.orig"
rm "${binDir}/${theTool}"
ln -s "${cctool}" "${binDir}/${theTool}"
done
done 9< <( port contents cctools | fgrep /bin/ )

On my 2 x 10.6.8 I've got :
#=> port installed perl*
The following ports are currently installed:
  perl5 @5.28.3_0+perl5_30 (active)
  perl5.30 @5.30.3_3 (active)

-- 
Bjarne D Mathiesen
Korsør ; Danmark ; Europa
---
denne besked er skrevet i et totalt M$-frit miljø
MacPro 2010 ; OpenCore + macOS 10.15.7 Catalina
2 x 3,46 GHz 6-Core Intel Xeon ; 256 GB 1333 MHz DDR3 ECC RDIMM
ATI Radeon RX 590 8 GB

Re: is macports getting rusty?

[Richard L. Hamilton]

I would expect that the buildbots also need to satisfy dependencies, and thus, 
heavily-depended on builds would tend to be done earlier, regardless of whether 
or not they were large and slow; and if they are large and slow, they're 
arguably delaying the building of even more smaller ports that don't depend on 
them, which is not necessarily a gain.

If I'm right about ports being built also by the buildbots in dependency order, 
any further tweak to build order would likely have minimal benefits.

The real improvement would be more buildbots, but that would take either a way 
of trusting volunteer buildbots (and that they were managed properly to produce 
fully correct results, and quite possibly dedicated to the purpose to avoid 
anything that would conflict or interfere), or an actual budget, or donated 
servers and server farm space, power, cooling, etc. Another improvement might 
be more people qualified, trusted, and authorized to babysit the buildbots, 
and/or automatic tickets on failed builds, since if a build on a buildbot 
fails, it's either a problem with the buildbot or with the port itself; and in 
the latter case, those building themselves will also have the problem.

> On Nov 29, 2021, at 09:15, Bill Cole 
>  wrote:
> 
> On 2021-11-29 at 08:32:46 UTC-0500 (Mon, 29 Nov 2021 13:32:46 +)
> Chris Jones 
> is rumored to have said:
> 
>> If you find yourself during an port update building rust from source, either 
>> just let it finish, or if you don't want your machine tied up building rust 
>> for some period, just cancel the build and try again later on, as as others 
>> have pointed out eventually the buildbots will provide the binary tarball 
>> for it and thus you will just pick that once its available.
> 
> This raises a question that I cannot find a clear answer to:
> 
> How are builds prioritized on the buildbots?
> 
> I would hope that in an ideal world, some priority is given to large, slow, 
> and heavily-depended-upon builds. It's also easier to say that than to 
> actually implement it, but it would help address a real pain point in using 
> MacPorts.
> 
> 
> -- 
> Bill Cole
> b...@scconsult.com or billc...@apache.org
> (AKA @grumpybozo and many *@billmail.scconsult.com addresses)
> Not Currently Available For Hire
> 

-- 
eMail:  mailto:rlha...@smart.net

Re: is macports getting rusty?

[Bill Cole]

On 2021-11-29 at 08:32:46 UTC-0500 (Mon, 29 Nov 2021 13:32:46 +)
Chris Jones 
is rumored to have said:

If you find yourself during an port update building rust from source, 
either just let it finish, or if you don't want your machine tied up 
building rust for some period, just cancel the build and try again 
later on, as as others have pointed out eventually the buildbots will 
provide the binary tarball for it and thus you will just pick that 
once its available.


This raises a question that I cannot find a clear answer to:

How are builds prioritized on the buildbots?

I would hope that in an ideal world, some priority is given to large, 
slow, and heavily-depended-upon builds. It's also easier to say that 
than to actually implement it, but it would help address a real pain 
point in using MacPorts.



--
Bill Cole
b...@scconsult.com or billc...@apache.org
(AKA @grumpybozo and many *@billmail.scconsult.com addresses)
Not Currently Available For Hire

Re: is macports getting rusty?

[Chris Jones]

Give the developers of rust itself a piece of your mind if you like, 
over how slow it is to compile. Good luck with that - I suspect to be 
THAT slow (although really? compiling a C compiler suite plus library 
isn't exactly fast either) they might be doing some things by brute 
force that they haven't got any other good way to do - like running a 
serious test suite, as just one example. And of course they're the ones 
that suffer most over how slow it is to compile, since they have to 
rebuild rust every time they want to test some seemingly minor change. 
So I suspect they're already doing what can to speed it up that they 
don't think sacrifices any other concern.


Actually building rust does NOT take that much longer than any other 
compiler tool kit, such as gcc or LLVM(clang) . This is in part because 
rust itself is based on LLVM and internally the build compiles its own 
LLVM instance before going on to build the rust specific parts on top of 
that. This accounts from probably something like 60-70% of the total 
build time.


If you find yourself during an port update building rust from source, 
either just let it finish, or if you don't want your machine tied up 
building rust for some period, just cancel the build and try again later 
on, as as others have pointed out eventually the buildbots will provide 
the binary tarball for it and thus you will just pick that once its 
available.


Chris

Re: is macports getting rusty?

[Richard L. Hamilton]

It is _possible_ to write reliable, secure code in C or even assembler. But for 
large complex software, it's darn unlikely; there's too many ways to mess up, 
even for capable and meticulous programmers, and most, even if bright and 
useful, aren't _that_ good.

Rust is supposed to eliminate a number of common types of problems, while still 
producing efficient code. So of course more projects will adopt it, and rewrite 
at least critical parts to use it (or something similar).

MacPorts is mainly a collection of software from elsewhere, and the mechanisms 
to automate installation and updates. That does not provide any control over 
such decisions of individual projects; MacPorts can perhaps feed bug fixes back 
to projects, but they don't have to accept them, and would not be likely to 
listen to complaints about depending on rust because of how slow it is to update

Give the developers of rust itself a piece of your mind if you like, over how 
slow it is to compile. Good luck with that - I suspect to be THAT slow 
(although really? compiling a C compiler suite plus library isn't exactly fast 
either) they might be doing some things by brute force that they haven't got 
any other good way to do - like running a serious test suite, as just one 
example. And of course they're the ones that suffer most over how slow it is to 
compile, since they have to rebuild rust every time they want to test some 
seemingly minor change. So I suspect they're already doing what can to speed it 
up that they don't think sacrifices any other concern.

A better question might be why aren't you getting/using one of the pre-built 
binary packages? On https://packages.macports.org/rust/ 
 I see for the latest rust version:

rust-1.56.1_3.darwin_13.x86_64.tbz2 2021-11-24 11:01101M
rust-1.56.1_3.darwin_14.x86_64.tbz2 2021-11-24 15:46101M
rust-1.56.1_3.darwin_15.x86_64.tbz2 2021-11-24 17:20101M
rust-1.56.1_3.darwin_16.x86_64.tbz2 2021-11-24 14:04101M
rust-1.56.1_3.darwin_17.x86_64.tbz2 2021-11-24 14:01101M
rust-1.56.1_3.darwin_18.x86_64.tbz2 2021-11-24 15:54101M
rust-1.56.1_3.darwin_19.x86_64.tbz2 2021-11-24 15:52101M
rust-1.56.1_3.darwin_20.arm64.tbz2  2021-11-25 15:0694M
rust-1.56.1_3.darwin_20.x86_64.tbz2 2021-11-24 10:38100M
rust-1.56.1_3.darwin_21.arm64.tbz2  2021-11-24 08:1594M
rust-1.56.1_3.darwin_21.x86_64.tbz2 2021-11-29 05:47100M

Look at the dates: they don't get pre-built instantly, so if you're updating 
early, you won't benefit from the pre-built version. There also appears to be a 
single defined variant (+debug) - the pre-built packages will only be used if 
that variant is NOT requested. See below for equivalent macOS versions.

Darwin version  macOS version
 1  10.0 Cheetah
 5  10.1 Puma (later minor versions, at any rate)
 6  10.2 Jaguar
 7  10.3 Panther
 8  10.4 Tiger
 9  10.5 Leopard
10  10.6 Snow Leopard
11  10.7 Lion
12  10.8 Mountain Lion
13  10.9 Mavericks
14  10.10 Yosemite
15  10.11 El Capitan
16  10.12 Sierra
17  10.13 High Sierra
18  10.14 Mojave
19  10.15 Catalina
20  11 Big Sur
21  12 Monterey



> On Nov 28, 2021, at 23:37, Kastus Shchuka  wrote:
> 
> Dear macports users,
> 
> Recently, more and more ports began to depend on rust and cargo.
> 
> Maybe rust is a wonderful language that will solve all problems of the world. 
> I just wonder, if it is so good, why it takes forever and a day (literally)  
> to compile? I've never seen anything taking that long to build. 
> 
> I've been using graphviz port for over 10 years, I guess. I had to delete it 
> today.
> 
> graphviz depends on gd2. gd2 depends on libheif. libheif depends on rav1e. 
> Now rav1e started depending on cargo-c, nasm, clang-13, cargo.
> An attempt to upgrade rav1e launched a build of cargo-c which I had to kill 
> as I did not have luxary to wait for tens of hours for it to finish.
> 
> I either have to keep outdated ports or stop using them and delete. 
> Unfortunately, the usable surface of macports started shrinking for me (or 
> should I call it "rusting"?). 
> 
> Another example is py-cryptography, which now requires rust to build. Until 
> binary package was made available, it took me over a day to upgrade 
> py-cryptography. 
> 
> I also now have a broken ImageMagic because its dependency chain pulls in 
> rust. And the list goes on and on.
> 
> I doubt people who rushed rust into macports are going to reconsider their 
> decisions. I am just sharing my experience with this "rusting"
> 
> Thank you 

Re: is macports getting rusty?

[Mark Anderson]

I think this is more a reflection of the open source landscape than
anything we're doing.

py-cryptography, for example, is 8% Rust, according to Github. I don't know
when they added it, but the latest version needs it. Same goes for rav1e.
It's one-third Rust.

Along with Go, you're going to see more and more Rust. I think the only way
around that is waiting for a binary package.

—Mark
___
Mark E. Anderson 
MacPorts Trac WikiPage 
GitHub Profile 



On Sun, Nov 28, 2021 at 11:37 PM Kastus Shchuka  wrote:

> Dear macports users,
>
> Recently, more and more ports began to depend on rust and cargo.
>
> Maybe rust is a wonderful language that will solve all problems of the
> world. I just wonder, if it is so good, why it takes forever and a day
> (literally)  to compile? I've never seen anything taking that long to
> build.
>
> I've been using graphviz port for over 10 years, I guess. I had to delete
> it today.
>
> graphviz depends on gd2. gd2 depends on libheif. libheif depends on rav1e.
> Now rav1e started depending on cargo-c, nasm, clang-13, cargo.
> An attempt to upgrade rav1e launched a build of cargo-c which I had to
> kill as I did not have luxary to wait for tens of hours for it to finish.
>
> I either have to keep outdated ports or stop using them and delete.
> Unfortunately, the usable surface of macports started shrinking for me (or
> should I call it "rusting"?).
>
> Another example is py-cryptography, which now requires rust to build.
> Until binary package was made available, it took me over a day to upgrade
> py-cryptography.
>
> I also now have a broken ImageMagic because its dependency chain pulls in
> rust. And the list goes on and on.
>
> I doubt people who rushed rust into macports are going to reconsider their
> decisions. I am just sharing my experience with this "rusting"
>
> Thank you for reading.
>
> -Kastus