Re: Fix: My dovecot install fails after macOS Sonoma 14.5 update and MacPorts update

[Ryan Schmidt]

On May 22, 2024, at 16:25, Gerben Wierda wrote:
> 
> I was able to circumvent this autoconf failure by installing gettext before 
> installing dovecot, I don't know if that is the best solution, but ot works 
> (as in, it now installs).

Yes that is the solution and I committed it here:

https://github.com/macports/macports-ports/commit/8addb4df7ea4e8da28e059e28052f68b8bad02cb

Only when you use the +apns variant, dovecot regenerates the configure script 
from configure.ac (because it applies a patch to configure.ac). configure.ac 
references the AM_ICONV macro so the port that provides that macro must be 
present at build time. From its name it seems like the AM_ICONV macro should be 
provided by the libiconv port but for whatever reason it's actually provided by 
the gettext port.

Re: livecheck and curl 8.7.1

[Ryan Schmidt]

> On Apr 20, 2024, at 22:12, Fred Wright wrote:
> 
> there was a case a couple of years ago where MacPorts decided to prefer 
> python.org to the mirrors (at my location), and the fetch (on 10.9) would 
> hang in a way that wasn't subject to a timeout, so it never gave up and never 
> moved on to the mirrors.  The response when I reported this was "gee, it's 
> supposed to have timeouts".  I don't know exactly what got fixed, but I 
> haven't seen this lately.

You're right, there are timeouts. 

If there is a server causing you trouble, you can tell MacPorts not to access 
it using the host blacklist setting in macports.conf. 

Re: 10.5 and gcc8 x86-64 ok but ppc bails with dlerror

[Ryan Schmidt]

On Apr 5, 2024, at 13:03, Riccardo Mottola wrote:
> 
> Odd/Even was an old practice, possibly not followed anymore.

It was an old practice of GNOME abandoned in 2020. Graphviz, Cairo, Pango, 
Pixman, Glib2 are examples, from ports I maintain(ed). The MacPorts "gnome" 
livecheck recipe still follows this rule. Never heard of gcc following it. All 
I remember about gcc's versioning is that prior to version 5, the branch name 
consisted of the first two numbers in the version number, and as of 5 it is 
only the first number. Thus the branches are e.g. 4.7, 4.8, 4.9, 5, 6, 7, etc. 

Re: 10.5 and gcc8 x86-64 ok but ppc bails with dlerror

[Ryan Schmidt]

On Apr 5, 2024, at 03:52, Riccardo Mottola wrote:
> 
> I propose  to keep even versions, because they are stable ones

Do you have a source for this claim? It's the first I've heard of it. As far as 
I know, all gcc version numbers are stable.

Re: how to handle 'internal' logs for failed builds

[Ryan Schmidt]

On Apr 4, 2024, at 17:29, Joshua Root wrote:
> 
> Do that on a branch, make some change to the affected port as well, and push 
> to your fork and let CI run.

That presumes it will fail in CI. 

If it succeeds in CI but fails on buildbot, suspect a case sensitivity issue. 

Re: New port for ldc2

[Ryan Schmidt]

On Mar 24, 2024, at 15:52, P Padil wrote:

> I’m trying to develop a new port for the d-lang ldc package (see attached), 
> but I’m having a hard time deciphering the macports guide as to how to 
> install something like this. It is a binary distributed in a tar.xz file 
> which has the bin, lib, etc, imports directories in the tar and, I would have 
> though, would be easy to link to the appropriate installion dirs.  However I 
> cannot find anything as to how to do this in a Portfile.  Any pointers/help 
> appreciated. 

There's no documentation specific to this case because it's not a usual way to 
distribute software. I don't recall encountering this need in MacPorts before. 

It looks like you're trying to write a portfile that installs a precompiled 
binary. MacPorts prefers to build from source. Can the port be made to do that 
instead?

Re: Unable to build Xcode projects using SwiftPM in MacPorts sandbox

[Ryan Schmidt]

On Mar 23, 2024, at 23:00, Zero King wrote:
> 
> I think it could be base's sandbox that prevented writes to the home 
> directory, where SwiftPM stores its cache.

If disabling sandboxing in macports.conf makes it work, then your suspicion is 
probably correct. 

MacPorts sets the HOME environment variable to point to a directory within 
workpath. It looks like it's ignoring that and trying to write to a 
subdirectory of the macports user's real home directory, 
/opt/local/var/macports/home. That would be a bug to file with Apple. It has 
been a long-standing problem that has affected MacPorts in other ways before. 

Re: Unable to access macports-infrastructure.git

[Ryan Schmidt]

> On Mar 12, 2024, at 19:46, Dave Allured wrote:
> 
> 
> I am getting an unusual builder error that is blocking my recent port update 
> on all of the builders.  Go to console view on build.macports.org, find port 
> `gjs-devel`, and look at the stdio logs below the commit message.
> 
> `fatal: unable to access 
> 'https://github.com/macports/macports-infrastructure.git/': Server aborted 
> the SSL handshake`
> 
> Now I know they put Dave detectors in all the traffic lights.  I thought 
> maybe that was the problem here, but Ryan has one too.  Please let me know 
> how to get my build started, and whether I owe money or something.  Thank you.

The build server has been under heavy load. Not completely sure why yet. It 
took me minutes to even pull up a buildbot web site. It's possible builds are 
failing as a result and will have to be retried once things calm down. I will 
try to make a change to reduce CPU use shortly. 

Re: jupytext port build cannot find jlpm

[Ryan Schmidt]

On Mar 12, 2024, at 10:00, Jonathan Stickel wrote:
> 
> command not found: jlpm
> 
> Recommendations for how to resolve this issue?

One solution is to figure out where this program is being called in the source 
code, then patch it to use the program name as it appears in MacPorts (e.g. in 
this case to add the python version to the end).


> Is there a way to set an alias during the build phase?
> 
> alias jlpm="jlpm-${python.branch}"

I can't recall seeing a port that sets a shell alias so that may not be 
possible.

Some ports do create a directory in workpath (e.g. ${workpath}/bin), then 
create symlinks in there having the desired names, and add ${workpath}/bin to 
PATH. But that's probably not needed in this case since you already have a 
directory that contains the files with the desired names (below).


> Or add 
> "${prefix}/Library/Frameworks/Python.framework/Versions/${python.branch}/bin" 
> to the path? I haven't been successful implementing either approach.

You can find examples of how to do that by running:

port file all|sort -u|xargs grep '\sPATH='

Note that you want ${frameworks_dir}; don't assume its value is 
${prefix}/Library/Frameworks


> I've been wondering if it is worth the trouble to create and maintain ports 
> for some of these python packages. These days they are often using advanced 
> python-internal build tools that are difficult to configure, while `pip 
> install ` often just works. By comparison, there is a port for Julia 
> but none of its packages, instead relying entirely on Julia's package 
> manager. I'm curious about the thoughts of others on this matter.

I don't think it's been the intention to add all python modules to MacPorts. 
Rather, we add modules that are dependencies of other ports or are useful on 
their own (especially if requested by a user). 

The python portgroup was enhanced to support many of the new build systems. 
Check out the python.pep517_backend option.

Re: FYI, Ryan's email is bouncing...

[Ryan Schmidt]

> Email sent to ryandesign at macports.org goes to macports at ryandesign.com, 
> which then bounces. Hopefully Ryan is still seeing this, or someone knows how 
> to get in touch with him.

My email provider deleted my forwarding rules and I didn't notice right away so 
this was broken for about 2 days. I'm happy you got a bounce message back; I 
thought they were just throwing the mail away. I believe I fixed the forwarding 
last night. 

Re: { darwin any } ports getting reinstalled after OS upgrade

[Ryan Schmidt]

On Sep 27, 2023, at 06:09, Christopher Jones wrote:
> 
> That just means they are *supported* on any drawn version. Each version 
> still have its own build.

But that's just what "any" means: they build identically on any OS version. 
That's why only one single archive gets produced for these ports by the build 
system. When MacPorts on any OS version installs such a port, they all grab and 
untar the same file from the server. 

Nils is right in pointing out that there should be no need to reinstall those 
ports when upgrading the OS, but nobody has written any code to make that 
happen yet. For now, MacPorts will consider any port to need to be reinstalled 
if it was installed on a different OS version, even if that port is marked 
"any". 

> b.t.w. When upgrading to a new major OS version, you anyway should follow the 
> migration instructions, which involves manually removing all ports anyway.

We do recommend that, but the piecemeal method Nils is using is also viable. 

Re: { darwin any } ports getting reinstalled after OS upgrade

[Ryan Schmidt]

On Sep 27, 2023, at 05:57, Nils Breunese wrote:
> 
> I noticed that ports that use ‘platform { darwin any }’ get reinstalled 
> during this process, and ‘port outdated’ shows ‘(platform darwin 22 ≠ 23)’. 
> Does MacPorts really need to reinstall these ports if they’re marked as being 
> suitable for any Darwin version?

I agree, MacPorts should not need to reinstall those ports, but there is 
evidently no code at present to let it know that. You could file an enhancement 
ticket in Trac. 

Re: Qt 5.10 skipped?

[Ryan Schmidt]

> On Aug 30, 2023, at 19:40, Jason Liu wrote:
> 
> I just noticed that Qt 5.9 and Qt 5.11 are available in MacPorts, but not Qt 
> 5.10. Was there a specific reason why 5.10 was skipped, or was it just that 
> no one got around to it before 5.11 was released?

Certainly qt 5.10.x existed in MacPorts. You can find bugs that were filed 
against it:

https://trac.macports.org/query?port=~qt5=~%405.10 


I couldn't immediately find the commit in which it was removed. It's a little 
difficult since the qt5 portfile is massive and there are also the qt5 
portgroups. But I suspect it was deleted because there exists no macOS version 
for which qt 5.10.x is the most recent one available. 

> And a somewhat tangential question: When using the qt5 PortGroup, is there a 
> way to specify a particular version of Qt 5.x that needs to be used? I do see 
> plenty of references to ${qt5.version}, but is that something that I am 
> allowed to set in my Portfile?

Looking near the top of the qt5-1.0 portgroup,

https://github.com/macports/macports-ports/blob/master/_resources/port1.0/group/qt5-1.0.tcl
 


the prominent placement of qt5.min_version suggests to me that it is intended 
for portfile authors to override it. 38 ports reference this variable.

I don't believe there is any way for you to specify a maximum qt version, nor a 
specific version. The qt5 portgroup will select the blessed qt version that is 
compatible with the user's OS version. 

Re: Buildbots down?

[Ryan Schmidt]

On Feb 6, 2023, at 14:09, Fred Wright wrote:
> On Thu, 2 Feb 2023, Ryan Schmidt wrote:
>> On Feb 2, 2023, at 18:25, Herby G wrote:
>>> 
>>> There haven't been any builds for recent commits to master on Github in 
>>> over 24 hours at the moment of writing this.
>>> 
>>> https://build.macports.org/ is also failing to load.
>>> 
>>> Can someone take a look?
>> 
>> Unfortunately, Winter Storm Mara delivered several days of freezing rain to 
>> the Austin area. The ice and wind has broken a large number of tree branches 
>> which has downed a large number of power lines. 30% of the city, including 
>> the buildbot machines, has been without power, in the case of the buildbot 
>> for 38 hours already. As of four hours ago the power company has stated they 
>> are unable to provide an estimate for when power will be restored.
>> 
>> https://twitter.com/austinenergy/status/1621235217909383169?cxt=HHwWgoC8zem15P8s
>> 
>> This affects the build.macports.org web page, all build machines*, and also 
>> updates to rsync.macports.org.
>> 
>> (*The Apple Silicon build machine is off-site and still online but can't get 
>> anything done without build.macports.org telling it what to do.)
> 
> Based on a couple of ports where I've had PRs merged recently, it looks like 
> the buildbots are mostly back in service, but Mojave and arm64 are still out.

Yes. Disk first aid found a minor problem on the Mojave disk that I had hoped 
to correct before putting it back into service by booting from the installer 
and running some fsck commands there, but it is hanging when booting from the 
installer. It is possible I created the installer image incorrectly.

I took the Apple Silicon machine offline in anticipation of upgrading it; see 
https://trac.macports.org/ticket/66637.

Re: configure step downloading resources - supported on buildbots?

[Ryan Schmidt]

On Feb 6, 2023, at 05:46, René J.V. Bertin wrote:

> Exploring Audacity's latest code I notice it requires the conan package 
> manager nowadays, for building "vendored" libraries. A number of those aren't 
> in MacPorts.
> 
> From what I've seen this downloaded stuff goes into `build.dir` but I seem to 
> recall that the build bots don't support downloading things during the build. 
> Is that correct and still the case?

You can download things during a build, but it is strongly discouraged.

Checksum mismatches for ports fetching from GitHub updated January 30, 2023

[Ryan Schmidt]

Last week we received four tickets about unusual checksum mismatches for 
distfiles that download automatically-generated tarballs from GitHub:

https://trac.macports.org/ticket/66796 (bsdmake)
https://trac.macports.org/ticket/66804 (libgit2)
https://trac.macports.org/ticket/66805 (rav1e)
https://trac.macports.org/ticket/66822 (cargo-c)

This appears to have been caused by a problem on GitHub that was resolved:

https://github.blog/2023-02-01-github-availability-report-january-2023/

> January 30 18:35 UTC (lasting 7 hours)
> 
> We upgraded our production Git binary with a recent version from
> upstream. The updates included a change to use an internal
> implementation of gzip when generating archives. This resulted in subtle
> changes to the contents of the “Download Source” links served by GitHub,
> leading to checksum mismatches. No content was changed.
> 
> After becoming aware of the impact to many communities, we rolled back
> the compression change to restore the previous behavior.
> 
> Similar to the above, we are still investigating the contributing
> factors of this incident, and will provide a more thorough update in
> next month’s report.

I've closed the tickets as fixed because GitHub resolved the problem by 
returning to the previous compression method. However if any commits were made 
that updated any ports that fetch from automatically-generated GitHub tarballs 
during the timeframe that GitHub was using the incorrect compression method (7 
hours after? before? during? 20230130T183500Z) we may have mirrored files 
generated with the wrong compression method, and now that the problem has been 
fixed users might encounter checksum mismatches if they fetch from GitHub 
instead of our mirror; we should identify and fix any such ports. For example, 
terragrunt-0.43 may have been affected when this change was committed on 
20230130T223341Z:

https://github.com/macports/macports-ports/commit/1100bde6eb3e1dc420a36675c39e46f821a2f41f

but it has subsequently been updated to a newer version so it's no longer 
affected.

Re: Buildbots down?

[Ryan Schmidt]

On Feb 2, 2023, at 18:25, Herby G wrote:
> 
> There haven't been any builds for recent commits to master on Github in over 
> 24 hours at the moment of writing this.
> 
> https://build.macports.org/ is also failing to load.
> 
> Can someone take a look?

Unfortunately, Winter Storm Mara delivered several days of freezing rain to the 
Austin area. The ice and wind has broken a large number of tree branches which 
has downed a large number of power lines. 30% of the city, including the 
buildbot machines, has been without power, in the case of the buildbot for 38 
hours already. As of four hours ago the power company has stated they are 
unable to provide an estimate for when power will be restored.

https://twitter.com/austinenergy/status/1621235217909383169?cxt=HHwWgoC8zem15P8s

This affects the build.macports.org web page, all build machines*, and also 
updates to rsync.macports.org.

(*The Apple Silicon build machine is off-site and still online but can't get 
anything done without build.macports.org telling it what to do.)

Re: macOS 13 builds down?

[Ryan Schmidt]

On Jan 28, 2023, at 07:59, Jimmy Yuen Ho Wong wrote:
> On Sat, Jan 28, 2023 at 1:57 PM Jimmy Yuen Ho Wong wrote:
> 
>> I've just upgraded to to macOS 13 from macOS 12 and noticed most of the 
>> ports I'm reinstalling are building from source. It looks like no ports for 
>> Ventura have ever been built. What's going on? 
> 
> Specifically, I'm inquiring only about the arm64 builds. x86_64 builds seem 
> to be fine.

https://trac.macports.org/ticket/66637

Re: Request to re-run buildbots for a few ports

[Ryan Schmidt]

On Nov 28, 2022, at 14:34, Kirill A. Korinsky wrote:

> May I ask anyone with access to reschedule build for ports: scalapack and 
> papilo?

Jeremy scheduled a rebuild of scalapack on 10.7 which succeeded. Looks like it 
had already succeeded on the other workers before.

I scheduled a rebuild of papilo on all workers. It failed on 10.6 because its 
dependency onetbb fails there. It succeeded on 10.9. The build on 10.8 should 
start soon. Looks like it already succeeded on the other workers before.

Re: Notability reqiurement for submitted ports?

[Ryan Schmidt]

On Oct 31, 2022, at 14:22, Zhenfu Shi wrote:

> Hi all,
> I’m thinking about submitting a small utility written in C++ to MP that is 
> really useful to only a small group of people (that is, rhythm gamers). Is 
> there any notability requirements for submitted ports, like they have in 
> Homebrew?

Nope! I would say feel free to submit a port for it, and if you want to 
maintain it, that's even better.

Re: having the "+test" or "+tests" variant propagate causes unexpected builds

[Ryan Schmidt]

On Oct 28, 2022, at 21:33, Daniel J. Luke wrote:
> 
> I don't think implementation difficulty is the barrier here - but that all 
> variants should just have the same behavior.
> 
> In my mind, the real problem is the need for +test variants, there should be 
> a way to just use the test phase - and perhaps changes to base/ to enable 
> that are a better option.

In one of my ports that has a tests variant, the reason why the variant exists 
is that the build system looks for certain test dependencies at configure time. 
If they're not there, it doesn't build the test suite and doesn't allow tests 
to be run later. I'm not sure how MacPorts could be improved to handle that 
better in the absence of a tests variant. Would you have MacPorts do the 
configuration in the configure phase, do the build in the build phase, and then 
redo the configuration and build in the test phase? Or would you suggest in 
this case that the test dependencies that are needed at configure time should 
be added unconditionally, so that even users who won't be running the tests 
need to install them? In my port's case the dependencies are probably small and 
that wouldn't make much of a difference, but I'm not sure it'll always be that 
way for all ports.

Re: Removed ports still listed on ports.macports.org

[Ryan Schmidt]

On Oct 30, 2022, at 01:06, Mojca Miklavec wrote:

> V ned., 30. okt. 2022 01:02 je oseba Nils Breunese napisala:
>> Hi,
>> 
>> The openjdk port was removed on August 25th after it had been obsoleted a 
>> while ago (https://github.com/macports/macports-ports/pull/15910), but I 
>> just noticed that https://ports.macports.org/port/openjdk/details/ still 
>> shows the openjdk port information from when it last existed.
>> 
>> I see there are more ports listed on 
>> https://ports.macports.org that have been 
>> removed from the ports tree, e.g. 
>> https://ports.macports.org/port/openjdk7-zulu/.

openjdk7-zulu has *not* been removed from the ports tree. It's there, it's just 
obsolete and can't be installed:

https://github.com/macports/macports-ports/blob/master/java/openjdk7-zulu/Portfile

It will be removed one year after it was marked as being obsolete.

>> Does ports.macports.org have some kind of 
>> mechanism that should clean up the entries for ports that no longer exist?
> 
> The port pages were kept on purpose (to let users find deleted ports easily 
> and potentially resurrect them instead of having to reinvent the wheel, for 
> historic / arcival reasons, ...), with the intention to clearly mark that 
> they have been deleted and they should not appear in searches within the site 
> unless explicitly requested.
> 
> Maybe the feedback should look different? Or do you have some other 
> suggestion?

That sounds like a fine idea, but 
https://ports.macports.org/port/openjdk/details/ does not say anywhere that 
openjdk has been deleted. (It says it is obsolete, which is different from 
deleted.) I seem to recall the ports web site used to say when a port was 
deleted but maybe it no longer does, or maybe it's broken for this specific 
port for some reason.

Re: Disabled key in launchd plists

[Ryan Schmidt]

On Sep 17, 2022, at 04:58, René J.V. Bertin wrote:
> 
> Ryan Schmidt wrote on 20220916::21:30:28 re: "Re: Disabled key in launchd 
> plists"
> 
>> My reading of the documentation is that the system will start any launchd 
>> plists at system startup time that are in the standard LaunchDaemons 
>> directories and that are not disabled.
> 
> That's possible, but it could be that it's done via launchctl. Academic 
> difference, probably.
> 
>> As I said, we don't want things the user installed with MacPorts to start 
>> unexpectedly at system startup time.
> 
> That would depend a bit on what kind of service, no, and to what extent the 
> unexpected load isn't that because documented? BTW, launch *agents* would 
> only be started when a user logs in. That, in a sense, is a trickier 
> situation. A MacPorts user is an admin, in principle, so in a situation to 
> decide on installing ports that contain a system daemon (which could perform 
> a crucial but strictly MacPorts-specific function). Deciding whether or not 
> other users on that system should run the same agents as him/her is another 
> thing.
> 
> I'm not familiar with `port load`; I presume (hope...) that it's unambiguous 
> as to the difference between agents and daemons, that the former have to be 
> started as the user, the latter "sudo" etc.
> 
>> I don't know if you can specify a directory to launchctl to have it load all 
>> plists in a directory.
> 
> I only noticed that yesterday too, first from looking through the sources and 
> then confirmed it in the manpage. It makes sense.
> 
>> If the plist is not in a standard LaunchDaemons directory, I assume that 
>> would load the plist immediately, but not at future system startups. Since 
>> that would be an undesirable situation I haven't ever tested it.
> 
> Why is that undesirable? As far as I see, the whole system is designed this 
> way. It works for agents (at least the ones symlinked in 
> /Library/LaunchAgents get started again after logging in, AFAIHS) but it 
> would make even more sense for daemons. You start (load) them once and they 
> keep running to perform the background service you (sometimes) want. You just 
> have to be aware of it.
> It's true that Apple could have foreseen a way to load either type 
> temporarily.
> 
> Or I could be wrong, the documentation isn't exactly without ambiguities (it 
> says "Note that per-user configuration files (LaunchAgents) must be owned by 
> the user loading them." which is clearly not true if they did not forget to 
> specify that this applies only to the ones in ~/Library/LaunchAgents).
> 
> I could reboot my system to test a few of these assumptions but somehow that 
> annoys me :)

I don't think I have anything further I want to add to this conversation. I've 
explained to the best of my knowledge how launchd/launchctl work and what 
MacPorts does with its launchd support and why. I'm sure you can perform any 
further investigations that interest you on your own.

Re: Disabled key in launchd plists

[Ryan Schmidt]

On Sep 15, 2022, at 14:48, René J.V. Bertin wrote:

> Ryan Schmidt wrote on 20220915::13:36:25 re: "Re: Disabled key in launchd 
> plists"
> 
>> Yes, to all of that, but none of that really relates to the question you 
>> asked about the Disabled key.
> 
> I think it does... because
> 
>> Merely being placed in a directory does not cause a launchd plist to be 
>> loaded.
> 
> means that someone has to load them explicitly.
> 
> Now I wasn't aware that you can give a directory as argument to `launchctl 
> [un]load`. Without that possibility I don't really see what the point of the 
> Disabled key is (so you have to un/load the plists concerned with -w or -F 
> ... so what?). And that's what my question was about.
> 
> 
>>> But would launchd even look where MacPorts installs these plists? I suppose 
>>> it doesn't...
>> 
>> MacPorts places symlinks to its launchd plists in /Library/LaunchDaemons so 
>> that launchd can find them. This has been working fine ever since launchd 
>> support was added to MacPorts for the release of Tiger over 17 years ago.
> 
> Yes, but not automatically (so I missed it for the port I'm currently 
> tinkering with). But if "Merely being placed in a directory does not cause a 
> launchd plist to be loaded" and you have to give a full path to launchctl, 
> removing those symlinks shouldn't make a difference, correct?

My reading of the documentation is that the system will start any launchd 
plists at system startup time that are in the standard LaunchDaemons 
directories and that are not disabled.

As I said, we don't want things the user installed with MacPorts to start 
unexpectedly at system startup time. That's why we mark them disabled. The user 
opts in to loading a port's launchd plist(s) by running the "port load" command 
(or the equivalent longer launchctl command(s)). This causes the plist to be 
loaded immediately and at all future system startups. Ports that use the 
standard MacPorts startupitems mechanism print a note to the user explaining 
this when such a port is installed.

I don't know if you can specify a directory to launchctl to have it load all 
plists in a directory. In my experience, you give the absolute path of the 
plist you want to load. If the plist is not in a standard LaunchDaemons 
directory, I assume that would load the plist immediately, but not at future 
system startups. Since that would be an undesirable situation I haven't ever 
tested it.

Re: Disabled key in launchd plists

[Ryan Schmidt]

On Sep 15, 2022, at 11:01, René J.V. Bertin wrote:
> 
> On Thursday September 15 2022 10:07:24 Ryan Schmidt wrote:
>> Apple's default value of Disabled is false, which means (according to the 
>> documentation I read) that the service will be started automatically the 
>> next time the system starts up
> 
> Or the user logs in, for agents. That's what I thought too until recently, 
> but the manpage also says 
> 
> "This document details the parameters that can be given to an XML property 
> list that can be loaded into launchd with launchctl."
> 
> The launchd manpage confirms this by stating that
> "The primary and preferred interface to launchd is via the launchctl(1) tool 
> which (among other options) allows the user or administrator to load and 
> unload jobs."
> 
> But it also says 
> 
> "Where possible, it is preferable for jobs to launch on demand based on 
> criteria specified in their respective configuration files."

Yes, to all of that, but none of that really relates to the question you asked 
about the Disabled key.

> which could imply that jobs can get loaded by simply being in the proper 
> directory.

Indeed Apple believes it is more efficient if you write launchd plists that 
work "on demand", that is to say, they specify under what conditions they 
should launch (e.g. when a connection comes in on a particular network port or 
when a file appears in a particular directory or at a particular time of day) 
and launchd then arranges for that to happen. launchd plists created by the 
MacPorts startupitem keyword to not do this* but manually-written plists that 
ports may install could do that if desired.

Merely being placed in a directory does not cause a launchd plist to be loaded.

Loading a launchd plist does not necessarily mean that a process will run. It 
just means that it is then possible for the OS to run the process under 
whatever conditions the launchd plist specifies. MacPorts-created launchd 
plists will indeed start a process running the moment they are loaded, but 
plists coded to be "on demand" would not necessarily if their launch conditions 
had not been met.

*
https://trac.macports.org/ticket/13807
https://trac.macports.org/ticket/60654


>> , but we want the plists our ports install to be disabled until the user 
>> enables them.
> 
> But would launchd even look where MacPorts installs these plists? I suppose 
> it doesn't...

MacPorts places symlinks to its launchd plists in /Library/LaunchDaemons so 
that launchd can find them. This has been working fine ever since launchd 
support was added to MacPorts for the release of Tiger over 17 years ago.

Re: Disabled key in launchd plists

[Ryan Schmidt]

On Sep 15, 2022, at 08:24, René J.V. Bertin wrote:

> I've noticed that most of the launchd plists installed by ports have 
> `Disabled: true`. There are a select few that don't, among which the mariadb 
> server. I have that port installed, and yet no mariadb launchd job loaded 
> which suggests that launchd does not auto-load plists, at least not from an 
> "unsupported" location like /opt/local/Library/Launch{Agents,Daemons}.
> 
> What's the actual function of this key? Just a convenience feature so you can 
> do `launchctl [un]load /path/to/*.plist` and only load the enabled jobs?

The Disabled key, and other keys of launchd plists, is described in the 
launchd.plist manpage:

https://www.manpagez.com/man/5/launchd.plist/

Apple's default value of Disabled is false, which means (according to the 
documentation I read) that the service will be started automatically the next 
time the system starts up, but we want the plists our ports install to be 
disabled until the user enables them. A port may have been installed for the 
user as a dependency; the user may have no idea that the port includes a 
launchd plist and probably doesn't want some process they didn't ask for to be 
running persistently on their computer, especially if that program opens 
network connections. An exception to this is ports that need to run a 
persistent program in order to serve their purpose and hopefully do not need a 
network connection in order to do it, in which case such a port would use 
"startupitem.autostart yes". certsync is an example of such a port.

plists installed with the MacPorts startupitem options set Disabled to true. 
The mariadb port does not use the MacPorts startupitem options but instead 
provides a pre-made plist in the port's files directory and installs that. I 
don't know why it has Disabled set to false.

Re: Upgrading py-poetry-core and pep517

[Ryan Schmidt]

On Sep 11, 2022, at 12:40, David Gilman wrote:
> 
> There's a new release of poetry and py-poetry-core and poetry. I have
> the PR ready at https://github.com/macports/macports-ports/pull/16088
> . However, I am unable to get it to cleanly upgrade from an older
> version of py-poetry-core.
> 
> I think the issue has to do with the pep517 machinery intertwining
> with Poetry. If you have an old version of py-poetry-core / poetry
> installed the Python imports to do the pep517 build are trying to
> import across old and new versions of py-poetry-core / poetry and
> refactoring internal to those packages is breaking the install.
> 
> I am pretty stumped here about what to do next. Is there functionality
> in the Portfile DSL to force an uninstall/reinstall? That would be a
> quick and easy fix.

No. If your port cannot build correctly when a previous version of itself is 
installed, you can use something like:

PortGroup conflicts_build 1.0
conflicts_build ${subport}

However this should be considered a temporary measure until the bug in the 
build system can be corrected.

> As far as the "python.pep517_backend" field in the Python portgroup
> goes, it needs to be left default (setuptools) otherwise you get a
> cyclical dependency of py-poetry-core on py-poetry-core.

Re: include files for cgxCADTools

[Ryan Schmidt]

On Aug 7, 2022, at 19:06, Ryan Schmidt wrote:

> configure.cxxflags-append -std=c++11

In fact, the Makefile already had -std=c++11, but your patchfile removed it. So 
don't remove it.

Re: include files for cgxCADTools

[Ryan Schmidt]

On Aug 6, 2022, at 09:47, Mark Brethen wrote:

> See log file for errors I’m getting with using build.env instead of 
> build.args.

The log has errors like this:


Os -arch x86_64 -Wall -I../../FbdReader/inc -I/opt/local/include/opencascade  
-c FbdReader.cpp  -o FbdReader.o
make: Os: No such file or directory


You asked for this to happen when you wrote in your patchfile:

CXX =

Don't do that. :)

Why not let make's default value for CXX be used if the user (or in this case 
MacPorts) doesn't specify it? To accomplish this, remove the "CXX =" line.



On Aug 6, 2022, at 16:58, Mark Brethen wrote:

> If passing build.env, need to remove cc, cxx, etc. assignments in makefile. 
> Even ‘cc=‘ will cause those errors.

Variables that you specify in the environment (build.env) are defaults for 
make. Commands in the Makefile can override or append to values from the 
environment. When you wrote "CXX =" in the Makefile above, you instructed make 
to ignore any CXX set in the environment and replace it with the empty string.

Variables you specify in arguments (build.args) override anything set in the 
Makefile. Use this if a Makefile sets a variable incorrectly (for example if a 
Makefile sets "CC = something") and you want to override it in MacPorts without 
having to write a patchfile for it. (Ideally Makefiles would use e.g. "CC ?= 
something" which means "set CC to something if CC is not already set.")

So decide on a case by case basis, for each variable, what is the most 
appropriate thing to do.

Re: include files for cgxCADTools

[Ryan Schmidt]

On Aug 6, 2022, at 12:14, Mark Brethen wrote:
> 
> It builds successfully when I blacklist Apple’s clang.

It used clang++ 14, which I think defaults to a newer language standard.

So don't blacklist Apple clang. Instead, specify what language standard you 
want, per my previous message.

Re: include files for cgxCADTools

[Ryan Schmidt]

On Aug 6, 2022, at 12:09, Mark Brethen wrote:

> Here’s the log file when using build.args instead of build.env and compiling 
> with Apple’s clang. Different failure this time.

You're getting warning about C++11 features:


/opt/local/include/opencascade/Standard_Handle.hxx:75:19: warning: rvalue 
references are a C++11 extension [-Wc++11-extensions]
handle (handle&& theHandle) : entity(theHandle.entity)
  ^
/opt/local/include/opencascade/Standard_Handle.hxx:118:30: warning: rvalue 
references are a C++11 extension [-Wc++11-extensions]
handle& operator= (handle&& theHandle)
 ^
/opt/local/include/opencascade/Standard_Handle.hxx:221:5: warning: explicit 
conversion functions are a C++11 extension [-Wc++11-extensions]
explicit operator bool () const
^~~~
/opt/local/include/opencascade/Standard_Handle.hxx:281:34: warning: default 
template arguments for a function template are a C++11 extension 
[-Wc++11-extensions]
template ::value>::type>
 ^ 
~
/opt/local/include/opencascade/Standard_Handle.hxx:289:34: warning: default 
template arguments for a function template are a C++11 extension 
[-Wc++11-extensions]
template ::value>::type>
 ^ 
~


So:


* You're evidently using opencascade
* opencascade evidently requires C++11 mode or later
* You're evidently not using C++11 mode or later


To fix it add these lines:


# opencascade requires C++11 or later
compiler.cxx_standard 2011
configure.cxxflags-append -std=c++11


And you're getting these errors:


/opt/local/include/opencascade/BRepLib_MakeWire.hxx:176:81: error: a space is 
required between consecutive right angle brackets (use '> >')
 
NCollection_List>& theGrVL);

^~

> >
/opt/local/include/opencascade/BRepLib_MakeWire.hxx:178:79: error: a space is 
required between consecutive right angle brackets (use '> >')
  void CreateNewVertices(const 
NCollection_List>& theGrVL, 
  ^~
  > 
>


If these problems persist after you're using C++11 mode, I guess the files 
mentioned will have to be changed in the ways indicated (in the opencascade 
port).

Re: MacPorts custom installation prefix and precompiled binaries

[Ryan Schmidt]

On Aug 7, 2022, at 08:05, Steven Smith wrote:

> Someday, one hopes that there will be a cleanup of all the tcl code that 
> breaks for prefix paths with spaces and non-ascii characters, but that’s 
> aspirational, not reality.

Many ports use build systems like autoconf that do not support paths that 
contain spaces or other special characters [1]. As such there's no point in 
spending time to fix MacPorts base or individual Portfiles to support paths 
that contain spaces or special characters.


[1] 
https://www.gnu.org/savannah-checkouts/gnu/autoconf/manual/autoconf-2.71/html_node/File-System-Conventions.html

> Autoconf uses shell-script processing extensively, so the file names that it 
> processes should not contain characters that are special to the shell. 
> Special characters include space, tab, newline, NUL, and the following:
> 
> " # $ & ' ( ) * ; < = > ? [ \ ` |
> 
> Also, file names should not begin with ‘~’ or ‘-’, and should contain neither 
> ‘-’ immediately after ‘/’ nor ‘~’ immediately after ‘:’. On Posix-like 
> platforms, directory names should not contain ‘:’, as this runs afoul of ‘:’ 
> used as the path separator.
> 
> These restrictions apply not only to the files that you distribute, but also 
> to the absolute file names of your source, build, and destination directories.
> 
> On some Posix-like platforms, ‘!’ and ‘^’ are special too, so they should be 
> avoided.

Re: SPOOLES MT routines

[Ryan Schmidt]

On Aug 5, 2022, at 17:45, Mark Brethen wrote:

> Note that the pre-build approach is overriding the makefile cflags, makefile 
> was patched to cflags += , etc. How do I prevent this?

Setting variables like CFLAGS as arguments (e.g. build.args) overrides anything 
specified in the Makefile. That's the purpose of specifying them as args.

If you don't want to override the Makefile, then specify them as environment 
variables instead (e.g. build.env).

Note that *.args and *.env have different quoting rules.

For *.args you would use e.g.

build.args  CFLAGS="foo bar baz"

For *.env you would use e.g.

build.env   "CFLAGS=foo bar baz"

You must also make sure that each environment variable is its own list item. In 
your portfile you're currently combining multiple different variables into a 
single string which isn't going to work for env. You'll want to use a list 
instead of a string. For example (untested):

set args[list "CFLAGS=${cflags}" "FFLAGS=${fflags}" 
"LDFLAGS=${ldflags}"]

build.args-append   {*}${args}

Re: Help with New Port - M2VDownsizer

[Ryan Schmidt]

On Aug 5, 2022, at 18:14, Robert Kennedy wrote:

> I have been playing around with CFLAGS += -Xarch_ppc -faltivec in my 
> Makefile, as suggested by Josh.
> 
> I noticed that if I include CFLAGS += -Xarch_ppc -faltivec in my Makefile, 
> the output from the clang compiler gets garbled but the M2VDownsizerbinary 
> appears to build fine.  
> E,g,
> /opt/local/var/macports/build/_Users_grinch_Macports_ports_multimedia_M2VDownsizer/M2VDownsizer/work/compwrap/cc/usr/bin/clang
>  -Os -arch x86_64 -Wall -g -Xarch_ppc -faltivec -Xarch_ppc64 -faltivec -I./ 
> -Iutils -Iutils/altivec -Impeg2dec -Impeg2enc 
> -I/System/Library/Frameworks/CoreServices.framework/Headers  -c 
> utils/mpegconsts.c -o obj/utils/mpegconsts.o
> 
> clang: clang: clang: clang: warning: warning: warning: warning: argument 
> unusedargument unusargument dured dur unused dinig ng compilation:uring '- 
> coacompilXarchatmpilio_ppatiornn: gu:c  -m''--faXXarletntciav rhch_ec_ppu'n
> usppc clang: warning: ar-faltedc guiv d-fmeecuralnt'
> ing compilatclang: warning: ioti uarn: '-Xarch_ppc -faltivec'
> clang: warning: argument unused during compilation: '-Xarch_ppc6nguvs4 uemece'
> d-faltivclang: nt decu'
>  unriwarning: argument unused during compilation: '-Xarch_ppc64 -faltivec'
> ng compilation: '-Xarch_ppc64 -faltivec'
> used during compilation: '-Xarch_ppc64 -faltivec'
> 
> I can avoid this garbled output if I turn off parallel builds by including 
> the following in the Portfile:
> 
> use_parallel_build    no
> 
> What is recommended in these circumstances considering that the garbled 
> output does not appear to affect building the final binary?  Should I turn 
> off parallel builds in the Portfile?

No, you should leave parallel building on because it speeds up builds on 
multiprocessor machines. If bugs are reported against a port and the error 
messages in the log are too garbled to decipher due to parallel building, ask 
the user to rerun the build with parallel building disabled.

Re: Issues with config.h

[Ryan Schmidt]

On Aug 5, 2022, at 16:43, Robert Kennedy wrote:
> 
> I have also patched config.h to address the ARCH issue.  It now reads:
> 
> #ifndef LINUX
>   #if defined(__ppc__) || defined(__ppc64__)
> #define ARCH_PPC
>   #elif defined(__i386__)
> #define ARCH_X86
>   #elif defined(__x86_64__)
>   #define ARCH_X86_64
> #elif defined ((__arm64__) && defined(__APPLE__)) || (__aarch64__)
> #define ARCH_ARM64
> #else
> #define ARCH_X86
> #endif
> #endif
> 
> But it looks like ARCH_PPC, ARCH_X86 is not used anywhere in the code.  It 
> might have been easier to just remove all this code.

Sure, that's how you could write defines for various architectures, but if 
they're not used in the code there's no point.

I see that ARCH_PPC is used in mpeg2dec/mpeg2_internal.h. I agree I don't see 
anywhere that ARCH_X86 is used.

> I will check whether the code uses WORDS_BIGENDIAN.  If not, I will just 
> patch out the problematic code:
> 
> #ifndef LINUX
> #define WORDS_BIGENDIAN 1
> #endif

I agree WORDS_BIGENDIAN is not used anywhere within this project's code, 
however it is a standard define name and I don't know whether it is used in any 
system headers that this project's code might be including.

Code that process endianness incorrectly will probably still compile and run 
but will generate incorrect data. So just the fact that it compiles doesn't 
mean it's right.


> My portfile already had PortGroup   makefile 1.0 so there was no need 
> to add it.

Ah, yes, I missed that. As you've now found, using the makefile portgroup means 
there's no need for you to list "use_configure no"; the fact that you had still 
listed "use_configure no" lead me to the incorrect assumption that you had not 
used the makefile portgroup.

Re: Issues with config.h

[Ryan Schmidt]

Early in config.h it says:


#ifndef LINUX
#if defined(__ppc__) || defined(__ppc64__)
#define ARCH_PPC
#elif defined(__i386__)
#define ARCH_X86
#endif
#else
#define ARCH_X86
#endif


So, on non-Linux systems, it sets ARCH_PPC when building for 32-bit or 64-bit 
PowerPC systems, it sets ARCH_X86 when building for 32-bit Intel systems, and 
it sets nothing when building for 64-bit Intel systems and all ARM systems. 
Since this software predates Apple Silicon systems you may have some work to do 
to patch it to work on those systems, depending on how ARCH_PPC and ARCH_X86 
are used in the code.


And these lines at the end of config.h are suspicious:


#ifndef LINUX
#define WORDS_BIGENDIAN 1
#endif


Certainly not all non-Linux systems are big-endian. Intel and Apple Silicon 
systems are little-endian.

Re: include files for cgxCADTools

[Ryan Schmidt]

On Aug 5, 2022, at 15:26, Mark Brethen wrote:
> 
> I don’t see these errors when building with macports-clang, only Apple’s 
> clang.

Ok. Certainly, different compilers can behave differently. I didn't try to see 
what the actual error was, since it was jumbled. If you have an non-jumbled 
error message you need help fixing, show us the error.

Re: Issues with config.h

[Ryan Schmidt]

On Aug 4, 2022, at 08:36, Robert Kennedy wrote:

> e.g.  
> ./config.h:29:9: warning: 'HAVE_STDINT_H' macro redefined
> #define HAVE_STDINT_H
> ^
> ./config.h:42:9: note: previous definition is here
> #define HAVE_STDINT_H 1

The config.h file clearly does define HAVE_STDINT_H (empty) on line 29 and then 
redefines it (with a value) on line 42, which is clearly wrong. Remove line 29 
and maybe the problem goes away.

The problem does seem to stem from the fact that config.h is getting included 
twice. For example one place that seems to happen is in mpeg2enc/fdctref.c 
which first includes mpeg2enc.h (which itself includes config.h) and then 
includes config.h again. The developer needs to decide whether it is valid to 
include config.h multiple times. If so, config.h needs to be written so that 
multiple inclusion works. Include guards are one way to do that. Otherwise, the 
developer needs to ensure that config.h only gets included once. Since the 
project is archived and the developer apparently no longer develops it, you get 
to decide.


On Aug 4, 2022, at 12:30, Robert Kennedy wrote:

> You will not be able to build the project without the Makefile and all the 
> Patches that I created.

You're right. This does make it a bit difficult to help.


> If you have any ideas on why the "macro redefined" warnings only appear when 
> the Project is being built by Macports without the "Includes guard" patch, 
> please let me know.

MacPorts sets many environment variables and flags and also applies certain 
protections like sandboxing. MacPorts also builds in parallel by default. These 
might account for such differences in behavior.


> And of course, please provide any feedback on the Portfile since I am a 
> newbie when it comes to writing Portfiles.

Since you are using "use_configure no", consider using the makefile portgroup; 
read it to find out how to use it.

You can delete the worksrcdir line since the value you've set it to is the 
default.

Since your distfile is unversioned, follow 
https://trac.macports.org/wiki/PortfileRecipes#unversioned-distfiles

Re: include files for cgxCADTools

[Ryan Schmidt]

On Aug 4, 2022, at 13:19, Mark Brethen wrote:

> I’m seeing this error when generate all the header dependencies for 
> FbdReader. This is confusing, which file has the error?

Error messages from several different files are being jumbled together due to 
parallel building. You can make it less confusing by disabling parallel 
building:

sudo port build use_parallel_build=no

Re: SPOOLES MT routines

[Ryan Schmidt]

On Aug 5, 2022, at 14:32, Mark Brethen wrote:
> 
> pre-build {
> set cflags  "${configure.cflags} [get_canonical_archflags cc]"
> set ldflags "${configure.ldflags} [get_canonical_archflags ld]"
> 
> # workaround for Rank mismatch between actual argument
> set ver [regsub -all {[[:alpha:]]+} ${compilers.gcc_default} {}]
> 
> if { ${ver} >= 10 } {
> set fflags "${configure.fflags} [get_canonical_archflags fc] 
> -fallow-argument-mismatch"
> } else {
> set fflags "${configure.fflags} [get_canonical_archflags fc]"
> }
> 
> set args"CFLAGS=${cflags} FFLAGS=${fflags} LDFLAGS=${ldflags}"
> 
> build.args-append   ${args}
> 
> # Build serial spooles library
> ui_info "Building spooles"
> system -W ${workpath}/spooles.2.2 "${args} ${build.cmd} lib"
> 
> # Extend library with multi-threading (MT) subroutines
> system -W ${workpath}/spooles.2.2/MT/src "${args} ${build.cmd} makeLib"
> }
> 
> 
> :info:build Building spooles
> :debug:build system -W 
> /opt/local/var/macports/build/_Users_marbre_ports_cad_CalculiX-ccx/CalculiX-ccx/work/spooles.2.2:
>  CFLAGS=-Os -arch x86_64 FFLAGS=-Os -m64 -fallow-argument-mismatch 
> LDFLAGS=-L/opt/local/lib -Wl,-headerpad_max_install_names 
> -Wl,-rpath,/opt/local/lib/libgcc -arch x86_64 /usr/bin/make lib
> :info:build sh: -arch: command not found
> 
> It worked when I patched the makefile with the flags, but these should 
> probably be passed at build. What did I miss?

Quoting?

set args"CFLAGS=\"${cflags}\" FFLAGS=\"${fflags}\" LDFLAGS=\"${ldflags}\""

Re: CalculiX is using libstdc++ (this installation is configured to use libc++)

[Ryan Schmidt]

On Jul 27, 2022, at 07:41, Mark Brethen wrote:
> 
> As the source does not use configure, I pass the compiler args:
> 
> pre-build {
>build.args-appendCXX=${configure.cxx} \
> CC=${configure.cc}
> }
> 
> For reasons I don’t understand, If I don’t put it in a pre-build override 
> it’s ignored.

Certainly not ignored, but conceivably not doing what you wanted.

If you just write:

   build.args-appendCXX=${configure.cxx} \
CC=${configure.cc}

outside of any phase, that takes effect the moment the Portfile is parsed. This 
will use whatever compiler value MacPorts chooses, modulated by any 
compiler.blacklist or similar setting you have set on lines before that line.

However, you mentioned the compilers portgroup, which allows you to set a 
compiler by choosing a variant. Variant blocks are parsed after the main part 
of the Portfile, so if you want any compiler modifications that might have been 
made by a variant block to be honored by these build.args, you have to do as 
you did and wait to set the build.args until right before you use them, e.g. in 
a pre-build block.

Re: openGL Framework

[Ryan Schmidt]

On Jul 23, 2022, at 05:38, Mark Brethen wrote:
> 
> if I switch to X11 server, should I remove /opt/X11 along with Xquartz.app?

If you are using the MacPorts xorg-server's X11.app and aren't using any other 
parts of XQuartz (such as its libraries) for any things outside of MacPorts, 
then sure, you can uninstall XQuartz. I don't know if simply deleting /opt/X11 
and /Applications/Utilities/XQuartz.app is sufficient to accomplish that. 
Consult the XQuartz documentation and/or web site or ask its developer.

Re: fetch timeout

[Ryan Schmidt]

On Jul 26, 2022, at 11:25, Mark Brethen wrote:
> 
> The mp curl no longer downloads from the target website, wias-berlin.de; web 
> download still works. The only changes were an Xcode update and reinstalling 
> commandlinetools and macports for big sur.

You'll have to talk to the administrators of that server to find out why it 
doesn't work properly.

Re: What to do with Mono (and msbuild) on ARM64?

[Ryan Schmidt]

On Jul 3, 2022, at 18:22, Mojca Miklavec wrote:

> One of the projects that I'm working on requires mono as a build dependency.
> At the moment mono is broken on ARM64 in MacPorts (I suspect that it
> works under HomeBrew).
> 
> I fixed it:
>https://github.com/macports/macports-ports/pull/14952/
> but there's a comment in the port saying
>> please update msbuild when updating mono
> and I'm unable to cope with that one (I don't know how to update msbuild).
> 
> While I can handle that local patch for myself, I would hate to send
> instructions to other developers working on macs explaining how to
> patch mono (or abandon the idea of using macports) if they want to
> build the sources.
> 
> Would it be acceptable to upgrade just mono without msbuild to avoid
> having a broken mono on M1?

Perhaps the maintainer of the msbuild port can update it.

Re: How about a MacPorts "system" (generic) python port?

[Ryan Schmidt]

On Jul 15, 2022, at 03:04, Chris Jones wrote:

> we already have a solution to this, for users that 'just want a python3'
> 
> > sudo port select python3 python310
> 
> replacing python310 with your preferred version.
> 
> I see no need for what you propose above in addition.

He wants something that can be used by ports. "port select" is for users to 
use, not for ports to use.

Re: openGL Framework

[Ryan Schmidt]

On Jul 19, 2022, at 17:18, Mark Brethen wrote:

> I’m also confused about the two x11 window environments (xorg-server vs 
> xquartz). I’m assuming Xquartz libraries, etc are in /opt/local, what about 
> xorg?

XQuartz is a standalone package that installs a complete X11 system, including 
the X11 server application XQuartz.app into /Applications/Utilities and the 
supporting X11 libraries into /opt/X11.

The xorg ports in MacPorts are the same software but you can pick and choose 
what you want to install, and we might have newer versions of some of the 
software than XQuartz since we update ports individually whereas XQuartz is a 
monolithic distribution. We call the X11 server application X11.app and install 
it into /Applications/MacPorts or whatever your applications_dir is set to and 
we install the X11 libraries into /opt/local or whatever your prefix is set to.

Both XQuartz and the xorg ports in MacPorts are maintained by Jeremy Huddleston 
Sequoia.

Any ports in MacPorts that need X11 libraries will use the ones provided by 
MacPorts X11 ports, not those provided by XQuartz.

If you want to run an X11 server, you can use either one, but most MacPorts 
users probably choose the MacPorts xorg-server port since that's one less thing 
to need to update separately and you don't end up with redundant copies of X11 
libraries.

Re: fetch timeout

[Ryan Schmidt]

On Jul 15, 2022, at 09:55, Chris Jones wrote:

> On 15/07/2022 3:49 pm, Mark Brethen wrote:
>> -rw-r--r--1 root  wheel  346545 Jan  1  2020 cert.pem
> 
> The above could be your problem, as that is very old, 2.5 years or so now. It 
> actually pre-dates the public release of macOS 11, which wasn't until 
> November that year, which makes it quite suspicious...
> 
> In comparison mine is from May this year, on macOS12. I would imagine the 
> same on macOS 11 to be much more up to date than the above.

I don't believe the above indicates a problem with Mark's system but rather a 
strangeness of macOS 11. On both an Intel MacBook Pro and on an Apple Silicon 
Mac mini I have access to, the creation and modification date of the macOS 11 
disk volume and all of its system files is and has always been Jan 1, 2020 
midnight California time. The same is not true of macOS 12 volumes on the same 
machines.

Re: bazel 4.2 and 3.7 no longer building with native clang

[Ryan Schmidt]

On Jul 13, 2022, at 11:46, Steven Smith wrote:

> How does one reconfigure the bazel Portfile to use a previous version of 
> clang that has a chance of working to build previous versions of bazel, which 
> are in turn necessary to build the tensor flow packages.

If certain compilers that MacPorts might choose will not build a port, add 
their names to compiler.blacklist and MacPorts will choose another compiler.

If you only want to exclude certain versions of a compiler (such as {clang < 
700 >= 900}), include the compiler_blacklist_versions portgroup.

If you've excluded all compilers MacPorts might choose, add a suitable compiler 
to compiler.fallback.

Re: Using on macOS 10.13 with clang++-mp-13

[Ryan Schmidt]

On Jul 3, 2022, at 13:38, Ken Cunningham wrote:

> You can (sometimes) use macports-libcxx to get past this. This however 
> introduces two versions of libc++.dylib into the softare mixture, and this 
> sometimes causes errors due to incompatibilities between libc++ versions (ODR 
> errors). If you are taking this route, linking in the newer libc++.a 
> statically has a higher chance of working properly (that is what Chrome does 
> on older MacOS systems).

If a port X links to a static library provided by a port Y, you should increase 
the revision of port X whenever port Y is updated to a new version so that your 
port X receives the new version of the static library. A comment should be left 
in port Y advising whoever updates it to do that.

Re: Compiler Selection

[Ryan Schmidt]

On Jul 3, 2022, at 13:39, Mark Brethen wrote:
> 
> According to the wiki:
> 
>> The compiler is chosen with the following precedence:
>> 
>>  • If configure.compiler is set in the Portfile, then that is the 
>> compiler that is used.
>>  • If the user sets default_compilers, the compiler is chosen from that 
>> list unless the compiler is blacklisted.
>>  • If the Portfile sets compiler.whitelist, the compiler is chosen from 
>> that list unless the compiler is blacklisted. 
>>  • If the Portfile sets compiler.fallback, the compiler is chosen from 
>> that list unless the compiler is blacklisted. 
>>  • A compiler is chosen (Xcode compilers if possible) that
>>  • is not blacklisted,
>>  • is known to work on the particular version of the OS, and
>>  • satisfies the requirements set in the Portfile.
> CalculiX requires gcc + fortran. I used ‘configure.compiler macports-gcc-11’ 
> (it will not build with clang). But this seems too rigid. My initial goal was 
> to set the latest version as default with the option of trying earlier 
> versions of macports-gcc. I tried using compilers PortGroup but could not get 
> that to work—I haven’t figured out the proper setup. Is it possible? I read 
> that variants are no longer recommended for this.

If you must build with gcc, it is customary to offer variants to let the user 
choose which version of gcc, from among the set of compatible versions of gcc 
which you specify. It is my understanding that the purpose of the compilers 
portgroup is to assist you with that task but I have no further information 
about it. Read the comments in the compilers portgroup for assistance or check 
how other ports use this portgroup.

Re: A bunch of failures with nodejs on the builders

[Ryan Schmidt]

On Jul 2, 2022, at 19:11, Marc Wilson wrote:
> 
> I’ve not heard of this before, and have never experienced it, despite having 
> two different Mac on 12.x and before that, on 11.x.

Macs running macOS 12 do not seem to be affected presumably because they're 
already running macOS 12; there would be no reason to put the macOS 12 
installer on them. Apple is only force-downloading the macOS 12 installer on 
some earlier systems; it is unclear whether this is a bug or is in some 
perverse way intentional on their part. Apple has not responded to the bug 
report I filed in March.

I noticed that both of the macOS 11 buildbot workers received the full 12 GB 
macOS installer application again, despite my having previously followed 
Chris's suggestion of removing the dmg from within them. Chris's workaround is 
effective so far on the macOS 10.15 buildbot worker but I don't know if that's 
just because it hasn't checked for updates recently enough.


> Surely you have upgrades disabled in System Preferences?

In System Preferences -> Software Update, "Automatically keep my Mac up to 
date" is unchecked. In Advanced..., "Check for updates" is checked (because I 
do want to be notified when new macOS updates are available), "Download new 
updates when available" is unchecked*, "Install macOS updates" and "Install app 
updates from the App Store" are grayed out and unchecked, and "Install system 
data files and security updates" is checked (because I do want the malware 
protection to be updated automatically).


* It was unchecked on the macOS 11 x86_64 buildbot worker and it is my 
intention to uncheck it on every buildbot worker but I see it was actually 
checked on the macOS 11 arm64 worker; I've now unchecked it there.

Re: A bunch of failures with nodejs on the builders

[Ryan Schmidt]

On May 5, 2022, at 07:59, Christopher Nielsen wrote:

>> In addition, as you may know, a bug in recent macOS causes it to download 
>> the 12+ GB macOS Monterey installer periodically which would have left it 
>> with only 8 GB available.
> 
> I recently discovered this behavior as well.
> 
> My solution is to leave the installer in place - ensuring it’s not 
> auto-downloaded again - while removing ‘SharedSupport.dmg’ from within the 
> app bundle. This frees up the vast majority of storage space, without 
> invalidating the installer itself.

This method has not been effective.

Re: Frotz build targets

[Ryan Schmidt]

On Jun 27, 2022, at 17:47, Nils Breunese wrote:
> 
> I don’t maintain any ports with GUI’s myself, but I believe flavors like x11, 
> sdl, etc. are usually provided as variants of a port instead of as subports. 
> You can also specify conflicts between variants if there are any to avoid 
> users installing conflicting variants. For instance the glib2 port contains 
> 'variant x11 conflicts quartz { … }’.

Many ports currently use variants for this, yes, but partly that's because they 
got done that way before the subport feature existed. Separate ports (whether 
implemented as subports within a single Portfile or as separate Portfiles) are 
better because, if done correctly, any number of those ports can be installed 
simultaneously, and we can build binaries of them on our automated buildbot 
system. (We only build ports' default variants.) This in turn is preferred by 
users, who would rather not have to compile things from source just because 
they want a different set of options compiled in.

The freeciv and freeciv-x11 ports are examples of this, and Chris has recently 
indicated a desire to undertake the restructuring necessary to do the same 
separation for glib, gtk, and everything depending on them; see 
https://trac.macports.org/ticket/63680.

Re: openGL Framework

[Ryan Schmidt]

On Jun 30, 2022, at 07:50, Mark Brethen wrote:
> 
> I noticed that -framework OpenGL exists in /System/Library/Frameworks. Is 
> there an OpenGL framework port that I should be using?

The MacPorts equivalent of the OpenGL framework is the mesa port.

Re: Frotz build targets

[Ryan Schmidt]

On Jun 26, 2022, at 17:39, Mark Brethen wrote:
> 
> Attaching a Portfile that uses variants—downside is you can only install one. 
> Maybe that is sufficient? I couldn’t find any documentation for using 
> subports.

What would you like to know about subports? Have you looked at any existing 
portfiles that use subports to see how they work?

Re: Frotz build targets

[Ryan Schmidt]

On Jun 26, 2022, at 14:23, Mark Brethen wrote:

> But is there an interest in building for X11 anymore?

I do not know about frotz, but in MacPorts ports in general, yes, there is 
still interest in X11. Ports that have optional X11 components can make them 
available in an +x11 variant.

Re: Python+MacOS Versions

[Ryan Schmidt]

On Jun 24, 2022, at 19:47, Joshua Root wrote:

> On 2022-6-25 05:47 , Jeremy Lavergne wrote:
>> Does anyone happen to know what versions of Python each MacOS shipped with?
>>  * I'm specifically hoping to find when python3.8 began being pre-installed 
>> on MacOS to update a PSPP build-dep that's only required on systems without 
>> python3.8.
>> https://github.com/macports/macports-ports/blob/master/math/pspp/Portfile#L48
>>* Also hoping to make a wiki table with known versions for future 
>> reference.
> 
> It should be possible to find the version of python shipped with each version 
> of macOS by looking at the tags at 
> . For example, clicking through 
> to python from 
> ,
>  you can see that 10.6.8 shipped with Python 2.6.1 (and some older versions).
> 
> As mentioned previously, python3 is part of the developer tools, and 
> unfortunately the corresponding source doesn't seem to be available anywhere 
> that I can find. I think the /usr/bin/python3 stub first appeared in 
> Catalina, with the actual python installation in Xcode 11, and was 3.7.3 at 
> the time, but don't quote me on that.

Agreed, there is no /usr/bin/python3 on Mojave. On Catalina, /usr/bin/python3 
--version returns "Python 3.7.3".

I didn't realize python3 wasn't part of the OS and was provided by Xcode. I 
guess that means that, unlike python2 on earlier systems, if a port needs 
python3 at runtime, it must use a MacPorts python3x port and not rely on the 
system python.

Re: Difference between build_arch and configure.build_arch, universal_archs and configure.universal_archs (was: Re: +universal for x864+arm64 on Macintel)

[Ryan Schmidt]

On Jun 15, 2022, at 16:49, Joshua Root wrote:

> $universal_archs contains exactly what is set in macports.conf (so does 
> $build_arch). ${configure.universal_archs} is the archs that will actually be 
> built with +universal, and may differ from the macports.conf value due to the 
> supported_archs of the Portfile and of the chosen SDK. (That is also the case 
> for ${configure.build_arch}.)

Thanks Josh, I need to print that out and frame it and put it on my wall so I 
don't forget again.

Re: GitHub Actions macOS-10.15 image deprecation

[Ryan Schmidt]

On Jun 15, 2022, at 23:24, aeiouaeiouaeiouaeiouaeiouaeiou wrote:
> 
> https://github.com/actions/virtual-environments/issues/5583
> 
> > GitHub Actions is starting the deprecation process for macOS 10.15. While 
> > the image is being deprecated, You may experience longer queue times during 
> > peak usage hours. Deprecation will begin on 5/31/2022 and the image will be 
> > fully unsupported by 8/30/2022
> 
> They also made the Monterey image available to all users, but I cannot test 
> it due to the lack of the necessary CI files 
> (https://github.com/macports/macports-ci-files/releases/tag/v2.7.2).

Josh added the file for Monterey.

Re: help with a regex please

[Ryan Schmidt]

On Jun 19, 2022, at 16:34, Fred Wright wrote:

> I don't know specificlly about reinplace, but here's an example with sed:

reinplace is sed -i ""

https://github.com/macports/macports-base/blob/master/src/port1.0/portutil.tcl#L951-L953

Re: +universal for x864+arm64 on Macintel

[Ryan Schmidt]

On Jun 14, 2022, at 12:33, René J.V. Bertin wrote:

> On Monday June 13 2022 04:34:46 Ryan Schmidt wrote:
> 
>> The standard universal variant has no content. (Universal support is 
>> implemented by adding the return value of procedures like 
>> [get_canonical_archflags cc] to CFLAGS.) Many ports implement their own 
>> universal variant by manually creating this empty universal variant 
>> ("variant universal {}") and using get_canonical_archflags where needed.
> 
> Bummer, looks like "base" is trying to be too smart: the manually created 
> variant is removed after the Portfile has been parsed, "due to < 2 supported 
> universal_archs".

Base is exactly smart enough. When supported_archs contains only 1 arch, it 
does make sense to offer a universal variant, therefore base prevents it.

Re: +universal for x864+arm64 on Macintel

[Ryan Schmidt]

On Jun 13, 2022, at 15:40, Chris Jones wrote:

> On 13 Jun 2022, at 1:06 pm, René J.V. Bertin wrote:
> 
>> Ryan Schmidt wrote on 20220613::04:34:46 re: "Re: +universal for x864+arm64 
>> on Macintel"
>> 
>>> As far as I know, the SDK would also need to support arm64, which SDKs 
>>> prior to macOS 11 don't. Therefore MacPorts doesn't support compiling for 
>>> macOS for arm64 prior to macOS 11.
>> 
>> What Apple call the SDK is essentially the combination of standard 
>> headerfiles (incl. the system-specific ones) plus the standard libraries 
>> (including the ones for system functions). Is there any reason to suspect 
>> that one cannot use the 10.11+ SDKs on earlier OS X versions?
> 
> Macports only supports buidling against  the sdk you would ‘naturally’ get 
> installing the Xcode/CLT version supported on a given OS. Whilst it might be 
> possible in a hacking way to install a newer SDK on alder OS, this is not 
> something macports can realistically support.

I added the MacOSX.sdk port specifically so that we could begin to explore the 
notion of using SDKs on OS versions whose Xcode/CLT versions do not provide 
those SDK versions. There are a number of situations where this might be 
helpful.

For example, the QuickTime framework headers were removed from the SDK at some 
point, though the QuickTime framework itself remained on the system for several 
versions after that so as not to break software that had been built with 
earlier SDKs. By using an earlier SDK, software that needs the QuickTime 
framework can be built on some later systems.

Another use case is that much software requires the latest SDK to build, even 
though it can be deployed to much earlier systems; now we have a possible 
avenue to allow building such software on those older systems. libsdl2 is an 
example of a port that can be built on 10.6 but doing so requires the 10.7 SDK 
(which was not provided in any Xcode version on 10.6) so the port guides the 
user through a manual process of installing the 10.7 SDK outside of MacPorts. 
My SDK port provides a hopefully more streamlined method of doing that within 
MacPorts.

There is no support in MacPorts base (or via a portgroup) yet for using such an 
SDK; it would have to be programmed into the port manually. I had submitted a 
PR to add such support to base, but it proved to be incomplete and I did not 
pursue completing it yet.

Re: +universal for x864+arm64 on Macintel

[Ryan Schmidt]

On Jun 13, 2022, at 04:29, René J.V. Bertin wrote:
> 
> On Monday June 13 2022 02:48:06 Ryan Schmidt wrote:
>> (and if MacPorts does not create a universal variant automatically in this 
>> case then the port should do so itself)
> 
> Doh, I should have thought about that. Still, it would be more elegant to 
> handle that situation with something like `universal_variant force` 
> (admitting that the logic isn't perfect).

The standard universal variant has no content. (Universal support is 
implemented by adding the return value of procedures like 
[get_canonical_archflags cc] to CFLAGS.) Many ports implement their own 
universal variant by manually creating this empty universal variant ("variant 
universal {}") and using get_canonical_archflags where needed.


> I suppose pre-10.11 installs should still be able to cross-compile for arm64, 
> with the right compiler and libraries installed, no?

As far as I know, the SDK would also need to support arm64, which SDKs prior to 
macOS 11 don't. Therefore MacPorts doesn't support compiling for macOS for 
arm64 prior to macOS 11.

> (Link step included btw, as all normally built clang compilers are 
> cross-compilers so a new-enough clang should be able to generate the correct 
> object code for Apple silicon.)

Re: MacPorts Status

[Ryan Schmidt]

On Jun 11, 2022, at 09:07, Jordan wrote:

> First of all thank you for your work! I use MacPorts exclusively for a few 
> different reasons and am looking to contribute to the project. I am starting 
> to spin up maintaining a few ports after a long break from macOS (mostly 
> because until M1 machines came around the hardware sucked if I am honest).

Welcome!


> There is one aspect of MacPorts that has me a little down in the dumps and 
> that's is its _apparent_ (but likely not true) neglected state. I think it's 
> perhaps under maintained currently due to a lack of people-power? I say so 
> because I'd love to see more people using MacPorts so I wonder if there's any 
> "modernisation" (for lack of a better word) to be had here?

It is true that there have always been many more things that could be done to 
improve MacPorts than there are people available to do those things.

But I am not certain what specifically you're referring to here. There are 
dozens of commits to MacPorts each day, several tickets filed each day, 
frequent discussions occur on the mailing lists. What do you perceive to be 
neglected?


> Things like the timeline on Trac are years old,

I'm not sure what timeline you're referring to. I see there is a Timeline 
button on the Trac toolbar, which I think I have probably never used before 
today, but clicking it now, it appears to be up to date with the latest changes 
from today:

https://trac.macports.org/timeline


> the use of `sudo` by default probably scares new users etc etc.

The use of sudo increases security and protects the user from potentially 
malicious changes to their files. I am not aware that the use of sudo scares 
new users, but if it does, then improved education of new users is the solution.


> I'm looking to contribute to some kind of effort to "modernise" (again, for 
> lack of a better word) MacPorts so perhaps some usage compared to things like 
> Homebrew can be gained back since I'd love to see this project used more.

Since most of us are not familiar with Homebrew, perhaps you could describe 
exactly what aspects of it you'd like to see adopted by MacPorts. But of course 
Homebrew and MacPorts are different projects with different goals written in 
different programming languages by different people so while we may borrow 
ideas from one another here and there they probably won't become very similar.

I'm all for making MacPorts easier to use, and there have been countless 
previous threads on the mailing list discussing ways to do that. But of course 
as with any volunteer-based open-source project, nothing actually gets done 
until somebody volunteers to do it, where "do it" includes writing the code, 
yes, but for significant or potentially controversial changes you probably want 
to have a discussion on the mailing list first to gain consensus that your 
proposed change should be made, to save you the time of writing code if that 
code is implementing something that wouldn't be accepted anyway.


> Also, as a tacked-on issue I am working on packaging a port and MacPorts is 
> smart enough to only remove the files within directories as present when the 
> port was imaged but I was hoping for a cheeky setting to change this so that 
> it "dumb"ly deletes the entire folder.
> 
> For example, the port is imaged and at the time of imaging has a 
> `${prefix}/lib/foo/bar` directory with a single file in it. If I immediately 
> uninstall the port then `bar` is correctly deleted since it is _exactly_ as 
> it was when the port was installed from its image. However, if I add files to 
> `bar` then MacPorts only removes those that single file that is part of the 
> port image, leaves the others alone, and because `bar` is now not-empty 
> leaves `bar`. I wonder if I can force MacPorts to always delete `bar`?

When a port is installed, only the files that it installs are tracked in the 
registry. Directories themselves are not tracked. When deactivating a port, if 
no files are left in a directory, the directory is deleted, otherwise it is 
kept.

You can write pre- and post- activate and deactivate blocks to do any 
additional tasks before or after MacPorts activates or deactivates a port. For 
example, there are several font ports that update the fontconfig cache after 
activation. It is not normal for a port to delete files it didn't install, but 
it is possible to program the port to do that, and I could imagine some use 
cases for that. Carefully consider whether that is the right thing to do. What 
are these files that you are deleting? Consider that when a user upgrades a 
port, that includes deactivating the old version of the port and activating the 
new version. Will the new version want to have those files you're deleting? 
Where did the files come from? Did the user create them? If so, the user may 
not expect MacPorts to delete them.

Re: +universal for x864+arm64 on Macintel

[Ryan Schmidt]

On Jun 13, 2022, at 02:58, Chris Jones wrote:

> I assume you have tried removing these two line from your portfile ?
> 
> 

You're pointing to the `universal_variant no` line and the `supported_archs 
x86_64` line.

Certainly, if the port installs files of more architectures than x86_64, those 
other architectures should be in the supported_archs line too.

The port also specifies `use_configure no` later, which is one of the 
conditions I was referring to previously which causes MacPorts not to create a 
universal variant (so the `universal_variant no` line is redundant). If a 
universal variant is desired for a port that does not have a configure phase 
(and does not use the cmake or makefile or xcode portgroups) the port must 
implement the universal variant itself.



>> On Jun 10, 2022, at 11:23, René J.V. Bertin wrote:
>>> When I tried to test the new +universal variant of a x864_64 + arm64 port 
>>> (port:VLC) installing from official DMGs on my 10.9.5 Mac I discovered that 
>>> the variant wasn't added because the port only supported the current build 
>>> architecture.

Also, just a reminder, MacPorts does not complain if you ask to install a port 
with a variant that does not exist.

Re: +universal for x864+arm64 on Macintel

[Ryan Schmidt]

On Jun 10, 2022, at 11:23, René J.V. Bertin wrote:

> When I tried to test the new +universal variant of a x864_64 + arm64 port 
> (port:VLC) installing from official DMGs on my 10.9.5 Mac I discovered that 
> the variant wasn't added because the port only supported the current build 
> architecture. Whaaat? It took me a while to realise that maybe "base" knew I 
> can't build arm64 binaries (supposedly; never tried) and thus ignored the 
> arm64 entry in the port's supported_archs list. The variant did appear when I 
> added i386 support (bogus in this case as the universal DMG doesn't contain 
> i386 binaries).
> 
> Such intelligence from "base" is probably a good thing but somewhere I feel 
> that it's inappropriate in this port's kind of situation. The universal DMG 
> exists after all, and my port installs from there, not by building anything. 
> Is there a way to override the logic and force the +universal variant?
> 
> The problem isn't necessarily entirely moot; port:VLC installs a library that 
> can be used by 3rd party (non-MacPorts) applications, and it doesn't strike 
> as impossible to build arm64 binaries on a system where MacPorts considers 
> that to be impossible.

In my view, MacPorts base is behaving correctly and doesn't need any changes, 
though your port might.

MacPorts base knows which architectures are supported by your OS version. OS X 
10.9 supports i386 and x86_64. macOS 11 is the first version that supports 
arm64.

MacPorts also knows which architectures are supported by a port based on its 
supported_archs line. If such a line is not present, all architectures are 
assumed to be supported.

The intersection of those two lists of architectures is the list of 
architectures for which the port can be installed.

If the number of items in the list of architectures for which the port can be 
installed is greater than one, and if certain other conditions are met, then 
MacPorts creates a universal variant. Otherwise it does not.

A port that installs from an upstream binary should set supported_archs to the 
list of architectures for which that binary was compiled. If that list of 
architectures is greater than one, then on some systems this port will offer a 
universal variant (and if MacPorts does not create a universal variant 
automatically in this case then the port should do so itself) and the port 
should handle that correctly: if the universal variant is requested by the 
user, the port should install files with those architectures. In the simple 
case of a 2-architecture universal binary, the port can just install the binary 
files unmodified. But if the user does not request the universal variant, then 
the port should install files of only the architecture that was requested, for 
example by using lipo to thin the universal binary. The libxl port shows one 
way this can be done.

Re: Fastly Mirrors Throttled to 1 MB/sec?

[Ryan Schmidt]

On Jun 6, 2022, at 08:54, Christopher Nielsen wrote:
> 
> Over the past few months, I’ve found that downloads from 
> ‘packages.macports.org’ and ‘distfiles.macports.org’ are consistently 
> throttled at 1 megabyte/second. And while that isn’t an issue for smaller 
> payloads, it significantly impacts binary installation/update times for 
> larger ports like Rust, the various LLVM ports, etc: The vast majority of 
> time is spent waiting for the download to occur, with the installation 
> occurring in a few seconds.
> 
> Cancelling the download via Ctrl-C results in the next mirror being selected 
> - which for me, is generally ‘ywg.ca.packages.macports.org’ - resulting in a 
> fast non-throttled download.
> 
> So the $100,000 question is: Are our Fastly-based mirrors intentionally 
> throttled at 1 megabyte/second?
> 
> If that’s intentional - particularly to reduce hosting costs - then no 
> worries. But just figured I’d ask, in case that’s not supposed to be 
> occurring.

Yes, it has always been that slow for me as well, which is part of the reason 
why I had not formally announced the fact that we are using this CDN. I need to 
ask them if there is some mistake in our configuration that is causing this.

Re: Reason for rpath usage on arm64

[Ryan Schmidt]

On May 30, 2022, at 01:01, Ken Cunningham wrote:

> ah yes, your question was arm64-specific so that led to the gcc assumption.
> 
> the way the cmake PG configures cmake breaks testing on all archs and systems.

It worked for me on Catalina x86_64 but not on Big Sur arm64, and I was trying 
to understand why this difference exists.


> this comes up regularly, eg
> 
> https://lists.macports.org/pipermail/macports-dev/2021-September/043754.html
> 
> Perhaps it might beneficially be changed.

I'm assuming that just changing the default in the cmake 1.1 portgroup could 
cause problems (or at least nonreproducibility) for ports already using the 
portgroup, so perhaps this change, plus any others we want to make to the cmake 
portgroup, should be in a new cmake 1.2 portgroup.

Re: privoxy-pki-bundle not Behaving as Desired – Request for Assistance

[Ryan Schmidt]

On May 24, 2022, at 15:31, Daniel J. Luke wrote:

> On May 24, 2022, at 3:56 PM, Ryan Schmidt wrote:
>> I wouldn't recommend anyone begin writing any code until it is discussed how 
>> the feature should work. That should avoid spending time writing code that 
>> won't work.
> 
> sure, if someone were working on a feature it would be reasonable for them to 
> discuss it here ... (but again, no on is).
> 
>>> - but there are of course possibilities here. We could (ab)use epoch,
>> 
>> epoch can't be used because epoch is not part of the archive filename, and 
>> because it is within the portfile author's control when to change the epoch.
> 
> then epoch can't ever be used for anything because it has the same 'fatal' 
> problem (if I bump epoch in a port I maintain, and use a version + revision 
> that there's already a package for ... I guess we tell people "don't do that" 
> - but there's nothing preventing someone from doing it by mistake)

Correct, don't do that.

epoch can be used if you use it for what it's for.

For example, consider you have a port foo with epoch 0, version 2.0, revision 
5. Archives have been built, users have installed them, it works fine.

Now you update the port to epoch 0, version 3.0, revision 0. Archives get 
built, some users upgrade to this version, and they report that it does not 
work. Nobody knows why, and to avoid this problem, you want to downgrade the 
port to the last working version. To do this, you set version back to 2.0, set 
revision back to 5, and increase epoch from 0 to 1. Any users who had already 
upgraded to version 3.0 revision 0 will be downgraded back to version 2.0 
revision 5. In doing so, they may receive the previously-built version 2.0 
revision 5 archives from the server, which are still fine. Users who never 
upgraded to version 3.0 revision 0 are not prompted to reinstall anything since 
that would be unnecessary.

Some time later a solution to the 3.0 problem is found and you wish to upgrade 
the port to 3.0 with a patch to fix the problem. To do this, you leave epoch at 
1 (as you will forever unless at some time in the future you again wish to make 
the version decrease), set version to 3.0 again, set revision to 1 this time 
(since revision 0 of 3.0 was bad), new archives will be built, users will 
upgrade, etc.


>>> just have portindex increment the revision for the ports that request to be 
>>> rebuilt when a new version of something is added,
>> 
>> How? The PortIndex file is generated by extracting certain information from 
>> each port. Right now, an up-to-date PortIndex file contains the fact that 
>> privoxy-pki-bundle has epoch 0, version 3.0.33, revision 3, because that's 
>> what it said in the privoxy-pki-bundle Portfile when that PortIndex entry 
>> was written. Suppose tomorrow I update curl-ca-bundle to a new version. By 
>> what means is the portindex program supposed to know that now, it should 
>> record revision 4 in the PortIndex file entry for privoxy-pki-bundle? If you 
>> are suggesting that it should read the existing entry from the PortIndex 
>> file, increment revision, and write it back, that won't work because it is 
>> valid to delete the PortIndex file and regenerate it from scratch.
> 
> Because it's impossible to change how portindex works? It's impossible to 
> store any state other than state that's already stored? I'm purposely not 
> writing up a comprehensive design here - since I've not volunteered to 
> implement it and I don't think it's worthwhile to do an exhaustive feature 
> design if no one is doing an implementation. (and also, if someone else 
> decides to implement it, I don't want to constrain them since I haven't 
> really dug into the details).
> 
> I feel like the few times we've discussed this we talk past each other 
> because I'm operating on the assumption that whoever implements a new feature 
> isn't constrained in what parts of MacPorts they would modify while you 
> appear to be focused on what would break with the smallest possible 
> implementation (and by extension, why that feature isn't possible to 
> implement).
> 
> Fundamentally the current situation is that we have cases where this kind of 
> dependency exists. We deal with it now by adding a comment + having a human 
> do something. I'm just saying that software could be written to do the right 
> thing here and I would welcome the automation. Implementation would, of 
> course, take time and require consideration of all of the other pieces of 
> MacPorts (and infra) that it touches.

When I said "How?" I was trying to convey that I could not imagine a way to 
accomplish what you propose. And your answer is that you won't discuss it. So 
we remain at an impasse.

Your comment that we shouldn't bother designing a feature if nobody is 
implementing it seems exactly backwards to me. Obviously a design is needed 
first before anyone would consider implementing anything.

Re: Reason for rpath usage on arm64

[Ryan Schmidt]

On May 24, 2022, at 10:22, Ken Cunningham wrote:
> 
>> Does anyone know why things seem to be using rpath on arm64 builds
> 
> Basically, it comes down to the fact that Apple has disallowed 
> DYLD_LIBRARY_PATH to work on newer systems.

> So gcc is installing it’s libraries with the @rpath linkage

So you're talking about how SIP doesn't pass DYLD_LIBRARY_PATH to child 
processes on El Capitan and later, which breaks the gcc test suite, therefore 
gcc now uses @rpath at build time to fix the problem, and does not rewrite the 
install_names at install time, therefore things with @rpath get installed.

That's not the issue that prompted me to start this thread.

When I enabled the test suite for the portmidi port (which does not use gcc)...

https://github.com/macports/macports-ports/commit/fd447d210e1814b1303c5013d088efa673470d77

...I set "DYLD_LIBRARY_PATH=." in test.env so that the executable it runs can 
find the library it just built. This worked fine on Catalina x86_64 but on Big 
Sur arm64 it failed with the error message:

dyld: Library not loaded: @rpath/libportmidi.2.dylib
  Referenced from: /path/to/portmidi/work/build/./qtest
  Reason: image not found

Seeing "@rpath" in this error message, I added 
"-DCMAKE_BUILD_WITH_INSTALL_RPATH=OFF" to configure.args (to counteract 
"-DCMAKE_BUILD_WITH_INSTALL_RPATH=ON" set by the cmake portgroup) and the 
problem went away.

Perhaps placing the blame on @rpath was the wrong conclusion, since I now 
realize that the install_names also use @rpath on Catalina x86_64 yet the test 
worked there. I just don't understand then why disabling @rpath on Big Sur 
arm64 made the tests work.

It seemed to me like I was seeing a lot of @rpath-related issues especially on 
arm64 lately, and this latest instance with portmidi prompted me to ask about 
it, but it could be that most or all of the other instances were related to the 
gcc situation you already discussed. I know we are using a different branch of 
gcc for arm64 so maybe that explains why the @rpath-related changes you 
mentioned only appear on arm64 so far.

Re: privoxy-pki-bundle not Behaving as Desired – Request for Assistance

[Ryan Schmidt]

On May 24, 2022, at 14:08, Daniel J. Luke wrote:

> On May 24, 2022, at 12:07 PM, Ryan Schmidt wrote:
>> I'm not convinced that I would be happy to see such a feature. Just a 
>> moment's consideration shows how many different parts of MacPorts would need 
>> modifications to support it, not only in base but also in the buildbot and 
>> in the package hosting infrastructure. And in the end, what we end up with 
>> is a rather fundamental violation of a principle we currently rely on, which 
>> is that a given version and revision of a port is (unless there is a bug) 
>> reproducible, immutable, deterministic.
> 
> The assumption you're making is that any possible implementation will be bad? 
> ;-)

No, just that with a few moments of thinking about the proposed feature, I 
could foresee several problems that would arise, and I wanted to convey some of 
those to at least communicate that it's a complex problem to solve. If somebody 
thinks this feature is so valuable that they want to devote time to solving the 
problems and coming up with a workable solution that's fine. Perhaps with more 
than a few moments' thought that would be possible.


> I don't think it's worth bikeshedding this when no one has done any work - 
> but if we step back and consider that all we'd be trying to do is automate 
> the process you already go through when reading a comment and rev-bumping a 
> port, it doesn't seem like it's a noncomputable problem.

I don't think the term bikeshedding is applicable here at this time. 
Bikeshedding refers to devolving into discussions about unimportant details. I 
don't believe I was doing that.

One possibility is to just introduce a new portfile keyword (the 
rebuild-if-these-ports-were-updated keyword) that MacPorts base doesn't 
actually use (just like base recognizes but does not use known_fail) but that 
the buildbot or another server process could use to increase the revision of 
the appropriate port and commit it. Then all other existing processes would go 
into effect to build and deploy the new package, update the portindexes, etc. 
Of course this opens up the possibility of difficulties in recognizing where 
the revision keyword is in any arbitrary portfile, especially one that has 
subports or one that does not specify the revision keyword. It's the same 
problem we encounter whenever we want to increase the revision of ports that 
depend on a library whose major version has increased. It's the reason why we 
now recommend all ports and subports specify their revision even when it is 0 
but not all portfiles do that yet and even if they did it can still be 
difficult to identify programmatically which revision keyword to increase when 
there's more than one.


> This is assuming that any possible implementation would result in 
> non-reproducible builds, which I also wouldn't support.

I didn't mean to suggest that builds would not be reproducible, just to point 
out that if we reuse archive filenames then depending on the freshness of the 
file on whichever server the user happens to reach they may receive the new 
correct file or an old incorrect one. Therefore it seems desirable not to reuse 
archive filenames, and the question then becomes how to differentiate the 
filenames.


> On the other hand, if no one is actually doing the work /and/ prominent 
> project leaders are saying they could never support such a feature, it's 
> unlikely that anyone will ever try to make improvements in this area, so it's 
> probably not even worth time defining what it would take for it to be useful 
> and added to MacPorts.

I don't want my comments to dissuade anyone from working on anything in 
MacPorts that they wish to work on. Again I just wanted to share the initial 
problems that I perceived with this proposed feature.


>> Or perhaps the proposal is that package filenames should grow even longer 
>> with an additional identifier -- which would hopefully be empty for ports 
>> not opting in to this new thing so that all our existing packages are not 
>> invalidated. If so, what would the new identifier be?
> 
> There isn't really a proposal yet

It is proposed in this thread that MacPorts could have a certain feature.


> (I don't think anyone is working on code)

I wouldn't recommend anyone begin writing any code until it is discussed how 
the feature should work. That should avoid spending time writing code that 
won't work.


> - but there are of course possibilities here. We could (ab)use epoch,

epoch can't be used because epoch is not part of the archive filename, and 
because it is within the portfile author's control when to change the epoch.


> just have portindex increment the revision for the ports that request to be 
> rebuilt when a new version of something is added,

How? The PortIndex file is generated by extracting certain informatio

Re: privoxy-pki-bundle not Behaving as Desired – Request for Assistance

[Ryan Schmidt]

On May 24, 2022, at 09:35, Daniel J. Luke wrote:

> - MacPorts could grow a feature by which a port could specify that it needs 
> to get rebuilt if something it depends on gets rebuilt (this would probably 
> require another identifier along with epoch-version-revision or would require 
> some magic behavior with one of the existing versioning numbers)

> So far, when people encounter this problem, there hasn't been enough 
> motivation for anyone to build a MacPorts feature to support it (but I'd be 
> happy to see one).

I'm not convinced that I would be happy to see such a feature. Just a moment's 
consideration shows how many different parts of MacPorts would need 
modifications to support it, not only in base but also in the buildbot and in 
the package hosting infrastructure. And in the end, what we end up with is a 
rather fundamental violation of a principle we currently rely on, which is that 
a given version and revision of a port is (unless there is a bug) reproducible, 
immutable, deterministic.

For example, imagine that at 1pm curl-ca-bundle is updated to a new version and 
privoxy-pki-bundle uses this hypothetical new feature that makes it reinstall 
when curl-ca-bundle is updated. To get there, we'd have to modify the registry 
to store what version of curl-ca-bundle was installed at the time that 
privoxy-pki-bundle was installed, and we'd have to extend the portindex to hold 
information about which ports should trigger a reinstall for another port, and 
we'd have to enhance base's understanding of outdatedness to make use of this 
new information, and enhance "port outdated" to display that information. And 
suppose we have modified the buildbot to know that it must rebuild 
privoxy-pki-bundle and upload a new package. (Currently it decides whether to 
build a new package in part based on whether a package with the correct 
filename already exists on the server, which now won't be sufficient criteria.) 
It takes awhile for newly uploaded packages to sync from the private server to 
the public one, so let's say by 2pm the new curl-ca-bundle and 
privoxy-pki-bundle packages have synced to the public server (although there's 
no guarantee that they'd both be there at the same time -- the new 
curl-ca-bundle package, having been built first, might arrive 30-60 minutes 
ahead of privoxy-pki-bundle). Our mirrors sync from the public server at 
varying intervals, so let's say that by 10pm the new packages are on all the 
mirrors. Prior to that time, old packages of the same filename might still be 
on some mirrors. What happens if a user upgrades to the new curl-ca-bundle at 
5pm? If the new curl-ca-bundle package is available on any mirror they access, 
that'll be used, otherwise the new curl-ca-bundle will be built from source. 
Fine. Then MacPorts will want to reinstall privoxy-pki-bundle, even though its 
version and revision are the same, due to the new hypothetical feature. It will 
check for a package on a mirror and it will find one! But will it be an old 
package or a newly rebuilt one? No one knows.

If the proposed workaround for this is that ports using this new reinstall 
feature should program themselves never to use prebuilt packages, then that's 
another reason to dislike the new feature. (Users like receiving binaries and 
not having to build things from source.) Or perhaps the proposal is that 
package filenames should grow even longer with an additional identifier -- 
which would hopefully be empty for ports not opting in to this new thing so 
that all our existing packages are not invalidated. If so, what would the new 
identifier be?

Reason for rpath usage on arm64

[Ryan Schmidt]

Does anyone know why things seem to be using rpath on arm64 builds? For example:

https://trac.macports.org/ticket/64204

The de-facto standard for things installed with MacPorts has been not to use 
rpath, since MacPorts predates the existence of rpath, and since things have 
worked fine without rpath, and since things often break when rpath starts being 
used. I am assuming that someone somewhere thinks that using rpath is a good 
idea now, perhaps especially on arm64. Anyone know the reason?

Re: privoxy-pki-bundle not Behaving as Desired – Request for Assistance

[Ryan Schmidt]

On May 23, 2022, at 18:35, Jim DeLaHunt wrote:

> 1. port curl-ca-bundle is a subport of port curl[2],
> 2. it looks to me like port curl-ca-bundle only got updated about 7 times in 
> the last 43 months[3]
> 3. it looks as if port curl-ca-bundle updates a file within curl,
> security/nss/lib/ckfw/builtins/certdata.txt, and does some make and
> install operations. Maybe building and installing curl itself does
> those same operations.
> 4. Thus, maybe a version update to port curl also functions as an
> update to port curl-ca-bundle, but I'm not sure.

The curl ca bundle is installed by the curl-ca-bundle port, not by the curl 
port. The curl port depends on the curl-ca-bundle port.

> If #4 is true, it makes me wonder if maybe port curl-ca-bundle's values for 
> certdata_updated (epoch time) and certdata_date should get updated each time 
> the curl version gets updated.

No, they shouldn't. certdata_updated and certdata_date are to be changed only 
when curl-ca-bundle is updated to a new version of certdata.txt. 
certdata_updated is used by the portfile to make MacPorts fetch certdata.txt 
from mozilla only within a short period after it was updated; after that time 
the port assumes MacPorts has mirrored the file and it then fetches only from 
MacPorts mirrors. This is to reduce the load on the mozilla server.

Re: privoxy-pki-bundle not Behaving as Desired – Request for Assistance

[Ryan Schmidt]

On May 23, 2022, at 21:16, Chris Jones wrote:

> On 23 May 2022, at 9:59 pm, Steven Smith wrote:
> 
>>> What has changed between the time that the buildbot built the package and 
>>> the time that the user installs it?
>> 
>> The certs in curl-ca-bundle are updated regularly to clear out expired certs.
>> 
>> Per the previous discussion, privoxy-pki-bundle uses these certs via a 
>> depends_lib, and unless a port revision is added by hand, the port 
>> inevitably will contain expired certs.
>> 
>> The “solution” appears to be to bump the revision of privoxy-pki-bundle by 
>> hand whenever curl-ca-bundle is updated. I’m trying to identify a more 
>> automated and robust way of accomplishing that.
> 
> The simple solution then is to just put a comment into the curl-ca-bundle 
> port next to the version/revision asking whomever updates it to bump the 
> revision of privoxy-pki-bundle at the same time. This simple but generally 
> effective solution is used in a number of ports with similar situations and 
> works well most of the time. I see no need to do anything more complex here, 
> particularly not to automate things such that the same port file installs 
> different things at different times. That lack of reproducibility is 
> definitely not wanted.

Right, this is what I already recommended. I'm happy to revbump 
privoxy-pki-bundle whenever I update curl-ca-bundle, but will forget if not 
reminded via a comment.

I do see that privoxy-pki-bundle depends on 
path:share/curl/curl-ca-bundle.crt:curl-ca-bundle, which means that certsync 
could also satisfy it. Whereas curl-ca-bundle is updated by me whenever mozilla 
releases a new certdata.txt, certsync installs a launchd plist that monitors 
the user's Keychain and whenever it is modified, it runs a program to generates 
a new ca bundle. If you intend to handle that situation, then you could do 
something similar and install a launchd plist that monitors the ca bundle and a 
program to regenerate your files whenever the ca bundle changes. Then revbumps 
would not be needed.

Re: privoxy-pki-bundle not Behaving as Desired – Request for Assistance

[Ryan Schmidt]

On May 22, 2022, at 13:43, Steven Smith wrote:

> A standard install command grabs pre-built stuff from 
> https://packages.macports.org/privoxy-pki-bundle. This pre-built stuff is 
> inevitably stale because it inevitably contains expired certs.

Again, why is it the case that what we built on the buildbot contains expired 
certs? Why would having the user build the port locally result in non-expired 
certs being used? What has changed between the time that the buildbot built the 
package and the time that the user installs it?

> We want to port to install stuff, just not pre-built expired stuff.
> 
> The desired behavior is to always install from “source,” i.e. the behavior 
> that goes with “port -s install” which always installs the latest up-to-date 
> PKI. 
> 
> How does one write a Portfile to prevent installs from being cached at 
> https://packages.macports.org/privoxy-pki-bundle ? 

It is possible to do that but I hesitate to tell you how because I am almost 
certain it is not the correct solution to whatever problem you are having.

Re: privoxy-pki-bundle not Behaving as Desired – Request for Assistance

[Ryan Schmidt]

On May 21, 2022, at 12:40, Steven Smith wrote:

> Is there a way to achieve the second objective: prevent MacPorts from 
> building and caching its own copy of the port contents, which for this port 
> will inevitably be filled with expired certificates?
> 
> Right now, avoiding this issue requires installing the port from “source” 
> with `port -s install`.

Again I don't understand the question.

If you don't want the port to install anything, write the destroot phase to do 
that. Except that a port must install at least one file. But this doesn't sound 
like what you're asking.

Why will what the port installs inevitably be filled with expired certificates? 
What would be the cause of that happening?

If the port's contents should be changed (in the binaries produced by the 
buildbot and on user systems) then change the port to install the new contents 
and increase its revision.

Perhaps there is something unique about this software that I don't understand. 
(I have no idea what this software is.)

Re: cargo.crates_github doesn't work anymore in the cargo_fetch PG

[Ryan Schmidt]

On May 18, 2022, at 07:26, Vadym-Valdis Yudaiev wrote:

> Looks like after some changes `cargo.crates_github` broke. The @harens 
> thought the `cargo._old_macos_compatibility` option was to blame. I do think 
> it broke even earlier.
> 
> https://github.com/macports/macports-ports/pull/13909#issuecomment-1059773013
> 
> Anyway, for the `joshuto` port, I need cargo crate from Github, which I can't 
> get anymore. For now, I've created a draft PR and will be waiting for a fix 
> in PG to continue.
> 
> https://github.com/macports/macports-ports/pull/14865

Sounds like https://trac.macports.org/ticket/65025

Re: privoxy-pki-bundle not Behaving as Desired – Request for Assistance

[Ryan Schmidt]

On May 17, 2022, at 15:31, Steven Smith wrote:

>> Whenever the curl-ca-bundle port is updated to a new version, the 
>> privoxy-pki-bundle port's revision should be increased so that it rebuilds 
>> with the new bundle.
> 
> 
> Thank you.
> 
> This is the part that I was hoping is automatic, without updating a revision: 
> the depends_lib would see that the “library” that the port depends upon has 
> been updated, and rebuilds itself because of the updated library dependency. 
> All without modification of the Portfile.
> 
> I infer from your response that this isn’t how depends_lib works.

Correct, that's not how it works. If you want a port to be rebuilt, increase 
its revision.

You may be thinking of a MacPorts feature called rev-upgrade which is designed 
to help users not encounter some types of breakage when someone forgets to 
increase a port's revision. The only type of breakage this deals with is 
dynamically linked libraries being missing: for example, a port is updated to a 
new version which provides a new major version of the library which therefore 
has a different filename but the revision of the ports that link with the 
library was not increased. As a stopgap until the revisions are increased, 
MacPorts will notice the problem and rebuilt the ports from source on the 
user's machine to get things working again.

Re: privoxy-pki-bundle not Behaving as Desired – Request for Assistance

[Ryan Schmidt]

On May 12, 2022, at 18:04, Steven Smith wrote:
> 
> I tried to write privoxy-pki-bundle to have these two basic behaviors:
> 
>   • Updates to the file ${prefix}/share/curl/curl-ca-bundle.crt via port 
> curl-ca-bundle will force reinstallation of the CA file 
> ${prefix}/etc/privoxy/CA/trustedCAs.pem with the new CAs.
>   • port install privoxy-pki-bundle will generate this CA file from 
> source; MacPorts will not use a buildbot to create and save an (inevitably 
> outdated) file of CAs to be installed when this port is installed.
> 
> Apparently, neither of these behaviors is working, as I see from the recent 
> update of curl-ca-bundle.
> 
> I’d appreciate any pointers to get this port configured and working the way 
> it should to make sure that CAs are automatically updated, and old CAs aren’t 
> installed.

I don't understand exactly what you're asking for. It sounds like you're saying 
the privoxy-pki-bundle port would like to have the curl-ca-bundle.crt file that 
the curl-ca-bundle port installs converted to a different format. If so, write 
the privoxy-pki-bundle port so that it has depends_build port:curl-ca-bundle 
and performs the conversion and installs the converted file. Whenever the 
curl-ca-bundle port is updated to a new version, the privoxy-pki-bundle port's 
revision should be increased so that it rebuilds with the new bundle. A comment 
can be added to the curl-ca-bundle port to remind people to do that when 
curl-ca-bundle is updated.

Re: A bunch of failures with nodejs on the builders

[Ryan Schmidt]

On Apr 25, 2022, at 07:54, David Gilman wrote:
> 
> There were a handful of pushes of new nodejs versions (at least nodejs
> 18 through 15) in the past week and a bunch of binary builders failed.
> Some of these look ephemeral and just need xcode reinstalled or
> whatever. I'd appreciate it if the admins could do that. There is also
> an exotic hard build failure.
> 
> On 10.6 and macOS 11 the build ran out of disk space.

I don't know if you meant 10.6 x86_64 or 10.6 i386. At the moment, 10.6 x86_64 
has 66.7 GB available and 10.6 i386 has 68.7 GB available. I cannot conceive of 
a build that could take that much space so I would not believe an "out of disk 
space" message here.

nodejs15, 16, 17, 18 are marked as "known fail" on OS X 10.8 and earlier, 
therefore the build system will not attempt to build them on those OS versions. 
If you were looking at build status information on ports.macports.org, it would 
be from the last time the build was attempted before "known fail" was added to 
them (and "known fail" was presumably added to them because nobody wanted to 
figure out how to fix whatever the problem was).

I looked at build logs for nodejs13 and some older nodejs versions from 10.6 
and it was complaining about the version of python. You could file bug reports 
in the issue tracker to get those ports fixed to use the right python, or 
whatever the problem is.


You didn't say whether 11 x86_64 or 11 arm64 but looking at the logs I assume 
11 x86_64:

https://build.macports.org/builders/ports-11_x86_64-builder/builds/70121/steps/install-port/logs/stdio

fatal error: /Library/Developer/CommandLineTools/usr/bin/libtool: can't write 
to output file (No space left on device)

At the moment, 11 x86_64 has 20.2 GB available. This might not be enough to 
build the largest ports especially if the build is memory intensive and some of 
the space gets used for swap. In addition, as you may know, a bug in recent 
macOS causes it to download the 12+ GB macOS Monterey installer periodically 
which would have left it with only 8 GB available. Looks like the automatic 
Monterey installer download happened most recently on 4/18 and that build 
failure log is from 4/20. I did already remove the installer but I guess it 
wasn't until after 4/20 since the log shows that after cleanup only 8 GB was 
available:

https://build.macports.org/builders/ports-11_x86_64-builder/builds/70121/steps/cleanup/logs/stdio

So retrying the builds now should succeed. I've triggered builds of any unbuilt 
nodejs ports on 11 x86_64 (which turned out to be only nodejs18):

https://build.macports.org/builders/ports-11_x86_64-watcher/builds/18511

The build succeeded.


11 arm64 had 25.2 GB available. I deleted the unrequested Monterey installer 
and now it has 38.7 GB free.


> On 10.15 there were missing xcode installation related errors.

All builders have of course always had Xcode and the Xcode command line tools 
installed. However, as you may know, a bug in recent macOS causes it to delete 
the CLT installation receipt at some point:

https://trac.macports.org/wiki/ProblemHotlist#reinstall-clt

The MacPorts cltversion portgroup and apparently some parts of the nodejs build 
use to determine information about the CLT. The conditions which cause the CLT 
receipt to be deleted are not known. Performing software updates may be one of 
the conditions. I checked and indeed the 10.15 builder's CLT receipt has once 
again disappeared, so I have once again reinstalled the CLT to put it back and 
scheduled new builds of any unbuilt nodejs ports on 10.15 (which turned out to 
be nodejs17 and nodejs18):

https://build.macports.org/builders/ports-10.15_x86_64-watcher/builds/28623

As of now, the nodejs17 build just completed successfully and the nodejs18 
build is beginning.


> On 10.7, 10.8 and 10.9 linking failed while linking
> node-v10.24.1/out/Release/torque
> 
> Undefined symbols for architecture x86_64:
>  "_macports_legacy_sysconf", referenced from:
>  v8::base::OS::AllocatePageSize() in libv8_libbase.a(platform-posix.o)
>  v8::base::OS::Allocate(void*, unsigned long, unsigned long,
> v8::base::OS::MemoryPermission) in libv8_libbase.a(platform-posix.o)

You're right, nodejs10 does have that build failure on 10.7, 10.8, 10.9, and 
10.6 as well. Sounds like a defect either in our legacy support or in the way 
this port is using it. Please file a ticket in the issue tracker. I don't see 
"-lMacportsLegacySupport" among the flags being given to the compiler, which is 
probably necessary to fix this. This flag is already in LDFLAGS but the build 
system may be ignoring it.

go ports fetching dependencies at build time

[Ryan Schmidt]

Question regarding this commit:

https://github.com/macports/macports-ports/commit/2c8bec5807d23120038f39d292a7e1cad625707d

Here, a port tfmigrate was updated from 0.3.2 to 0.3.3 and changed to allow it 
to fetch dependencies at build time.

What transpired between 0.3.2 and 0.3.3 that made it necessary to allow the 
port to do its own fetching again? This seems to happen to various go ports 
with some frequency. What can we do to improve the situation so that this does 
not become necessary so often?

Re: Location to store binaries

[Ryan Schmidt]

On Apr 17, 2022, at 08:45, Marcus Calhoun-Lopez wrote:

> I have been careful to submit all changes upstreams.
> So far, most of them have been merged already.
> The only changes I have not submitted are the ones that apply to systems 
> prior to 10.7, which Rust does not support.
> These patches are small and can easily be kept as patch files within the port.
> 
> However, that is not the reason for the fork.
> The fork would *only* be for the Release feature of gitbub that allows the 
> upload of binaries.
> Specifically the upload of the bootstrap compilers for unsupported systems.

I understand that. If you were able to submit all of your fixes to upstream, 
then presumably it would be possible for them to build the next version of 
their bootstrap compiler in a way that was compatible with older systems and 
then maintaining your fork would not be necessary. That's why I asked.

Re: Location to store binaries

[Ryan Schmidt]

On Apr 16, 2022, at 10:36, Marcus Calhoun-Lopez wrote:

> Right now, I put them in a release section of a fork of Rust:
> https://github.com/MarcusCalhoun-Lopez/rust/releases
> 
> Could a similar fork be created under the auspices of the MacPorts project?

Yes, it could be. We haven't done that for any other port before so I'm not 
sure why it should happen for rust now, but yes, it could be done.

I think forks don't preserve releases, pull requests, issues, etc. If you would 
like those things preserved, you could transfer your repo to the MacPorts 
organization.

All of this presumes that upstream will not accept whatever changes you've 
made. Have we already confirmed that that is the case?

Re: macOS 12 x86_64 buildbot weirdness

[Ryan Schmidt]

On Apr 8, 2022, at 10:06, Herby Gwrote:

> Something weird might be going on with the macOS 12 x86_64 buildbot
> 
> Two builds of the same latest Rust fail with completely different errors:
> 
> 1. 
> https://build.macports.org/builders/ports-12_x86_64-builder/builds/26275/steps/install-port/logs/stdio
> 2. 
> https://build.macports.org/builders/ports-12_x86_64-builder/builds/26314/steps/install-port/logs/stdio
> 
> Run #1 fails signal 6 (SIGABRT?):
> rustc exited with signal: 6
> 
> Did not run successfully: signal: 6
> error: could not compile `openssl-sys`
> 
> Run #2 fails due to missing object file (maybe the process got sig'd mid-run 
> before it could write out the object file?):
> error: could not write output to 
> /opt/local/var/macports/build/_opt_bblocal_var_buildworker_ports_build_ports_lang_rust/rust/work/rustc-1.60.0-src/build/x86_64-apple-darwin/stage2-tools/x86_64-apple-darwin/release/deps/rayon_core-9641c1e2ac352716.rayon_core.ce832f4d-cgu.2.rcgu.o:
>  No such file or directory
> 
> error: could not compile `rayon-core` due to previous error
> Either way, something odd is happening.  Maybe the machine should have some 
> diagnostics run on memory and CPU.

I do not think it relates to the issue you mentioned, but there has been some 
network instability today, causing the Apple Silicon buildbot worker to 
disconnect from the master several times and causing some other builders not to 
start builds after being unable to connect to GitHub. I restarted any failed 
builds. The 10.6 i386 worker also experienced one of its customary kernel 
panics during a build, and I rebooted it and the build was reattempted.

I restart all build machines periodically to try to avoid problems. Most of 
them (except for 10.6 i386) were restarted yesterday.

The machine that runs the macOS 12 VM is the same Xserve that runs several 
other VMs, so I do not suspect any hardware problems. Xserves use 
error-correcting memory.

I'll try another build of rust now.

https://build.macports.org/builders/ports-12_x86_64-builder/builds/26360

Re: revision control downloads

[Ryan Schmidt]

On Mar 23, 2022, at 16:31, Rainer Müller wrote:
> On 22/03/2022 22.23, Ryan Schmidt wrote:
> 
>> As I said we do already and have for a long time strongly discouraged using 
>> revision control systems for downloads.
> 
> As these days most web frontends allow to download a tarball, we could
> also think about discouraging fetching from VCS completely.

Obviously wherever that option exists we should be using it already. The github 
portgroup for example makes that easy to do for projects hosted at GitHub.

There are many ports that fetch from git because they need git submodules (as 
you mentioned), which those automatically-generated tarballs do not include. 
Ideally the developers of those projects would provide release downloads; if 
they do, we should obviously be using those (e.g. with the github portgroup's 
"github.tarball_from releases").

Some projects have complicated build processes which include fetching from git 
somewhere in them. Aseprite is an example of that.

Some git hosting providers seem to be unable to provide reproducible tarball 
download capability, despite the fact that GitHub and others have been able to. 
GoogleSource.com is an example. See https://github.com/google/gitiles/issues/84

Re: revision control downloads

[Ryan Schmidt]

On Mar 22, 2022, at 13:08, Daniel J. Luke wrote:

> On Mar 21, 2022, at 9:20 PM, Ryan Schmidt wrote:
>> Ports that fetch their sources from a revision control system do not enjoy 
>> the protection of checksums. Although ports that fetch source from a 
>> revision control system specify which tag or commit hash to fetch, it is 
>> conceivable that a developer with sufficient access to that repository could 
>> delete an old tag and replace it with a new tag of the same name that 
>> contains different software. This is one of the reasons why we recommend 
>> ports fetch using distfiles, and the vast majority do. We might consider 
>> recommending that ports that fetch directly from a git repository 
>> (fetch.type git) never use a tag, and always use the commit hash 
>> corresponding to that tag, since replacing the contents of the repository 
>> while keeping the same sha1 hash is, as far as I know, still impossible in 
>> the general case. (Yes, it is possible to engineer a sha1 collision, but 
>> only if you can carefully control both the old and new files. Generating a 
>> sha1 collision against some existing old file is a very different matter.)
> 
> I haven't reviewed the current literature but one thing is certain - attacks 
> only get better. We should strongly discourage any use of revision control as 
> the source.

As I said we do already and have for a long time strongly discouraged using 
revision control systems for downloads.


> We could shorten the window where a person could get sources that don't match 
> what the maintainer expected with a little infrastructure magic. Roughly in 
> order of presumed level of effort:
> 
> - provide a place for maintainers to manually upload distfiles that they can 
> generate from a source checkout
> - provide some port command for maintainers to upload distfiles (as a 
> convenience for the above upload process)
> - have a process that automatically generates distfiles from the checked out 
> source and puts it on our mirrors (builder could probably do this since it's 
> already going to check out the source), then have base look for the distfile 
> on our mirrors first (even if the portfile specifies to check out from a 
> revision control system)

Rainer did some work on automating the creation of distfiles from revision 
control systems but it has not been touched in some years. Since Rainer hasn't 
been very active here lately maybe someone else would like to pick it up and 
finish the work.

https://github.com/macports/macports-base/tree/vcs-fetch

Re: Codesigning everything and combatting malicious code

[Ryan Schmidt]

On Mar 21, 2022, at 23:02, Joshua Root wrote:

> We could ad-hoc codesign everything, which would not improve security at all, 
> but would get GateKeeper to ease up a bit on restrictions on incoming network 
> connections and the like.

> Assurance that binaries have not changed after being installed would be nice 
> I suppose.

Ok, those could be useful qualities. A binary could change because of malicious 
actions or just because the disk it's stored on is failing. Preventing the 
binary from being used in either of those cases is probably good.

If we did ad-hoc codesiging in MacPorts then we would not need to use any 
Developer ID and we would have the same safeguards for Intel and PowerPC 
systems that we already have automatically on Apple Silicon systems. Would that 
be of any benefit?

"man codesign" says "Significant restrictions apply to the use of ad-hoc signed 
code; consult documentation before using this." I don't know which 
documentation it means. I found this page discussing the restrictions:

https://apple.stackexchange.com/questions/288291/what-are-the-restrictions-of-ad-hoc-code-signing

Most of the answer provided there is incomprehensible to me except for the 
conclusion that an ad-hoc signed binary is "basically [...] the same as a 
non-signed binary". Are we sure that ad-hoc codesigning is enough to pacify 
GateKeeper? Since all binaries must be codesigned on Apple Silicon, does that 
mean that GateKeeper never has anything to complain about on Apple Silicon 
systems?

The code signature appears to be stored within the file being codesigned. (I 
ad-hoc codesigned an unsigned binary produced by MacPorts and its size 
increased by 25K.) What happens if file is completely replaced? I assume the 
previous file's code signature does nothing to prevent that. We have at times 
had the situation where a user runs a third-party installer that installs older 
files on top of the files MacPorts installed, causing various problems. But I'm 
guessing codesigning wouldn't do anything to prevent that situation, in which 
case codesigning doesn't really prevent modification of the file in the way 
that one might expect.


> Codesigning is a in the end just a mechanism, and there are policy questions 
> that need to be thought through before it can be useful.

Yup, and thanks for raising some of those questions in your response!

Codesigning everything and combatting malicious code

[Ryan Schmidt]

Sorry, this got long.

I want to move a discussion to the macports-dev list that began with a user's 
question to macports-mgr. This user ran a third-party utility which reported 
that some files installed by MacPorts were not codesigned and asked if it was a 
concern.

I replied that the files installed by the MacPorts installer are codesigned but 
that MacPorts predates the existence of codesigning and nobody has yet added 
code to MacPorts that arranges for files installed by ports to be codesigned. 
Individual ports might codesign their files if that's how their build system 
was written. On Apple Silicon all (code?) files must be codesigned and the 
linker (?) takes care of ad-hoc codesigning the files automatically. I also 
said I wasn't really clear on what purpose codesigning serves; MacPorts and 
macOS got by fine without it for years.

The user replied asking if codesigning might reduce the risk of compromised 
open source packages, referencing this incident in which the developer of 
node-ipc deliberately released a version containing malicious code:

https://www.schneier.com/blog/archives/2022/03/developer-sabotages-open-source-software-package.html

This is a hypothetical question since node-ipc is not in MacPorts and as far as 
I know a similar situation has not happened with any software that we do have 
in MacPorts. And since then node-ipc appears to have been forked by a different 
developer and a new non-malicious version has been released.

I felt that the broader MacPorts developer community might care to hear a reply 
to this or to discuss whether and how we should modify MacPorts base to 
codesign everything installed by ports.

~

Certainly the situation with node-ipc was undesirable but I don't see how 
MacPorts codesigning its files would solve it. I'll talk about that in a 
minute. First let me discuss what safeguards MacPorts already contains.

MacPorts ports already fetch specific maintainer-tested versions of the source 
code. If a developer releases a new version, nothing changes in MacPorts until 
a MacPorts contributor updates the Portfile to use that new version. Before 
doing so the MacPorts contributor is expected to verify that the port at least 
compiles on one macOS version; that the port's test phase runs, if the port has 
one; that "port lint" doesn't show any easily fixable mistakes; and ideally 
that some basic functionality of the installed files works. If the port has 
subports, this applies to all subports as well.

MacPorts already checksums distfiles. The MacPorts master build server saves a 
copy of distfiles (assuming the checksum matches at the time) and these are 
mirrored on servers around the world. If a developer replaces a distfile on 
their server with something else (maliciously or not), MacPorts will not 
install the port using that distfile; it will issue a checksum mismatch error 
and tell the user how to report it to us. Or MacPorts might download the 
previous "good" file from a MacPorts mirror, bypassing the developer's new 
replacement file. When we notice such a "stealth update" has occurred, proper 
procedure is to compare the old and new file to see whether something malicious 
has happened so that we can decide whether we should modify the port to use the 
new file or to bypass it. (We might also bypass a new developer distfile if it 
contains no relevant differences from the old one.)

Ports that fetch their sources from a revision control system do not enjoy the 
protection of checksums. Although ports that fetch source from a revision 
control system specify which tag or commit hash to fetch, it is conceivable 
that a developer with sufficient access to that repository could delete an old 
tag and replace it with a new tag of the same name that contains different 
software. This is one of the reasons why we recommend ports fetch using 
distfiles, and the vast majority do. We might consider recommending that ports 
that fetch directly from a git repository (fetch.type git) never use a tag, and 
always use the commit hash corresponding to that tag, since replacing the 
contents of the repository while keeping the same sha1 hash is, as far as I 
know, still impossible in the general case. (Yes, it is possible to engineer a 
sha1 collision, but only if you can carefully control both the old and new 
files. Generating a sha1 collision against some existing old file is a very 
different matter.)

Some ports' build systems fetch additional files from the Internet at build 
time. This too is discouraged in MacPorts. Who knows whether such build systems 
download specific versions of those files or whether their integrity is 
verified in any way. This represents another avenue by which unverified files 
(which might have been replaced with malicious versions) could make their way 
into a build of a port. And if that one server that the build system is 
programmed to download from is down, the installation fails. Ports should 
specify all their 

Re: python2.7 pip install fails with SSLError

[Ryan Schmidt]

On Mar 17, 2022, at 14:49, Steven Smith wrote:

>> macOS 12.3 removes /usr/bin/python
> 
> This package encapsulates everything in a virtualenv, so that wasn’t the 
> issue. In case anyone runs into something similar, my logs say this the 
> removal of /usr/local/lib/.Python is the issue:
> 
>> dyld[545]: Library not loaded: @executable_path/../.Python
>>  Referenced from: 
>> /private/var/calendarserver/Library/CalendarServer/virtualenv
>> /bin/python
>>  Reason: tried: 
>> '/private/var/calendarserver/Library/CalendarServer/virtualenv/bin/../.Python'
>>  (no such file), '/usr/local/lib/.Python' (no such file), '/usr/lib/.Python' 
>> (no such file)

Hmm, I'm not familiar with any ".Python" but I don't know Python very well.

macOS doesn't put anything in /usr/local/lib so the reference to that can 
probably be ignored.

Re: python2.7 pip install fails with SSLError

[Ryan Schmidt]

On Mar 17, 2022, at 14:37, Steven Smith wrote:

> Thanks everyone for your responses and suggestions.
> 
> Short story: I used the existing `sudo port install calendar-contacts-server` 
> and … it works!
> 
> Even though this project is old and archived, I’m not aware of any open 
> source that replaces its functionality, and it’s now fully reliant on a 
> MacPorts stack, so I expect it will work indefinitely now.
> 
> The long story and the reason for my initial query: I had originally 
> implemented a working instance without MacPorts, then wrote the Portfile to 
> get a MacPorts version, then adopted the “if it’s not broke don’t fix it” 
> attitude with my original, non-MacPorts Calendar server. So I kept that 
> running until this week when there was some breaking change in the macOS 12.3 
> Python environment.

macOS 12.3 removes /usr/bin/python.

https://lists.macports.org/pipermail/macports-dev/2022-February/044121.html


> I tried a quick manual install and ran into this issue, and believe I got 
> tangled up in stuff I had to fix a couple years ago to get the MacPorts 
> version working. IIRC the solution involves setting the correct environment 
> variables, as done in the Portfile.
> 
> There’s one little issue I ran into, and I’d appreciate a suggested fix. The 
> pip installer wants to see sqlparse==0.2.0 in the virtualenv, which it 
> installs correctly, but for some reason breaks later when it finds a 
> MacPorts-installed version of py27-sqlparse. The solution is to just 
> uninstall this port. How should this conflict best be expressed in the 
> Portfile?

If this failure occurs at build time, use "PortGroup conflicts_build 1.0" and 
"conflicts_build py27-sqlparse".

Re: python2.7 pip install fails with SSLError

[Ryan Schmidt]

On Mar 17, 2022, at 04:39, Chris Jones wrote:

> Does the port *have* to use python 2.7 ? Is it not compatible with python 3.x 
> ?

See https://github.com/apple/ccs-calendarserver/issues/537 where Steven asked 
upstream for guidance on migrating to Python 3. There was no reply. The project 
is no longer being developed.

Re: python2.7 pip install fails with SSLError

[Ryan Schmidt]

On Mar 14, 2022, at 20:18, Jim DeLaHunt wrote:

> On 2022-03-14 17:09, Steven Smith wrote:
>> I’m trying to update calendar-contacts-server to macOS 12.3. This requires 
>> installing a bunch of stuff with python2.7’s pip.
>> I’m hitting an SSLError at the command:
>>> /opt/local/bin/python2.7 -m pip install -I --user setuptools==18.5
>> Is there a way of pointing MacPorts python2.7 pip at a set of up-to-date 
>> CAs? [Say e.g. from curl-ca-bundle?]
> 
> I recall discussion recently on either the macports-dev or the macports-user 
> lists about SSL tools failing due to expiry of Let's Encrypt certificates. I 
> can't locate a link to those threads quickly, and I'm not sure if it applies 
> to your situation.
> 
> Have a look at
> 

Steven should not be affected by those issues since he uses macOS 12.3 and "on 
macOS 10.15 and later and on macOS 10.12, no fixes appear to be needed".

Re: OpenSSH 8.9p1 deprecated variants cleanup feedback request

[Ryan Schmidt]

Just a brief reply,

I agree that we don't usually want to be in the business of forever maintaining 
patches, so they should be sent to the upstream developers for inclusion, and 
if they refuse them, then we either keep maintaining the patches (if they are 
valuable and someone wants to continue maintaining them) or we remove them. So 
far someone has wanted to maintain this openssh patch. I don't use it so I 
don't know how valuable it is.

It's not uncommon for someone to forget to delete a patchfile when they remove 
the reference to it from a Portfile. If you find such a situation, you can 
delete the unused patchfiles. There might be a very small subset of those cases 
where the unreferenced patchfile was deliberately retained on the theory that 
it might be needed again in the near future. But deleting it is fine since if 
it is needed again it can always be retrieved from the repository history.

Re: LLVM port mp-llvm-* select groups mislinks certain programs

[Ryan Schmidt]

On Mar 14, 2022, at 14:02, Christopher Jones wrote:

> You can easily see the issue with
> 
>> sudo port -d select llvm mp-llvm-13
> 
> the linkage starts out in sync, but then starts to diverge..
> 
> 
> DEBUG: ln -sf /opt/local/bin/llvm-symbolizer-mp-13 
> /opt/local/bin/llvm-symbolizer
> DEBUG: rm -f /opt/local/bin/llvm-tblgen
> DEBUG: rm -f /opt/local/bin/llvmc
> DEBUG: ln -sf /opt/local/bin/opt-mp-13 /opt/local/bin/macho-dump
> DEBUG: ln -sf /opt/local/bin/llvm-tblgen-mp-13 /opt/local/bin/opt
> 
> 
> I tried applying the patch as supplied, and then reran port select, but it 
> didn't’ make any difference. 
> 
> I confess I am not that familiar with how the port select mechanism works - 
> does the port need to be rev-bump in order to get rebuilt with the patch 
> (which would be a bit of a pain) ?

Yes, like I said.

Re: LLVM port mp-llvm-* select groups mislinks certain programs

[Ryan Schmidt]

On Mar 14, 2022, at 00:28, 조성빈 wrote:

> Hello,
> 
> I’m not sure who to CC so just sending it to the dev list:
> 
> it seems that the mp-llvm-* select groups of the llvm-12, llvm-13 (and 
> possibly
> more?) ports mislink certain programs.

Could you describe exactly what is mislinked?


> I think adding a line over opt-mp-* will
> fix the bug, but I’m not an expert on these ports.

Could be. If so, the revision would need to be increased as well.


> Thanks.
> 
> <0001-llvm-fix-broken-mp-llvm-groups.patch>

Re: /usr/bin/python will be removed in macOS 12.3!

[Ryan Schmidt]

On Feb 24, 2022, at 12:35, Bjarne D Mathiesen wrote:

> "#!/usr/bin/env python" might not be a problem
> iff
>   the macports python is also installed as a prerequisite
>   the ${PATH} variable includes '/opt/local/bin'

The /opt/local/bin/python symlink will only exist if the user has used "sudo 
port select python ...". This symlink is intended only for the user's 
convenience. We don't want ports to rely on its existence (or behave 
differently if it exists).

Re: Patch ./configure, or use autogen.sh instead?

[Ryan Schmidt]

On Feb 21, 2022, at 16:32, Jim DeLaHunt wrote:

> But I also thought, why not also patch the m4 macros which generate the 
> configure script? It keeps the codebase on the user's machine consistent. It 
> keeps the patches in the MacPorts repository. What harm could it do?
> 
> I'm glad you asked that!  It turns out that this codebase's Makefile includes 
> rules to rerun aclocal, autoconf, and automake if any of the m4
> macro files are newer than aclocal.m4, or configure, or config.status,
> or a few other files.  When the Portfile patches these m4 scripts, it
> updates their modification times to the present, which is newer than 
> aclocal.m4.  Thus at build time, the Makefile attempts to perform the
> configuration phase all over again. port -t (trace mode) exposes that
> this adds dependencies on those autotools.
> 
> I worked around this by adding a post-patch clause which used the
> "touch -r" macro to reset the modification times of the files I patched to be 
> the same an an unpatched macro file. This keeps the Makefile happy.
> 
> It might turn out to be smarter to remove the patches and the touches of 
> those macro files. It makes for a simpler Portfile. I have not decided.

It's not just this codebase that does that; it's a feature of autotools. That's 
one of the reasons why I said to decide whether to patch the generated files or 
the files they're generated from, not both.

Re: /usr/bin/python will be removed in macOS 12.3!

[Ryan Schmidt]

On Feb 19, 2022, at 11:06, Gerben Wierda wrote:

> But how does one (a) find out the dependency exists and (b) make sure the 
> software finds python from MacPorts?

General-purpose answers to either question are not possible. Each affected port 
will need to be investigated and fixed in its own unique way.

/usr/bin/python might be used

* directly by the Portfile specifying /usr/bin/python somewhere
* in a patchfile that references /usr/bin/python explicitly
* as a shebang line (e.g. "#!/usr/bin/python" or "#!/usr/bin/env python") in 
files used by the port at build time and/or installed by the port 
* by a build system that uses the first binary it finds named "python"
* in unknowable other ways

To tell the build system to use a particular python, one might need to

* set the PYTHON environment variable at configure time -- this is accomplished 
by setting the configure.python option
* specify some other configure environment variable or argument
* patch files
* do something else

Re: Patch ./configure, or use autogen.sh instead?

[Ryan Schmidt]

On Feb 17, 2022, at 14:10, Jim DeLaHunt wrote:
> 
> On 2022-02-16 23:12, Jason Liu wrote:
>> autogen.sh is quite often just a wrapper around autoreconf -i. If you run 
>> 'autoreconf --install' (which is what 'use_autoreconf yes' runs), in 
>> many/most cases it's equivalent to running autogen.sh.
>> 
> 
> Thank you for the link, Jason.
> 
> The contents of this project's `autogen.sh`[1] are not simply a call to 
> `autoreconf -i`. There are 346 lines, 3 functions, and a sequence of calls to 
> autotools components like aclocal, autoheader, libtoolize, autoconf, and 
> automake. Interestingly, it calls aclocal twice: once
> before autoheader and libtoolize, and again after.
> 
> This `autogen.sh` was first checked in back in 2002, with quite a similar 
> structure to what it has today. I am guessing that it predates
> `autoreconf`, and that the project has never decided to do the work of
> switching over to `autoreconf`.
> 
> I think I don't want to take on that switchover as part of improving
> the MacPorts port of this codebase.
> 
> Thank you for the tips, everyone. I am learning a lot, even if I'm
> not checking in code as a result.
> 
> [1] 

It is very common for projects to offer an autogen.sh script. If they do, it is 
preferable to use it, rather than just run autoreconf. Even if just running 
autoreconf works today, you don't know in what ways the project's authors will 
change their script over time. I do find that the need for separate scripts is 
diminishing and I do see projects dropping their custom scripts in favor of 
just using autoreconf, but you're right to leave that decision to the project's 
developers.

While "autogen.sh" is still the most popular name for such a script, it is 
recommended for the script to be named "bootstrap" instead, so you may find 
projects that use that script name.

Some autogen.sh and bootstrap scripts end by running the configure script with 
the args that were supplied to it. In this case, if you wish, you can set 
"configure.cmd ./autogen.sh" and add autoconf, automake, libtool, and any other 
needed dependencies to depends_build manually.

Some autogen.sh and bootstrap scripts either never run the configure script or 
they honor an environment variable that you can set to tell it not to do so. In 
this case, you could use "use_autoreconf yes" and "autoreconf.cmd ./autogen.sh" 
and "autoreconf.args" (to clear it) and possibly "autoreconf.env NOCONFIGURE=1" 
or similar (read the script to find out). MacPorts will then run autogen.sh for 
you as its own isolated command before running the configure script as it 
normally would. Overriding autoreconf.cmd deletes the autoconf, automake, and 
libtool dependencies that "use_autoreconf yes" adds, so you have to add them 
again manually. Do so after overriding autoreconf.cmd, not before.

I agree with the previous commentary that recommends patching the configure 
script and not regenerating it from a patched configure.ac, if that's simpler. 
Make sure that you decide which method you want to use, and only patch either 
the generated files or the files they're generated from, as appropriate to your 
decision.