Asking for review on a couple PRs
Hi all, If there are any committers who might have a bit of free time to review and hopefully merge a couple of PRs, it would allow me to submit updates to some dependent ports. * PR #20943: shaderc <https://github.com/macports/macports-ports/pull/20943> * PR #21065: msdfgen <https://github.com/macports/macports-ports/pull/21065> Both of these ports are new dependencies; shaderc is a new dependency that is needed in order to enable Vulkan support in warzone2100, and msdfgen is a new dependency for Godot 4. -- Jason Liu
Re: [macports-base] rev-upgrade complains for weakly linked dependencies
Aha, I figured someone else would have encountered a similar situation and filed a ticket already, but my searches in Trac weren't coming up with anything. Since the ticket is already 7 years old, I guess that means that it won't be getting fixed any time soon, so I guess I'll just have to use the hack I alluded to in my original message. -- Jason Liu On Mon, Oct 16, 2023 at 6:04 PM wrote: > On 10/16/23 at 1:14 PM, Jason Liu wrote: > > > Hi all, > > > > I'm getting complaints from rev-upgrade for a library binary that is > weakly > > linked to a system framework which does exist on newer versions of macOS, > > but doesn't exist on older macOSes. rev-upgrade thinks that it's a broken > > file, and offers to rebuild the port; but of course rebuilding the port > > won't do any good, because the system framework doesn't exist. In > addition, > > it's a bit difficult to simply remove the weak linking, because this > piece > > of the library is distributed as a pre-compiled binary blob by the > upstream > > devs, so it's not like I could modify some source code or CMakeLists > file. > > Is this something that could be fixed in macports-base, so that > rev-upgrade > > doesn't complain if a weakly linked dependency doesn't exist? (I could > file > > a Trac ticket if anyone thinks it's something worth pursuing.) > > > I believe the ticket for this issue is > https://trac.macports.org/ticket/52700 >
[macports-base] rev-upgrade complains for weakly linked dependencies
Hi all, I'm getting complaints from rev-upgrade for a library binary that is weakly linked to a system framework which does exist on newer versions of macOS, but doesn't exist on older macOSes. rev-upgrade thinks that it's a broken file, and offers to rebuild the port; but of course rebuilding the port won't do any good, because the system framework doesn't exist. In addition, it's a bit difficult to simply remove the weak linking, because this piece of the library is distributed as a pre-compiled binary blob by the upstream devs, so it's not like I could modify some source code or CMakeLists file. Is this something that could be fixed in macports-base, so that rev-upgrade doesn't complain if a weakly linked dependency doesn't exist? (I could file a Trac ticket if anyone thinks it's something worth pursuing.) P.S.: Supposed, I could use `install_name_tool` and set the offending dependency to the root folder, '/', but that seems awfully hacky to me https://stackoverflow.com/questions/4824885/remove-dependent-shared-library-from-a-dylib -- Jason Liu
Re: librsvg, and what MacPorts is for
> > That said, I presume there's a strong overall consensus that on current > hardware, we run the current (supported) versions of software, and that > older operating systems and hardware are supported only on a "if it doesn't > hurt anything" basis. > I'm not sure this is 100% accurate, either. I think it's pretty much up to the maintainers of each individual port, but I know that at least for me, since I have some hardware that, for example, can't be upgraded beyond 10.11, I tend to maintain my ports using a "try our very best to make the current version of the software work on older macOSes" mentality (hence the existence of things like the legacysupport PortGroup). In some cases, that may mean making a version-numbered subport of an older release that still works on an older macOS, like what we have had to do with MoltenVK. Is this potentially a risk when there are security patches that only work on newer macOSes? Yes, but using an older version of macOS in and of itself can be considered a security risk, so I think it's up to the user to make the decision whether to run an out-of-date version of some piece of software that is still being made available through MacPorts. -- Jason Liu On Tue, Oct 10, 2023 at 3:26 PM Perry E. Metzger wrote: > And Mascguy didn't seem to care to explain the situation, which I clearly > didn't understand. Okay, That makes more sense and is acceptable. > > That said, I presume there's a strong overall consensus that on current > hardware, we run the current (supported) versions of software, and that > older operating systems and hardware are supported only on a "if it doesn't > hurt anything" basis. > > Perry > > > On 10/10/23 14:08, Gregorio Litenstein wrote: > > In general terms I (who am absolutely nobody) agree with you, but there's > one thing I believe you're not taking into account and it's that this is a > fallback version for users with ancient hardware. > > The main `librsvg` port is currently at `2.56.3`, which was released two > months ago > > @Chris, I belive OP didn't realize it's not the main port. > > > Gregorio Litenstein Goldzweig [image: glit_qr_4.png] > Médico Cirujano > > >- Fono: +56 9 96343643 >- E-Mail: g.litenst...@gmail.com > > > On 10 Oct 2023 15:07 -0300, Chris Jones > , wrote: > > Hi, > > I am not sure what you are complaining about. Version 2.56.3, whilst not > the absolute latest version a pretty up to date rust based version, is > already used on Darwin 10 and newer. Your mail below seems to imply the old > C version is used everywhere, which just isn't the case. What am I missing > here ? > > Chris > > On 10 Oct 2023, at 6:48 pm, Perry E. Metzger > wrote: > > See the following thread: > https://github.com/macports/macports-ports/pull/20744 — but to summarize, > Mascguy does not want to update librsvg to a safe / modern one because > ancient versions of MacOS can't support Rust. > > So I don't want to be a pain in the neck, but I have little interest in > MacPorts if the point is to preserve compatibility with MacOS 10.5 at the > expense of having the thousands of users of current Macs and current MacOS > have a dangerously insecure version of a basic SVG graphics library that > other things depend on. > > (The upstream librsvg maintainers have washed their hands of the old C > version and don't support it any more, and for good reason. The Rust > version of the library provides a far more secure codebase.) > > I don't know how other people feel here, but I don't work on MacPorts > because I like retrocomputing, but rather because I want to use Unix tools > on my modern Macs. > > If we're all on the same page that the priority is current MacOS users, > then we need to make sure that policy is well understood by all and we need > to update ports that are being held back for the benefit of people using an > OS from 2007. > > If the consensus is that we prioritize ancient versions of MacOS with > three users (or sometimes none) over the experience the bulk of the users > have, that's fine, and I'll accept it, but then I'm switching to Brew, and > I will advise others to do the same, and will explain that current versions > of MacPorts cannot be trusted to have safe software because the people > involved prioritize support for ancient versions of the operating system. > > I will accept whatever the consensus is. > > Perry > > >
Blacklist for architectures?
Hi all, Is there some sort of equivalent to 'compiler.blacklist' for supported archs? In other words, is there some way for us to remove a selection from the default list of 'supported_archs', rather than manually listing what is essentially a whitelist in 'supported_archs'? Or would I use vanilla Tcl and something like 'lsearch' or 'lremove' to remove it from the default list? -- Jason Liu
Re: Regarding github CI
I believe you are talking about automatically updating Portfiles? I feel like this has the potential to be incredibly dangerous. As someone who works on ports that often have a very large number of dependencies, I have sometimes encountered situations where if the dependencies get updated too far beyond what the port can accept, then the port ends up broken until the upstream devs update their code to use the new version of the dependency. If MacPorts had the ability to have ports be dependent on specific versions of a dependency like Apt does on Debian, then I might be more inclined to accept such an idea, but MacPorts basically only has one version of a port available at any given time. -- Jason Liu On Wed, Sep 6, 2023 at 5:02 PM Kirill A. Korinsky via macports-dev < macports-dev@lists.macports.org> wrote: > Folks, > > As you may have noticed a few weeks ago, I opened a PR > https://github.com/macports/macports-ports/pull/20092 in which I propose > enabling the execution of port test as part of GitHub's CI. > > I understand that not all ports currently have tests, but a lot of them > does. > > My goal here is to enhance the quality of tests for the ports, which will > allow me to proceed with a bot that runs `port livecheck...` and detects > when ports are updated. Subsequently, it will open a PR to update a port > with the updated version and checksums, but only if such a port has tests. > > Why? To automate the process of updating ports. We have a vast number of > ports, and many of them can be updated quite easily. Enabling the execution > of port tests on GitHub will ensure that such automatically generated PRs > are of sufficient quality to be merged. > > What's the ultimate goal? To keep MacPorts dynamically updated. > > -- > wbr, Kirill > >
Fwd: Re: Re: Qt 5.10 skipped?
-- Forwarded message - From: Date: Fri, Sep 1, 2023 at 10:21 PM Subject: Re: Re: Re: Qt 5.10 skipped? To: Jason Liu My email is once again blocked from posting to the mailing lists; please forward this. On 9/1/23 at 5:34 PM, Jason Liu wrote: > Yes, I do have the crash report that gets generated by macOS when the > application crashes. I haven't reported it anywhere, because this is an > old, obsolete version of both the software and the version of Qt that's > involved. MacPorts’ trac would still be a good place to report this. > I thought about using the 'symbolicatecrash' tool that comes with Xcode, > but I've never had any luck getting it to work properly. I suppose I could > also use 'atos' to symbolicate line by line, but I don't even know where to > begin getting my hands on the dSYM build symbols from the MacPorts build of > Qt 5.11. I am not familiar with these tools (mainly since I have not needed to use them yet), however my impression is that (1) they are not very useful for certain “release“ builds which do not have debugging symbols to begin with, and/or (2) they are only needed for a debug build where symbols have been stripped. I do not know whether rebuilding the relevant qt511 ports with the +debug variant enabled works or will help get a more useful crash report, but that is what I would suggest trying next.
Re: Re: Qt 5.10 skipped?
Yes, I do have the crash report that gets generated by macOS when the application crashes. I haven't reported it anywhere, because this is an old, obsolete version of both the software and the version of Qt that's involved. I thought about using the 'symbolicatecrash' tool that comes with Xcode, but I've never had any luck getting it to work properly. I suppose I could also use 'atos' to symbolicate line by line, but I don't even know where to begin getting my hands on the dSYM build symbols from the MacPorts build of Qt 5.11. -- Jason Liu On Fri, Sep 1, 2023 at 3:33 PM wrote: > On 8/31/23 at 6:39 PM, Jason Liu wrote: > > > Does anyone have any objections if I were to resurrect Qt 5.10? One of > the > > ports that I am working on crashes under certain circumstances when using > > qt511, which is the newest Qt that works on macOS 10.11. The version of > Qt > > that was bundled with the pre-built binary distribution was Qt 5.10. > > Admittedly, I am also able to compile the port using qt59, and there are > no > > crashes. In addition, I can also compile the port using qt513 on macOS > > 10.13, and there are also no crashes (qt512 also doesn't currently > exist). > > > > I'm not really sure how much Qt changes with each version. So on the one > > hand, it would be nice to be able to get the port working with the same > > version of Qt that it originally shipped with, but on the other hand, it > > seems like overkill to bring back such a large set of ports as qt510-* > just > > for the sake of a single port on old versions of macOS. > > > > What are everyone's thoughts on this? > > > > -- > > Jason Liu > > > I think it would be much better to investigate the crash first. Do you > have a crash report/have you reported the crash somewhere? >
Re: Qt 5.10 skipped?
Does anyone have any objections if I were to resurrect Qt 5.10? One of the
ports that I am working on crashes under certain circumstances when using
qt511, which is the newest Qt that works on macOS 10.11. The version of Qt
that was bundled with the pre-built binary distribution was Qt 5.10.
Admittedly, I am also able to compile the port using qt59, and there are no
crashes. In addition, I can also compile the port using qt513 on macOS
10.13, and there are also no crashes (qt512 also doesn't currently exist).
I'm not really sure how much Qt changes with each version. So on the one
hand, it would be nice to be able to get the port working with the same
version of Qt that it originally shipped with, but on the other hand, it
seems like overkill to bring back such a large set of ports as qt510-* just
for the sake of a single port on old versions of macOS.
What are everyone's thoughts on this?
--
Jason Liu
On Wed, Aug 30, 2023 at 11:26 PM Ryan Schmidt
wrote:
>
>
> On Aug 30, 2023, at 19:40, Jason Liu wrote:
>
> I just noticed that Qt 5.9 and Qt 5.11 are available in MacPorts, but not
> Qt 5.10. Was there a specific reason why 5.10 was skipped, or was it just
> that no one got around to it before 5.11 was released?
>
>
> Certainly qt 5.10.x existed in MacPorts. You can find bugs that were filed
> against it:
>
> https://trac.macports.org/query?port=~qt5=~%405.10
>
> I couldn't immediately find the commit in which it was removed. It's a
> little difficult since the qt5 portfile is massive and there are also the
> qt5 portgroups. But I suspect it was deleted because there exists no macOS
> version for which qt 5.10.x is the most recent one available.
>
> And a somewhat tangential question: When using the qt5 PortGroup, is there
> a way to specify a particular version of Qt 5.x that needs to be used? I do
> see plenty of references to ${qt5.version}, but is that something that I am
> allowed to set in my Portfile?
>
>
> Looking near the top of the qt5-1.0 portgroup,
>
>
> https://github.com/macports/macports-ports/blob/master/_resources/port1.0/group/qt5-1.0.tcl
>
> the prominent placement of qt5.min_version suggests to me that it is
> intended for portfile authors to override it. 38 ports reference this
> variable.
>
> I don't believe there is any way for you to specify a maximum qt version,
> nor a specific version. The qt5 portgroup will select the blessed qt
> version that is compatible with the user's OS version.
>
Qt 5.10 skipped?
Hi everyone,
I just noticed that Qt 5.9 and Qt 5.11 are available in MacPorts, but not
Qt 5.10. Was there a specific reason why 5.10 was skipped, or was it just
that no one got around to it before 5.11 was released?
And a somewhat tangential question: When using the qt5 PortGroup, is there
a way to specify a particular version of Qt 5.x that needs to be used? I do
see plenty of references to ${qt5.version}, but is that something that I am
allowed to set in my Portfile?
--
Jason Liu
Re: Quick favor for those with access to 10.12 machines
That's a great tip, I had no idea someone was uploading the macOS SDKs onto GitHub! Thanks, Kirill! -- Jason Liu On Wed, Aug 23, 2023 at 2:22 PM Kirill A. Korinsky wrote: > You may use github "mirror" of SDK for difference macOS, for example > > > https://github.com/search?q=repo%3Aphracker%2FMacOSX-SDKs%20kCGImageByteOrder32Big=code > <https://github.com/search?q=repo:phracker/MacOSX-SDKs%20kCGImageByteOrder32Big=code> > > -- > wbr, Kirill > > On 23. Aug 2023, at 20:14, Jason Liu wrote: > > > -- Forwarded message - > From: > Date: Wed, Aug 23, 2023 at 9:07 AM > Subject: Re: Quick favor for those with access to 10.12 machines > To: Jason Liu > > > On 8/23/23 at 1:18 AM, Jason Liu wrote: > > > Hi all, > > > > I have what is hopefully a quick favor to ask of people who have access > to > > a machine with 10.12. Could someone search for whether the string > > "kCGImageByteOrder32Big" can be found anywhere inside of > > /System/Library/Frameworks/CoreGraphics.framework? > > > > In other words, run a > > > > grep -R kCGImageByteOrder32Big > > /System/Library/Frameworks/CoreGraphics.framework > > > > and let me know whether you get any hits. > > > > -- > > Jason Liu > > > > > > I do not have access macOS 10.12, but I know kCGImageByteOrder32Big is > defined in CGImage.h in the SDKs for 10.12 and later. There were many > constants introduced in 10.12 which are equivalent to and/or replacements > for earlier ones. In the case of kCGImageByteOrder32Big it has the same > value as kCGBitmapByteOrder32Big. > > > > Could you please forward this to the macports-dev list? It is currently > rejecting my email. > > >
Fwd: Quick favor for those with access to 10.12 machines
-- Forwarded message - From: Date: Wed, Aug 23, 2023 at 9:07 AM Subject: Re: Quick favor for those with access to 10.12 machines To: Jason Liu On 8/23/23 at 1:18 AM, Jason Liu wrote: > Hi all, > > I have what is hopefully a quick favor to ask of people who have access to > a machine with 10.12. Could someone search for whether the string > "kCGImageByteOrder32Big" can be found anywhere inside of > /System/Library/Frameworks/CoreGraphics.framework? > > In other words, run a > > grep -R kCGImageByteOrder32Big > /System/Library/Frameworks/CoreGraphics.framework > > and let me know whether you get any hits. > > -- > Jason Liu > I do not have access macOS 10.12, but I know kCGImageByteOrder32Big is defined in CGImage.h in the SDKs for 10.12 and later. There were many constants introduced in 10.12 which are equivalent to and/or replacements for earlier ones. In the case of kCGImageByteOrder32Big it has the same value as kCGBitmapByteOrder32Big. Could you please forward this to the macports-dev list? It is currently rejecting my email.
Quick favor for those with access to 10.12 machines
Hi all, I have what is hopefully a quick favor to ask of people who have access to a machine with 10.12. Could someone search for whether the string "kCGImageByteOrder32Big" can be found anywhere inside of /System/Library/Frameworks/CoreGraphics.framework? In other words, run a grep -R kCGImageByteOrder32Big /System/Library/Frameworks/CoreGraphics.framework and let me know whether you get any hits. -- Jason Liu
Re: Xcode PortGroup: Xcode compiles code twice?
Yes, you're right that clearing the build phase doesn't prove anything, but
I'm not sure I'm following your other point. Are you saying that "make
install" will compile the source code? I was under the impression that you
need to manually run "make" in order to actually compile the source code,
hence the traditional magic formula of './configure ; make ; make install'.
Without the first make, the "make install" shouldn't have anything to
install. Or am I wrong about that?
--
Jason Liu
On Tue, Aug 15, 2023 at 2:54 PM Joshua Root wrote:
> On 16/8/2023 04:29, Jason Liu wrote:
> > Hi everyone,
> >
> > I'm working on a Portfile that uses the xcode PortGroup, and I've
> > noticed something that surprised me: It seems that the MacPorts build is
> > compiling the source code during the build phase, and then compiling the
> > source code AGAIN during the destroot phase? Is this correct, or am I
> > starting to hallucinate? Because when I add a 'build {}' to my Portfile,
> > which in theory should cause nothing to be compiled, all of the compiled
> > products are still somehow coming into existence and getting placed into
> > ${destroot}.
>
> I don't know if your project is in fact building things twice, but
> clearing the build phase doesn't prove anything one way or the other,
> because the install target depends on the targets that build the program
> and will thus run them first. You will usually see a similar thing
> happen if you just run 'make install' with a makefile based project.
>
> - Josh
>
Xcode PortGroup: Xcode compiles code twice?
Hi everyone,
I'm working on a Portfile that uses the xcode PortGroup, and I've noticed
something that surprised me: It seems that the MacPorts build is compiling
the source code during the build phase, and then compiling the source code
AGAIN during the destroot phase? Is this correct, or am I starting to
hallucinate? Because when I add a 'build {}' to my Portfile, which in
theory should cause nothing to be compiled, all of the compiled products
are still somehow coming into existence and getting placed into ${destroot}.
--
Jason Liu
Re: Portfile for CEF: Build from source or pre-built binary?
On Sun, Aug 6, 2023 at 10:19 PM Ryan Schmidt wrote: > Most of the buildbot workers have about 90GB disk space total, sometimes > with as little as 15GB free. mpbb is currently coded to attempt to free > disk space by uninstalling ports until at least 15GB is free. Well, considering that CEF needs ~200 GB of free disk space, that would seem to preclude the possibility of building from source on the MacPorts buildbots, no? -- Jason Liu On Sun, Aug 6, 2023 at 10:19 PM Ryan Schmidt wrote: > On Aug 6, 2023, at 16:57, Jason Liu wrote: > > > > On their website, CEF states that building from source on a Mac requires > "At least 16 GB of RAM and 200 GB of free disk space" [1]. > > My opinion remains that most ports should build from source if that's > possible. If a project is closed source and only a binary is available, > then obviously we can't build from source. > > Ports that require an unusually large amount of free disk space should be > coded to verify that enough free disk space exists first (e.g. in a > pre-fetch block). > > Most of the buildbot workers have about 90GB disk space total, sometimes > with as little as 15GB free. mpbb is currently coded to attempt to free > disk space by uninstalling ports until at least 15GB is free. >
Portfile for CEF: Build from source or pre-built binary?
Hi everyone, I'm working on writing a Portfile for the Chromium Embedded Framework (CEF), and I have a question for the mailing list. The question is the perennial one of: should the port be built from source, or should the port download the pre-built binary distribution? In this particular case, I don't think it's necessarily a question of how difficult or complex it is to build from source; rather, I think it's a question of the resources required to build from source. If we were to build CEF from source, it would require downloading the entire source code for both CEF itself and Chromium. On their website, CEF states that building from source on a Mac requires "At least 16 GB of RAM and 200 GB of free disk space" [1]. In addition, a full build from source apparently takes "Approximately 4 hours with a fast internet connection (100 Mbps) and fast build machine (2.4 GHz, 16 logical cores, SSD)" [2]. On the other hand, Spotify hosts the official pre-built binary distributions of CEF [3]; the "standard distribution" seems to hover at around 200 MB in size. If building from source takes that many resources, I guess it's no wonder why Spotify (but not Google?) hosts pre-built binaries. Thoughts? Opinions? [1] https://bitbucket.org/chromiumembedded/cef/wiki/AutomatedBuildSetup.md#markdown-header-macos-configuration [2] https://bitbucket.org/chromiumembedded/cef/wiki/MasterBuildQuickStart.md#markdown-header-mac-os-x-setup [3] https://cef-builds.spotifycdn.com/index.html -- Jason Liu
Re: Unable to fetch sources from GitLab instance
Indeed, I was downloading the tarball and manually putting it into the distfiles directory, so that I could get the rest of the Portfile written. I guess I'll just have to take it on faith that one of the newer buildbots will manage to save the distfile into distfiles.macports.org before the older builders attempt their builds. I'm aware of the curl issues on older Macs, since this topic was recently discussed on this mailing list as well... which is why I tried manually running /usr/bin/curl on the tarball, which downloaded fine on both my 10.13 High Sierra and 10.11 El Capitan machines. That's why I was confused when MacPorts couldn't seem to download the file. I was under the impression that MacPorts uses the Apple/system libcurl, which... is the same as what /usr/bin/curl uses? Or am I mistaken about that? In any case, thanks Dave, for trying it out and letting me know that I'm not slowly losing my mind writing these Portfiles. (Or maybe I am anyway, haha) -- Jason Liu On Sat, Jul 29, 2023 at 10:26 PM Dave Allured - NOAA Affiliate via macports-dev wrote: > Jason, port fetch with your unmodified portfile works fine for me. > Monterey x86_64, Macports 2.81, freshly updated. I think your portfile > recipe for this gitlab instance is good. There have been several recent > reports of Macports fetch problems. These often relate to down level Apple > native versions of curl on older Macs, improper certificate updates on > servers, etc. > > There is a shortcut if it is only a single tar file. Instead of spending > time diagnosing, just manually download the distfile, which you already > have. Then manually insert that into > (prefix)/var/macports/distfiles/librist, then re-run port install, as > described in https://trac.macports.org/wiki/ProblemHotlist#fetch-failures. > Port install will then skip the problematic download and be happy with the > local copy. When your portfile is later merged into Macports, the distfile > will be propagated to various mirrors, which will then serve anyone having > problems with the original server. > > Also see: https://trac.macports.org/wiki/ProblemHotlist#letsencrypt > > > On Sat, Jul 29, 2023 at 9:14 AM Jason Liu wrote: > >> By the way, here is my Portfile for librist, in case anyone wants to try >> and see whether fetching from VideoLAN's GitLab instance works for them. >> -- >> Jason Liu >> >> >> On Fri, Jul 28, 2023 at 11:47 PM Jason Liu wrote: >> >>> Hi all, >>> >>> I suppose I should be directing this question to René Bertin, since he's >>> the maintainer of the VLC port, but I'm sending this to the entire dev >>> mailing list, in case anyone else has any ideas. >>> >>> I'm trying to create a Portfile for librist, which is one of the >>> projects that's being hosted on https://code.videolan.org. My question >>> to René (and everyone else) is: Have you had any luck trying to fetch >>> sources from code.videolan.org? When I try to use the gitlab PortGroup, >>> MacPorts doesn't seem to find a tarball at the URL that is generated by the >>> PortGroup. However, I'm able to download the file using a web browser, and >>> even curl downloads the file just fine: >>> >>> /usr/bin/curl -LROJk >>> https://code.videolan.org/rist/librist/-/archive/v0.2.7/librist-v0.2.7.tar.bz2 >>> >>> Does anyone have any explanation for the behavior that I'm seeing? >>> -- >>> Jason Liu >>> >>
Re: Unable to fetch sources from GitLab instance
By the way, here is my Portfile for librist, in case anyone wants to try and see whether fetching from VideoLAN's GitLab instance works for them. -- Jason Liu On Fri, Jul 28, 2023 at 11:47 PM Jason Liu wrote: > Hi all, > > I suppose I should be directing this question to René Bertin, since he's > the maintainer of the VLC port, but I'm sending this to the entire dev > mailing list, in case anyone else has any ideas. > > I'm trying to create a Portfile for librist, which is one of the projects > that's being hosted on https://code.videolan.org. My question to René > (and everyone else) is: Have you had any luck trying to fetch sources from > code.videolan.org? When I try to use the gitlab PortGroup, MacPorts > doesn't seem to find a tarball at the URL that is generated by the > PortGroup. However, I'm able to download the file using a web browser, and > even curl downloads the file just fine: > > /usr/bin/curl -LROJk > https://code.videolan.org/rist/librist/-/archive/v0.2.7/librist-v0.2.7.tar.bz2 > > Does anyone have any explanation for the behavior that I'm seeing? > > -- > Jason Liu > Portfile Description: Binary data
Unable to fetch sources from GitLab instance
Hi all, I suppose I should be directing this question to René Bertin, since he's the maintainer of the VLC port, but I'm sending this to the entire dev mailing list, in case anyone else has any ideas. I'm trying to create a Portfile for librist, which is one of the projects that's being hosted on https://code.videolan.org. My question to René (and everyone else) is: Have you had any luck trying to fetch sources from code.videolan.org? When I try to use the gitlab PortGroup, MacPorts doesn't seem to find a tarball at the URL that is generated by the PortGroup. However, I'm able to download the file using a web browser, and even curl downloads the file just fine: /usr/bin/curl -LROJk https://code.videolan.org/rist/librist/-/archive/v0.2.7/librist-v0.2.7.tar.bz2 Does anyone have any explanation for the behavior that I'm seeing? -- Jason Liu
Re: Problems fetching from GitHub on 10.8?
I see... I didn't realize that MacPorts base was actually linking against libcurl, and not just simply calling the curl command-line utility. -- Jason Liu On Fri, Jul 21, 2023 at 3:30 PM Kirill A. Korinsky wrote: > Well... you need to build a MacPorts against some fresh enough libcurl :) > Old system hasn't got enough. > > I'm using bootstrap MacPorts with curl (and python) for old systems :) And > it works very stable. > > Can you use normal MacPorts and link MacPorts against curl from MacPorts? > Yes, you can. But if you remove curl it won't work anymore. Completley. > > -- > wbr, Kirill > > On 21. Jul 2023, at 21:19, Jason Liu wrote: > > Thanks Kirill. I'm still reading my way through that Trac ticket, but I > think I'm starting to get the gist of what's going on. The interesting > thing, though, is that I have the MacPorts curl (and curl-ca-bundle) > installed on the 10.8 VM already. I would have thought that MacPorts base > would look for and use MacPorts curl and curl-ca-bundle if they're > installed? > > -- > Jason Liu > > > On Fri, Jul 21, 2023 at 2:41 PM Kirill A. Korinsky > wrote: > >> See: https://trac.macports.org/ticket/51516 >> >> You may also enjoy my ansible playbook: >> https://github.com/catap/macos-ansible-playbooks >> >> -- >> wbr, Kirill >> >> On 21. Jul 2023, at 19:45, Jason Liu wrote: >> >> >> On my Mountain Lion VM, a Portfile that I'm working on updating doesn't >> seem to be able to fetch the distfile from GitHub, but the same Portfile is >> able to fetch just fine from my El Capitan VM. Is anyone else on 10.8 >> experiencing this issue when attempting to fetch directly from GitHub? >> >> -- >> Jason Liu >> >> >
Re: Problems fetching from GitHub on 10.8?
Thanks Kirill. I'm still reading my way through that Trac ticket, but I think I'm starting to get the gist of what's going on. The interesting thing, though, is that I have the MacPorts curl (and curl-ca-bundle) installed on the 10.8 VM already. I would have thought that MacPorts base would look for and use MacPorts curl and curl-ca-bundle if they're installed? -- Jason Liu On Fri, Jul 21, 2023 at 2:41 PM Kirill A. Korinsky wrote: > See: https://trac.macports.org/ticket/51516 > > You may also enjoy my ansible playbook: > https://github.com/catap/macos-ansible-playbooks > > -- > wbr, Kirill > > On 21. Jul 2023, at 19:45, Jason Liu wrote: > > > On my Mountain Lion VM, a Portfile that I'm working on updating doesn't > seem to be able to fetch the distfile from GitHub, but the same Portfile is > able to fetch just fine from my El Capitan VM. Is anyone else on 10.8 > experiencing this issue when attempting to fetch directly from GitHub? > > -- > Jason Liu > >
Problems fetching from GitHub on 10.8?
On my Mountain Lion VM, a Portfile that I'm working on updating doesn't seem to be able to fetch the distfile from GitHub, but the same Portfile is able to fetch just fine from my El Capitan VM. Is anyone else on 10.8 experiencing this issue when attempting to fetch directly from GitHub? -- Jason Liu
Get version number of another port from inside Portfile?
Hi all, I'm working on a Portfile where the source code needs to gather a list of all of the dependencies' version numbers into a file before it will compile. The project has a utility python script that will gather these version numbers, but it assumes that all of the dependencies have been downloaded locally and will be compiled from within the project. However, all of the dependencies are already available through MacPorts. So my question is: Is there some sort of convenient way to obtain the version number of another port from inside a Portfile? Or do I just need to do what I am doing now, which is system "port -q info --version " -- Jason Liu
Re: Installing zstd CMake files
Would it make sense to make it into a variant, instead of a separate subport? -- Jason Liu On Wed, May 10, 2023 at 1:16 AM wrote: > The package opencolorio uses CMake to find minizip-ng. > The minizip-ng’s CMake files assume that the CMake files of zstd exist. > However, the CMake files of zstd are installed if and only if zstd is > built with CMake (*not* make). > According to the README.md of zstd, "`make` is the officially maintained > build system of this project.” > Further, zstd is a dependency of clang. > > My proposed solution is to create a subport of zstd that just installed > the CMake files > (https://github.com/macports/macports-ports/pull/18605/). > > This is *far* from an ideal solution, and I was hoping someone else would > have a better idea. > > Thanks, > Marcus >
Re: path:-style depspec to libc++.dylib?
Did you try double backslash or putting quotes around the path? -- Jason Liu On Sun, May 7, 2023 at 6:18 PM René J.V. Bertin wrote: > Hi, > > I tried to write a path:-style depspec to libc++.dylib, but that gives me > a regexp compilation error. Can that be avoided by escaping those pluses in > the filename and if so, how? A simple backslash didn't do it for me. > > Thanks, > R. >
Re: having the "+test" or "+tests" variant propagate causes unexpected builds
My own personal opinion has been that +test/+tests and +debug, by default, should not propagate through the chain of dependencies; and then perhaps there might be some way to enable propagation (maybe with a command line option?). However, if I recall correctly, all variants propagate through the dependency chain, so it might be difficult to make certain variant keywords behave differently? -- Jason Liu On Sun, Oct 23, 2022 at 1:58 PM Ken Cunningham < ken.cunningham.web...@gmail.com> wrote: > Various ports implement a “test” or “tests” variant to allow extra > features and deps required for testing to be enabled. > > This variant, when requested, will propagate up the chain to all the > ports, however. There is no real use case where someone would desire the > test/s variant to propagate up. > > This generates needless builds, and often enables features people neither > need nor want, and then guarantees manual rebuilds, forever, of the > involved ports. > > I recently came back to a massive building project involving clang and > llvm when I was trying to build “mesa +tests”. Because clang-15 and llvm-15 > also have a “+tests” variant, and had not yet been installed, port was > building those (and possibly others) with the tests variant rather than use > the prebuilt binary. > > Of course I just aborted the huge llvm/clang-15 build, cleaned them up, > and installed them separately. But others would probably not know to do > this. > > I had suggested a few years ago we might namespace the test/tests > variants, by having a convention that the portname be prepended to the test > variant, to be more specific and avoid this — but not a widely acceptable > idea at that time. So we’re still in the same situation… > > Is it possible that a “test” or “tests” variant might not be propagated up > the ports chain by base, instead? > > > K > > > > PS. A similar thing happens with “+debug” variants, another common variant > that you *usually* don’t want propagated up to *every single port* in the > chain either. > > This one is occasionally something that people would want up their chain, > but it is so fragile of a plan to rely on variant propagation (ie if you > have the port installed already, it won’t be reinstalled with the “+debug” > variant), that such rare users might best install each port they want to be > installed as “debug” do that specifically. Certainly most of us don’t want > clang-15 installed with it’s debug variant when you’re trying to debug some > little port.
Re: Frotz build targets
On Mon, Jun 27, 2022 at 5:35 PM Mark Brethen wrote:
> Can a subport have variants?
Yes, they can. As Nils points out, subports can have variants that are
specific to that subport by using conditional logic:
if {$subport eq "mySubport"} {
variant theVariant description {This variant only exists for subport
"mySubport"} {
...
}
}
--
Jason Liu
On Mon, Jun 27, 2022 at 6:17 PM Nils Breunese wrote:
> I believe variants are global for a port, but Iyou could use conditional
> logic based on which subport gets installed for a variant. What are you
> trying to achieve?
>
> Nils.
>
> > Op 27 jun. 2022, om 23:35 heeft Mark Brethen
> het volgende geschreven:
> >
> > Can a subport have variants?
> >
> > Mark Brethen
> > mark.bret...@gmail.com
> >
> >
> >
> >> On Jun 26, 2022, at 9:03 PM, Mark Brethen
> wrote:
> >>
> >> I’ll do that. It’s been so long that I’ve forgotten how they’re set up.
> >>
> >> Mark Brethen
> >> mark.bret...@gmail.com
> >>
> >>
> >>
> >>> On Jun 26, 2022, at 8:56 PM, Ryan Schmidt
> wrote:
> >>>
> >>> On Jun 26, 2022, at 17:39, Mark Brethen wrote:
> >>>>
> >>>> Attaching a Portfile that uses variants—downside is you can only
> install one. Maybe that is sufficient? I couldn’t find any documentation
> for using subports.
> >>>
> >>> What would you like to know about subports? Have you looked at any
> existing portfiles that use subports to see how they work?
> >>>
> >>
> >
>
>
Re: Codesigning everything and combatting malicious code
xisting certificate if GateKeeper is enabled. > Everything would presumably have to be re-signed and reinstalled. > I believe Josh is correct. A certificate revocation means having to start over from scratch and "re-publishing" everything with a new certificate. This is also what happens when a developer's certificate gets revoked on any of Apple's app stores. = Finally, I'm of the belief that code signing would not prevent, nor was it designed to prevent, malicious or buggy code from making its way into distributed software. It didn't prevent the Log4j bug/vulnerability, and it wouldn't have prevented the node-ipc situation from occurring. In my opinion, it's not, nor should it be, our job to guarantee that we are providing bug-free and non-malicious software from upstream third-party sources. Our current merge review process already implements many of the best practices used by open-source projects to try to guard against such software from making it from upstream all the way to being released by our distribution network. However, I believe that it IS our role to strive to provide bug-free and secure first-party software, i.e. MacPorts base, MacPorts legacy support, Portfiles, and the other pieces of code that are directly released by the group of volunteers that make up "The MacPorts Project"™. I'm only a package maintainer, and so I don't expect my opinion to have much weight on final decisions, but I'm of the opinion that the current situation should stay as it is: we release a MacPorts installer that has been code signed, but we should not be code signing any of the ports using a Developer ID certificate. Ad hoc signing of ports might be useful, as long as it doesn't involve the need for a Developer ID certificate. -- Jason Liu On Mon, Mar 21, 2022 at 9:21 PM Ryan Schmidt wrote: > Sorry, this got long. > > I want to move a discussion to the macports-dev list that began with a > user's question to macports-mgr. This user ran a third-party utility which > reported that some files installed by MacPorts were not codesigned and > asked if it was a concern. > > I replied that the files installed by the MacPorts installer are > codesigned but that MacPorts predates the existence of codesigning and > nobody has yet added code to MacPorts that arranges for files installed by > ports to be codesigned. Individual ports might codesign their files if > that's how their build system was written. On Apple Silicon all (code?) > files must be codesigned and the linker (?) takes care of ad-hoc > codesigning the files automatically. I also said I wasn't really clear on > what purpose codesigning serves; MacPorts and macOS got by fine without it > for years. > > The user replied asking if codesigning might reduce the risk of > compromised open source packages, referencing this incident in which the > developer of node-ipc deliberately released a version containing malicious > code: > > > https://www.schneier.com/blog/archives/2022/03/developer-sabotages-open-source-software-package.html > > This is a hypothetical question since node-ipc is not in MacPorts and as > far as I know a similar situation has not happened with any software that > we do have in MacPorts. And since then node-ipc appears to have been forked > by a different developer and a new non-malicious version has been released. > > I felt that the broader MacPorts developer community might care to hear a > reply to this or to discuss whether and how we should modify MacPorts base > to codesign everything installed by ports. > > ~ > > Certainly the situation with node-ipc was undesirable but I don't see how > MacPorts codesigning its files would solve it. I'll talk about that in a > minute. First let me discuss what safeguards MacPorts already contains. > > MacPorts ports already fetch specific maintainer-tested versions of the > source code. If a developer releases a new version, nothing changes in > MacPorts until a MacPorts contributor updates the Portfile to use that new > version. Before doing so the MacPorts contributor is expected to verify > that the port at least compiles on one macOS version; that the port's test > phase runs, if the port has one; that "port lint" doesn't show any easily > fixable mistakes; and ideally that some basic functionality of the > installed files works. If the port has subports, this applies to all > subports as well. > > MacPorts already checksums distfiles. The MacPorts master build server > saves a copy of distfiles (assuming the checksum matches at the time) and > these are mirrored on servers around the world. If a developer replaces a > distfile on their server with something else (maliciously or not), MacPorts > will not install the port using that distfile; it will issue a checksum > mismatch error an
Re: /usr/bin/python will be removed in macOS 12.3!
SIP is only the beginning of your boot disk headaches. Starting with Catalina, and continuing into Big Sur, Apple made it increasingly difficult to make any modifications to anything in the system volume. In Catalina, the root volume was split into two: the System volume and Data volume, and the System volume is mounted read-only. Starting in Big Sur, the System volume is not just read-only, but the entire volume is now also cryptographically signed, and is referred to as a Signed System Volume (SSV). https://eclecticlight.co/2020/06/25/big-surs-signed-system-volume-added-security-protection/ I remember reading online somewhere that an SSV which has been unsealed can never be fully re-sealed, even if you bless a new snapshot, but I can't seem to find where it was that I saw that anymore. See also: https://apple.stackexchange.com/questions/395508/can-i-mount-the-root-system-filesystem-as-writable-in-big-sur -- Jason Liu On Tue, Feb 22, 2022 at 6:17 PM Gerben Wierda via macports-dev < macports-dev@lists.macports.org> wrote: > > For a test I was planning to adapt the migration approach. Save the > ‘requested’ list. Uninstall all ports. Temporary mv /usr/bin/python out of > the way. Build the requested list. > > > > I can do this on a secondary system. > > > > This would tell me at least that the ports I have installed all install > without /usr/bin/python, which takes care of any build dependencies. > > I did try, but failed. I was able to turn off system integrity protection, > but that still leaves me with a boot disk that is read-only, so temporarily > moving /usr/bin/python out of the way did not work. I haven’t investigated > going beyond simply disabling SIP. > > So far, then, no success at trying to emulate the 12.3 macOS situation > regarding python on macOS 12.2.1. > > G
Re: Patch ./configure, or use autogen.sh instead?
autogen.sh is quite often just a wrapper around autoreconf -i. If you run 'autoreconf --install' (which is what 'use_autoreconf yes' runs), in many/most cases it's equivalent to running autogen.sh. https://stackoverflow.com/questions/50044091/what-is-the-job-of-autogen-sh-when-building-a-c-package-on-linux -- Jason Liu On Wed, Feb 16, 2022 at 11:15 PM Jim DeLaHunt wrote: > On 2022-02-15 01:08, Joshua Root wrote: > > > > On 2022-2-15 19:28 , Jim DeLaHunt wrote: > >> I am working on a port[1], where I want to patch a couple of autoconf > >> macros[2] and configure.ac > >> The codebase supplies a script, ./autogen.sh . It runs autoconf, > >> libtool, etc. etc. and regenerates the ./configure script, > >> incorporating the fixes from the patched macros > >> > >> I see two ways to solve this: > >> a) tell the Portfile to use ./autogen.sh as the configure command. Let > >> every user rebuild the ./configure script > >> b) patch the ./configure script with the same fixes as the autoconf > >> macros > > > > ...We have a shortcut for option a) by the way: 'use_autoreconf yes'. > That > > adds the needed deps for you and usually works fine on its own (plus > > 'autoreconf.args -fi' if you want to regenerate absolutely everything > > from scratch), but if special options are needed, you can also set > > 'autoreconf.cmd ./autogen.sh' > > Thank you for the reply, Joshua. > > You mention autoreconf. I just learned something new! I did not know > about autoreconf[1], a tool included in autoconf. > > It looks like upstream does not use autoreconf. They have a shell script > `autogen.sh`. For all I know it is something they created themselves. > Like autoreconf, it runs autoconf, automake, libtoolize, etc. etc. I > have no assurance that I can replace it with autoreconf. > > Thus I think I will try running ./autogen.sh in place of ./configure, > and see how far that gets me. It is good to know that patching > ./configure is not out of the question. > > Best regards, > --Jim DeLaHunt > > [1] > < > https://www.gnu.org/savannah-checkouts/gnu/autoconf/manual/autoconf-2.71/html_node/autoreconf-Invocation.html > > >
Re: license_noconflict ignored on builders?
Yes, I went back and re-read that before I posted. The second example might qualify as having answered my question, but since "openssl" is both the name of a port and one of the licenses, maybe that's not the best example to be using? Especially considering that the keyword just above "license_noconflict" is "license", and the second example in that entry uses "freetype", which again is both the name of a port and one of the licenses. -- Jason Liu On Fri, Jan 14, 2022 at 7:23 PM Ryan Schmidt wrote: > On Jan 14, 2022, at 18:14, Jason Liu wrote: > > > Oh, was license_noconflict supposed to be used with the name of a port? > I thought it was supposed to specify a license that didn't conflict! > > See the documentation for license_noconflict at > > https://guide.macports.org/#reference.keywords > >
Re: license_noconflict ignored on builders?
Oh, was license_noconflict supposed to be used with the name of a port? I thought it was supposed to specify a license that didn't conflict! -- Jason Liu On Fri, Jan 14, 2022 at 6:35 PM Joshua Root wrote: > On 2022-1-15 03:58 , Jason Liu wrote: > > I added a license_noconflict > > <https://github.com/macports/macports-ports/pull/13641> to one of my > > portfiles a couple of days ago, and it appears that the builders are > > still flagging it as not distributable: > > > > "warzone2100" is not distributable because its license "gpl" conflicts > > with license "OpenSSL" of dependency "openssl11" > > > > Is it possible that the license_noconflict is being ignored? > > The Portfile says "license_noconflict openssl". It doesn't say > anything about openssl11. > > Looking at the explanation in the comment, it looks like you actually > mean "license_noconflict asciidoctor"? > > - Josh >
license_noconflict ignored on builders?
I added a license_noconflict <https://github.com/macports/macports-ports/pull/13641> to one of my portfiles a couple of days ago, and it appears that the builders are still flagging it as not distributable: "warzone2100" is not distributable because its license "gpl" conflicts with license "OpenSSL" of dependency "openssl11" Is it possible that the license_noconflict is being ignored? -- Jason Liu
Question regarding supported_archs noarch
Hi all, A hopefully quick and simple question regarding 'supported_archs noarch'. Is 'noarch' meant to specify that the build itself isn't supposed to be architecture-specific, or does it mean that the final output products are not architecture-specific? I have a subport that basically runs through the entire build, including compiling the source code, but then in the end simply throws away those compiled objects, and the only files that are retained (i.e. staged into destroot) are the game data files, which are basically just a bunch of text files and images (which are not architecture-specific). It seems that the upstream developers wrote the makefiles this way because some of the game data files get generated during the compile process, and they never bothered to cleanly split off the generating of the game data files into a separate makefile that can be run by itself. -- Jason Liu
Re: [10.6.8] certbot & pyOpenSSL problems
> > One reason, I'm staying on 10.6.8 is that I really like Spaces. I'm > (primarily) using Remote Desktop to admin them as they are "head-less". > Another reason not going the Linux way is, that I find the macOS > desktop to be far superior to the Linux desktops. > You might be surprised. There have been a lot of advancements in terms of Linux desktop polish, and you can even turn your entire desktop environment into an (almost) exact clone of macOS' behavior, including Exposé and Spaces, and even the application dock along the bottom. If you've ever heard of Linus Tech Tips, they're even currently doing a multi-part video series on "daily computing and gaming on Linux by a couple of people who have never used Linux" challenge. -- Jason Liu On Sun, Dec 19, 2021 at 9:41 AM Bjarne D Mathiesen wrote: > Ken Cunningham wrote: > >> System Version: Mac OS X 10.6.8 (10K549) Kernel Version: Darwin 10.8.0 > > > > It appears that you are running a 10.6.8 system (on a macmini, I think I > saw?) > > Correct, I've got three of those : > PowerMac10,1 w/ PowerPC 7447A (G4) > Presently, I don't use it > > MacMini1,1 (originally Intel Core Duo 1.66 GHz) > MacMini2,1 > both of these have been upgraded to > Intel Core 2 Duo 2 GHz > 4 GB RAM > although the MacMini1,1 is only able to see 2 GB [crying] > and the MacMini2,1 can only use 3 GB of the 4 GB > > All of them have 2 x original OWC miniStack w/ the HDs in AppleRAID-1 > > > > and I think you appear to be using it as a server for various things. > > Correct - they have the full xAMP stack > Apache 2.4 - MySQL 5.7 - PHP 8.1 > Postfix - Dovecot - sqlgrey > > > > > In another spot, I see that it is a 32bit EFI Mac: > > > >> platform: macOS-10.6.8-i386-64bit > > Also correct > > > > > If you are not using for much else beyond being a server, I thought I > would let you know that this tiny like c program: > > > > https://github.com/demonicsweaters/make_single_eltorito.c > > > > that compiles in 10 seconds or so, will slightly modify an Ubuntu 64 bit > installer image to properly boot a 32bit EFI Mac, and you can then install > Ubuntu 64 bit on there. > > > > Once it is installed, you’re good to go, and you can use 64bit software > everywhere, including any 64 bit software you download rather than get from > the main repo, like Spotify, Google Chrome, etc. > > > > I have this now running on 3 older Macs with 32bit EFI, and it works > just beautifully. And if you are just using that as a server, you would be > most likely many miles ahead to be running Ubuntu 20.04 with webmin than > trying to keep teasing SnowLeopard along as a server. > > > > Just my $0.02 for whatever. > > [thumbs-up][nerd] Thank you for the hint [nerd][thumbs-up] > > Presently, I'm very much satisfied with my setup, but going to Linux on > them is definitely one of the options I'm considering if I get into too > many problems w/ being up-to-date. This is the 1st really serious > problem I've had where I've had to abandon the MacMini for an > alternative solution. Of course, not being able to upgrade to MySQL 8 is > a bit of a nuisance, but not something I'm presently worrying about. > > One reason, I'm staying on 10.6.8 is that I really like Spaces. > I'm (primarily) using Remote Desktop to admin them as they are "head-less". > > Another reason not going the Linux way is, that I find the macOS desktop > to be far superior to the Linux desktops. > > > > > Ken > > > > (PS - dual booting a MacOSX install of 10.6 or 10.7 works just fine too, > using ReFIND: https://www.rodsbooks.com/refind/ ) > > > > Thanks. > One reason to originally staying on 10.6.8 was that the MacMini1,1 isn't > able to go to 10.7 without some hacking - at least not before upgrading > the CPU. And I like to keep them at par on the software as the > MacMini1,1 is my testing platform and backup. > > -- > Bjarne D Mathiesen > Korsør ; Danmark ; Europa > --- > denne besked er skrevet i et totalt M$-frit miljø > MacPro 2010 ; OpenCore + macOS 10.15.7 Catalina > 2 x 3,46 GHz 6-Core Intel Xeon ; 256 GB 1333 MHz DDR3 ECC RDIMM > ATI Radeon RX 590 8 GB >
Re: fetch phase: sourceforge with 302 redirects?
Yes, using the `curl -IL` command to obtain the part of the path I need to extract (which is very slightly different from the web browser URL), did the trick. Thanks! :) -- Jason Liu On Thu, Dec 16, 2021 at 4:29 PM Daniel J. Luke wrote: > On Dec 16, 2021, at 4:24 PM, Jason Liu wrote: > > I'm working on a new portfile that has its source stored on sourceforge. > MacPorts is having trouble obtaining the tarball, because apparently the > mirrors are pointing to the wrong file, and if I put the full URL into > `master_sites`, it's unable to find the tarball at all. It seems that > sourceforge is using 301 redirects to point to the actual file. If I use > the URL with a `curl -L`, the correct file downloads just fine. Is there > any way to get MacPorts to follow redirects during the fetch phase? If at > all possible, I'd like to avoid manually using curl through a `system` > call, but I suppose it could work as a last resort. > > Does this sourceforge example help? > > https://trac.macports.org/wiki/howto/AvoidRedirects > > -- > Daniel J. Luke > >
fetch phase: sourceforge with 302 redirects?
Hi everyone, I'm working on a new portfile that has its source stored on sourceforge. MacPorts is having trouble obtaining the tarball, because apparently the mirrors are pointing to the wrong file, and if I put the full URL into ` master_sites`, it's unable to find the tarball at all. It seems that sourceforge is using 301 redirects to point to the actual file. If I use the URL with a `curl -L`, the correct file downloads just fine. Is there any way to get MacPorts to follow redirects during the fetch phase? If at all possible, I'd like to avoid manually using curl through a `system` call, but I suppose it could work as a last resort. -- Jason Liu
Re: Question about `platforms` and `${os.platform}`
On Sun, Dec 12, 2021 at 3:03 AM Joshua Root wrote:
> On 2021-12-12 11:54 , Jason Liu wrote:
>
>> On Sat, Dec 11, 2021 at 6:41 PM Chris Jones > <mailto:jon...@hep.phy.cam.ac.uk>> wrote:
>>
>>>
>>> No, because that would render the port non functional on non darwin
>>> OSes. You should only specify the darwin platform when it is
>>> actually required, e.g. when then making a os.major conditional that
>>> inly makes sense in darwin platforms.
>>
>>
>>
>> Wait a minute, didn't Ryan say that "We don't really expect many ports
>> to be installable on other operating systems or for maintainers to test
>> anything on other operating systems"? So we /should/ worry about the
>> port being non-functional on non-Darwin OSes? I'm confused now
>
>
> Ports need to be "functional" on non-darwin platforms to the extent that
> things like portindex, 'port mirror', and 'port info' work. They don't
> need to be actually installable.
Ah, so it seems like you're saying that the metadata-related stuff that we
typically place near the top of the portfiles (e.g. name, version,
categories, description, etc.) should all still be "functional" outside of
any sort of specific platform context (and thus shouldn't be wrapped inside
of a `platform darwin {`), but that it would be alright to wrap the stuff
related to the build phases (i.e. fetch, extract, patch, configure, build,
destroot, etc.) inside of a `platform darwin {` to hide it from other
platforms, because the build phases only relate to the "installable" part?
Am I getting that right? Or am I still confused?
--
Jason Liu
On Sun, Dec 12, 2021 at 3:03 AM Joshua Root wrote:
> On 2021-12-12 11:54 , Jason Liu wrote:
> > On Sat, Dec 11, 2021 at 6:41 PM Chris Jones > <mailto:jon...@hep.phy.cam.ac.uk>> wrote:
> >
> > No, because that would render the port non functional on non darwin
> > OSes. You should only specify the darwin platform when it is
> > actually required, e.g. when then making a os.major conditional that
> > inly makes sense in darwin platforms.
> >
> >
> > Wait a minute, didn't Ryan say that "We don't really expect many ports
> > to be installable on other operating systems or for maintainers to test
> > anything on other operating systems"? So we /should/ worry about the
> > port being non-functional on non-Darwin OSes? I'm confused now
>
> Ports need to be "functional" on non-darwin platforms to the extent that
> things like portindex, 'port mirror', and 'port info' work. They don't
> need to be actually installable.
>
> > That's why I originally thought that `platforms darwin` implied that the
> > portfile was only intended to be valid on Darwin. Now that I've learned
> > that `platforms darwin` doesn't actually do anything, I'm just...
> confused.
>
> It means the port is expected to only be installable on darwin. Ports
> need to be more or less "valid" anywhere. The platforms option "doesn't
> actually do anything" in precisely the same way that statement is true
> for the description, i.e. what it does is simply make that information
> known to users.
>
> - Josh
>
Re: Question about `platforms` and `${os.platform}`
On Sat, Dec 11, 2021 at 6:41 PM Chris Jones
wrote:
> No, because that would render the port non functional on non darwin OSes.
> You should only specify the darwin platform when it is actually required,
> e.g. when then making a os.major conditional that inly makes sense in
> darwin platforms.
>
Wait a minute, didn't Ryan say that "We don't really expect many ports to
be installable on other operating systems or for maintainers to test
anything on other operating systems"? So we *should* worry about the port
being non-functional on non-Darwin OSes? I'm confused now
That's why I originally thought that `platforms darwin` implied that the
portfile was only intended to be valid on Darwin. Now that I've learned
that `platforms darwin` doesn't actually do anything, I'm just... confused.
--
Jason Liu
On Sat, Dec 11, 2021 at 6:41 PM Chris Jones
wrote:
>
>
> On 11 Dec 2021, at 10:42 pm, Jason Liu wrote:
>
>
> On Sat, Dec 11, 2021 at 5:20 PM Ryan Schmidt
> wrote:
>
>>
>>
>> On Dec 10, 2021, at 15:07, Jason Liu wrote:
>>
>>>
>>> A conversation in one of my PRs has brought up an interesting question
>>> that I've been wondering about for a long time. In Portfiles, whenever I've
>>> had a test for `${os.major} <= xx`, I've typically always added an
>>> additional check for darwin in the front, i.e.:
>>>
>>> if {${os.platform} eq "darwin" && ${os.major} <= xx} {
>>>
>>> I've done it that way because I basically copied what I saw from other
>>> Portfiles, and because I get gently admonished by the committers when I
>>> forget to.
>>
>>
>> Yes, please always do that.
>
>
> Then would it be easier (or even kosher) to simply wrap the majority of
> the Portfile inside of a single
>
> if {${os.platform} eq "darwin"} {
>
> and be done with it, instead of needing to add one each and every time I
> have a conditional involving `${os.major}`?
>
>
> No, because that would render the port non functional on non darwin OSes.
> You should only specify the darwin platform when it is actually required,
> e.g. when then making a os.major conditional that inly makes sense in
> darwin platforms.
>
> So basically, following the recommendations as currently given.
>
> Chris
>
>
> I was under the impression that the `platforms darwin` line means that the
>>> entire Portfile is supposed to be valid only for `${os.platform} eq
>>> "darwin"`, no? (In other words, my understanding is that a line such as
>>> `platforms darwin freebsd openbsd` is meant to signify that "this Portfile
>>> is supposed to be valid for the listed platforms".) If that's not the case,
>>> then what is the purpose of `platforms darwin`?
>>
>>
>> The platforms line is not used by MacPorts in any way at this time, other
>> than to display it in the output of "port info". There is a ticket about
>> possibly using it in the future as a way to indicate which OS versions the
>> port is compatible with, but I don't think that got beyond the idea phase.
>>
>
> That seems like a pity, and a bit of a waste, considering that platforms
> is being included in each and every Portfile.
>
> --
> Jason Liu
>
>
> On Sat, Dec 11, 2021 at 5:20 PM Ryan Schmidt
> wrote:
>
>>
>>
>> On Dec 10, 2021, at 15:07, Jason Liu wrote:
>>
>> > A conversation in one of my PRs has brought up an interesting question
>> that I've been wondering about for a long time. In Portfiles, whenever I've
>> had a test for `${os.major} <= xx`, I've typically always added an
>> additional check for darwin in the front, i.e.:
>> >
>> > if {${os.platform} eq "darwin" && ${os.major} <= xx} {
>> >
>> > I've done it that way because I basically copied what I saw from other
>> Portfiles, and because I get gently admonished by the committers when I
>> forget to.
>>
>> Yes, please always do that.
>>
>>
>> > But I've also always wondered why it's necessary.
>>
>> Your Portfile could be parsed or accessed by non-Darwin operating
>> systems. For example, before we took over server hosting duties in 2016,
>> all of the server VMs including the one that generated the PortIndex files
>> (except the actual macOS build machines) were running Linux; perhaps some
>> day we will once again want to try using a server OS other than macOS for
>> this task. There are even some users using Linux to test various things in
>> MacPorts. We don't really expect many ports to be installable on oth
Re: Question about `platforms` and `${os.platform}`
On Sat, Dec 11, 2021 at 5:46 PM Ryan Schmidt
wrote:
> On Dec 11, 2021, at 16:41, Jason Liu wrote:
>
>>
>> and be done with it, instead of needing to add one each and every time I
>> have a conditional involving `${os.major}`?
>
>
> I wouldn't expect you to need to write more than a couple such
> conditionals in a portfile so I wouldn't expect it to be that inconvenient.
Actually, I find myself needing to use them more and more. For example, I
now often have to check for macOS < 10.12, due to the large overhaul that
happened in AppKit between 10.11 and 10.12. My workarounds for older
versions of macOS are contained inside those `if {${os.major} <= 15}`
conditionals.
Josh did recently commit "darwin" as a default for platforms, so it is no
> longer mandatory to explicitly put a platforms line in each portfile.
>
Does this mean that we can start removing `platforms darwin` from our
portfiles? Or is it one of those "despite the fact that it's a default, you
have to put it in the file anyway"?
--
Jason Liu
On Sat, Dec 11, 2021 at 5:46 PM Ryan Schmidt
wrote:
> On Dec 11, 2021, at 16:41, Jason Liu wrote:
>
> > Then would it be easier (or even kosher) to simply wrap the majority of
> the Portfile inside of a single
> >
> > if {${os.platform} eq "darwin"} {
>
> The shorthand way to do that is:
>
> platform darwin {
>
>
> > and be done with it, instead of needing to add one each and every time I
> have a conditional involving `${os.major}`?
>
> I wouldn't expect you to need to write more than a couple such
> conditionals in a portfile so I wouldn't expect it to be that inconvenient.
>
> We use platform darwin blocks to isolate things that are obviously
> Mac-specific. It would be odd to see such a block enclosing most of a
> portfile, even bits which are obviously not Mac-specific.
>
>
> >> The platforms line is not used by MacPorts in any way at this time,
> other than to display it in the output of "port info". There is a ticket
> about possibly using it in the future as a way to indicate which OS
> versions the port is compatible with, but I don't think that got beyond the
> idea phase.
> >
> > That seems like a pity, and a bit of a waste, considering that platforms
> is being included in each and every Portfile.
>
> Well, it's as much a pity as any unimplemented feature in MacPorts.
>
> Josh did recently commit "darwin" as a default for platforms, so it is no
> longer mandatory to explicitly put a platforms line in each portfile.
>
>
Re: Question about `platforms` and `${os.platform}`
On Sat, Dec 11, 2021 at 5:20 PM Ryan Schmidt
wrote:
>
>
> On Dec 10, 2021, at 15:07, Jason Liu wrote:
>
>>
>> A conversation in one of my PRs has brought up an interesting question
>> that I've been wondering about for a long time. In Portfiles, whenever I've
>> had a test for `${os.major} <= xx`, I've typically always added an
>> additional check for darwin in the front, i.e.:
>>
>> if {${os.platform} eq "darwin" && ${os.major} <= xx} {
>>
>> I've done it that way because I basically copied what I saw from other
>> Portfiles, and because I get gently admonished by the committers when I
>> forget to.
>
>
> Yes, please always do that.
Then would it be easier (or even kosher) to simply wrap the majority of the
Portfile inside of a single
if {${os.platform} eq "darwin"} {
and be done with it, instead of needing to add one each and every time I
have a conditional involving `${os.major}`?
I was under the impression that the `platforms darwin` line means that the
>> entire Portfile is supposed to be valid only for `${os.platform} eq
>> "darwin"`, no? (In other words, my understanding is that a line such as
>> `platforms darwin freebsd openbsd` is meant to signify that "this Portfile
>> is supposed to be valid for the listed platforms".) If that's not the case,
>> then what is the purpose of `platforms darwin`?
>
>
> The platforms line is not used by MacPorts in any way at this time, other
> than to display it in the output of "port info". There is a ticket about
> possibly using it in the future as a way to indicate which OS versions the
> port is compatible with, but I don't think that got beyond the idea phase.
>
That seems like a pity, and a bit of a waste, considering that platforms is
being included in each and every Portfile.
--
Jason Liu
On Sat, Dec 11, 2021 at 5:20 PM Ryan Schmidt
wrote:
>
>
> On Dec 10, 2021, at 15:07, Jason Liu wrote:
>
> > A conversation in one of my PRs has brought up an interesting question
> that I've been wondering about for a long time. In Portfiles, whenever I've
> had a test for `${os.major} <= xx`, I've typically always added an
> additional check for darwin in the front, i.e.:
> >
> > if {${os.platform} eq "darwin" && ${os.major} <= xx} {
> >
> > I've done it that way because I basically copied what I saw from other
> Portfiles, and because I get gently admonished by the committers when I
> forget to.
>
> Yes, please always do that.
>
>
> > But I've also always wondered why it's necessary.
>
> Your Portfile could be parsed or accessed by non-Darwin operating systems.
> For example, before we took over server hosting duties in 2016, all of the
> server VMs including the one that generated the PortIndex files (except the
> actual macOS build machines) were running Linux; perhaps some day we will
> once again want to try using a server OS other than macOS for this task.
> There are even some users using Linux to test various things in MacPorts.
> We don't really expect many ports to be installable on other operating
> systems or for maintainers to test anything on other operating systems, but
> just take 2 seconds when you're writing an OS version conditional to think
> about what you're trying to express, and then express it, including
> checking os.platform. Typical forms include the one you mentioned:
>
> if {${os.platform} eq "darwin" && ${os.major} <= xx}
>
> (<, <=, ==, >=, >)
>
> And the other one:
>
> if {${os.platform} ne "darwin" || ${os.major} <= xx}
>
> (<, <=, ==, >=, >)
>
> Usually the decision about which to use comes down to whether the thing
> you're doing is Mac-specific or not. For example, if you were writing a
> conditional to use the Security framework on recent macOS and openssl on
> older macOS, what should happen if perchance the port is used on non-macOS?
> In this case, the answer is that frameworks are a Mac thing, so you would
> want to use openssl for non-macOS.
>
>
> > I was under the impression that the `platforms darwin` line means that
> the entire Portfile is supposed to be valid only for `${os.platform} eq
> "darwin"`, no? (In other words, my understanding is that a line such as
> `platforms darwin freebsd openbsd` is meant to signify that "this Portfile
> is supposed to be valid for the listed platforms".) If that's not the case,
> then what is the purpose of `platforms darwin`?
>
> The platforms line is not used by MacPorts in any way at this time, other
> than to display it in the output of "port info". There is a ticket about
> possibly using it in the future as a way to indicate which OS versions the
> port is compatible with, but I don't think that got beyond the idea phase.
>
>
>
Re: Significant security vulnerability discovered in Log4j
On Sat, Dec 11, 2021 at 1:32 PM Eric Gallager wrote: > > so... is there anything to do about this in MacPorts? > There's probably nothing that can be done in terms of the MacPorts packages. It's basically dependent on upstream developers to patch anything that might be affected. It was more of a general warning to anyone on the mailing list that might be running a web server. ...I don't think any of these are the same thing, are they? > Based on my googling, jakarta-log4j is some sort of wrapper that allows Jakarta to use log4j, so it's quite possible that the jakarta-log4j package is affected. Depending on how closely the C++ port follows the original Java in the log4cxx package, it might also be affected; the same applies to the log4perl packages. -- Jason Liu On Sat, Dec 11, 2021 at 1:32 PM Eric Gallager wrote: > On Fri, Dec 10, 2021 at 6:00 PM Jason Liu wrote: > > > > In case everyone hadn't heard the news. If anyone is running Log4j for > logging on any of your web servers, you might want to read this. > > > > WIRED: 'The Internet Is On Fire' > > A vulnerability in the Log4j logging framework has security teams > scrambling to put in a fix. > > > > -- > > Jason Liu > > so... is there anything to do about this in MacPorts? > > $ port search log4j > jakarta-log4j @1.2.16 (java, devel) > Java logging API > > log4cxx @0.10.0_1 (devel) > log4cxx is a port to C++ of the log4j project > > log4jdbc @1.1 (java) > JDBC driver that can log SQL and/or JDBC calls > > p5-log-dispatch-config @1.40.0 (perl) > Log::Dispatch::Config - Log4j for Perl > > p5-log-log4perl @1.540.0 (perl) > Log4j implementation for Perl > > p5.28-log-dispatch-config @1.40.0 (perl) > Log::Dispatch::Config - Log4j for Perl > > p5.28-log-log4perl @1.540.0 (perl) > Log4j implementation for Perl > > p5.30-log-dispatch-config @1.40.0 (perl) > Log::Dispatch::Config - Log4j for Perl > > p5.30-log-log4perl @1.540.0 (perl) > Log4j implementation for Perl > > p5.32-log-dispatch-config @1.40.0 (perl) > Log::Dispatch::Config - Log4j for Perl > > p5.32-log-log4perl @1.540.0 (perl) > Log4j implementation for Perl > > Found 11 ports. > $ port installed `port -q search log4j` > The following ports are currently installed: > jakarta-log4j @1.2.16_0 (active) > log4jdbc @1.1_0 (active) > p5.28-log-log4perl @1.540.0_0 (active) > p5.30-log-log4perl @1.540.0_0 (active) > p5.32-log-log4perl @1.540.0_0 (active) > $ > > ...I don't think any of these are the same thing, are they? >
Significant security vulnerability discovered in Log4j
In case everyone hadn't heard the news. If anyone is running Log4j for logging on any of your web servers, you might want to read this. WIRED: 'The Internet Is On Fire' <https://www.wired.com/story/log4j-flaw-hacking-internet/> A vulnerability in the Log4j logging framework has security teams scrambling to put in a fix. -- Jason Liu
Question about `platforms` and `${os.platform}`
Hi everyone,
A conversation in one of my PRs has brought up an interesting question that
I've been wondering about for a long time. In Portfiles, whenever I've had
a test for `${os.major} <= xx`, I've typically always added an additional
check for darwin in the front, i.e.:
if {${os.platform} eq "darwin" && ${os.major} <= xx} {
I've done it that way because I basically copied what I saw from other
Portfiles, and because I get gently admonished by the committers when I
forget to. But I've also always wondered why it's necessary. I was under
the impression that the `platforms darwin` line means that the entire
Portfile is supposed to be valid only for `${os.platform} eq "darwin"`, no?
(In other words, my understanding is that a line such as `platforms darwin
freebsd openbsd` is meant to signify that "this Portfile is supposed to be
valid for the listed platforms".) If that's not the case, then what is the
purpose of `platforms darwin`?
--
Jason Liu
Re: Remove pending builds from buildbot?
The builds from the previous commit were fine, and completed successfully. If the builders are smart enough to detect that nothing was changed in the Portfile and are able to skip trying to recompile the package, then I suppose it would be fine to simply leave the build requests in the queue. -- Jason Liu On Mon, Dec 6, 2021 at 2:40 PM Chris Jones wrote: > > Normally making a commit that does not change the revision or version will > not trigger a rebuild, assuming the previous port revision built OK on a > given platform. If this is the case then again the builds do not need to be > cancelled. If the previous builds in fact did not build, then yes another > rebuild will be attempted, so if nothing has changed to fix the previous > build failure it will fail again. If this is the case then its best not > avoid making such commits until you are able to do so at the same times as > fixing the build failures. If this cannot be done the the port should use > the known_fail pattern for the OSes known to fail, to avoid needless > failing rebuilds. > > Chris > > On 6 Dec 2021, at 7:35 pm, Jason Liu wrote: > > > Neither the revision nor the version numbers were changed in the Portfile > (in fact, no changes were made to the Portfile at all), only the new file > got added to the files/ folder. But builds got queued up on the buildbot > anyway after the PR got merged. > > Adding the file to the port (PR #12976 > <https://github.com/macports/macports-ports/pull/12976>) was only in > preparation for the PR that actually includes bugfixes (PR #13266 > <https://github.com/macports/macports-ports/pull/13266>), which obviously > will change what is installed, and do indeed need builds to be rerun. I'm > trying to get the queued builds for PR #12976 removed from the builders. > > -- > Jason Liu > > > On Mon, Dec 6, 2021 at 2:02 PM Chris Jones > wrote: > >> Hi, >> >> If the revision and version numbers have changed, because what is >> installed on disc has changed, then the builds need to be rerun, otherwise >> you are forcing the long builds on users, which is worse than tying up the >> buildbots for a while. >> >> Cheers Chris >> >> On 6 Dec 2021, at 6:10 pm, Jason Liu wrote: >> >> >> Hi all, >> >> Would it be possible to remove the queued builds for my PR that was >> recently merged on the 10.13 >> <https://build.macports.org/builders/ports-10.13_x86_64-watcher> and >> 12_x86_64 <https://build.macports.org/builders/ports-12_x86_64-watcher> >> watchers? >> That PR only added an auxiliary file, and really shouldn't be re-built. >> Since there's such a long queue on the 10.13 and 12 builders, I'm thinking >> that cancelling a fairly length build that isn't needed would help clear up >> the backlog a bit faster. If there's an easy way to cancel the build on all >> of the builders (not just 10.13 and 12), that would probably be even >> better. On the other hand, if cancelling the build would be a hassle, then >> don't worry about it. >> >> The commit is identified by the hash >> >> 14e7b489f630... >> (Jason Liu ) >> >> -- >> Jason Liu >> >>
Re: Remove pending builds from buildbot?
Neither the revision nor the version numbers were changed in the Portfile (in fact, no changes were made to the Portfile at all), only the new file got added to the files/ folder. But builds got queued up on the buildbot anyway after the PR got merged. Adding the file to the port (PR #12976 <https://github.com/macports/macports-ports/pull/12976>) was only in preparation for the PR that actually includes bugfixes (PR #13266 <https://github.com/macports/macports-ports/pull/13266>), which obviously will change what is installed, and do indeed need builds to be rerun. I'm trying to get the queued builds for PR #12976 removed from the builders. -- Jason Liu On Mon, Dec 6, 2021 at 2:02 PM Chris Jones wrote: > Hi, > > If the revision and version numbers have changed, because what is > installed on disc has changed, then the builds need to be rerun, otherwise > you are forcing the long builds on users, which is worse than tying up the > buildbots for a while. > > Cheers Chris > > On 6 Dec 2021, at 6:10 pm, Jason Liu wrote: > > > Hi all, > > Would it be possible to remove the queued builds for my PR that was > recently merged on the 10.13 > <https://build.macports.org/builders/ports-10.13_x86_64-watcher> and > 12_x86_64 <https://build.macports.org/builders/ports-12_x86_64-watcher> > watchers? > That PR only added an auxiliary file, and really shouldn't be re-built. > Since there's such a long queue on the 10.13 and 12 builders, I'm thinking > that cancelling a fairly length build that isn't needed would help clear up > the backlog a bit faster. If there's an easy way to cancel the build on all > of the builders (not just 10.13 and 12), that would probably be even > better. On the other hand, if cancelling the build would be a hassle, then > don't worry about it. > > The commit is identified by the hash > > 14e7b489f630... > (Jason Liu ) > > -- > Jason Liu > >
Remove pending builds from buildbot?
Hi all, Would it be possible to remove the queued builds for my PR that was recently merged on the 10.13 <https://build.macports.org/builders/ports-10.13_x86_64-watcher> and 12_x86_64 <https://build.macports.org/builders/ports-12_x86_64-watcher> watchers? That PR only added an auxiliary file, and really shouldn't be re-built. Since there's such a long queue on the 10.13 and 12 builders, I'm thinking that cancelling a fairly length build that isn't needed would help clear up the backlog a bit faster. If there's an easy way to cancel the build on all of the builders (not just 10.13 and 12), that would probably be even better. On the other hand, if cancelling the build would be a hassle, then don't worry about it. The commit is identified by the hash 14e7b489f630... (Jason Liu ) -- Jason Liu
PR 12976: request for review and merge
Could someone take a quick look, and if there aren't any issues, merge it? I have a series of bugfixes that are waiting for this to get merged first. https://github.com/macports/macports-ports/pull/12976 -- Jason Liu
Re: 10.15 Xcode version: Buildbot, vs. GitHub CI
The code isn't actually mine... I simply took the code from other
Portfiles, e.g. the wxWidgets 1.0 PortGroup
<https://github.com/macports/macports-ports/blob/master/_resources/port1.0/group/wxWidgets-1.0.tcl#L206-L208>,
phantomjs-qt
<https://github.com/macports/macports-ports/blob/master/aqua/phantomjs-qt/Portfile#L375-L384>,
qt5
<https://github.com/macports/macports-ports/blob/master/aqua/qt5/Portfile#L1123-L1132>,
qt6
<https://github.com/macports/macports-ports/blob/master/aqua/qt6/Portfile#L664-L669>,
etc. Although admittedly, the code in those Portfiles are slightly more
generalized than mine, since I'm only interested in the specific case of
the macOS 10.12 SDK on macOS 10.11.
--
Jason Liu
On Wed, Dec 1, 2021 at 12:50 PM Chris Jones
wrote:
>
> you are making a number of assumptions in the code below on where things
> are installed, that I don't think are universally valid. I think it
> would be better to use the macPort utilities to query the SDK version
> and/or Xcode version instead.
>
> Chris
>
> On 01/12/2021 5:14 pm, Jason Liu wrote:
> > A very similar situation occurs on macOS 10.11: It's fairly common to
> > install Xcode 8.2.1 on macOS 10.11, and Xcode 8.2.1 comes with the macOS
> > 10.12 SDK. So, in my upcoming fixes that allows godot to compile on
> > older macOSes, I have the following check:
> >
> > if {${os.platform} eq "darwin" && ${os.major} <= 15} {
> >
> > set sdks_dir
> ${developer_dir}/Platforms/MacOSX.platform/Developer/SDKs
> > set add_appkit_wrapper yes
> > if {![catch {file lstat $sdks_dir/MacOSX10.12.sdk finfo}]} {
> > set add_appkit_wrapper no
> > }
> > }
> >
> > Thus, if the Portfile detects the situation of the macOS 10.12 SDK being
> > installed on macOS 10.11, then it won't add my AppKit compatibility
> > wrapper file.
> >
> > I suspect a similar technique might need to be put in place to account
> > for Xcode 11 vs 12 being installed on macOS 10.15.
> >
> > --
> > Jason Liu
> >
> >
> > On Wed, Dec 1, 2021 at 10:32 AM Chris Jones > <mailto:jon...@hep.phy.cam.ac.uk>> wrote:
> >
> > Hi,
> >
> > On 01/12/2021 3:18 pm, Christopher Nielsen wrote:
> > > Just encountered an interesting situation, where a Swift-based
> > port builds successfully via 10.15 CI, but fails on our buildbot.
> > >
> > > This appears to be related to Xcode versions: Presently our 10.15
> > buildbot has Xcode 11.7, whereas 10.15 CI has Xcode 12.4.
> > >
> > > This brings up two questions:
> > > * Would it be feasible to update our 10.15 buildbot to a newer
> > Xcode release? Or are there certain ports/situations that
> > necessitate remaining with 11.7?
> > > * Do we have control over the Xcode version used for GitHub CI,
> > or is 12.4 the only option?
> > >
> > > Ideally both should utilize the same Xcode 12 release,
> > specifically one that still ships with the 10.15 SDK. So the choices
> > would be 12.0, 12.0.1, and 12.1.
> > >
> > > Thoughts?
> > >
> >
> > MacPorts cannot mandate what Xcode a user has installed. Both Xcode
> 11
> > and 12 are valid options to have on macOS10.15 (and, I might be wrong
> > here, but from memory Ryan specifically keeps the builder on Xcode
> > 11 to
> > avoid issues with Xcode 12 also shipping the macOS11 SDK).
> >
> > So basically the port needs to handle both, in whatever way is
> > appropriate. I guess this is the new mint port
> >
> >
> >
> https://github.com/macports/macports-ports/commit/cfc6d01aa685a5a9cc30264bc2a7e9d1badf587e
> > <
> https://github.com/macports/macports-ports/commit/cfc6d01aa685a5a9cc30264bc2a7e9d1badf587e
> >
> >
> > I see there is a check in there on the Darwin version. It sounds this
> > this should be changed to a test specifically on the Xcode(CLT)
> > versions
> > installed, if the requirement is really Xcode 12 and above, and not
> > really the Darwin version.
> >
> > Chris
> >
>
Re: 10.15 Xcode version: Buildbot, vs. GitHub CI
A very similar situation occurs on macOS 10.11: It's fairly common to
install Xcode 8.2.1 on macOS 10.11, and Xcode 8.2.1 comes with the macOS
10.12 SDK. So, in my upcoming fixes that allows godot to compile on older
macOSes, I have the following check:
if {${os.platform} eq "darwin" && ${os.major} <= 15} {
set sdks_dir ${developer_dir}/Platforms/MacOSX.platform/Developer/SDKs
set add_appkit_wrapper yes
if {![catch {file lstat $sdks_dir/MacOSX10.12.sdk finfo}]} {
set add_appkit_wrapper no
}
}
Thus, if the Portfile detects the situation of the macOS 10.12 SDK being
installed on macOS 10.11, then it won't add my AppKit compatibility wrapper
file.
I suspect a similar technique might need to be put in place to account for
Xcode 11 vs 12 being installed on macOS 10.15.
--
Jason Liu
On Wed, Dec 1, 2021 at 10:32 AM Chris Jones
wrote:
> Hi,
>
> On 01/12/2021 3:18 pm, Christopher Nielsen wrote:
> > Just encountered an interesting situation, where a Swift-based port
> builds successfully via 10.15 CI, but fails on our buildbot.
> >
> > This appears to be related to Xcode versions: Presently our 10.15
> buildbot has Xcode 11.7, whereas 10.15 CI has Xcode 12.4.
> >
> > This brings up two questions:
> > * Would it be feasible to update our 10.15 buildbot to a newer Xcode
> release? Or are there certain ports/situations that necessitate remaining
> with 11.7?
> > * Do we have control over the Xcode version used for GitHub CI, or is
> 12.4 the only option?
> >
> > Ideally both should utilize the same Xcode 12 release, specifically one
> that still ships with the 10.15 SDK. So the choices would be 12.0, 12.0.1,
> and 12.1.
> >
> > Thoughts?
> >
>
> MacPorts cannot mandate what Xcode a user has installed. Both Xcode 11
> and 12 are valid options to have on macOS10.15 (and, I might be wrong
> here, but from memory Ryan specifically keeps the builder on Xcode 11 to
> avoid issues with Xcode 12 also shipping the macOS11 SDK).
>
> So basically the port needs to handle both, in whatever way is
> appropriate. I guess this is the new mint port
>
>
>
> https://github.com/macports/macports-ports/commit/cfc6d01aa685a5a9cc30264bc2a7e9d1badf587e
>
> I see there is a check in there on the Darwin version. It sounds this
> this should be changed to a test specifically on the Xcode(CLT) versions
> installed, if the requirement is really Xcode 12 and above, and not
> really the Darwin version.
>
> Chris
>
Re: Question regarding builders and require_active_variants
Is there no way to make the buildbots "smarter", so that if the Portfile has a `require_active_variants` line, to simply just go ahead and install the dependency with that variant enabled? I'm not necessarily asking for this sort of behavior in MacPorts base, which would affect all users, but only on the buildbots, so that they could complete a larger number of package builds. Or would this be considered "too smart by half"? -- Jason Liu On Wed, Nov 17, 2021 at 11:24 AM Christopher Nielsen < masc...@rochester.rr.com> wrote: > Makes sense. And yes, it’s definitely frustrating that Quartz-only > gtk2/gtk3 ports don’t build on our buildbots. > > Ultimately I’m going to fix this issue once and for all, via segregated > subports for X11 and Quartz. (That will need to be done for all of the key > foundational ports, including: glib2, gtk2, gtk3, gtk-osx-application-XXX, > etc.) > > In case anyone’s interested in the game plan: New non-conflicting subports > will be created, independent from our existing ones. That ensures we don’t > break any existing ports, and we’ll be able to gradually migrate dependents > to the new scheme. > > However, all of that is going to take a few months. Nonetheless, it’s > definitely on the radar! > > In the interim, I’d be curious whether there’s a short-term > buildbot-related fix for Quartz-only ports like RawTherapee. Josh/Ryan…? > > >> Jason, are you asking relative to the new port you just created, for > RawTherapee? > > > > Yes, although it's a generally applicable question. >
Re: Question regarding builders and require_active_variants
> > Jason, are you asking relative to the new port you just created, for > RawTherapee? Yes, although it's a generally applicable question. -- Jason Liu On Wed, Nov 17, 2021 at 9:43 AM Christopher Nielsen < masc...@rochester.rr.com> wrote: > Jason, are you asking relative to the new port you just created, for > RawTherapee? > > > As I was working on a port recently, a question came to mind which I’m > curious about. When using `require_active_variants` in a Portfile, are the > builders smart enough to then install dependencies using the required > variants? Because when I install the port locally, I have to manually go > and install the dependencies myself with the required variants. > > > > Jason Liu >
Question regarding builders and require_active_variants
Hi all, As I was working on a port recently, a question came to mind which I'm curious about. When using `require_active_variants` in a Portfile, are the builders smart enough to then install dependencies using the required variants? Because when I install the port locally, I have to manually go and install the dependencies myself with the required variants. -- Jason Liu
Re: Using Apple Clang to get OpenMP code (no clang-X.X required to compile)
Maybe it would be useful to add this workaround to the info provided in https://trac.macports.org/wiki/CompilerSelection#Parallelism ? -- Jason Liu On Fri, Nov 12, 2021 at 9:09 PM Eric Borisch wrote: > Just looking for anyone interested to provide feedback on this: > >https://github.com/macports/macports-ports/pull/12933 > > It uses '-Xpreprocessor -fopenmp' to get Apple's clang to compile OpenMP > code; it still relies on libomp for omp.h / -lomp, but doesn't require > clang-x.x to compile. > > Not an official route, but it works; thanks to M.P. for the tip. > > Thanks, > - Eric >
Apple Clang and port:libomp
Hi all, I feel like I should already know the answer to this question, but my brain is too tired at the moment to try to dig it out from my memory banks. If a MacPorts build is using Apple Clang, then does that mean that it will be unable to see port:libomp for OpenMP support? I've got a port that I'm working on that is giving the following messages: :info:configure -- Could NOT find OpenMP_C (missing: OpenMP_C_FLAGS OpenMP_C_LIB_NAMES) :info:configure -- Could NOT find OpenMP_CXX (missing: OpenMP_CXX_FLAGS OpenMP_CXX_LIB_NAMES) :info:configure -- Could NOT find OpenMP (missing: OpenMP_C_FOUND OpenMP_CXX_FOUND) (Note: I do already have depends_lib-append port:libomp in my Portfile.) Does this mean that I need to blacklist Apple Clang in order for the build to be able to see libomp? Or is there something like a PortGroup that I'm missing? -- Jason Liu
Re: upgrade to openssl 3.0.0
That was also the Blender devs' claim, which I assume is why they decided it wasn't necessary to include the GPL-OpenSSL exception text, since any licensing conflicts would self-resolve once Blender starts using OpenSSL 3.0. But currently, their pre-built release binary downloads and compiles OpenSSL 1.1.1g, which is still under the OpenSSL license, not Apache 2.0. -- Jason Liu On Sat, Oct 2, 2021 at 2:25 PM Joshua Root wrote: > Blender is GPL-2+, which means it can be distributed when linked with > OpenSSL 3.0, since GPL-3 is compatible with Apache-2. > > - Josh > > On 2021-10-3 05:09 , Jason Liu wrote: > > I hope the question that I'm about to ask doesn't induce > > "Inception"-style migraines, but since it directly relates to one of the > > ports I maintain, here goes. What if one of my ports indirectly uses > > openssl through one of its dependencies, and the licensing terms in the > > current version of my port only covers openssl 1.1.1, but not 3.0? > > > > To use a specific example, Blender uses openssl 1.1.1 indirectly, by way > > of using networking through python. I contacted the Blender devs about > > this, and even though they acknowledged that they were unknowingly using > > OpenSSL without including the OpenSSL license, the only thing they ended > > up doing was to add the OpenSSL license to their licenses directory. > > They refuse, even now, to add the chunk of text specifying the > > GPL-OpenSSL exception to their licenses. Somehow their legal team seems > > to think that's enough to allow them to distribute pre-compiled > > binaries, but the MacPorts automatic license checking is flagging my > > Blender port as being non-distributable. Since my port is a couple of > > versions behind the latest release of Blender (there are some new > > dependent libraries that I need to submit first), how should we specify > > in our Portfiles that one of the dependencies should continue using the > > old openssl11, without adding the old_openssl PortGroup, and thus a > > direct dependency on openssl? Does this mean that the dependencies which > > are directly dependent on openssl will need new variants, e.g. +openssl > > and +openssl11? > > > > -- > > Jason Liu >
Re: upgrade to openssl 3.0.0
I hope the question that I'm about to ask doesn't induce "Inception"-style
migraines, but since it directly relates to one of the ports I maintain,
here goes. What if one of my ports indirectly uses openssl through one
of its dependencies, and the licensing terms in the current version of my
port only covers openssl 1.1.1, but not 3.0?
To use a specific example, Blender uses openssl 1.1.1 indirectly, by way
of using networking through python. I contacted the Blender devs about
this, and even though they acknowledged that they were unknowingly using
OpenSSL without including the OpenSSL license, the only thing they ended up
doing was to add the OpenSSL license to their licenses directory. They
refuse, even now, to add the chunk of text specifying the GPL-OpenSSL
exception to their licenses. Somehow their legal team seems to think that's
enough to allow them to distribute pre-compiled binaries, but the MacPorts
automatic license checking is flagging my Blender port as being
non-distributable. Since my port is a couple of versions behind the latest
release of Blender (there are some new dependent libraries that I need to
submit first), how should we specify in our Portfiles that one of the
dependencies should continue using the old openssl11, without adding the
old_openssl PortGroup, and thus a direct dependency on openssl? Does this
mean that the dependencies which are directly dependent on openssl will
need new variants, e.g. +openssl and +openssl11?
--
Jason Liu
On Sat, Oct 2, 2021 at 1:17 PM Ken Cunningham <
ken.cunningham.web...@gmail.com> wrote:
> openssl 3.0.0 is out, with a new and very favourable Apache-2 license that
> will let many previously non-distributable ports become distributable.
> However, there are 758 ports that indicate they link against openssl. That
> is a daunting number of ports indeed.
>
> One option might be to move all of our openssl ports (1.0, 1.1, and 3.x)
> into subdirectories out of ${prefix}, have no default openssl installed in
> $[prefix} directly, create a new PortGroup, openssl_select or some such,
> add that PG to all 758 ports, default it to openssl 1.1.1, and then allow
> people to gradually move the 758 ports to openssl 3.0.0 as they get to it.
>
> Although that is possible with suitable effort, I don’t think that is
> workable for many reasons, the most obvious being that we would then have
> to modify every single port in some way to find an openssl installed into a
> non-default prefix. Creating the PG and adding it to 758 ports might be
> work enough, but then finding the right way to force all 758 ports to build
> properly against an openssl that is not in the default prefix is the real
> horror, and seems like a nightmare of wasted labours.
>
> So it would appear that the same option that was chosen last time, ie
> upgrade the default openssl in ${prefix} to the newer openssl, and then fix
> the subset of ports that will not build against it to use an older openssl
> appears both the better option and lot less work (assuming most ports do
> build against openssl 3.0.0, which seems to be the case so far). Some will
> disagree, but I put it to you that it is going to be far less work in the
> end to force a few % of the ports to a specific alternate openssl than
> force all of them, all the time, forever.
>
> Most things I have attempted to rebuild over the past few days have
> rebuilt without any issues, but a few things don’t build with openssl 3.0.0
> yet and will need to stay with openssl 1.1.1 for a while until patched or
> updated (or forever). That will require both forcing those ports to find an
> alternate openssl installation, and also (the tricky part) forcing them to
> ignore the openssl in the default prefix.
>
> To support this, there is a new openssl11 port that provides openssl 1.1.1
> tucked away in a subdir, much like we have openssl10, and a few new options
> were added to the old_openssl PortGroup to allow most ports to be forced to
> the alternate openssl with minimal fuss. Add the PortGroup, spec the
> branch, and choose the method, for the most part.
>
> If this plan holds, I would anticipate that we move ports that we find
> need to stay on openssl 1.1.1 to openssl11 using the old_openssl PortGroup
> soon or now, before we upgrade to openssl 3.0.0 to minimize fuss. Then once
> we have done that, upgrading the default openssl to 3.0.0 and revbumping
> the deps would occur.
>
> So I am asking that anyone with an interest in a port that uses openssl
> please try building it against openssl 3.0.0 in the PR below, and if there
> is a problem that can’t be solved by an update or a patch, consider trying
> to use the old_openssl PortGroup to fix the build and move it over. As
> there are so many ports, it would help if people pitched in with the ones
> that are
Re: changing 'configure' options for testing
In addition to the code Ken just provided, he and others have helped me
with some test-related "protection" code, so that in addition to a
+tests variant, there is the following section of code (I usually place it
after any build-related sections, and before any destroot-related sections,
in order to keep my portfiles laid out in the same order as the port
phases):
pre-test {
if {![variant_isset tests]} {
ui_error "'tests' variant must be activated to enable test support"
error "Please enable the 'tests' variant and try again"
}
}
--
Jason Liu
On Tue, Sep 28, 2021 at 11:13 AM Ken Cunningham <
ken.cunningham.web...@gmail.com> wrote:
>
> On 2021-09-28 7:44 a.m., Daniel J. Luke wrote:
> > On Sep 20, 2021, at 10:20 AM, Daniel J. Luke wrote:
> >> On Sep 20, 2021, at 8:15 AM, Frank Dean wrote:
> >>> Daniel J. Luke writes:
> >>>> The newest version of clamav uses cmake for builds. In the
> 'configure' stage, I have it disabling tests because otherwise it won't
> build without the test dependencies installed (check and pytest).
> >>>>
> >>>> Do we have a template or example of a canonical way to handle this? I
> don't see an obvious hook for when someone is running `port test` to change
> configure.args (I could, of course, add a post-extract/pre-configure and do
> some non-declaritive test to see if `port test` is being run and use that
> to branch - but that feels like a bad design choice).
> >>> The rapidjson port implements a 'tests' variant to handle a similar
> >>> situation. I used the same pattern for the libosmium port. The tests
> >>> can then be run with `sudo port -d test current +tests`.
> >> That works, I guess.
> >>
> >> Is there interest in having base auto-add +tests if `port test` is
> called? (I haven't looked at base/ code in a while, but it seems possible).
> I like to imagine a future where we have enough infrastructure that we
> would run `port test` for any ports that have test.run set.
> > On this same train of thought - clamav tests run against the installed
> libraries (like some other ports tend to do) - in long-ago times I'd solve
> this by setting DYLD_LIBRARY_PATH - but since SIP started removing these
> that no longer works normally. I think trace mode has a(n elaborate)
> workaround where it copies binaries and executes the copies, but that
> doesn't seem like something I should implement in a portfile ;-) [This is
> another instance where if trace mode were the default, things would 'just
> work']
> >
> > Has anyone else already solved this?
>
>
> The way our cmake PortGroup sets things up, running tests is harder. You
> can configure cmake to allow testing to find the libraries in the build
> tree. See for example what I did in libcbor:
>
>
> variant tests description {enable tests} {
> depends_test-append port:cmocka
> configure.args-append -DWITH_TESTS=ON
> configure.pre_args-replace -DCMAKE_BUILD_WITH_INSTALL_RPATH:BOOL=ON \
> -DCMAKE_BUILD_WITH_INSTALL_RPATH:BOOL=OFF
> test.runyes
> test.target test
> }
>
> Ken
>
>
>
>
>
>
> >
>
Re: Updating omniORBpy with python variants
Did you try manually running the `portindex` command to have MacPorts re-scan your portfiles? -- Jason Liu On Tue, Sep 21, 2021 at 1:33 PM Thomas Lockhart wrote: > I’m having trouble updating the omniORBpy port file to support the latest > version of omniORBpy and latest versions of python. It’s been a good while > since I’ve done this, but (with the latest macports installed) it only > wants to recognize the orginal 27 and 36 ports as available even though > I’ve now defined others (or think I have at least). > > Any hints on what I’m missing? Are the available ports cached somewhere > else in the macports tree??? > > TIA > > - Tom > >
Re: iTerm2 Question
If I'm not mistaken, to get a .app to show up in the /Applications/MacPorts
folder, you need to place the .app into ${destroot}$prefix/Applications at
some point during the destroot phase.
--
Jason Liu
On Sat, May 22, 2021 at 5:07 PM Mark Anderson wrote:
> So I can get iTerm2 to build correctly using Xcode 1.0 portgroup which
> solves a lot of the makefile problems, but I'm unsure what to do for the
> destroot phase to drop the .app into /Application/Macports. Any help would
> be appreciated.
>
> —Mark
> ___
> Mark E. Anderson
> MacPorts Trac WikiPage <https://trac.macports.org/wiki/mark>
> GitHub Profile <https://github.com/markemer>
>
>
Re: Publicizing MacPorts [installation]
On Sat, May 22, 2021 at 1:35 PM Ken Cunningham < ken.cunningham.web...@gmail.com> wrote: > > Yes, thanks for the tips! I am pretty sure that it is possible to automate >> it one way or another. But my point is that it would be helpful to have a >> one-liner to install MacPorts and maintain it as a part of the main >> repository. > > > This was of course suggested years ago as well, when homebrew first did > it, but at that time was that it was both not needed and not a useful > addition to MacPorts, if I recall the full email exchange correctly, so we > let the idea die. > I feel like having a one-liner to install MacPorts, similar to Homebrew, would be incredibly useful, especially for people who are not tech savvy. It seems that many of us on the mailing list, including myself, already have our own home-grown scripts to automate installing MacPorts. One thing that I particularly like about the Homebrew installer is that it automatically installs the CLT... I've been doing something similar in my own MacPorts install script for around a decade. My script even automatically accepts the Xcode license by using a small chunk of expect. I realize that from the perspective of the MacPorts developers, we might not want to be taking over control of this step from the user. But from personal experience as a sysadmin, even this seemingly minor step can be a fairly high hurdle for people who are not tech savvy. Another thing that my script tries to automate is to add /opt/local to everyone's $PATH if the script detects that SIP is disabled. On Sat, May 22, 2021 at 2:31 PM Joshua Root wrote: > > I think the best thing we could do to facilitate one-liner command > line installation is set up a redirect so you can download the latest > binary installer for your OS version without having to construct > its not-so-easy-to-derive name yourself. It would then be simple to > download the .pkg and feed it to installer(8). > This would be incredibly useful, and would allow me to cut out around 40-50 lines of code from my MacPorts install script (which is currently constructing the not-so-easy-to-derive name myself). A single permalink redirect would also allow the installation instructions to be simplified on the MacPorts website, instead of what's currently there: 3. Install MacPorts for your version of the operating system: * macOS Big Sur v11 <https://github.com/macports/macports-base/releases/download/v2.7.0/MacPorts-2.7.0-11-BigSur.pkg> * macOS Cataline v10.15 <https://github.com/macports/macports-base/releases/download/v2.7.0/MacPorts-2.7.0-10.15-Catalina.pkg> * macOS Mojave v10.14 <https://github.com/macports/macports-base/releases/download/v2.7.0/MacPorts-2.7.0-10.14-Mojave.pkg> * Older OS? See here. <https://www.macports.org/install.php#installing> -- Jason Liu On Sat, May 22, 2021 at 1:35 PM Ken Cunningham < ken.cunningham.web...@gmail.com> wrote: > > Yes, thanks for the tips! I am pretty sure that it is possible to > automate it one way or another. But my point is that it would be helpful to > have a one-liner to install MacPorts and maintain it as a part of the main > repository. > > This was of course suggested years ago as well, when homebrew first did > it, but at that time was that it was both not needed and not a useful > addition to MacPorts, if I recall the full email exchange correctly, so we > let the idea die. > > I wrote up a MacPorts install script for Jeremy’s Xquartz project here < > https://github.com/XQuartz/XQuartz/blob/master/install-or-update-macports.sh> > that adds a bit more trickery he needed, but the basic guts was extremely > simple and what I recommend to people who complain that MacPorts is > extremely difficult to get installed and they claim to have spent hours and > hours and hours trying to make it work: > > == > > cd /tmp > git clone -b release-2.7 https://github.com/macports/macports-base.git > cd macports-base > ./configure && make && sudo make install > > > and then add to the $PATH as usual > > ==
Re: Buildbot Performance
On Mon, May 17, 2021 at 7:54 PM Christopher Nielsen < masc...@rochester.rr.com> wrote: > > Pinning our buildbot VMs to specific NUMA nodes can result in starvation, > when multiple VMs assigned to a given node are all busy. That would also > result in underutilization of the other node, if VMs assigned to that are > idle. > > Does that make sense? What you say does make sense, but I believe that what I am suggesting is different than what I think you are describing. I'm not suggesting that the guests get pinned to different NUMA nodes. Instead, I'm suggesting that all guests get pinned to the same 6 nodes. AFAICT, the starvation situation that you're describing can occur when some of the guests get pinned to a different set of NUMA nodes. If all of the guests are pinned to the same nodes, then theoretically, those guests should behave as if they all have equal access to the same pool of physical cores. There shouldn't be any underutilization of any particular node, because the node has been pinned for all of the guests; this means that the hypervisor should be in control of scheduling the utilization of the node among all of the guests which have that node pinned. -- Jason Liu On Mon, May 17, 2021 at 7:54 PM Christopher Nielsen < masc...@rochester.rr.com> wrote: > Well, my recommendation for our setup, is to avoid pinning. > > Why? For several reasons: > * Out-of-the-box, ESX/ESXi makes a best-effort attempt to schedule all > vCPUs for a given VM, on a single NUMA node. > * Even when that’s not possible, the hypervisor schedules VM vCPUs to > hyperthread pairs. > > Pinning our buildbot VMs to specific NUMA nodes can result in starvation, > when multiple VMs assigned to a given node are all busy. That would also > result in underutilization of the other node, if VMs assigned to that are > idle. > > Does that make sense? > > > On 2021-05-17-M, at 19:34, Jason Liu wrote: > > > > If the guests on a virtual server are exerting a heavy enough load that > the virtual host is not able to obtain the resources it needs, then the > entire system's performance, both physical and virtual, can be affected. > I'm not claiming to be familiar enough with the specifics of the situation > to claim that this is what's happening, but if it is, then using CPU > pinning can help. It's basically analogous to the situation where you > overcommit the memory too much, and the virtual host isn't able to have > enough memory to do the tasks it needs to. > > > > On the virtual servers which I set up for my customers, I have the > hypervisor set up to automatically pin all but 2 cores. So for example, on > an 8 core machine, I pin cores 0-5 for all of the VM settings, so that none > of the guests are able to use cores 6 and 7. This effectively removes 2 > cores from the pool of CPUs available for the guest to use, which means > that the virtual host will always have those 2 cores available to it at all > times. This allows my customer to run, on average, 10 guests on the virtual > server simultaneously, and everything stays performant, even under heavy > vCPU loads. > > > > The other thing I do is that I tell my customers to never overcommit on > memory. My rule of thumb is that the sum of all simultaneously running > guests should never exceed the amount of physical RAM minus 4 GB. So, on a > server with 128 GB of RAM, the sum of the memory allocated to the guests > running simultaneously should never add up to more than 124 GB of memory. > > > >> On Mon, May 17, 2021 at 6:24 PM Ryan Schmidt > wrote: > >> > >>> On May 17, 2021, at 13:13, Jason Liu wrote: > >>> > >>> Regarding CPU overcommitment: Are the virtual hosts doing any sort of > CPU pinning? Many virtualization products have the ability to specify which > of the pCPU cores a guest is allowed to use. As far as I can remember, > products like KVM and ESXi can do CPU pinning, while VirtualBox cannot. > >> > >> Nope, nothing like that is set up. Is there any reason why we would > want that? >
Re: Buildbot Performance
> > We don’t want any type of pinning, as that will further exacerbate the > situation. > Why would that be? Do the virtual servers have a low number of physical cores or something? -- Jason Liu On Mon, May 17, 2021 at 6:26 PM Christopher Nielsen < masc...@rochester.rr.com> wrote: > We don’t want any type of pinning, as that will further exacerbate the > situation. > > > On 2021-05-17-M, at 18:24, Ryan Schmidt wrote: > > > >> On May 17, 2021, at 13:13, Jason Liu wrote: > >> > >> Regarding CPU overcommitment: Are the virtual hosts doing any sort of > CPU pinning? Many virtualization products have the ability to specify which > of the pCPU cores a guest is allowed to use. As far as I can remember, > products like KVM and ESXi can do CPU pinning, while VirtualBox cannot. > > > > Nope, nothing like that is set up. Is there any reason why we would want > that? >
Re: Buildbot Performance
If the guests on a virtual server are exerting a heavy enough load that the virtual host is not able to obtain the resources it needs, then the entire system's performance, both physical and virtual, can be affected. I'm not claiming to be familiar enough with the specifics of the situation to claim that this is what's happening, but if it is, then using CPU pinning can help. It's basically analogous to the situation where you overcommit the memory too much, and the virtual host isn't able to have enough memory to do the tasks it needs to. On the virtual servers which I set up for my customers, I have the hypervisor set up to automatically pin all but 2 cores. So for example, on an 8 core machine, I pin cores 0-5 for all of the VM settings, so that none of the guests are able to use cores 6 and 7. This effectively removes 2 cores from the pool of CPUs available for the guest to use, which means that the virtual host will always have those 2 cores available to it at all times. This allows my customer to run, on average, 10 guests on the virtual server simultaneously, and everything stays performant, even under heavy vCPU loads. The other thing I do is that I tell my customers to never overcommit on memory. My rule of thumb is that the sum of all simultaneously running guests should never exceed the amount of physical RAM minus 4 GB. So, on a server with 128 GB of RAM, the sum of the memory allocated to the guests running simultaneously should never add up to more than 124 GB of memory. -- Jason Liu On Mon, May 17, 2021 at 6:24 PM Ryan Schmidt wrote: > On May 17, 2021, at 13:13, Jason Liu wrote: > > > Regarding CPU overcommitment: Are the virtual hosts doing any sort of > CPU pinning? Many virtualization products have the ability to specify which > of the pCPU cores a guest is allowed to use. As far as I can remember, > products like KVM and ESXi can do CPU pinning, while VirtualBox cannot. > > Nope, nothing like that is set up. Is there any reason why we would want > that? > >
Re: Buildbot Performance
On Sun, May 16, 2021 at 3:38 PM Ryan Schmidt wrote: > > > On May 16, 2021, at 09:48, Christopher Nielsen wrote: > >> Upgrading them to six-core Xeons would absolutely help, for sure. But I’m >> quite certain that we could also improve the situation, by reducing the >> level of CPU overcommitment. And reducing the vCPUs per VM would help, as >> we simply don’t have the physical CPU power to support eight/VM. >> > > When you say reduce CPU overcommitment, by what means are you referring > to, other than reducing CPUs per VM? Regarding CPU overcommitment: Are the virtual hosts doing any sort of CPU pinning? Many virtualization products have the ability to specify which of the pCPU cores a guest is allowed to use. As far as I can remember, products like KVM and ESXi can do CPU pinning, while VirtualBox cannot. -- Jason Liu On Sun, May 16, 2021 at 3:38 PM Ryan Schmidt wrote: > > > On May 16, 2021, at 09:48, Christopher Nielsen wrote: > > > In terms of the ratio of vCPUs to GB of RAM, 1:1 isn’t totally > unreasonable. However, we should also reserve 2 GB of RAM for the OS, > including the disk cache. So perhaps 6 vCPUs would be a better choice. > > MacPorts base hasn't ever considered reserving any RAM for the OS. I > cannot confirm that any changes are needed to the formula of how many jobs > we start based on how much RAM is available, but if there are, then they > should be agreed upon and made in base first; then the buildbot setup can > be adjusted accordingly. > > > > As for the total physical CPUs available on our Xserves, here’s the rub: > While hyperthreading does provide some benefit, best-case it generally only > provides 50% more headroom. And sometimes it’s as low as 25%. > > I'm aware. > > > > So if we assume best-case, our Xserve’s only provide the processing > power of 12 CPU cores, when accounting for hyperthreading. So even if only > two builders are active, we’re already well overcommitted on CPU. And with > three or more going, I’d bet the hypervisor is spending more time on > scheduling and pre-emption, than actual processing time. > > I cannot confirm or deny your claims about the hypervisor. > > > > By way of comparison, I’m running on a modest 2008-era MacPro, with only > eight physical CPU cores… and no hyper threading. Plus the Xeons in my > MacPro are one major generation behind the Nehalem-based CPUs on our > Xserves. Yet, my port build times are anywhere from 2x to 10x faster than > we’re seeing on our builders. (And no, that’s not an exaggeration.) > > Are you looking at the time to build just the port, or are you including > the time to install all the dependencies? Because on your system, you > already have the dependencies installed, or if not, they just need to be > downloaded and installed once. On the buildbot, on the other hand, > dependencies are deactivated between builds, and are sometimes activated > and deactivated many times before the main port is built. This can result > in a significant and in my opinion an unnecessarily large amount of time > being taken for dealing with dependencies first. I believe this can be > improved upon. For example, instead of deactivating *all* active ports > between each build and between each dependency, we could develop a way to > deactive only the active ports that are not needed by the next build or > dependency. This would have the added benefit of reducing disk wear, which > has already been a problem before. See > https://trac.macports.org/ticket/62621 > > It is definitely expected that any individual VM will take longer to build > if other VMs on the same host are also busy. Reducing the number of CPUs > each VM has will just ensure that the builds take longer, even if other VMs > on the same host are not busy. > > > > So we need to do something, as the buildbots simply can’t keep up. > > I think they've been keeping up fine, except when a bunch of huge things > need to be built, which I find understandable. > > > > Upgrading them to six-core Xeons would absolutely help, for sure. But > I’m quite certain that we could also improve the situation, by reducing the > level of CPU overcommitment. And reducing the vCPUs per VM would help, as > we simply don’t have the physical CPU power to support eight/VM. > > When you say reduce CPU overcommitment, by what means are you referring > to, other than reducing CPUs per VM? > >
Re: Ports updated without maintainer notification?
In this particular situation, the libraries in question are ones that I specifically added to MacPorts in order to allow my Blender port to build. No other ports use those libraries, although obviously there's nothing to prevent other software from starting to use them in the future. -- Jason Liu On Tue, May 11, 2021 at 7:08 PM Nils Breunese wrote: > Ryan Schmidt wrote: > > > Holding back a library to be compatible with one port may not be the > right choice for other ports that depends on the library. > > If other ports would want to use the latest version of foo-lib, but > Blender needs specifically version 1.15 of foo-lib, it might be an idea to > have both a foo-lib port with the latest version and a foo-lib-1.15 port > that Blender can then depend on. > > Nils.
Re: Ports updated without maintainer notification?
> Adding openmaintainer makes things easier with non-committer maintainers > in particular, which is why it's recommended to new maintainers, but it's > not required. If there's a good chance that someone not familiar with the > nuances of a port will inadvertently break something, then openmaintainer > is not the right choice. > As I said in the last message that I just sent, the risk of updating the particular ports I mentioned are that they are dependencies for the blender port, and might cause blender to fail. Every time I have updated those libraries in the past, I have always made sure that the updated version compiles against the blender port. Not only that, but I did run into one instance where blender was compiling successfully against a newer version of one of the libraries, but the Blender app was crashing during runtime whenever I tried to render a project. But you need to respond to tickets and PRs within 3 days, or changes can be > merged anyway under the maintainer timeout rule. > Not usually a problem. Hopefully it's fairly evident that I've been pretty active since I started submitting ports around a year ago. Also note that openmaintainer is not carte blanche for others to make > whatever changes they want to your ports. > That's why I was so confused when I suddenly realized that some of the ports I maintain were updated to new versions, and I couldn't find any PRs or Trac tickets referring to those changes. I had a moment of real panic, because at that moment, I had no way of guaranteeing that my blender port in the public ports tree wasn't suddenly broken. -- Jason Liu On Fri, May 7, 2021 at 12:43 PM Joshua Root wrote: > On 2021-5-8 02:02 , Jason Liu wrote: > > > > If your ports are marked openmaintainer, that gives permission to > > others to make minor modifications to your ports without notifying > > you. Not all changes happen via PRs; some are committed directly to > > master. > > > > > > Does this mean that it's okay to have ports with only myself as > > maintainer? When I started submitting my first ports around a year ago, > > I was told that I should always add openmaintainer in addition to myself. > > Adding openmaintainer makes things easier with non-committer maintainers > in particular, which is why it's recommended to new maintainers, but > it's not required. If there's a good chance that someone not familiar > with the nuances of a port will inadvertently break something, then > openmaintainer is not the right choice. But you need to respond to > tickets and PRs within 3 days, or changes can be merged anyway under the > maintainer timeout rule. > > Also note that openmaintainer is not carte blanche for others to make > whatever changes they want to your ports. Committers are expected to > apply good judgement when making changes to ports maintained by others, > and to take responsibility for fixing any problem introduced in doing > so. If a change is at all risky or there is any doubt as to the correct > approach, running it by the maintainer first is the right thing to do. > > <https://guide.macports.org/#project.update-policies> > > - Josh >
Re: Ports updated without maintainer notification?
> Did you also update all the other outdated ports on your ‘local’ machine, > or did you just cherry-pick the updated osl from the current master ? > > If so it is really not a good idea to do that, as it means, as appears > above, you could get an updated binary tarball install that was built > against another updated port you do not have. > I have a cron job (actually a launchd job) that runs port selfupdate and port upgrade outdated every night on my machine. I did cherry-pick the updated osl portfile from the current master in git, but only because I noticed that the portfile had changed in the git tree but the updated ports weren't showing up as being outdated. You should *always* keep all your ports updated and consistent. > > if you run > > > sudo port -d sync > > sudo port update outdated > > does that help > > Running port -d sync results in the updated ports now showing up in the list of outdated ports. I was under the impression that port selfupdate ran port sync as a part of the self-update process? Or am I mistaken, and port sync needs to be added to my nightly script? Just finally to note, there is nothing wrong with the current osl builds, > they are available (apart from arm) down to 10.9 > It's not the individual libraries that I'm worried about. It's the fact that these libraries are dependencies for my Blender port. Blender typically uses slightly outdated versions of all of its dependent libraries. Very complex libraries are a particular risk. For example, trying to use the newest version of usd is causing blender builds to fail on my machine. I remember a time when ports with lots of dependencies, such as gimp, could be broken for weeks at a time, because updates to one or more of its dependencies was causing gimp builds to fail. Admittedly, this was an experience from a decade ago, but it did leave a lasting impression. Is there some way that I can signal not to update certain libraries without verifying against blender? Should I leave some sort of warning comment in the portfiles? -- Jason Liu On Fri, May 7, 2021 at 12:15 PM Christopher Jones wrote: > > OSL in particular appears to be a problem on my machine. I've copied the > newer version of the portfile directly to my local machine, and tried to > build it, but it's failing to build because osl is indirectly dependent on > opencolorio (by way of openimageio), and apparently there's a new problem > with either opencolorio or openimageio: > > :info:build dyld: Symbol not found: > __ZN4YAML6detail9node_data12empty_scalarE > :info:build Referenced from: /opt/local/lib/libOpenColorIO.1.dylib > :info:build Expected in: /opt/local/lib/libyaml-cpp.0.6.dylib > :info:build in /opt/local/lib/libOpenColorIO.1.dylib > :info:build /bin/sh: line 1: 34490 Trace/BPT trap: 5 > :info: build > opt/local/var/macports/build/_Users_jasonliu_ports_graphics_osl/osl/work/build/bin/oslc > -q > -I/opt/local/var/macports/build/_Users_jasonliu_ports_graphics_osl/osl/work/OpenShadingLanguage-1.11.13.0/src/shaders > -I/opt/local/var/macports/build/_Users_jasonliu_ports_graphics_osl/osl/work/OpenShadingLanguage-1.11.13.0/src/shaders > -I/opt/local/var/macports/build/_Users_jasonliu_ports_graphics_osl/osl/work/OpenShadingLanguage-1.11.13.0/src/shaders > /opt/local/var/macports/build/_Users_jasonliu_ports_graphics_osl/osl/work/OpenShadingLanguage-1.11.13.0/src/shaders/emitter.osl > -o > /opt/local/var/macports/build/_Users_jasonliu_ports_graphics_osl/osl/work/build/src/shaders/emitter.oso > > > > Did you also update all the other outdated ports on your ‘local’ machine, > or did you just cherry-pick the updated osl from the current master ? > > If so it is really not a good idea to do that, as it means, as appears > above, you could get an updated binary tarball install that was built > against another updated port you do not have. > > You should *always* keep all your ports updated and consistent. > > if you run > > > sudo port -d sync > > sudo port update outdated > > does that help > > > Just finally to note, there is nothing wrong with the current osl builds, > they are available (apart from arm) down to 10.9 > > https://ports.macports.org/port/osl/summary > > Chris > > > -- > Jason Liu > > > On Fri, May 7, 2021 at 7:32 AM Ryan Schmidt > wrote: > >> >> >> On May 7, 2021, at 01:59, Jason Liu wrote: >> >> > I've run across a situation that has left me confused. I started >> updating some of the portfiles for which I'm the maintainer, and then I >> noticed that the portfiles seem to have already been updated in git. >> However, I can't find any PRs for such an update, and I was never notified >> that the ports for which I'm the maintainer was getting upd
Re: Ports updated without maintainer notification?
> If your ports are marked openmaintainer, that gives permission to others > to make minor modifications to your ports without notifying you. Not all > changes happen via PRs; some are committed directly to master. > Does this mean that it's okay to have ports with only myself as maintainer? When I started submitting my first ports around a year ago, I was told that I should always add openmaintainer in addition to myself. If you let us know specifically which ports, we could take a look. > The ports are osl, oidn, and embree. > In addition, I have run a "port selfupdate" on my machine, and yet the >> MacPorts on my machine isn't seeing the new version of the port. Is >> something broken, either on my machine, or on GitHub? >> > > If your MacPorts is configured to get ports via rsync, it can take an hour > for changes to propagate from git to the main rsync server, and up to a day > longer for changes to propagate from there to other rsync mirrors. > OSL in particular appears to be a problem on my machine. I've copied the newer version of the portfile directly to my local machine, and tried to build it, but it's failing to build because osl is indirectly dependent on opencolorio (by way of openimageio), and apparently there's a new problem with either opencolorio or openimageio: :info:build dyld: Symbol not found: __ZN4YAML6detail9node_data12empty_scalarE :info:build Referenced from: /opt/local/lib/libOpenColorIO.1.dylib :info:build Expected in: /opt/local/lib/libyaml-cpp.0.6.dylib :info:build in /opt/local/lib/libOpenColorIO.1.dylib :info:build /bin/sh: line 1: 34490 Trace/BPT trap: 5 :info: build opt/local/var/macports/build/_Users_jasonliu_ports_graphics_osl/osl/work/build/bin/oslc -q -I/opt/local/var/macports/build/_Users_jasonliu_ports_graphics_osl/osl/work/OpenShadingLanguage-1.11.13.0/src/shaders -I/opt/local/var/macports/build/_Users_jasonliu_ports_graphics_osl/osl/work/OpenShadingLanguage-1.11.13.0/src/shaders -I/opt/local/var/macports/build/_Users_jasonliu_ports_graphics_osl/osl/work/OpenShadingLanguage-1.11.13.0/src/shaders /opt/local/var/macports/build/_Users_jasonliu_ports_graphics_osl/osl/work/OpenShadingLanguage-1.11.13.0/src/shaders/emitter.osl -o /opt/local/var/macports/build/_Users_jasonliu_ports_graphics_osl/osl/work/build/src/shaders/emitter.oso -- Jason Liu On Fri, May 7, 2021 at 7:32 AM Ryan Schmidt wrote: > > > On May 7, 2021, at 01:59, Jason Liu wrote: > > > I've run across a situation that has left me confused. I started > updating some of the portfiles for which I'm the maintainer, and then I > noticed that the portfiles seem to have already been updated in git. > However, I can't find any PRs for such an update, and I was never notified > that the ports for which I'm the maintainer was getting updated... usually, > if someone submits a PR for a portfile for which I'm the maintainer, I get > a notification through GitHub. > > If your ports are marked openmaintainer, that gives permission to others > to make minor modifications to your ports without notifying you. Not all > changes happen via PRs; some are committed directly to master. > > If there is an urgent issue that needs to be fixed in a port, someone else > might make that fix, even if the port is not marked openmaintainer. > > If you let us know specifically which ports, we could take a look. > > > > In addition, I have run a "port selfupdate" on my machine, and yet the > MacPorts on my machine isn't seeing the new version of the port. Is > something broken, either on my machine, or on GitHub? > > If your MacPorts is configured to get ports via rsync, it can take an hour > for changes to propagate from git to the main rsync server, and up to a day > longer for changes to propagate from there to other rsync mirrors.
Ports updated without maintainer notification?
Hi all, I've run across a situation that has left me confused. I started updating some of the portfiles for which I'm the maintainer, and then I noticed that the portfiles seem to have already been updated in git. However, I can't find any PRs for such an update, and I was never notified that the ports for which I'm the maintainer was getting updated... usually, if someone submits a PR for a portfile for which I'm the maintainer, I get a notification through GitHub. In addition, I have run a "port selfupdate" on my machine, and yet the MacPorts on my machine isn't seeing the new version of the port. Is something broken, either on my machine, or on GitHub? -- Jason Liu
Port conflicts message: provide more info?
Hi all, When trying to install a conflicting (sub)port, here's the current error message: ---> Computing dependencies for Error: Can't install because conflicting ports are active: Error: Follow https://guide.macports.org/#project.tickets to report a bug. Error: Processing of port failed Perhaps it would be useful to add some additional information to this error message? Something like: Error: Can't install because conflicting ports are active: Error: Either uninstall before trying to install , Error: or deactivate before trying to install by running Error: sudo port deactivate Error: If neither of these suggestions work, then Error: please follow https://guide.macports.org/#project.tickets to report a bug. Error: Processing of port failed What are everyones' thoughts? (I haven't personally run across a situation where a port conflicts with more than one active port, so I'm not sure whether the wording of my above suggestion is optimal.) -- Jason Liu
Re: M1 CPU features
In addition to SIMDe, projects like Blender are using sse2neon to allow them to compile Blender on arm64 while using libraries that use Intel intrinsics and haven't been ported for use with Arm Neon. https://github.com/DLTcollab/sse2neon I plan to package sse2neon after I update a couple of the other ports I'm maintaining. Maybe I'll try packaging SIMDe at the same time. However, since I don't own an M1 Mac, I'm hoping that people on this mailing list would be able to help test my portfile before I submit them. Any volunteers? :) -- Jason Liu On Mon, Apr 26, 2021 at 3:16 PM Georges Martin wrote: > And this article describes SIMDe: > > > https://simd-everywhere.github.io/blog/2020/06/22/transitioning-to-arm-with-simde.html > > SIMD Everywhere (SIMDe) provides fast, portable, permissively-licensed > (MIT) implementations of the x86 APIs which allow you to run code designed > for x86/x86_64 CPUs pretty much anywhere, including on Arm (using NEON if > available). With almost no source code changes, you can recompile your x86 > SIMD code for Arm (or POWER, or WebAssembly, etc.). > > > ...that is not yet packaged for MacPorts ;-) ;-) ;-) > > G. > > Le 26 avr. 2021 à 20:56, Georges Martin a écrit : > > Aha, hw.optional! That's useful, thanks Georges! > > > You're welcome :-) You also have: > > hw.optional.amx_version: 2 > hw.optional.arm64: 1 > hw.targettype: J313 > > "amx" is the Neural Engine and I think "J313" is the code name for the M1. > > You may find this article very interesting: > > https://levelup.gitconnected.com/armv9-what-is-the-big-deal-4528f20f78f3 > > It describes the new ARMv9 instruction set with SVE2 and how it compares > to Intel/AMD MMX/SSE/AVX and NEON/SVE. > > Question is: would Apple adopt ARMv9 with SVE2 in a M2 for a future Mac > Pro ? ;-) > > G. > > Le 26 avr. 2021 à 20:44, Jason Liu a écrit : > > Aha, hw.optional! That's useful, thanks Georges! > > -- > Jason Liu > > > On Mon, Apr 26, 2021 at 2:16 PM Georges Martin wrote: > >> $ sysctl hw.optional | grep -E 'neon|armv8' >> hw.optional.neon: 1 >> hw.optional.neon_hpfp: 1 >> hw.optional.neon_fp16: 1 >> hw.optional.armv8_1_atomics: 1 >> hw.optional.armv8_crc32: 1 >> hw.optional.armv8_2_fhm: 1 >> hw.optional.armv8_2_sha512: 1 >> hw.optional.armv8_2_sha3: 1 >> >> Le 26 avr. 2021 à 19:55, Jason Liu a écrit : >> >> On Mon, Apr 26, 2021 at 1:42 PM Christopher Jones < >> jon...@hep.phy.cam.ac.uk> wrote: >> >>> Thats not at all surprising as those instruction sets are very much >>> specific to X86_64 systems. >>> >>> RISC processors, Arm, do have their own sets of SIMD instructions (e.g. >>> Neon), but they are entirely different to those on X86_64 machines. >>> >>> Whether or these are supported on Apple’s M1 processors I have no idea. >>> >> >> It looks like the M1 supports Neon SIMD instructions, but not SVE SIMD >> (which I guess is supposed to be similar to AVX?): >> >> https://discussions.apple.com/thread/252073619 >> >> -- >> Jason Liu >> >> >> On Mon, Apr 26, 2021 at 1:42 PM Christopher Jones < >> jon...@hep.phy.cam.ac.uk> wrote: >> >>> >>> >>> On 26 Apr 2021, at 6:28 pm, Jason Liu wrote: >>> >>> Thanks Arno :) >>> >>> I'm kind of surprised that the M1 doesn't seem to support any SSE or >>> AVX >>> >>> >>> >>> Thats not at all surprising as those instruction sets are very much >>> specific to X86_64 systems. >>> >>> RISC processors, Arm, do have their own sets of SIMD instructions (e.g. >>> Neon), but they are entirely different to those on X86_64 machines. >>> >>> Whether or these are supported on Apple’s M1 processors I have no idea. >>> >>> Chris >>> >>> >>> >>> >>> Does "sysctl machdep.cpu.features" return anything? >>> >>> -- >>> Jason Liu >>> >>> >>> On Mon, Apr 26, 2021 at 1:23 PM Arno Hautala wrote: >>> >>>> > On 26 Apr 2021, at 13:20, Jason Liu wrote: >>>> > >>>> > sysctl machdep.cpu.brand_string ; sysctl machdep.cpu | grep -i >>>> "avx\|sse” >>>> >>>> $ sysctl machdep.cpu.brand_string ; sysctl machdep.cpu | grep -i >>>> "avx\|sse” >>>> machdep.cpu.brand_string: Apple M1 >>>> >>>> -- >>>> arno s hautala/-| a...@alum.wpi.edu >>>> >>>> pgp b2c9d448 >>>> >>>> >>>> >>> >> > >
Re: M1 CPU features
On Mon, Apr 26, 2021 at 2:56 PM Georges Martin wrote: > Question is: would Apple adopt ARMv9 with SVE2 in a M2 for a future Mac > Pro ? ;-) > It might not happen for the M2, but I think it's virtually guaranteed that Apple will adopt ARMv9 at some point. With the success of the M1, it looks like Apple will be sticking with ARM for a while, and as we all know (maybe with the exception of their laptop cameras), Apple loves to use the newest technology. -- Jason Liu On Mon, Apr 26, 2021 at 2:56 PM Georges Martin wrote: > Aha, hw.optional! That's useful, thanks Georges! > > > You're welcome :-) You also have: > > hw.optional.amx_version: 2 > hw.optional.arm64: 1 > hw.targettype: J313 > > "amx" is the Neural Engine and I think "J313" is the code name for the M1. > > You may find this article very interesting: > > https://levelup.gitconnected.com/armv9-what-is-the-big-deal-4528f20f78f3 > > It describes the new ARMv9 instruction set with SVE2 and how it compares > to Intel/AMD MMX/SSE/AVX and NEON/SVE. > > Question is: would Apple adopt ARMv9 with SVE2 in a M2 for a future Mac > Pro ? ;-) > > G. > > Le 26 avr. 2021 à 20:44, Jason Liu a écrit : > > Aha, hw.optional! That's useful, thanks Georges! > > -- > Jason Liu > > > On Mon, Apr 26, 2021 at 2:16 PM Georges Martin wrote: > >> $ sysctl hw.optional | grep -E 'neon|armv8' >> hw.optional.neon: 1 >> hw.optional.neon_hpfp: 1 >> hw.optional.neon_fp16: 1 >> hw.optional.armv8_1_atomics: 1 >> hw.optional.armv8_crc32: 1 >> hw.optional.armv8_2_fhm: 1 >> hw.optional.armv8_2_sha512: 1 >> hw.optional.armv8_2_sha3: 1 >> >> Le 26 avr. 2021 à 19:55, Jason Liu a écrit : >> >> On Mon, Apr 26, 2021 at 1:42 PM Christopher Jones < >> jon...@hep.phy.cam.ac.uk> wrote: >> >>> Thats not at all surprising as those instruction sets are very much >>> specific to X86_64 systems. >>> >>> RISC processors, Arm, do have their own sets of SIMD instructions (e.g. >>> Neon), but they are entirely different to those on X86_64 machines. >>> >>> Whether or these are supported on Apple’s M1 processors I have no idea. >>> >> >> It looks like the M1 supports Neon SIMD instructions, but not SVE SIMD >> (which I guess is supposed to be similar to AVX?): >> >> https://discussions.apple.com/thread/252073619 >> >> -- >> Jason Liu >> >> >> On Mon, Apr 26, 2021 at 1:42 PM Christopher Jones < >> jon...@hep.phy.cam.ac.uk> wrote: >> >>> >>> >>> On 26 Apr 2021, at 6:28 pm, Jason Liu wrote: >>> >>> Thanks Arno :) >>> >>> I'm kind of surprised that the M1 doesn't seem to support any SSE or >>> AVX.... >>> >>> >>> >>> Thats not at all surprising as those instruction sets are very much >>> specific to X86_64 systems. >>> >>> RISC processors, Arm, do have their own sets of SIMD instructions (e.g. >>> Neon), but they are entirely different to those on X86_64 machines. >>> >>> Whether or these are supported on Apple’s M1 processors I have no idea. >>> >>> Chris >>> >>> >>> >>> >>> Does "sysctl machdep.cpu.features" return anything? >>> >>> -- >>> Jason Liu >>> >>> >>> On Mon, Apr 26, 2021 at 1:23 PM Arno Hautala wrote: >>> >>>> > On 26 Apr 2021, at 13:20, Jason Liu wrote: >>>> > >>>> > sysctl machdep.cpu.brand_string ; sysctl machdep.cpu | grep -i >>>> "avx\|sse” >>>> >>>> $ sysctl machdep.cpu.brand_string ; sysctl machdep.cpu | grep -i >>>> "avx\|sse” >>>> machdep.cpu.brand_string: Apple M1 >>>> >>>> -- >>>> arno s hautala/-| a...@alum.wpi.edu >>>> >>>> pgp b2c9d448 >>>> >>>> >>>> >>> >> >
Re: M1 CPU features
On Mon, Apr 26, 2021 at 1:42 PM Christopher Jones wrote: > Thats not at all surprising as those instruction sets are very much > specific to X86_64 systems. > > RISC processors, Arm, do have their own sets of SIMD instructions (e.g. > Neon), but they are entirely different to those on X86_64 machines. > > Whether or these are supported on Apple’s M1 processors I have no idea. > It looks like the M1 supports Neon SIMD instructions, but not SVE SIMD (which I guess is supposed to be similar to AVX?): https://discussions.apple.com/thread/252073619 -- Jason Liu On Mon, Apr 26, 2021 at 1:42 PM Christopher Jones wrote: > > > On 26 Apr 2021, at 6:28 pm, Jason Liu wrote: > > Thanks Arno :) > > I'm kind of surprised that the M1 doesn't seem to support any SSE or > AVX > > > > Thats not at all surprising as those instruction sets are very much > specific to X86_64 systems. > > RISC processors, Arm, do have their own sets of SIMD instructions (e.g. > Neon), but they are entirely different to those on X86_64 machines. > > Whether or these are supported on Apple’s M1 processors I have no idea. > > Chris > > > > > Does "sysctl machdep.cpu.features" return anything? > > -- > Jason Liu > > > On Mon, Apr 26, 2021 at 1:23 PM Arno Hautala wrote: > >> > On 26 Apr 2021, at 13:20, Jason Liu wrote: >> > >> > sysctl machdep.cpu.brand_string ; sysctl machdep.cpu | grep -i >> "avx\|sse” >> >> $ sysctl machdep.cpu.brand_string ; sysctl machdep.cpu | grep -i >> "avx\|sse” >> machdep.cpu.brand_string: Apple M1 >> >> -- >> arno s hautala/-| a...@alum.wpi.edu >> >> pgp b2c9d448 >> >> >> >
Re: M1 CPU features
That's it?! For all of machdep.cpu?! That's surprisingly minimal :/ -- Jason Liu On Mon, Apr 26, 2021 at 1:25 PM Gary Palter wrote: > [palter@miniMe ~/VLM/IssuesAndWiki.wiki](155)$ sysctl > machdep.cpu.brand_string ; sysctl machdep.cpu | grep -i "avx\|sse" > machdep.cpu.brand_string: Apple M1 > [palter@miniMe ~/VLM/IssuesAndWiki.wiki](156)$ sysctl machdep.cpu > machdep.cpu.cores_per_package: 8 > machdep.cpu.core_count: 8 > machdep.cpu.logical_per_package: 8 > machdep.cpu.thread_count: 8 > machdep.cpu.brand_string: Apple M1 > [palter@miniMe ~/VLM/IssuesAndWiki.wiki](157)$ > > - Gary > > > On Apr 26, 2021, at 1:20 PM, Jason Liu wrote: > > Can someone who owns an M1 Mac run the following command and let me know > what the output is? > > sysctl machdep.cpu.brand_string ; sysctl machdep.cpu | grep -i "avx\|sse" > > -- > Jason Liu > > >
Re: M1 CPU features
Thanks Arno :) I'm kind of surprised that the M1 doesn't seem to support any SSE or AVX Does "sysctl machdep.cpu.features" return anything? -- Jason Liu On Mon, Apr 26, 2021 at 1:23 PM Arno Hautala wrote: > > On 26 Apr 2021, at 13:20, Jason Liu wrote: > > > > sysctl machdep.cpu.brand_string ; sysctl machdep.cpu | grep -i "avx\|sse” > > $ sysctl machdep.cpu.brand_string ; sysctl machdep.cpu | grep -i "avx\|sse” > machdep.cpu.brand_string: Apple M1 > > -- > arno s hautala/-| a...@alum.wpi.edu > > pgp b2c9d448 > > >
M1 CPU features
Can someone who owns an M1 Mac run the following command and let me know what the output is? sysctl machdep.cpu.brand_string ; sysctl machdep.cpu | grep -i "avx\|sse" -- Jason Liu
Re: Publicizing MacPorts
On Wed, Apr 21, 2021 at 10:42 PM Ken Cunningham < ken.cunningham.web...@gmail.com> wrote: > A set of pages with some screenshots of some of the cool GUI software and > games, like SNAP has for Ubuntu. To piggyback off Ken's statement, as a part of the modernization efforts for the website, it would be nice to have a section or infobox on the MacPorts front page with either a featured "App of the Day" or "App of the Week", similar to what is in the iOS/Mac App Store. Just off the top of my head, I can think of over a dozen candidates, including GIMP, Inkscape, VLC, Blender, Audacity, KeePassXC, Battle for Wesnoth, SuperTuxKart, etc. We could even automatically populate the text of the infobox with the long_description directly from the portfiles. Add some pretty screenshots, and of course, the bottom would always say something like: To install this app using MacPorts, run this command sudo port install gimp in a terminal window. We could even make it so that if a visitor to the website clicks on the text for the "sudo port install" line, it automatically copies the command to their clipboard. I think that just adding this one thing could make the MacPorts website feel more lively and dynamic. I can also help with a redesign or modernization of the public-facing website. I did some professional web design work to put myself through college. I'm more of a web programmer than a graphic designer, but taking good-looking screenshots of apps shouldn't be too difficult. Even better would be to scrape the screenshots directly from the project authors' websites, since they are much more likely to have up-to-date screenshots whenever they make changes to their UI. -- Jason Liu On Wed, Apr 21, 2021 at 10:42 PM Ken Cunningham < ken.cunningham.web...@gmail.com> wrote: > various people have come along with some good ideas over the five years > I’ve been around. > > A set of pages with some screenshots of some of the cool GUI software and > games, like SNAP has for Ubuntu. > > A rewrite of the very old and dated and cobwebby website. > > To be honest — these people in the end have not been encouraged to > proceed. “I’ll get to that” or “Who is going to do that?” or “Why should we > do that?” are far more common responses. > > So — start there, I would volunteer. > > Ken
Re: Publicizing MacPorts
On Mon, Apr 19, 2021 at 1:25 PM Craig Treleaven wrote: > Also, why should we consider that MacPorts is in competition with > Homebrew? Both MacPorts and Homebrew seem to have a sufficient number of > contributors to keep going for the foreseeable future. Nether packaging > system has to "win" nor does the other have to "lose". The projects do > have differing philosophies that may make one more suitable than the other > for particular users. > Although this is a nice sentiment, I believe the reality is that MacPorts is in fact in competition with Homebrew. And not just Homebrew, but with other package managers as well, such as Munki, and even to some extent other deployment products such as Jamf/Casper and Jenkins. My belief is that the total number of systems running macOS as its operating system is the entire "pie" of systems that could potentially use one of these macOS package managers. In addition, the vast majority of users will only use one package management product, hence my opinion of why it's a pie with a limited number of potential users that gets divided up. The possible exception to only using one product per machine might be, say, in an enterprise setting (you can read my personal anecdote below for an example). In addition, it has traditionally been the case that package management systems say on their websites that installing multiple package managers on one machine can cause problems... e.g. MacPorts doesn't work well with Fink and Homebrew, Homebrew doesn't work well with Fink and MacPorts, etc. People on this mailing list are tech savvy enough to deal with the potential conflicts that might occur regarding environment variables, $PATH order, etc. but the vast majority of users won't be, and thus would stick to using only one package manager. If one product "wins" by getting installed on a particular system, then the others "lose". Personal Anecdote In one of my former lives, I used to work as a systems administrator for one of the science departments at a local university, and we used more than one package management system (PMS) on our Macs. The central, university-wide IT support group used a combination of Munki and Jenkins to deploy proprietary commercial software that requires license keys, such as Adobe and Microsoft products. The department-level IT support staff have superuser access on each of the computers, but we didn't have the ability to contribute our own software packages to the central IT's Munki/Jenkins servers. So, several science departments banded together to also deploy MacPorts on their departments' Macs, so that we could manage software packages that were of interest to the individual departments. We were also aware of other departments that used Homebrew as their departmental PMS. In fact, the central IT group even added a Homebrew "installer" to the list of software available through Munki, but refused to add the MacPorts installer (the sysadmin that was the head of managing the Munki servers loved Homebrew and Ruby, and absolutely hated Tcl). This is one of the few instances where I could see the benefit of having more than one PMS installed on the same machine. But for home computers that are being managed by a single family member, or even computers (laptops) that are only ever used by a single person, I highly doubt that they would willingly go through the hassle of obtaining their software through more than one PMS. -- Jason Liu On Mon, Apr 19, 2021 at 1:25 PM Craig Treleaven wrote: > > On Apr 19, 2021, at 10:47 AM, Karl-Michael Schindler < > karl-michael.schind...@physik.uni-halle.de> wrote: > > > > Am 19.04.2021 um 14:00 schrieb macports-dev-requ...@lists.macports.org: > >> > >> Date: Mon, 19 Apr 2021 13:24:51 +0200 > >> From: Mojca Miklavec > >> Subject: Re: Publicizing MacPorts > >> Message-ID: < > calbomsbrso9ao2sgvonvp69hskephjiwzhscmwmi_f6cfay...@mail.gmail.com> > >> Content-Type: text/plain; charset="UTF-8" > >> > >> We should probably be publicising that MacPorts works just fine with > >> M1 more aggressively from the very beginning. > >> > >> If anyone is willing to volunteer to do PR for MacPorts ... > >> > >> Mojca > > > > I intend to do so, probably to a limited extend only. As a first step, I > have checked the upstream download pages of our top ten downloads. Less > than 5 of them mention macports properly. My next steps is to write to > them. I also want to extent the maintainer part of the docs with the direct > to port maintainer to check upstream download and install pages. > > > > Michael. > > People don’t install MacPorts or Homebrew just to have a package > manager—they install a package manager as a prerequisi
Re: License GPL-2 conflicts with OpenSSLException
On Fri, Apr 16, 2021 at 9:21 PM Joshua Root wrote: > OpenSSLException conflicts with everything except the OpenSSL license. > Wait, what? If OpenSSLException conflicts with everything except the OpenSSL license, doesn't that make it tautologically equivalent to the OpenSSL license itself? (i.e. the sentence is basically the same as saying "if and only if") I thought the entire point of OpenSSLException was to specify that the authors have made the exception (in writing) that allows their GPL-licensed project to link against OpenSSL? Wouldn't that imply that OpenSSLException has to be compatible with something (i.e. the GPL) in addition to the OpenSSL license? -- Jason Liu On Fri, Apr 16, 2021 at 9:21 PM Joshua Root wrote: > On 2021-4-17 11:16 , Ryan Schmidt wrote: > > > https://build.macports.org/builders/ports-10.15_x86_64-builder/builds/55652/steps/gather-archives/logs/stdio > > > >> "hydrogen" is not distributable because its license "GPL-2" conflicts > with license "OpenSSLException" of dependency "qt5-qtbase" > > > > Does this make sense or is there an error in the script? Why would GPL-2 > or anything conflict with OpenSSLException? It's just an exception. It > lifts some restrictions imposed by the GPL. It shouldn't be imposing > additional restrictions itself, should it? > > Implementation detail. OpenSSLException conflicts with everything except > the OpenSSL license. If you see that, it means that all the other > license options also conflicted (in this case, GPL-3 or LGPL-3). > > - Josh >
Re: 11_x86 build - build failures due to 'no space on device'
> > For some reason fseventsd was taking 14GB of memory. > Not really surprising, for a machine being used as a builder. The constant creation and deletion of files when compiling software is bound to be generating a ton of file system events. -- Jason Liu On Thu, Apr 8, 2021 at 1:58 PM Ryan Schmidt wrote: > > > On Apr 8, 2021, at 07:25, Ryan Schmidt wrote: > > > We now have 21GB free. > > 31GB free after rebooting. For some reason fseventsd was taking 14GB of > memory. The VM only has 8GB of real memory so a lot of swap was being used. > This is probably also the explanation for why that builder has gotten a > larger backlog of builds lately. > >
Re: FAQ wording (was: Re: Desolate Condition)
>
> What if the end of the first sentence was changed to: "ports will be
> installed for only the architecture you're currently running on."?
>
Besides being the most concisely worded, I also find this version to be the
most unambiguous. I think leaving the words "build/built" and "compile" out
of the sentence entirely would be for the best.
--
Jason Liu
On Mon, Apr 5, 2021 at 10:15 AM wowfunha...@gmail.com
wrote:
> > It's a tricky thing to state both concisely and accurately because the
> > compilation may indeed happen on the user's computer after installation
> > is requested. Or it may not. The ambiguity of whether "be compiled" is a
> > verb in the passive voice ("the code is being compiled") or a
> > description of a state ("this is code compiled for x86_64") may reflect
> > the author's awareness of the undecidedness of which one will actually
> > happen.
> >
> > Improvements to the text's clarity are very welcome, of course.
> >
> > - Josh
>
> What if the end of the first sentence was changed to: "ports will be
> installed for only the architecture you're currently running on."?
> Basically mirroring the other change made on Saturday.
>
>
>
Re: Built archive not uploaded
On Sat, Mar 27, 2021 at 2:01 PM Joshua Root wrote: > > The 10.15 builder is working through a large backlog after its downtime, > and is currently handling commits from around 300 hours ago. On Sat, Mar 27, 2021 at 1:44 PM Mojca Miklavec wrote: > > I didn't try to look into details, but note that the 10.15 builder has > more than 500 jobs in the backlog, so it might also be just a matter > of time. Thanks for pointing this out... I had no idea that the 10.15 builder had such a huge backlog of pending builds. -- Jason Liu On Sat, Mar 27, 2021 at 2:01 PM Joshua Root wrote: > On 2021-3-28 04:39 , Jason Liu wrote: > > Hi all, > > > > It seems that the 10.15 build for port:oidn either failed or something > > went wrong when uploading the built package to the public archive: > > > > https://packages.macports.org/oidn/ <https://packages.macports.org/oidn/ > > > > > > Could someone with access to the builders take a look and see what the > > problem might be? > > > > https://ports.macports.org/port/oidn/ > > <https://ports.macports.org/port/oidn/> isn't providing me with any > > useful clues. > > If you click the health indicator for 10.15 you will see the build that > that status is based on. In this case it was on the 23rd of January when > the port was updated to version 1.2.1. And indeed that version is on the > packages server. > > The 10.15 builder is working through a large backlog after its downtime, > and is currently handling commits from around 300 hours ago. > > See also <https://github.com/macports/macports-webapp/issues/135>. > > - Josh >
Built archive not uploaded
Hi all, It seems that the 10.15 build for port:oidn either failed or something went wrong when uploading the built package to the public archive: https://packages.macports.org/oidn/ Could someone with access to the builders take a look and see what the problem might be? https://ports.macports.org/port/oidn/ isn't providing me with any useful clues. -- Jason Liu
Re: INSTALL_PATH with spaces
Technically, MacPorts does have an equivalent inside ${prefix}, and that
would be the ${prefix}/Library directory. You could create an Application
Support directory in there, e.g. ${prefix}/Library/Application
Support/MacPass, and have the app load its plugins from that location. In
this way, you would still be staying in-line with MacPorts' mtree layout.
--
Jason Liu
On Sat, Mar 6, 2021 at 9:19 AM Janosch Peters via macports-dev <
macports-dev@lists.macports.org> wrote:
>
> > Am 06.03.2021 um 02:17 schrieb Ryan Schmidt :
> >
> > I would abandon this attempt because a port shouldn't be installing
> anything in a user's $HOME. Ports should accommodate being installed by one
> user but being used by another user, for example. And the user account we
> use on our build server is unlikely to be the same as the user account that
> the user is using.
> >
>
> You are of course right about not installing anything into $HOME. I
> changed the port to install into /Library/Application Support/MacPass and
> also changed the „parent“ app MacPass to load plugins from this location.
>
> If I were a seasoned Objective-C developer I would have changed the path
> to something below ${prefix}. If I find time to dig deeper into Objective-C
> I will do that, but for the time being I think /Library/Application Support
> is good enough.
>
> The PR for the new port MacPassHTTP is here:
> https://github.com/macports/macports-ports/pull/10162
>
>
Need help testing updated Blender portfile
Hi everyone, I'd like to ask whether anyone who has a Mac with *10.11 or 10.12* might be willing to test out a portfile for me before I submit the PR. The upstream devs bumped the minimum required versions of macOS and Xcode in Blender 2.90, but other than saying "require relatively recent versions" in the change log, they don't really give a good reason for doing this. I patched/reverted the version numbers using my portfile, and it still seems to compile fine on my machine. However, I no longer have any way of testing it, because my 10.11 machine has fallen victim to the 2011 MacBook Pro GPU flaw. Here are instructions for how to test my new portfile: 1. Grab my portfile <https://github.com/jasonliu--/macports-ports/tree/blender-2.90.0/graphics/blender> (and also everything inside the files/ folder), and do a "port install" on it. 2. Once the build completes, go and download some of the Blender demo files <https://www.blender.org/download/demo-files/>. I would recommend "Splash Fox <https://cloud.blender.org/p/gallery/5f4cd4441b96699a87f70fcb>" from the Blender Splash Screens <https://www.blender.org/download/demo-files/#blender-splash-screens> section, and/or "Nishita Sky Demo <https://cloud.blender.org/p/gallery/5f4d1791cc1d7c5e0e8832d4>" from the Cycles <https://www.blender.org/download/demo-files/#cycles> section. 3. Open the file in Blender. 4. See whether you can zoom in and out using the mouse scroll wheel. If you have a physical mouse scroll wheel that you can click, then try holding down the scroll wheel and dragging your mouse around inside the viewport to rotate the camera around the scene. 5. Try rendering a still image of the scene by hitting the F12 key. If you're feeling brave and believe that your machine is powerful enough, you can try rendering the entire animation by hitting Command-F12. (Note: Rendering animations is resource intensive and can take a very long time!) If Blender fails to behave as expected or crashes at any point during my instructions, please let me know. Conversely, if everything worked without any problems, please also let me know. -- Jason Liu
Re: 99 bottles of beer on the wall...
Down to only 2 pages and < 50 open PRs... it's starting to look like my Inbox Zero! Also, we're fast approaching PR #10,000! -- Jason Liu On Mon, Jan 25, 2021 at 3:39 PM Ken Cunningham < ken.cunningham.web...@gmail.com> wrote: > the PR list is < 100 for the first time in a very very long time…. > > Keep ‘er going! > > K
Re: "cask" ports just keep on rolling in...
> > The destroot phase needs to — what — record the files that will be > installed in some kind of an index file? And also know about the stuff that > gets installed into ~/Library, etc. > You would probably need to get the list of files installed by the installer by running either pkgutil --files or lsbom, if we're talking about a PKG installer. (Or maybe it would be easier to just grab the bom file in its entirety.) Typically running an ls -R on the .app bundle inside a DMG installer would be sufficient for those kinds of installers. But then this begs the question: what happens to all of the work that went into the build-from-source packages? Wouldn't this end up rendering the hundreds of hours I spent getting the Blender package to work a complete waste? -- Jason Liu On Fri, Feb 5, 2021 at 5:31 PM Ken Cunningham < ken.cunningham.web...@gmail.com> wrote: > What would we really need to do this properly? > > The current fetch, checksum phases are OK. > > The extract phase can be used as is for some software (simple copies), but > for other software we don’t want to extract it, we want to run the > installer. > > The configure and build phases need to be overridden and disappear. > > The destroot phase needs to — what — record the files that will be > installed in some kind of an index file? And also know about the stuff that > gets installed into ~/Library, etc. > > Then run the “cask” destroot without installing any files into the actual > destroot: > > copy the apps and stuff > or > run the installer > or > extract from the pkg > or > … > > and then NOT have the entire software repackaged into a MP archive file to > be stored 12 different times… > > Or some such plan > > Best, > > Ken > > > > > > On Feb 5, 2021, at 11:00 AM, Ken Cunningham < > ken.cunningham.web...@gmail.com> wrote: > > With no plan, we’ll just keep getting more and more of these. > > <https://github.com/macports/macports-ports/pull/9936> > > This kind of port just repackages the DMG into many tgz archives. > > It’s wasteful. People want them. > > What we should have instead of this is some kinda tech that > > 1. downloads the DMG > 2. installs the app > 3. records some way of uninstalling everything > > > What we have instead is a repackaging of the DMG into many, identical, > system-specific archive bundles. > > Yuk. > > Ken > > >
Re: Desolate Condition
When popular software titles either get added as new ports, or when they get a version update, maybe there should be some sort of post. For example, I believe that LibreOffice recently got added to the ports tree, although I'm not sure how complete of a package it is compared to the DMG installer that is available directly from libreoffice.org; it's possible that it might still be a work-in-progress. However, whenever one of the popular titles reaches parity with the upstream authors' DMG installer, then we could post something along the lines of " is now available as an official package in MacPorts!" I realize that what is considered a "popular software title" is very subjective. Maybe "high visibility title" might be a more appropriate phrase? I'm not sure. But anyway, that's my suggestion. -- Jason Liu On Sat, Jan 30, 2021 at 12:50 PM Christopher Nielsen < masc...@rochester.rr.com> wrote: > *On 2021-01-30-S, at 12:00, Rainer Müller > wrote:* > > > Yes, we do have a Twitter account [1], but we are not using it much. > Mostly we announce new releases and maintenance stuff there. > > I added a new SocialMedia wiki page [2] to document the existence of such > social media presence. This also has the list of users that already have > access and the signatures they use for tweets. > > Help with this would be very welcome! I can give out access to more users > via TweetDeck [3], if you tell me your Twitter handle. > > Rainer > > [1] https://twitter.com/macports > [2] https://trac.macports.org/wiki/SocialMedia > [3] https://tweetdeck.twitter.com > > > You’re welcome to add me. My Twitter handle is @ChrisNielsen333. > > In terms of current tweet-worthy items, there are at least two that I’d > suggest: > * The recent migration to libjpeg-turbo. It was quick, painless, and a big > win for our users! > * All of the great work folks have been doing, to support Apple’s M1 Macs. > > Anything else? >
Re: Bash version on Big Sur?
Great, thanks! :) -- Jason Liu On Thu, Jan 28, 2021 at 2:01 PM Arno Hautala wrote: > > On 28 Jan 2021, at 13:11, Jason Liu wrote: > > > > What is the version of the built-in bash shell in Big Sur? > > > > Is it still "3.2.57(1)-release"? > > Yes > > And for reference: > > /bin/sh > /bin/bash > GNU bash, version 3.2.57(1)-release (arm64-apple-darwin20) > Copyright (C) 2007 Free Software Foundation, Inc. > > > /bin/ksh > version sh (AT Research) 93u+ 2012-08-01 > > > /bin/csh > /bin/tcsh > tcsh 6.21.00 (Astron) 2019-05-08 (unknown-apple-darwin) options > wide,nls,dl,bye,al,kan,sm,rh,color,filec > > > /bin/zsh > zsh 5.8 (x86_64-apple-darwin20.0) > > > -- > arno s hautala/-| a...@alum.wpi.edu > > pgp b2c9d448 > > >
Bash version on Big Sur?
Hi all, What is the version of the built-in bash shell in Big Sur? Is it still "3.2.57(1)-release"? -- Jason Liu
Re: Desolate Condition
> > One advantage that HomeBrew does have, though, is cachet: There are so > many times when articles - or even organizations, such as Google - simply > recommend using HomeBrew… with no mention of MacPorts. > > So, my feeling is that we need to up our public relations game. Do we have > an active social media presence, for example? (Twitter in particular?) > > Of note, while I’m not an expert in social media relations, I’d happily > volunteer to help with it. > > Thoughts? > I completely agree with this point. Due to MacPorts' low social media visibility, and thus low mind share in this day and age, it seems that lots of people, even including software authors, don't seem to consider MacPorts as a viable (or at the very least, a mainstream) method of obtaining software. I see plenty of open source projects have a blurb on their websites saying " is available through Homebrew using 'brew install --cask '", but I never see an equivalent blurb for MacPorts these days. I also agree with Andrew Janke's point that "MacPorts feels like more of a "pro" thing and serious sysadmin tool, and its command output can be kind of technical and intimidating." MacPorts essentially adds a *nix-style package management system onto a Mac, and these *nix PMSes are also (in)famous for feeling technical and intimidating. Perhaps a GUI like Pallet would help in this regard? There seems to be much higher comfort levels with GUI-based "app stores", even among non-technical users. -- Jason Liu On Tue, Jan 26, 2021 at 10:12 AM Christopher Nielsen < masc...@rochester.rr.com> wrote: > > *Ken Cunningham wrote:* > homebrew is in shambles. > > their long-touted "no-sudo" and "no PATH" advantage from installing into > /usr/local has been eliminated by Apple as the horrible security threat it > always was. They have to retool into /opt/homebrew and make 10,000 builds > respect the build args now. > > They stripped out all their universal handling code a few years ago, can't > put it back, and so can't do the critical universal builds any more. They > tell everyone universal is wasteful, lipo things manually, and run the > x86_64 homebrew on Apple Silicon. > > So MacPorts, which works great from 10.4 PPC to 11.x arm64, is the place > to be. > > > Personnally, I’ve never actually tried HomeBrew, as I didn’t want anything > installed into core OS areas. And after choosing MacPorts years ago - 10+ > at this point? - I’ve always been very happy with the experience. Enough so > that I’m finally giving back, as a contributor! > > One advantage that HomeBrew does have, though, is cachet: There are so > many times when articles - or even organizations, such as Google - simply > recommend using HomeBrew… with no mention of MacPorts. > > So, my feeling is that we need to up our public relations game. Do we have > an active social media presence, for example? (Twitter in particular?) > > Of note, while I’m not an expert in social media relations, I’d happily > volunteer to help with it. > > Thoughts? >
Re: port "cask" -- installing prebuilt binaries
>
> A more efficient method that overrides activate/deactivate and does not
> needlessly de everything would seem useful.
>
Since the dmg itself is a compressed archive, it would seem to make sense
that activate/deactivate would operate directly on the dmg, no? In other
words, the distfile *is* the archive, so recompressing of the destroot
should be skipped entirely. Instead, the dmg distfile should either be
copied or moved over to where the activate/deactivate archive usually lives.
--
Jason Liu
On Thu, Dec 24, 2020 at 5:27 AM Ken Cunningham <
ken.cunningham.web...@gmail.com> wrote:
> I've been building a few of these "cask" installers for my own use.
>
> I'm finding using the normal port activate/deactivate process quite
> inefficient for this...
>
> For example, SageMath is a 1.5GB dmg, and is about 4GB installed.
> Downloading the dmg, decompressing it and copying the contents into
> destroot, recompressing it, then activating it and decompressing it again
> takes just forever to do and seems wasteful on space as well.
>
> A more efficient method that overrides activate/deactivate and does not
> needlessly de everything would seem useful.
>
> Also a recommended method to avoid mirroring the distfiles ("NoMirror"
> license I presume) and not caching the archives (clearing archive_sites I
> suppose) are needed.
>
> Ken
Re: Package managers and package versioning
and then install the
>> old versions of the dependencies that are compatible with the old package.
>>
>
> If all versions of a port install to the same location on disk as they do
> currently and only one version of a port can be active at a time, then your
> proposal here -- "deactivate all of the dependencies that were too new" --
> will break all the ports that depend on the newer dependencies.
>
Yes, that is precisely what I am proposing: that doing so would break all
ports that depend on the newer dependencies. This is exactly what happens
in PMSes like APT and YUM. And very similar to what happens if you were to
do a "port deactivate libpng", the PMS will provide a helpful warning
message telling you that the following list of packages will be broken if
you proceed... do you still wish to continue? [y/N].
I realize that there are probably many people screaming at me through their
monitors right now saying, "That's a TERRIBLE idea! Why would we want to
allow users to break a bunch of stuff using our system?!" Other PMSes will
warn you when you are about to do something like that, but the fact is that
they will still allow you to do it if you insist on continuing. Again, I
point out that this is where MacPorts' activate/deactivate system could
really shine, because MacPorts still retains on local disk the other
version that was already installed, and you can switch back and forth
between versions using "port activate". This is still much better than APT,
which will simply uninstall the newer version of the dependency and replace
it with the older version (i.e. you can only have one copy of a package
installed on the system at one time).
For us, the single product we are providing is MacPorts, which has a very
> broad focus: ten thousand software packages intended to work together. That
> goal isn't always met: sometimes updating one port breaks another, until we
> fix it. But treating each of those ten thousand ports as a separate
> top-level product that needs its own world of dependencies would be an
> unfathomable disaster.
>
> Don't forget also that separate libraries would be loaded into memory
> separately. If you have a zillion copies of gettext and libpng installed,
> one for each port that uses them, and then you want to use something huge
> like Qt or WebkKt, then you're loading hundreds of copies of gettext or
> libpng into memory.
>
But isn't this precisely what NPM does? Apple's take (and possibly NPM's
too) would be to say, "storage and memory are cheap these days". I don't
know where all this supposed cheap stuff is available for sale, but I
certainly don't have the money to just go out and upgrade the storage and
memory on all of my computers.
I hope it's clear by now, but just in case we're still in "nothing is
obvious" territory, let me clearly state that my personal preferences
generally lean more towards the side of the *nix-like philosophy of
"sharing/reusing as many resources as possible is better". However, I think
you probably have to admit one thing about NPM is that its packages are
very self-contained.
I thought it was being proposed that all versions of a port would install
> to different locations on disk so that many could be active simultaneously,
> which would avoid the one problem while causing many new ones, most of
> which we haven't begun to discuss yet.
>
Let me try an example.
>
> Suppose I have zlib installed at:
> /opt/local/port/zlib/1.2.11/{bin,include,lib}
>
> So I hope this npm-style hierarchy is not what anyone is proposing.
>
No, I certainly hope nothing that I had said before was giving the
impression that this is what I was suggesting, and in my mind this would be
a bad idea. However, I think it's impossible to deny that all of the PMSes
mentioned so far, APT, YUM, NPM, Yarn, etc. have all been quite successful,
and they all give users the ability to install specific versions of
packages.
--
Jason Liu
On Sun, Oct 4, 2020 at 5:34 PM Ryan Schmidt wrote:
>
>
> On Oct 4, 2020, at 14:09, Jason Liu wrote:
>
> >> Where by "this" you again mean the ability to specify a
> previously-available version, not an arbitrary version that was never in a
> portfile.
> >
> > Yes, of course. If you look at the folder for the Blender package on the
> Debian repo
> >
> > http://deb.debian.org/debian/pool/main/b/blender/
> >
> > from my previous example, there are many versions of Blender that were
> skipped. For instance, they skipped version 2.82 and went from 2.79b right
> to 2.82a. And they skipped versions 2.83.0-2.83.4, and went from 2.82a
> right to 2.83.5. Obviously, there would be no way for a user to try to
> install Blender version 2.83.2 using apt-get, because that version was
> never packaged (i
Re: Package managers and package versioning
be required in every single portfile. If these (required)
variables also had the ability to specify version ranges, we could
effectively cap each version of a portfile with a maximum compatible
version of macOS and Xcode, which could be raised as compatibility patches
are added for newer macOSes and Xcodes.
macos_version_compatibility <=10.15.7
(I prefer to use Darwin version)
darwin_version_compatibility <=19
(this portfile would be compatible up to Catalina, but not yet compatible
with macOS 11)
xcode_version_compatibility >=8.2.1 && <=12beta2
(or whatever)
The benefit of this would be as follows. Let's say that we went to a system
of having one portfile for each version of a particular package. Let's also
say that "somepackage" was currently on version 2.4.3, and it is not
compatible with macOS 11. If the upstream authors of "somepackage" were to
add patches that made it compatible with macOS 11 and released it as 2.4.4,
then the new portfile for 2.4.4 would increment darwin_version_compatibility
<=20. However, the portfile for 2.4.3 wouldn't need to be changed at all,
because it wasn't, and never will be, compatible with macOS 11. If a port
maintainer were to come along and backport the macOS 11 patches into the
2.4.3 port, then they could update the 2.4.3 portfile to also be compatible
with darwin <=20. A similar concept applies to Xcode version, ARM support,
etc.
Thus, the idea of keeping a separate portfile for each version of a package
would answer the question of "how you could build historical "Portfiles" on
a newer OS or compiler version", because the answer is: you never would.
Once a "Portfile-2.4.3" was pegged to a maximum of Catalina, it would
potentially never need to be updated again, because it is not compatible
when macOS 11 gets released. Our new "port" command would be able to
indicate to users on a macOS 11 machine that only "Portfile-2.4.4" was
available, because the old historical "Portfile-2.4.3" isn't compatible
with macOS 11.
This is all work that we are here volunteering to do to improve our current
> port collection, but asking everyone to retroactively also fix these
> problems in all older versions would not be acceptable. We don't have a
> sufficient number of contributors to keep our current collection of ports
> up to date, so we certainly don't have enough contributors to keep all
> previous versions of our collection working as well.
>
This is why it would be important in a "new" MacPorts to be able to cap old
versions of a portfile with a maximum compatible version of stuff. ("stuff"
meaning dependencies, macOS version, Xcode version, etc.) This would allow
older versions of portfiles to slowly fade into obsolescence as their
maximum compatible versions of dependencies become more and more
out-of-date. There would be no need to keep retroactively updating them
once they've been capped with maximum compatible versions.
If compatibility patches can be backported, then great, but if they can't
be for any reason, then that's fine too, the old versions of the portfiles
will continue to work for older versions of macOS/Xcode. But in my opinion
that would be a much better situation than what exists now, where there's
only a single Portfile (and a single version of the package) for all users
on all versions of macOS. However, I once again reiterate that I also
believe it would require a massive amount of effort to implement this new
regime.
You might even have to compile each of those copies from source, since each
> copy is going to a different location, and MacPorts binaries are not
> designed to be relocatable. Making our binaries relocatable is something
> I'm interested in but that is also a large challenge.
>
MacPorts binaries aren't designed to be relocatable, because MacPorts, in
my opinion, operates under a very *nix-like philosophy of sharing and
reusing resources. This is also why many macOS apps *are* relocatable:
because they don't share resources with anything else, and thus can play in
their own sandbox. Regardless of where you relocate this sandbox to, there
is very little chance of the app breaking when you move it, because all of
its toys (like the gettext discussed above) move along with it. In many
cases, sharing/reusing resources and relocatability are direct trade-offs
and in opposition to one another. (Obviously, things like @rpath,
@executable_path, @loader_path, etc. were designed to help break this
linkage/trade-off.)
--
Jason Liu
On Sun, Oct 4, 2020 at 3:00 AM Ryan Schmidt wrote:
>
>
> > On Oct 3, 2020, at 20:23, Jason Liu wrote:
> >
> >> On Fri, Oct 2, 2020 at 9:00 PM Ryan Schmidt wrote:
> >>
> >> On Oct 2, 2020, at 17:42, Lothar Haeger wrote:
> >>> Instead of creating separate copies of perl for each version, it
> would've
Re: Supporting installing arbitrary port versions (was: Re: port "cask" -- installing prebuilt binaries)
e pre-compiled .deb archives. One of the most important ways in which an APT or YUM repository differs from a MacPorts ports tree is this: what kinds of data get downloaded to a client's computer during a sync operation. From "man port" on my Mac: sync > Performs a sync operation only on the ports tree of a MacPorts > installation, *pulling in the latest revision available of the Portfiles > from the MacPorts rsync server*. > On an APT- or YUM-based Linux machine, none of the spec files for any package get pulled down to a client machine during a sync operation. Instead, what gets synced is a database that only contains metadata about packages. This metadata is made up of entries from all of the .dsc files in the entire Debian repository. So, when a user runs this command: apt-get install blender=2.79.b+dfsg0-7 apt-get will first attempt to download the pre-compiled .deb archive. If a package doesn't have a .deb available, then it will download all three items and attempt to build from source. In this way, apt-get is basically downloading the old portfile for Blender 2.79b when it gets the third item, the .debian.tar.xz file. On Sat, Oct 3, 2020 at 11:00 AM Ryan Schmidt wrote: > On Oct 3, 2020, at 06:05, Lothar Haeger wrote: >> >> Implementing all this would be a major project and I totally understand >> that it's nothing to jump at without considering all the consequences and >> outside factors. >> > > It would be a major project if it were clear how it could possibly work > and it were just a matter of doing it. But so far I have no idea how it > could work. > As I said in my previous message, in the worst-case scenario it would require a complete rewrite of MacPorts. Hopefully after I described how an APT repo works, it's a bit more clear why I said that. Right now, MacPorts has no concept of older and newer versions of a portfile. For example, in my Blender port, there's no such thing as a "Portfile-2.82a", "Portfile-2.83.0", "Portfile-2.83.1", etc. There's only a single Portfile... the newest one. If I change the Portfile to a newer version of Blender, I'm basically overwriting the same spec file for Blender, not creating a brand new spec file for each new version of Blender, which is how it is done for an APT or YUM repo. -- Jason Liu On Sat, Oct 3, 2020 at 11:00 AM Ryan Schmidt wrote: > > > On Oct 3, 2020, at 06:05, Lothar Haeger wrote: > > >> It's because, besides being an unimaginably large amount of work in > rearranging our code to do it, I have absolutely no idea how it would be > accomplished without providing the user with unlimited opportunities to > create broken combinations of port versions, which would generate an > unlimited number of bug reports that we would then need to respond to, and > my goal in MacPorts is to reduce, not increase, the likelihood that users > would find something broken or need to contact us to help troubleshoot it. > >> > >> If you have an idea for how it could be done without such problems > arising, please open a new topic and describe it and we can talk through a > few scenarios and see if it works. > > > > I do have some ideas (not mine, just looking at how it works elsewhere), > but not the time to drive something of this size, to be honest. Basically, > we'd need to > > - keep and distribute all Portfile versions (including patches and > files), not just the lastest > > - add required versions to dependencies (can be version ranges e.g. > >=1.2.3 or 2.* or 1.2.3-4.5.6) > > - add support to install individual ports in into separate folders or > give them individual names on install time (much like with the perl > versions), so multiple versions of a port can be installed in parallel. > > - add command line parameters to the port command so users can > optionally specify a version to install (defaults to latest) and a pre/post > fix to install location or file names or both (depends on how stuff gets > implemented, of course) > > - add support for dependency resolution including version information. > So if two ports have different, non overlapping version requirements for a > dependency, that dependency gets installed twice. Making sure each port > then actually uses its matching version is probably one of the trickier > parts here. > > > > Implementing all this would be a major project and I totally understand > that it's nothing to jump at without considering all the consequences and > outside factors. > > It would be a major project if it were clear how it could possibly work > and it were just a matter of doing it. But so far I have no idea how it > could work. > > Just looking at your idea to distribute all portfile versions, let's start > with the fact that po
Package managers and package versioning
ing the active_variants PortGroup or require_active_variants) As for MacPorts, it's not that we haven't implemented it because we're > lazy. It's because, besides being an unimaginably large amount of work in > rearranging our code to do it, I have absolutely no idea how it would be > accomplished without providing the user with unlimited opportunities to > create broken combinations of port versions, which would generate an > unlimited number of bug reports that we would then need to respond to, and > my goal in MacPorts is to reduce, not increase, the likelihood that users > would find something broken or need to contact us to help troubleshoot it. > I agree, it would be a monumental amount of work. In the worst-case scenario, it would require a complete rewrite of MacPorts, including redesigning the entire server infrastructure as well as a top-to-bottom rewrite of the client software. At a minimum, it would probably require significantly changing how the MacPorts "port" command works. For example, I have so far submitted two versions of my MaterialX port: version 1.37.1 and 1.37.2. If I were to do a "port install materialx @1.37.1" (even though the current version in MacPorts is 1.37.2), then the MacPorts client software would first need to somehow go out onto the internet and fetch the portfile for MaterialX version 1.37.1. If you're interested how other PMSes accomplish this, I could go into further details. However, I disagree that it would provide unlimited opportunities to create broken combinations of port versions. This "unlimited"-ness would be prevented by restricting dependencies to a range of version numbers (or just a single version number). In reality, it would most likely reduce the number of bug reports, since you would be able to cap the maximum version of a dependency that a package was compatible with. Right now, MacPorts doesn't have that ability. -- Jason Liu
Need help with USD port
Hi all,
I'm trying to put together a portfile for a library called Universal Scene
Description, and I'm getting some strange errors when I try to compile it
on my machine. I'm looking for people with macOS 10.12 or newer to try to
compile USD and see whether they are getting the same compile errors. To
start with, here's my machine info:
macOS 10.11.6 15G22010
Xcode 8.2.1 8C1002
Here's what I'd like people to test:
1. Download USD version 19.11 from GitHub (yes, it has to be 19.11,
because that's the version Blender uses):
https://github.com/PixarAnimationStudios/USD/releases/tag/v19.11
2. Extract the source code.
3. mkdir /tmp/usd
4. cd USD-19.11/build_scripts
5. python2.7 build_usd.py --no-python /tmp/usd
IMPORTANT: Yes, you *have* to run the build script using Python 2.7. The
upstream authors didn't add any sort of Python 3 support to USD until
version 20.05.
Theoretically, the build script should try to use Xcode Clang to do the
compilation. Just to make sure, check the log file located in
/tmp/usd/build/USD-19.11/log.txt to verify that the build script is finding
AppleClang, and not any of the MacPorts Clangs that you might have
installed on your machine.
I'm looking for compile errors that look something like this:
USD-19.11/pxr/usd/lib/sdf/namespaceEdit.cpp:114:9: error: constructor for
> 'pxrInternal_v0_19__pxrReserved__::SdfNamespaceEdit_Namespace::_Node'
> must explicitly initialize the const member '_originalPath'
> _Node(const TfToken& name) : _key(name) { }
> ^
> USD-19.11/pxr/usd/lib/sdf/namespaceEdit.cpp:207:23: note: '_originalPath'
> declared here
> const SdfPath _originalPath;
> ^
> USD-19.11/pxr/usd/lib/sdf/namespaceEdit.cpp:115:9: error: constructor for
> 'pxrInternal_v0_19__pxrReserved__::SdfNamespaceEdit_Namespace::_Node'
> must explicitly initialize the const member '_originalPath'
> _Node(const _TargetKey& key) : _key(key.key) { }
> ^
> USD-19.11/pxr/usd/lib/sdf/namespaceEdit.cpp:207:23: note: '_originalPath'
> declared here
> const SdfPath _originalPath;
> ^
> USD-19.11/pxr/usd/lib/sdf/namespaceEdit.cpp:116:9: error: constructor for
> 'pxrInternal_v0_19__pxrReserved__::SdfNamespaceEdit_Namespace::_Node'
> must explicitly initialize the const member '_originalPath'
> _Node(const SdfPath& path) : _key(_GetKey(path)) { }
> ^
>
Regardless of build success or failure, please let me know which macOS and
Xcode/CLT you are using. If you are getting build failures, please also let
me know whether or not it is because you are seeing the same compile errors
that I'm seeing.
P.S.: My portfile won't actually be using the Python build script included
with USD, since MacPorts takes care of most of the actions performed by the
script; the portfile will be running CMake directly.
--
Jason Liu
Re: Apple ARM binary codesign issue
> > I've read that number of duplicates is one of the ways they determine > issue priority internally. > I can confirm that this is indeed supposed to be the case. Our Apple engineering liaison at the university where I used to work as a sysadmin frequently repeated this during our Apple developer meetings. These Apple developer meetings were where the IT staff from various departments around campus gathered once a month to discuss issues with the latest betas and issues with software apps being used by faculty, staff, and students. One of the main purposes of these meetings were to bring up potential bugs in the betas, and to have as many of the meeting attendees go back and try to duplicate the bug, and then report it. In fact, we even passed around the titles of the bug reports to the mailing list, so that it could get recorded as a duplicate of the same issue. Even better, if the original person's bug report was lucky enough to get a bug report number in Apple's Radar bug tracker, the Apple liaison would look it up and pass it along to everyone at the meeting, so that we could reference the original bug report (again, to increase the duplicate count). -- Jason Liu On Tue, Sep 22, 2020 at 4:17 PM Joshua Root wrote: > On 2020-9-23 05:33 , Ryan Schmidt wrote: > > > > Send feedback through the Feedback Assistant app. > > Yes, everyone with any issues with Apple preview software should do this > early and often. I've read that number of duplicates is one of the ways > they determine issue priority internally. > > - Josh >
