Re: if we are going to support universal arm/x86_64 ports, we'll likely need a multi-architecture gcc again

[Eric Gallager via macports-dev]

On Sun, Sep 13, 2020 at 1:21 PM Ken Cunningham <
ken.cunningham.web...@gmail.com> wrote:

> I’m not sure if the MacPorts plan is to support universal arm/x86_64
> builds, but I can imagine that for some years we might want to consider
> doing that.
>
> The last gcc version we had that supported multiarchitecture building was
> apple-gcc42 (Intel, PPC).
>
> We might have built a suitable newer cross-compiling gcc by updating the
> current gcc darwindriver.c with Apple’s lipo enhancements and doing a
> formal cross-compiling installation, but we never got around to it, and our
> current gcc versions are single arch (although support 32 and 64bit in
> each). There are some projects on github that did this (Frederick
> Devernay’s, among others).
>
> But with arm now on the horizon, we will need a cross-compiling gcc again
> if we are to have cross-arch universal support, which I can certainly see a
> need for. (A gcc that supports darwin arm has not yet been released, but it
> is currently working and exists).
>
> This is (IMHO) a major MacPorts advantage, as our universal support is
> unusually robust.
>
> Ken


It'd be great to see these changes contributed back upstream to the
official version of GCC in the GNU toolchain

Re: gpg_verify 1.0 feedback

[Steven Smith]

> I imagine there could be a much simpler interface for this using this 
> PortGroup, making it more appealing to add it to ports. … The PortGroup would 
> handle fetching the .sig or .asc file

Certainly port group gpg_verify can be improved.

However, reliance on accessing key servers is a Very Bad Idea™️ and doomed to 
fail. See the internet and previous discussions on this port group for the 
myriad reasons why this is.

> I think that gpg signature verification is something that belongs in
> maintainer-facing tools rather than in Portfiles. If the maintainer
> verifies the distfile's signature before updating the checksums, the
> user gets close to the same assurances while avoiding a lot of complexity.
> 
> I always verify before updating if the project provides signatures, and
> I would hope others do the same, but also wouldn't be surprised if some
> don't. 

Checksumming commands, especially gnupg ones, are arcane, error-prone, and add 
complication and time *especially* when done by hand.

If they’re not automated, they won’t be used. I wouldn’t trust them to be used 
(even by myself) for these reasons.

The simplest solution is a gpg verification block in the Portfile in the very 
few cases where it’s appropriate, rather than building an additional macports 
tool to do this. But if someone wants to take this on, then fine as long as 
it’s automated.

Steve

smime.p7s
Description: S/MIME cryptographic signature

Re: gpg_verify 1.0 feedback

[Joshua Root]

I think that gpg signature verification is something that belongs in
maintainer-facing tools rather than in Portfiles. If the maintainer
verifies the distfile's signature before updating the checksums, the
user gets close to the same assurances while avoiding a lot of complexity.

I always verify before updating if the project provides signatures, and
I would hope others do the same, but also wouldn't be surprised if some
don't. Better tools might help improve things.

- Josh

Need help with USD port

[Jason Liu]

Hi all,

I'm trying to put together a portfile for a library called Universal Scene
Description, and I'm getting some strange errors when I try to compile it
on my machine. I'm looking for people with macOS 10.12 or newer to try to
compile USD and see whether they are getting the same compile errors. To
start with, here's my machine info:

macOS 10.11.6 15G22010
Xcode 8.2.1 8C1002

Here's what I'd like people to test:

   1. Download USD version 19.11 from GitHub (yes, it has to be 19.11,
   because that's the version Blender uses):
   https://github.com/PixarAnimationStudios/USD/releases/tag/v19.11
   2. Extract the source code.
   3. mkdir /tmp/usd
   4. cd USD-19.11/build_scripts
   5. python2.7 build_usd.py --no-python /tmp/usd

IMPORTANT: Yes, you *have* to run the build script using Python 2.7. The
upstream authors didn't add any sort of Python 3 support to USD until
version 20.05.

Theoretically, the build script should try to use Xcode Clang to do the
compilation. Just to make sure, check the log file located in
/tmp/usd/build/USD-19.11/log.txt to verify that the build script is finding
AppleClang, and not any of the MacPorts Clangs that you might have
installed on your machine.

I'm looking for compile errors that look something like this:

USD-19.11/pxr/usd/lib/sdf/namespaceEdit.cpp:114:9: error: constructor for
> 'pxrInternal_v0_19__pxrReserved__::SdfNamespaceEdit_Namespace::_Node'
> must explicitly initialize the const member '_originalPath'
> _Node(const TfToken& name) : _key(name) { }
> ^
> USD-19.11/pxr/usd/lib/sdf/namespaceEdit.cpp:207:23: note: '_originalPath'
> declared here
> const SdfPath _originalPath;
>   ^
> USD-19.11/pxr/usd/lib/sdf/namespaceEdit.cpp:115:9: error: constructor for
> 'pxrInternal_v0_19__pxrReserved__::SdfNamespaceEdit_Namespace::_Node'
> must explicitly initialize the const member '_originalPath'
> _Node(const _TargetKey& key) : _key(key.key) { }
> ^
> USD-19.11/pxr/usd/lib/sdf/namespaceEdit.cpp:207:23: note: '_originalPath'
> declared here
> const SdfPath _originalPath;
>   ^
> USD-19.11/pxr/usd/lib/sdf/namespaceEdit.cpp:116:9: error: constructor for
> 'pxrInternal_v0_19__pxrReserved__::SdfNamespaceEdit_Namespace::_Node'
> must explicitly initialize the const member '_originalPath'
> _Node(const SdfPath& path) : _key(_GetKey(path)) { }
> ^
>

Regardless of build success or failure, please let me know which macOS and
Xcode/CLT you are using. If you are getting build failures, please also let
me know whether or not it is because you are seeing the same compile errors
that I'm seeing.

P.S.: My portfile won't actually be using the Python build script included
with USD, since MacPorts takes care of most of the actions performed by the
script; the portfile will be running CMake directly.

-- 
Jason Liu

Re: Apple ARM binary codesign issue

[Ryan Schmidt]

On Sep 24, 2020, at 08:51, Ruben Di Battista wrote:

> Ok, that's what I didn't know. I thought It was mandatory for Apple Store 
> software and not everything...  

Yup. As of macOS Big Sur 11 beta 6, released September 3, 2020, codesigning is 
mandatory for everything on ARM Macs. How to deal with that in MacPorts is what 
this thread is about. See 
https://developer.apple.com/documentation/macos-release-notes/macos-big-sur-11-universal-apps-beta-release-notes:

> New in macOS 11 on Apple silicon Mac computers, and starting in macOS Big Sur 
> 11 beta 6, the operating system enforces that any executable must be signed 
> before it’s allowed to run. There isn’t a specific identity requirement for 
> this signature: a simple ad-hoc signature is sufficient. This new behavior 
> doesn’t change the long-established policy that our users and developers can 
> run arbitrary code on their Macs, and is designed to simplify the execution 
> policies on Apple silicon Mac computers and enable the system to better 
> detect code modifications. This new policy doesn’t apply to translated x86 
> binaries running under Rosetta 2, nor does it apply to macOS 11 running on 
> Intel-based platforms.

Re: Thoughts on switching our archive compression method

[Ryan Schmidt]

On Sep 24, 2020, at 08:39, Lothar Haeger wrote:
> 
> Reading "xz" and "archive" in the same sentence reminds me of 
> https://www.nongnu.org/lzip/xz_inadequate.html
> 
> While it's easy to read the above as a rant of a disappointed competitor, he 
> seems to make some valid points about why lzma/xz should be avoided, 
> especially for long term archival purposes.

Thanks for pointing out that article. I wasn't aware of it or the deficiencies 
in the xz format that it alleges exist.

It sounds like lzip might be a better format, but macOS tar / libarchive 
doesn't know how to decompress lzip files, and lzip is GPL-2+ licensed so I 
would be wary about bundling it with MacPorts which is BSD licensed.

Re: Apple ARM binary codesign issue

[Ruben Di Battista]

Ok, that's what I didn't know. I thought It was mandatory for Apple Store
software and not everything... 

On Thu, 24 Sep 2020, 13:35 Ryan Schmidt,  wrote:

> On Sep 23, 2020, at 03:37, Ruben Di Battista wrote:
>
> > Can't be an easier choice to push globally  a linker switch, if it
> exists, to disable codesigning altogether for MP software?
>
> macOS 11 on ARM now require codesigning. Binaries that are not codesigned
> cannot be used at all.
>
>

Re: Thoughts on switching our archive compression method

[Lothar Haeger]

Reading "xz" and "archive" in the same sentence reminds me of 
https://www.nongnu.org/lzip/xz_inadequate.html 


While it's easy to read the above as a rant of a disappointed competitor, he 
seems to make some valid points about why lzma/xz should be avoided, especially 
for long term archival purposes.

> Am 24.09.2020 um 04:36 schrieb Ryan Schmidt :
> 
> On Sep 23, 2020, at 04:29, Dan Ports wrote:
> 
>> Also, bz2 is particularly slow at decompression, so even xz is likely
>> an improvement there.
> 
> As far as compression/decompression time goes, the normal gzip, bzip2 and xz 
> tools are single-threaded, as far as I know, so there is an opportunity to go 
> faster by using parallel processing.
> 
> For bzip2, you can install lbzip2 and configure MacPorts to use it; it then 
> compresses and decompresses in parallel, even bz2 files that were not created 
> with lbzip2. I have been using this on my Mac for years.
> 
> For xz, there's pixz. It sounds like xz files cannot be decompressed in 
> parallel unless they were created with pixz. We could install pixz on the 
> buildbot servers and use it to create the archives. This would improve 
> compress/decompress performance on the buildbot servers, and users could 
> choose to install pixz to get improved decompression performance or we can 
> consider bundling it with MacPorts base in the future; it is BSD licensed.
> 

Re: port "cask" -- installing prebuilt binaries

[Lothar Haeger]

> Am 15.08.2020 um 10:37 schrieb Lothar Haeger :
> 
>> The variant name is: +prebuilt
> 
> I like that name for its conciseness, lack of an underscore and correct 
> tense. :-)

So here's an example how this could look like in real life: 
https://github.com/macports/macports-ports/pull/8503 


Opinions?

gpg_verify 1.0 feedback

[Christopher Chavez]

The gpg_verify 1.0 PortGroup was added one year ago. I count 5 ports which are 
using it (4 of which were setup by the author of the PortGroup soon after it 
was introduced). However there are many more ports which could be using it. It 
would be nice if one day the MacPorts guide encouraged using this PortGroup. 
But I think there are some issues with this PortGroup's design that hinder its 
proliferation. Would it be worth improving?

I tried adding gpg_verify to gnupg2 recently. I eventually got it working, but 
I wasn't happy enough with the result to be convinced I should submit the 
result in a PR, especially since someone else maintained the port. I found 
gpg_verify much more tedious to add than compared to other PortGroups. I think 
the main reason is that it exposes what I consider "implementation details" the 
PortGroup should instead handle itself. One example is creating the separate 
post-checksum phase; I'm not aware when else a port would want to do PGP 
verification, so this is something that I think could be done in the PortGroup. 
Another is passing the complete distfile name to 
gpg_verify.verify_gpg_signature, when passing it without prepending 
${distpath}/ should be sufficient (I think `global distpath` would work, as 
done in the cargo_fetch 1.0 PortGroup). I also wonder if fetching the signature 
file could be handled by the PortGroup rather than requiring the portfile to 
add it to its distfiles and checksums.

Some other minor issues: The PGP verification step should print some 
acknowledgement when it is run with `port -v` and `port -d`; currently it only 
prints when there's an error (and without a precise reason specified). And 
rather than using a temporary .gnupg directory for each port, maybe a single 
permanent .gnupg directory somewhere in ${prefix}/var/macports should be used, 
so that keys don't have to be imported from scratch for each port build. I 
would also prefer the phrasing "PGP verification" rather than "GPG 
verification": PGP is the desired protocol, whereas GPG is just a common 
implementation of it (and it shouldn't matter whether it's the one used).

I really do think the PortGroup would be easier to use if all ports had to do 
is put the fingerprint(s) in the Portfile, rather than put the entire pubkey in 
the ports tree for each Portfile which might use it. If this means having to 
fetch the pubkeys, the issue with doing so (as the PortGroup mentions) is the 
intermittency of existing keyservers. I suggested using the high-availability 
keyserver keys.openpgp.org, but that was deemed too new for MacPorts to rely 
on. One year later, it is still around, is somewhat widely used—e.g. as the 
default keyserver in PGP suites—and claims 99.9998% uptime so far:

freenode #hagrid 2020-08-23 10:21 UTC  since recording uptime with 
uptimerobot on 2019-07-09, we had one downtime of roughly one minute, which was 
2019-11-11 23:24

I believe it would be good enough for MacPorts' purposes of retrieving a key 
for a given fingerprint. The problem with using it now is that GnuPG refuses to 
allow importing keys without owner information, which keys.openpgp.org requires 
key owners' permission to distribute: https://dev.gnupg.org/T4393. I have tried 
several keys for open source projects, none of which whose owner's info is 
distributed by keys.openpgp.org. MacPorts might be stuck with storing pubkeys, 
but maybe there is a better way of doing so.

I imagine there could be a much simpler interface for this using this 
PortGroup, making it more appealing to add it to ports. One possible idea:

gpg_verify.use_gpg_verification yes
gpg_verify.fingerprint hex_fingerprint_1 [hex_fingerprint2 …]
gpg_verify.signature_ext (sig|asc)

The PortGroup would handle fetching the .sig or .asc file for the distfile(s), 
and import the key using the fingerprint (if not already imported).

(Maybe the first line is redundant—using the PortGroup or its options at all 
would be what enables PGP verification.)

If custom signature filenames or URLs are frequently needed, then there might 
instead be options like `gpg_verify.signature_name sigfile` (fetching 
${master_sites}sigfile), or `gpg_verify.signature_url url` (fetching a complete 
URL to a signature file).

Maybe other package managers have ideas on how to handle PGP verification of 
upstream distfiles.

Christopher A. Chavez

Re: promote older systems support

[miniupnp]

+1

Le 07/09/2020 à 02:52, Ruben Di Battista a écrit :
> +1
>
> On Sun, 6 Sep 2020, 22:41 Ken Cunningham,
>  > wrote:
>
> there is considerable grousing on the homebrew forums about their
> ruthless policy for dropping support for systems more or less
> right after apple does, and some of those forum posts about this
> topic have many thousands of views.
>
> as you folks have put such effort into older systems support over
> the years, it might not be a waste of time to mention that support
> on the MacPorts main page .
>
> It is a primary differentiating feature.
>
> Ken
>