Re: port "cask" -- installing prebuilt binaries

2020-12-29 Thread Ken Cunningham
Another prime example for a cask port, but showing yet different issues, looks 
to be osxfuse.

osxfuse is required for a number of other ports, but the current osxfuse port 
is three years out of date because it’s no long open source, and users need to 
use the upstream installer.

For this one, we certainly don’t want to destroot it into DESTROOT/PREFIX and 
compress it into a tbz2 as per the MacPorts normal methods.

We would like our cask port to just download the DMG with the installer pkg on 
it, and then run the installer.

And then, somehow, tell base we have it installed to fulfill the needs of the 
ports that depend on it.

Ken



Re: port "cask" -- installing prebuilt binaries

2020-12-24 Thread Ken Cunningham



> On Dec 24, 2020, at 10:58 AM, Jason Liu  wrote:
> 
> A more efficient method that overrides activate/deactivate and does not 
> needlessly de everything would seem useful.
> 
> Since the dmg itself is a compressed archive, it would seem to make sense 
> that activate/deactivate would operate directly on the dmg, no? In other 
> words, the distfile is the archive, so recompressing of the destroot should 
> be skipped entirely. Instead, the dmg distfile should either be copied or 
> moved over to where the activate/deactivate archive usually lives.
> 
> -- 
> Jason Liu


Yes, we have challenges ahead of us sorting this out. For an example, here’s a 
little tiny Portfile that is supposed to install SageMath. It’s a prime 
candidate for a “cask” port, and does a fine job of exhibiting the problems to 
be faced.

Put it in a folder called “binary/sagemath-binary” to see it work, if you like:

===

# -*- coding: utf-8; mode: tcl; tab-width: 4; indent-tabs-mode: nil; 
c-basic-offset: 4 -*- vim:fenc=utf-8:ft=tcl:et:sw=4:ts=4:sts=4

PortSystem  1.0

namesagemath-binary
categories  binary math
platforms   darwin
license Permissive NoMirror
maintainers nomaintainer
description free open-source mathematics software system licensed under 
the GPL
long_description${description}. This port installs the binary build from 
upstream \
recommended for this version of MacOS.

homepagehttps://www.sagemath.org/index.html

universal_variant   no
archive_sites

# https://www.sagemath.org/download-mac.html
# not all have been listed here
master_siteshttp://www.cecm.sfu.ca/sage/osx/intel/ \
http://mirrors.mit.edu/sage/osx/intel/ \
https://mirror.marwan.ma/sage/osx/intel/ \

http://ftp.sun.ac.za/ftp/pub/mirrors/www.sagemath.org/osx/intel/ \
https://sagemath.mirror.ac.za/osx/intel/ \
http://ftp.leg.uct.ac.za/pub/packages/sage/osx/intel/ \
http://ftp.riken.jp/sagemath/osx/intel/ \
https://mirrors.tuna.tsinghua.edu.cn/sagemath/osx/intel/ \
http://mirror.aarnet.edu.au/pub/sage/osx/intel/ \
https://mirror.dogado.de/sage/osx/intel/

# 10.15 and up
if { ${os.platform} eq "darwin" && ${os.major} >= 19  } {
supported_archs x86_64 arm64
version 9.2
distnamesage-${version}-OSX_10.15.7-x86_64.app
use_dmg yes
checksums   rmd160  398838206e0011ce20a346e78d5db5278e1b2dec \
sha256  
fa6eb93368d4f7c220cbdd7a1483c1ef9d9b718b0f179c17c2acf14fb74f10c1 \
size1986453988
}

use_configure   no
build {}

destroot {
copy ${worksrcpath}/SageMath-${version}.app ${destroot}${applications_dir}
xinstall -m 0640 ${worksrcpath}/README.txt 
${destroot}${prefix}/share/${name}-${version}
}

===

Re: port "cask" -- installing prebuilt binaries

2020-12-24 Thread Jason Liu
>
> A more efficient method that overrides activate/deactivate and does not
> needlessly de everything would seem useful.
>

Since the dmg itself is a compressed archive, it would seem to make sense
that activate/deactivate would operate directly on the dmg, no? In other
words, the distfile *is* the archive, so recompressing of the destroot
should be skipped entirely. Instead, the dmg distfile should either be
copied or moved over to where the activate/deactivate archive usually lives.

-- 
Jason Liu


On Thu, Dec 24, 2020 at 5:27 AM Ken Cunningham <
ken.cunningham.web...@gmail.com> wrote:

> I've been building a few of these "cask" installers for my own use.
>
> I'm finding using the normal port activate/deactivate process quite
> inefficient for this...
>
> For example, SageMath is a 1.5GB dmg, and is about 4GB installed.
> Downloading the dmg, decompressing it and copying the contents into
> destroot, recompressing it, then activating it and decompressing it again
> takes just forever to do and seems wasteful on space as well.
>
> A more efficient method that overrides activate/deactivate and does not
> needlessly de everything would seem useful.
>
> Also a recommended method to avoid mirroring the distfiles ("NoMirror"
> license I presume) and not caching the archives (clearing archive_sites I
> suppose) are needed.
>
> Ken


Re: port "cask" -- installing prebuilt binaries

2020-12-24 Thread Ken Cunningham
I've been building a few of these "cask" installers for my own use.

I'm finding using the normal port activate/deactivate process quite inefficient 
for this...

For example, SageMath is a 1.5GB dmg, and is about 4GB installed. Downloading 
the dmg, decompressing it and copying the contents into destroot, recompressing 
it, then activating it and decompressing it again takes just forever to do and 
seems wasteful on space as well.

A more efficient method that overrides activate/deactivate and does not 
needlessly de everything would seem useful.

Also a recommended method to avoid mirroring the distfiles ("NoMirror" license 
I presume) and not caching the archives (clearing archive_sites I suppose) are 
needed.

Ken

Re: port "cask" -- installing prebuilt binaries

2020-12-13 Thread Mark Anderson
I like the idea of a separate thing like cask, (if only in labeling) but we
don't always need it/shouldn't use it exclusively.

Like maybe for restrictive non-free stuff like Chrome a different mode
makes sense, but with iTerm, I'm just trying to allow install on <10.15 and
knowing the iTerm guy, pretty soon, less than 11.0

Maybe just:
---> iTerm2 is only available via binary on this system, source is
available on 10.15+

---> Chrome is only available via binary

I'm not suggesting a Chrome port BTW, it's just the first thing I thought
of since I'm using it right this second to send this.

—Mark
___
Mark E. Anderson 
MacPorts Trac WikiPage 
GitHub Profile 



On Sun, Dec 13, 2020 at 4:42 PM Mark Anderson  wrote:

> I like the idea of a separate thing like cask, (if only in labeling) but
> we don't always need it/shouldn't use it exclusively.
>
> Like maybe for restrictive non-free stuff like Chrome a different mode
> makes sense, but with iTerm, I'm just trying to allow install on <10.15 and
> knowing the iTerm guy, pretty soon, less than 11.0
>
> Maybe just:
> ---> iTerm2 is only available via binary on this system, source is
> available on 10.15+
>
> ---> Chrome is only available via binary
>
> I'm not suggesting a Chrome port BTW, it's just the first thing I thought
> of since I'm using it right this second to send this.
>
>
> Thanks,
> —Mark
> ___
> Mark E. Anderson 
> Find me on LinkedIn 
>
>
> On Sun, Dec 13, 2020 at 4:30 PM Nils Breunese  wrote:
>
>> Ken Cunningham  wrote:
>>
>> So, I'm looking to install iTerm2 for old systems from binary as building
>> is becoming increasingly impossible - have we come to a consensus on any of
>> this?
>>
>> —Mark
>> ___
>> Mark E. Anderson > >
>> MacPorts Trac WikiPage 
>> GitHub Profile 
>>
>>
>>
>>
>>
>> I continue to believe that in general trying to shoehorn "cask" binary
>> installs as some variant of a port that is generally meant to build from
>> source is a recipe for nothing but endless trouble. Homebrew has a
>> completely different subsystem for cask installs that makes it really clear
>> what you are getting, and this is very desirable, I agree.
>>
>>
>> IMHO binary-only install port should have some clearly recognizable port
>> name that does not cause confusion about what it is, and does not obscure
>> or trample a port's existing variants (which a "prebuilt" or other similar
>> variant name would do, if there were other variants). We have port name
>> distinctions for a great many ports in MacPorts now (the perl, python, php,
>> etc, etc, etc port families, for example). Having a naming family for
>> binary-only ports is No Big Deal.
>>
>>
>> Chris has suggested a category inclusion, which is pure and uses macports
>> unique functionality, but IMHO is unrecognizable for 99.% of users who
>> would never notice that a given port is added to a certain category or
>> subcategory.
>>
>>
>> But we should resolve this, as many people want it, whatever is decided
>> by the managers, who so far have expressed no opinion, ergo it is
>> unresolved.
>>
>>
>> Why is having binary ports without a special indicator a problem?
>> MacPorts has already has ports that use upstream binaries, mostly for ports
>> that are either impossible to build from source (Google Cloud SDK source is
>> not available AFAIK for instance), or very hard/time-consuming (OpenJDK for
>> instance) to build. I maintain a couple of ports like these and they don’t
>> have a specific name or variant or anything indicating that they use
>> upstream binaries and I don’t see a problem with that really. If someone
>> would ever decide it’s a good idea to switch to building OpenJDK from
>> source via a Portfile, that could just be transparently done without any
>> users having to switch to a different port name or variant, and that seems
>> fine to me.
>>
>> It might even make sense for some ports (like iTerm2 perhaps?) to build
>> from source on macOS versions for which that is feasible, and use an
>> upstream binary on OS versions for which it isn’t.
>>
>> Nils.
>>
>


Re: port "cask" -- installing prebuilt binaries

2020-12-13 Thread Nils Breunese
Ken Cunningham  wrote:

>> So, I'm looking to install iTerm2 for old systems from binary as building
>> is becoming increasingly impossible - have we come to a consensus on any of
>> this?
>> 
>> —Mark
>> ___
>> Mark E. Anderson 
>> MacPorts Trac WikiPage 
>> GitHub Profile 
>> 
>> 
>> 
> 
> I continue to believe that in general trying to shoehorn "cask" binary 
> installs as some variant of a port that is generally meant to build from 
> source is a recipe for nothing but endless trouble. Homebrew has a completely 
> different subsystem for cask installs that makes it really clear what you are 
> getting, and this is very desirable, I agree.
> 
> 
> 
> IMHO binary-only install port should have some clearly recognizable port name 
> that does not cause confusion about what it is, and does not obscure or 
> trample a port's existing variants (which a "prebuilt" or other similar 
> variant name would do, if there were other variants). We have port name 
> distinctions for a great many ports in MacPorts now (the perl, python, php, 
> etc, etc, etc port families, for example). Having a naming family for 
> binary-only ports is No Big Deal.
> 
> 
> 
> Chris has suggested a category inclusion, which is pure and uses macports 
> unique functionality, but IMHO is unrecognizable for 99.% of users who 
> would never notice that a given port is added to a certain category or 
> subcategory.
> 
> 
> 
> But we should resolve this, as many people want it, whatever is decided by 
> the managers, who so far have expressed no opinion, ergo it is unresolved.
> 

Why is having binary ports without a special indicator a problem? MacPorts has 
already has ports that use upstream binaries, mostly for ports that are either 
impossible to build from source (Google Cloud SDK source is not available AFAIK 
for instance), or very hard/time-consuming (OpenJDK for instance) to build. I 
maintain a couple of ports like these and they don’t have a specific name or 
variant or anything indicating that they use upstream binaries and I don’t see 
a problem with that really. If someone would ever decide it’s a good idea to 
switch to building OpenJDK from source via a Portfile, that could just be 
transparently done without any users having to switch to a different port name 
or variant, and that seems fine to me.

It might even make sense for some ports (like iTerm2 perhaps?) to build from 
source on macOS versions for which that is feasible, and use an upstream binary 
on OS versions for which it isn’t.

Nils.

Re: port "cask" -- installing prebuilt binaries

2020-12-13 Thread Mark Anderson
So yeah - I think that a name and separate port is a good idea. I'm also on
board for the category for the 0.0001% of us that would use it.

maybe category: binary and binary-PortName like we do with like
py38-something


—Mark
___
Mark E. Anderson 
MacPorts Trac WikiPage 
GitHub Profile 



On Sun, Dec 13, 2020 at 3:15 PM Ken Cunningham <
ken.cunningham.web...@gmail.com> wrote:

> So, I'm looking to install iTerm2 for old systems from binary as building
> is becoming increasingly impossible - have we come to a consensus on any of
> this?
>
> —Mark
> ___
> Mark E. Anderson  >
> MacPorts Trac WikiPage 
> GitHub Profile 
>
>
>
>
>
> I continue to believe that in general trying to shoehorn "cask" binary
> installs as some variant of a port that is generally meant to build from
> source is a recipe for nothing but endless trouble. Homebrew has a
> completely different subsystem for cask installs that makes it really clear
> what you are getting, and this is very desirable, I agree.
>
>
> IMHO binary-only install port should have some clearly recognizable port
> name that does not cause confusion about what it is, and does not obscure
> or trample a port's existing variants (which a "prebuilt" or other similar
> variant name would do, if there were other variants). We have port name
> distinctions for a great many ports in MacPorts now (the perl, python, php,
> etc, etc, etc port families, for example). Having a naming family for
> binary-only ports is No Big Deal.
>
>
> Chris has suggested a category inclusion, which is pure and uses macports
> unique functionality, but IMHO is unrecognizable for 99.% of users who
> would never notice that a given port is added to a certain category or
> subcategory.
>
>
> But we should resolve this, as many people want it, whatever is decided by
> the managers, who so far have expressed no opinion, ergo it is unresolved.
>
>
> K
>
>
>
>
>
>
>


Re: port "cask" -- installing prebuilt binaries

2020-12-13 Thread Ken Cunningham

So, I'm looking to install iTerm2 for old systems from binary as building
is becoming increasingly impossible - have we come to a consensus on any of
this?

—Mark
___
Mark E. Anderson https://lists.macports.org/mailman/listinfo/macports-dev>>
MacPorts Trac WikiPage >
GitHub Profile >





I continue to believe that in general trying to shoehorn "cask" binary 
installs as some variant of a port that is generally meant to build from 
source is a recipe for nothing but endless trouble. Homebrew has a 
completely different subsystem for cask installs that makes it really 
clear what you are getting, and this is very desirable, I agree.



IMHO binary-only install port should have some clearly recognizable port 
name that does not cause confusion about what it is, and does not 
obscure or trample a port's existing variants (which a "prebuilt" or 
other similar variant name would do, if there were other variants). We 
have port name distinctions for a great many ports in MacPorts now (the 
perl, python, php, etc, etc, etc port families, for example). Having a 
naming family for binary-only ports is No Big Deal.



Chris has suggested a category inclusion, which is pure and uses 
macports unique functionality, but IMHO is unrecognizable for 99.% 
of users who would never notice that a given port is added to a certain 
category or subcategory.



But we should resolve this, as many people want it, whatever is decided 
by the managers, who so far have expressed no opinion, ergo it is 
unresolved.



K








Re: port "cask" -- installing prebuilt binaries

2020-12-13 Thread Mark Anderson
So, I'm looking to install iTerm2 for old systems from binary as building
is becoming increasingly impossible - have we come to a consensus on any of
this?

—Mark
___
Mark E. Anderson 
MacPorts Trac WikiPage 
GitHub Profile 



On Thu, Sep 24, 2020 at 6:55 AM Lothar Haeger  wrote:

>
> Am 15.08.2020 um 10:37 schrieb Lothar Haeger :
>
> The variant name is: +prebuilt
>
>
> I like that name for its conciseness, lack of an underscore and correct
> tense. :-)
>
>
> So here's an example how this could look like in real life:
> https://github.com/macports/macports-ports/pull/8503
>
> Opinions?
>


Re: Supporting installing arbitrary port versions (was: Re: port "cask" -- installing prebuilt binaries)

2020-10-03 Thread Jason Liu
> Just looking at your idea to distribute all portfile versions, let's start
> with the fact that portfile syntax has evolved over time.
>

This is where portfile syntax itself can, and probably should, be
versioned. Maybe by incrementing PortSystem? i.e. PortSystem 1.3, 1.4, 2.0,
etc. Similar to how the HTML standard specification's version number has
changed over time, from HTML 2 all the way to the current HTML 5.

If MacPorts allowed the user to pick an arbitrary old Portfile to install,
> it is extremely likely that the user would experience an error.
>

>
If you are suggesting that we should not only distribute all old portfiles
> but also keep them all updated as needed, that would be a ludicrous amount
> of work that nobody would be willing to do.
>

This is where being able to restrict dependencies to specific version
ranges would be incredibly useful. If you could say that a portfile's
dependency requires a version >= 1.3.0 && <= 1.5.2, then you would not need
to update that old portfile once the dependency's portfile gets updated to
a version beyond 1.5.2.

Well you spoke of keeping all Portfile versions. So perhaps you're
> suggesting the user should be able to select from any past version of the
> port. That's slightly more reasonable than allowing the user to request any
> version of the software regardless of whether that version has ever been in
> the portfile, but still completely impossible and unsupportable as far as I
> can see so far.
>

Other PMSes like APT and YUM do exactly that. They keep all versions of
their package's spec files. How they do this is that their package
repositories have a different architecture than how MacPorts is currently
laid out.

[Note: What follows below is a fairly lengthy description of how another
package management system works. If you have no interest in such things,
here is probably a good place to stop reading.]

Allow me to take the Blender package in the Debian repo as an example. Here
is the entry for Debian Testing, which is the version of Debian that I use
(Debian Stable tends to be a bit too outdated for my taste):

https://packages.debian.org/bullseye/blender

If you look at the info on the right side of that page, one of the sections
is "Download Source Package blender:". There are three items listed:

   - [blender_2.83.5+dfsg-2.dsc]
   
   - [blender_2.83.5+dfsg.orig.tar.xz]
   

   - [blender_2.83.5+dfsg-2.debian.tar.xz]
   


Before I go into further detail about the three items above, another
interesting thing to take a quick look at is the actual folder for the
Blender package on the Debian repo:

http://deb.debian.org/debian/pool/main/b/blender/

There you will see the three files I listed above, and also many .deb
files. Those .deb files are basically pre-compiled archives of the package,
which have been built for individual architectures e.g. amd64, i386, arm64,
etc. Now, let me go through each of the three items.

The first item, the .dsc file, is a Debian source control
 file. It is a text file that contains many
fields which would be quite familiar to anyone familiar with our portfiles.

The second item, the .orig.tar.xz file, is a copy of the upstream source
code. It's basically the equivalent of a MacPorts distfile, like what's
stored on https://distfiles.macports.org.

The third item, the .debian.tar.xz file, is the one that is the most
interesting. If we took the entire contents of this folder

https://github.com/macports/macports-ports/blob/master/graphics/blender/

and zipped it up into an archive, that would be the MacPorts equivalent to
the third item. If you extract the third item, you will find a debian/
folder that contains all of the spec files for the APT package of Blender.
In general, if you were to concatenate the two files "control" and "rules"
together into a single file, it would look a lot like our notion of a
portfile. The file called "control" will already look familiar: in fact,
this file is used to generate the .dsc file when the package gets built.
The "rules" file contains the equivalent of what we use in portfiles to
control the various phases of the MacPorts build process, e.g. pre-fetch,
post-patch, depends_lib, configure.args, post-destroot, etc.

Now, here is how an APT repository "keeps all Portfile versions". If you go
back and look at actual folder for the Blender package on the Debian repo:

http://deb.debian.org/debian/pool/main/b/blender/

you will see that there are multiple versions of Blender in that folder.
For each version, the three items I discussed are present, along with some
pre-compiled .deb archives. One of the most important ways in which an APT
or YUM repository differs from a MacPorts ports tree is this: what 

Re: Supporting installing arbitrary port versions (was: Re: port "cask" -- installing prebuilt binaries)

2020-10-03 Thread Ryan Schmidt



On Oct 3, 2020, at 06:05, Lothar Haeger wrote:

>> It's because, besides being an unimaginably large amount of work in 
>> rearranging our code to do it, I have absolutely no idea how it would be 
>> accomplished without providing the user with unlimited opportunities to 
>> create broken combinations of port versions, which would generate an 
>> unlimited number of bug reports that we would then need to respond to, and 
>> my goal in MacPorts is to reduce, not increase, the likelihood that users 
>> would find something broken or need to contact us to help troubleshoot it.
>> 
>> If you have an idea for how it could be done without such problems arising, 
>> please open a new topic and describe it and we can talk through a few 
>> scenarios and see if it works.
> 
> I do have some ideas (not mine, just looking at how it works elsewhere), but 
> not the time to drive something of this size, to be honest. Basically, we'd 
> need to 
> - keep and distribute all Portfile versions (including patches and files), 
> not just the lastest
> - add required versions to dependencies (can be version ranges e.g. >=1.2.3 
> or 2.* or 1.2.3-4.5.6)
> - add support to install individual ports in into separate folders or give 
> them individual names on install time (much like with the perl versions), so 
> multiple versions of a port can be installed in parallel.
> - add command line parameters to the port command so users can optionally 
> specify a version to install (defaults to latest) and a pre/post fix to 
> install location or file names or both (depends on how stuff gets 
> implemented, of course)
> - add support for dependency resolution including version information. So if 
> two ports have different, non overlapping version requirements for a 
> dependency, that dependency gets installed twice. Making sure each port then 
> actually uses its matching version is probably one of the trickier parts here.
> 
> Implementing all this would be a major project and I totally understand that 
> it's nothing to jump at without considering all the consequences and outside 
> factors. 

It would be a major project if it were clear how it could possibly work and it 
were just a matter of doing it. But so far I have no idea how it could work.

Just looking at your idea to distribute all portfile versions, let's start with 
the fact that portfile syntax has evolved over time. A portfile from ten years 
ago can't even necessarily be parsed by today's MacPorts ("livecheck.type" used 
to be called "livecheck.check"; there used to be "cd" and "suffix" procedures 
that ports used to use) or it may behave differently than it did then 
("platform" blocks used to behave like variants and their code was evaluated at 
the end of the portfile rather than where it's declared; the way that the 
arguments of startupitems and configure.env/build.env/etc need to be quoted was 
changed).

Libraries change over time. Sometimes when a library changes it means that the 
other ports that use that library need to be patched. Obviously those patches 
were not in the old versions of the portfile that predate the new library, and 
patches for the current version of the software couldn't necessarily be used on 
old versions without being rewritten.

Compilers and operating systems evolve. Xcode 12, for example, prohibits 
implicitly declared functions. We're adding patches to ports as fast as we can 
to fix these problems, but old versions of the portfile would not have those 
patches.

If MacPorts allowed the user to pick an arbitrary old Portfile to install, it 
is extremely likely that the user would experience an error.

If you are suggesting that we should not only distribute all old portfiles but 
also keep them all updated as needed, that would be a ludicrous amount of work 
that nobody would be willing to do.


>> I'm speaking of the user being able to specify an arbitrary version.
> 
> So do I.

Well you spoke of keeping all Portfile versions. So perhaps you're suggesting 
the user should be able to select from any past version of the port. That's 
slightly more reasonable than allowing the user to request any version of the 
software regardless of whether that version has ever been in the portfile, but 
still completely impossible and unsupportable as far as I can see so far.


>> I consider it a feature, not a bug, that we offer multiple ports for 
>> different versions of perl, php, python, ruby, gcc, clang. It enables the 
>> user to install multiple versions of those languages that can all be active 
>> and available at the same time. If the user has one script that works with 
>> python 3.8 and another that requires python 3.6, no problem. If they have 
>> one web site that works with php 7.4 and another that needs php 7.2, no 
>> problem. If we only had a single python or php port that only let the user 
>> choose a single version to install at a time, that would not be possible.
> 
> I totally agree that this is a great feature. 

Re: port "cask" -- installing prebuilt binaries

2020-10-03 Thread Ken Cunningham
Ok.

So contenders for the metadata identifying non-buildable binary-only ports we 
have:

name suffix (like the hs- ports had)
category only
variant as identifier

We have spent far too long on this nonsense. Some admin pick one, then lets 
move on.

K



Re: port "cask" -- installing prebuilt binaries

2020-10-03 Thread Lothar Haeger


> Am 03.10.2020 um 03:00 schrieb Ryan Schmidt :
> 
> Lothar, I agree with most of your reasoning for why a variant is a reasonable 
> choice for indicating a prebuilt binary. I am less concerned with how to 
> indicate prebuilt status than I am with whether we should do it at all.

Interesting, back to the very beginning… 

I kind of already accepted that an indicator should be found, not questioning 
is usefulness anymore. Personally I do not care much about it, but obviously 
others do and in general additional information is considered a good thing, so 
why not add it? All I really care about in this context is, that however we 
implement it, no existing or easily conceivable future feature will suffer from 
it. 



Re: port "cask" -- installing prebuilt binaries

2020-10-03 Thread Lothar Haeger
Ryan,

> Am 03.10.2020 um 03:00 schrieb Ryan Schmidt :
> 
>> Just MacPorts did not implement it yet and when the necessity arose a 
>> seemingly simple workaround was chosen instead of solving the underlying 
>> problem.
> 
> As for MacPorts, it's not that we haven't implemented it because we're lazy. 

my apologies for expressing this poorly enough to be misunderstood. I did not 
mean to suggest laziness was a reason to chose one way over another, neither in 
this case nor in general. 

> It's because, besides being an unimaginably large amount of work in 
> rearranging our code to do it, I have absolutely no idea how it would be 
> accomplished without providing the user with unlimited opportunities to 
> create broken combinations of port versions, which would generate an 
> unlimited number of bug reports that we would then need to respond to, and my 
> goal in MacPorts is to reduce, not increase, the likelihood that users would 
> find something broken or need to contact us to help troubleshoot it.
> 
> If you have an idea for how it could be done without such problems arising, 
> please open a new topic and describe it and we can talk through a few 
> scenarios and see if it works.

I do have some ideas (not mine, just looking at how it works elsewhere), but 
not the time to drive something of this size, to be honest. Basically, we'd 
need to 
- keep and distribute all Portfile versions (including patches and files), not 
just the lastest
- add required versions to dependencies (can be version ranges e.g. >=1.2.3 or 
2.* or 1.2.3-4.5.6)
- add support to install individual ports in into separate folders or give them 
individual names on install time (much like with the perl versions), so 
multiple versions of a port can be installed in parallel.
- add command line parameters to the port command so users can optionally 
specify a version to install (defaults to latest) and a pre/post fix to install 
location or file names or both (depends on how stuff gets implemented, of 
course)
- add support for dependency resolution including version information. So if 
two ports have different, non overlapping version requirements for a 
dependency, that dependency gets installed twice. Making sure each port then 
actually uses its matching version is probably one of the trickier parts here.

Implementing all this would be a major project and I totally understand that 
it's nothing to jump at without considering all the consequences and outside 
factors. 

> I'm speaking of the user being able to specify an arbitrary version.

So do I.

> I consider it a feature, not a bug, that we offer multiple ports for 
> different versions of perl, php, python, ruby, gcc, clang. It enables the 
> user to install multiple versions of those languages that can all be active 
> and available at the same time. If the user has one script that works with 
> python 3.8 and another that requires python 3.6, no problem. If they have one 
> web site that works with php 7.4 and another that needs php 7.2, no problem. 
> If we only had a single python or php port that only let the user choose a 
> single version to install at a time, that would not be possible.

I totally agree that this is a great feature. Only it's limited to a few 
versions of a few ports due to the way it's being implemented. Supporting this 
for more ports and version does not scale well, twice as many ports/versions 
cause twice as much work. With thousands of ports in the tree the current 
approach will always be limited to a handful of ports. If this was implemented 
as general features to a) install any port version and b) install several 
instances of any port, we'd get the same for each and every port/version out of 
the box.

Re: port "cask" -- installing prebuilt binaries

2020-10-02 Thread Ryan Schmidt
Lothar, I agree with most of your reasoning for why a variant is a reasonable 
choice for indicating a prebuilt binary. I am less concerned with how to 
indicate prebuilt status than I am with whether we should do it at all.

For ports that are not available any way other than prebuilt, I'm not sure 
what's gained by adding a default variant that can't be turned off. It tells 
the user it's a prebuilt binary, but will they really care?

As for the scenario where we want to compile on some OS versions and offer a 
prebuilt binary on others, do we have specific examples of ports where that 
would be used? I forget but I would hope this situation arises very 
infrequently. I suppose osxfuse might be one, where we can compile our own on < 
10.11 but have to use a prebuilt binary on 10.11 and later because it is a 
kernel extension and Apple requires that those be signed by a real developer on 
10.11 and later. But even then: why not just make the port compile on < 10.11 
and use a prebuilt binary on >= 10.11, which is kind of what the port should be 
doing now except that it has bugs? Why does that need to be communicated to (or 
selectable by) the user via a variant or any other means?


To your other point:

On Oct 2, 2020, at 17:42, Lothar Haeger wrote:

> MacPorts already uses port names to store non-naming information in one 
> common use case: when multiple versions of a port should be maintained in 
> parallel. Think of perl: p5, p5.26, p5.28, p5.30 and the gazillions of 
> derived port variants. Basically every single perl extension exists in 
> multiple versions to make all versions of perl happy. Same for python and 
> some others.
> Instead of creating separate copies of perl for each version, it would've 
> probably been smarter to fix the limitation in MacPorts that made this 
> workaround necessary, i.e. its inability to maintain and install all but the 
> latest version of a port. RPM can do it, DEB can do it, MSI can do it, 
> nothing unusual in the grand scheme of package managers in general to be able 
> to choose a specific version to install. Just MacPorts did not implement it 
> yet and when the necessity arose a seemingly simple workaround was chosen 
> instead of solving the underlying problem.

I have no familiarity with rpm, deb, msi, or other package managers so I cannot 
say whether or how they allow the user to select which version of a package to 
install. As for MacPorts, it's not that we haven't implemented it because we're 
lazy. It's because, besides being an unimaginably large amount of work in 
rearranging our code to do it, I have absolutely no idea how it would be 
accomplished without providing the user with unlimited opportunities to create 
broken combinations of port versions, which would generate an unlimited number 
of bug reports that we would then need to respond to, and my goal in MacPorts 
is to reduce, not increase, the likelihood that users would find something 
broken or need to contact us to help troubleshoot it.

If you have an idea for how it could be done without such problems arising, 
please open a new topic and describe it and we can talk through a few scenarios 
and see if it works.

I'm speaking of the user being able to specify an arbitrary version. If you're 
instead thinking that the port maintainer would specify a list of valid 
versions or something, that might be more feasible, but still not without some 
of the above problems.

I consider it a feature, not a bug, that we offer multiple ports for different 
versions of perl, php, python, ruby, gcc, clang. It enables the user to install 
multiple versions of those languages that can all be active and available at 
the same time. If the user has one script that works with python 3.8 and 
another that requires python 3.6, no problem. If they have one web site that 
works with php 7.4 and another that needs php 7.2, no problem. If we only had a 
single python or php port that only let the user choose a single version to 
install at a time, that would not be possible.




Re: port "cask" -- installing prebuilt binaries

2020-10-02 Thread Lothar Haeger



> Am 02.10.2020 um 17:02 schrieb Ken Cunningham 
> :
> 
> So this issue seems still active, but without a decided plan here, people are 
> just starting to PR their favourite approach, which is of course exactly what 
> we are tring to avoid. 

Ken, it's not "people", it's just me who provided an example to revive this 
discussion after weeks of silence, and I even announced it and asked for 
feedback in this very thread more than a week ago. 

> The idea of identifying them by a variant suffers from the fact that these 
> are obviously not variants, they are immutable downloaded binaries without a 
> non-prebuilt version. So that is just confusing. 

That's your personal reception, but what you find confusing may be perfectly 
logical to others, certainly to me. Here's another attempt to explain why I 
think MacPorts "variants" feature would be a good fit for the purpose:

- each "traditional" variant of a port results in a different binary (compare 
their checksums...) with possibly slightly different functionality. All 
traditional variants of a port are still considered the same 
application/product. This would as well apply to a prebuilt binary 
redistributed by MacPorts.
- traditional variants may or may not be combined freely, in many cases it's 
actually either-or. MacPorts supports that via the "conflicts" keyword in 
variants definitions. "prebuilt" variants would obviously conflict with other 
variants, nothing unusual here (the current port tree has ~2200 variants out of 
~6600 declaring conflicts).
- all ports with a prebuilt variant can easily be identified and even 
uninstalled with a single command (try the latter with a prefix...)
- prebuilt and compiled-from-source variants can coexist for the same port 
(both would obviously conflict and could no be combined, though). 
- switching from prebuilt to compiled and back would be just like switching 
between any other variants.
- variants can be tied to OS versions, so a single port could supply a prebuilt 
binary for some OS versions and be compiled from source for others - a existing 
real-world use case
- a "prebuilt" variant can easily be added to existing binary ports (with a 
prefix, affected ports would have to be renamed with all consequences arising 
from that)
- an existing "prebuilt" variant can easily be removed if an updated port 
version finally allows compiling from source. 

> There are very rare ports like ghc where a downloaded binary rebuilds the 
> macports version. In that rare case only, a prebuilt variant sorta makes some 
> sense, but our previous naming scheme where we called it "ghc-bootstrap" was 
> probably more appropriate, TBH.

I generally do not think it's a good idea to mix different kinds of information 
in a single fields of function. A "name" is a name and not a combination of 
name and something else. "Something else" should go into the "something else" 
field or function instead. That's a very low-level design principle that has 
proven beneficial in almost every context. Whenever it get's ignored, sooner or 
later funny side effects come up that could've been avoided if a proper 
solution was implemented instead of some (at the time) seemingly easier 
workaround.
MacPorts already uses port names to store non-naming information in one common 
use case: when multiple versions of a port should be maintained in parallel. 
Think of perl: p5, p5.26, p5.28, p5.30 and the gazillions of derived port 
variants. Basically every single perl extension exists in multiple versions to 
make all versions of perl happy. Same for python and some others.
Instead of creating separate copies of perl for each version, it would've 
probably been smarter to fix the limitation in MacPorts that made this 
workaround necessary, i.e. its inability to maintain and install all but the 
latest version of a port. RPM can do it, DEB can do it, MSI can do it, nothing 
unusual in the grand scheme of package managers in general to be able to choose 
a specific version to install. Just MacPorts did not implement it yet and when 
the necessity arose a seemingly simple workaround was chosen instead of solving 
the underlying problem. I'm pretty sure something similar awaits us when a 
category-or-variant-like piece of information is stored as part of a port name.

Some complications that come to mind:
- to provide the same app prebuilt and compiled-from-source in parallel would 
require two differently named ports.
- Switching between them would require uninstalling one and installing another.
- Some way to prevent a user installing both of them must be implemented
- unlike other related ports, both yould not eben start with the application 
name and would not be listed next to each other in alphabetical port listings
- Those two ports may end up with different maintainers and latest availbale 
versions. Users might have to decide if they rather install the outdated but 
compiled version or a newer prebuilt port. 
- Which of those two ports 

Re: port "cask" -- installing prebuilt binaries

2020-10-02 Thread Ken Cunningham


> On Oct 2, 2020, at 8:02 AM, Ken Cunningham  
> wrote:
> 
> Chris says put them in a category. Logical, but I don't know if most users 
> would note that when trying to install them, as it is not perhaps obvious 
> enough these are different beasts.

Also, it it is decided to go ahead with the idea of identifying these using 
only a category, I’m not certain how to handle the situation where the port can 
build on newer systems, but needs a prebuilt binary on older systems, and that 
line might move over time as newer systems become older and need the binary 
instead of building.

K

Re: port "cask" -- installing prebuilt binaries

2020-10-02 Thread Ken Cunningham
So this issue seems still active, but without a decided plan here, people are 
just starting to PR their favourite approach, which is of course exactly what 
we are tring to avoid. 

The idea of identifying them by a variant suffers from the fact that these are 
obviously not variants, they are immutable downloaded binaries without a 
non-prebuilt version. So that is just confusing. 

There are very rare ports like ghc where a downloaded binary rebuilds the 
macports version. In that rare case only, a prebuilt variant sorta makes some 
sense, but our previous naming scheme where we called it "ghc-bootstrap" was 
probably more appropriate, TBH.

Chris says put them in a category. Logical, but I don't know if most users 
would note that when trying to install them, as it is not perhaps obvious 
enough these are different beasts.

Really obvious is to tag a small suffix on the name, which nobody could ever 
miss. Then everyone will know immediately what they are dealing with, including 
those of us who resolve tickets. That's why I like that idea.

So lets get some decision, whatever it is, before we have to clean up a big 
mess as people Bogart ahead with different plans.

Ken

Re: port "cask" -- installing prebuilt binaries

2020-09-24 Thread Lothar Haeger

> Am 15.08.2020 um 10:37 schrieb Lothar Haeger :
> 
>> The variant name is: +prebuilt
> 
> I like that name for its conciseness, lack of an underscore and correct 
> tense. :-)

So here's an example how this could look like in real life: 
https://github.com/macports/macports-ports/pull/8503 


Opinions?

Re: port "cask" -- installing prebuilt binaries

2020-08-15 Thread Ken Cunningham
Doesn't the port `ghc` do exactly this? 
https://github.com/macports/macports-ports/blob/master/lang/ghc/Portfile#L31-L33
 The variant name is: +prebuilt

throwaway bootstrap compiler only, totally different issue




Re: port "cask" -- installing prebuilt binaries

2020-08-15 Thread Lothar Haeger

> Am 15.08.2020 um 09:47 schrieb Eric F (iEFdev) :
> 
> Doesn't the port `ghc` do exactly this? 
> https://github.com/macports/macports-ports/blob/master/lang/ghc/Portfile#L31-L33
>  
> 

Yes, looks like it does. Three more seem to work that way, too, `mate-icons` 
even in reverse mode:

`port echo variant:rebuild variant:prebuilt
mate-icons
cabal
ghc
stack`

> The variant name is: +prebuilt

I like that name for its conciseness, lack of an underscore and correct tense. 
:-)

> Besides that… Haven't followed this discussion very closely, but I think I'm 
> more into what
> [I think] someone said earlier - that MP should try to build everything, or 
> try to fix it so it can.

Yes, I think we all agree on that. This thread is about those cases where it's 
not feasible to build from source, yet providing the port is desirable.

Re: port "cask" -- installing prebuilt binaries

2020-08-15 Thread Eric F (iEFdev)

On 8/15/20 8:48 , Lothar Haeger wrote:
>> Am 15.08.2020 um 01:11 schrieb Ken Cunningham 
>> :
>>
>> so -- what would a user typing
> Obviously that variant would conflict with all other variants, much like 
> +ruby21 conflicts with +ruby22. So the user would type
>
> sudo port -v install macvim +prebuild_binary

Doesn't the port `ghc` do exactly this? 
https://github.com/macports/macports-ports/blob/master/lang/ghc/Portfile#L31-L33
The variant name is: +prebuilt

—

Besides that… Haven't followed this discussion very closely, but I think I'm 
more into what
[I think] someone said earlier - that MP should try to build everything, or try 
to fix it so it can.
/2¢

 · Eric


Re: port "cask" -- installing prebuilt binaries

2020-08-15 Thread Ken Cunningham



On 2020-08-14 11:48 p.m., Lothar Haeger wrote:



Am 15.08.2020 um 01:11 schrieb Ken Cunningham :

so -- what would a user typing

Obviously that variant would conflict with all other variants, much like 
+ruby21 conflicts with +ruby22. So the user would type

sudo port -v install macvim +prebuild_binary


Well, I guess we've all made our points, as much as they are going to be 
made.


For me, it is a needless layer of extra confusion to attempt to list a 
preinstalled binary as a build variant for a port for something that is 
in no way a build variant and doesn't involve building the port. But 
apparently to some, that seems quite logical.


"Cask" ports, if allowed, would have almost nothing to do with the usual 
ports, have different downloads, different installation procedures, 
different deactivate procedures, different updating schedules, different 
os version support, and basically -- are totally different ports.


Having them as variants of other ports is just trouble and makes for 
extremely messy portfiles. Nothing mentioned here would be leading me to 
be changing that opinion, but then I won't be calling this either. If 
such ports are ever accepted into MacPorts, which is also not up to me.


At least if they do start to come in, it will have been with a modicum 
of forethought.


Ken





Re: port "cask" -- installing prebuilt binaries

2020-08-15 Thread Lothar Haeger



> Am 15.08.2020 um 01:11 schrieb Ken Cunningham 
> :
> 
> so -- what would a user typing

Obviously that variant would conflict with all other variants, much like 
+ruby21 conflicts with +ruby22. So the user would type

sudo port -v install macvim +prebuild_binary




Re: port "cask" -- installing prebuilt binaries

2020-08-14 Thread Arno Hautala


> On 14 Aug 2020, at 19:11, Ken Cunningham  
> wrote:
> 
> so -- what would a user typing
> 
> sudo port -v install macvim +prebuild_binary -huge +big +ruby25 +tcl +xim
> 
> be expected to get? It's a total shambles.
> 

They’d see the same thing they see if they tried to run:

> sudo port install macvim +ruby25 +ruby24
> 
> Error: MacVim: Variant ruby24 conflicts with ruby25
> Error: Unable to open port: Error evaluating variants

Or in this case, maybe even a more specific error message to call out that the 
“prebuild_binary” variant is not compatible with any other variants.

> sudo port install macvim +prebuild_binary -huge +big +ruby25 +tcl +xim
> 
> Error: MacVim: Variant prebuild_binary conflicts with all other variants
> Error: Unable to open port: Error evaluating variants

I fail to see how variant isn’t _precisely_ the correct way to handle this.

I’d imagine anything with this variant would look something like:

> $ port variants macvim
> MacVim has the variants:
>prebuild_binary: Install a pre-built binary supplied by the original 
> developer
>  * conflicts with all other available variants
>big: Build big feature set
>  * conflicts with huge
>cscope: Enable source code browsing with cscope
> [+]huge: Build huge feature set
>  * conflicts with big
> ...


-- 
arno  s  hautala/-|   a...@alum.wpi.edu

pgp b2c9d448




Re: port "cask" -- installing prebuilt binaries

2020-08-14 Thread Ken Cunningham

I mean, you can maybe describe the “feature set” of the prebuilt binary in
the variant description and forbid all the other variants… Can’t you?


No doubt, with sufficient effort, we could take something inherently 
simple and make a complicated and undecipherable version of it.




PS: Maybe “prebuilt_binary” is not the right wording since most of the
ports can already be installed as “prebuilt_binary”. Maybe it should be
something like “bin_only” ?


IMHO, bikeshedding the variant name doesn't get past the fact that this 
is not a build variant, it a completely different animal.



K



Re: port "cask" -- installing prebuilt binaries

2020-08-14 Thread Ruben Di Battista
I mean, you can maybe describe the “feature set” of the prebuilt binary in
the variant description and forbid all the other variants… Can’t you?

PS: Maybe “prebuilt_binary” is not the right wording since most of the
ports can already be installed as “prebuilt_binary”. Maybe it should be
something like “bin_only” ?

PPS: As I already said, ports that are open source should be fixed at least
on the buildbot  machines in order to be able to build them from sources
(as the upstream developer did) allowing few, highly selected, exceptions
(as the java stuff that was already mentioned).

  _
-. .´  |
  ',  ;|∞∞
˜˜ |∞ RdB
,.,|∞∞
  .'   '.  |
-'   `’
https://rdb.is


On 15 August 2020 at 01:11:23, Ken Cunningham (
ken.cunningham.web...@gmail.com) wrote:

consider:

$ port variants macvim
Warning: port definitions are more than two weeks old, consider updating
them by running 'port selfupdate'.
MacVim has the variants:
   big: Build big feature set
 * conflicts with huge
   cscope: Enable source code browsing with cscope
[+]huge: Build huge feature set
 * conflicts with big
   lua: Enable Lua scripting
   perl: Enable Perl scripting
   python27: Enable Python scripting
   python36: Enable Python scripting
 * conflicts with python37
   python37: Enable Python scripting
 * conflicts with python36
   ruby: Compatibility variant, requires +ruby18
 * requires ruby18
   ruby18: Enable Ruby scripting
 * conflicts with ruby19 ruby20 ruby21 ruby22 ruby23 ruby24 ruby25
   ruby19: Enable Ruby scripting
 * conflicts with ruby18 ruby20 ruby21 ruby22 ruby23 ruby24 ruby25
   ruby20: Enable Ruby scripting
 * conflicts with ruby18 ruby19 ruby21 ruby22 ruby23 ruby24 ruby25
   ruby21: Enable Ruby scripting
 * conflicts with ruby18 ruby19 ruby20 ruby22 ruby23 ruby24 ruby25
   ruby22: Enable Ruby scripting
 * conflicts with ruby18 ruby19 ruby20 ruby21 ruby23 ruby24 ruby25
   ruby23: Enable Ruby scripting
 * conflicts with ruby18 ruby19 ruby20 ruby21 ruby22 ruby24 ruby25
   ruby24: Enable Ruby scripting
 * conflicts with ruby18 ruby19 ruby20 ruby21 ruby22 ruby23 ruby25
   ruby25: Enable Ruby scripting
 * conflicts with ruby18 ruby19 ruby20 ruby21 ruby22 ruby23 ruby24
   tcl: Enable Tcl scripting
   xim: Build with support for X Input Method



so -- what would a user typing

sudo port -v install macvim +prebuild_binary -huge +big +ruby25 +tcl +xim

be expected to get? It's a total shambles. Variants for this are not
workable.  So let's move on to a plan that might actually work. Or not.
I think, as has been stated, that the whole idea is possibly too flawed
to be considered.


K


Re: port "cask" -- installing prebuilt binaries

2020-08-14 Thread Ken Cunningham

consider:

$ port variants macvim
Warning: port definitions are more than two weeks old, consider updating 
them by running 'port selfupdate'.

MacVim has the variants:
   big: Build big feature set
 * conflicts with huge
   cscope: Enable source code browsing with cscope
[+]huge: Build huge feature set
 * conflicts with big
   lua: Enable Lua scripting
   perl: Enable Perl scripting
   python27: Enable Python scripting
   python36: Enable Python scripting
 * conflicts with python37
   python37: Enable Python scripting
 * conflicts with python36
   ruby: Compatibility variant, requires +ruby18
 * requires ruby18
   ruby18: Enable Ruby scripting
 * conflicts with ruby19 ruby20 ruby21 ruby22 ruby23 ruby24 ruby25
   ruby19: Enable Ruby scripting
 * conflicts with ruby18 ruby20 ruby21 ruby22 ruby23 ruby24 ruby25
   ruby20: Enable Ruby scripting
 * conflicts with ruby18 ruby19 ruby21 ruby22 ruby23 ruby24 ruby25
   ruby21: Enable Ruby scripting
 * conflicts with ruby18 ruby19 ruby20 ruby22 ruby23 ruby24 ruby25
   ruby22: Enable Ruby scripting
 * conflicts with ruby18 ruby19 ruby20 ruby21 ruby23 ruby24 ruby25
   ruby23: Enable Ruby scripting
 * conflicts with ruby18 ruby19 ruby20 ruby21 ruby22 ruby24 ruby25
   ruby24: Enable Ruby scripting
 * conflicts with ruby18 ruby19 ruby20 ruby21 ruby22 ruby23 ruby25
   ruby25: Enable Ruby scripting
 * conflicts with ruby18 ruby19 ruby20 ruby21 ruby22 ruby23 ruby24
   tcl: Enable Tcl scripting
   xim: Build with support for X Input Method



so -- what would a user typing

sudo port -v install macvim +prebuild_binary -huge +big +ruby25 +tcl +xim

be expected to get? It's a total shambles. Variants for this are not 
workable.  So let's move on to a plan that might actually work. Or not. 
I think, as has been stated, that the whole idea is possibly too flawed 
to be considered.



K



Re: port "cask" -- installing prebuilt binaries

2020-08-14 Thread Lothar Haeger



> Am 07.08.2020 um 01:13 schrieb Ken Cunningham 
> :
> An example: let’s take the port MacVIM, which is ripe for this treatment.
> 
> We can build and install the current macvim on some newer systems - let’s say 
> 10.12 to current.
> 
> port -v install macvim gets you that.
> 
> We will all know what that port represents, without trouble, if there is a 
> ticket.
> 
> Now say on 10.7 to 10.11, we want to install a binary version of macvim. 
> macvim won't build on those systems, but there is a binary that works.

Good example, really. If we can handle this case of a hybrid port properly, the 
case where a port is totally binary only and won't build from source ever is 
covered as well.

> We need a clear, identifiable port name to install the macvim binary. It 
> should not be called macvim, that is just a *crazy* recipe for trouble. You’d 
> never be able to keep up with what was what. We need the port to clearly be 
> recognizable as the macvim binary. We need a new name. 


Totally disagree here, though. If we have two ports with different names that 
provide the same application *that* is exactly the point where the confusion 
begins. I'd rather have one port that handles both cases properly. Think of two 
port files in one, sharing the equalities but differentiation where required. A 
couple of ports do that already and Portfile syntax already allows that today.

In case of trouble, a look at the Portfile (the only one existing, thus correct 
one) shows where the if/then/else forks and which version is installed which 
way. Verbose mode during install would also print all the info required to see 
what's happening as well.

And if we can do this depending on macOS version, we can differentiate based on 
other criteria as well. So a single port could theoretically provide both 
install modes for all macOS versions (provided the port builds on all versions 
and a binary is available for all versions as well). All we'd need to make it 
possible would be a way for the user to tell the port command which route to 
choose. 

Here's where I see a variant as an option as this is an already existing 
mechanism through which a user can select which kind of final binary he wants 
to be installed. Each variant results in a different binary (being compiled by 
MacPorts) already, so using another variant as a way to choose a pre-built 
binary where available seems a simple and logical extension of what we already 
have and use.
Ports that are always built from sourec to not implement that variant, ports 
that always use a binary implement it and define it as the default variant. 
Hybrid ports implement it - possibly for a few older macOS versions - and 
defaults to building from source where possible. If a port can be build from 
source or be installed from binary in a certain environment, the user has the 
option of requesting "+pre-built" and will just get that.



Re: port "cask" -- installing prebuilt binaries

2020-08-07 Thread Ryan Schmidt



On Aug 7, 2020, at 10:49, Ken Cunningham wrote:

> By the way, for an SDK port, Gcenx has done one in an interesting way that 
> might be useful too.
> 
> 

Well that's just my port with further modifications. When I get back to looking 
into this port again, I'll continue from where I left off with my port; I don't 
want to have to puzzle out what someone else did to my port without talking to 
me about it. People are welcome to make changes to ports and contribute the 
changes back to us, at which point we can discuss their merits, or they can 
modify ports for their own personal use, which appears to be what's happened 
here, in which case it's theirs and not my concern.



Re: port "cask" -- installing prebuilt binaries

2020-08-07 Thread Nils Breunese
OpenJDK is open source, so it can be built from source. However, it is quite a 
complex and lengthy build. As the maintainer of the openjdk* ports I decided to 
use the binaries provided by the AdoptOpenJDK (for the HotSpot VM and OpenJ9 VM 
based JDK’s) and GraalVM projects. I personally wouldn’t want to replicate 
their builds in MacPorts and I don’t think anyone would enjoy waiting for 
MacPorts to build OpenJDK from source on their machine, because that can take a 
couple of hours. But theoretically the openjdk* ports could be changed to build 
from source. If anyone wants to have a go at that, be my guest!

Nils.

> Op 7 aug. 2020 om 10:09 heeft Georges Martin  het 
> volgende geschreven:
> 
> Good point.
> 
> And I just updated postgresql-jdbc, which depends on binary-only openjdk*... 
> :-|
> 
>> Le 7 août 2020 à 08:59, Ryan Schmidt  a écrit :
>> 
>> 
>> 
 On Aug 5, 2020, at 07:52, Georges Martin wrote:
>>> 
>>> If MacPorts starts to mix both approaches, I worry we may end up having 
>>> (open source) packages depending on closed source, binary packages.
>> 
>> We already have that, by necessity. For example, php-excel is an open source 
>> php module that depends on the closed-source libxl library, for which we 
>> have a port. php-oracle, an open source php module distributed as part of 
>> php itself, depends on the closed-source oracle-instantclient library, for 
>> which we have a port.



Re: port "cask" -- installing prebuilt binaries

2020-08-07 Thread Ken Cunningham
nope, variant is just wrong in all ways.

K


Re: port "cask" -- installing prebuilt binaries

2020-08-07 Thread Arno Hautala


> On 7 Aug 2020, at 09:04, Ken Cunningham  
> wrote:
> 
> If we use only category to mark as a binary port, what if we have a binary 
> port added (e.g. say lazarus) but then we later find out we can build that 
> port on some systems? Then we want the binary lazarus moved to 
> lazarus-binary, so we can have a real lazarus port that builds from source 
> named lazarus.

> I'm back to using the name to mark binary ports. I believe we will regret not 
> doing that in the end if we don't do it at the start.


I think the naming could indeed be a headache and a port effectively 
conflicting with itself (binary vs source) is something that should be avoided. 
I just don’t think the name is the right place for that.

I think I now prefer the variant as the right way to label and deconflict this. 
Yes, it implies it’s an option, but ports already provide default variants and 
a portfile could already fail the install if the “prebuilt” variant is manually 
removed from an install request of a port that only provides a prebuilt option.

The variants.conf file could even list ‘-prebuilt’ to keep these ports from 
“poisoning” your installation as dependents.

Using the variant solves the naming conflict, is more visible than using a 
category, and doesn’t overload the port name with metadata.

My only quibble would be that I don’t particularly like “binary” or “prebuilt” 
for the variant name. Both could be confused with referring to installing from 
an archive built by the MacPorts build server. Something like “vendor_provided” 
is lengthy, but maybe clearer as to what it means?

-- 
arno  s  hautala/-|   a...@alum.wpi.edu

pgp b2c9d448



Re: port "cask" -- installing prebuilt binaries

2020-08-07 Thread Ken Cunningham


On 2020-08-07, at 12:10 AM, Ryan Schmidt wrote:

> 
> 
> On Aug 6, 2020, at 21:20, Ruben Di Battista wrote:
> 
>> If they've been built somewhere by the upstream developer, shouldn't we be 
>> able to build them too and hence instead of providing them as binaries we 
>> would  fix the ports in order to build on old systems? I guess it's a matter 
>> of how hard is to fix the build environment on old systems... 
> 
> That would be my preference too. We've already addressed many of the reasons 
> why something would fail on an older system: we have newer cctools, ld64, 
> clang available in MacPorts. The one big remaining issue is that a lot of 
> Mac-specific software can only build with the latest SDK, so we need to add a 
> MacOSX.sdk port with subports to install each major version of the SDK. I 
> worked on such a port awhile back but didn't finish it: 
> https://github.com/ryandesign/macports-ports/blob/MacOSX.sdk/devel/MacOSX.sdk/Portfile
> 
> An alternative solution is to devise a method where certain ports can opt 
> into being built on a newer system. See 
> https://trac.macports.org/ticket/60878 for more on that proposal.
> 

If we can do this, I also agree we should, for many reasons.

But still there are many things that we can't.

By the way, for an SDK port, Gcenx has done one in an interesting way that 
might be useful too.






Re: port "cask" -- installing prebuilt binaries

2020-08-07 Thread Ken Cunningham
A few more issues:

Handling supported_archs in binary-only ports. What if binary comes only as a 
universal binary, but the user asks for only one arch? We want that one binary 
to satisfy all the situations it can.

Handling universal requests. What if user asks for universal? If binary comes 
with three arches but user asks for only two? etc

Handling std library link checking? Binaries might come with all kinds of 
different linkages, including internal stdlibs (tenfourfox). Many will need 
link checking disabled, but some will need it enabled...

If we use only category to mark as a binary port, what if we have a binary port 
added (e.g. say lazarus) but then we later find out we can build that port on 
some systems? Then we want the binary lazarus moved to lazarus-binary, so we 
can have a real lazarus port that builds from source named lazarus. ( please 
ignore the fact we already have a lazarus port - this is just illustrative).

I'm back to using the name to mark binary ports. I believe we will regret not 
doing that in the end if we don't do it at the start.

K

Re: port "cask" -- installing prebuilt binaries

2020-08-07 Thread Ken Cunningham
yes, it's variant names that can't have + and - in them. Momentarily crossed 
that wire.

K

> On Aug 7, 2020, at 00:01, Ryan Schmidt  wrote:
> 
> 
> 
>> On Aug 6, 2020, at 09:10, Ken Cunningham wrote:
>> 
>> So a port that installs the Zoom teleconferencing application would be 
>> called "zoom_binary". (We can't use "zoom-binary" because variants us the + 
>> and - to do their thing.)
> 
> Calling it "zoom-binary" would be fine. Yes, "+" and "-" are used to select 
> and deselect variants, but MacPorts allows "-" in port names; we already have 
> thousands of ports with "-" in their names.
> 


Re: port "cask" -- installing prebuilt binaries

2020-08-07 Thread Georges Martin
Good point.

And I just updated postgresql-jdbc, which depends on binary-only openjdk*... :-|

> Le 7 août 2020 à 08:59, Ryan Schmidt  a écrit :
> 
> 
> 
>> On Aug 5, 2020, at 07:52, Georges Martin wrote:
>> 
>> If MacPorts starts to mix both approaches, I worry we may end up having 
>> (open source) packages depending on closed source, binary packages.
> 
> We already have that, by necessity. For example, php-excel is an open source 
> php module that depends on the closed-source libxl library, for which we have 
> a port. php-oracle, an open source php module distributed as part of php 
> itself, depends on the closed-source oracle-instantclient library, for which 
> we have a port.


Re: port "cask" -- installing prebuilt binaries

2020-08-07 Thread Ryan Schmidt



On Aug 6, 2020, at 21:20, Ruben Di Battista wrote:

> If they've been built somewhere by the upstream developer, shouldn't we be 
> able to build them too and hence instead of providing them as binaries we 
> would  fix the ports in order to build on old systems? I guess it's a matter 
> of how hard is to fix the build environment on old systems... 

That would be my preference too. We've already addressed many of the reasons 
why something would fail on an older system: we have newer cctools, ld64, clang 
available in MacPorts. The one big remaining issue is that a lot of 
Mac-specific software can only build with the latest SDK, so we need to add a 
MacOSX.sdk port with subports to install each major version of the SDK. I 
worked on such a port awhile back but didn't finish it: 
https://github.com/ryandesign/macports-ports/blob/MacOSX.sdk/devel/MacOSX.sdk/Portfile

An alternative solution is to devise a method where certain ports can opt into 
being built on a newer system. See https://trac.macports.org/ticket/60878 for 
more on that proposal.



Re: port "cask" -- installing prebuilt binaries

2020-08-07 Thread Ryan Schmidt



> On Aug 6, 2020, at 13:34, Arno Hautala wrote:
> 
>> On 6 Aug 2020, at 14:06, Ken Cunningham wrote:
>> 
>> Along with name-suffix, a category was actually one of the first suggestions 
>> in the original post 50 posts ago, but how does one see which ports you have 
>> installed that are members of that category?
>> 
>> with a name suffix, easy.
>> 
>> port -v installed | grep binary
> 
> 
> Assuming a category of “binary”:
> 
>> port echo category:binary and installed
> 

Searching your installed ports by name or category is equally easy.

port installed category:binary

port installed name:binary$



Re: port "cask" -- installing prebuilt binaries

2020-08-07 Thread Ryan Schmidt



On Aug 6, 2020, at 10:32, Arno Hautala wrote:

> Have any developers asked for source not to be mirrored?

A few; see "port echo license:nomirror"



Re: port "cask" -- installing prebuilt binaries

2020-08-07 Thread Ryan Schmidt



On Aug 6, 2020, at 09:10, Ken Cunningham wrote:

> So a port that installs the Zoom teleconferencing application would be called 
> "zoom_binary". (We can't use "zoom-binary" because variants us the + and - to 
> do their thing.)

Calling it "zoom-binary" would be fine. Yes, "+" and "-" are used to select and 
deselect variants, but MacPorts allows "-" in port names; we already have 
thousands of ports with "-" in their names.



Re: port "cask" -- installing prebuilt binaries

2020-08-07 Thread Ryan Schmidt



On Aug 5, 2020, at 07:52, Georges Martin wrote:

> If MacPorts starts to mix both approaches, I worry we may end up having (open 
> source) packages depending on closed source, binary packages.

We already have that, by necessity. For example, php-excel is an open source 
php module that depends on the closed-source libxl library, for which we have a 
port. php-oracle, an open source php module distributed as part of php itself, 
depends on the closed-source oracle-instantclient library, for which we have a 
port.

Re: port "cask" -- installing prebuilt binaries

2020-08-06 Thread Ruben Di Battista
I also believe we should apply this strategy to "binary only" non-open
source software.

I have a question tho, probably addressed mostly to @Ken and those that
manage the ports builds on old systems, in facts, about hybrid ports (as
defined by @Herby: open source software that we are unable to build on some
systems and then we ship it as binary only).

If they've been built somewhere by the upstream developer, shouldn't we be
able to build them too and hence instead of providing them as binaries we
would  fix the ports in order to build on old systems? I guess it's a
matter of how hard is to fix the build environment on old systems...


On Fri, 7 Aug 2020, 03:56 Herby G,  wrote:

> I certainly see your points. Here are my thoughts:
>
> On the subject of binary-only ports, you'll have two kinds:
>
>- binary ports that are *entirely* binary _only_, and were *never*
>source-built, no matter the platforms or macOS version (like Zoom for
>example)
>- "hybrid" ports that are source-compiled under certain conditions,
>and fetching a binary in other cases
>
> The first thing to consider is what's the ratio of pure-binary to "hybrid"
> binary ports? I feel like "hybrid" ones would be in the minority.
> Homebrew's "casks" are mostly pure binary-only.
>
> One thing that can be done to simplify things is have the condition that
> only "purely" binary ports can be in the "binary" category (for
> simplicity's sake).
>
> Hybrid ports are going to be a hassle no matter what, and that would be
> true for any port that has conditional logic that changes the port's
> behavior depending on macOS version and other factors (besides
> architecture).
>
> When someone is filing a ticket, we'd want to know their macOS version +
> environmental details anyway, and that tells you what you need to know as a
> maintainer to troubleshoot a hybrid port that you are maintaining.
>
> Now the headaches you illustrate have a lot to do with how a hybrid port
> would be implemented.
>
> Two ways that immediately come to mind are:
>
>- sub-ports: your main port installs a source-built or binary-only
>sub-port depending on macOS version
>- same port but using conditional logic internally
>
> In the first case, each sub-port would have to have a different name, so
> that's that.
> In the case of the second, yes, that can lead to the scenario you
> illustrated, but as I mentioned above, once we know the user's environment,
> we can know what's installed.
>
> I am not against having different names for a binary-only vs source-built
> version of a port.  But that only matters in the case where the same piece
> of software is source-based in one place and binary-only elsewhere.
>
> I think using a category to mark binary-only ports is far more optimal
> than packing that meaning into the port's name, especially when ports can
> choose almost any name they want.
>
> Additionally, as I've mentioned previously, it's much easier for tooling
> like the MacPorts website, the port command and more to tell based off a
> category than a list of blessed regexes for determining whether a port
> _might be_ binary only (as it's possible some other port may have "binary"
> in its name even though it may not be).
>
>
> On Thu, Aug 6, 2020 at 7:13 PM Ken Cunningham <
> ken.cunningham.web...@gmail.com> wrote:
>
>>
>>
>> > On Aug 6, 2020, at 2:34 PM, Herby G  wrote:
>> >
>> > > If we decide to go ahead with this, and if we decide to primarily use
>> a category to mark these, we will need a plan for how to manage a name
>> collision conflict when there are two ports that install the same software,
>> one by building from source (on newer systems) and one by installing a
>> binary (on older systems).
>>
>> > This does not introduce any new mechanism or concept that does not
>> already currently exist in MacPorts.
>> >
>>
>> yes it does. An example: let’s take the port MacVIM, which is ripe for
>> this treatment.
>>
>> We can build and install the current macvim on some newer systems - let’s
>> say 10.12 to current.
>>
>> port -v install macvim gets you that.
>>
>> We will all know what that port represents, without trouble, if there is
>> a ticket.
>>
>> Now say on 10.7 to 10.11, we want to install a binary version of macvim.
>> macvim won't build on those systems, but there is a binary that works.
>>
>> We need a clear, identifiable port name to install the macvim binary. It
>> should not be called macvim, that is just a *crazy* recipe for trouble.
>> You’d never be able to keep up with what was what. We need the port to
>> clearly be recognizable as the macvim binary.
>>
>> We need a new name. No trickery with categories is acceptable here.
>>
>> The “macvim” port might (or might not) point a user to the macvim_binary
>> port if needed, but it can’t be the same port with the same name, and it
>> most definitely should not install the binary instead. If we go down that
>> road, we’ll never know what is going on, and this whole idea 

Re: port "cask" -- installing prebuilt binaries

2020-08-06 Thread Herby G
I certainly see your points. Here are my thoughts:

On the subject of binary-only ports, you'll have two kinds:

   - binary ports that are *entirely* binary _only_, and were *never*
   source-built, no matter the platforms or macOS version (like Zoom for
   example)
   - "hybrid" ports that are source-compiled under certain conditions, and
   fetching a binary in other cases

The first thing to consider is what's the ratio of pure-binary to "hybrid"
binary ports? I feel like "hybrid" ones would be in the minority.
Homebrew's "casks" are mostly pure binary-only.

One thing that can be done to simplify things is have the condition that
only "purely" binary ports can be in the "binary" category (for
simplicity's sake).

Hybrid ports are going to be a hassle no matter what, and that would be
true for any port that has conditional logic that changes the port's
behavior depending on macOS version and other factors (besides
architecture).

When someone is filing a ticket, we'd want to know their macOS version +
environmental details anyway, and that tells you what you need to know as a
maintainer to troubleshoot a hybrid port that you are maintaining.

Now the headaches you illustrate have a lot to do with how a hybrid port
would be implemented.

Two ways that immediately come to mind are:

   - sub-ports: your main port installs a source-built or binary-only
   sub-port depending on macOS version
   - same port but using conditional logic internally

In the first case, each sub-port would have to have a different name, so
that's that.
In the case of the second, yes, that can lead to the scenario you
illustrated, but as I mentioned above, once we know the user's environment,
we can know what's installed.

I am not against having different names for a binary-only vs source-built
version of a port.  But that only matters in the case where the same piece
of software is source-based in one place and binary-only elsewhere.

I think using a category to mark binary-only ports is far more optimal than
packing that meaning into the port's name, especially when ports can choose
almost any name they want.

Additionally, as I've mentioned previously, it's much easier for tooling
like the MacPorts website, the port command and more to tell based off a
category than a list of blessed regexes for determining whether a port
_might be_ binary only (as it's possible some other port may have "binary"
in its name even though it may not be).


On Thu, Aug 6, 2020 at 7:13 PM Ken Cunningham <
ken.cunningham.web...@gmail.com> wrote:

>
>
> > On Aug 6, 2020, at 2:34 PM, Herby G  wrote:
> >
> > > If we decide to go ahead with this, and if we decide to primarily use
> a category to mark these, we will need a plan for how to manage a name
> collision conflict when there are two ports that install the same software,
> one by building from source (on newer systems) and one by installing a
> binary (on older systems).
>
> > This does not introduce any new mechanism or concept that does not
> already currently exist in MacPorts.
> >
>
> yes it does. An example: let’s take the port MacVIM, which is ripe for
> this treatment.
>
> We can build and install the current macvim on some newer systems - let’s
> say 10.12 to current.
>
> port -v install macvim gets you that.
>
> We will all know what that port represents, without trouble, if there is a
> ticket.
>
> Now say on 10.7 to 10.11, we want to install a binary version of macvim.
> macvim won't build on those systems, but there is a binary that works.
>
> We need a clear, identifiable port name to install the macvim binary. It
> should not be called macvim, that is just a *crazy* recipe for trouble.
> You’d never be able to keep up with what was what. We need the port to
> clearly be recognizable as the macvim binary.
>
> We need a new name. No trickery with categories is acceptable here.
>
> The “macvim” port might (or might not) point a user to the macvim_binary
> port if needed, but it can’t be the same port with the same name, and it
> most definitely should not install the binary instead. If we go down that
> road, we’ll never know what is going on, and this whole idea would best be
> left not done.
>
> Like
>
>
>
>


Re: port "cask" -- installing prebuilt binaries

2020-08-06 Thread Ken Cunningham


> On Aug 6, 2020, at 4:13 PM, Ken Cunningham  
> wrote:
> 
> We need a new name. No trickery with categories is acceptable here.

Picture the ticket:

hello, user. Uhm before I can help you, does

port list echo categories:binary and installed | grep macvim say “binary” ?

No —wait — I meant does:

port echo leaves: binary and installed and macvim say “binary”

no — heck - got it wrong again — how about does

port -v installed category:binary and installed and “I’m having a stroke” 
indicate you’re having a stroke? 

Oh — forget about it….

Ken

Re: port "cask" -- installing prebuilt binaries

2020-08-06 Thread Ken Cunningham



> On Aug 6, 2020, at 2:34 PM, Herby G  wrote:
> 
> > If we decide to go ahead with this, and if we decide to primarily use a 
> > category to mark these, we will need a plan for how to manage a name 
> > collision conflict when there are two ports that install the same software, 
> > one by building from source (on newer systems) and one by installing a 
> > binary (on older systems).

> This does not introduce any new mechanism or concept that does not already 
> currently exist in MacPorts.
> 

yes it does. An example: let’s take the port MacVIM, which is ripe for this 
treatment.

We can build and install the current macvim on some newer systems - let’s say 
10.12 to current.

port -v install macvim gets you that.

We will all know what that port represents, without trouble, if there is a 
ticket.

Now say on 10.7 to 10.11, we want to install a binary version of macvim. macvim 
won't build on those systems, but there is a binary that works.

We need a clear, identifiable port name to install the macvim binary. It should 
not be called macvim, that is just a *crazy* recipe for trouble. You’d never be 
able to keep up with what was what. We need the port to clearly be recognizable 
as the macvim binary.

We need a new name. No trickery with categories is acceptable here.

The “macvim” port might (or might not) point a user to the macvim_binary port 
if needed, but it can’t be the same port with the same name, and it most 
definitely should not install the binary instead. If we go down that road, 
we’ll never know what is going on, and this whole idea would best be left not 
done.

Like





Re: port "cask" -- installing prebuilt binaries

2020-08-06 Thread Herby G
> If we decide to go ahead with this, and if we decide to primarily use a
category to mark these, we will need a plan for how to manage a name
collision conflict when there are two ports that install the same software,
one by building from source (on newer systems) and one by installing a
binary (on older systems).

Unless I am misunderstanding, I think that question is answered by what is
our current policy today for that?

We are discussing ports just like any other, they just have an additional
category ("binary" or "binary_only").

The question of conflict could still arise even if I were discussing two
other ports, one in the "sysutils" category, and one in the "net" category.

This does not introduce any new mechanism or concept that does not already
currently exist in MacPorts.

On Thu, Aug 6, 2020 at 5:04 PM Ken Cunningham <
ken.cunningham.web...@gmail.com> wrote:

>
>
> On Aug 6, 2020, at 12:28 PM, Herby G  wrote:
>
> > so far, name-suffix is winning on all fronts...with no downsides yet.
>
> I don't plan on pushing the issue, but I have to say that I don't agree.
>
> Using a name suffix isn't clean, as you may include other non-binary ports
> that may happen to have the word "binary" in their name.
>
> A category allows you a cleaner approach as you can now represent that a
> port is binary as an _attribute_ of the port, rather than overloading the
> name.
>
> This will make it easier to write port utilities and commands that target
> binary ports.
>
> We can easily add an alias that could let you do things like "port -v
> binary_only" which would transparently do the "category:binary".
>
> Additionally, if using a category, you can see the list of binary ports in
> a clean way when browsing ports in the MacPorts website, it makes it easier
> to do things like add an icon to signify binary only if a given port is in
> the "binary" category, and not make possibly mistaken assumptions off of
> the name.
>
> On Thu, Aug 6, 2020 at 3:02 PM Ken Cunningham <
> ken.cunningham.web...@gmail.com> wrote:
>
>> category-only identifier is
>>
>> less clear and less obvious
>> harder to remember how to search for
>> name conflicts with a non-binary version (eg for newer systems that can
>> build it)
>>
>> so far, name-suffix is winning on all fronts...with no downsides yet.
>>
>> K
>
>
>
> If we decide to go ahead with this, and if we decide to primarily use a
> category to mark these, we will need a plan for how to manage a name
> collision conflict when there are two ports that install the same software,
> one by building from source (on newer systems) and one by installing a
> binary (on older systems).
>
> I would suggest the binary install version of the port be called
> “portname_binary” unless someone has a better idea for how to manage this
> issue.
>
> Ken
>
>
>


Re: port "cask" -- installing prebuilt binaries

2020-08-06 Thread Ken Cunningham


> On Aug 6, 2020, at 12:28 PM, Herby G  wrote:
> 
> > so far, name-suffix is winning on all fronts...with no downsides yet.
> 
> I don't plan on pushing the issue, but I have to say that I don't agree.
> 
> Using a name suffix isn't clean, as you may include other non-binary ports 
> that may happen to have the word "binary" in their name.
> 
> A category allows you a cleaner approach as you can now represent that a port 
> is binary as an _attribute_ of the port, rather than overloading the name.
> 
> This will make it easier to write port utilities and commands that target 
> binary ports.
> 
> We can easily add an alias that could let you do things like "port -v 
> binary_only" which would transparently do the "category:binary".
> 
> Additionally, if using a category, you can see the list of binary ports in a 
> clean way when browsing ports in the MacPorts website, it makes it easier to 
> do things like add an icon to signify binary only if a given port is in the 
> "binary" category, and not make possibly mistaken assumptions off of the name.
> 
> On Thu, Aug 6, 2020 at 3:02 PM Ken Cunningham 
> mailto:ken.cunningham.web...@gmail.com>> 
> wrote:
> category-only identifier is
> 
> less clear and less obvious
> harder to remember how to search for
> name conflicts with a non-binary version (eg for newer systems that can build 
> it)
> 
> so far, name-suffix is winning on all fronts...with no downsides yet.
> 
> K


If we decide to go ahead with this, and if we decide to primarily use a 
category to mark these, we will need a plan for how to manage a name collision 
conflict when there are two ports that install the same software, one by 
building from source (on newer systems) and one by installing a binary (on 
older systems).

I would suggest the binary install version of the port be called 
“portname_binary” unless someone has a better idea for how to manage this issue.

Ken




Re: port "cask" -- installing prebuilt binaries

2020-08-06 Thread Ken Cunningham


> On Aug 6, 2020, at 1:42 PM, Jason Liu  wrote:
> 
> All that said, one more question.  As I now understand it, the idea is to 
> download a binary-only installer (from the publisher’s web site) and launch 
> it.  Someone still has to answer any and all dialogs that such installers 
> always present.  So, it fact, the administrator has to sit at the machine and 
> click “OK” ad nauseam.  Previously, I thought we were going to create a new 
> binary image that would avoid such tedium.  Do I have this right?  Or is 
> there some scripting trickery wrapped around the installer?
> 
> The situation of dialog boxes and clicking "OK" ad nauseam is, in most cases, 
> completely unnecessary. Installing binary-only installers (.dmg or .pkg) can 
> be accomplished exclusively using the command line:
> 
> If the installer is a .dmg:
> 
> # hdiutil mount software-title.dmg
> 
> Once the DMG is mounted, if it's just the app, then
> 
> # cp -R "/Volumes/Mounted DMG/Software Title.app" /Applications
> 
> on the other hand, if the contents are a .pkg, then
> 
> # /usr/bin/installer -package "/Volumes/Mounted DMG/Software Installer.pkg" 
> -target "/Volumes/Macintosh HD"
> 
> Finally, unmount the DMG:
> 
> # hdiutil unmount "/Volumes/Mounted DMG"
> 
> For the vast majority of cases, no manual user intervention is necessary. In 
> fact, software deployment tools such as Jamf/Casper, Munki, and even Apple 
> MDM, use this method to perform non-interactive remote installations of Mac 
> software.
>  
> — 


MacPorts can do much of this automatically, I belive:






Ken

Re: port "cask" -- installing prebuilt binaries

2020-08-06 Thread Jason Liu
>
> All that said, one more question.  As I now understand it, the idea is to
> download a binary-only installer (from the publisher’s web site) and launch
> it.  Someone still has to answer any and all dialogs that such installers
> always present.  So, it fact, the administrator has to sit at the machine
> and click “OK” ad nauseam.  Previously, I thought we were going to create a
> new binary image that would avoid such tedium.  Do I have this right?  Or
> is there some scripting trickery wrapped around the installer?


The situation of dialog boxes and clicking "OK" ad nauseam is, in most
cases, completely unnecessary. Installing binary-only installers (.dmg or
.pkg) can be accomplished exclusively using the command line:

If the installer is a .dmg:

# hdiutil mount software-title.dmg

Once the DMG is mounted, if it's just the app, then

# cp -R "/Volumes/Mounted DMG/Software Title.app" /Applications

on the other hand, if the contents are a .pkg, then

# /usr/bin/installer -package "/Volumes/Mounted DMG/Software Installer.pkg"
-target "/Volumes/Macintosh HD"

Finally, unmount the DMG:

# hdiutil unmount "/Volumes/Mounted DMG"

For the vast majority of cases, no manual user intervention is necessary.
In fact, software deployment tools such as Jamf/Casper, Munki, and even
Apple MDM, use this method to perform non-interactive remote installations
of Mac software.

-- 
Jason Liu


On Thu, Aug 6, 2020 at 3:16 PM Craig Treleaven 
wrote:

> On Aug 6, 2020, at 11:29 AM, Ken Cunningham <
> ken.cunningham.web...@gmail.com> wrote:
>
> All valid points. I thought we had more-or-less got past the “should we”
> and moved on to the “how should we”,
>
> I am not necessarily championing this, but people are submitting these,
> and there is demand.here are nearly 4000 cask installer formulae on brew
> now. If similar binary-only ports are going to be accepted, I was hoping
> for a mechanism to identify and control them somewhat, for the reasons you
> mention and more.
>
>
> So, pretend I don’t know how Homebrew’s cask system works.  (I don’t.)
>
> 1) As a user, what is the advantage of this kind of system versus other
> avenues for software (i.e. the Mac App Store or direct download of a dmg
> from the developer web site)?
>
>
> convenience, really. You can install something with a short command
> instead of wading through finding the right installer on a website. Updates
> are handled transparently. You can install 10 or 20 software packages onto
> a set of systems with one command. You can make a list of the 75 software
> packages you like to have and install them all with one command. You can
> tell your grandmother how to install zoom with four words to paste into a
> terminal instead of a complicated set of download and install instructions.
>
> Doesn’t most such software include an auto-updater?
>
>
> Sometimes.
>
> If so, won’t that conflict with MacPorts update handling?
>
>
> Yes.
>
> A potential disadvantage would be the time lag from a new version being
> released until a new ‘cask’ is available, right?
>
>
> Yes
>
>
> If I understand correctly, this is a facility that would only benefit
> system administrators that have a fleet of Macs that are more-or-less
> configured the same.  It would help them to automate the process of
> installing and updating software whether the software is open-source or
> only available as a binary.  (But not software that is only on the Mac App
> Store.)  Is that right?
>
> That is a worthy audience but why is this something that MacPorts should
> address?  I believe there are already 'multiple device management systems'
> out there that support macOS.  And if Homebrew already provides 4,000
> packages, why do we want to do the same?  Is it not possible to combine
> Homebrew’s casks with open source software installed by MacPorts?
>
> MacPorts is not in competition with Homebrew.  Really.  The projects have
> different objectives and goals and that is perfectly fine.  Both
> communities have sufficient support to continue for the foreseeable
> future.  We don’t have to “defeat them” to “win” or vice versa.  We seem to
> be aiming to replicate their cask system.  Should we not be aiming to
> provide a system that is demonstrably *better* than what is currently
> available?
>
> All that said, one more question.  As I now understand it, the idea is to
> download a binary-only installer (from the publisher’s web site) and launch
> it.  Someone still has to answer any and all dialogs that such installers
> always present.  So, it fact, the administrator has to sit at the machine
> and click “OK” ad nauseam.  Previously, I thought we were going to create a
> new binary image that would avoid such tedium.  Do I have this right?  Or
> is there some scripting trickery wrapped around the installer?
>
> Personally, I don’t see any compelling reason why the MacPorts project
> should want to go in this direction.  The first paragraph on our homepage
> says:
>
> "The MacPorts Project is an 

Re: port "cask" -- installing prebuilt binaries

2020-08-06 Thread Jason Liu
>
> 2) My impression is that developers of commercial software would, in many
> cases, NOT want a third party (like MacPorts) to be distributing their
> software.  From their perspective, a third party introduces considerably
> more risk that users may end up with maliciously altered software.  Can we
> not expect to get takedown notices from certain publishers?
>

At a university where I used to work as a systems administrator, they
managed software on their fleet of Macs using a combination of Munki and
Jenkins. This included commercial software, including such titles such as
Autodesk AutoCAD, Adobe Creative Suite (before it became Creative Cloud),
Microsoft Office (before it became 365), and EndNote.

There were never any takedown notices from any of the commercial
publishers. In fact, the central IT group that developed and maintained the
Munki+Jenkins framework was frequently in contact with the software
companies, and worked with them on how to better deploy their software
titles using the framework. Their discussions typically revolved around how
to push out the licenses together with the software installers during
installations and (sometimes) remote updates.

As far as I know, the companies never, ever complained about our usage of
Munki and Jenkins. The commercial companies are typically only worried
about whether or not a customer has purchased the correct number of
licenses from them to cover the number of seats. They don't care how the
customer goes about deploying their software (or in fact are willing to
help customers with figuring out ways to better deploy their software using
the customers' deployment frameworks).

In theory, proprietary software could be deployed using MacPorts as well. I
have in fact done this. Our department had research groups that purchased
some proprietary Mac software that happened to be fairly Unix-like in terms
of its installation process. My fellow Mac admin colleague and I set up a
local MacPorts server (maybe it would be considered a mirror?), and we
wrote our own portfile for this proprietary software. We even contacted the
company to troubleshoot how we could make sure that our port contacted the
software's license server (which we set up on a different machine than our
local MacPorts server) during the installation process. The company never
complained about our using MacPorts to deploy their software. Their support
engineers even commented about how "awesome" and innovative it was that we
were using MacPorts to deploy commercial software. They thought that
MacPorts could only be used to deploy open source software.

-- 
Jason Liu


On Thu, Aug 6, 2020 at 11:04 AM Craig Treleaven 
wrote:

> > On Aug 6, 2020, at 10:10 AM, Ken Cunningham <
> ken.cunningham.web...@gmail.com> wrote:
> >
> > How about I float a suggestion? We could append "_binary" to the name.
> Otherwise leave the categories, notes, etc as they are now.
> >
> > So a port that installs the Zoom teleconferencing application would be
> called "zoom_binary". (We can't use "zoom-binary" because variants us the +
> and - to do their thing.) You could find all your installed binaries with a
> simple grep...
> >
> > Open to a more descriptive, shorter suffix if anyone is thinking of one.
> >
> > Once we find metadata we agree on, then two more points to work out.
> > 1. Should we mirror the binary installers (no, I'd say)?
> > 2. We should require a mechanism to remove all traces of the software
> when uninstalling it, I think.
> >
>
> So, pretend I don’t know how Homebrew’s cask system works.  (I don’t.)
>
> 1) As a user, what is the advantage of this kind of system versus other
> avenues for software (i.e. the Mac App Store or direct download of a dmg
> from the developer web site)?  Doesn’t most such software include an
> auto-updater?  If so, won’t that conflict with MacPorts update handling?  A
> potential disadvantage would be the time lag from a new version being
> released until a new ‘cask’ is available, right?
>
> 2) My impression is that developers of commercial software would, in many
> cases, NOT want a third party (like MacPorts) to be distributing their
> software.  From their perspective, a third party introduces considerably
> more risk that users may end up with maliciously altered software.  Can we
> not expect to get takedown notices from certain publishers?
>
> 3) Is the MacPorts mirror network willing to contribute bandwidth for
> distributing non-open source software?  Will we sour our relations with
> some of the mirrors if we use/abuse their bandwidth this way?  Why do we
> want to use our bandwidth that way?
>
> Craig
>
>


Re: port "cask" -- installing prebuilt binaries

2020-08-06 Thread Herby G
> All that said, one more question.  As I now understand it, the idea is to
download a binary-only installer (from the publisher’s web site) and launch
it.  Someone still has to answer any and all dialogs that such installers
always present.  So, it fact, the administrator has to sit at the machine
and click “OK” ad nauseam.  Previously, I thought we were going to create a
new binary image that would avoid such tedium.  Do I have this right?  Or
is there some scripting trickery wrapped around the installer?

That is not always the case.  There is plenty of binary-only software
that's literally a binary or an app bundle in a .dmg or zip file equivalent.

A good example of this is the port for the 1Password CLI (
https://github.com/macports/macports-ports/blob/master/security/1password-cli/Portfile),
or a .app bundle meant to be extracted into your Applications directory.

Those are excellent candidates for what Ken is talking about.  Even for
some ports that are way too onerous to build, some of the utilities for
handling binary-only ports can be used for making it easier to provide the
pre-built distributable(s) of such a piece of software, just as Nils was
saying at the start of this thread.

On Thu, Aug 6, 2020 at 3:16 PM Craig Treleaven 
wrote:

> On Aug 6, 2020, at 11:29 AM, Ken Cunningham <
> ken.cunningham.web...@gmail.com> wrote:
>
> All valid points. I thought we had more-or-less got past the “should we”
> and moved on to the “how should we”,
>
> I am not necessarily championing this, but people are submitting these,
> and there is demand.here are nearly 4000 cask installer formulae on brew
> now. If similar binary-only ports are going to be accepted, I was hoping
> for a mechanism to identify and control them somewhat, for the reasons you
> mention and more.
>
>
> So, pretend I don’t know how Homebrew’s cask system works.  (I don’t.)
>
> 1) As a user, what is the advantage of this kind of system versus other
> avenues for software (i.e. the Mac App Store or direct download of a dmg
> from the developer web site)?
>
>
> convenience, really. You can install something with a short command
> instead of wading through finding the right installer on a website. Updates
> are handled transparently. You can install 10 or 20 software packages onto
> a set of systems with one command. You can make a list of the 75 software
> packages you like to have and install them all with one command. You can
> tell your grandmother how to install zoom with four words to paste into a
> terminal instead of a complicated set of download and install instructions.
>
> Doesn’t most such software include an auto-updater?
>
>
> Sometimes.
>
> If so, won’t that conflict with MacPorts update handling?
>
>
> Yes.
>
> A potential disadvantage would be the time lag from a new version being
> released until a new ‘cask’ is available, right?
>
>
> Yes
>
>
> If I understand correctly, this is a facility that would only benefit
> system administrators that have a fleet of Macs that are more-or-less
> configured the same.  It would help them to automate the process of
> installing and updating software whether the software is open-source or
> only available as a binary.  (But not software that is only on the Mac App
> Store.)  Is that right?
>
> That is a worthy audience but why is this something that MacPorts should
> address?  I believe there are already 'multiple device management systems'
> out there that support macOS.  And if Homebrew already provides 4,000
> packages, why do we want to do the same?  Is it not possible to combine
> Homebrew’s casks with open source software installed by MacPorts?
>
> MacPorts is not in competition with Homebrew.  Really.  The projects have
> different objectives and goals and that is perfectly fine.  Both
> communities have sufficient support to continue for the foreseeable
> future.  We don’t have to “defeat them” to “win” or vice versa.  We seem to
> be aiming to replicate their cask system.  Should we not be aiming to
> provide a system that is demonstrably *better* than what is currently
> available?
>
> All that said, one more question.  As I now understand it, the idea is to
> download a binary-only installer (from the publisher’s web site) and launch
> it.  Someone still has to answer any and all dialogs that such installers
> always present.  So, it fact, the administrator has to sit at the machine
> and click “OK” ad nauseam.  Previously, I thought we were going to create a
> new binary image that would avoid such tedium.  Do I have this right?  Or
> is there some scripting trickery wrapped around the installer?
>
> Personally, I don’t see any compelling reason why the MacPorts project
> should want to go in this direction.  The first paragraph on our homepage
> says:
>
> "The MacPorts Project is an open-source community initiative to design an
> easy-to-use system for compiling, installing, and upgrading either
> command-line, X11 or Aqua based open-source software on the Mac operating
> 

Re: port "cask" -- installing prebuilt binaries

2020-08-06 Thread Ken Cunningham



> On Aug 6, 2020, at 12:28 PM, Herby G  wrote:
> 
> > so far, name-suffix is winning on all fronts...with no downsides yet.
> 
> I don't plan on pushing the issue, but I have to say that I don't agree.
> 

I originally suggested both, and got no purchase on either, so had to push one 
of them to get some reaction :> I get the feeling if I had pushed the category 
idea this morning, we’d all be talking now about how the name method would be 
so much better :>



I would use the name — it’s just so much easier. I can see what’s installed 
when someone posts a ticket by the name they used. I can search for them. I can 
remember the command to search for them. I can tell someone else how to search 
for them. It doesn’t conflict with the non-binary port if one exists.

But truly -- that is really the least of the worries here.

So long as:

The binary install can be EASILY recognized by all.
You can FIND them all easily, and delete them.

I don’t care.

Ken

Re: port "cask" -- installing prebuilt binaries

2020-08-06 Thread Ken Cunningham


> On Aug 6, 2020, at 12:16 PM, Craig Treleaven  wrote:
> 
> 

> Personally, I don’t see any compelling reason why the MacPorts project should 
> want to go in this direction.  The first paragraph on our homepage says:

Actually, I’m perfectly happy with a blanket “NO” on all this, if that is the 
outcome.

What I am trying to avoid is a bunch of these slipping in with no organization 
to it at all, for all these reasons.

And IF it is done, please make it SIMPLE.

no cryptic commands that 1 person in 10,000 can remember. MacPorts is already 
too full of that kind of thing.

K



Re: port "cask" -- installing prebuilt binaries

2020-08-06 Thread Herby G
> so far, name-suffix is winning on all fronts...with no downsides yet.

I don't plan on pushing the issue, but I have to say that I don't agree.

Using a name suffix isn't clean, as you may include other non-binary ports
that may happen to have the word "binary" in their name.

A category allows you a cleaner approach as you can now represent that a
port is binary as an _attribute_ of the port, rather than overloading the
name.

This will make it easier to write port utilities and commands that target
binary ports.

We can easily add an alias that could let you do things like "port -v
binary_only" which would transparently do the "category:binary".

Additionally, if using a category, you can see the list of binary ports in
a clean way when browsing ports in the MacPorts website, it makes it easier
to do things like add an icon to signify binary only if a given port is in
the "binary" category, and not make possibly mistaken assumptions off of
the name.

On Thu, Aug 6, 2020 at 3:02 PM Ken Cunningham <
ken.cunningham.web...@gmail.com> wrote:

> category-only identifier is
>
> less clear and less obvious
> harder to remember how to search for
> name conflicts with a non-binary version (eg for newer systems that can
> build it)
>
> so far, name-suffix is winning on all fronts...with no downsides yet.
>
> K


Re: port "cask" -- installing prebuilt binaries

2020-08-06 Thread Craig Treleaven
> On Aug 6, 2020, at 11:29 AM, Ken Cunningham  
> wrote:
> 
> All valid points. I thought we had more-or-less got past the “should we” and 
> moved on to the “how should we”, 
> 
> I am not necessarily championing this, but people are submitting these, and 
> there is demand.here are nearly 4000 cask installer formulae on brew now. If 
> similar binary-only ports are going to be accepted, I was hoping for a 
> mechanism to identify and control them somewhat, for the reasons you mention 
> and more.
> 
>> 
>> So, pretend I don’t know how Homebrew’s cask system works.  (I don’t.)  
>> 
>> 1) As a user, what is the advantage of this kind of system versus other 
>> avenues for software (i.e. the Mac App Store or direct download of a dmg 
>> from the developer web site)?  
> 
> convenience, really. You can install something with a short command instead 
> of wading through finding the right installer on a website. Updates are 
> handled transparently. You can install 10 or 20 software packages onto a set 
> of systems with one command. You can make a list of the 75 software packages 
> you like to have and install them all with one command. You can tell your 
> grandmother how to install zoom with four words to paste into a terminal 
> instead of a complicated set of download and install instructions. 
> 
>> Doesn’t most such software include an auto-updater?  
> 
> Sometimes.
> 
>> If so, won’t that conflict with MacPorts update handling?  
> 
> Yes.
> 
>> A potential disadvantage would be the time lag from a new version being 
>> released until a new ‘cask’ is available, right?
> 
> Yes

If I understand correctly, this is a facility that would only benefit system 
administrators that have a fleet of Macs that are more-or-less configured the 
same.  It would help them to automate the process of installing and updating 
software whether the software is open-source or only available as a binary.  
(But not software that is only on the Mac App Store.)  Is that right?

That is a worthy audience but why is this something that MacPorts should 
address?  I believe there are already 'multiple device management systems' out 
there that support macOS.  And if Homebrew already provides 4,000 packages, why 
do we want to do the same?  Is it not possible to combine Homebrew’s casks with 
open source software installed by MacPorts?

MacPorts is not in competition with Homebrew.  Really.  The projects have 
different objectives and goals and that is perfectly fine.  Both communities 
have sufficient support to continue for the foreseeable future.  We don’t have 
to “defeat them” to “win” or vice versa.  We seem to be aiming to replicate 
their cask system.  Should we not be aiming to provide a system that is 
demonstrably *better* than what is currently available?

All that said, one more question.  As I now understand it, the idea is to 
download a binary-only installer (from the publisher’s web site) and launch it. 
 Someone still has to answer any and all dialogs that such installers always 
present.  So, it fact, the administrator has to sit at the machine and click 
“OK” ad nauseam.  Previously, I thought we were going to create a new binary 
image that would avoid such tedium.  Do I have this right?  Or is there some 
scripting trickery wrapped around the installer?

Personally, I don’t see any compelling reason why the MacPorts project should 
want to go in this direction.  The first paragraph on our homepage says:

"The MacPorts Project is an open-source community initiative to design an 
easy-to-use system for compiling, installing, and upgrading either 
command-line, X11 or Aqua based open-source software on the Mac operating 
system . To that end we provide the command-line 
driven MacPorts software package under a 3-Clause BSD License 
, and through it easy access to 
thousands of ports that greatly simplify 
 the task of compiling and installing 
 open-source software on your Mac.”

This effectively our charter and binary-only doesn’t fit.  The watchword at 
Apple is “a thousand no’s for every yes”.  IOW, focus on doing the right things 
really well and don’t get distracted trying to do all the other stuff just 
because it is possible.  Just because we _could_ provide binary packages 
doesn’t mean that we should.  

IMHO, YMMV, etc.

Craig



Re: port "cask" -- installing prebuilt binaries

2020-08-06 Thread Christopher Jones


> On 6 Aug 2020, at 8:02 pm, Ken Cunningham  
> wrote:
> 
> category-only identifier is
> 
> less clear and less obvious
> harder to remember how to search for
> name conflicts with a non-binary version (eg for newer systems that can build 
> it)
> 
> so far, name-suffix is winning on all fronts...with no downsides yet.

For you maybe…

For me, the port category is clearly the better approach. I fail to see how

port echo category:binary and installed

is any harder to run than

port -v installed | grep binar

If anything, your suggestion involves the use of a pipe and secondary command 
(grep) so could be argued to be more complicated.

FWIW, I dislike the proposal of using the port name here, if we are going to do 
this, when there is already a mechanism in macports (the categories) for 
exactly this use case.

Chris

> 
> K



smime.p7s
Description: S/MIME cryptographic signature


Re: port "cask" -- installing prebuilt binaries

2020-08-06 Thread Ken Cunningham
category-only identifier is

less clear and less obvious
harder to remember how to search for
name conflicts with a non-binary version (eg for newer systems that can build 
it)

so far, name-suffix is winning on all fronts...with no downsides yet.

K

Re: port "cask" -- installing prebuilt binaries

2020-08-06 Thread Arno Hautala


> On 6 Aug 2020, at 14:06, Ken Cunningham  
> wrote:
> 
> Along with name-suffix, a category was actually one of the first suggestions 
> in the original post 50 posts ago, but how does one see which ports you have 
> installed that are members of that category?
> 
> with a name suffix, easy.
> 
> port -v installed | grep binary


Assuming a category of “binary”:

> port echo category:binary and installed


There’s also `search` for identifying ports in a specific category:

> port search --category binary


-- 
arno  s  hautala/-|   a...@alum.wpi.edu

pgp b2c9d448



Re: port "cask" -- installing prebuilt binaries

2020-08-06 Thread Ken Cunningham


> On Aug 6, 2020, at 10:50 AM, Herby G  wrote:

> a "binary"/"binary_only" (or a new MacPorts specific name for these) as a 
> *category*, just as we have the "aqua" category
> in this way, binary-only apps can continue to be represented in respected 
> categories with "binary" as their secondary or third category

Along with name-suffix, a category was actually one of the first suggestions in 
the original post 50 posts ago, but how does one see which ports you have 
installed that are members of that category?

with a name suffix, easy.

port -v installed | grep binary



> a portgroup with the name selected above, which as mentioned earlier, could 
> perhaps have:
> a blank build section and use_configure disabled since binary ports are 
> extract-only
> automatic support to destroot and install .app bundles into the 
> $prefix/Applications dir
> by default turning on any options that might exist to disable 
> caching/mirroring of the distfile, and disable attempting to check MacPorts 
> official mirrors for the same.
> The port maintainer should be able to disable this to return to standard 
> MacPorts caching + mirroring behavior at his/her discretion
> perhaps lint additions to warn the user if they are indeed doing a build or 
> configure on a port that has been marked as being in the "binary" category
> In this way, we don't need to have major changes to the MacPorts structure, 
> but now can track binary-only ports.
> If any special sub-commands need to be added for binary-only ports, these 
> commands need only target ports in this "binary" category.
> 


I think all this other stuff would be best implemented as simple templates for 
binary-only ports, one for dmg, one for pkg, one for simple xz/tar.gz./etc 
usual compressed files.

I have come to basically hate/strongly dislike PortGroups, as they make a total 
mess out of understanding what the H*LL is going on, esp for new contributors.

K



Re: port "cask" -- installing prebuilt binaries

2020-08-06 Thread Herby G
Adding my 2 cents to this as well (why not).

I personally think a conservative approach makes sense here.

We can keep the entirety of MacPorts structure exactly the way it is today,
and add 3 things:

   - a "binary"/"binary_only" (or a new MacPorts specific name for these)
   as a *category*, just as we have the "aqua" category
  - in this way, binary-only apps can continue to be represented in
  respected categories with "binary" as their secondary or third category
   - a portgroup with the name selected above, which as mentioned earlier,
   could perhaps have:
  - a blank build section and use_configure disabled since binary ports
  are extract-only
  - automatic support to destroot and install .app bundles into
  the $prefix/Applications dir
  - by default turning on any options that might exist to disable
  caching/mirroring of the distfile, and disable attempting to
check MacPorts
  official mirrors for the same.
 - The port maintainer should be able to disable this to return to
 standard MacPorts caching + mirroring behavior at his/her discretion
  - perhaps lint additions to warn the user if they are indeed doing a
   build or configure on a port that has been marked as being in the "binary"
   category

In this way, we don't need to have major changes to the MacPorts structure,
but now can track binary-only ports.
If any special sub-commands need to be added for binary-only ports, these
commands need only target ports in this "binary" category.

On Thu, Aug 6, 2020 at 12:02 PM Ken Cunningham <
ken.cunningham.web...@gmail.com> wrote:

>
>
> > On Aug 6, 2020, at 8:23 AM, Arno Hautala  wrote:
> >
> >> On 6 Aug 2020, at 10:10, Ken Cunningham <
> ken.cunningham.web...@gmail.com> wrote:
> >>
> >> How about I float a suggestion? We could append "_binary" to the name.
> Otherwise leave the categories, notes, etc as they are now.
> >
> > Could all of the “cask” ports be put in a second ports tree?
>
> Very very difficult to implement. Not impossible. You’d have to have some
> new logic in the “port” command, and a new keyword like “cask” or “binary”
> or some better keyword, that searches a new ports tree. Realistically — not
> likely to ever happen, and if it does, not likely to happen any time soon.
>
>
> > Any source-based ports that wanted to depend on those would also need to
> go in that tree or at least couldn’t be in the source-only tree. The tree
> wouldn’t ship by default, or at least would have to be enabled
> (“uncommented”) in a config file.
>
> Now extremely difficult to implement.
>
> >
> > Personally, I dislike the idea of a port name suffix, but an attribute
> that could be searched for is a good idea.
>
>
> When you type “port search XYZ”, the idea is to somehow immediately see
> that XYZ is a prebuilt binary you might or might not want install. If it’s
> not in the name, I don’t see how to do it.
>
> When you want to know what prebuilt binaries you have installed so you can
> purge them all, you need to have a simple command you can type to identify
> them all.
>
> port -v installed | grep binary
>
> is pretty simple. Hopefully someone on this list knows a better way, or
> perhaps there is a way to make a command internal to “port” do it bettter.
>
> K


Re: port "cask" -- installing prebuilt binaries

2020-08-06 Thread Ken Cunningham



> On Aug 6, 2020, at 8:23 AM, Arno Hautala  wrote:
> 
>> On 6 Aug 2020, at 10:10, Ken Cunningham  
>> wrote:
>> 
>> How about I float a suggestion? We could append "_binary" to the name. 
>> Otherwise leave the categories, notes, etc as they are now. 
> 
> Could all of the “cask” ports be put in a second ports tree?

Very very difficult to implement. Not impossible. You’d have to have some new 
logic in the “port” command, and a new keyword like “cask” or “binary” or some 
better keyword, that searches a new ports tree. Realistically — not likely to 
ever happen, and if it does, not likely to happen any time soon.


> Any source-based ports that wanted to depend on those would also need to go 
> in that tree or at least couldn’t be in the source-only tree. The tree 
> wouldn’t ship by default, or at least would have to be enabled 
> (“uncommented”) in a config file.

Now extremely difficult to implement.

> 
> Personally, I dislike the idea of a port name suffix, but an attribute that 
> could be searched for is a good idea.


When you type “port search XYZ”, the idea is to somehow immediately see that 
XYZ is a prebuilt binary you might or might not want install. If it’s not in 
the name, I don’t see how to do it.

When you want to know what prebuilt binaries you have installed so you can 
purge them all, you need to have a simple command you can type to identify them 
all.

port -v installed | grep binary

is pretty simple. Hopefully someone on this list knows a better way, or perhaps 
there is a way to make a command internal to “port” do it bettter.

K

Re: port "cask" -- installing prebuilt binaries

2020-08-06 Thread Arno Hautala
> On 6 Aug 2020, at 11:03, Craig Treleaven  wrote:
> 
> So, pretend I don’t know how Homebrew’s cask system works.  (I don’t.)  
> 
> 1) As a user, what is the advantage of this kind of system versus other 
> avenues for software (i.e. the Mac App Store or direct download of a dmg from 
> the developer web site)?  Doesn’t most such software include an auto-updater? 
>  If so, won’t that conflict with MacPorts update handling?  A potential 
> disadvantage would be the time lag from a new version being released until a 
> new ‘cask’ is available, right?

It basically becomes _an_ App Store for software not in _the_ App Store. I 
don’t know how homebrew handles this either, but I think installing the 
software with MacPorts and then updating it on it’s own would either break the 
MacPorts install, or MacPorts would need to try to detect this and “repair” it 
by reinstalling or updating the MacPorts database with the updating files. 
Neither situation sounds ideal. I don’t know that there’s going to be an easy 
way to disable the auto-update features other than user education.

> 2) My impression is that developers of commercial software would, in many 
> cases, NOT want a third party (like MacPorts) to be distributing their 
> software.  From their perspective, a third party introduces considerably more 
> risk that users may end up with maliciously altered software.  Can we not 
> expect to get takedown notices from certain publishers?

I can both see a difference and at the same time question what the difference 
is between mirroring source archives and binary archives. I think a clear line 
could be whether the software is released with any sort of open source license.

> 3) Is the MacPorts mirror network willing to contribute bandwidth for 
> distributing non-open source software?  Will we sour our relations with some 
> of the mirrors if we use/abuse their bandwidth this way?  Why do we want to 
> use our bandwidth that way?

I would expect this would be reasonably similar to how source archives are 
currently handled. Have any developers asked for source not to be mirrored?

-- 
arno  s  hautala/-|   a...@alum.wpi.edu

pgp b2c9d448

Re: port "cask" -- installing prebuilt binaries

2020-08-06 Thread Ken Cunningham
All valid points. I thought we had more-or-less got past the “should we” and 
moved on to the “how should we”, 

I am not necessarily championing this, but people are submitting these, and 
there is demand.here are nearly 4000 cask installer formulae on brew now. If 
similar binary-only ports are going to be accepted, I was hoping for a 
mechanism to identify and control them somewhat, for the reasons you mention 
and more.

> 
> So, pretend I don’t know how Homebrew’s cask system works.  (I don’t.)  
> 
> 1) As a user, what is the advantage of this kind of system versus other 
> avenues for software (i.e. the Mac App Store or direct download of a dmg from 
> the developer web site)?  

convenience, really. You can install something with a short command instead of 
wading through finding the right installer on a website. Updates are handled 
transparently. You can install 10 or 20 software packages onto a set of systems 
with one command. You can make a list of the 75 software packages you like to 
have and install them all with one command. You can tell your grandmother how 
to install zoom with four words to paste into a terminal instead of a 
complicated set of download and install instructions. 

> Doesn’t most such software include an auto-updater?  

Sometimes.

> If so, won’t that conflict with MacPorts update handling?  

Yes.

> A potential disadvantage would be the time lag from a new version being 
> released until a new ‘cask’ is available, right?

Yes

> 
> 2) My impression is that developers of commercial software would, in many 
> cases, NOT want a third party (like MacPorts) to be distributing their 
> software.  From their perspective, a third party introduces considerably more 
> risk that users may end up with maliciously altered software.  Can we not 
> expect to get takedown notices from certain publishers?

I suggest if we agree to do this, we do not mirror any installers partly for 
this reason.  Each would have it’s SHA code, so security would be as good as 
whatever good security is now. 

Software developers seem OK with "brew cask”, based on their being 4000 of them 
so far, and I have seen software people open PRs to update their own casks when 
they release a new version — so in general, they seem OK. Dunno about details 
beyond that.


> 
> 3) Is the MacPorts mirror network willing to contribute bandwidth for 
> distributing non-open source software?  

I doubt it. There is no need to mirror these.

> Will we sour our relations with some of the mirrors if we use/abuse their 
> bandwidth this way?

Probably.

>  Why do we want to use our bandwidth that way?

We don’t.

> 
> Craig



Re: port "cask" -- installing prebuilt binaries

2020-08-06 Thread Arno Hautala
> On 6 Aug 2020, at 10:10, Ken Cunningham  
> wrote:
> 
> How about I float a suggestion? We could append "_binary" to the name. 
> Otherwise leave the categories, notes, etc as they are now. 

Could all of the “cask” ports be put in a second ports tree? Any source-based 
ports that wanted to depend on those would also need to go in that tree or at 
least couldn’t be in the source-only tree. The tree wouldn’t ship by default, 
or at least would have to be enabled (“uncommented”) in a config file.

The downside to this is that I could see it quickly becoming like the Fink 
“unstable” tree if something popular is only available as a “binary” and that 
or something that depends on it becomes popular.

Personally, I dislike the idea of a port name suffix, but an attribute that 
could be searched for is a good idea.

-- 
arno  s  hautala/-|   a...@alum.wpi.edu

pgp b2c9d448




Re: port "cask" -- installing prebuilt binaries

2020-08-06 Thread Craig Treleaven
> On Aug 6, 2020, at 10:10 AM, Ken Cunningham  
> wrote:
> 
> How about I float a suggestion? We could append "_binary" to the name. 
> Otherwise leave the categories, notes, etc as they are now. 
> 
> So a port that installs the Zoom teleconferencing application would be called 
> "zoom_binary". (We can't use "zoom-binary" because variants us the + and - to 
> do their thing.) You could find all your installed binaries with a simple 
> grep...
> 
> Open to a more descriptive, shorter suffix if anyone is thinking of one.
> 
> Once we find metadata we agree on, then two more points to work out. 
> 1. Should we mirror the binary installers (no, I'd say)?
> 2. We should require a mechanism to remove all traces of the software when 
> uninstalling it, I think.
> 

So, pretend I don’t know how Homebrew’s cask system works.  (I don’t.)  

1) As a user, what is the advantage of this kind of system versus other avenues 
for software (i.e. the Mac App Store or direct download of a dmg from the 
developer web site)?  Doesn’t most such software include an auto-updater?  If 
so, won’t that conflict with MacPorts update handling?  A potential 
disadvantage would be the time lag from a new version being released until a 
new ‘cask’ is available, right?

2) My impression is that developers of commercial software would, in many 
cases, NOT want a third party (like MacPorts) to be distributing their 
software.  From their perspective, a third party introduces considerably more 
risk that users may end up with maliciously altered software.  Can we not 
expect to get takedown notices from certain publishers?

3) Is the MacPorts mirror network willing to contribute bandwidth for 
distributing non-open source software?  Will we sour our relations with some of 
the mirrors if we use/abuse their bandwidth this way?  Why do we want to use 
our bandwidth that way?

Craig



Re: port "cask" -- installing prebuilt binaries

2020-08-06 Thread Ken Cunningham
How about I float a suggestion? We could append "_binary" to the name. 
Otherwise leave the categories, notes, etc as they are now. 

So a port that installs the Zoom teleconferencing application would be called 
"zoom_binary". (We can't use "zoom-binary" because variants us the + and - to 
do their thing.) You could find all your installed binaries with a simple 
grep...

Open to a more descriptive, shorter suffix if anyone is thinking of one.

Once we find metadata we agree on, then two more points to work out. 
1. Should we mirror the binary installers (no, I'd say)?
2. We should require a mechanism to remove all traces of the software when 
uninstalling it, I think.

K






Re: port "cask" -- installing prebuilt binaries

2020-08-05 Thread Christopher Chavez




On 8/5/2020 8:12 AM, Lothar Haeger wrote:



Am 05.08.2020 um 14:52 schrieb Georges Martin mailto:jrjsm...@gmail.com>>:

If MacPorts starts to mix both approaches, I worry we may end up
having (open source) packages depending on closed source, binary
packages. And have less control on ensuring a consistent, compatible
distribution.


That's already the case e.g. with port that use one of the openjdk*
ports as dependency. Also, basically all ports using portgroup java
depend on a binary java distribution. I'd rather see the dependencies
being part of the macports tree, so we can at least control versions and
variants, than trying to keep the depending ports out of MP.

To minimize this risk, I guess Macports could exclude binary ports from
being used as dependencies to other ports. As soon as such ports are
reliably recognizable, that is. There should be a way to allow such
dependencies on a port by port basis, though.



While OpenJDK is not closed-source, the sentiment here appears to be
that MacPorts is treating it as if it were closed-source.

Regarding Java ports specifically: they normally do not specify an exact
Java distribution to use. By using the Java portgroup (which they should
if they don't already), they only specify what versions of Java they're
compatible with, and a fallback Java distribution to install only if a
compatible Java distribution isn't already installed (whether through
MacPorts, or outside of it—e.g. Oracle JDK). Someone could potentially
build Java from source if they wanted to, and then MacPorts would use
that as the Java distribution for these ports.

I would keep in mind the tradeoff between the limited effort available
from maintainers/volunteers and "control" (often entailing duplication
of effort).


Re: port "cask" -- installing prebuilt binaries

2020-08-05 Thread Lothar Haeger

> Am 05.08.2020 um 15:11 schrieb Ken Cunningham 
> :
> 
> the indicator should not be a variant. Variant indicates an option to install 
> one way or the other, and a given port on a given system will not have such 
> an option.
Well, it's just a suggestion, here's another one: I suggested a port group in 
an earlier discussion about this topic, still think it could be useful. This PG 
would then

- add a well-known category or some other identifier
- disable patch/config/build phases by default
- provide a default destroot phase to handle the most common use cases (*.app 
from the root of an DMG distfile goes into /Apps/MacPorts, while a cmd line 
tool would go into /opt/local/bin. etc)



Re: port "cask" -- installing prebuilt binaries

2020-08-05 Thread Christopher Chavez

On 8/4/2020 12:52 PM, Ruben Di Battista wrote:

One of the reasons I chose Macports for is the fact it builds its own
tree from source and it ships basically open source only software.


I think there have been cases where MacPorts' preference/insistence on
building from source, even when prebuilt binaries are available, may
have been excessive and/or eventually had negative consequences. One set
of examples is the various ports for Java libraries, many of which are
now outdated or broken. They all appear to be open source, but many also
have a prebuilt non-architecture-specific binary (.jar file) readily
available. Many of these ports no longer build from source with newer
JDK versions (even for some up-to-date ports), but the prebuilt binaries
for these ports often still work with newer Java runtime versions. Had
these ports used the upstream binary instead of building from source, I
wonder if fewer of them would be broken, and that they would've actually
been easier to keep updated and working.


Re: port "cask" -- installing prebuilt binaries

2020-08-05 Thread Lothar Haeger

> Am 05.08.2020 um 14:52 schrieb Georges Martin :
> 
> If MacPorts starts to mix both approaches, I worry we may end up having (open 
> source) packages depending on closed source, binary packages. And have less 
> control on ensuring a consistent, compatible distribution.


That's already the case e.g. with port that use one of the openjdk* ports as 
dependency. Also, basically all ports using portgroup java depend on a binary 
java distribution. I'd rather see the dependencies being part of the macports 
tree, so we can at least control versions and variants, than trying to keep the 
depending ports out of MP.

To minimize this risk, I guess Macports could exclude binary ports from being 
used as dependencies to other ports. As soon as such ports are reliably 
recognizable, that is. There should be a way to allow such dependencies on a 
port by port basis, though.




Re: port "cask" -- installing prebuilt binaries

2020-08-05 Thread Ken Cunningham
> Sure, let's repeat this a few more times until everyone *really* got that. :-)

That's exactly why the indicator should not be a variant. Variant indicates an 
option to install one way or the other, and a given port on a given system will 
not have such an option.

K


Re: port "cask" -- installing prebuilt binaries

2020-08-05 Thread Lothar Haeger



> Am 05.08.2020 um 14:28 schrieb Ken Cunningham 
> :
> 
> I wasn't imagining a variant for binary-only installs.

That's just an idea on how this could be presented to users. Also an easy way 
to identify ports that are basically binary redistribution (`port list 
variant:prebuilt`). Also would allow to provide a single port both ways, and an 
easy way to configure user preference via existing mechanisms.

> If a port can be built, we should build it.

Sure, let's repeat this a few more times until everyone *really* got that. :-) 

> But for things we can't build, (like Zoom or Microsoft Word) installing a 
> binary is the only option and some ports of this type seem to be what people 
> are submitting recently.

There may be ports where providing it both ways makes sense (e.g. some ports 
may not build on certain macOS/XCode versions but on others), though, so if we 
can find a way to provide both in parallel, that could help.

> This is fine -- some work to maintain -- but I was just wondering if we might 
> coalesce around a scheme to make it obvious somehow to users that these were 
> different animals than our usual ports.

I totally agree with this, just think this is merely a starting point to 
optimize creation and handling of such ports.

Re: port "cask" -- installing prebuilt binaries

2020-08-05 Thread Georges Martin


> Le 5 août 2020 à 12:52, Lothar Haeger  a écrit :
> 
> Even if MacPorts also provided closed source stuff it would still be your 
> choice to install or not install it.

Problem is with dependencies on closed source packages where you have no choice.

If I use apt or dnf as part of a Linux distribution, or MacPorts as an add-in 
package manager to macOS, I know that I can rely on a consistent and compatible 
set of open source packages and their dependencies. If necessary, I know I can 
contribute, fix, upgrade or add to those packages.

If I use chocolatey on Windows or mas on macOS, I know they are more « 
command-line interface App Stores » than package managers, they have less 
dependencies to manage and consistency to ensure, I have less to no control on 
the building/packaging and... that’s ok.

If MacPorts starts to mix both approaches, I worry we may end up having (open 
source) packages depending on closed source, binary packages. And have less 
control on ensuring a consistent, compatible distribution.

Just my 2 euro cents... :-q

Re: port "cask" -- installing prebuilt binaries

2020-08-05 Thread Ken Cunningham
I wasn't imagining a variant for binary-only installs. If a port can be built, 
we should build it.

But for things we can't build, (like Zoom or Microsoft Word) installing a 
binary is the only option and some ports of this type seem to be what people 
are submitting recently.

This is fine -- some work to maintain -- but I was just wondering if we might 
coalesce around a scheme to make it obvious somehow to users that these were 
different animals than our usual ports.

K

Re: port "cask" -- installing prebuilt binaries

2020-08-05 Thread Lothar Haeger

> Am 04.08.2020 um 19:52 schrieb Ruben Di Battista :
> 
> So my take here is to not provide pre-built binaries packages if not strictly 
> unavoidable, like for the osxfuse project (since it was open source before). 

I think we all agree that building from source is preferred, so nothing should 
change for OSS ports. There is no reason to fear anything you get from MP now 
would no longer be available in the future. OSS would still be built from 
source.

I could imagine to implement a binary re-distribution scheme as a well-known 
variant e.g. "+prebuilt", so that it became possible for a single port to 
support both build/distribution schemes. That way dependencies could simply 
reference the port as usual, while each user could choose the variant she 
prefers. It would not even be necessary to decide on one exclusive scheme for 
each port, but have both options in parallel, wherever that might make sense. 
One could then also set "+prebuild" (or "-prebuilt") in variants.conf, so if a 
prebuilt binary is available, it would be used, if not everything would work as 
it does now.

> One of the reasons I chose Macports for is the fact it builds its own tree 
> from source and it ships basically open source only software. 

Not sure why "ships basically only open source" would be a reason to prefer 
MacPorts over HomeBrew, as long as either of them provided what you want (OSS) 
in the way you want it (build from source). Even if MacPorts also provided 
closed source stuff it would still be your choice to install or not install it. 
I see no reason how providing more binary-only ports in a more formalized way 
would make MacPorts a lesser choice.



Re: port "cask" -- installing prebuilt binaries

2020-08-04 Thread Ruben Di Battista
Ehi, sorry I was not clear enough. Sometimes I’m very bad at expressing
what I have in mind, I’m sorry.

Hmm, I hadn't heard about that, but I don't run 10.14+ other than for
testing.

https://github.com/osxfuse/osxfuse/issues/590

Huh? Are you suggesting *disallowing* the use of precompiled binaries?
That would be crazy.

No, no. What I meant was to limit “cask behavior”, that for what I
understand is “binary-only” distribution, to only the ports that really
really are needed (like, for my use case, osxfuse).

The current approach seems reasonable and philosophically closer to the
FOSS movement (i.e. shipping default variants distributable ports as
binaries…). Maybe expanding the matrix of builds to more variants would be
also cool, but I imagine that this would require an insane amount of space
on the servers...

  _
-. .´  |
  ',  ;|∞∞
˜˜ |∞ RdB
,.,|∞∞
  .'   '.  |
-'   `’
https://rdb.is


On 4 August 2020 at 22:35:47, Fred Wright (f...@fwright.net) wrote:


On Wed, 29 Jul 2020, Daniel J. Luke wrote:
> On Jul 29, 2020, at 9:30 PM, Fred Wright  wrote:
[...]
>> Personally, I'd prefer the MacPorts approach if it were less stingy
>> with the binary archives. Ideally, one should be *able* to build from
>> source, but not be *required* to do so.
>
> How is it stingy? We have binary archives for everything that the
> buildbots can build that the licenses allow us to distribute, right?

Aside from the usual issues with non-default variants and certain old OS
versions, the main problem seems to be what appears to be an overly strict
interpretation of "distributable".

As a random example, consider ffmpeg. The license for ffmpeg shows as
GPL-2+. Although GPL prohibits binary-only distributions at the "meta
level", it does *not* demand that sources be bundled with the binaries.
It simply says that if they're not, there has to be clear information
available to the user on how to obtain the sources. It doesn't even
demand a working build procedure, just the sources. It might get a bit
fuzzy in cases where the sources are patched, in which case it's not 100%
clear that providing the original source plus the patches is adequate, or
whether it's necessary to provide the actual patched sources (though the
two are certainly morally equivalent), but in any case, MacPorts clearly
does both. Anyone can get the upstream source archive(s) via "port
fetch", get the sources in tree form via "port extract", and get the
patched sources via "port patch". I don't see how this fails to meet the
GPL requirements for any MacPorts user.

Now if one were to be really paranoid, one might consider that providing
binaries on servers that are accessible by means other than MacPorts could
constitute a GPL violation, due to not having the "clear path to sources"
that MacPorts provides. But if that's a concern, it could be easily
addressed by including a README.sources file in every directory on the
archive servers, either pointing to the corresponding source on a distfile
mirror (or directly upstream if necessary), or perhaps just pointing to
the MacPorts homepage.

And to pick a random sub-example, Debian offers ffmpeg as a binary
package.

> port, by default, will use the binary archives unless you tell it to
> build from source instead.

BTW, on an only mildly related note, I've seen a number of cases lately
where it successfully fetches a binary archive and than fails to fetch the
corresponding checksum file. It seems that it gets only one chance to do
this, and only from the same mirror where it fetched the archive. It's
become common for me to need a "cleanup pass" after doing "upgrade
outdated" across my VMs, where I manually retry the failed upgrades.

On Tue, 4 Aug 2020, Ruben Di Battista wrote:

> There's is one compelling need for having "binary only" install, and that
> is for the port "osxfuse", that is currently broken for 10.14+.
>
> There was a discussion about it on the Github project about the choice of
> making it close closed source... Nonetheless it would be useful to have
it
> in order to provide things like fuse file systems and so on.

Hmm, I hadn't heard about that, but I don't run 10.14+ other than for
testing.

I was involved in fixing some osxfuse bugs right around the time that
10.10 came out, making kext signature checking mandatory. On 10.9, it
warns you about an unsigned kext, but you can tell it to proceed. The fix
at the time (for the relevant OS versions) was to have MacPorts fetch the
binary distribution and extract the signed prebuilt kext from it, while
building the rest from sources. This is done starting with 10.9, to avoid
the dialog box, even though it's not strictly necessary until 10.10+.

It sounds like some more tightening of the security noose has caused
additional problems for osxfuse.

On Tue, 4 Aug 2020, Ruben Di Battista wrote:

> So my take here is to not provide pre-built binaries packages if not
> strictly unavoidable, like for 

Re: port "cask" -- installing prebuilt binaries

2020-08-04 Thread Mark Anderson
If I understand correctly the reason ffmpeg is a problem is because you can
build it in a "non-free" way. In fact, I think the variant is +non_free.

I used to use homebrew cask and macports, but then they rolled cask into
homebrew and I'll be damned if I install regular old homebrew. I'd love a
way to do the same with ports, but I honestly have no idea exactly how it
would work and how to differentiate the FOSS stuff from the binary only
stuff.

—Mark


On Tue, Aug 4, 2020 at 4:35 PM Fred Wright  wrote:

>
> On Wed, 29 Jul 2020, Daniel J. Luke wrote:
> > On Jul 29, 2020, at 9:30 PM, Fred Wright  wrote:
> [...]
> >> Personally, I'd prefer the MacPorts approach if it were less stingy
> >> with the binary archives.  Ideally, one should be *able* to build from
> >> source, but not be *required* to do so.
> >
> > How is it stingy? We have binary archives for everything that the
> > buildbots can build that the licenses allow us to distribute, right?
>
> Aside from the usual issues with non-default variants and certain old OS
> versions, the main problem seems to be what appears to be an overly strict
> interpretation of "distributable".
>
> As a random example, consider ffmpeg.  The license for ffmpeg shows as
> GPL-2+.  Although GPL prohibits binary-only distributions at the "meta
> level", it does *not* demand that sources be bundled with the binaries.
> It simply says that if they're not, there has to be clear information
> available to the user on how to obtain the sources.  It doesn't even
> demand a working build procedure, just the sources.  It might get a bit
> fuzzy in cases where the sources are patched, in which case it's not 100%
> clear that providing the original source plus the patches is adequate, or
> whether it's necessary to provide the actual patched sources (though the
> two are certainly morally equivalent), but in any case, MacPorts clearly
> does both.  Anyone can get the upstream source archive(s) via "port
> fetch", get the sources in tree form via "port extract", and get the
> patched sources via "port patch".  I don't see how this fails to meet the
> GPL requirements for any MacPorts user.
>
> Now if one were to be really paranoid, one might consider that providing
> binaries on servers that are accessible by means other than MacPorts could
> constitute a GPL violation, due to not having the "clear path to sources"
> that MacPorts provides.  But if that's a concern, it could be easily
> addressed by including a README.sources file in every directory on the
> archive servers, either pointing to the corresponding source on a distfile
> mirror (or directly upstream if necessary), or perhaps just pointing to
> the MacPorts homepage.
>
> And to pick a random sub-example, Debian offers ffmpeg as a binary
> package.
>
> > port, by default, will use the binary archives unless you tell it to
> > build from source instead.
>
> BTW, on an only mildly related note, I've seen a number of cases lately
> where it successfully fetches a binary archive and than fails to fetch the
> corresponding checksum file.  It seems that it gets only one chance to do
> this, and only from the same mirror where it fetched the archive.  It's
> become common for me to need a "cleanup pass" after doing "upgrade
> outdated" across my VMs, where I manually retry the failed upgrades.
>
> On Tue, 4 Aug 2020, Ruben Di Battista wrote:
>
> > There's is one compelling need for having "binary only" install, and that
> > is for the port "osxfuse", that is currently broken for 10.14+.
> >
> > There was a discussion about it on the Github project about the choice of
> > making it close closed source... Nonetheless it would be useful to have
> it
> > in order to provide things like fuse file systems and so on.
>
> Hmm, I hadn't heard about that, but I don't run 10.14+ other than for
> testing.
>
> I was involved in fixing some osxfuse bugs right around the time that
> 10.10 came out, making kext signature checking mandatory.  On 10.9, it
> warns you about an unsigned kext, but you can tell it to proceed.  The fix
> at the time (for the relevant OS versions) was to have MacPorts fetch the
> binary distribution and extract the signed prebuilt kext from it, while
> building the rest from sources.  This is done starting with 10.9, to avoid
> the dialog box, even though it's not strictly necessary until 10.10+.
>
> It sounds like some more tightening of the security noose has caused
> additional problems for osxfuse.
>
> On Tue, 4 Aug 2020, Ruben Di Battista wrote:
>
> > So my take here is to not provide pre-built binaries packages if not
> > strictly unavoidable, like for the osxfuse project (since it was open
> > source before).
>
> Huh?  Are you suggesting *disallowing* the use of precompiled binaries?
> That would be crazy.
>
> The amount of both human and computer time that's wasted rebuilding the
> same binaries from the same sources with the same options is humongous.
>
> Another thing that's often overlooked is that 

Re: port "cask" -- installing prebuilt binaries

2020-08-04 Thread Fred Wright



On Wed, 29 Jul 2020, Daniel J. Luke wrote:

On Jul 29, 2020, at 9:30 PM, Fred Wright  wrote:

[...]
Personally, I'd prefer the MacPorts approach if it were less stingy 
with the binary archives.  Ideally, one should be *able* to build from 
source, but not be *required* to do so.


How is it stingy? We have binary archives for everything that the 
buildbots can build that the licenses allow us to distribute, right?


Aside from the usual issues with non-default variants and certain old OS 
versions, the main problem seems to be what appears to be an overly strict 
interpretation of "distributable".


As a random example, consider ffmpeg.  The license for ffmpeg shows as 
GPL-2+.  Although GPL prohibits binary-only distributions at the "meta 
level", it does *not* demand that sources be bundled with the binaries. 
It simply says that if they're not, there has to be clear information 
available to the user on how to obtain the sources.  It doesn't even 
demand a working build procedure, just the sources.  It might get a bit 
fuzzy in cases where the sources are patched, in which case it's not 100% 
clear that providing the original source plus the patches is adequate, or 
whether it's necessary to provide the actual patched sources (though the 
two are certainly morally equivalent), but in any case, MacPorts clearly 
does both.  Anyone can get the upstream source archive(s) via "port 
fetch", get the sources in tree form via "port extract", and get the 
patched sources via "port patch".  I don't see how this fails to meet the 
GPL requirements for any MacPorts user.


Now if one were to be really paranoid, one might consider that providing 
binaries on servers that are accessible by means other than MacPorts could 
constitute a GPL violation, due to not having the "clear path to sources" 
that MacPorts provides.  But if that's a concern, it could be easily 
addressed by including a README.sources file in every directory on the 
archive servers, either pointing to the corresponding source on a distfile 
mirror (or directly upstream if necessary), or perhaps just pointing to 
the MacPorts homepage.


And to pick a random sub-example, Debian offers ffmpeg as a binary 
package.


port, by default, will use the binary archives unless you tell it to 
build from source instead.


BTW, on an only mildly related note, I've seen a number of cases lately 
where it successfully fetches a binary archive and than fails to fetch the 
corresponding checksum file.  It seems that it gets only one chance to do 
this, and only from the same mirror where it fetched the archive.  It's 
become common for me to need a "cleanup pass" after doing "upgrade 
outdated" across my VMs, where I manually retry the failed upgrades.


On Tue, 4 Aug 2020, Ruben Di Battista wrote:


There's is one compelling need for having "binary only" install, and that
is for the port "osxfuse", that is currently broken for 10.14+.

There was a discussion about it on the Github project about the choice of
making it close closed source... Nonetheless it would be useful to have it
in order to provide things like fuse file systems and so on.


Hmm, I hadn't heard about that, but I don't run 10.14+ other than for 
testing.


I was involved in fixing some osxfuse bugs right around the time that 
10.10 came out, making kext signature checking mandatory.  On 10.9, it 
warns you about an unsigned kext, but you can tell it to proceed.  The fix 
at the time (for the relevant OS versions) was to have MacPorts fetch the 
binary distribution and extract the signed prebuilt kext from it, while 
building the rest from sources.  This is done starting with 10.9, to avoid 
the dialog box, even though it's not strictly necessary until 10.10+.


It sounds like some more tightening of the security noose has caused 
additional problems for osxfuse.


On Tue, 4 Aug 2020, Ruben Di Battista wrote:


So my take here is to not provide pre-built binaries packages if not
strictly unavoidable, like for the osxfuse project (since it was open
source before).


Huh?  Are you suggesting *disallowing* the use of precompiled binaries? 
That would be crazy.


The amount of both human and computer time that's wasted rebuilding the 
same binaries from the same sources with the same options is humongous.


Another thing that's often overlooked is that installing precompiled 
binaries, especially when signed, is more secure than building from 
sources.  This is because the attack surface of the tools required to 
install binaries is far smaller than the attack surface of the toolchains 
needed to build them.  If you think this is purely hypothetical, consider 
the early Unix hack that implemented a backdoor that didn't appear in any 
source code.



One of the reasons I chose Macports for is the fact it builds its own tree
from source and it ships basically open source only software.


Anyone is free to configure it to do that.  I suppose there could be a 
strict source-only option that flatly 

Re: port "cask" -- installing prebuilt binaries

2020-08-04 Thread Nils Breunese
I already maintain some ports that install binaries from upstream, like the 
openjdk* ports, which are very complex to build from source and take many, many 
hours to build.

I also maintain some Go ports that build from source, but that are not 
available on some older OS versions because the go compiler doesn’t seem to be 
available for those older OS versions. See for instance 
https://ports.macports.org/port/kustomize and 
https://ports.macports.org/port/skaffold/summary. I wonder if those application 
would work on those older OS versions if I would switch those ports to install 
binaries from upstream.

Nils.

> Op 4 aug. 2020 om 19:48 heeft Ruben Di Battista  
> het volgende geschreven:
> 
> 
> There's is one compelling need for having "binary only" install, and that is 
> for the port "osxfuse", that is currently broken for 10.14+.
> 
> There was a discussion about it on the Github project about the choice of 
> making it close closed source... Nonetheless it would be useful to have it in 
> order to provide things like fuse file systems and so on.
> 
> 
> 
>> On Tue, 4 Aug 2020, 15:38 Lothar Haeger,  wrote:
>> 
>> 
>>> Am 30.07.2020 um 08:03 schrieb Ken Cunningham 
>>> :
>>> I only raise the idea as people are already doing this, and submitting such 
>>> ports, and before we have too many, there is an opportunity to say how it 
>>> should best be done (custom category, naming convention, etc).
>> Reminds me of an earlier conversation at 
>> https://github.com/macports/macports-ports/pull/6767#discussion_r402584006 
>> 
>> I do see some benefits is formalizing binary-only ports and to adapt the 
>> build and distribution scheme for it. Could save resources and development 
>> time and make those ports easily recognizable for those who care about the 
>> different way it was built.
>> 
>> 


Re: port "cask" -- installing prebuilt binaries

2020-08-04 Thread Ruben Di Battista
So my take here is to not provide pre-built binaries packages if not
strictly unavoidable, like for the osxfuse project (since it was open
source before).

One of the reasons I chose Macports for is the fact it builds its own tree
from source and it ships basically open source only software.

Otherwise there would be no difference with homebrew and it would make
little sense to prefer Macports UX, that is arguably worse than Homebrew's
IMHO (especially Ruby DSL formula vs TCL Portfile).

On Tue, 4 Aug 2020, 19:47 Ruben Di Battista, 
wrote:

> There's is one compelling need for having "binary only" install, and that
> is for the port "osxfuse", that is currently broken for 10.14+.
>
> There was a discussion about it on the Github project about the choice of
> making it close closed source... Nonetheless it would be useful to have it
> in order to provide things like fuse file systems and so on.
>
>
>
> On Tue, 4 Aug 2020, 15:38 Lothar Haeger,  wrote:
>
>>
>>
>> Am 30.07.2020 um 08:03 schrieb Ken Cunningham <
>> ken.cunningham.web...@gmail.com>:
>>
>> I only raise the idea as people are already doing this, and submitting such 
>> ports, and before we have too many, there is an opportunity to say how it 
>> should best be done (custom category, naming convention, etc).
>>
>> Reminds me of an earlier conversation at
>> https://github.com/macports/macports-ports/pull/6767#discussion_r402584006
>>
>>
>> I do see some benefits is formalizing binary-only ports and to adapt the
>> build and distribution scheme for it. Could save resources and development
>> time and make those ports easily recognizable for those who care about the
>> different way it was built.
>>
>>
>>


Re: port "cask" -- installing prebuilt binaries

2020-08-04 Thread Ruben Di Battista
There's is one compelling need for having "binary only" install, and that
is for the port "osxfuse", that is currently broken for 10.14+.

There was a discussion about it on the Github project about the choice of
making it close closed source... Nonetheless it would be useful to have it
in order to provide things like fuse file systems and so on.



On Tue, 4 Aug 2020, 15:38 Lothar Haeger,  wrote:

>
>
> Am 30.07.2020 um 08:03 schrieb Ken Cunningham <
> ken.cunningham.web...@gmail.com>:
>
> I only raise the idea as people are already doing this, and submitting such 
> ports, and before we have too many, there is an opportunity to say how it 
> should best be done (custom category, naming convention, etc).
>
> Reminds me of an earlier conversation at
> https://github.com/macports/macports-ports/pull/6767#discussion_r402584006
>
>
> I do see some benefits is formalizing binary-only ports and to adapt the
> build and distribution scheme for it. Could save resources and development
> time and make those ports easily recognizable for those who care about the
> different way it was built.
>
>
>


Re: port "cask" -- installing prebuilt binaries

2020-08-04 Thread Lothar Haeger


> Am 30.07.2020 um 08:03 schrieb Ken Cunningham 
> :
> I only raise the idea as people are already doing this, and submitting such 
> ports, and before we have too many, there is an opportunity to say how it 
> should best be done (custom category, naming convention, etc).
Reminds me of an earlier conversation at 
https://github.com/macports/macports-ports/pull/6767#discussion_r402584006 
 

I do see some benefits is formalizing binary-only ports and to adapt the build 
and distribution scheme for it. Could save resources and development time and 
make those ports easily recognizable for those who care about the different way 
it was built.




Re: port "cask" -- installing prebuilt binaries

2020-07-30 Thread Ken Cunningham
> >> From the user's perspective, how does that differ from a port that's 
> > available as a binary archive?  I presume the idea is that it directly uses 
> > a precompiled binary from the upstream source, but from the user's 
> > perspective, does it really matter whether it was a binary from upstream or 
> > a binary from the buildbots?
> > 
> > Or is this for ports where upstream doesn't provide source at all?
> > 
> > Personally, I'd prefer the MacPorts approach if it were less stingy with 
> > the binary archives.  Ideally, one should be *able* to build from source, 
> > but not be *required* to do so.
> 
> How is it stingy? We have binary archives for everything that the buildbots 
> can build that the licenses allow us to distribute, right?
> 
> port, by default, will use the binary archives unless you tell it to build 
> from source instead.
> -- 

These “cask” binaries are generally the official downloadable software packages 
you would get from the project or website of the company or group providing the 
software, built the way they build it, with their libraries integrated into the 
binaries, their update mechanisms, their range of allowed deployment targets. 

There are upsides and downsides to this. Clearly if we can build it, we should. 
But there are things we can’t build, for example “Zoom” is a good one. But 
there are hundreds of others. Until now, MacPorts didn’t try to provide these 
things, but there seems to be a demand for command-line binary installers.

Having used homebrew cask to install about 100 of these in one fell swoop, it 
was convenient, until the warts started showing up and I had to uninstall about 
1/2 of them for various reasons.

I only raise the idea as people are already doing this, and submitting such 
ports, and before we have too many, there is an opportunity to say how it 
should best be done (custom category, naming convention, etc).

K

Re: port "cask" -- installing prebuilt binaries

2020-07-29 Thread Daniel J. Luke
On Jul 29, 2020, at 9:30 PM, Fred Wright  wrote:
> On Wed, 29 Jul 2020, Ken Cunningham wrote:
>> there seems to be demand for replicating the “binary only” installers of 
>> homebrew cask.
>> 
>> we have a few ports that do that now, and I see more and more coming in.
> 
>> From the user's perspective, how does that differ from a port that's 
> available as a binary archive?  I presume the idea is that it directly uses a 
> precompiled binary from the upstream source, but from the user's perspective, 
> does it really matter whether it was a binary from upstream or a binary from 
> the buildbots?
> 
> Or is this for ports where upstream doesn't provide source at all?
> 
> Personally, I'd prefer the MacPorts approach if it were less stingy with the 
> binary archives.  Ideally, one should be *able* to build from source, but not 
> be *required* to do so.

How is it stingy? We have binary archives for everything that the buildbots can 
build that the licenses allow us to distribute, right?

port, by default, will use the binary archives unless you tell it to build from 
source instead.
-- 
Daniel J. Luke



Re: port "cask" -- installing prebuilt binaries

2020-07-29 Thread Fred Wright


On Wed, 29 Jul 2020, Ken Cunningham wrote:


there seems to be demand for replicating the “binary only” installers of 
homebrew cask.

we have a few ports that do that now, and I see more and more coming in.


From the user's perspective, how does that differ from a port that's 
available as a binary archive?  I presume the idea is that it directly 
uses a precompiled binary from the upstream source, but from the user's 
perspective, does it really matter whether it was a binary from upstream 
or a binary from the buildbots?


Or is this for ports where upstream doesn't provide source at all?

Personally, I'd prefer the MacPorts approach if it were less stingy with 
the binary archives.  Ideally, one should be *able* to build from source, 
but not be *required* to do so.


Fred Wright

port "cask" -- installing prebuilt binaries

2020-07-29 Thread Ken Cunningham
there seems to be demand for replicating the “binary only” installers of 
homebrew cask.

we have a few ports that do that now, and I see more and more coming in.

I have used homebrew cask from time to time to see what it’s up to. It is 
convenient at times. I also downloaded a lot of things that didn’t actually 
work on the system I was using, or were time-bombed demos that timed out after 
a week — or updated on me when I wasn’t really planning on updating just then — 
or installed but just plain didn’t work (Lazarus) — so there are downsides to 
it to.

homebrew has nearly 4000 of these already. Probably would be pretty easy to 
just write a little translator to use their cask formulas, I wouldn’t be 
surprised :>

Anyway, if we’re going to do this, and there is demand, perhaps we should have 
a category or a naming convention to make it clear that a prebuilt downloaded 
binary is what we’re installing?

K