Re: compiler_wrapper portgroup

[Chris Jones]

Hi,

It solves a number of problems, but mostly it boils down to not all build 
systems or projects supporting the standard rules we use in macports for 
passing flags erc. to the underlying compilers. Legacy support PG for instance 
requires this to function and a number of ports, or a bunch of ports using a 
given build system, did not work with it. E.g.

1 The go based ports. I experimented for a long time trying to get them to work 
with legacy-support in the ‘standard’ way but in the end it just was not 
compatible with how this build system works. So I added a wrapper there, 
locally at first in the PG and with that all the go ports now build down to I 
think 10.7, whereas before it was limited to I think 10.12 and newer.

2. Rust. Same deal as go. Now works down to 10.9.

3. Bazel. This build system really goes out of its way to not allow you to 
modify its build environment. Again, wrapping the compilers in scripts which 
you then tell the build system to use is a great way to circumvent this.

4. Finally, macports has support for ccache integrated, but it is a little hit 
and miss as to where it works. It didn’t work with bazel, but with the wrapper 
now will does. Similarly for all the python group ports, they also use the 
wrapper and as such work with ccache. If you have ever tried to work on ports 
like py-tensorflow or py-pytorch having ccache now working is a huge boost. 
Same for the makefile PG, it also now works with ccache.

So.. after implementing compiler wrappers in a number of places I decided it 
made sense to centralise it in one place.

Cheers Chris

> On 7 May 2021, at 5:59 pm, Joshua Root  wrote:
> 
> Recently, quite a lot of changes have been made to a number of portgroups, 
> eventually ending up with a new compiler_wrapper portgroup which is now 
> included by several others.
> 
> Chris, for those who have not kept up with all the changes, would you be able 
> to explain what problems all this is solving, how it solves them, and why 
> this particular approach was chosen?
> 
> - Josh

compiler_wrapper portgroup

[Joshua Root]

Recently, quite a lot of changes have been made to a number of 
portgroups, eventually ending up with a new compiler_wrapper portgroup 
which is now included by several others.


Chris, for those who have not kept up with all the changes, would you be 
able to explain what problems all this is solving, how it solves them, 
and why this particular approach was chosen?


- Josh

Re: Ports updated without maintainer notification?

[Joshua Root]

On 2021-5-8 02:02 , Jason Liu wrote:


If your ports are marked openmaintainer, that gives permission to
others to make minor modifications to your ports without notifying
you. Not all changes happen via PRs; some are committed directly to
master.


Does this mean that it's okay to have ports with only myself as 
maintainer? When I started submitting my first ports around a year ago, 
I was told that I should always add openmaintainer in addition to myself.


Adding openmaintainer makes things easier with non-committer maintainers 
in particular, which is why it's recommended to new maintainers, but 
it's not required. If there's a good chance that someone not familiar 
with the nuances of a port will inadvertently break something, then 
openmaintainer is not the right choice. But you need to respond to 
tickets and PRs within 3 days, or changes can be merged anyway under the 
maintainer timeout rule.


Also note that openmaintainer is not carte blanche for others to make 
whatever changes they want to your ports. Committers are expected to 
apply good judgement when making changes to ports maintained by others, 
and to take responsibility for fixing any problem introduced in doing 
so. If a change is at all risky or there is any doubt as to the correct 
approach, running it by the maintainer first is the right thing to do.




- Josh

Re: Ports updated without maintainer notification?

[Christopher Jones]

> OSL in particular appears to be a problem on my machine. I've copied the 
> newer version of the portfile directly to my local machine, and tried to 
> build it, but it's failing to build because osl is indirectly dependent on 
> opencolorio (by way of openimageio), and apparently there's a new problem 
> with either opencolorio or openimageio:
> 
> :info:build dyld: Symbol not found: __ZN4YAML6detail9node_data12empty_scalarE
> :info:build   Referenced from: /opt/local/lib/libOpenColorIO.1.dylib
> :info:build   Expected in: /opt/local/lib/libyaml-cpp.0.6.dylib
> :info:build  in /opt/local/lib/libOpenColorIO.1.dylib
> :info:build /bin/sh: line 1: 34490 Trace/BPT trap: 5
> :info: build  
> opt/local/var/macports/build/_Users_jasonliu_ports_graphics_osl/osl/work/build/bin/oslc
>  -q 
> -I/opt/local/var/macports/build/_Users_jasonliu_ports_graphics_osl/osl/work/OpenShadingLanguage-1.11.13.0/src/shaders
>  
> -I/opt/local/var/macports/build/_Users_jasonliu_ports_graphics_osl/osl/work/OpenShadingLanguage-1.11.13.0/src/shaders
>  
> -I/opt/local/var/macports/build/_Users_jasonliu_ports_graphics_osl/osl/work/OpenShadingLanguage-1.11.13.0/src/shaders
>  
> /opt/local/var/macports/build/_Users_jasonliu_ports_graphics_osl/osl/work/OpenShadingLanguage-1.11.13.0/src/shaders/emitter.osl
>  -o 
> /opt/local/var/macports/build/_Users_jasonliu_ports_graphics_osl/osl/work/build/src/shaders/emitter.oso


Did you also update all the other outdated ports on your ‘local’ machine, or 
did you just cherry-pick the updated osl from the current master ? 

If so it is really not a good idea to do that, as it means, as appears above, 
you could get an updated binary tarball install that was built against another 
updated port you do not have.

You should *always* keep all your ports updated and consistent.

if you run

> sudo port -d sync
> sudo port update outdated

does that help >

Just finally to note, there is nothing wrong with the current osl builds, they 
are available (apart from arm) down to 10.9

https://ports.macports.org/port/osl/summary 


Chris

> 
> -- 
> Jason Liu
> 
> 
> On Fri, May 7, 2021 at 7:32 AM Ryan Schmidt  > wrote:
> 
> 
> On May 7, 2021, at 01:59, Jason Liu wrote:
> 
> > I've run across a situation that has left me confused. I started updating 
> > some of the portfiles for which I'm the maintainer, and then I noticed that 
> > the portfiles seem to have already been updated in git. However, I can't 
> > find any PRs for such an update, and I was never notified that the ports 
> > for which I'm the maintainer was getting updated... usually, if someone 
> > submits a PR for a portfile for which I'm the maintainer, I get a 
> > notification through GitHub.
> 
> If your ports are marked openmaintainer, that gives permission to others to 
> make minor modifications to your ports without notifying you. Not all changes 
> happen via PRs; some are committed directly to master.
> 
> If there is an urgent issue that needs to be fixed in a port, someone else 
> might make that fix, even if the port is not marked openmaintainer.
> 
> If you let us know specifically which ports, we could take a look.
> 
> 
> > In addition, I have run a "port selfupdate" on my machine, and yet the 
> > MacPorts on my machine isn't seeing the new version of the port. Is 
> > something broken, either on my machine, or on GitHub?
> 
> If your MacPorts is configured to get ports via rsync, it can take an hour 
> for changes to propagate from git to the main rsync server, and up to a day 
> longer for changes to propagate from there to other rsync mirrors.



smime.p7s
Description: S/MIME cryptographic signature

Re: Ports updated without maintainer notification?

[Jason Liu]

> If your ports are marked openmaintainer, that gives permission to others
> to make minor modifications to your ports without notifying you. Not all
> changes happen via PRs; some are committed directly to master.
>

Does this mean that it's okay to have ports with only myself as maintainer?
When I started submitting my first ports around a year ago, I was told that
I should always add openmaintainer in addition to myself.

If you let us know specifically which ports, we could take a look.
>

The ports are osl, oidn, and embree.


> In addition, I have run a "port selfupdate" on my machine, and yet the
>> MacPorts on my machine isn't seeing the new version of the port. Is
>> something broken, either on my machine, or on GitHub?
>>
>
> If your MacPorts is configured to get ports via rsync, it can take an hour
> for changes to propagate from git to the main rsync server, and up to a day
> longer for changes to propagate from there to other rsync mirrors.
>

OSL in particular appears to be a problem on my machine. I've copied the
newer version of the portfile directly to my local machine, and tried to
build it, but it's failing to build because osl is indirectly dependent on
opencolorio (by way of openimageio), and apparently there's a new problem
with either opencolorio or openimageio:

:info:build dyld: Symbol not found:
__ZN4YAML6detail9node_data12empty_scalarE
:info:build   Referenced from: /opt/local/lib/libOpenColorIO.1.dylib
:info:build   Expected in: /opt/local/lib/libyaml-cpp.0.6.dylib
:info:build  in /opt/local/lib/libOpenColorIO.1.dylib
:info:build /bin/sh: line 1: 34490 Trace/BPT trap: 5
:info: build
opt/local/var/macports/build/_Users_jasonliu_ports_graphics_osl/osl/work/build/bin/oslc
-q
-I/opt/local/var/macports/build/_Users_jasonliu_ports_graphics_osl/osl/work/OpenShadingLanguage-1.11.13.0/src/shaders
-I/opt/local/var/macports/build/_Users_jasonliu_ports_graphics_osl/osl/work/OpenShadingLanguage-1.11.13.0/src/shaders
-I/opt/local/var/macports/build/_Users_jasonliu_ports_graphics_osl/osl/work/OpenShadingLanguage-1.11.13.0/src/shaders
/opt/local/var/macports/build/_Users_jasonliu_ports_graphics_osl/osl/work/OpenShadingLanguage-1.11.13.0/src/shaders/emitter.osl
-o
/opt/local/var/macports/build/_Users_jasonliu_ports_graphics_osl/osl/work/build/src/shaders/emitter.oso

-- 
Jason Liu


On Fri, May 7, 2021 at 7:32 AM Ryan Schmidt  wrote:

>
>
> On May 7, 2021, at 01:59, Jason Liu wrote:
>
> > I've run across a situation that has left me confused. I started
> updating some of the portfiles for which I'm the maintainer, and then I
> noticed that the portfiles seem to have already been updated in git.
> However, I can't find any PRs for such an update, and I was never notified
> that the ports for which I'm the maintainer was getting updated... usually,
> if someone submits a PR for a portfile for which I'm the maintainer, I get
> a notification through GitHub.
>
> If your ports are marked openmaintainer, that gives permission to others
> to make minor modifications to your ports without notifying you. Not all
> changes happen via PRs; some are committed directly to master.
>
> If there is an urgent issue that needs to be fixed in a port, someone else
> might make that fix, even if the port is not marked openmaintainer.
>
> If you let us know specifically which ports, we could take a look.
>
>
> > In addition, I have run a "port selfupdate" on my machine, and yet the
> MacPorts on my machine isn't seeing the new version of the port. Is
> something broken, either on my machine, or on GitHub?
>
> If your MacPorts is configured to get ports via rsync, it can take an hour
> for changes to propagate from git to the main rsync server, and up to a day
> longer for changes to propagate from there to other rsync mirrors.

Re: Ports updated without maintainer notification?

[Christopher Jones]

> 
> In addition, I have run a "port selfupdate" on my machine, and yet the 
> MacPorts on my machine isn't seeing the new version of the port. Is something 
> broken, either on my machine, or on GitHub?

If you aren’t already, I strongly recommend using a direct git clone of the 
ports tree rather than the default rsync’ed tarball. i.e.something like this in 
your sources.conf

~/Projects/MacPorts/ports > tail -3 /opt/local/etc/macports/sources.conf

#rsync://rsync.macports.org/macports/release/tarballs/ports.tar [default]
file:///Users/chris/Projects/MacPorts/ports [default]

where the path above points to where ever you choose to git clone the ports 
repo.

Once you have done this, ‘port sync’ automatically updates this via a git 
rebase rather than the rsync tarball.

e.g.

Oberon ~/Projects/MacPorts/ports > port -d sync
--->  Updating the ports tree
Synchronizing local ports tree from file:///Users/chris/Projects/MacPorts/ports
DEBUG: /opt/local/bin/git pull --rebase --autostash
DEBUG: system -W /Users/chris/Projects/MacPorts/ports: /opt/local/bin/git pull 
--rebase --autostash
Current branch master is up to date.
DEBUG: system: /opt/local/bin/portindex /Users/chris/Projects/MacPorts/ports
Creating port index in /Users/chris/Projects/MacPorts/ports

Total number of ports parsed:   0 
Ports successfully parsed:  0 
Ports failed:   0 
Up-to-date ports skipped:   26290


the -d option is very useful here as you get to see the git rebase, tother with 
the portindex update.

If you do this regularly, and before submitting any PR’s, it is basically 
impossible  to ‘miss’ updates to any ports.

Chris

smime.p7s
Description: S/MIME cryptographic signature

Re: Ports updated without maintainer notification?

[Ryan Schmidt]

On May 7, 2021, at 01:59, Jason Liu wrote:

> I've run across a situation that has left me confused. I started updating 
> some of the portfiles for which I'm the maintainer, and then I noticed that 
> the portfiles seem to have already been updated in git. However, I can't find 
> any PRs for such an update, and I was never notified that the ports for which 
> I'm the maintainer was getting updated... usually, if someone submits a PR 
> for a portfile for which I'm the maintainer, I get a notification through 
> GitHub.

If your ports are marked openmaintainer, that gives permission to others to 
make minor modifications to your ports without notifying you. Not all changes 
happen via PRs; some are committed directly to master.

If there is an urgent issue that needs to be fixed in a port, someone else 
might make that fix, even if the port is not marked openmaintainer.

If you let us know specifically which ports, we could take a look.


> In addition, I have run a "port selfupdate" on my machine, and yet the 
> MacPorts on my machine isn't seeing the new version of the port. Is something 
> broken, either on my machine, or on GitHub?

If your MacPorts is configured to get ports via rsync, it can take an hour for 
changes to propagate from git to the main rsync server, and up to a day longer 
for changes to propagate from there to other rsync mirrors.

Ports updated without maintainer notification?

[Jason Liu]

Hi all,

I've run across a situation that has left me confused. I started updating
some of the portfiles for which I'm the maintainer, and then I noticed that
the portfiles seem to have already been updated in git. However, I can't
find any PRs for such an update, and I was never notified that the ports
for which I'm the maintainer was getting updated... usually, if someone
submits a PR for a portfile for which I'm the maintainer, I get a
notification through GitHub.

In addition, I have run a "port selfupdate" on my machine, and yet the
MacPorts on my machine isn't seeing the new version of the port. Is
something broken, either on my machine, or on GitHub?

-- 
Jason Liu