Re: git: e2650af157bc - main - Make CPU_SET macros compliant with other implementations
On Mon, Jan 03, 2022 at 12:35:39AM +0100, Stefan Esser wrote:
> Am 02.01.22 um 23:16 schrieb Konstantin Belousov:
> > On Sun, Jan 02, 2022 at 10:45:14PM +0100, Stefan Esser wrote:
> >> Am 02.01.22 um 20:51 schrieb Antoine Brodin:
> [...]
> >> Python 3.8.12 (default, Dec 31 2021, 10:50:47)
> > import os
> > os.sched_getaffinity(0)
> >> Traceback (most recent call last):
> >> File "", line 1, in
> >> OSError: [Errno 34] Result too large
> >>
> >> This is a Python interpreter problem: it seems that the wrapper
> >> for the sched_getaffinity() function that has been introduced by
> >> kib in is buggy.
> >>
> >> As a work-around I have added a patch to comment out the
> >> os.sched_getaffinity(0) call (which used to cause an Attribute
> >> error that was caught by try/except, before).
> >>
> >> See ports commit 507c189b2876.
> >
> > Buggy in which way?
>
> My assumption was that the wrapper in the Python interpreter in
> Modules/posixmodules.c function os_sched_getaffinity_impl() does
> not work with the FreeBSD implementation of sched_getaffinity().
>
> The relevant code in the Python wrapper is:
>
> ncpus = NCPUS_START;
> while (1) {
> setsize = CPU_ALLOC_SIZE(ncpus);
> mask = CPU_ALLOC(ncpus);
> if (mask == NULL)
> return PyErr_NoMemory();
> if (sched_getaffinity(pid, setsize, mask) == 0)
> break;
> CPU_FREE(mask);
> if (errno != EINVAL)
> return posix_error();
> if (ncpus > INT_MAX / 2) {
> PyErr_SetString(PyExc_OverflowError, "could not allocate "
> "a large enough CPU set");
> return NULL;
> }
> ncpus = ncpus * 2;
> }
>
> NCPUS_START is 8 * sizeof(unsigned long) = 64 on a 64 bit CPU.
>
> > Our cpuset_getaffinity(2) syscall returns ERANGE for cpuset size not
> > equal to CPU_SETSIZE. It seems that python source expects EINVAL in
> > this case.
>
> Yes, anything except EINVAL will cause the loop to exit prematurely.
>
> > I can change the wrapper to translate ERANGE to EINVAL. sched_setaffinity()
> > probably would require a symmetrical patch, but lets postpone it.
>
> Yes.
>
> > diff --git a/lib/libc/gen/sched_getaffinity.c
> > b/lib/libc/gen/sched_getaffinity.c
> > index 2ae8c5b763a3..8748d7a60278 100644
> > --- a/lib/libc/gen/sched_getaffinity.c
> > +++ b/lib/libc/gen/sched_getaffinity.c
> > @@ -26,11 +26,29 @@
> > * SUCH DAMAGE.
> > */
> >
> > +#include
> > #include
> > +#include
> >
> > int
> > sched_getaffinity(pid_t pid, size_t cpusetsz, cpuset_t *cpuset)
> > {
> > + /*
> > +* Be more Linux-compatible:
> > +* - return EINVAL in passed size is less than size of cpuset_t
> > +* in advance, instead of ERANGE from the syscall
> > +* - if passed size is larger than the size of cpuset_t, be
> > +* permissive by claming it back to sizeof(cpuset_t) and
> > +* zeroing the rest.
> > +*/
> > + if (cpusetsz < sizeof(cpuset_t))
> > + return (EINVAL);
> > + if (cpusetsz > sizeof(cpuset_t)) {
> > + memset((char *)cpuset + sizeof(cpuset_t), 0,
> > + cpusetsz - sizeof(cpuset_t));
> > + cpusetsz = sizeof(cpuset_t);
> > + }
> > +
> > return (cpuset_getaffinity(CPU_LEVEL_WHICH, CPU_WHICH_PID,
> > pid == 0 ? -1 : pid, cpusetsz, cpuset));
> > }
>
> I have rebuilt the C library with this patch, but it did not fix
> the problem, since the value checked in the loop is errno, not
> the return code of sched_getaffinity().
I see, thank you for noting this.
>
> The following code is tested to work:
>
> #include
> #include
>
> int
> sched_getaffinity(pid_t pid, size_t cpusetsz, cpuset_t *cpuset)
> {
> int result;
>
> result = cpuset_getaffinity(CPU_LEVEL_WHICH, CPU_WHICH_PID,
> pid == 0 ? -1 : pid, cpusetsz, cpuset);
>
> if (result && errno == ERANGE)
> errno = EINVAL;
>
> return (result);
> }
I want to be more permissive outright, in particular, allow larger cpusets
than kernel handles. Updated patch is below.
diff --git a/lib/libc/gen/sched_getaffinity.c b/lib/libc/gen/sched_getaffinity.c
index 2ae8c5b763a3..a26d098deb83 100644
--- a/lib/libc/gen/sched_getaffinity.c
+++ b/lib/libc/gen/sched_getaffinity.c
@@ -26,11 +26,30 @@
* SUCH DAMAGE.
*/
+#include
#include
+#include
int
sched_getaffinity(pid_t pid, size_t cpusetsz, cpuset_t *cpuset)
{
+ /*
+* Be more Linux-compatible:
+* - return EINVAL in passed size is less than size of cpuset_t
+* in advance, instead of ERANGE from the syscall
+* - if passed size is larger than the size of cpuset_t, be
+* permissive by claming it back to sizeof(cpuset_t) and
+* zeroing the rest.
+*/
+ if (cpusetsz < sizeof(cpuset_t)) {
+ errno = EINVAL;
+ return (-1);
+ if (cpusetsz > sizeof(cpuset_t)) {
+
Re: git: e2650af157bc - main - Make CPU_SET macros compliant with other implementations
Am 02.01.22 um 23:16 schrieb Konstantin Belousov:
> On Sun, Jan 02, 2022 at 10:45:14PM +0100, Stefan Esser wrote:
>> Am 02.01.22 um 20:51 schrieb Antoine Brodin:
[...]
>> Python 3.8.12 (default, Dec 31 2021, 10:50:47)
> import os
> os.sched_getaffinity(0)
>> Traceback (most recent call last):
>> File "", line 1, in
>> OSError: [Errno 34] Result too large
>>
>> This is a Python interpreter problem: it seems that the wrapper
>> for the sched_getaffinity() function that has been introduced by
>> kib in is buggy.
>>
>> As a work-around I have added a patch to comment out the
>> os.sched_getaffinity(0) call (which used to cause an Attribute
>> error that was caught by try/except, before).
>>
>> See ports commit 507c189b2876.
>
> Buggy in which way?
My assumption was that the wrapper in the Python interpreter in
Modules/posixmodules.c function os_sched_getaffinity_impl() does
not work with the FreeBSD implementation of sched_getaffinity().
The relevant code in the Python wrapper is:
ncpus = NCPUS_START;
while (1) {
setsize = CPU_ALLOC_SIZE(ncpus);
mask = CPU_ALLOC(ncpus);
if (mask == NULL)
return PyErr_NoMemory();
if (sched_getaffinity(pid, setsize, mask) == 0)
break;
CPU_FREE(mask);
if (errno != EINVAL)
return posix_error();
if (ncpus > INT_MAX / 2) {
PyErr_SetString(PyExc_OverflowError, "could not allocate "
"a large enough CPU set");
return NULL;
}
ncpus = ncpus * 2;
}
NCPUS_START is 8 * sizeof(unsigned long) = 64 on a 64 bit CPU.
> Our cpuset_getaffinity(2) syscall returns ERANGE for cpuset size not
> equal to CPU_SETSIZE. It seems that python source expects EINVAL in
> this case.
Yes, anything except EINVAL will cause the loop to exit prematurely.
> I can change the wrapper to translate ERANGE to EINVAL. sched_setaffinity()
> probably would require a symmetrical patch, but lets postpone it.
Yes.
> diff --git a/lib/libc/gen/sched_getaffinity.c
> b/lib/libc/gen/sched_getaffinity.c
> index 2ae8c5b763a3..8748d7a60278 100644
> --- a/lib/libc/gen/sched_getaffinity.c
> +++ b/lib/libc/gen/sched_getaffinity.c
> @@ -26,11 +26,29 @@
> * SUCH DAMAGE.
> */
>
> +#include
> #include
> +#include
>
> int
> sched_getaffinity(pid_t pid, size_t cpusetsz, cpuset_t *cpuset)
> {
> + /*
> + * Be more Linux-compatible:
> + * - return EINVAL in passed size is less than size of cpuset_t
> + * in advance, instead of ERANGE from the syscall
> + * - if passed size is larger than the size of cpuset_t, be
> + * permissive by claming it back to sizeof(cpuset_t) and
> + * zeroing the rest.
> + */
> + if (cpusetsz < sizeof(cpuset_t))
> + return (EINVAL);
> + if (cpusetsz > sizeof(cpuset_t)) {
> + memset((char *)cpuset + sizeof(cpuset_t), 0,
> + cpusetsz - sizeof(cpuset_t));
> + cpusetsz = sizeof(cpuset_t);
> + }
> +
> return (cpuset_getaffinity(CPU_LEVEL_WHICH, CPU_WHICH_PID,
> pid == 0 ? -1 : pid, cpusetsz, cpuset));
> }
I have rebuilt the C library with this patch, but it did not fix
the problem, since the value checked in the loop is errno, not
the return code of sched_getaffinity().
The following code is tested to work:
#include
#include
int
sched_getaffinity(pid_t pid, size_t cpusetsz, cpuset_t *cpuset)
{
int result;
result = cpuset_getaffinity(CPU_LEVEL_WHICH, CPU_WHICH_PID,
pid == 0 ? -1 : pid, cpusetsz, cpuset);
if (result && errno == ERANGE)
errno = EINVAL;
return (result);
}
Regards, STefan
OpenPGP_signature
Description: OpenPGP digital signature
Re: git: e2650af157bc - main - Make CPU_SET macros compliant with other implementations
Am 02.01.22 um 22:45 schrieb Stefan Esser:
> Am 02.01.22 um 20:51 schrieb Antoine Brodin:
[...]
>> http://beefy18.nyi.freebsd.org/data/main-amd64-default/pe2d17ded99d5_s5169832c96/logs/errors/slurm-wlm-20.02.7.log
>
> This is easily fixed with this patch:
>
> --- src/plugins/task/affinity/affinity.c.orig 2021-05-12 20:23:20 UTC
> +++ src/plugins/task/affinity/affinity.c
> @@ -297,7 +297,7 @@ void reset_cpuset(cpu_set_t *new_mask, cpu_set_t *cur_
> if (slurm_getaffinity(1, sizeof(full_mask), &full_mask)) {
> /* Try to get full CPU mask from process init */
> CPU_ZERO(&full_mask);
> -#ifdef __FreeBSD__
> +#if defined(__FreeBSD__) && !defined(CPU_ALLOC)
> CPU_OR(&full_mask, cur_mask);
> #else
> CPU_OR(&full_mask, &full_mask, cur_mask);
>
> This effectively removes the conditional compilation that was
> required due to the different CPU_OR signature.
>
> BUT: There are many other build issues in this port, that do not
> depend on the CPU_SET macros.
>
> Since the build succeeds on -STABLE, there must be other changes
> in the build configuration on -CURRENT, which lead to #include of
> Linux specific headers and try to use pushd/popd in /bin/sh (under
> the assumption of Bash installed as /bin/sh).
>
> I'll look into these issues, but they must be somewhere in the
> build system, not the sources being compiled.
>
> I'm sure that the patch above is required, but I'm not going to
> commit it right now, since I want to understand the other build
> issues first.
I have committed the patch as addda1277abe, together with a small
change that prevents the configure script from enabling the build
of slurmrestd if GLIB is found (as may be the case when building
with "make" in the port directory instead of with poudriere).
Regards, STefan
OpenPGP_signature
Description: OpenPGP digital signature
Re: git: e2650af157bc - main - Make CPU_SET macros compliant with other implementations
On Sun, Jan 02, 2022 at 10:45:14PM +0100, Stefan Esser wrote:
> Am 02.01.22 um 20:51 schrieb Antoine Brodin:
> > Hi,
> >
> > It seems that the 2 main ports failing are math/py-numpy (503 ports
> > skipped) and sysutils/slurm-wlm (232 ports skipped)
>
> Hi Antoine,
>
> thank you for the information!
>
> > Failure logs:
> > http://beefy18.nyi.freebsd.org/data/main-amd64-default/pe2d17ded99d5_s5169832c96/logs/errors/py38-numpy-1.20.3,1.log
>
> Python 3.8.12 (default, Dec 31 2021, 10:50:47)
> >>> import os
> >>> os.sched_getaffinity(0)
> Traceback (most recent call last):
> File "", line 1, in
> OSError: [Errno 34] Result too large
>
> This is a Python interpreter problem: it seems that the wrapper
> for the sched_getaffinity() function that has been introduced by
> kib in is buggy.
>
> As a work-around I have added a patch to comment out the
> os.sched_getaffinity(0) call (which used to cause an Attribute
> error that was caught by try/except, before).
>
> See ports commit 507c189b2876.
Buggy in which way?
Our cpuset_getaffinity(2) syscall returns ERANGE for cpuset size not
equal to CPU_SETSIZE. It seems that python source expects EINVAL in
this case.
I can change the wrapper to translate ERANGE to EINVAL. sched_setaffinity()
probably would require a symmetrical patch, but lets postpone it.
diff --git a/lib/libc/gen/sched_getaffinity.c b/lib/libc/gen/sched_getaffinity.c
index 2ae8c5b763a3..8748d7a60278 100644
--- a/lib/libc/gen/sched_getaffinity.c
+++ b/lib/libc/gen/sched_getaffinity.c
@@ -26,11 +26,29 @@
* SUCH DAMAGE.
*/
+#include
#include
+#include
int
sched_getaffinity(pid_t pid, size_t cpusetsz, cpuset_t *cpuset)
{
+ /*
+* Be more Linux-compatible:
+* - return EINVAL in passed size is less than size of cpuset_t
+* in advance, instead of ERANGE from the syscall
+* - if passed size is larger than the size of cpuset_t, be
+* permissive by claming it back to sizeof(cpuset_t) and
+* zeroing the rest.
+*/
+ if (cpusetsz < sizeof(cpuset_t))
+ return (EINVAL);
+ if (cpusetsz > sizeof(cpuset_t)) {
+ memset((char *)cpuset + sizeof(cpuset_t), 0,
+ cpusetsz - sizeof(cpuset_t));
+ cpusetsz = sizeof(cpuset_t);
+ }
+
return (cpuset_getaffinity(CPU_LEVEL_WHICH, CPU_WHICH_PID,
pid == 0 ? -1 : pid, cpusetsz, cpuset));
}
Re: git: e2650af157bc - main - Make CPU_SET macros compliant with other implementations
Am 02.01.22 um 20:51 schrieb Antoine Brodin:
> Hi,
>
> It seems that the 2 main ports failing are math/py-numpy (503 ports
> skipped) and sysutils/slurm-wlm (232 ports skipped)
Hi Antoine,
thank you for the information!
> Failure logs:
> http://beefy18.nyi.freebsd.org/data/main-amd64-default/pe2d17ded99d5_s5169832c96/logs/errors/py38-numpy-1.20.3,1.log
Python 3.8.12 (default, Dec 31 2021, 10:50:47)
>>> import os
>>> os.sched_getaffinity(0)
Traceback (most recent call last):
File "", line 1, in
OSError: [Errno 34] Result too large
This is a Python interpreter problem: it seems that the wrapper
for the sched_getaffinity() function that has been introduced by
kib in is buggy.
As a work-around I have added a patch to comment out the
os.sched_getaffinity(0) call (which used to cause an Attribute
error that was caught by try/except, before).
See ports commit 507c189b2876.
> http://beefy18.nyi.freebsd.org/data/main-amd64-default/pe2d17ded99d5_s5169832c96/logs/errors/slurm-wlm-20.02.7.log
This is easily fixed with this patch:
--- src/plugins/task/affinity/affinity.c.orig 2021-05-12 20:23:20 UTC
+++ src/plugins/task/affinity/affinity.c
@@ -297,7 +297,7 @@ void reset_cpuset(cpu_set_t *new_mask, cpu_set_t *cur_
if (slurm_getaffinity(1, sizeof(full_mask), &full_mask)) {
/* Try to get full CPU mask from process init */
CPU_ZERO(&full_mask);
-#ifdef __FreeBSD__
+#if defined(__FreeBSD__) && !defined(CPU_ALLOC)
CPU_OR(&full_mask, cur_mask);
#else
CPU_OR(&full_mask, &full_mask, cur_mask);
This effectively removes the conditional compilation that was
required due to the different CPU_OR signature.
BUT: There are many other build issues in this port, that do not
depend on the CPU_SET macros.
Since the build succeeds on -STABLE, there must be other changes
in the build configuration on -CURRENT, which lead to #include of
Linux specific headers and try to use pushd/popd in /bin/sh (under
the assumption of Bash installed as /bin/sh).
I'll look into these issues, but they must be somewhere in the
build system, not the sources being compiled.
I'm sure that the patch above is required, but I'm not going to
commit it right now, since I want to understand the other build
issues first.
Regards, STefan
OpenPGP_signature
Description: OpenPGP digital signature
Re: git: e2650af157bc - main - Make CPU_SET macros compliant with other implementations
On Fri, Dec 31, 2021 at 10:22 AM Stefan Esser wrote: > Am 31.12.21 um 09:01 schrieb Antoine Brodin: > > On Thu, Dec 30, 2021 at 11:54 AM Stefan Eßer wrote: > >> > >> The branch main has been updated by se: > >> > >> URL: > >> https://cgit.FreeBSD.org/src/commit/?id=e2650af157bc7489deaf2c9054995f0f88a6e5da > >> > >> commit e2650af157bc7489deaf2c9054995f0f88a6e5da > >> Author: Stefan Eßer > >> AuthorDate: 2021-12-30 11:20:32 + > >> Commit: Stefan Eßer > >> CommitDate: 2021-12-30 11:20:32 + > >> > [...] > >> Ports that have added -D_WITH_CPU_SET_T to build on -CURRENT do > >> no longer require that option. > >> > >> The FreeBSD version has been bumped to 1400046 to reflect this > >> incompatible change. > >> > >> Reviewed by:kib > >> MFC after: 2 weeks > >> Relnotes: yes > >> Differential Revision: https://reviews.freebsd.org/D33451 > > > > Hi, > > > > This breaks a lot of ports, like lang/python38. > > Could these kinds of changes on public headers be tested with an > > exp-run, and reverted in the mean-time? > > I'm sorry for the breakage. The commit had the goal to lessen > port build problems caused by the misled assumptions that the > port was being built on a GLIBC based system. > > In the case of the Python language ports, one additional macro > was required and has been added in commit cb65d4432aed11. > > Since the official package builders have not been upgraded to > a -CURRENT with this change, they are not affected. But I'll > watch the failed build logs on beefy18. Hi, It seems that the 2 main ports failing are math/py-numpy (503 ports skipped) and sysutils/slurm-wlm (232 ports skipped) Failure logs: http://beefy18.nyi.freebsd.org/data/main-amd64-default/pe2d17ded99d5_s5169832c96/logs/errors/py38-numpy-1.20.3,1.log http://beefy18.nyi.freebsd.org/data/main-amd64-default/pe2d17ded99d5_s5169832c96/logs/errors/slurm-wlm-20.02.7.log Antoine > I'm not opposed to a revert and exp-run, but I'm convinced that > any fall-out from this change is easily fixed, and I'm willing > to quickly fix any ports or base system components affected. > > Regards, STefan
Re: git: e2650af157bc - main - Make CPU_SET macros compliant with other implementations
On Fri, Dec 31, 2021 at 11:34:34AM -0700, Warner Losh wrote: > On Fri, Dec 31, 2021 at 11:29 AM Konstantin Belousov > wrote: > > > On Fri, Dec 31, 2021 at 01:08:25PM -0500, Ed Maste wrote: > > > Some time ago I started a best practises doc for potentially > > > disruptive src changes (and have received some feedback, including > > > from folks on this thread). I'll paste it here for further discussion. > > > --- > > > This is the suggested process for introducing tool chain and other > > > changes in the src tree that may cause significant disruption to > > > ports. Some examples of potentially disruptive changes are: > > > > > > - major compiler updates > > > - OpenSSL updates > > > - adding a library or system call (such as memfd) that is already > > > present on other systems > > > - changing the semantics or APIs of existing libraries > > > > > > The goal of this document is not to be overly prescriptive, but to > > > document a process that has produced good results in the past, avoid > > > surprises among ports committers and maintainers, and clarify the > > > expectation on port maintainers to collaborate on addressing fallout > > > from the potentially disruptive change. The project gets the best > > > results when everybody works together, in good faith, to solve > > > problems with disruptive changes. > > > > > > Disruptive change process: > > > > > > 1. Request a ports exp-run with the desired change. This is used to > > > determine the initial impact of the change. If the exp-run shows no > > > impact or minimal impact the rest of the process may be skipped. > > > > > > 2. Verify that important packages build, and fix identified failures. > > > Maintainers of important packages should be prepared to assist. > > > Important (critical?) packages include: > > > > > > - pkg > > > - binutils > > > - gcc > > > … (need to expand this list) > > > > > > 3. Post a Heads-Up email to at least the FreeBSD-current and > > > FreeBSD-ports mailing lists with a proposed schedule. Where > > > appropriate add other mailing lists, such as FreeBSD-toolchain. Allow > > > at least three weeks between the Heads-Up email and the commit. > > > > > > 4. In the period between the Heads-Up email and the commit, developers > > > proposing the change and maintainers of ports affected by the change > > > work together to resolve any ports failures. > > And what to do if developers are not 'collaborative'? For my case, there > > was a silence from ports maintainers, even after > > - a tool was proposed > > - a request for feedback was issued > > > > There's a timeout in ports. If the maintainer is unresponsive, you can > proceed. > There's some tweaking we can do to the timeouts, to be sure, but I've had > several > things time out and I was good to proceed with my changes. Timeouts in ports are only for maintainer approval of ports changes. If you need somebody to (help) develop the changes, timeout does not provide any support. > > For base changes in ABI, maybe we need a faster expectation for feedback as > well as allowing breakage when the timeout is reached and there aren't some > compelling reasons not to proceed. [We do not allow ABI changes at all, except additions of interfaces FWIW] > > > > > > > > 5. Request additional exp-runs as necessary (by adding a comment in > > > the existing PR). > > > > > > 6. Commit may proceed once all important/critcal ports build, and either: > > > > > > - The deadline proposed in the Heads-Up email has been reached > > > - There is a concensus that remaining failures are insignificant (for > > > example, a small number of unmaintained leaf ports are the only > > > outstanding failures) > > > > > > 7. Collaborate on fixing any outstanding issues (e.g. broken leaf ports) > > > > This is good wishes, at best. This assessment is backed by my experience > > both with ino64, and with sched_get/setaffinity. Either source changes > > are blocked indefinitely, or source committer is tasked with fixing all > > broken ports. > > > > It's never that absolute: I've made changes I knew would break ports that > were deemed to be unsupported enough to just mark broken. But having a list > of things that might be broken allowed me to work with portmgr to make this > call. In this case, I did not expected that addition of two functions would break anything. Apparently it did, because autoconf software checked linkage with them, ignoring availability of prototype. And after the functions presence was detected, software assumed that whole set of glibc-compatible API of CPU_XXX is there. Requesting exp-runs for addition of each exported function is not constructive.
Re: git: e2650af157bc - main - Make CPU_SET macros compliant with other implementations
On Fri, Dec 31, 2021 at 11:29 AM Konstantin Belousov wrote: > On Fri, Dec 31, 2021 at 01:08:25PM -0500, Ed Maste wrote: > > Some time ago I started a best practises doc for potentially > > disruptive src changes (and have received some feedback, including > > from folks on this thread). I'll paste it here for further discussion. > > --- > > This is the suggested process for introducing tool chain and other > > changes in the src tree that may cause significant disruption to > > ports. Some examples of potentially disruptive changes are: > > > > - major compiler updates > > - OpenSSL updates > > - adding a library or system call (such as memfd) that is already > > present on other systems > > - changing the semantics or APIs of existing libraries > > > > The goal of this document is not to be overly prescriptive, but to > > document a process that has produced good results in the past, avoid > > surprises among ports committers and maintainers, and clarify the > > expectation on port maintainers to collaborate on addressing fallout > > from the potentially disruptive change. The project gets the best > > results when everybody works together, in good faith, to solve > > problems with disruptive changes. > > > > Disruptive change process: > > > > 1. Request a ports exp-run with the desired change. This is used to > > determine the initial impact of the change. If the exp-run shows no > > impact or minimal impact the rest of the process may be skipped. > > > > 2. Verify that important packages build, and fix identified failures. > > Maintainers of important packages should be prepared to assist. > > Important (critical?) packages include: > > > > - pkg > > - binutils > > - gcc > > … (need to expand this list) > > > > 3. Post a Heads-Up email to at least the FreeBSD-current and > > FreeBSD-ports mailing lists with a proposed schedule. Where > > appropriate add other mailing lists, such as FreeBSD-toolchain. Allow > > at least three weeks between the Heads-Up email and the commit. > > > > 4. In the period between the Heads-Up email and the commit, developers > > proposing the change and maintainers of ports affected by the change > > work together to resolve any ports failures. > And what to do if developers are not 'collaborative'? For my case, there > was a silence from ports maintainers, even after > - a tool was proposed > - a request for feedback was issued > There's a timeout in ports. If the maintainer is unresponsive, you can proceed. There's some tweaking we can do to the timeouts, to be sure, but I've had several things time out and I was good to proceed with my changes. For base changes in ABI, maybe we need a faster expectation for feedback as well as allowing breakage when the timeout is reached and there aren't some compelling reasons not to proceed. > > > > 5. Request additional exp-runs as necessary (by adding a comment in > > the existing PR). > > > > 6. Commit may proceed once all important/critcal ports build, and either: > > > > - The deadline proposed in the Heads-Up email has been reached > > - There is a concensus that remaining failures are insignificant (for > > example, a small number of unmaintained leaf ports are the only > > outstanding failures) > > > > 7. Collaborate on fixing any outstanding issues (e.g. broken leaf ports) > > This is good wishes, at best. This assessment is backed by my experience > both with ino64, and with sched_get/setaffinity. Either source changes > are blocked indefinitely, or source committer is tasked with fixing all > broken ports. > It's never that absolute: I've made changes I knew would break ports that were deemed to be unsupported enough to just mark broken. But having a list of things that might be broken allowed me to work with portmgr to make this call. Warner
Re: git: e2650af157bc - main - Make CPU_SET macros compliant with other implementations
On Fri, Dec 31, 2021 at 01:08:25PM -0500, Ed Maste wrote: > Some time ago I started a best practises doc for potentially > disruptive src changes (and have received some feedback, including > from folks on this thread). I'll paste it here for further discussion. > --- > This is the suggested process for introducing tool chain and other > changes in the src tree that may cause significant disruption to > ports. Some examples of potentially disruptive changes are: > > - major compiler updates > - OpenSSL updates > - adding a library or system call (such as memfd) that is already > present on other systems > - changing the semantics or APIs of existing libraries > > The goal of this document is not to be overly prescriptive, but to > document a process that has produced good results in the past, avoid > surprises among ports committers and maintainers, and clarify the > expectation on port maintainers to collaborate on addressing fallout > from the potentially disruptive change. The project gets the best > results when everybody works together, in good faith, to solve > problems with disruptive changes. > > Disruptive change process: > > 1. Request a ports exp-run with the desired change. This is used to > determine the initial impact of the change. If the exp-run shows no > impact or minimal impact the rest of the process may be skipped. > > 2. Verify that important packages build, and fix identified failures. > Maintainers of important packages should be prepared to assist. > Important (critical?) packages include: > > - pkg > - binutils > - gcc > … (need to expand this list) > > 3. Post a Heads-Up email to at least the FreeBSD-current and > FreeBSD-ports mailing lists with a proposed schedule. Where > appropriate add other mailing lists, such as FreeBSD-toolchain. Allow > at least three weeks between the Heads-Up email and the commit. > > 4. In the period between the Heads-Up email and the commit, developers > proposing the change and maintainers of ports affected by the change > work together to resolve any ports failures. And what to do if developers are not 'collaborative'? For my case, there was a silence from ports maintainers, even after - a tool was proposed - a request for feedback was issued > > 5. Request additional exp-runs as necessary (by adding a comment in > the existing PR). > > 6. Commit may proceed once all important/critcal ports build, and either: > > - The deadline proposed in the Heads-Up email has been reached > - There is a concensus that remaining failures are insignificant (for > example, a small number of unmaintained leaf ports are the only > outstanding failures) > > 7. Collaborate on fixing any outstanding issues (e.g. broken leaf ports) This is good wishes, at best. This assessment is backed by my experience both with ino64, and with sched_get/setaffinity. Either source changes are blocked indefinitely, or source committer is tasked with fixing all broken ports.
Re: git: e2650af157bc - main - Make CPU_SET macros compliant with other implementations
Some time ago I started a best practises doc for potentially disruptive src changes (and have received some feedback, including from folks on this thread). I'll paste it here for further discussion. --- This is the suggested process for introducing tool chain and other changes in the src tree that may cause significant disruption to ports. Some examples of potentially disruptive changes are: - major compiler updates - OpenSSL updates - adding a library or system call (such as memfd) that is already present on other systems - changing the semantics or APIs of existing libraries The goal of this document is not to be overly prescriptive, but to document a process that has produced good results in the past, avoid surprises among ports committers and maintainers, and clarify the expectation on port maintainers to collaborate on addressing fallout from the potentially disruptive change. The project gets the best results when everybody works together, in good faith, to solve problems with disruptive changes. Disruptive change process: 1. Request a ports exp-run with the desired change. This is used to determine the initial impact of the change. If the exp-run shows no impact or minimal impact the rest of the process may be skipped. 2. Verify that important packages build, and fix identified failures. Maintainers of important packages should be prepared to assist. Important (critical?) packages include: - pkg - binutils - gcc … (need to expand this list) 3. Post a Heads-Up email to at least the FreeBSD-current and FreeBSD-ports mailing lists with a proposed schedule. Where appropriate add other mailing lists, such as FreeBSD-toolchain. Allow at least three weeks between the Heads-Up email and the commit. 4. In the period between the Heads-Up email and the commit, developers proposing the change and maintainers of ports affected by the change work together to resolve any ports failures. 5. Request additional exp-runs as necessary (by adding a comment in the existing PR). 6. Commit may proceed once all important/critcal ports build, and either: - The deadline proposed in the Heads-Up email has been reached - There is a concensus that remaining failures are insignificant (for example, a small number of unmaintained leaf ports are the only outstanding failures) 7. Collaborate on fixing any outstanding issues (e.g. broken leaf ports)
Re: git: e2650af157bc - main - Make CPU_SET macros compliant with other implementations
On Fri, Dec 31, 2021 at 9:31 AM Kyle Evans wrote: > On Fri, Dec 31, 2021 at 10:19 AM Konstantin Belousov > wrote: > > > > On Fri, Dec 31, 2021 at 09:37:16AM -0600, Kyle Evans wrote: > > > On Fri, Dec 31, 2021 at 4:22 AM Stefan Esser wrote: > > > > > > > > Am 31.12.21 um 09:01 schrieb Antoine Brodin: > > > > > On Thu, Dec 30, 2021 at 11:54 AM Stefan Eßer > wrote: > > > > >> > > > > >> The branch main has been updated by se: > > > > >> > > > > >> URL: > https://cgit.FreeBSD.org/src/commit/?id=e2650af157bc7489deaf2c9054995f0f88a6e5da > > > > >> > > > > >> commit e2650af157bc7489deaf2c9054995f0f88a6e5da > > > > >> Author: Stefan Eßer > > > > >> AuthorDate: 2021-12-30 11:20:32 + > > > > >> Commit: Stefan Eßer > > > > >> CommitDate: 2021-12-30 11:20:32 + > > > > >> > > > > [...] > > > > >> Ports that have added -D_WITH_CPU_SET_T to build on -CURRENT > do > > > > >> no longer require that option. > > > > >> > > > > >> The FreeBSD version has been bumped to 1400046 to reflect this > > > > >> incompatible change. > > > > >> > > > > >> Reviewed by:kib > > > > >> MFC after: 2 weeks > > > > >> Relnotes: yes > > > > >> Differential Revision: https://reviews.freebsd.org/D33451 > > > > > > > > > > Hi, > > > > > > > > > > This breaks a lot of ports, like lang/python38. > > > > > Could these kinds of changes on public headers be tested with an > > > > > exp-run, and reverted in the mean-time? > > > > > > > > I'm sorry for the breakage. The commit had the goal to lessen > > > > port build problems caused by the misled assumptions that the > > > > port was being built on a GLIBC based system. > > > > > > > > > > Given that we've now iterated on this a couple of times, this likely > > > should have all been backed out and exp-run'd *way* sooner. > > Exp-runs are great when there is a path forward from fixing the breakage. > > In the case where you have some random ports broken, and no ports > maintainers > > responses for explicit queries, it is basically a deadlock. > > > > I definitely cannot go over some random but large set of ports fixing > them, > > while maintainers are silent. > > > > We do not know in advance that maintainers are silent, because we do > not know in advance which ports are even affected. We can't outright > claim there's a problem without knowing the scope. Nevertheless, yes, > you're right- this is even a conversation we've had recently re: > toolchain breakage, and I don't recall what the outcome of that > conversation was. > > > In fact, with this set of changes, I initially provided some tools in > base > > that were intended to ease the ports life, but still required some > (minimal) > > involvement from the maintainers, like passing -DWITH_CPU_SET_T to C/C++ > > compiler. I asked more than once if these tools are desirable helper or > > not, with no avail. > > > > And that was indeed helpful, thanks! > > > So my only route forward was to leave the state in the minimal damaging > > mode as I see it from bug reports, and wait for maintainers to do > > _something_. I am very grateful that Stefan took the torch and started > > massaging the CPU_XXX ugliness into more compatibility with glibc. This > > again happens in the same silence mode from maintainers, so if we want a > > progress in this area, it have to go this way. > > > > None of this is really applicable to this specific commit, though. The > breakage identified this time was that a couple more definitions were > expected, which does lean towards iteration on the patch rather than > yet requiring the maintainer or ports committer aide. > > I suspect the answer for other scenarios is that we run the exp-run > and shoot out a solicitation for help to -ports@ to cast a broader > net. Ports committers already get stuck with the fallout from this > stuff when maintainers don't notice or step up to the plate or even > when the breakage is just bad enough. I imagine you'd have no problem > catching some folks willing to help be proactive rather than reactive. > > > > > > > > In the case of the Python language ports, one additional macro > > > > was required and has been added in commit cb65d4432aed11. > > > > > > > > Since the official package builders have not been upgraded to > > > > a -CURRENT with this change, they are not affected. But I'll > > > > watch the failed build logs on beefy18. > > Right. > > > > > > > > This is a mindset that we all take, but we really need to work towards > > > improving. Once we're watching fallout logs on the official builders, > > > we've already lost. This is the kind of thing that helps promote the > > > idea that -CURRENT isn't stable enough for production uses: we start > > > accepting that we can be a little more lenient on identifying > > > ports-breaking changes because it's -CURRENT and we lose a fraction of > > > the ports tree because we've only sniped off individual ports as they > > > come up. > > > > > > portmgr@ is able an
Re: git: e2650af157bc - main - Make CPU_SET macros compliant with other implementations
On Fri, Dec 31, 2021 at 10:19 AM Konstantin Belousov wrote: > > On Fri, Dec 31, 2021 at 09:37:16AM -0600, Kyle Evans wrote: > > On Fri, Dec 31, 2021 at 4:22 AM Stefan Esser wrote: > > > > > > Am 31.12.21 um 09:01 schrieb Antoine Brodin: > > > > On Thu, Dec 30, 2021 at 11:54 AM Stefan Eßer wrote: > > > >> > > > >> The branch main has been updated by se: > > > >> > > > >> URL: > > > >> https://cgit.FreeBSD.org/src/commit/?id=e2650af157bc7489deaf2c9054995f0f88a6e5da > > > >> > > > >> commit e2650af157bc7489deaf2c9054995f0f88a6e5da > > > >> Author: Stefan Eßer > > > >> AuthorDate: 2021-12-30 11:20:32 + > > > >> Commit: Stefan Eßer > > > >> CommitDate: 2021-12-30 11:20:32 + > > > >> > > > [...] > > > >> Ports that have added -D_WITH_CPU_SET_T to build on -CURRENT do > > > >> no longer require that option. > > > >> > > > >> The FreeBSD version has been bumped to 1400046 to reflect this > > > >> incompatible change. > > > >> > > > >> Reviewed by:kib > > > >> MFC after: 2 weeks > > > >> Relnotes: yes > > > >> Differential Revision: https://reviews.freebsd.org/D33451 > > > > > > > > Hi, > > > > > > > > This breaks a lot of ports, like lang/python38. > > > > Could these kinds of changes on public headers be tested with an > > > > exp-run, and reverted in the mean-time? > > > > > > I'm sorry for the breakage. The commit had the goal to lessen > > > port build problems caused by the misled assumptions that the > > > port was being built on a GLIBC based system. > > > > > > > Given that we've now iterated on this a couple of times, this likely > > should have all been backed out and exp-run'd *way* sooner. > Exp-runs are great when there is a path forward from fixing the breakage. > In the case where you have some random ports broken, and no ports maintainers > responses for explicit queries, it is basically a deadlock. > > I definitely cannot go over some random but large set of ports fixing them, > while maintainers are silent. > We do not know in advance that maintainers are silent, because we do not know in advance which ports are even affected. We can't outright claim there's a problem without knowing the scope. Nevertheless, yes, you're right- this is even a conversation we've had recently re: toolchain breakage, and I don't recall what the outcome of that conversation was. > In fact, with this set of changes, I initially provided some tools in base > that were intended to ease the ports life, but still required some (minimal) > involvement from the maintainers, like passing -DWITH_CPU_SET_T to C/C++ > compiler. I asked more than once if these tools are desirable helper or > not, with no avail. > And that was indeed helpful, thanks! > So my only route forward was to leave the state in the minimal damaging > mode as I see it from bug reports, and wait for maintainers to do > _something_. I am very grateful that Stefan took the torch and started > massaging the CPU_XXX ugliness into more compatibility with glibc. This > again happens in the same silence mode from maintainers, so if we want a > progress in this area, it have to go this way. > None of this is really applicable to this specific commit, though. The breakage identified this time was that a couple more definitions were expected, which does lean towards iteration on the patch rather than yet requiring the maintainer or ports committer aide. I suspect the answer for other scenarios is that we run the exp-run and shoot out a solicitation for help to -ports@ to cast a broader net. Ports committers already get stuck with the fallout from this stuff when maintainers don't notice or step up to the plate or even when the breakage is just bad enough. I imagine you'd have no problem catching some folks willing to help be proactive rather than reactive. > > > > > In the case of the Python language ports, one additional macro > > > was required and has been added in commit cb65d4432aed11. > > > > > > Since the official package builders have not been upgraded to > > > a -CURRENT with this change, they are not affected. But I'll > > > watch the failed build logs on beefy18. > Right. > > > > > This is a mindset that we all take, but we really need to work towards > > improving. Once we're watching fallout logs on the official builders, > > we've already lost. This is the kind of thing that helps promote the > > idea that -CURRENT isn't stable enough for production uses: we start > > accepting that we can be a little more lenient on identifying > > ports-breaking changes because it's -CURRENT and we lose a fraction of > > the ports tree because we've only sniped off individual ports as they > > come up. > > > > portmgr@ is able and willing to run exp-runs for changes like this, we > > really need to take advantage of that to avoid this kind of follow-up. > There, you are blocking src changes by putting unreasonable requirements > on src committers to fix ports breakage. I am willing to wor
Re: git: e2650af157bc - main - Make CPU_SET macros compliant with other implementations
On Fri, Dec 31, 2021 at 09:37:16AM -0600, Kyle Evans wrote: > On Fri, Dec 31, 2021 at 4:22 AM Stefan Esser wrote: > > > > Am 31.12.21 um 09:01 schrieb Antoine Brodin: > > > On Thu, Dec 30, 2021 at 11:54 AM Stefan Eßer wrote: > > >> > > >> The branch main has been updated by se: > > >> > > >> URL: > > >> https://cgit.FreeBSD.org/src/commit/?id=e2650af157bc7489deaf2c9054995f0f88a6e5da > > >> > > >> commit e2650af157bc7489deaf2c9054995f0f88a6e5da > > >> Author: Stefan Eßer > > >> AuthorDate: 2021-12-30 11:20:32 + > > >> Commit: Stefan Eßer > > >> CommitDate: 2021-12-30 11:20:32 + > > >> > > [...] > > >> Ports that have added -D_WITH_CPU_SET_T to build on -CURRENT do > > >> no longer require that option. > > >> > > >> The FreeBSD version has been bumped to 1400046 to reflect this > > >> incompatible change. > > >> > > >> Reviewed by:kib > > >> MFC after: 2 weeks > > >> Relnotes: yes > > >> Differential Revision: https://reviews.freebsd.org/D33451 > > > > > > Hi, > > > > > > This breaks a lot of ports, like lang/python38. > > > Could these kinds of changes on public headers be tested with an > > > exp-run, and reverted in the mean-time? > > > > I'm sorry for the breakage. The commit had the goal to lessen > > port build problems caused by the misled assumptions that the > > port was being built on a GLIBC based system. > > > > Given that we've now iterated on this a couple of times, this likely > should have all been backed out and exp-run'd *way* sooner. Exp-runs are great when there is a path forward from fixing the breakage. In the case where you have some random ports broken, and no ports maintainers responses for explicit queries, it is basically a deadlock. I definitely cannot go over some random but large set of ports fixing them, while maintainers are silent. In fact, with this set of changes, I initially provided some tools in base that were intended to ease the ports life, but still required some (minimal) involvement from the maintainers, like passing -DWITH_CPU_SET_T to C/C++ compiler. I asked more than once if these tools are desirable helper or not, with no avail. So my only route forward was to leave the state in the minimal damaging mode as I see it from bug reports, and wait for maintainers to do _something_. I am very grateful that Stefan took the torch and started massaging the CPU_XXX ugliness into more compatibility with glibc. This again happens in the same silence mode from maintainers, so if we want a progress in this area, it have to go this way. > > > In the case of the Python language ports, one additional macro > > was required and has been added in commit cb65d4432aed11. > > > > Since the official package builders have not been upgraded to > > a -CURRENT with this change, they are not affected. But I'll > > watch the failed build logs on beefy18. Right. > > This is a mindset that we all take, but we really need to work towards > improving. Once we're watching fallout logs on the official builders, > we've already lost. This is the kind of thing that helps promote the > idea that -CURRENT isn't stable enough for production uses: we start > accepting that we can be a little more lenient on identifying > ports-breaking changes because it's -CURRENT and we lose a fraction of > the ports tree because we've only sniped off individual ports as they > come up. > > portmgr@ is able and willing to run exp-runs for changes like this, we > really need to take advantage of that to avoid this kind of follow-up. There, you are blocking src changes by putting unreasonable requirements on src committers to fix ports breakage. I am willing to work together with ports maintainers, but I am not willing to handle things in silence and neglect of other' (my) work. I have similar experience with ino64 FWIW, but I was too naive at that time and indeed tried to fix all ports breakage, including digging into rust/ghc builds. I learned since, I will not do that again. > > > > > > I'm not opposed to a revert and exp-run, but I'm convinced that > > any fall-out from this change is easily fixed, and I'm willing > > to quickly fix any ports or base system components affected. > > > > That's probably not necessary at this point given that we're now N > commits deep into the cpuset.h/sched.h saga, but I really would have > liked to see us be more open to the idea. > > Thanks, > > Kyle Evans
Re: git: e2650af157bc - main - Make CPU_SET macros compliant with other implementations
From: Kyle Evans wrote on Date: Fri, 31 Dec 2021 09:37:16 -0600 : > . . . > This is a mindset that we all take, but we really need to work towards > improving. Once we're watching fallout logs on the official builders, > we've already lost. This is the kind of thing that helps promote the > idea that -CURRENT isn't stable enough for production uses: Hmm. Quoting: https://www.freebsd.org/where/ QUOTE If you are interested in a purely experimental snapshot release of FreeBSD-CURRENT (AKA 14.0-CURRENT), aimed at developers and bleeding-edge testers only, then please see the FreeBSD Snapshot Releases page. END QUOTE I thought that FreeBSD avoided promoting -CURRENT for production uses. > we start > accepting that we can be a little more lenient on identifying > ports-breaking changes because it's -CURRENT and we lose a fraction of > the ports tree because we've only sniped off individual ports as they > come up. > . . . === Mark Millard marklmi at yahoo.com
Re: git: e2650af157bc - main - Make CPU_SET macros compliant with other implementations
On Fri, Dec 31, 2021 at 4:22 AM Stefan Esser wrote: > > Am 31.12.21 um 09:01 schrieb Antoine Brodin: > > On Thu, Dec 30, 2021 at 11:54 AM Stefan Eßer wrote: > >> > >> The branch main has been updated by se: > >> > >> URL: > >> https://cgit.FreeBSD.org/src/commit/?id=e2650af157bc7489deaf2c9054995f0f88a6e5da > >> > >> commit e2650af157bc7489deaf2c9054995f0f88a6e5da > >> Author: Stefan Eßer > >> AuthorDate: 2021-12-30 11:20:32 + > >> Commit: Stefan Eßer > >> CommitDate: 2021-12-30 11:20:32 + > >> > [...] > >> Ports that have added -D_WITH_CPU_SET_T to build on -CURRENT do > >> no longer require that option. > >> > >> The FreeBSD version has been bumped to 1400046 to reflect this > >> incompatible change. > >> > >> Reviewed by:kib > >> MFC after: 2 weeks > >> Relnotes: yes > >> Differential Revision: https://reviews.freebsd.org/D33451 > > > > Hi, > > > > This breaks a lot of ports, like lang/python38. > > Could these kinds of changes on public headers be tested with an > > exp-run, and reverted in the mean-time? > > I'm sorry for the breakage. The commit had the goal to lessen > port build problems caused by the misled assumptions that the > port was being built on a GLIBC based system. > Given that we've now iterated on this a couple of times, this likely should have all been backed out and exp-run'd *way* sooner. > In the case of the Python language ports, one additional macro > was required and has been added in commit cb65d4432aed11. > > Since the official package builders have not been upgraded to > a -CURRENT with this change, they are not affected. But I'll > watch the failed build logs on beefy18. > This is a mindset that we all take, but we really need to work towards improving. Once we're watching fallout logs on the official builders, we've already lost. This is the kind of thing that helps promote the idea that -CURRENT isn't stable enough for production uses: we start accepting that we can be a little more lenient on identifying ports-breaking changes because it's -CURRENT and we lose a fraction of the ports tree because we've only sniped off individual ports as they come up. portmgr@ is able and willing to run exp-runs for changes like this, we really need to take advantage of that to avoid this kind of follow-up. > > I'm not opposed to a revert and exp-run, but I'm convinced that > any fall-out from this change is easily fixed, and I'm willing > to quickly fix any ports or base system components affected. > That's probably not necessary at this point given that we're now N commits deep into the cpuset.h/sched.h saga, but I really would have liked to see us be more open to the idea. Thanks, Kyle Evans
Re: git: e2650af157bc - main - Make CPU_SET macros compliant with other implementations
Am 31.12.21 um 09:01 schrieb Antoine Brodin: > On Thu, Dec 30, 2021 at 11:54 AM Stefan Eßer wrote: >> >> The branch main has been updated by se: >> >> URL: >> https://cgit.FreeBSD.org/src/commit/?id=e2650af157bc7489deaf2c9054995f0f88a6e5da >> >> commit e2650af157bc7489deaf2c9054995f0f88a6e5da >> Author: Stefan Eßer >> AuthorDate: 2021-12-30 11:20:32 + >> Commit: Stefan Eßer >> CommitDate: 2021-12-30 11:20:32 + >> [...] >> Ports that have added -D_WITH_CPU_SET_T to build on -CURRENT do >> no longer require that option. >> >> The FreeBSD version has been bumped to 1400046 to reflect this >> incompatible change. >> >> Reviewed by:kib >> MFC after: 2 weeks >> Relnotes: yes >> Differential Revision: https://reviews.freebsd.org/D33451 > > Hi, > > This breaks a lot of ports, like lang/python38. > Could these kinds of changes on public headers be tested with an > exp-run, and reverted in the mean-time? I'm sorry for the breakage. The commit had the goal to lessen port build problems caused by the misled assumptions that the port was being built on a GLIBC based system. In the case of the Python language ports, one additional macro was required and has been added in commit cb65d4432aed11. Since the official package builders have not been upgraded to a -CURRENT with this change, they are not affected. But I'll watch the failed build logs on beefy18. I'm not opposed to a revert and exp-run, but I'm convinced that any fall-out from this change is easily fixed, and I'm willing to quickly fix any ports or base system components affected. Regards, STefan OpenPGP_signature Description: OpenPGP digital signature
Re: git: e2650af157bc - main - Make CPU_SET macros compliant with other implementations
On Thu, Dec 30, 2021 at 11:54 AM Stefan Eßer wrote: > > The branch main has been updated by se: > > URL: > https://cgit.FreeBSD.org/src/commit/?id=e2650af157bc7489deaf2c9054995f0f88a6e5da > > commit e2650af157bc7489deaf2c9054995f0f88a6e5da > Author: Stefan Eßer > AuthorDate: 2021-12-30 11:20:32 + > Commit: Stefan Eßer > CommitDate: 2021-12-30 11:20:32 + > > Make CPU_SET macros compliant with other implementations > > The introduction of improved compatibility with some 3rd > party software, but caused the configure scripts of some ports to > assume that they were run in a GLIBC compatible environment. > > Parts of sched.h were made conditional on -D_WITH_CPU_SET_T being > added to ports, but there still were compatibility issues due to > invalid assumptions made in autoconfigure scripts. > > The differences between the FreeBSD version of macros like CPU_AND, > CPU_OR, etc. and the GLIBC versions was in the number of arguments: > FreeBSD used a 2-address scheme (one source argument is also used as > the destination of the operation), while GLIBC uses a 3-adderess > scheme (2 source operands and a separately passed destination). > > The GLIBC scheme provides a super-set of the functionality of the > FreeBSD macros, since it does not prevent passing the same variable > as source and destination arguments. In code that wanted to preserve > both source arguments, the FreeBSD macros required a temporary copy of > one of the source arguments. > > This patch set allows to unconditionally provide functions and macros > expected by 3rd party software written for GLIBC based systems, but > breaks builds of externally maintained sources that use any of the > following macros: CPU_AND, CPU_ANDNOT, CPU_OR, CPU_XOR. > > One contributed driver (contrib/ofed/libmlx5) has been patched to > support both the old and the new CPU_OR signatures. If this commit > is merged to -STABLE, the version test will have to be extended to > cover more ranges. > > Ports that have added -D_WITH_CPU_SET_T to build on -CURRENT do > no longer require that option. > > The FreeBSD version has been bumped to 1400046 to reflect this > incompatible change. > > Reviewed by:kib > MFC after: 2 weeks > Relnotes: yes > Differential Revision: https://reviews.freebsd.org/D33451 Hi, This breaks a lot of ports, like lang/python38. Could these kinds of changes on public headers be tested with an exp-run, and reverted in the mean-time? Antoine
