Re: having the "+test" or "+tests" variant propagate causes unexpected builds
On Oct 30, 2022, at 11:42 PM, Ken Cunningham wrote: > I wonder if there was consideration given way-back-when to the idea of having > NO propagation of variants at all. It was an intentional design. > Anything you wanted to apply to every port, you would put in variants.conf. > Otherwise, your variant applied to the port you were currently installing, > and that is it. The behavior predates variants.conf > To me — that makes quite a bit of sense, actually…might solve a lot of > problems. > > But I’m sure some scenario arises where it was needed. Maybe? We could probably figure out what depends on the current behavior and then decide whether to change it or not. Back in the beginning, we didn't really know how much to use or not use variants and over time we've (ab)used variants for a lot more than just enabling or disabling a set of features on a given port. -- Daniel J. Luke
Re: having the "+test" or "+tests" variant propagate causes unexpected builds
I wonder if there was consideration given way-back-when to the idea of having NO propagation of variants at all. Anything you wanted to apply to every port, you would put in variants.conf. Otherwise, your variant applied to the port you were currently installing, and that is it. To me — that makes quite a bit of sense, actually…might solve a lot of problems. But I’m sure some scenario arises where it was needed. K
Re: having the "+test" or "+tests" variant propagate causes unexpected builds
I have the same situation in a port I maintain. I would like to be able to specify pre/post phase actions inside a phase block (so when running ‘port test’ a pre-configure that does the necessary setup will run) I haven’t looked at base/ to see how I’d want to implement it, but I think the port files would look cleaner (and we wouldn’t need variant-specific magic), so complexity there seems worthwhile. -- Daniel J. Luke > On Oct 30, 2022, at 6:23 AM, Ryan Schmidt wrote: > > On Oct 28, 2022, at 21:33, Daniel J. Luke wrote: >> >> I don't think implementation difficulty is the barrier here - but that all >> variants should just have the same behavior. >> >> In my mind, the real problem is the need for +test variants, there should be >> a way to just use the test phase - and perhaps changes to base/ to enable >> that are a better option. > > In one of my ports that has a tests variant, the reason why the variant > exists is that the build system looks for certain test dependencies at > configure time. If they're not there, it doesn't build the test suite and > doesn't allow tests to be run later. I'm not sure how MacPorts could be > improved to handle that better in the absence of a tests variant. Would you > have MacPorts do the configuration in the configure phase, do the build in > the build phase, and then redo the configuration and build in the test phase? > Or would you suggest in this case that the test dependencies that are needed > at configure time should be added unconditionally, so that even users who > won't be running the tests need to install them? In my port's case the > dependencies are probably small and that wouldn't make much of a difference, > but I'm not sure it'll always be that way for all ports. > >
Re: having the "+test" or "+tests" variant propagate causes unexpected builds
Test dependencies exist already. MacPorts Guideguide.macports.org--Daniel J. LukeOn Oct 30, 2022, at 7:30 AM, Nils Breunese wrote:Maybe MacPorts could introduce the concept of ‘test dependencies’ (dependencies only required for running tests). I work with Apache Maven quite a bit, and apart from build and runtime dependencies (which MacPorts already distinguishes) it also supports declaring test scope dependencies, that don’t need to be installed until tests are actually run: https://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html#Dependency_ScopeNils.Op 30 okt. 2022 om 11:23 heeft Ryan Schmidt het volgende geschreven:On Oct 28, 2022, at 21:33, Daniel J. Luke wrote:I don't think implementation difficulty is the barrier here - but that all variants should just have the same behavior.In my mind, the real problem is the need for +test variants, there should be a way to just use the test phase - and perhaps changes to base/ to enable that are a better option.In one of my ports that has a tests variant, the reason why the variant exists is that the build system looks for certain test dependencies at configure time. If they're not there, it doesn't build the test suite and doesn't allow tests to be run later. I'm not sure how MacPorts could be improved to handle that better in the absence of a tests variant. Would you have MacPorts do the configuration in the configure phase, do the build in the build phase, and then redo the configuration and build in the test phase? Or would you suggest in this case that the test dependencies that are needed at configure time should be added unconditionally, so that even users who won't be running the tests need to install them? In my port's case the dependencies are probably small and that wouldn't make much of a difference, but I'm not sure it'll always be that way for all ports.
Re: having the "+test" or "+tests" variant propagate causes unexpected builds
Maybe MacPorts could introduce the concept of ‘test dependencies’ (dependencies only required for running tests). I work with Apache Maven quite a bit, and apart from build and runtime dependencies (which MacPorts already distinguishes) it also supports declaring test scope dependencies, that don’t need to be installed until tests are actually run: https://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html#Dependency_Scope Nils. > Op 30 okt. 2022 om 11:23 heeft Ryan Schmidt het > volgende geschreven: > > On Oct 28, 2022, at 21:33, Daniel J. Luke wrote: >> >> I don't think implementation difficulty is the barrier here - but that all >> variants should just have the same behavior. >> >> In my mind, the real problem is the need for +test variants, there should be >> a way to just use the test phase - and perhaps changes to base/ to enable >> that are a better option. > > In one of my ports that has a tests variant, the reason why the variant > exists is that the build system looks for certain test dependencies at > configure time. If they're not there, it doesn't build the test suite and > doesn't allow tests to be run later. I'm not sure how MacPorts could be > improved to handle that better in the absence of a tests variant. Would you > have MacPorts do the configuration in the configure phase, do the build in > the build phase, and then redo the configuration and build in the test phase? > Or would you suggest in this case that the test dependencies that are needed > at configure time should be added unconditionally, so that even users who > won't be running the tests need to install them? In my port's case the > dependencies are probably small and that wouldn't make much of a difference, > but I'm not sure it'll always be that way for all ports. > >
Re: having the "+test" or "+tests" variant propagate causes unexpected builds
On Oct 28, 2022, at 21:33, Daniel J. Luke wrote: > > I don't think implementation difficulty is the barrier here - but that all > variants should just have the same behavior. > > In my mind, the real problem is the need for +test variants, there should be > a way to just use the test phase - and perhaps changes to base/ to enable > that are a better option. In one of my ports that has a tests variant, the reason why the variant exists is that the build system looks for certain test dependencies at configure time. If they're not there, it doesn't build the test suite and doesn't allow tests to be run later. I'm not sure how MacPorts could be improved to handle that better in the absence of a tests variant. Would you have MacPorts do the configuration in the configure phase, do the build in the build phase, and then redo the configuration and build in the test phase? Or would you suggest in this case that the test dependencies that are needed at configure time should be added unconditionally, so that even users who won't be running the tests need to install them? In my port's case the dependencies are probably small and that wouldn't make much of a difference, but I'm not sure it'll always be that way for all ports.
Re: having the "+test" or "+tests" variant propagate causes unexpected builds
I don't think implementation difficulty is the barrier here - but that all variants should just have the same behavior. In my mind, the real problem is the need for +test variants, there should be a way to just use the test phase - and perhaps changes to base/ to enable that are a better option. > On Oct 23, 2022, at 7:58 PM, Jason Liu wrote: > > My own personal opinion has been that +test/+tests and +debug, by default, > should not propagate through the chain of dependencies; and then perhaps > there might be some way to enable propagation (maybe with a command line > option?). > > However, if I recall correctly, all variants propagate through the dependency > chain, so it might be difficult to make certain variant keywords behave > differently? > > -- > Jason Liu > > > On Sun, Oct 23, 2022 at 1:58 PM Ken Cunningham > wrote: > Various ports implement a “test” or “tests” variant to allow extra features > and deps required for testing to be enabled. > > This variant, when requested, will propagate up the chain to all the ports, > however. There is no real use case where someone would desire the test/s > variant to propagate up. > > This generates needless builds, and often enables features people neither > need nor want, and then guarantees manual rebuilds, forever, of the involved > ports. > > I recently came back to a massive building project involving clang and llvm > when I was trying to build “mesa +tests”. Because clang-15 and llvm-15 also > have a “+tests” variant, and had not yet been installed, port was building > those (and possibly others) with the tests variant rather than use the > prebuilt binary. > > Of course I just aborted the huge llvm/clang-15 build, cleaned them up, and > installed them separately. But others would probably not know to do this. > > I had suggested a few years ago we might namespace the test/tests variants, > by having a convention that the portname be prepended to the test variant, to > be more specific and avoid this — but not a widely acceptable idea at that > time. So we’re still in the same situation… > > Is it possible that a “test” or “tests” variant might not be propagated up > the ports chain by base, instead? > > > K > > > > PS. A similar thing happens with “+debug” variants, another common variant > that you *usually* don’t want propagated up to *every single port* in the > chain either. > > This one is occasionally something that people would want up their chain, but > it is so fragile of a plan to rely on variant propagation (ie if you have the > port installed already, it won’t be reinstalled with the “+debug” variant), > that such rare users might best install each port they want to be installed > as “debug” do that specifically. Certainly most of us don’t want clang-15 > installed with it’s debug variant when you’re trying to debug some little > port. -- Daniel J. Luke
Re: having the "+test" or "+tests" variant propagate causes unexpected builds
On Sun, Oct 23, 2022 at 1:58 PM Ken Cunningham wrote: > > Various ports implement a “test” or “tests” variant to allow extra features > and deps required for testing to be enabled. > > This variant, when requested, will propagate up the chain to all the ports, > however. There is no real use case where someone would desire the test/s > variant to propagate up. > > This generates needless builds, and often enables features people neither > need nor want, and then guarantees manual rebuilds, forever, of the involved > ports. > > I recently came back to a massive building project involving clang and llvm > when I was trying to build “mesa +tests”. Because clang-15 and llvm-15 also > have a “+tests” variant, and had not yet been installed, port was building > those (and possibly others) with the tests variant rather than use the > prebuilt binary. > > Of course I just aborted the huge llvm/clang-15 build, cleaned them up, and > installed them separately. But others would probably not know to do this. > > I had suggested a few years ago we might namespace the test/tests variants, > by having a convention that the portname be prepended to the test variant, to > be more specific and avoid this — but not a widely acceptable idea at that > time. So we’re still in the same situation… > > Is it possible that a “test” or “tests” variant might not be propagated up > the ports chain by base, instead? > > > K > > > > PS. A similar thing happens with “+debug” variants, another common variant > that you *usually* don’t want propagated up to *every single port* in the > chain either. > > This one is occasionally something that people would want up their chain, but > it is so fragile of a plan to rely on variant propagation (ie if you have the > port installed already, it won’t be reinstalled with the “+debug” variant), > that such rare users might best install each port they want to be installed > as “debug” do that specifically. Certainly most of us don’t want clang-15 > installed with it’s debug variant when you’re trying to debug some little > port. I wonder if this is what's tripping me up in 65981: https://trac.macports.org/ticket/65981
Re: having the "+test" or "+tests" variant propagate causes unexpected builds
My own personal opinion has been that +test/+tests and +debug, by default, should not propagate through the chain of dependencies; and then perhaps there might be some way to enable propagation (maybe with a command line option?). However, if I recall correctly, all variants propagate through the dependency chain, so it might be difficult to make certain variant keywords behave differently? -- Jason Liu On Sun, Oct 23, 2022 at 1:58 PM Ken Cunningham < ken.cunningham.web...@gmail.com> wrote: > Various ports implement a “test” or “tests” variant to allow extra > features and deps required for testing to be enabled. > > This variant, when requested, will propagate up the chain to all the > ports, however. There is no real use case where someone would desire the > test/s variant to propagate up. > > This generates needless builds, and often enables features people neither > need nor want, and then guarantees manual rebuilds, forever, of the > involved ports. > > I recently came back to a massive building project involving clang and > llvm when I was trying to build “mesa +tests”. Because clang-15 and llvm-15 > also have a “+tests” variant, and had not yet been installed, port was > building those (and possibly others) with the tests variant rather than use > the prebuilt binary. > > Of course I just aborted the huge llvm/clang-15 build, cleaned them up, > and installed them separately. But others would probably not know to do > this. > > I had suggested a few years ago we might namespace the test/tests > variants, by having a convention that the portname be prepended to the test > variant, to be more specific and avoid this — but not a widely acceptable > idea at that time. So we’re still in the same situation… > > Is it possible that a “test” or “tests” variant might not be propagated up > the ports chain by base, instead? > > > K > > > > PS. A similar thing happens with “+debug” variants, another common variant > that you *usually* don’t want propagated up to *every single port* in the > chain either. > > This one is occasionally something that people would want up their chain, > but it is so fragile of a plan to rely on variant propagation (ie if you > have the port installed already, it won’t be reinstalled with the “+debug” > variant), that such rare users might best install each port they want to be > installed as “debug” do that specifically. Certainly most of us don’t want > clang-15 installed with it’s debug variant when you’re trying to debug some > little port.
having the "+test" or "+tests" variant propagate causes unexpected builds
Various ports implement a “test” or “tests” variant to allow extra features and deps required for testing to be enabled. This variant, when requested, will propagate up the chain to all the ports, however. There is no real use case where someone would desire the test/s variant to propagate up. This generates needless builds, and often enables features people neither need nor want, and then guarantees manual rebuilds, forever, of the involved ports. I recently came back to a massive building project involving clang and llvm when I was trying to build “mesa +tests”. Because clang-15 and llvm-15 also have a “+tests” variant, and had not yet been installed, port was building those (and possibly others) with the tests variant rather than use the prebuilt binary. Of course I just aborted the huge llvm/clang-15 build, cleaned them up, and installed them separately. But others would probably not know to do this. I had suggested a few years ago we might namespace the test/tests variants, by having a convention that the portname be prepended to the test variant, to be more specific and avoid this — but not a widely acceptable idea at that time. So we’re still in the same situation… Is it possible that a “test” or “tests” variant might not be propagated up the ports chain by base, instead? K PS. A similar thing happens with “+debug” variants, another common variant that you *usually* don’t want propagated up to *every single port* in the chain either. This one is occasionally something that people would want up their chain, but it is so fragile of a plan to rely on variant propagation (ie if you have the port installed already, it won’t be reinstalled with the “+debug” variant), that such rare users might best install each port they want to be installed as “debug” do that specifically. Certainly most of us don’t want clang-15 installed with it’s debug variant when you’re trying to debug some little port.
