Re: changing 'configure' options for testing
Ryan asked (when he had this same issue about six months ago) why our cmake
PortGroup sets it ON when it breaks all the in-tree tests, and whether we
should change the PG to turn it OFF all the time. (The default is OFF I
believe.)
In fact, we probably should. I'm not sure why we set it like it was.
Ken
On Tue, Sep 28, 2021 at 1:49 PM Daniel J. Luke wrote:
> Thanks Ken and Jason!
>
> This is helpful (now I just need to figure out why a couple of tests fail
> when run this way vs. when run against the installed libraries).
>
> I was worried that the installed rpath would be different with
> CMAKE_BUILD_WITH_INSTALL_RPATH:BOOL=OFF, but it looks like it gets set to
> $prefix in destroot, so the installed binaries are fine.
>
> > On Sep 28, 2021, at 11:38 AM, Jason Liu wrote:
> >
> > In addition to the code Ken just provided, he and others have helped me
> with some test-related "protection" code, so that in addition to a +tests
> variant, there is the following section of code (I usually place it after
> any build-related sections, and before any destroot-related sections, in
> order to keep my portfiles laid out in the same order as the port phases):
> >
> > pre-test {
> > if {![variant_isset tests]} {
> > ui_error "'tests' variant must be activated to enable test
> support"
> > error "Please enable the 'tests' variant and try again"
> > }
> > }
> >
> > --
> > Jason Liu
> >
> >
> > On Tue, Sep 28, 2021 at 11:13 AM Ken Cunningham <
> ken.cunningham.web...@gmail.com> wrote:
> >
> > On 2021-09-28 7:44 a.m., Daniel J. Luke wrote:
> > > On Sep 20, 2021, at 10:20 AM, Daniel J. Luke
> wrote:
> > >> On Sep 20, 2021, at 8:15 AM, Frank Dean wrote:
> > >>> Daniel J. Luke writes:
> > The newest version of clamav uses cmake for builds. In the
> 'configure' stage, I have it disabling tests because otherwise it won't
> build without the test dependencies installed (check and pytest).
> >
> > Do we have a template or example of a canonical way to handle this?
> I don't see an obvious hook for when someone is running `port test` to
> change configure.args (I could, of course, add a post-extract/pre-configure
> and do some non-declaritive test to see if `port test` is being run and use
> that to branch - but that feels like a bad design choice).
> > >>> The rapidjson port implements a 'tests' variant to handle a similar
> > >>> situation. I used the same pattern for the libosmium port. The
> tests
> > >>> can then be run with `sudo port -d test current +tests`.
> > >> That works, I guess.
> > >>
> > >> Is there interest in having base auto-add +tests if `port test` is
> called? (I haven't looked at base/ code in a while, but it seems possible).
> I like to imagine a future where we have enough infrastructure that we
> would run `port test` for any ports that have test.run set.
> > > On this same train of thought - clamav tests run against the installed
> libraries (like some other ports tend to do) - in long-ago times I'd solve
> this by setting DYLD_LIBRARY_PATH - but since SIP started removing these
> that no longer works normally. I think trace mode has a(n elaborate)
> workaround where it copies binaries and executes the copies, but that
> doesn't seem like something I should implement in a portfile ;-) [This is
> another instance where if trace mode were the default, things would 'just
> work']
> > >
> > > Has anyone else already solved this?
> >
> >
> > The way our cmake PortGroup sets things up, running tests is harder. You
> > can configure cmake to allow testing to find the libraries in the build
> > tree. See for example what I did in libcbor:
> >
> >
> > variant tests description {enable tests} {
> > depends_test-append port:cmocka
> > configure.args-append -DWITH_TESTS=ON
> > configure.pre_args-replace -DCMAKE_BUILD_WITH_INSTALL_RPATH:BOOL=ON
> \
> > -DCMAKE_BUILD_WITH_INSTALL_RPATH:BOOL=OFF
> > test.runyes
> > test.target test
> > }
> >
> > Ken
>
> --
> Daniel J. Luke
>
>
Re: changing 'configure' options for testing
In addition to the code Ken just provided, he and others have helped me
with some test-related "protection" code, so that in addition to a
+tests variant, there is the following section of code (I usually place it
after any build-related sections, and before any destroot-related sections,
in order to keep my portfiles laid out in the same order as the port
phases):
pre-test {
if {![variant_isset tests]} {
ui_error "'tests' variant must be activated to enable test support"
error "Please enable the 'tests' variant and try again"
}
}
--
Jason Liu
On Tue, Sep 28, 2021 at 11:13 AM Ken Cunningham <
ken.cunningham.web...@gmail.com> wrote:
>
> On 2021-09-28 7:44 a.m., Daniel J. Luke wrote:
> > On Sep 20, 2021, at 10:20 AM, Daniel J. Luke wrote:
> >> On Sep 20, 2021, at 8:15 AM, Frank Dean wrote:
> >>> Daniel J. Luke writes:
> The newest version of clamav uses cmake for builds. In the
> 'configure' stage, I have it disabling tests because otherwise it won't
> build without the test dependencies installed (check and pytest).
>
> Do we have a template or example of a canonical way to handle this? I
> don't see an obvious hook for when someone is running `port test` to change
> configure.args (I could, of course, add a post-extract/pre-configure and do
> some non-declaritive test to see if `port test` is being run and use that
> to branch - but that feels like a bad design choice).
> >>> The rapidjson port implements a 'tests' variant to handle a similar
> >>> situation. I used the same pattern for the libosmium port. The tests
> >>> can then be run with `sudo port -d test current +tests`.
> >> That works, I guess.
> >>
> >> Is there interest in having base auto-add +tests if `port test` is
> called? (I haven't looked at base/ code in a while, but it seems possible).
> I like to imagine a future where we have enough infrastructure that we
> would run `port test` for any ports that have test.run set.
> > On this same train of thought - clamav tests run against the installed
> libraries (like some other ports tend to do) - in long-ago times I'd solve
> this by setting DYLD_LIBRARY_PATH - but since SIP started removing these
> that no longer works normally. I think trace mode has a(n elaborate)
> workaround where it copies binaries and executes the copies, but that
> doesn't seem like something I should implement in a portfile ;-) [This is
> another instance where if trace mode were the default, things would 'just
> work']
> >
> > Has anyone else already solved this?
>
>
> The way our cmake PortGroup sets things up, running tests is harder. You
> can configure cmake to allow testing to find the libraries in the build
> tree. See for example what I did in libcbor:
>
>
> variant tests description {enable tests} {
> depends_test-append port:cmocka
> configure.args-append -DWITH_TESTS=ON
> configure.pre_args-replace -DCMAKE_BUILD_WITH_INSTALL_RPATH:BOOL=ON \
> -DCMAKE_BUILD_WITH_INSTALL_RPATH:BOOL=OFF
> test.runyes
> test.target test
> }
>
> Ken
>
>
>
>
>
>
> >
>
Re: changing 'configure' options for testing
On 2021-09-28 7:44 a.m., Daniel J. Luke wrote:
On Sep 20, 2021, at 10:20 AM, Daniel J. Luke wrote:
On Sep 20, 2021, at 8:15 AM, Frank Dean wrote:
Daniel J. Luke writes:
The newest version of clamav uses cmake for builds. In the 'configure' stage, I
have it disabling tests because otherwise it won't build without the test
dependencies installed (check and pytest).
Do we have a template or example of a canonical way to handle this? I don't see
an obvious hook for when someone is running `port test` to change
configure.args (I could, of course, add a post-extract/pre-configure and do
some non-declaritive test to see if `port test` is being run and use that to
branch - but that feels like a bad design choice).
The rapidjson port implements a 'tests' variant to handle a similar
situation. I used the same pattern for the libosmium port. The tests
can then be run with `sudo port -d test current +tests`.
That works, I guess.
Is there interest in having base auto-add +tests if `port test` is called? (I
haven't looked at base/ code in a while, but it seems possible). I like to
imagine a future where we have enough infrastructure that we would run `port
test` for any ports that have test.run set.
On this same train of thought - clamav tests run against the installed
libraries (like some other ports tend to do) - in long-ago times I'd solve this
by setting DYLD_LIBRARY_PATH - but since SIP started removing these that no
longer works normally. I think trace mode has a(n elaborate) workaround where
it copies binaries and executes the copies, but that doesn't seem like
something I should implement in a portfile ;-) [This is another instance where
if trace mode were the default, things would 'just work']
Has anyone else already solved this?
The way our cmake PortGroup sets things up, running tests is harder. You
can configure cmake to allow testing to find the libraries in the build
tree. See for example what I did in libcbor:
variant tests description {enable tests} {
depends_test-append port:cmocka
configure.args-append -DWITH_TESTS=ON
configure.pre_args-replace -DCMAKE_BUILD_WITH_INSTALL_RPATH:BOOL=ON \
-DCMAKE_BUILD_WITH_INSTALL_RPATH:BOOL=OFF
test.run yes
test.target test
}
Ken
Re: changing 'configure' options for testing
On Sep 20, 2021, at 10:20 AM, Daniel J. Luke wrote: > On Sep 20, 2021, at 8:15 AM, Frank Dean wrote: >> Daniel J. Luke writes: >>> The newest version of clamav uses cmake for builds. In the 'configure' >>> stage, I have it disabling tests because otherwise it won't build without >>> the test dependencies installed (check and pytest). >>> >>> Do we have a template or example of a canonical way to handle this? I don't >>> see an obvious hook for when someone is running `port test` to change >>> configure.args (I could, of course, add a post-extract/pre-configure and do >>> some non-declaritive test to see if `port test` is being run and use that >>> to branch - but that feels like a bad design choice). >> >> The rapidjson port implements a 'tests' variant to handle a similar >> situation. I used the same pattern for the libosmium port. The tests >> can then be run with `sudo port -d test current +tests`. > > That works, I guess. > > Is there interest in having base auto-add +tests if `port test` is called? (I > haven't looked at base/ code in a while, but it seems possible). I like to > imagine a future where we have enough infrastructure that we would run `port > test` for any ports that have test.run set. On this same train of thought - clamav tests run against the installed libraries (like some other ports tend to do) - in long-ago times I'd solve this by setting DYLD_LIBRARY_PATH - but since SIP started removing these that no longer works normally. I think trace mode has a(n elaborate) workaround where it copies binaries and executes the copies, but that doesn't seem like something I should implement in a portfile ;-) [This is another instance where if trace mode were the default, things would 'just work'] Has anyone else already solved this? -- Daniel J. Luke
