On 09/09/16 14:17, Dirk Eddelbuettel wrote: > | The plan was to avoid seeing eg, lintian hardening-no-bindnow warnings > | on packages with compiled extensions. I tried injecting dpkg-buildflags > | LDFLAGS output into the MAKEFLAGS environment for R CMD INSTALL but it > | doesn't appear to work. It is possible this is also not particularly > | important (and/or might interfere with R's own build config) - I don't > | really see security as a major R issue. > > It's an R issue. R "remembers" CC, CFLAGS, CXX, CXXFLAGS, etc pp from its > configure run and reuses them. When R runs 'R CMD INSTALL' you can add to > this on a per-package basis via the PKG_* variants from inside the package. > But we can't override from the outside. Minor annoyance (ie would be nice to > strip '-g' sometimes, toggle -O2 for -O3, ...). Needs an upstream change.
I understood (from how the CDBS makefile worked) that you could set an env var MAKEFLAGS encapsulating other build flags and it would be used. I was trying to use the same mechanism, ie MAKEFLAGS=LDFLAGS=-z,now R CMD INSTALL ... Is this mechanism no longer useable (or have I misunderstood what it should be able to do)? > > | > I had once tried to bring some perspective from the R ecosystem to that, > but > | > as I recall I pretty much failed to get my point across. R already does > /a > | > lot/ here via R CMD check, and it may be worthwhile to milk that angle > rather > | > than to shoehorn R (which builds packages somewhat idiosyncratically) into > | > Debian's framework. Anyway, minor 'day 2' detail. > | > | I suspect our testing is most interesting where non-R dependencies are > | concerned, particularly where we remove bundled copies and force the use > | of system ones. > | > | As with vignette building below, it requires the set of build-deps to be > | much bigger (and particularly for bioconductor, tends to require big > | dataset packages available). > > CRAN and BioConductor don't seem to know/care about chroot and minimal > environments. They tend to build on "fat" systems with everything installed. > > But they also do it across three or more OSs which we don't so we can take > advantage of Linux tricks where they can't. > > Some of the test philosophy is the same and hence redundant. R cares a lot > about build "stability" and "possibility" (ie no API changes), we care a lot > about changes in the toolchain (and so do they). Eg Brian Ripley in Oxford > tends to also check again as-of-yet unreleased (!!) gcc versions to catch > things early (but those checks are extra -- at the core where tests are > automated they are more sensible). > > | My workflow is to run autopkgtest (in an schroot) as a postbuild step, > | which tests the built deb (so no workaround for testing a not-yet > | installed package is required, and ensuring the binary dependencies are > > I fully agree that chroot tests are good, but our build does thosse ... > > | correct), and autodep8 ensures that there is at least a `library(RLIB)` > | test defined for every package. > > ... because as I tried to explain last time, _each_ R CMD INSTALL involved > in _each_ of our builds includes a load of the just built library. From what > I understood then and understand now you are duplicating that. But I may > miss sufficient coffee... Anyway Yes, I've read #752609 and I think I understood your meaning - that anything coming directly from CRAN/bioconductor should have fairly strong guarantees of correctness, and re-checking that at build time is extra unnecessary work for us. (Although I suspect there must be some edge cases we might nevertheless catch at build-time, I can't admittedly come up with a concrete one right now). I don't think it's quite the same argument as autopkgtest though. Running autopkgtest immediately post-build checks that the runtime Depends: are correct. (Hopefully using a substvar instead of a manual list should mostly make this issue go away, but I've certainly seen uploaded packages with insufficient Depends: to be loaded). Running them periodically after (as ci.d.n does) tests for bit rot due to whatever reason (eg, dependencies which should be versioned but aren't). > > | (stdout from R CMD INSTALL includes "checking if the package can be > | loaded" or something like that, but I'm not sure what that is actually > | testing in practice) > > The very 'library(foo)' for a the package r-cran-foo you are building. > > Which checks dynamic linker as well as R aspects (as the R code needs to > syntactially parseable etc pp). It is a good test. I just fell over it > yesterday on one or two small packages I am building right now. > > | > | > It might also be interesting to explore: > | > | > > | > | > * handling binary dependencies between R compiled extensions > | > | > | > | What exactly do you mean here? I admit I'd welcome if this dh module > | > | would be available in sid soon to enable switching packages to it to get > | > | it fully tested soon. This kind of features might wait for later > | > | versions. > | > | See, eg, r-cran-rprotobuf (LinkingTo: Rcpp) or r-bioc-biostrings > | (LinkingTo: IRanges). > > Being the author of Rcpp which is used by 750+ CRAN packages via LinkingTo > (whereas we recommended Depends in the day), I am aware of its limitation. It > is (pretty much) R's version of Build-Depends for the C++ case of header-only > libraries. > > | My understanding is that this means that these packages build a binary > | extension using header files from another R package, and may link to the > | so file from the other package at runtime. > > Not necessarily a so file. > > It is for C++ code where you can get by with headers only. Most prominent > example is the BH package I maintain -- it has 120mb of Boost but __without > linking__ as that makes deployment so much easier across OS. > > BH, RcppArmadillo, RcppEigen, ... all get used by _their_ client packages > just via headers. What they _would_ need in standalone use (ie if your built > a main()) they get from R: LAPACK, BLAS, ... > > | I am wondering if this should be interpreted as a tight dependency on > | the LinkingTo: package, eg > | > | Built-Using: r-cran-rcpp (=version) or > | Depends: r-cran-rcpp (=version) > > No. > > | or if we need to trigger a transition when a package which can be the > | subject of LinkingTo: changes. > > No. > > | I'm speculating a bit here - I don't know how often this is a > | problem/how stable in practice package ABIs are. This might just be > | adding more work for a rare problem. > > Yes. That's a nice easy answer to those questions, thanks. No special treatment of this header needed, then.