Re: [R-pkg-devel] Optional library dependencies for compiled code
I'd like to add, for bundling `zstd` and `lz4`, I was told by Professor Ripley that I must first check for system installations and use that when possible, because of possible security issues being fixed in later updates. I think that's a good idea, and can be set up with a configure script: 1) check for later version system installations e.g. with `pkg-config`, 2) use system if it exists, 3) otherwise compile bundled `zstd`. On Wed, Jan 29, 2020 at 8:40 AM Iñaki Ucar wrote: > On Wed, 29 Jan 2020 at 15:11, Guido Kraemer > wrote: > > > > I am developing an R package that can make (optional, but strongly > > recommended) use of a number of external compression algorithms. What is > > the best strategy if I want to distribute this as an R package? Because > > the package is for reading large external data sets, the package will > > probably not be very useful without compression. > > > > - Make a hard dependency on all external libraries and make user > > manually disable it if they cannot install a library for some reason? > > > > - Make no dependency on external libraries and have the user compile it > > manually if they want compression? > > > > - Look for installed libraries and use only these? (If so: How would I > > do this during the installation of an R package?) > > > > - How does this work on Windows? > > > > The possible compressions are: blosc, zlib, bzip2, lz4, xz > > You could take a look at the fst package [1], which bundles the zstd > library [2]. And that's probably everything you need. Or you may want > to directly use the fst package if your target is tabular data. > > [1] https://cran.r-project.org/web/packages/fst/index.html > [2] https://github.com/facebook/zstd > > __ > R-package-devel@r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-package-devel > [[alternative HTML version deleted]] __ R-package-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-package-devel
Re: [R-pkg-devel] Optional library dependencies for compiled code
Complex system dependencies are not cleanly handled by the R packaging system,
but Suggesting other packages is. You could create or re-use an existing
package for each external dependency and possibly even let the user specify
which one they want to your dependent package the way DBI handles database
drivers. This gives you a clean isolated system dependency for each
"compression driver" that you can handle with configure to be slick or force
the user to address the system dependency with a simple external call.
On January 29, 2020 7:30:25 AM PST, Dirk Eddelbuettel wrote:
>
>Guido,
>
>On 29 January 2020 at 15:11, Guido Kraemer wrote:
>| I am developing an R package that can make (optional, but strongly
>| recommended) use of a number of external compression algorithms. What
>is
>| the best strategy if I want to distribute this as an R package?
>Because
>
>It's actually a fairly difficult issue.
>
>I have packages on GitHub but not on CRAN for that very reason (of
>difficult
>to bundle external libraries).
>
>| the package is for reading large external data sets, the package will
>
>| probably not be very useful without compression.
>|
>| - Make a hard dependency on all external libraries and make user
>| manually disable it if they cannot install a library for some reason?
>
>Could do, but probably means the package cannot get onto CRAN as the
>build
>will fail "as shipped" ?
>
>Also forces users to set up build options etc. Hard. May reduce use.
>
>| - Make no dependency on external libraries and have the user compile
>it
>| manually if they want compression?
>
>Same, no? Too much user dependency.
>
>| - Look for installed libraries and use only these? (If so: How would
>I
>| do this during the installation of an R package?)
>
>I once added the ability to use a system / existing 'nlopt' library to
>the
>'nloptr' package. The package still defaults to downloading a tar.gz
>and
>building during its build, but can use the system library for a
>speedup. You
>could look at that. All done via configure.
>
>There are other "tricks". At least one package I know cheats (at least
>on
>Linux), comes empty and tells you to install what is needed and
>reinstall.
>That passes the letter of the law ("hey, the package builds on CRAN")
>but
>hardly the spirit of the law.
>
>| - How does this work on Windows?
>
>Even less easily.
>
>| The possible compressions are: blosc, zlib, bzip2, lz4, xz
>
>Some of these are a given with R.
>
>For my Debian builds I enable at least zlib, bzip2, lzma -- but I have
>to
>"pretty soon" look into a similar list for another project. My plan
>there is
>bundle the sources of the missing compressors (and other libraries I
>need)
>and trigger a static library build (for each, or maybe all) from
>configure.
>
>Dirk
--
Sent from my phone. Please excuse my brevity.
__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel
Re: [R-pkg-devel] Optional library dependencies for compiled code
Guido,
On 29 January 2020 at 15:11, Guido Kraemer wrote:
| I am developing an R package that can make (optional, but strongly
| recommended) use of a number of external compression algorithms. What is
| the best strategy if I want to distribute this as an R package? Because
It's actually a fairly difficult issue.
I have packages on GitHub but not on CRAN for that very reason (of difficult
to bundle external libraries).
| the package is for reading large external data sets, the package will
| probably not be very useful without compression.
|
| - Make a hard dependency on all external libraries and make user
| manually disable it if they cannot install a library for some reason?
Could do, but probably means the package cannot get onto CRAN as the build
will fail "as shipped" ?
Also forces users to set up build options etc. Hard. May reduce use.
| - Make no dependency on external libraries and have the user compile it
| manually if they want compression?
Same, no? Too much user dependency.
| - Look for installed libraries and use only these? (If so: How would I
| do this during the installation of an R package?)
I once added the ability to use a system / existing 'nlopt' library to the
'nloptr' package. The package still defaults to downloading a tar.gz and
building during its build, but can use the system library for a speedup. You
could look at that. All done via configure.
There are other "tricks". At least one package I know cheats (at least on
Linux), comes empty and tells you to install what is needed and reinstall.
That passes the letter of the law ("hey, the package builds on CRAN") but
hardly the spirit of the law.
| - How does this work on Windows?
Even less easily.
| The possible compressions are: blosc, zlib, bzip2, lz4, xz
Some of these are a given with R.
For my Debian builds I enable at least zlib, bzip2, lzma -- but I have to
"pretty soon" look into a similar list for another project. My plan there is
bundle the sources of the missing compressors (and other libraries I need)
and trigger a static library build (for each, or maybe all) from configure.
Dirk
--
http://dirk.eddelbuettel.com | @eddelbuettel | e...@debian.org
__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel
