Re: [R-pkg-devel] Best practices for CRAN package using Go
Simon,
Thanks! I look forward to packaging the next ADBC driver release and
submitting.
Cheers,
-dewey
On 2023-07-12 20:01, Simon Urbanek wrote:
Dewey,
you will definitely need to include all the necessary sources for your
package. You may want to have a look at the "Using Rust"[1] document
linked from the CRAN policy. I think Go is quite similar to Rust in
that sense so you should use the same approach, i.e. checking for
system and user installations (for go the official location is
/usr/local/go/bin/go and it may not be on the PATH), declaring Go
version dependency and making sure your package has included all module
dependency sources (i.e. don't use install-time module
resolution/download).
If you need to include a large source tar ball that is not permissible
on CRAN, I'd recommend using Zenodo.org since it is specifically
designed to facilitate longevity and reproducibility (as opposed to
Github or other transient storage that may disappear at any point).
All that said, you may run into the same issues as Rust (errors and
segfaults due to limited interoperability of compilers) so use with
care and test well. External bindings like Rust or Go are only provided
on "best effort" basis.
Cheers,
Simon
[1] - https://cran.r-project.org/web/packages/using_rust.html
PS: go is now available on the CRAN macOS builder machines and the Mac
Builder (https://mac.r-project.org/macbuilder/submit.html).
On 13/07/2023, at 2:36 AM, Dewey Dunnington
wrote:
Thank you! It seems I needed the refresher on CRAN policy regarding
downloading sources: it seems like the go.sum/go.mod provide
sufficient checksumming to comply with the policy, as you noted (with
`go mod vendor` as a backup if this turns out to not be acceptable).
Downloading Go is probably out based on the advice for Rust that
explicitly forbids this.
Cheers!
-dewey
On 2023-07-10 11:09, Ivan Krylov wrote:
В Thu, 06 Jul 2023 15:22:26 -0300
Dewey Dunnington пишет:
I've wrapped two of these drivers for R that seem to build and
install on MacOS, Linux, and Windows [3][4]; however, I am not sure
if the pattern I used is suitable for CRAN or whether these packages
will have to be GitHub-only for the foreseeable future.
There are a few parts to following the CRAN policy [*] regarding
external dependencies.
I think (but don't know for sure) that your package will not be
allowed
to download Go by itself. The policy says: "Only as a last resort and
with the agreement of the CRAN team should a package download
pre-compiled software."
An already installed Go should be able to "first look to see if [a
dependency] is already installed and if so is of a suitable version"
when installing the dependencies of the Go part of the code. The
go.mod
and go.sum files specify the exact versions and checksums of the
dependencies, which satisfies the requirement for fixed dependency
versions ("it is acceptable to download them as part of installation,
but do ensure that the download is of a fixed version rather than the
latest"), so your package seems to be fine in this respect.
One more thing: when bootstrapping the source package, can you run go
mod vendor [**] in order to bundle *all* the Go dependencies together
with the package? Is the resulting directory prohibitively large?
Would
it satisfy the CRAN policy preference to "include the library sources
in the package and compile them as part of package installation"
without requiring Internet access? Unfortunately, I don't know enough
about Go to answer these questions myself. I think that a small
bundle
of vendored Go code would be preferrable for CRAN but *not*
preferrable
for packaging in a GNU/Linux distro like Debian where dynamic linking
(in the widest possible sense) is a strong preference.
--
Best regards,
Ivan
[*] https://cran.r-project.org/web/packages/policies.html
[**] https://go.dev/ref/mod#vendoring
__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel
__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel
Re: [R-pkg-devel] Best practices for CRAN package using Go
Dewey,
you will definitely need to include all the necessary sources for your package.
You may want to have a look at the "Using Rust"[1] document linked from the
CRAN policy. I think Go is quite similar to Rust in that sense so you should
use the same approach, i.e. checking for system and user installations (for go
the official location is /usr/local/go/bin/go and it may not be on the PATH),
declaring Go version dependency and making sure your package has included all
module dependency sources (i.e. don't use install-time module
resolution/download).
If you need to include a large source tar ball that is not permissible on CRAN,
I'd recommend using Zenodo.org since it is specifically designed to facilitate
longevity and reproducibility (as opposed to Github or other transient storage
that may disappear at any point).
All that said, you may run into the same issues as Rust (errors and segfaults
due to limited interoperability of compilers) so use with care and test well.
External bindings like Rust or Go are only provided on "best effort" basis.
Cheers,
Simon
[1] - https://cran.r-project.org/web/packages/using_rust.html
PS: go is now available on the CRAN macOS builder machines and the Mac Builder
(https://mac.r-project.org/macbuilder/submit.html).
> On 13/07/2023, at 2:36 AM, Dewey Dunnington wrote:
>
> Thank you! It seems I needed the refresher on CRAN policy regarding
> downloading sources: it seems like the go.sum/go.mod provide sufficient
> checksumming to comply with the policy, as you noted (with `go mod vendor` as
> a backup if this turns out to not be acceptable). Downloading Go is probably
> out based on the advice for Rust that explicitly forbids this.
>
> Cheers!
>
> -dewey
>
> On 2023-07-10 11:09, Ivan Krylov wrote:
>> В Thu, 06 Jul 2023 15:22:26 -0300
>> Dewey Dunnington пишет:
>>> I've wrapped two of these drivers for R that seem to build and
>>> install on MacOS, Linux, and Windows [3][4]; however, I am not sure
>>> if the pattern I used is suitable for CRAN or whether these packages
>>> will have to be GitHub-only for the foreseeable future.
>> There are a few parts to following the CRAN policy [*] regarding
>> external dependencies.
>> I think (but don't know for sure) that your package will not be allowed
>> to download Go by itself. The policy says: "Only as a last resort and
>> with the agreement of the CRAN team should a package download
>> pre-compiled software."
>> An already installed Go should be able to "first look to see if [a
>> dependency] is already installed and if so is of a suitable version"
>> when installing the dependencies of the Go part of the code. The go.mod
>> and go.sum files specify the exact versions and checksums of the
>> dependencies, which satisfies the requirement for fixed dependency
>> versions ("it is acceptable to download them as part of installation,
>> but do ensure that the download is of a fixed version rather than the
>> latest"), so your package seems to be fine in this respect.
>> One more thing: when bootstrapping the source package, can you run go
>> mod vendor [**] in order to bundle *all* the Go dependencies together
>> with the package? Is the resulting directory prohibitively large? Would
>> it satisfy the CRAN policy preference to "include the library sources
>> in the package and compile them as part of package installation"
>> without requiring Internet access? Unfortunately, I don't know enough
>> about Go to answer these questions myself. I think that a small bundle
>> of vendored Go code would be preferrable for CRAN but *not* preferrable
>> for packaging in a GNU/Linux distro like Debian where dynamic linking
>> (in the widest possible sense) is a strong preference.
>> --
>> Best regards,
>> Ivan
>> [*] https://cran.r-project.org/web/packages/policies.html
>> [**] https://go.dev/ref/mod#vendoring
>
> __
> R-package-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-package-devel
>
__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel
Re: [R-pkg-devel] Best practices for CRAN package using Go
Thank you! It seems I needed the refresher on CRAN policy regarding
downloading sources: it seems like the go.sum/go.mod provide sufficient
checksumming to comply with the policy, as you noted (with `go mod
vendor` as a backup if this turns out to not be acceptable). Downloading
Go is probably out based on the advice for Rust that explicitly forbids
this.
Cheers!
-dewey
On 2023-07-10 11:09, Ivan Krylov wrote:
В Thu, 06 Jul 2023 15:22:26 -0300
Dewey Dunnington пишет:
I've wrapped two of these drivers for R that seem to build and
install on MacOS, Linux, and Windows [3][4]; however, I am not sure
if the pattern I used is suitable for CRAN or whether these packages
will have to be GitHub-only for the foreseeable future.
There are a few parts to following the CRAN policy [*] regarding
external dependencies.
I think (but don't know for sure) that your package will not be allowed
to download Go by itself. The policy says: "Only as a last resort and
with the agreement of the CRAN team should a package download
pre-compiled software."
An already installed Go should be able to "first look to see if [a
dependency] is already installed and if so is of a suitable version"
when installing the dependencies of the Go part of the code. The go.mod
and go.sum files specify the exact versions and checksums of the
dependencies, which satisfies the requirement for fixed dependency
versions ("it is acceptable to download them as part of installation,
but do ensure that the download is of a fixed version rather than the
latest"), so your package seems to be fine in this respect.
One more thing: when bootstrapping the source package, can you run go
mod vendor [**] in order to bundle *all* the Go dependencies together
with the package? Is the resulting directory prohibitively large? Would
it satisfy the CRAN policy preference to "include the library sources
in the package and compile them as part of package installation"
without requiring Internet access? Unfortunately, I don't know enough
about Go to answer these questions myself. I think that a small bundle
of vendored Go code would be preferrable for CRAN but *not* preferrable
for packaging in a GNU/Linux distro like Debian where dynamic linking
(in the widest possible sense) is a strong preference.
--
Best regards,
Ivan
[*] https://cran.r-project.org/web/packages/policies.html
[**] https://go.dev/ref/mod#vendoring
__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel
[R-pkg-devel] Best practices for CRAN package using Go
Hi all, I'm wondering if there is any precedent for using Go in a CRAN R package. Go is a particularly good language for writing database drivers for the Arrow Database Connectivity framework [1][2] and - importantly - there are several active developers writing these drivers from which R users might be able to benefit (notably, the Snowflake driver, for which there isn't currently a readily-available R connector). I've wrapped two of these drivers for R that seem to build and install on MacOS, Linux, and Windows [3][4]; however, I am not sure if the pattern I used is suitable for CRAN or whether these packages will have to be GitHub-only for the foreseeable future. The pattern is roughly the same as for Rust (build a static library and link it), and the build process has similar requirements (a `go` binary with which `go build` can be run; the build process downloads dependencies as it goes just like `cargo build`). Thanks in advance for any insight! Cheers, -dewey [1] https://arrow.apache.org/adbc [2] https://cran.r-project.org/package=adbcdrivermanager [3] https://github.com/apache/arrow-adbc/tree/main/r/adbcflightsql#readme [4] https://github.com/apache/arrow-adbc/tree/main/r/adbcsnowflake#readme __ R-package-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-package-devel
