nealrichardson commented on a change in pull request #9898:
URL: https://github.com/apache/arrow/pull/9898#discussion_r612896932
##########
File path: r/vignettes/developing.Rmd
##########
@@ -0,0 +1,457 @@
+---
+title: "Developer Documentation"
+output: rmarkdown::html_vignette
+vignette: >
+ %\VignetteIndexEntry{Developer Documentation}
+ %\VignetteEngine{knitr::rmarkdown}
+ %\VignetteEncoding{UTF-8}
+---
+
+```{r setup options, include=FALSE}
+knitr::opts_chunk$set(error = TRUE, eval = FALSE)
+
+# Get environment variables describing what to evaluate
+run <- tolower(Sys.getenv("RUN_DEVDOCS", "false")) == "true"
+macos <- tolower(Sys.getenv("DEVDOCS_MACOS", "false")) == "true"
+ubuntu <- tolower(Sys.getenv("DEVDOCS_UBUNTU", "false")) == "true"
+sys_install <- tolower(Sys.getenv("DEVDOCS_SYSTEM_INSTALL", "false")) == "true"
+
+# Update the source knit_hook to save the chunk (if it is marked to be saved)
+knit_hooks_source <- knitr::knit_hooks$get("source")
+knitr::knit_hooks$set(source = function(x, options) {
+ # Extra paranoia about when this will write the chunks to the script, we will
+ # only save when:
+ # * CI is true
+ # * RUN_DEVDOCS is true
+ # * options$save is TRUE (and a check that not NULL won't crash it)
+ if (as.logical(Sys.getenv("CI", FALSE)) && run && !is.null(options$save) &&
options$save)
+ cat(x, file = "script.sh", append = TRUE, sep = "\n")
+ # but hide the blocks we want hidden:
+ if (!is.null(options$hide) && options$hide) {
+ return(NULL)
+ }
+ knit_hooks_source(x, options)
+})
+```
+
+```{bash, save=run, hide=TRUE}
+# Stop on failure, echo input as we go
+set -e
+set -x
+```
+
+This document is intended only for developers of Arrow or the Arrow R package.
If you're looking for how to install Arrow, [the instructions in the
readme](https://arrow.apache.org/docs/r/#installation) or if you're on linux,
see `vignette("install", package = "arrow")` for more details. What follows is
a description of how to build the various components that make up the Arrow
project and R package as well as some common troubleshooting and workflows
developers use.
+
+Even for developers, most contributions can be accomplished with the
instructions in [R-only development](#r-only-development). But if you're
working on both the C++ library and the R package, the [Developer environment
setup](#-developer-environment-setup) section will guide you through setting up
a developer environment.
+
+## R-only development
+
+Windows and macOS users who wish to contribute to the R package and
+don’t need to alter the Arrow C++ library may be able to obtain a
+recent version of the library without building from source. On macOS,
+you may install the C++ library using [Homebrew](https://brew.sh/):
+
+``` shell
+# For the released version:
+brew install apache-arrow
+# Or for a development version, you can try:
+brew install apache-arrow --HEAD
+```
+
+On Windows, you can download a .zip file with the arrow dependencies from the
+[nightly
repository](https://arrow-r-nightly.s3.amazonaws.com/libarrow/bin/windows/),
+and then set the `RWINLIB_LOCAL` environment variable to point to that
+zip file before installing the `arrow` R package. Version numbers in that
+repository correspond to dates, and you will likely want the most recent.
+
+## Developer environment setup
+
+If you need to alter both the Arrow C++ library and the R package code, or if
you can’t get a binary version of the latest C++ library elsewhere, you’ll need
to build it from source too.
+
+First, install the C++ library. See the [developer
+guide](https://arrow.apache.org/docs/developers/cpp/building.html) for more
details and a full list of configuration options.
+
+### Install dependencies {.tabset}
+
+The Arrow library will by default use system dependencies if they are suitable
or build them during its own build process. The only dependencies that one
should need to install outside of the build process are `cmake` (for
configuring the build) and `openssl` if you are building with S3 support.
+
+#### macOS
+```{bash, save=run & macos}
+brew install cmake openssl
+```
+
+#### Ubuntu
+```{bash, save=run & ubuntu}
+sudo apt install -y cmake libcurl4-openssl-dev libssl-dev
+```
+
+
+### Configure the Arrow build {.tabset}
+
+You can choose to build and then install the Arrow library into a user-defined
directory or into a system-level directory. You only need to do one of these
two options. It’s recommended to make a `build` directory inside of the `cpp`
directory of the Arrow git repository (it is git-ignored).
+
+#### Install to another directory
+
+It is recommended that you install the arrow library to a user-level directory
to be used in development. In this example we will install it to a directory
called `dist` that has the same parent as our `arrow` checkout. The directory
name `dist` and the location is merely a convention. It could be named or
located anywhere you would like. However, note that your installation of the
Arrow R package will point to this directory and need it to remain intact for
the package to continue to work. This is one reason we recommend *not* placing
it inside of the arrow git checkout.
+
+```{bash, save=run & !sys_install}
+export ARROW_HOME=$(pwd)/dist
+mkdir $ARROW_HOME
+```
+
+
+_Special instructions on Linux:_ You will need to set `LD_LIBRARY_PATH` to the
same directory as `LIB_DIR` before launching R and using Arrow. One way to do
this is to add it to your profile (we use `~/.bash_profile` here, but you might
need to put this in a different file depending on your setup). On macOS we do
not need to do this because the macOS shared library paths are hardcoded to
their locations during build time.
Review comment:
I don't think `LIB_DIR` is mentioned in this document so that's not
helpful (anymore)
##########
File path: r/vignettes/developing.Rmd
##########
@@ -0,0 +1,457 @@
+---
+title: "Developer Documentation"
+output: rmarkdown::html_vignette
+vignette: >
+ %\VignetteIndexEntry{Developer Documentation}
+ %\VignetteEngine{knitr::rmarkdown}
+ %\VignetteEncoding{UTF-8}
+---
+
+```{r setup options, include=FALSE}
+knitr::opts_chunk$set(error = TRUE, eval = FALSE)
+
+# Get environment variables describing what to evaluate
+run <- tolower(Sys.getenv("RUN_DEVDOCS", "false")) == "true"
+macos <- tolower(Sys.getenv("DEVDOCS_MACOS", "false")) == "true"
+ubuntu <- tolower(Sys.getenv("DEVDOCS_UBUNTU", "false")) == "true"
+sys_install <- tolower(Sys.getenv("DEVDOCS_SYSTEM_INSTALL", "false")) == "true"
+
+# Update the source knit_hook to save the chunk (if it is marked to be saved)
+knit_hooks_source <- knitr::knit_hooks$get("source")
+knitr::knit_hooks$set(source = function(x, options) {
+ # Extra paranoia about when this will write the chunks to the script, we will
+ # only save when:
+ # * CI is true
+ # * RUN_DEVDOCS is true
+ # * options$save is TRUE (and a check that not NULL won't crash it)
+ if (as.logical(Sys.getenv("CI", FALSE)) && run && !is.null(options$save) &&
options$save)
+ cat(x, file = "script.sh", append = TRUE, sep = "\n")
+ # but hide the blocks we want hidden:
+ if (!is.null(options$hide) && options$hide) {
+ return(NULL)
+ }
+ knit_hooks_source(x, options)
+})
+```
+
+```{bash, save=run, hide=TRUE}
+# Stop on failure, echo input as we go
+set -e
+set -x
+```
+
+This document is intended only for developers of Arrow or the Arrow R package.
If you're looking for how to install Arrow, [the instructions in the
readme](https://arrow.apache.org/docs/r/#installation) or if you're on linux,
see `vignette("install", package = "arrow")` for more details. What follows is
a description of how to build the various components that make up the Arrow
project and R package as well as some common troubleshooting and workflows
developers use.
+
+Even for developers, most contributions can be accomplished with the
instructions in [R-only development](#r-only-development). But if you're
working on both the C++ library and the R package, the [Developer environment
setup](#-developer-environment-setup) section will guide you through setting up
a developer environment.
+
+## R-only development
+
+Windows and macOS users who wish to contribute to the R package and
+don’t need to alter the Arrow C++ library may be able to obtain a
+recent version of the library without building from source. On macOS,
+you may install the C++ library using [Homebrew](https://brew.sh/):
+
+``` shell
+# For the released version:
+brew install apache-arrow
+# Or for a development version, you can try:
+brew install apache-arrow --HEAD
+```
+
+On Windows, you can download a .zip file with the arrow dependencies from the
Review comment:
This is also true for Linux now
##########
File path: r/vignettes/developing.Rmd
##########
@@ -0,0 +1,457 @@
+---
+title: "Developer Documentation"
+output: rmarkdown::html_vignette
+vignette: >
+ %\VignetteIndexEntry{Developer Documentation}
+ %\VignetteEngine{knitr::rmarkdown}
+ %\VignetteEncoding{UTF-8}
+---
+
+```{r setup options, include=FALSE}
+knitr::opts_chunk$set(error = TRUE, eval = FALSE)
+
+# Get environment variables describing what to evaluate
+run <- tolower(Sys.getenv("RUN_DEVDOCS", "false")) == "true"
+macos <- tolower(Sys.getenv("DEVDOCS_MACOS", "false")) == "true"
+ubuntu <- tolower(Sys.getenv("DEVDOCS_UBUNTU", "false")) == "true"
+sys_install <- tolower(Sys.getenv("DEVDOCS_SYSTEM_INSTALL", "false")) == "true"
+
+# Update the source knit_hook to save the chunk (if it is marked to be saved)
+knit_hooks_source <- knitr::knit_hooks$get("source")
+knitr::knit_hooks$set(source = function(x, options) {
+ # Extra paranoia about when this will write the chunks to the script, we will
+ # only save when:
+ # * CI is true
+ # * RUN_DEVDOCS is true
+ # * options$save is TRUE (and a check that not NULL won't crash it)
+ if (as.logical(Sys.getenv("CI", FALSE)) && run && !is.null(options$save) &&
options$save)
+ cat(x, file = "script.sh", append = TRUE, sep = "\n")
+ # but hide the blocks we want hidden:
+ if (!is.null(options$hide) && options$hide) {
+ return(NULL)
+ }
+ knit_hooks_source(x, options)
+})
+```
+
+```{bash, save=run, hide=TRUE}
+# Stop on failure, echo input as we go
+set -e
+set -x
+```
+
+This document is intended only for developers of Arrow or the Arrow R package.
If you're looking for how to install Arrow, [the instructions in the
readme](https://arrow.apache.org/docs/r/#installation) or if you're on linux,
see `vignette("install", package = "arrow")` for more details. What follows is
a description of how to build the various components that make up the Arrow
project and R package as well as some common troubleshooting and workflows
developers use.
+
+Even for developers, most contributions can be accomplished with the
instructions in [R-only development](#r-only-development). But if you're
working on both the C++ library and the R package, the [Developer environment
setup](#-developer-environment-setup) section will guide you through setting up
a developer environment.
+
+## R-only development
+
+Windows and macOS users who wish to contribute to the R package and
+don’t need to alter the Arrow C++ library may be able to obtain a
+recent version of the library without building from source. On macOS,
+you may install the C++ library using [Homebrew](https://brew.sh/):
+
+``` shell
+# For the released version:
+brew install apache-arrow
+# Or for a development version, you can try:
+brew install apache-arrow --HEAD
+```
+
+On Windows, you can download a .zip file with the arrow dependencies from the
+[nightly
repository](https://arrow-r-nightly.s3.amazonaws.com/libarrow/bin/windows/),
Review comment:
~~Unfortunately~~ As it turns out, this URL does not return useful
information (the bintray version did). Surely there's an S3 permission that can
be set that would make that list (XML) with contents, but a better way is
probably to encourage use of the package itself, like:
```
> nightly <- s3_bucket("arrow-r-nightly")
> nightly$ls("libarrow/bin")
[1] "libarrow/bin/centos-7" "libarrow/bin/centos-8"
[3] "libarrow/bin/debian-10" "libarrow/bin/debian-9"
[5] "libarrow/bin/ubuntu-16.04" "libarrow/bin/ubuntu-18.04"
[7] "libarrow/bin/windows"
```
We could even add a function to download the latest binary (appropriate for
your OS) and unzip it/put it in the right place. Out of scope here, just saying.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
