[ https://issues.apache.org/jira/browse/ARROW-8880?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Brian Schultheiss updated ARROW-8880: ------------------------------------- Description: When R install tries to run a binary install, it looks for an exact match on the binary version, say "0.17.1.zip" from [https://dl.bintray.com/ursalabs/arrow-r/libarrow/bin/ubuntu-18.04/]. The problem is that even though "0.17.1" is pushed to CRAN as an official release, there is a time period (like right now) where bintray does not have an official binary build, just a date stamped build: arrow-0.17.0.20200516.zip arrow-0.17.0.20200517.zip arrow-0.17.0.20200518.zip arrow-0.17.0.zip arrow-0.17.1.20200517.zip arrow-0.17.1.20200519.zip arrow-0.17.1.20200520.zip I'd like to suggest adding a new environment variable trigger that would allow for the scanning of bintray for a recent timestamped version, if the specific release number is not present. I'd like to suggest enhancing the linux code: [https://github.com/apache/arrow/blob/02f7be33d1c32d1636323e6fb90c63cb01bf44af/r/tools/linuxlibs.R#L39-L47] with scanning functionality: {color:#c1c7d0}try_download <- function(from_url, to_file, scan_dates = FALSE) {{color} {color:#c1c7d0} try({color} {color:#c1c7d0} suppressWarnings({color} {color:#c1c7d0} download.file(from_url, to_file, quiet = quietly){color} {color:#c1c7d0} ),{color} {color:#c1c7d0} silent = quietly{color} {color:#c1c7d0} ){color} {color:#c1c7d0} if (!file.exists(to_file)) {{color} {color:#c1c7d0} if (scan_dates) {{color} {color:#0747a6} scan_dates <- format(Sys.Date()-(0:10),"%Y%m%d"){color} {color:#0747a6} for (scan_date in scan_dates) {{color} {color:#0747a6} base_url <- tools::file_path_sans_ext(from_url){color} {color:#0747a6} ext <- tools::file_ext(from_url){color} {color:#0747a6} scan_url <- sprintf("%s.%s.%s", base_url, scan_date, ext){color} {color:#0747a6} if (try_download(from_url = scan_url, to_file, scan_dates = FALSE)) {{color} {color:#0747a6} return(TRUE){color} {color:#0747a6} }{color} {color:#0747a6} }{color} {color:#0747a6} }{color} {color:#0747a6} return(FALSE){color} {color:#c1c7d0} } else {{color} {color:#c1c7d0} return(TRUE){color} {color:#c1c7d0} }{color} \{color:#c1c7d0}} And then augment the calling function: [https://github.com/apache/arrow/blob/02f7be33d1c32d1636323e6fb90c63cb01bf44af/r/tools/linuxlibs.R#L55] with: binary_scan_ok <- !identical(tolower(Sys.getenv("LIBARROW_BINARY_SCAN", "false")), "false") {color:#c1c7d0}if (try_download(binary_url, libfile, {color}scan_dates = binary_scan_ok{color:#c1c7d0})) {{color} This would allow automated builds to set the scan option, and then find and install the most recent daily build in lieu of an official binary build being in place. was: When R install tries to run a binary install, it looks for an exact match on the binary version, say "0.17.1.zip" from [https://dl.bintray.com/ursalabs/arrow-r/libarrow/bin/ubuntu-18.04/]. The problem is that even though "0.17.1" is pushed to CRAN as an official release, there is a time period (like right now) where bintray does not have an official binary build, just a date stamped build: arrow-0.17.0.20200516.zip arrow-0.17.0.20200517.zip arrow-0.17.0.20200518.zip arrow-0.17.0.zip arrow-0.17.1.20200517.zip arrow-0.17.1.20200519.zip arrow-0.17.1.20200520.zip I'd like to suggest adding a new environment variable trigger that would allow for the scanning of bintray for a recent timestamped version, if the specific release number is not present. I'd like to suggest enhancing the linux code: [https://github.com/apache/arrow/blob/02f7be33d1c32d1636323e6fb90c63cb01bf44af/r/tools/linuxlibs.R#L39-L47] with scanning functionality: {color:#c1c7d0}try_download <- function(from_url, to_file, scan_dates = FALSE) {{color} {color:#c1c7d0} try({color} {color:#c1c7d0} suppressWarnings({color} {color:#c1c7d0} download.file(from_url, to_file, quiet = quietly){color} {color:#c1c7d0} ),{color} {color:#c1c7d0} silent = quietly{color} {color:#c1c7d0} ){color} {color:#c1c7d0} if (!file.exists(to_file)) {{color} {color:#c1c7d0} {color:#0747a6} if (scan_dates) {{color}{color} {color:#0747a6} scan_dates <- format(Sys.Date()-(0:10),"%Y%m%d"){color} {color:#0747a6} for (scan_date in scan_dates) {{color} {color:#0747a6} base_url <- tools::file_path_sans_ext(from_url){color} {color:#0747a6} ext <- tools::file_ext(from_url){color} {color:#0747a6} scan_url <- sprintf("%s.%s.%s", base_url, scan_date, ext){color} {color:#0747a6} if (try_download(from_url = scan_url, to_file, scan_dates = FALSE)) {{color} {color:#0747a6} return(TRUE){color} {color:#0747a6} }{color} {color:#0747a6} }{color} {color:#0747a6} return(FALSE){color} {color:#0747a6} } else {{color} {color:#0747a6} return(FALSE){color} {color:#0747a6} }{color} {color:#c1c7d0} } else {{color} {color:#c1c7d0} return(TRUE){color} {color:#c1c7d0} }{color} {color:#c1c7d0}}{color} And then augment the calling function: [https://github.com/apache/arrow/blob/02f7be33d1c32d1636323e6fb90c63cb01bf44af/r/tools/linuxlibs.R#L55] with: {color:#0747a6}binary_scan_ok <- !identical(tolower(Sys.getenv("LIBARROW_BINARY_SCAN", "false")), "false"){color} {color:#c1c7d0}if (try_download(binary_url, libfile{color:#0747a6}, {color}{color}{color:#0747a6}scan_dates = binary_scan_ok{color}{color:#c1c7d0})) {{color} This would allow automated builds to set the scan option, and then find and install the most recent daily build in lieu of an official binary build being in place. > [R][Linux] Make R Binary Install Friendlier > ------------------------------------------- > > Key: ARROW-8880 > URL: https://issues.apache.org/jira/browse/ARROW-8880 > Project: Apache Arrow > Issue Type: Improvement > Components: R > Affects Versions: 0.17.1 > Environment: Linux (Ubuntu) > Reporter: Brian Schultheiss > Priority: Minor > > When R install tries to run a binary install, it looks for an exact match on > the binary version, say "0.17.1.zip" from > [https://dl.bintray.com/ursalabs/arrow-r/libarrow/bin/ubuntu-18.04/]. > The problem is that even though "0.17.1" is pushed to CRAN as an official > release, there is a time period (like right now) where bintray does not have > an official binary build, just a date stamped build: > > arrow-0.17.0.20200516.zip > arrow-0.17.0.20200517.zip > arrow-0.17.0.20200518.zip > arrow-0.17.0.zip > arrow-0.17.1.20200517.zip > arrow-0.17.1.20200519.zip > arrow-0.17.1.20200520.zip > > I'd like to suggest adding a new environment variable trigger that would > allow for the scanning of bintray for a recent timestamped version, if the > specific release number is not present. > I'd like to suggest enhancing the linux code: > [https://github.com/apache/arrow/blob/02f7be33d1c32d1636323e6fb90c63cb01bf44af/r/tools/linuxlibs.R#L39-L47] > with scanning functionality: > {color:#c1c7d0}try_download <- function(from_url, to_file, scan_dates = > FALSE) {{color} > {color:#c1c7d0} try({color} > {color:#c1c7d0} suppressWarnings({color} > {color:#c1c7d0} download.file(from_url, to_file, quiet = > quietly){color} > {color:#c1c7d0} ),{color} > {color:#c1c7d0} silent = quietly{color} > {color:#c1c7d0} ){color} > {color:#c1c7d0} if (!file.exists(to_file)) {{color} > {color:#c1c7d0} if (scan_dates) {{color} > {color:#0747a6} scan_dates <- > format(Sys.Date()-(0:10),"%Y%m%d"){color} > {color:#0747a6} for (scan_date in scan_dates) {{color} > {color:#0747a6} base_url <- > tools::file_path_sans_ext(from_url){color} > {color:#0747a6} ext <- > tools::file_ext(from_url){color} > {color:#0747a6} scan_url <- sprintf("%s.%s.%s", > base_url, scan_date, ext){color} > {color:#0747a6} if (try_download(from_url = > scan_url, to_file, scan_dates = FALSE)) {{color} > {color:#0747a6} return(TRUE){color} > {color:#0747a6} }{color} > {color:#0747a6} }{color} > {color:#0747a6} }{color} > {color:#0747a6} return(FALSE){color} > {color:#c1c7d0} } else {{color} > {color:#c1c7d0} return(TRUE){color} > {color:#c1c7d0} }{color} > \{color:#c1c7d0}} > > And then augment the calling function: > > > [https://github.com/apache/arrow/blob/02f7be33d1c32d1636323e6fb90c63cb01bf44af/r/tools/linuxlibs.R#L55] > > > > with: > > binary_scan_ok <- !identical(tolower(Sys.getenv("LIBARROW_BINARY_SCAN", > "false")), "false") > {color:#c1c7d0}if (try_download(binary_url, libfile, {color}scan_dates = > binary_scan_ok{color:#c1c7d0})) {{color} > > This would allow automated builds to set the scan option, and then find and > install the most recent daily build in lieu of an official binary build being > in place. > > > > > > > > > > > > > -- This message was sent by Atlassian Jira (v8.3.4#803005)