Re: [R] Downloading a directory of text files into R

[Rui Barradas]

Às 23:06 de 25/07/2023, Bob Green escreveu:

Hello,

I am seeking advice as to how I can download the 833 files from this 
site:"http://home.brisnet.org.au/~bgreen/Data/;


I want to be able to download them to perform a textual analysis.

If the 833 files, which are in a Directory with two subfolders were on 
my computer I could read them through readtext. Using readtext I get the 
error:


 > x = readtext("http://home.brisnet.org.au/~bgreen/Data/*;)
Error in download_remote(file, ignore_missing, cache, verbosity) :
   Remote URL does not end in known extension. Please download the file 
manually.


 > x = readtext("http://home.brisnet.org.au/~bgreen/Data/Dir/()")
Error in download_remote(file, ignore_missing, cache, verbosity) :
   Remote URL does not end in known extension. Please download the file 
manually.


Any suggestions are appreciated.

Bob

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide 
http://www.R-project.org/posting-guide.html

and provide commented, minimal, self-contained, reproducible code.

Hello,

The following code downloads all files in the posted link.



suppressPackageStartupMessages({
  library(rvest)
})

# destination directory, change this at will
dest_dir <- "~/Temp"

# first get the two subfolders from the Data webpage
link <- "http://home.brisnet.org.au/~bgreen/Data/;
page <- read_html(link)
page %>%
  html_elements("a") %>%
  html_text() %>%
  grep("/$", ., value = TRUE) -> sub_folder

# create relevant disk sub-directories, if
# they do not exist yet
for(subf in sub_folder) {
  d <- file.path(dest_dir, subf)
  if(!dir.exists(d)) {
success <- dir.create(d)
msg <- paste("created directory", d, "-", success)
message(msg)
  }
}

# prepare to download the files
dest_dir <- file.path(dest_dir, sub_folder)
source_url <- paste0(link, sub_folder)

success <- mapply(\(src, dest) {
  # read each Data subfolder
  # and get the file names therein
  # then lapply 'download.file' to each filename
  pg <- read_html(src)
  pg %>%
html_elements("a") %>%
html_text() %>%
grep("\\.txt$", ., value = TRUE) %>%
lapply(\(x) {
  s <- paste0(src, x)
  d <- file.path(dest, x)
  tryCatch(
download.file(url = s, destfile = d),
warning = function(w) w,
error = function(e) e
  )
})
}, source_url, dest_dir)

lengths(success)
# http://home.brisnet.org.au/~bgreen/Data/Hanson1/
#   84
# http://home.brisnet.org.au/~bgreen/Data/Hanson2/
#  749

# matches the question's number
sum(lengths(success))
# [1] 833



Hope this helps,

Rui Barradas

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] plotly

[Thomas Subia via R-help]

 Colleagues,
Here is my reproducible code.
library(plotly)

t <- list(
  family = "Arial",
  size = 12,
  color = "black",
  face="bold")


t3 <- list(
  family = "Arial",
  size = 12,
  color = "black",
  face="bold")

t5 <- list(
  family = "Arial",
  size = 12,
  color = "black",
  face="bold")

plot_ly(
  domain = list(x = c(0, .5), y = c(0,0.5)),
  value = 319,
  title = list(text = "Example\n4,800 Target",font=t),
  type = "indicator",
  mode = "gauge+number+delta",
  number = list(valueformat = "%.2f",font=t5), 
  delta = list(reference = 4800, valueformat = "%.2f",font=t3),
  gauge = list(
    axis = list(range = list(NULL, 9600), 
    tickfont = list(size = 12, face="bold",family = "Arial")),
    bar = list(color = "black"),
    steps = list(
  list(range = c(0, 4800), color = 'AAFF00'),
  list(range = c(4800, 9600), color = 'ffb2b2')),
    threshold = list(
  line = list(color = "black", width = 8),
  thickness = 0.75,
  value = 4800)))
Question 1:How can I save the plotly graph with a specific graph size?Using 
ggsave, one can define the height, width and unit of measure, is there 
something similar in plotly?Since I am creating the same gauge plot but with 
different data, I'd like all the resulting graphs to have the same size. I've 
googled this but to no avail,
Question 2:How can I adjust the tick marks of the gauge? Ideally the tick marks 
should be set at seq(0,9600,1200)
Thanks everyone!
Thomas Subia

On Saturday, July 22, 2023 at 08:10:52 AM PDT, Boris Steipe 
 wrote:  
 
 ChatGPT-4:

-- Query:  --

Thank you. The `value` in the code example is 2874. The plot shows a large 
number at the center and a smaller value computed as a delta relative to the 
`reference` of 4800. But the large value is given as 2870, and the smaller 
value is given as -1930 i.e. both values are rounded. Can I control the 
precision of these two numbers?


-- Response:  ---

In Plotly, you can control the precision of the numbers by specifying the 
`valueformat` and `delta.valueformat` parameters. These parameters accept a 
string in the format that is used by the C `printf` function. For example, 
`%.2f` specifies a number with two digits after the decimal point. Here is an 
example:

```R
plotly::plot_ly(
  domain = list(x = c(0, 1), y = c(0, 1)),
  value = 2874,
  title = list(text = "Generic"),
  type = "indicator",
  mode = "gauge+number+delta",
  number = list(valueformat = "%.2f"), 
  delta = list(reference = 4800, valueformat = "%.2f"),
  gauge = list(
    axis = list(range = list(NULL, 5000), 
                tickfont = list(size = 20, family = "Arial")),
    bar = list(color = "blue"),
    steps = list(
      list(range = c(0, 4800), color = "white"),
      list(range = c(4800, 6000), color = "red")),
    threshold = list(
      line = list(color = "black", width = 6),
      thickness = 0.75,
      value = 4800)))
```
In this example, both the main number and the delta value will be displayed 
with two decimal places. You can adjust these strings to display the numbers 
with the precision you prefer.

-

Here you go. I have no idea how it does this, and have a discussion on elements 
of "Sein und Zeit" and book 19 of the Odyssey, and tutor me in Italian 
conversation, and draft Mermaid mind-maps - but here we are.


:-)






> 
> \In this example, the value was 2324, the gauge displays 2320.
> I'm not sure why this occurs. Changing the statement to 56, results in the 
> gauge reading 60. I'm not sure what needs to be changed in the script or the 
> environment to stop rounding.




> On Jul 22, 2023, at 10:43, Boris Steipe  wrote:
> 
> What do you mean "Rounded"?
> What do you expect, what do you get instead?
> 
> ?
> 
> 
> 
> 
> 
>> On Jul 22, 2023, at 10:40, Thomas Subia via R-help  
>> wrote:
>> 
>> Colleagues,
>> Thanks for the update.
>> My colleagues at work have run this script but the resulting graph output 
>> for value is rounded. How can one turn this annoying feature off?
>> I've googled this but to no avail.
>> 
>> 
>> 
>> 
>> [[alternative HTML version deleted]]
>> 
>> __
>> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
>> https://stat.ethz.ch/mailman/listinfo/r-help
>> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
>> and provide commented, minimal, self-contained, reproducible code.
> 
> __
> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

  
[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and 

Re: [R] Downloading a directory of text files into R

[Jeff Newmiller]

You cannot read files using name patterns. You can use list.files with patterns 
on your local filesystems, and you can use RCurl or httr contributed packages 
to parse out the web listing of files returned by the web server.  See the 
example in ?RCurl. Then you can download the individual files in an lapply or 
for loop.

On July 25, 2023 3:06:07 PM PDT, Bob Green  wrote:
>Hello,
>
>I am seeking advice as to how I can download the 833 files from this 
>site:"http://home.brisnet.org.au/~bgreen/Data/;
>
>I want to be able to download them to perform a textual analysis.
>
>If the 833 files, which are in a Directory with two subfolders were on my 
>computer I could read them through readtext. Using readtext I get the error:
>
>> x = readtext("http://home.brisnet.org.au/~bgreen/Data/*;)
>Error in download_remote(file, ignore_missing, cache, verbosity) :
>  Remote URL does not end in known extension. Please download the file 
> manually.
>
>> x = readtext("http://home.brisnet.org.au/~bgreen/Data/Dir/()")
>Error in download_remote(file, ignore_missing, cache, verbosity) :
>  Remote URL does not end in known extension. Please download the file 
> manually.
>
>Any suggestions are appreciated.
>
>Bob
>
>__
>R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
>https://stat.ethz.ch/mailman/listinfo/r-help
>PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
>and provide commented, minimal, self-contained, reproducible code.

-- 
Sent from my phone. Please excuse my brevity.

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Seeking Assistance: Plotting Sea Current Vectors in R

[Jim Lemon]

Hi Kostas,
The function vectorField in the plotrix package may do what you want.
See the example.

Jim

On Tue, Jul 25, 2023 at 9:30 PM konstantinos christodoulou
 wrote:
>
> Dear Rcommunity,
>
> I hope this email finds you well. I am writing to seek your assistance with
> a data visualization problem I am facing while working with R.
>
> Problem Description:
>
> I have a dataframe named "df" containing the following columns:
> "longitude", "latitude", "sea_currents_mag", and "sea_currents_direction".
> The dataframe includes sea current estimations, with information about
> magnitude (m/s) and direction (degrees) at various longitude and latitude
> coordinates. The study domain covers the Eastern Mediterranean Sea (23E to
> 36W) and extends from 31S to 37N. It is important to note that the
> longitude and latitude coordinates are not evenly spaced across the domain.
>
> Objective: I am seeking guidance on how to create a plot that visualizes
> the sea current vectors (arrows) at each coordinate. Additionally, if
> possible, I would like the borders of the surrounding countries to be
> included in the plot to provide geographic context.
>
> Specific Requests:
>
>1. Help with plotting sea current vectors (arrows) based on
>"sea_currents_mag" and "sea_currents_direction" at each corresponding
>longitude and latitude coordinate.
>2. Assistance with including the borders of the surrounding countries in
>the plot to provide geographic context.
>
> I would highly appreciate any advice, code examples, or packages that could
> assist me in achieving this visualization goal.
>
> Thank you very much for taking the time to read my email, and I look
> forward to any assistance you can provide.
>
> Best regards,
> Kostas
>
> [[alternative HTML version deleted]]
>
> __
> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Downloading a directory of text files into R

[Ben Bolker]

 Where is readtext() from?

  Some combination of scraping

http://home.brisnet.org.au/~bgreen/Data/Hanson1/

and

http://home.brisnet.org.au/~bgreen/Data/Hanson2/


to recover the required file names:

library(rvest)
read_html("http://home.brisnet.org.au/~bgreen/Data/Hanson1/;) |> 
html_element("body") |> html_element("table") |> html_table()


will get you most of the way there ...

then an lapply() or for loop to download all the bits ...?



On 2023-07-25 6:06 p.m., Bob Green wrote:

Hello,

I am seeking advice as to how I can download the 833 files from this 
site:"http://home.brisnet.org.au/~bgreen/Data/;


I want to be able to download them to perform a textual analysis.

If the 833 files, which are in a Directory with two subfolders were on 
my computer I could read them through readtext. Using readtext I get the 
error:


 > x = readtext("http://home.brisnet.org.au/~bgreen/Data/*;)
Error in download_remote(file, ignore_missing, cache, verbosity) :
   Remote URL does not end in known extension. Please download the file 
manually.


 > x = readtext("http://home.brisnet.org.au/~bgreen/Data/Dir/()")
Error in download_remote(file, ignore_missing, cache, verbosity) :
   Remote URL does not end in known extension. Please download the file 
manually.


Any suggestions are appreciated.

Bob

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide 
http://www.R-project.org/posting-guide.html

and provide commented, minimal, self-contained, reproducible code.


--
Dr. Benjamin Bolker
Professor, Mathematics & Statistics and Biology, McMaster University
Director, School of Computational Science and Engineering
(Acting) Graduate chair, Mathematics & Statistics
> E-mail is sent at my convenience; I don't expect replies outside of 
working hours.


__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] [EXT] Downloading a directory of text files into R

[Andrew Robinson via R-help]

Hi Bob,

there may be more efficient ways to go about it but I would use R to scrape the 
contents of

http://home.brisnet.org.au/~bgreen/Data/Hanson1/
http://home.brisnet.org.au/~bgreen/Data/Hanson2/

in order to form the URLs of the files, and then loop over the URLs.

Cheers,

Andrew

--
Andrew Robinson
Chief Executive Officer, CEBRA and Professor of Biosecurity,
School/s of BioSciences and Mathematics & Statistics
University of Melbourne, VIC 3010 Australia
Tel: (+61) 0403 138 955
Email: a...@unimelb.edu.au
Website: https://researchers.ms.unimelb.edu.au/~apro@unimelb/

I acknowledge the Traditional Owners of the land I inhabit, and pay my respects 
to their Elders.
On 26 Jul 2023 at 8:07 AM +1000, Bob Green , wrote:
External email: Please exercise caution

Hello,

I am seeking advice as to how I can download the 833 files from this
site:"http://home.brisnet.org.au/~bgreen/Data;

I want to be able to download them to perform a textual analysis.

If the 833 files, which are in a Directory with two subfolders were
on my computer I could read them through readtext. Using readtext I
get the error:

> x = readtext("http://home.brisnet.org.au/~bgreen/Data/*;)
Error in download_remote(file, ignore_missing, cache, verbosity) :
Remote URL does not end in known extension. Please download the
file manually.

> x = readtext("http://home.brisnet.org.au/~bgreen/Data/Dir/()")
Error in download_remote(file, ignore_missing, cache, verbosity) :
Remote URL does not end in known extension. Please download the
file manually.

Any suggestions are appreciated.

Bob

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.


[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] Downloading a directory of text files into R

[Bob Green]

Hello,

I am seeking advice as to how I can download the 833 files from this 
site:"http://home.brisnet.org.au/~bgreen/Data/;


I want to be able to download them to perform a textual analysis.

If the 833 files, which are in a Directory with two subfolders were 
on my computer I could read them through readtext. Using readtext I 
get the error:


> x = readtext("http://home.brisnet.org.au/~bgreen/Data/*;)
Error in download_remote(file, ignore_missing, cache, verbosity) :
  Remote URL does not end in known extension. Please download the 
file manually.


> x = readtext("http://home.brisnet.org.au/~bgreen/Data/Dir/()")
Error in download_remote(file, ignore_missing, cache, verbosity) :
  Remote URL does not end in known extension. Please download the 
file manually.


Any suggestions are appreciated.

Bob

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Seeking Assistance: Plotting Sea Current Vectors in R

[PIKAL Petr]

Not sure if correct but Chatgpt answer is:

Here are some popular R packages for sea current vectors:

oce: The "oce" package provides a wide range of functions for oceanographic 
data analysis, including handling sea current data. It allows you to work with 
current data in various formats and provides functions for visualization and 
basic analysis.

oceanmap: The "oceanmap" package is specifically designed for the analysis and 
visualization of oceanographic data. It includes functions for mapping 
oceanographic variables, including sea current vectors, using ggplot2-based 
plotting techniques.

rOceans: This package is part of the "rOpenSci" project and focuses on 
accessing various oceanographic datasets, including sea current data. It 
provides functions to download and work with sea current data from online 
sources.

marmap: While "marmap" is primarily designed for bathymetric data (depth 
measurements of the ocean), it can be useful for visualizing sea current 
vectors in the context of oceanographic maps.

ncdf4: The "ncdf4" package allows you to work with netCDF files, which are 
commonly used to store oceanographic data, including sea current data. It 
enables you to read, write, and manipulate netCDF files in R.

oceanoGrafia: The "oceanoGrafia" package provides tools for oceanographic data 
analysis, including sea current vectors. It supports various data formats and 
offers functions for data processing and visualization.

ocean: The "ocean" package is designed for oceanographic data analysis and 
visualization, including sea current data. It offers functions for data 
manipulation, transformation, and plotting.

Cheers Petr


-Original Message-
From: R-help  On Behalf Of konstantinos 
christodoulou
Sent: Tuesday, July 25, 2023 1:29 PM
To: r-help mailing list 
Subject: [R] Seeking Assistance: Plotting Sea Current Vectors in R

Dear Rcommunity,

I hope this email finds you well. I am writing to seek your assistance with a 
data visualization problem I am facing while working with R.

Problem Description:

I have a dataframe named "df" containing the following columns:
"longitude", "latitude", "sea_currents_mag", and "sea_currents_direction".
The dataframe includes sea current estimations, with information about 
magnitude (m/s) and direction (degrees) at various longitude and latitude 
coordinates. The study domain covers the Eastern Mediterranean Sea (23E to
36W) and extends from 31S to 37N. It is important to note that the longitude 
and latitude coordinates are not evenly spaced across the domain.

Objective: I am seeking guidance on how to create a plot that visualizes the 
sea current vectors (arrows) at each coordinate. Additionally, if possible, I 
would like the borders of the surrounding countries to be included in the plot 
to provide geographic context.

Specific Requests:

   1. Help with plotting sea current vectors (arrows) based on
   "sea_currents_mag" and "sea_currents_direction" at each corresponding
   longitude and latitude coordinate.
   2. Assistance with including the borders of the surrounding countries in
   the plot to provide geographic context.

I would highly appreciate any advice, code examples, or packages that could 
assist me in achieving this visualization goal.

Thank you very much for taking the time to read my email, and I look forward to 
any assistance you can provide.

Best regards,
Kostas

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see 
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Osobní údaje: Informace o zpracování a ochraně osobních údajů obchodních 
partnerů PRECHEZA a.s. jsou zveřejněny na: 
https://www.precheza.cz/zasady-ochrany-osobnich-udaju/ | Information about 
processing and protection of business partner’s personal data are available on 
website: https://www.precheza.cz/en/personal-data-protection-principles/
Důvěrnost: Tento e-mail a jakékoliv k němu připojené dokumenty jsou důvěrné a 
podléhají tomuto právně závaznému prohláąení o vyloučení odpovědnosti: 
https://www.precheza.cz/01-dovetek/ | This email and any documents attached to 
it may be confidential and are subject to the legally binding disclaimer: 
https://www.precheza.cz/en/01-disclaimer/

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

[R] Seeking Assistance: Plotting Sea Current Vectors in R

[konstantinos christodoulou]

Dear Rcommunity,

I hope this email finds you well. I am writing to seek your assistance with
a data visualization problem I am facing while working with R.

Problem Description:

I have a dataframe named "df" containing the following columns:
"longitude", "latitude", "sea_currents_mag", and "sea_currents_direction".
The dataframe includes sea current estimations, with information about
magnitude (m/s) and direction (degrees) at various longitude and latitude
coordinates. The study domain covers the Eastern Mediterranean Sea (23E to
36W) and extends from 31S to 37N. It is important to note that the
longitude and latitude coordinates are not evenly spaced across the domain.

Objective: I am seeking guidance on how to create a plot that visualizes
the sea current vectors (arrows) at each coordinate. Additionally, if
possible, I would like the borders of the surrounding countries to be
included in the plot to provide geographic context.

Specific Requests:

   1. Help with plotting sea current vectors (arrows) based on
   "sea_currents_mag" and "sea_currents_direction" at each corresponding
   longitude and latitude coordinate.
   2. Assistance with including the borders of the surrounding countries in
   the plot to provide geographic context.

I would highly appreciate any advice, code examples, or packages that could
assist me in achieving this visualization goal.

Thank you very much for taking the time to read my email, and I look
forward to any assistance you can provide.

Best regards,
Kostas

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.