Re: [R] Using multiple dat files
Às 18:59 de 06/11/2024, Sibylle Stöckli escreveu: Dear Rui Dear Bert Many thanks Solution filelist <- list.files(path = "O:/Data-Work/2.../Daten_RA-MeteoCH_1990-2021", pattern='*.dat', all.files= T, full.names= T) AAR<-read.table(filelist[1]) It seems therefore that there is no other way than read in individually > 100 weather tables using read.tables., right? Using file.choose() doesn't change the work. Yes my .dat files are data.frames str(W[[1]]) 'data.frame': 11688 obs. of 7 variables: $ year : num 1990 1990 1990 1990 1990 1990 1990 1990 1990 1990 ... $ DOY : num 1 2 3 4 5 6 7 8 9 10 ... $ Ta : num -2.67 -2.77 -2.23 -2.21 -0.98 0.82 0.49 -1.02 -2.31 -3.36 ... $ Tmin : num -3.5 -3.7 -4.26 -2.87 -2.98 0.3 -0.83 -1.27 -3 -3.82 ... $ Tmax : num -1.13 -0.15 -0.13 -0.45 1 1.87 1.72 -0.35 -0.85 -2.3 ... $ Precip: num 0 0 0 0 0.45 1.81 0.03 0 0 0 ... $ rSSD : num 0 0.08 0 0 0.08 0 0 0 0 0 ... *Gesendet: *Mittwoch, 6. November 2024 um 18:28 *Von: *"Bert Gunter" *An: *"Sibylle Stöckli" *CC: *[email protected] *Betreff: *Re: [R] Using multiple dat files Not quite sure if I understand you. list.files() simply returns a character vector(not a list). You can simply use a vector index to select whatever file you wish to read. So if your desired filename is the 5th element of filelist above, something like myfile <- read.table(filename[5], ...) You can also use regular expressions to choose a bunch of files that have some common signature to their names that you can read in simultaneously using your "filelist" vector of names via something like: myfiles <- lapply(grep("weath", filelist, value = TRUE), \(x) read.table(x,...)) ### Note that the result of lapply *is* a list, so use list indexing for extraction from myfiles. You can also choose files to read interactively (via a GUI interface) using file.choose() instead of using list.files() if you prefer to do it that way. Cheers, Bert On Wed, Nov 6, 2024 at 8:25 AM Sibylle Stöckli via R-help mailto:[email protected]>> wrote: Dear community To import multiple .dat weather files I am using list.files(). I intend to use the R package “ClimInd” to calculate different agroclimatic indicators. Question: Is there another solution to import multiple .dat files so that I can select elements from the list, e.g. one specific weather file (example AAR_DailyWeather)? # Import multiple .dat files weather data filelist <- list.files(path = "O:/Data-Work/……./Daten_RA-MeteoCH_1990-2021", pattern='*.dat', all.files= T, full.names= T) W <- lapply(filelist, function(x) read.table(x, header = TRUE, sep = "", colClasses = "numeric", comment.char = "")) W[[1]] > dd(data = W[[1]]$Precip, time.scale = W[[1]]$year) Fehler in W[[1]]$year : $ operator is invalid for atomic vectors Kind regards Sibylle __ [email protected] <mailto:[email protected]> mailing list -- To UNSUBSCRIBE and more, see https://stat.ethz.ch/mailman/listinfo/r-help <https://stat.ethz.ch/mailman/ listinfo/r-help> PLEASE do read the posting guide https://www.R-project.org/posting- guide.html <https://www.R-project.org/posting-guide.html> and provide commented, minimal, self-contained, reproducible code. Hello, If read.table as you have posted it solves the problem of reading one file, then the following should read all of them. # No further options passed to read.table. Was that the problem? AAR_list <- lapply(filelist, read.table) # You can also set the returned list's names, like Bert "suggested" AAR_list <- setNames(AAR_list, basename(filelist)) Or you can test in a small subset of the files # If this works then it's probably safe to read them all # (and you don't have to, if it doesn't) AAR_list <- lapply(filelist[1:3], read.table) Hope this helps, Rui Barradas -- Este e-mail foi analisado pelo software antivírus AVG para verificar a presença de vírus. www.avg.com __ [email protected] mailing list -- To UNSUBSCRIBE and more, see https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide https://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Using multiple dat files
On 11/5/24 22:29, Sibylle Stöckli via R-help wrote:
Dear community
To import multiple .dat weather files I am using list.files().
I intend to use the R package “ClimInd” to calculate different agroclimatic
indicators.
Question: Is there another solution to import multiple .dat files so that I can
select elements from the list, e.g. one specific weather file (example
AAR_DailyWeather)?
# Import multiple .dat files weather data
filelist <- list.files(path = "O:/Data-Work/……./Daten_RA-MeteoCH_1990-2021",
pattern='*.dat', all.files= T, full.names= T)
W <- lapply(filelist, function(x) read.table(x, header = TRUE, sep = "", colClasses =
"numeric", comment.char = ""))
W[[1]]
`W[[1]]` should be a data.frame, but it will not have the name
"AAR_DailyWeather" unless you assign names using the values in `filelist`.
dd(data = W[[1]]$Precip, time.scale = W[[1]]$year)
It is unclear what this code snippet is supposed to represent. There is
no function named `dd` in the base packages. There is a `dd` function in
package Hmisc but if you use it you need to assign the results to an
object name. Just executing the call will do nothing except perhaps
checking to see if there are errors.
Fehler in W[[1]]$year : $ operator is invalid for atomic vectors
Also unclear is how that error message came about. It seems to indicate
that W[[1]] is not a dataframe. You can examine the `W` object with
`str(W)`.
Looking at the help page for ClimInd::dd it appears that the
'time.scale' argument is supposed to signify what time scale to assume,
namely one of {month, season, year}. Pretty crappy documentation since
it is unstated what type the allowable values are and the usage example
shows
|time.scale = YEAR|
And there does not appear to be any definition of what the value of `YEAR` is
supposed to be.
It appears to me that you have been assigned a task in a poorly
documented package.
Contact the package author.
--
David.
Kind regards
Sibylle
__
[email protected] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide https://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
__
[email protected] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide https://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Re: [R] Using multiple dat files
Sorry, wrong language. "through the function's closures" in my email should be: through the function's chain of environments. (A function in R *is* a closure). -- Bert On Wed, Nov 6, 2024 at 12:46 PM Bert Gunter wrote: > "It seems therefore that there is no other way than read in individually > > 100 weather tables using read.tables., right? Using file.choose() doesn't > change the work." > > Yes. With that many files, file.choose() does not make sense. However, I > still do not understand what is the problem with using lapply() on the > character vector of file names with read.table() as you did in your > original post to read in all the files as components of the W list. As data > frames are also lists, you can extract individual columns as you did before > using the $ extractor, i.e. W[[1]]$year, etc.. You can also use data frame > indices for the columns, i.e. W[[1]][ ,1] or W[[1]][ ,"year"] . All of > which I believe you know. > > However, I will hazard a *guess* (so exercise due diligence and check) as > to the cause of your original error, > > "dd(data = W[[1]]$Precip, time.scale = W[[1]]$year) > Fehler in W[[1]]$year : $ operator is invalid for atomic vectors' > > even though I know **nothing** about the dd() function. My guess is: > "data" is usually an argument to a function that tells it to use > "nonstandard evaluation" to look for arguments and other names in the > function first in the "data" argument, rather than by following R's > "standard" evaluation by looking first through the function's closures. > **If** this guess is correct, the call you gave above should be something > like: > > dd(data = W[[1]], time.scale = year, precip = Precip,...) > > where precip (small "p") is a formal argument of the dd() function and > Precip is a column in the data frame W[[1]]. If this is wrong, my > apologies, and feel free to ignore without responding. > > Best, > Bert > > On Wed, Nov 6, 2024 at 10:59 AM Sibylle Stöckli > wrote: > >> Dear Rui >> Dear Bert >> >> Many thanks >> >> Solution >> filelist <- list.files(path = >> "O:/Data-Work/2.../Daten_RA-MeteoCH_1990-2021", pattern='*.dat', all.files= >> T, full.names= T) >> AAR<-read.table(filelist[1]) >> >> It seems therefore that there is no other way than read in individually > >> 100 weather tables using read.tables., right? Using file.choose() doesn't >> change the work. >> >> Yes my .dat files are data.frames >> >> > str(W[[1]])'data.frame': 11688 obs. of 7 variables: >> $ year : num 1990 1990 1990 1990 1990 1990 1990 1990 1990 1990 ... >> $ DOY : num 1 2 3 4 5 6 7 8 9 10 ... >> $ Ta: num -2.67 -2.77 -2.23 -2.21 -0.98 0.82 0.49 -1.02 -2.31 -3.36 ... >> $ Tmin : num -3.5 -3.7 -4.26 -2.87 -2.98 0.3 -0.83 -1.27 -3 -3.82 ... >> $ Tmax : num -1.13 -0.15 -0.13 -0.45 1 1.87 1.72 -0.35 -0.85 -2.3 ... >> $ Precip: num 0 0 0 0 0.45 1.81 0.03 0 0 0 ... >> $ rSSD : num 0 0.08 0 0 0.08 0 0 0 0 0 ... >> >> >> *Gesendet: *Mittwoch, 6. November 2024 um 18:28 >> *Von: *"Bert Gunter" >> *An: *"Sibylle Stöckli" >> *CC: *[email protected] >> *Betreff: *Re: [R] Using multiple dat files >> Not quite sure if I understand you. >> >> list.files() simply returns a character vector(not a list). You can >> simply use a vector index to select whatever file you wish to read. So if >> your desired filename is the 5th element of filelist above, something like >> >> myfile <- read.table(filename[5], ...) >> >> You can also use regular expressions to choose a bunch of files that have >> some common signature to their names that you can read in simultaneously >> using your "filelist" vector of names via something like: >> >> myfiles <- lapply(grep("weath", filelist, value = TRUE), \(x) >> read.table(x,...)) >> ### Note that the result of lapply *is* a list, so use list indexing for >> extraction from myfiles. >> >> >> You can also choose files to read interactively (via a GUI interface) >> using file.choose() instead of using list.files() if you prefer to do it >> that way. >> >> Cheers, >> Bert >> >> On Wed, Nov 6, 2024 at 8:25 AM Sibylle Stöckli via R-help < >> [email protected]> wrote: >> >>> Dear community >>> >>> To import multiple .dat weather files I am using list.files(). >>> I intend to use the R package “Clim
Re: [R] Using multiple dat files
"It seems therefore that there is no other way than read in individually > 100 weather tables using read.tables., right? Using file.choose() doesn't change the work." Yes. With that many files, file.choose() does not make sense. However, I still do not understand what is the problem with using lapply() on the character vector of file names with read.table() as you did in your original post to read in all the files as components of the W list. As data frames are also lists, you can extract individual columns as you did before using the $ extractor, i.e. W[[1]]$year, etc.. You can also use data frame indices for the columns, i.e. W[[1]][ ,1] or W[[1]][ ,"year"] . All of which I believe you know. However, I will hazard a *guess* (so exercise due diligence and check) as to the cause of your original error, "dd(data = W[[1]]$Precip, time.scale = W[[1]]$year) Fehler in W[[1]]$year : $ operator is invalid for atomic vectors' even though I know **nothing** about the dd() function. My guess is: "data" is usually an argument to a function that tells it to use "nonstandard evaluation" to look for arguments and other names in the function first in the "data" argument, rather than by following R's "standard" evaluation by looking first through the function's closures. **If** this guess is correct, the call you gave above should be something like: dd(data = W[[1]], time.scale = year, precip = Precip,...) where precip (small "p") is a formal argument of the dd() function and Precip is a column in the data frame W[[1]]. If this is wrong, my apologies, and feel free to ignore without responding. Best, Bert On Wed, Nov 6, 2024 at 10:59 AM Sibylle Stöckli wrote: > Dear Rui > Dear Bert > > Many thanks > > Solution > filelist <- list.files(path = > "O:/Data-Work/2.../Daten_RA-MeteoCH_1990-2021", pattern='*.dat', all.files= > T, full.names= T) > AAR<-read.table(filelist[1]) > > It seems therefore that there is no other way than read in individually > > 100 weather tables using read.tables., right? Using file.choose() doesn't > change the work. > > Yes my .dat files are data.frames > > > str(W[[1]])'data.frame':11688 obs. of 7 variables: > $ year : num 1990 1990 1990 1990 1990 1990 1990 1990 1990 1990 ... > $ DOY : num 1 2 3 4 5 6 7 8 9 10 ... > $ Ta: num -2.67 -2.77 -2.23 -2.21 -0.98 0.82 0.49 -1.02 -2.31 -3.36 ... > $ Tmin : num -3.5 -3.7 -4.26 -2.87 -2.98 0.3 -0.83 -1.27 -3 -3.82 ... > $ Tmax : num -1.13 -0.15 -0.13 -0.45 1 1.87 1.72 -0.35 -0.85 -2.3 ... > $ Precip: num 0 0 0 0 0.45 1.81 0.03 0 0 0 ... > $ rSSD : num 0 0.08 0 0 0.08 0 0 0 0 0 ... > > > *Gesendet: *Mittwoch, 6. November 2024 um 18:28 > *Von: *"Bert Gunter" > *An: *"Sibylle Stöckli" > *CC: *[email protected] > *Betreff: *Re: [R] Using multiple dat files > Not quite sure if I understand you. > > list.files() simply returns a character vector(not a list). You can simply > use a vector index to select whatever file you wish to read. So if your > desired filename is the 5th element of filelist above, something like > > myfile <- read.table(filename[5], ...) > > You can also use regular expressions to choose a bunch of files that have > some common signature to their names that you can read in simultaneously > using your "filelist" vector of names via something like: > > myfiles <- lapply(grep("weath", filelist, value = TRUE), \(x) > read.table(x,...)) > ### Note that the result of lapply *is* a list, so use list indexing for > extraction from myfiles. > > > You can also choose files to read interactively (via a GUI interface) > using file.choose() instead of using list.files() if you prefer to do it > that way. > > Cheers, > Bert > > On Wed, Nov 6, 2024 at 8:25 AM Sibylle Stöckli via R-help < > [email protected]> wrote: > >> Dear community >> >> To import multiple .dat weather files I am using list.files(). >> I intend to use the R package “ClimInd” to calculate different >> agroclimatic indicators. >> >> Question: Is there another solution to import multiple .dat files so that >> I can select elements from the list, e.g. one specific weather file >> (example AAR_DailyWeather)? >> >> >> # Import multiple .dat files weather data >> filelist <- list.files(path = >> "O:/Data-Work/……./Daten_RA-MeteoCH_1990-2021", pattern='*.dat', all.files= >> T, full.names= T) >> W <- lapply(filelist, function(x) read.table(x, header = TRUE, sep = "", >> colClasses = "numeric", comment.char = "")) >>
Re: [R] Using multiple dat files
Not quite sure if I understand you.
list.files() simply returns a character vector(not a list). You can simply
use a vector index to select whatever file you wish to read. So if your
desired filename is the 5th element of filelist above, something like
myfile <- read.table(filename[5], ...)
You can also use regular expressions to choose a bunch of files that have
some common signature to their names that you can read in simultaneously
using your "filelist" vector of names via something like:
myfiles <- lapply(grep("weath", filelist, value = TRUE), \(x)
read.table(x,...))
### Note that the result of lapply *is* a list, so use list indexing for
extraction from myfiles.
You can also choose files to read interactively (via a GUI interface) using
file.choose() instead of using list.files() if you prefer to do it that way.
Cheers,
Bert
On Wed, Nov 6, 2024 at 8:25 AM Sibylle Stöckli via R-help <
[email protected]> wrote:
> Dear community
>
> To import multiple .dat weather files I am using list.files().
> I intend to use the R package “ClimInd” to calculate different
> agroclimatic indicators.
>
> Question: Is there another solution to import multiple .dat files so that
> I can select elements from the list, e.g. one specific weather file
> (example AAR_DailyWeather)?
>
>
> # Import multiple .dat files weather data
> filelist <- list.files(path =
> "O:/Data-Work/……./Daten_RA-MeteoCH_1990-2021", pattern='*.dat', all.files=
> T, full.names= T)
> W <- lapply(filelist, function(x) read.table(x, header = TRUE, sep = "",
> colClasses = "numeric", comment.char = ""))
> W[[1]]
>
> > dd(data = W[[1]]$Precip, time.scale = W[[1]]$year)
> Fehler in W[[1]]$year : $ operator is invalid for atomic vectors
>
> Kind regards
> Sibylle
>
>
>
> __
> [email protected] mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
> https://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>
[[alternative HTML version deleted]]
__
[email protected] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide https://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Re: [R] Using multiple dat files
Às 06:29 de 06/11/2024, Sibylle Stöckli via R-help escreveu: Dear community To import multiple .dat weather files I am using list.files(). I intend to use the R package “ClimInd” to calculate different agroclimatic indicators. Question: Is there another solution to import multiple .dat files so that I can select elements from the list, e.g. one specific weather file (example AAR_DailyWeather)? # Import multiple .dat files weather data filelist <- list.files(path = "O:/Data-Work/……./Daten_RA-MeteoCH_1990-2021", pattern='*.dat', all.files= T, full.names= T) W <- lapply(filelist, function(x) read.table(x, header = TRUE, sep = "", colClasses = "numeric", comment.char = "")) W[[1]] dd(data = W[[1]]$Precip, time.scale = W[[1]]$year) Fehler in W[[1]]$year : $ operator is invalid for atomic vectors Kind regards Sibylle __ [email protected] mailing list -- To UNSUBSCRIBE and more, see https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide https://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. Hello, I find it strange that the error is complaining about the 2nd argument, W[[1]]$year. It seems that column W[[1]]$Precip exists but not year. What does str(W[[1]]) return? A data.frame? And why sep = "" when reading the files, aren't those files csv files? Hope this helps, Rui Barradas -- Este e-mail foi analisado pelo software antivírus AVG para verificar a presença de vírus. www.avg.com __ [email protected] mailing list -- To UNSUBSCRIBE and more, see https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide https://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
