[R] Huge differences in Ram Consumption with different versions of R on the same scripts
Hello R Help, I have some R vignettes that run fpp2 time series regressions. I have run these for a long time using R and a 12 core computer system. I ran each core using Linux to run a vignette on that core, so that all 12 could work concurrently. With 48GB of ram, the ram never filled up. I ran these regressions for hours, one data set right after the other on each core. Recently, I switched to Oracle Linux 8 and R 4.2 Now, with the same scripts, and the same data, the ram fills up and R reserves 4.2GB per instance in some cases. This results in all the ram being consumed and the swap space on the system activating constantly so that the performance is abysmal. It begins using 12Gb of swap space in addition to the ram consumed. It bogs the system so bad that one can't even establish new terminal sessions. Is there a way to specify a maximum ram allowance in an R vignette? If that’s not possible, what resources can you recommend to help identify the reason for the change in memory use? Why would a different version of R/Linux use 2GB per instance, while another uses 4.4GB? What kind of troubleshooting or programming techniques should I research for this kind of concern? Robert Knight [[alternative HTML version deleted]] __ R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] AIC
https://cran.r-project.org/web/packages/quantreg/quantreg.pdf Seems to be by the guy who has been writing about A1c and quantile regression since at least 1978. His work is cited in several papers on the subject. I would copy-paste his name here, but this from an iPhone. On Fri, Nov 25, 2022 at 6:45 PM Rolf Turner wrote: > On Fri, 25 Nov 2022 12:04:10 +0530 > ASHLIN VARKEY wrote: > > > How to calculate AIC , when the distribution have only quantile > > function > > This mailing list is for questions about the R language. It is not > for questions about statistical theory. You might get some useful > information from stackexchange, but you will have to make your > question much clearer and much more explicit. Give a reasonably > detailed example. Do not expect your readers to be telepathic. > > It is not at all clear to me that your question actually makes any > sense at all. > > cheers, > > Rolf Turner > > -- > Honorary Research Fellow > Department of Statistics > University of Auckland > Phone: +64-9-373-7599 ext. 88276 > > __ > R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide > http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code. > -- Robert Knight Developer of Meal Plan and Grocery List maker https://play.google.com/store/apps/details?id=io.robertknight.MPGL [[alternative HTML version deleted]] __ R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Date read correctly from CSV, then reformatted incorrectly by R
Richard,
This response was awe-inspiring. Thank you.
-Original Message-
From: R-help On Behalf Of Richard O'Keefe
Sent: Sunday, November 21, 2021 8:55 PM
To: Philip Monk
Cc: R Project Help
Subject: Re: [R] Date read correctly from CSV, then reformatted incorrectly
by R
CSV data is very often strangely laid out. For analysis, Buffer Date
Reading
100... ...
100... ...
and so on is more like what a data frame should be. I get quite annoyed
when I finally manage to extract data from a government agency only to find
that my tax money has been spent on making it harder to access than it
needed to be.
(1) You do NOT need any additional library to convert dates.
?strptime is quite capable.
(2) Just because reshaping CAN be done in R doesn't mean it
SHOULD be. Instead of reading data in as the wrong format
and then hacking it into shape every time, it makes sense
to convert the data once and only once, then load the
converted data. It took just a couple of minutes to write
(CSVDecoder read: 'transpose-in.csv') bindOwn: [:source |
(CSVEncoder write: 'transpose-out.csv') bindOwn: [:target |
source next bind: [:header | "Label date-1 ... date-n"
target nextPut: {header first. 'Date'. 'Reading'}.
[source atEnd] whileFalse: [
source next bind: [:group |
group with: header keysAndValuesDo: [:index :reading :date |
1 < index ifTrue: [
(date subStrings: '/') bind: [:dmy |
(dmy third,'-',dmy second,'-',dmy first) bind: [:iso |
target nextPut: {group first. iso.
reading}]]].
in another programming language, run it, and turn your example into
Buffer,Date,Reading
100,2016-10-28,2.437110889
100,2016-11-19,-8.69674895
100,2016-12-31,3.239299816
100,2017-01-16,2.443183304
100,2017-03-05,2.346743827
200,2016-10-28,2.524329899
200,2016-11-19,-7.688862068
...
You could do the same kind of thing easily in Perl, Python, F#, ...
Then just read the table in using
read.csv("transpose-out.csv", colClasses = c("integer","Date","numeric"))
and you're away laughing.
(3) Of course you can do the whole thing in base R.
h <- read.csv("transpose-in.csv", header=FALSE, nrows=1,
stringsAsFactors=FALSE)
d <- strptime(h[1,-1], format="%d/%m/%Y")
b <- read.csv("transpose-in.csv", header=FALSE, skip=1)
r <- expand.grid(Date=d, Buffer=b[,1])
r$Result <- as.vector(t(as.matrix(b[,-1])))
Lessons:
(A) You don't have to read a CSV file (or any other) all in one piece.
This pays off when the structure is irregular.
(B) You don't HAVE to accept or convert column names.
(C) strptime is your friend.
(D) expand.grid is particularly handy for "matrix form" CSV data.
(E) Someone who suggests doing something in another language because
it is easier can end up with egg on his face when doing the whole
thing in R turns out to be easier, simpler, and far more obvious.
(A) really is an important lesson.
(F) It's *amazing* what you can do in base R. It is useful to
familiarise yourself with its capabilities before considering other
packages. Compositional data? Not in base R. Correspondence
analysis? Not in base R. Data reshaping? Very much there.
On Sun, 21 Nov 2021 at 06:09, Philip Monk wrote:
> Hello,
>
> Simple but infuriating problem.
>
> Reading in CSV of data using :
>
> ```
> # CSV file has column headers with date of scene capture in format
> dd/mm/ # check.names = FALSE averts R incorrectly processing dates
> due to '/'
> data <- read.csv("C:/R_data/Bungala (b2000) julian.csv", check.names =
> FALSE)
>
> # Converts data table from wide (many columns) to long (many rows) and
> creates the new object 'data_long'
> # Column 1 is the 'Buffer' number (100-2000), Columns 2-25 contain
> monthly data covering 2 years (the header row being the date, and rows
> 2-21 being a value for each buffer).
> # Column headers for columns 2:25 are mutated into a column called
> 'Date', values for each buffer and each date into the column 'LST'
> data_long <- data %>% pivot_longer(cols = 2:25, names_to = "Date",
> values_to = "LST")
>
> # Instructs R to treat the 'Date' column data as a date data_long$Date
> <- as.Date(data_long$Date) ```
>
> Using str(data), I can see that R has correctly read the dates in the
> format %d/%m/%y (e.g. 15/12/2015) though has the data type as chr.
>
> Once changing the type to 'Date', however, the date is reconfigured.
> For instance, 15/01/2010 (15 January 2010), becomes 0015-01-20.
>
> I've tried ```data_long$Date <- as.Date(data_long$Date, format =
> "%d/%m.%y")```, and also ```tryformat c("%d/%m%y")```, but either the
> error persists or I get ```NA```.
>
> How do I make R change Date from 'chr' to 'date' without it going wrong?
>
> Suggestions/hints/solutions would be most
Re: [R] Fwd: Merging multiple csv files to new file
It might be easier to settle on the desired final csv layout and use Python
to copy the rows via line reads. Python doesn't care about the data type
in a given "cell", numeric or char, whereas the type errors R would
encounter would make the task very difficult.
On Wed, Nov 3, 2021, 10:36 AM gabrielle aban steinberg <
gabrielleabansteinb...@gmail.com> wrote:
> Hello, I would like to merge 18 csv files into a master data csv file, but
> each file has a different number of columns (mostly found in one or more of
> the other cvs files) and different number of rows.
>
> I have tried something like the following in R Studio (cloud):
>
> all_data_fit_files <- rbind("dailyActivity_merged.csv",
> "dailyCalories_merged.csv", "dailyIntensities_merged.csv",
> "dailySteps_merged.csv", "heartrate_seconds_merged.csv",
> "hourlyCalories_merged.csv", "hourlyIntensities_merged.csv",
> "hourlySteps_merged.csv", "minuteCaloriesNarrow_merged.csv",
> "minuteCaloriesWide_merged.csv", "minuteIntensitiesNarrow_merged.csv",
> "minuteIntensitiesWide_merged.csv", "minuteMETsNarrow_merged.csv",
> "minuteSleep_merged.csv", "minuteStepsNarrow_merged.csv",
> “minuteStepsWide_merged.csv", "sleepDay_merged.csv",
> "minuteStepsWide_merged.csv", "sleepDay_merged.csv",
> "weightLogInfo_merged.csv")
>
>
>
> But I am getting the following error:
>
> Error: unexpected input in "rlySteps_merged.csv",
> "minuteCaloriesNarrow_merged.csv", "minuteCaloriesWide_merged.csv",
> "minuteIntensitiesNarrow_merged.csv",
> "minuteIntensitiesWide_merged.csv", "minuteMETsNarrow_merged.csv"
>
>
> (Maybe the R Studio free trial/usage is underpowered for my project?)
>
> [[alternative HTML version deleted]]
>
> __
> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
> http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>
[[alternative HTML version deleted]]
__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Re: [R] problem for strsplit function
My method would be to use parse and deparse and substitute. It would iterate over each file name and build a new list of file names with the last four characters removed to have only the left side, and only the last four remaining to have only the right side. Then a new dataframe would be created of the partial file names. Deparse and substitute to get the file names into a string, then use character removal on the sides, put the file name into a new vector, and then create the relevant data frame if desired. This allows one to Rely on their software development metaphor. It might lack a certain finess, but the metaphor is either a loom or a boxing match against a CSV so it’s fun. :) Sent from my iPhone > On Jul 9, 2021, at 10:33 PM, Rolf Turner wrote: > > > This discussion has developed in such a way that it seems a better > subject line would be "problem for the hairsplit function". :-) > > cheers, > > Rolf Turner > > -- > Honorary Research Fellow > Department of Statistics > University of Auckland > Phone: +64-9-373-7599 ext. 88276 > > __ > R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code. __ R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Wayland backend for R
Perhaps software rendering would work.
Export RSTUDIO_CHROMIUM_ARGUMENTS="--disable-gpu"
/usr/lib/rstudio/bin/rstudio
On Wed, Jun 23, 2021, 10:01 AM Phillips Rogfield
wrote:
> Hello Paul,
>
> thank you for your kind advice.
>
> RStudio doesn't start at all this way. It gives me the following error:
>
> $ QT_QPA_PLATFORM=wayland rstudio
> Warning: Ignoring XDG_SESSION_TYPE=wayland on Gnome. Use
> QT_QPA_PLATFORM=wayland to run on Wayland anyway.
> QSocketNotifier: Can only be used with threads started with QThread
> Failed to load client buffer integration: wayland-egl
>
> WebEngineContext used before QtWebEngine::initialize() or OpenGL context
> creation failed.
> qt.qpa.wayland: No shell integration named "xdg-shell" found
> qt.qpa.wayland: No shell integration named "xdg-shell-v6" found
> qt.qpa.wayland: No shell integration named "wl-shell" found
> qt.qpa.wayland: No shell integration named "ivi-shell" found
> qt.qpa.wayland: Loading shell integration failed.
> qt.qpa.wayland: Attempted to load the following shells ("xdg-shell",
> "xdg-shell-v6", "wl-shell", "ivi-shell")
> qt.qpa.wayland: Wayland does not support QWindow::requestActivate()
>
> Best regards.
>
> On 23/06/2021 02:22, Paul Murrell wrote:
> > Hi
> >
> > I do not know of any Wayland backend for R.
> >
> > You might be able to try the R Studio IDE configured for Wayland and
> > see how its graphics device performs ?
> > https://github.com/rstudio/rstudio/issues/4686
> >
> > Paul
> >
> > On 23/06/21 1:25 am, Phillips Rogfield wrote:
> >> I have compiled R from source and I had to install the X11 libraries.
> >>
> >> I use Wayland, and I am having problems plotting on X11 (I guess it uses
> >> XWayland) with this version.
> >>
> >> Is there a Wayland backend for R?
> >>
> >> Some configuration option I need to turn on in order to use it?
> >>
> >> __
> >> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> >> https://stat.ethz.ch/mailman/listinfo/r-help
> >>
> >> PLEASE do read the posting guide
> >> http://www.R-project.org/posting-guide.html
> >>
> >> and provide commented, minimal, self-contained, reproducible code.
> >
>
> __
> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
> http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>
[[alternative HTML version deleted]]
__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Re: [R] CentOS 8: installing R
Openblas-threads is in the appstream repository rather than power tools. https://centos.pkgs.org/8/centos-appstream-x86_64/openblas-threads-0.3.3-5.el8.x86_64.rpm.html On Mon, May 24, 2021, 2:56 PM Marc Schwartz via R-help wrote: > Hi Roger, > > I can't speak to the details here, albeit, there was a thread back in > April on R-Devel (also not the right list for this topic), where one of > the Fedora maintainers provided some insights for CentOS 7.x, where > there were missing/incompatible tool chain issues. > > I would recommend re-posting this to r-sig-fedora, which is focused upon > R on Red Hat and Fedora based Linux distributions, including CentOS: > >https://stat.ethz.ch/mailman/listinfo/r-sig-fedora > > The Fedora/EPEL R maintainers do monitor and respond on that list, and > can likely provide you with more informed replies. > > Regards, > > Marc Schwartz > > Roger Bos wrote on 5/24/21 3:41 PM: > > Dear all, > > > > I seem to be having an impossible time install R on my centos 8 virtual > > machine (I know centos 8 is no longer maintained, but it is a work server > > so I have no choice in the matter.). I installed EPEL and enabled > > PowerTools, but I cannot install R due to conflicts in the requirements > for > > openblas-devel. Could some help me get past this issue? Thanks in > > advance, Roger > > > > [RCOAdmin@usd1sapp101 ~]$ sudo dnf install R > > CentOS-8 - PowerTools > >14 kB/s | 4.3 > kB > > 00:00 > > Extra Packages for Enterprise Linux Modular 8 - x86_64 > > 30 kB/s | > 15 kB > > 00:00 > > Extra Packages for Enterprise Linux 8 - x86_64 > > 52 kB/s | > 15 kB > > 00:00 > > Error: > > Problem: package R-devel-4.0.5-1.el8.x86_64 requires R-core-devel = > > 4.0.5-1.el8, but none of the providers can be installed > >- package R-4.0.5-1.el8.x86_64 requires R-devel = 4.0.5-1.el8, but > none > > of the providers can be installed > >- package R-core-devel-4.0.5-1.el8.x86_64 requires openblas-devel, but > > none of the providers can be installed > >- conflicting requests > >- nothing provides openblas(x86-32) = 0.3.3-5.el8 needed by > > openblas-devel-0.3.3-5.el8.i686 > >- nothing provides openblas-threads(x86-32) = 0.3.3-5.el8 needed by > > openblas-devel-0.3.3-5.el8.i686 > >- nothing provides openblas(x86-64) = 0.3.3-5.el8 needed by > > openblas-devel-0.3.3-5.el8.x86_64 > >- nothing provides openblas-threads(x86-64) = 0.3.3-5.el8 needed by > > openblas-devel-0.3.3-5.el8.x86_64 > > (try to add '--skip-broken' to skip uninstallable packages or '--nobest' > to > > use not only best candidate packages) > > [RCOAdmin@usd1sapp101 ~]$ sudo dnf install openblass-devel > > Last metadata expiration check: 0:00:31 ago on Mon 24 May 2021 09:34:56 > PM > > CEST. > > No match for argument: openblass-devel > > Error: Unable to find a match: openblass-devel > > [RCOAdmin@usd1sapp101 ~]$ sudo dnf install openblas-devel > > Last metadata expiration check: 0:00:43 ago on Mon 24 May 2021 09:34:56 > PM > > CEST. > > Error: > > Problem: cannot install the best candidate for the job > >- nothing provides openblas(x86-64) = 0.3.3-5.el8 needed by > > openblas-devel-0.3.3-5.el8.x86_64 > >- nothing provides openblas-threads(x86-64) = 0.3.3-5.el8 needed by > > openblas-devel-0.3.3-5.el8.x86_64 > > (try to add '--skip-broken' to skip uninstallable packages or '--nobest' > to > > use not only best candidate packages) > > [RCOAdmin@usd1sapp101 ~]$ sudo dnf install openblas-devel --nobest > > Last metadata expiration check: 0:00:52 ago on Mon 24 May 2021 09:34:56 > PM > > CEST. > > Error: > > Problem: conflicting requests > >- nothing provides openblas(x86-32) = 0.3.3-5.el8 needed by > > openblas-devel-0.3.3-5.el8.i686 > >- nothing provides openblas-threads(x86-32) = 0.3.3-5.el8 needed by > > openblas-devel-0.3.3-5.el8.i686 > >- nothing provides openblas(x86-64) = 0.3.3-5.el8 needed by > > openblas-devel-0.3.3-5.el8.x86_64 > >- nothing provides openblas-threads(x86-64) = 0.3.3-5.el8 needed by > > openblas-devel-0.3.3-5.el8.x86_64 > > (try to add '--skip-broken' to skip uninstallable packages) > > [RCOAdmin@usd1sapp101 ~]$ > > > > [[alternative HTML version deleted]] > > > > __ > > R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see > > https://stat.ethz.ch/mailman/listinfo/r-help > > PLEASE do read the posting guide > http://www.R-project.org/posting-guide.html > > and provide commented, minimal, self-contained, reproducible code. > > > > __ > R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide > http://www.R-project.org/posting-guide.html > and provide commented,
Re: [R] Variable labels
Actually, I just found exactly what you want. Before that though, I am
having a hard time finding any such cool job despite having even had
classes with some great professors in economics at UND, and so I work in a
completely non data related thing.
Here is exactly what you want, code included. Then on the right side of
Rstudio you can click the word "desc" and get a table of the variable name
and it's description.
variable <- c("year", "inv", "pop", "price", "linv", "lpop", "lprice", "t",
"invpc", "linvpc", "lprice_1", "linvpc_1", "gprice", "ginvpc")
description <- c("1947-1988","real housing inv, millions $","population,
1000s","housing price index; 1982 = 1",
"log(inv)","log(pop)","log(price)","time trend: t=1,...,42","per
capita inv: inv/pop",
"log(invpc)","lprice[_n-1]","linvpc[_n-1]","lprice -
lprice_1","linvpc - linvpc_1")
desc <- cbind(variable, description)
Robert D. Knight, MBA
Developer of Meal Plan and Grocery List maker for Android and iOS.
https://play.google.com/store/apps/details?id=io.robertknight.MPGL
On Wed, May 12, 2021 at 9:49 PM Steven Yen wrote:
> I insert variable with the expss function as shown below. No error
> message. My question is, how to save the variable labels in the data
> frame so that I can click to read the labels. Thank you.
>
> mydata<-read_excel("data/Excel/hseinv.xlsx",na=".")
> library(expss)
> mydata=apply_labels(mydata,
> year ="1947-1988",
> inv="real housing inv, millions $",
> pop="population, 1000s",
> price ="housing price index; 1982 = 1",
> linv ="log(inv)",
> lpop ="log(pop)",
> lprice ="log(price)",
> t ="time trend: t=1,...,42",
> invpc ="per capita inv: inv/pop",
> linvpc ="log(invpc)",
> lprice_1="lprice[_n-1]",
> linvpc_1="linvpc[_n-1]",
> gprice ="lprice - lprice_1",
> ginvpc ="linvpc - linvpc_1")
>
> __
> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
> http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>
[[alternative HTML version deleted]]
__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Re: [R] Variable labels
Hi Steven,
You make great sense wanting to have labels for your variables. When in
RStudio, the little arrow beside "mydata" in the Environment tab can be
clicked and you see all the variables there. And so you would like to see
a description under the variable names. Here is one way to accomplish
that. The following is not pseudocode, it's the actual code you should use.
Step 1, create a function that applies an attribute called "description" to
a variable.
desc <- function(obj) attr(obj, "description")
Step 2, use attribute to apply the description
attr(mydata$invpc, "description") <- "Per capita inventory"
Step 3, Now you can either click the arrow beside "mydata" on the
environment tab and see that written description with the word
"description" in quotes. You can also type
desc(mydata$invpc)
And that will provide you the associated description in text form.
Robert D. Knight, MBA
Developer of Meal Plan and Grocery List maker for Android and iOS.
https://play.google.com/store/apps/details?id=io.robertknight.MPGL
On Wed, May 12, 2021 at 9:49 PM Steven Yen wrote:
> I insert variable with the expss function as shown below. No error
> message. My question is, how to save the variable labels in the data
> frame so that I can click to read the labels. Thank you.
>
> mydata<-read_excel("data/Excel/hseinv.xlsx",na=".")
> library(expss)
> mydata=apply_labels(mydata,
> year ="1947-1988",
> inv="real housing inv, millions $",
> pop="population, 1000s",
> price ="housing price index; 1982 = 1",
> linv ="log(inv)",
> lpop ="log(pop)",
> lprice ="log(price)",
> t ="time trend: t=1,...,42",
> invpc ="per capita inv: inv/pop",
> linvpc ="log(invpc)",
> lprice_1="lprice[_n-1]",
> linvpc_1="linvpc[_n-1]",
> gprice ="lprice - lprice_1",
> ginvpc ="linvpc - linvpc_1")
>
> __
> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
> http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>
[[alternative HTML version deleted]]
__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Re: [R] Is there anyone who uses both R and Python here? How do you debug? Perhaps in RStudio?
An iterative process works well. Python to get the data desired and then Rscript script.r from a command line. My process involves building a script in R using, using Rstudio, Pycharm, VS Code, Kate, or some other editor. Then using data input built with Python as input to Rscript. The R scripts produce excel files or CSV data for other use RStudio is amazing for some slow pace academic work. The "expected a numeric but got a char" error appeared to often for my needs and so the workflows wound up with Python building data that's already cleaned for use in R to avoid data import troubles. My code use a functional paradigm rather than object oriented paradigm. Python does more than just munge my data since it handled many mathematic operations on it, but it's ultimate purpose is to clean large amounts of data to avoid import errors in R. On Wed, Jan 27, 2021, 1:49 AM C W wrote: > Hello all, > > I'm a long time R user, but recently also using Python. I noticed that > RStudio rolled out Python through reticulate. It's great so far! > > My question is, how do you debug in Python? > > In R, I simply step through the code script in my console with cmd+enter. > But you can't do that with Python, some of them are objects. > > Here's my example. > class person: > def __init__(self, id, created_at, name, attend_date, distance): > """Create a new `person`. > """ > self._id = id > self.created_at = created_at > self.name = name > self.attend_date = attend_date > self.distance = distance > > @classmethod > def get_person(self, employee): > """Find and return a person by. > """ > return person(employee['created_at'], >employee['id'], >employee['name'], >employee['attend_date'], >employee['distance'] >) > > The error message says self._id was 'str', but expecting an 'int'. I can't > do: > > self._id = 5 > I guess it's "hidden". Can't really assign and test like that. > > It seems hardcore Python programmers just use a debugger, and do not > understand the greatness of interactive IDE and console. I'd still like to > stay in IDE, hopefully. > > So, how are the R users coping with object classes? Do you just instantiate > every time? What if you got 10 of these class person objects to debug? > > I know this may be a Python question. But, I really wanted to see from a R > user's working experience. > > Thanks a lot, > > Mike > > [[alternative HTML version deleted]] > > __ > R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide > http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code. > [[alternative HTML version deleted]] __ R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] How to pass a character string with a hyphen
Strip the left characters and strip the right characters into their own variables using one of the methods that can do that. Then pass it using something like paste(left, "-", right). On Tue, Nov 17, 2020, 2:43 PM Jeff Reichman wrote: > R-Help > > > > How does one pass a character string containing a hyphen? I have a function > that accesses an api if I hard code the object, for example > > > > key_key <- "-" > > > > it works but when I pass the key code to the function (say something like > key_code <- code_input) it returns only . So R is seeing a string with > a negative operator I'm assuming > > > > Jeff > > > [[alternative HTML version deleted]] > > __ > R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide > http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code. > [[alternative HTML version deleted]] __ R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Can anyone advise me on running R and Rstudio on an AWS virtual machine
Would recommend a CentOS 7 HVM image, from centos.org <http://centos.org/> as the direct vendor rather than others in the market. Activate a t3.nano with 512MB of ram, and then follow the instructions to get Studio server installed. Create a new user for Studio-server. Also, ensure that you have a backup plan because the security effects of RStudio server in that situation are not extremely well studied. RStudio server doesn’t require web servers or anything like that. A t3a.nano would be about 10% cheaper, but slower on the CPU clock speed, which might reduce performance. If you are using Dplyr and multithreading your work then you might want to move up in the t3 offerings until you get one that has dual vCPUs, but the cost increases pretty quickly. It’s great to have a Studio version that never changes or updates on you. Ensure a backup plan in case the server gets hacked in some fashion. You could restrict access to your own IP address in the AWS security group settings which would drastically minimize the risk of that. Robert Knight > On Oct 14, 2020, at 12:00 PM, Chris Evans wrote: > > This is a funny one and if it's off topic here, I would be grateful if I > could be guided to where it would be on topic. I have done some searching but > not very successfully so far. > > Situation: I am doing some analyses of data that are stored in a postgres > database in the AWS cloud and using the RJDBC and dplyr packages for the > specifics of yanking the data to my own machine. They work and worked fine > when the database was in Redshift last year. However, I am getting error > messages on data transfer that I think are down to my very slow broadband > where I am now and that pushed me to think I should move to doing the > analyses on a virtual machine in the AWS cloud so the link from the data to > the machine is fast and so only getting the code up there and the results > down will go through my broadband. I would like to be able to work > interactively and I have some experience of that, (which is clearly not > R-help business!): I am fairly happy I can do that. If not, just ssh terminal > access to the VM would be OK and I'm used to that too. > > My suspicions that the errors are down to my broadband are the trigger but I > suspect that such a set may be the only way for me to go quite soon as these > particular data sets are growing fast (they're not huge yet, whole R saved > session image is 28Mb). I can see being able to upscale a VM in the cloud is > the sensible way to go. However, this is currently a bit outside my > experience. > > I have searched and found this from Amazon: > [ > https://aws.amazon.com/marketplace/pp/B07ZDBJ42H/ref=portal_asin_url#pdp-reviews > | > https://aws.amazon.com/marketplace/pp/B07ZDBJ42H/ref=portal_asin_url#pdp-reviews > ] > and a VM Ubuntu with R and Rstudio sounds perfect: pretty much replicating my > laptop. I can try that out for free by the look of it (can that be true?!) > but I would like to get any advice I can first. > > I also found: > [ > https://techcommunity.microsoft.com/t5/educator-developer-blog/hosting-rserver-and-rstudio-on-azure/ba-p/744389 > | > https://techcommunity.microsoft.com/t5/educator-developer-blog/hosting-rserver-and-rstudio-on-azure/ba-p/744389 > ] > but I would like to keep things on Amazon if I can (no great fan of Amazon or > M$ but sometimes I have to swallow my scruples). > > Coming back to AWS I also found: > [ https://blog.martinez.fyi/post/cloud-computing-with-r-and-aws/ | > https://blog.martinez.fyi/post/cloud-computing-with-r-and-aws/ ] > [ https://www.r-bloggers.com/2018/06/interacting-with-aws-from-r/ | > https://www.r-bloggers.com/2018/06/interacting-with-aws-from-r/ ] > and > [ > https://github.com/cloudyr/aws.ec2/commit/7566a353cc92082202f5646c7e1010df10c26dc5 > | > https://github.com/cloudyr/aws.ec2/commit/7566a353cc92082202f5646c7e1010df10c26dc5 > ] > all of which look pertinent and that last package looks as if it might be > another way to go to offload work up to the VM if I can create one. > > However, the first two pages are from 2018 and things in this cloud VM world > look to me to change very, very fast. That aws.ec2 package looks to be in an > orphaned or semi-orphaned state. In all, I thought I would ask here to see if > anyone has any fairly current advice about creating an AWS VM on which to run > R (with the luxury of being able to upscale it, subject to my pocket money, > as my needs may grow). > > TIA, > > Chris > > -- > Small contribution in our coronavirus rigours: > https://www.coresystemtrust.org.uk/home/free-options-to-replace-paper-core-forms-during-the-coronavirus-pandemic/ >
[R] Some R code works on Linux, but not Linux via Windows Subsystem Linux
RE: Some R code works on Linux, but not Linux via Windows Subsystem Linux This is taking data from a CSV and placing it into a data frame. This is R 3.6.3 inside Windows Subsystem for Linux v2, Ubuntu 18.04. The exact same code, unchanged and on the same computer, works correctly in Ubuntu 18.04 and other Linux systems directly if the computer is dual booted into one of those rather than Windows. Error in FUN(X[[i]], …) : only defined on a data frame with all numeric variables Calls: Summary.data.frame -> lapply -> FUN Any idea why the FUN function would error on Windows Subsytem for Linux, but not Linux itself? Any insight into the basic mechanism of how that could vary between systems? Haven't yet checked to see if the data is even getting imported via WSL. The script runs using Rscript as opposed to running interactively via the R console. Robert D. Knight, MBA Developer of Meal Plan and Grocery List maker for Android and iOS. https://play.google.com/store/apps/details?id=io.robertknight.MPGL https://itunes.apple.com/us/app/meal-plan-and-grocery-list/id1452755707 [[alternative HTML version deleted]] __ R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Error trapping in R
Some use try blocks, like found in other languages. Put the code you want to try inside the block. https://www.robertknight.io/blog/try-blocks-in-r-for-error-handling/ contains a quick example. The example doesn’t raise exceptions or anything, it just contains it for you so the script keeps going. I like handling errors with if statements inside of try blocks. Robert > On Feb 27, 2019, at 2:55 PM, Bernard Comcast > wrote: > > What is the recommended way to trap errors in R? My main need is to be able > to trap an error and then skip a section of code if an error has occurred. In > VB for Excel I used the “On Error goto .” construct to do this. > > Bernard > Sent from my iPhone so please excuse the spelling!" > __ > R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code. [[alternative HTML version deleted]] __ R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
