Thank you very much. This is incredibly helpful, I just added an R package
and put a bunch of code in it which works very well. I just had a quick
follow-up question.
Suppose across the uncurated data-sets, stage of cancer progression is
entered in the following way, where the column headers are all different
names:
Example Data-set 1 Example Data-set 2 Example Data-set
3
Tumor stage: T3b pt stage: 3 stage: I
Tumor stage: T2b pt stage: 2 stage:
II
Tumor stage: T3a pt stage: 1 stage:
IV
Tumor stage: T4c pt stage: 4 stage:
IV
What would the code in the R package I'm creating look like such that the
corresponding code in each of these R scripts could be:
curated$stage <- curate_stage (uncurated$characteristics_ch1.5[or whatever
the column header is called])
That is, what code in the function curate_stage() would make it such that
the output would be:
Stage Output Data-set 1 Stage Output Data-set 2 Stage Output
Data-set 3
3
3 1
2
2 2
3
1 4
4
4 4
Your helpful advice from before was that:
curate <- function(characteristic, word="grade: ") {
tmp <- sub(word, "", characteristic, fixed=TRUE)
tmp[tmp=="I"] <- "low"
tmp[tmp=="II"] <- "low"
tmp[tmp=="III"] <- "high"
tmp
}
I'm wondering about situations when it's not the same exact wording (ie I
can't just find "grade: " and take the value after it), but is very similar
(ie when the word "stage" appears in the row but is preceded and followed by
distinct words) (ie "Tumor stage: ", "pt stage: ", and "stage").
Thanks in advance-- you guys are saving me countless hours of repetitive
coding, allowing me to move onto the more important and interesting parts of
the study. I really appreciate all your suggestions and help.
Sincerely,
Ben
On Tue, Jun 7, 2011 at 12:53 PM, Duncan Murdoch <[email protected]>wrote:
> On 07/06/2011 12:41 PM, Ben Ganzfried wrote:
>
>> Hi,
>>
>> My project is set up the following way:
>> root directory contains the following folders:
>> folders: "Breast_Cancer" AND "Colorectal_Cancer" AND "Lung_Cancer" AND
>> "Prostate_Cancer"
>>
>> I want to create a file, call it: "repeating_functions.R" and place it in
>> the root directory such that I can call these functions from within the
>> sub-folders in each type of cancer. My confusion is that I'm not sure of
>> the syntax to make this happen. For example:
>>
>> Within the "Prostate_Cancer" folder, I have the following folders:
>> "curated" AND "src" AND "uncurated"
>>
>> Within "uncurated" I have a ton of files, one of which could be:
>> PMID5377_fullpdata.csv
>>
>> within "src" I have my R scripts, the one corresponding to the above
>> "uncurated" file would be:
>> PMID5377_curation.R
>>
>> Here's the problem I'm trying to address:
>> Many of the uncurated files will require the same R code to curate them
>> and
>> I find myself spending a lot of time copying and pasting the same code
>> over
>> and over. I've spent at least 40 hours copying code I've already written
>> and
>> pasting it into a new dataset. There has simply got to be a better way to
>> do this.
>>
>
> There is: you should put your common functions in a package. Packages are
> a good way to organize your own code, you don't need to publish them. (You
> will get a warning if you put "Not for distribution" into the License field
> in the DESCRIPTION file, but it's just a warning.) You can also put
> datasets in a package; this makes sense if they are relatively static. If
> you get new data every day you probably wouldn't.
>
> A common example of the code I'll write in an "uncurated" file is the
>> following (let's call the following snippet of code UNCURATED_EXAMPLE1):
>> ##characteristics_ch1.2 -> G
>> tmp<- uncurated$characteristics_ch1.2
>> tmp<- sub("grade: ","",tmp,fixed=TRUE)
>> tmp[tmp=="I"]<- "low"
>> tmp[tmp=="II"]<- "low"
>> tmp[tmp=="III"]<- "high"
>> curated$G<- tmp
>>
>> The thing that changes depending on the dataset is *typically* the column
>> header (ie "uncurated$characteristics_ch1.2" might be
>> "uncurated$description" or "uncurated_characteristics_ch1.7" depending on
>> the dataset), although sometimes I want to substitute different words (ie
>> "grade" can be referred to in many different ways).
>>
>> What's the easiest way to automate this? I'd like, at a minimum, to make
>> UNCURATED_EXAMPLE1 look like the following:
>> tmp<- uncurated$characteristics_ch1.2
>> insert_call_to_repeating_functions.R_and_access_("grade")_function
>> curated$G<- tmp
>>
>> It would be even better if I could say, for Prostate_Cancer, write one R
>> script that standardizes all the "uncurated" datasets; rather than writing
>> 100 different R scripts. Although I don't know how feasible this is.
>>
>
> Both of those sound very easy. For example,
>
> curate <- function(characteristic, word="grade: ") {
> tmp <- sub(word, "", characteristic, fixed=TRUE)
>
> tmp[tmp=="I"] <- "low"
> tmp[tmp=="II"] <- "low"
> tmp[tmp=="III"] <- "high"
> tmp
> }
>
> Then your script would just need one line
>
> curated$G <- curate(uncurated$characteristics_ch1.2)
>
> I don't know where you'll find the names of all the datasets, but if you
> can get them into a vector, it's pretty easy to write a loop that calls
> curate() for each one.
>
> Deciding how much goes in the package and how much is one-off code that
> stays with a particular dataset is a judgment call. I'd guess based on your
> description that curate() belongs in the package but the rest doesn't, but
> you know a lot more about the details than I do.
>
> Duncan Murdoch
>
>> I'm sorry if this sounds confusing. Basically, I have thousands of
>> "uncurated" datasets with clinical information and I'm trying to
>> standardize
>> all the datasets via R scripts so that all the information is standardized
>> for statistical analysis. Not all of the datasets contain the same
>> information, but many of them do contain similar data (ie age, stage,
>> grade,
>> days_to_recurrence, and many others). Furthermore, in many cases the
>> standardization code is very similar across datasets (ie I'll want to
>> delete
>> the words "Age: " before the actual number). But this is not always the
>> case (ie sometimes a dataset will not put the different patient data (ie
>> age, stage, grade) in separate columns, instead putting it all in one
>> column, so I have to write a different function to split it by the ";" and
>> make a new table that is separated by column). Anyway, I would be forever
>> grateful for any advice to make this quicker and am happy to provide any
>> clarifications.
>>
>> Thank you very much.
>>
>> Ben
>>
>> [[alternative HTML version deleted]]
>>
>> ______________________________________________
>> [email protected] mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-help
>> PLEASE do read the posting guide
>> http://www.R-project.org/posting-guide.html
>> and provide commented, minimal, self-contained, reproducible code.
>>
>
>
[[alternative HTML version deleted]]
______________________________________________
[email protected] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
