Oh, I see.
Actually I forgot to check whether if could be an advantage to use the
URLencoding () function on queryUrl. I'll check later.
Br. Frede
Sendt fra Samsung mobil
-------- Oprindelig meddelelse --------
Fra: arun
Dato:12/04/2014 12.45 (GMT+01:00)
Til: Frede Aakmann Tøgersen
Cc: Eliza Botto ,"R. Help"
Emne: Re: [R] calling in inverted commas
HI,
Please ignore the previous message. I copied your codes directly from the
email. For some reason, the urlPattern <- ... showed some special characters.
I manually fixed it and now it is working.
urlPattern1<-("http://disc2.nascom.nasa.gov/daac-bin/Giovanni/tovas/Giovanni_cgi.pl?west=%s&north=%s&east=%s&south=%s¶ms=0%%7C3B42_V7&plot_type=Time+Plot&byr=1998&bmo=01&bdy=1&eyr=2007&emo=12&edy=31&begin_date=1998%%2F01%%2F01&end_date=2014%%2F01%%2F31&cbar=cdyn&cmin=&cmax=&yaxis=ydyn&ymin=&ymax=&yint=&ascres=0.25x0.25&global_cfg=tovas.global.cfg.pl&instance_id=TRMM_V7&prod_id=3B42_daily&action=ASCII+Output";)
fileDestination <- getwd()
fileNames <- paste("precip", df2[,1], df2[,2], sep = "_")
fileNames <- paste(fileNames, "txt", sep = ".")
files <- file.path(fileDestination, fileNames)
for (i in 1:nrow(df2)){
queryUrl <- sprintf(urlPattern1, df2[i, 1], df2[i, 2], df2[i, 1], df2[i, 2])
download.file(queryUrl, files[i])
}
## import data in first file
precip <- read.table(files[1], skip = 4, header = TRUE, na.strings = "-9999.9",
sep = "", check.names = FALSE, stringsAsFactors = FALSE)
head(precip,2)
# Time(year:month:day) AccRain
#1 1998:01:01 0
#2 1998:01:02 0
A.K.
On , arun <smartpink...@yahoo.com> wrote:
HI,
Thanks for the link.
I should have used ?sprintf(). BTW, I am not able to reproduce your results.
urlPattern <-
"http://disc2.nascom.nasa.gov/daac-bin/Giovanni/tovas/Giovanni_cgi.pl?west=%s&north=%s&east=%s&south=%¶ms=0|3B42_V7&plot_type=Time+Plot&byr=1998&bmo=01&bdy=1&eyr=2007&emo=12&edy=31&begin_date=1998%%2F01%%2F01&end_date=2014%%2F01%%2F31&cbar=cdyn&cmin=&cmax=&yaxis=ydyn&ymin=&ymax=&yint=&ascres=0.25x0.25&global_cfg=tovas.global.cfg.pl&instance_id=TRMM_V7â_id=3B42_daily&action=ASCII+Output"
##
## some coordinates
df2 <- data.frame(Longitude = c(68.25, 68.75, 69.25), Latitude = c(24.75,
25.25, 25.75))
fileDestination <- getwd()
fileNames <- paste("precip", df2[,1], df2[,2], sep = "_")
fileNames <- paste(fileNames, "txt", sep = ".")
files <- file.path(fileDestination, fileNames)
for (i in 1:nrow(df2)){
queryUrl <- sprintf(urlPattern, df2[i, 1], df2[i, 2], df2[i, 1], df2[i,
2]) download.file(queryUrl, files[i])
}
precip <- read.table(files[1], skip = 4, header = TRUE, na.strings = "-9999.9",
+ sep = "", check.names = FALSE, stringsAsFactors = FALSE)
Error in read.table(files[1], skip = 4, header = TRUE, na.strings = "-9999.9",
:
more columns than column names
So, I checked the file "precip_68.25_24.75.txt"
Giovanni Error Message
Giovanni Error Message Page
Error: instance configuration file is not found.
Error: product configuration file is not found.
Please send email to Help Desk: h...@daac.gs
fc.nasa.gov
sessionInfo()
R version 3.0.2 (2013-09-25)
Platform: x86_64-unknown-linux-gnu (64-bit)
A.K.
On Saturday, April 12, 2014 3:24 AM, Frede Aakmann Tøgersen <fr...@vestas.com>
wrote:
Hi
Sigh I'm getting a headache seeing ugly formatted R code. Arun, your code is
almost unreadable. Have a look at e.g.
http://yihui.name/en/2010/04/formatr-farewell-to-ugly-r-code/
Now to the substantial. Why not use the sprintf() function for formatting the
url instead of the more involved approach using several gsub and regular
expressions that not many people (including myself) easily understand.
## here is a small example using sprintf(), see ?sprintf
## %s is format specifier for string
exampleStr <- "west=%s&north=%s&east=%s&south=%s"
sprintf(exampleStr, 1, 2, 3, 4)
## > [1] "west=1&north=2&east=3&south=4"
## since % is used in format specification then if a literal % is needed in the
string as here
## where you have e.g. %2F01% then escape those % with a %, i.e. % becomes %%
in the string
## I have done that in urlPattern:
urlPattern <-
"http://disc2.nascom.nasa.gov/daac-bin/Giovanni/tovas/Giovanni_cgi.pl?west=%s&north=%s&east=%s&south=%s¶ms=0|3B42_V7&plot_type=Time+Plot&byr=1998&bmo=01&bdy=1&eyr=2007&emo=12&edy=31&begin_date=1998%%2F01%%2F01&end_date=2014%%2F01%%2F31&cbar=cdyn&cmin=&cmax=&yaxis=ydyn&ymin=&ymax=&yint=&ascres=0.25x0.25&global_cfg=tovas.global.cfg.pl&instance_id=TRMM_V7â_id=3B42_daily&action=ASCII+Output"
## some coordinates
df2 <- data.frame(Longitude = c(68.25, 68.75, 69.25), Latitude = c(24.75,
25.25, 25.75))
fileDestination <- c("C:/Temp")
fileNames <- paste("precip", df2[,1], df2[,2], sep = "_")
fileNames <- paste(fileNames, "txt", sep = ".")
files <- file.path(fileDestination, fileNames)
for (i in 1:nrow(df2)){
queryUrl <- sprintf(urlPattern, df2[i, 1], df2[i, 2], df2[i, 1], df2[i, 2])
download.file(queryUrl, files[i])
}
## import data in first file
precip <- read.table(files[1], skip = 4, header = TRUE, na.strings = "-9999.9",
sep = "", check.names = FALSE, stringsAsFactors = FALSE)
head(precip)
Here is another way using the url() function instead of download.file()
## Escaping the trouble with saving and reading files one can also do it this
way
## having the dataframes in a named list
precipList <- vector("list", nrow(df2))
names(precipList) <- fileNames
for (i in 1:nrow(df2)){
queryUrl <- sprintf(urlPattern, df2[i, 1], df2[i, 2], df2[i, 1], df2[i, 2])
u <- url(queryUrl, open = "r")
precipList[[i]] <- read.table(u, skip = 4, header = TRUE, na.strings =
"-9999.9",
sep = "", check.names = FALSE, stringsAsFactors = FALSE)
close(u)
}
str(precipList)
Have a nice day.
Yours sincerely / Med venlig hilsen
Frede Aakmann Tøgersen
Specialist, M.Sc., Ph.D.
Plant Performance & Modeling
Technology & Service Solutions
T +45 9730 5135
M +45 2547 6050
fr...@vestas.com
http://www.vestas.com
Company reg. name: Vestas Wind Systems A/S
This e-mail is subject to our e-mail disclaimer statement.
Please refer to www.vestas.com/legal/notice<http://www.vestas.com/legal/notice>
If you have received this e-mail in error please contact the sender.
> -----Original Message-----
> From: r-help-boun...@r-project.org [mailto:r-help-boun...@r-project.org]
> On Behalf Of arun
> Sent: 12. april 2014 07:16
> To: r-help@r-project.org
> Subject: Re: [R] calling in inverted commas
>
> Hi,
> I noticed some special characters after sending.
> It should be:
> toreplace <- gsub(".*\\.pl\\?(west\\=.*south=.*)\\¶ms.*","\\1",Url1)
> end <- gsub(".*south=.*(\\¶ms.*)","\\1",Url1)
>
>
> A.K.
>
>
> On Saturday, April 12, 2014 1:06 AM, arun <smartpink...@yahoo.com>
> wrote:
> HI,
>
> Not sure if this helps:
> df2 <- data.frame(Col1=c(68.25, 68.75, 69.25), Col2=c(24.75, 25.25, 25.75))
>
>
> Url1 <- "http://disc2.nascom.nasa.gov/daac-
> bin/Giovanni/tovas/Giovanni_cgi.pl?west=68.25&north=24.75&east=68.25&s
> outh=24.75¶ms=0%7C3B42_V7&plot_type=Time+Plot&byr=1998&bmo=01&
> bdy=1&eyr=2007&emo=12&edy=31&begin_date=1998%2F01%2F01&end_da
> te=2014%2F01%2F31&cbar=cdyn&cmin=&cmax=&yaxis=ydyn&ymin=&ymax
> =&yint=&ascres=0.25x0.25&global_cfg=tovas.global.cfg.pl&instance_id=TRM
> M_V7â_id=3B42_daily&action=ASCII+Output"
>
>
> toreplace <- gsub(".*\\.pl\\?(west\\=.*south=.*)\\¶ms.*","\\1",Url1)
> begin <- gsub("(.*\\.pl\\?).*","\\1",Url1)
> end <- gsub(".*south=.*(\\¶ms.*)","\\1",Url1)
>
> vec2 <- strsplit(toreplace,"&")[[1]]
>
> ##replace ".csv" with ".txt" and use write.table() if you need as text files.
>
>
> lapply(seq_len(nrow(df2)),function(i) { val1 <-
> as.vector(rep(unlist(df2[i,]),2));replaced <-
> do.call(paste,list(lapply(seq_along(vec2),function(i) gsub("[-
> +]?(\\d*[.])?\\d+",val1[i],vec2[i])),collapse="&")); UrlNew <-
> paste0(begin,replaced,end); res <-
> read.csv(UrlNew,header=TRUE,stringsAsFactors=FALSE,skip=4,sep="",check.
> names=FALSE);
> write.csv(res,file=paste0("file_",paste(val1[1:2],collapse="_"),".csv"),row.na
> mes=FALSE,quote=FALSE)})
>
> sapply(list.files(pattern="file_"),function(x) {x1 <-
> read.csv(x,header=TRUE,stringsAsFactors=FALSE,check.names=FALSE);
> dim(x1)[1]})
> #file_68.25_24.75.csv file_68.75_25.25.csv file_69.25_25.75.csv
>
> # 3652 3652 3652
>
> ### reading the downloaded file
>
> lapply(list.files(pattern="file_"),function(x) {x1 <-
> read.csv(x,header=TRUE,stringsAsFactors=FALSE,check.names=FALSE);
> x1[5:7,]})
>
> #[[1]]
>
> # Time(year:month:day) AccRain
>
> #5 1998:01:05 0.000
>
> #6 1998:01:06 0.000
> #7 1998:01:07 0.984
>
> #
>
> #[[2]]
>
> # Time(year:month:day) AccRain
> #5 1998:01:05 0.0000
> #6 1998:01:06 0.0925
> #7 1998:01:07 0.2643
> #
> #[[3]]
>
> # Time(year:month:day) AccRain
> #5 1998:01:05 0.0000
> #6 1998:01:06 0.7043
>
> #7 1998:01:07 0.5340
>
>
>
> A.K.
>
>
> On Friday, April 11, 2014 2:38 PM, eliza botto <eliza_bo...@hotmail.com>
> wrote:
> Dear useRs,
> Here are three steps for downloading a file from a certain website in R. Here
> you see that in "URL" command
> (west=68.25&north=24.75&east=68.25&south=24.75) are actually the first
> and second column values of 1st row of a matrix called df2 (300 rows and 2
> columns). more precisely, df2[1,][1]=68.25, df2[1,][2]=24.75.
> Is there a way that I can make a loop so that remaining rows and columns get
> inculcated in the link as such and then get saved at the desired mentioned
> location in 300 different text files?
> URL<-("http://disc2.nascom.nasa.gov/daac-
> bin/Giovanni/tovas/Giovanni_cgi.pl?west=68.25&north=24.75&east=68.25&s
> outh=24.75¶ms=0%7C3B42_V7&plot_type=Time+Plot&byr=1998&bmo=01&
> bdy=1&eyr=2007&emo=12&edy=31&begin_date=1998%2F01%2F01&end_da
> te=2014%2F01%2F31&cbar=cdyn&cmin=&cmax=&yaxis=ydyn&ymin=&ymax
> =&yint=&ascres=0.25x0.25&global_cfg=tovas.global.cfg.pl&instance_id=TRM
> M_V7â_id=3B42_daily&action=ASCII+Output")
>
> destfile <- "C:\\Users\\Eliza\\Desktop\\AA\\love.txt"
> download.file(URL, destfile)
>
> Any suggestion?
> Thankyou very much in advance,
> Eliza
> [[alternative HTML version deleted]]
>
> ______________________________________________
> R-help@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-
> guide.html
> and provide commented, minimal, self-contained, reproducible code.
>
>
> ______________________________________________
> R-help@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-
> guide.html
> and provide commented, minimal, self-contained, reproducible code.
[[alternative HTML version deleted]]
______________________________________________
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
