Re: [R] download.file strict certificate revocation check
Ivan, SSL connect error & we definitely have MITM doing certificate interference. No change with True or False with R_LIBCURL_SSL_REVOKE_BEST_EFFORT Environment variable results should be attached. -Original Message- From: Ivan Krylov Sent: Wednesday, October 4, 2023 8:52 AM To: John Neset Cc: r-help@R-project.org Subject: Re: [R] download.file strict certificate revocation check WARNING: This is an external email. Do not click links or open attachments unless you recognize the sender and know the content is safe. В Wed, 4 Oct 2023 13:09:47 + John Neset пишет: > Trying to do this, reference FAQ- > 2.18 The Internet download functions fail. > (c) A MITM proxy (typically in enterprise environments) makes it > impossible to validate that certificates haven't been revoked. One can > switch to only best effort revocation checks via an environment > variable: see ?download.file. Here's what help(download.file) has to say: >> On Windows with ‘method = "libcurl"’, when R was linked with >> ‘libcurl’ with ‘Schannel’ enabled, the connection fails if it >> cannot be established that the certificate has not been revoked. >> Some MITM proxies present particularly in corporate environments >> do not work with this behavior. It can be changed by setting >> environment variable ‘R_LIBCURL_SSL_REVOKE_BEST_EFFORT’ to >> ‘TRUE’, with the consequence of reducing security. Does it help to Sys.setenv(...) this environment variable before downloading? If not, please provide your sessionInfo() and the full error message. -- Best regards, Ivan Confidentiality Notice - This communication and any attachments are for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure, distribution or copying is prohibited. If you are not the intended recipient(s), please contact the sender by replying to this e-mail and destroy/delete all copies of this e-mail message. __ R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] download.file strict certificate revocation check
This is sad news indeed.
https://cran.r-project.org/web/checks/check_summary_by_maintainer.html
lists Jim as the maintainer of clinsig, crank, eventInterval,
plotrix, and prettyR.
> library(packageRank)
> packageRank(c("clinsig", "crank", "eventInterval", "plotrix", "prettyR"))
date packages downloads rank percentile
1 2023-10-03 clinsig 1 14,454 of 18,0240.0
2 2023-10-03 crank 2 11,344 of 18,024 22.8
3 2023-10-03 eventInterval 4 8,001 of 18,024 51.0
4 2023-10-03 plotrix 3,082310 of 18,024 98.3
5 2023-10-03 prettyR90 1,954 of 18,024 89.1
It seems that at least plotrix and prettyR would be worth rescuing ...
volunteers ... ? (prettyR has 1 strong reverse dep, plotrix has many ...)
Ben Bolker
On 2023-10-04 6:30 p.m., Jim Lemon wrote:
Hello,
I am very sad to let you know that my husband Jim died in 18th September. I
apologise for not letting you know earlier but I had trouble finding the
password for his phone.
Kind regards,
Juel Briggs
On Thu, 5 Oct 2023, 02:07 Ivan Krylov
В Wed, 4 Oct 2023 14:32:49 +
John Neset пишет:
No change with True or False with R_LIBCURL_SSL_REVOKE_BEST_EFFORT
Judging by the screenshot, it looks like you've set an R variable
R_LIBCURL_SSL_REVOKE_BEST_EFFORT instead of setting an environment
variable using Sys.setenv:
Sys.setenv('R_LIBCURL_SSL_REVOKE_BEST_EFFORT' = 'TRUE')
(Use Sys.getenv to verify the result.)
For the next time, most people on the R-help mailing list would
probably appreciate it if you copied and pasted the text from the R
console instead of attaching screenshots.
--
Best regards,
Ivan
__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
[[alternative HTML version deleted]]
__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Re: [R] download.file strict certificate revocation check
Hello,
I am very sad to let you know that my husband Jim died in 18th September. I
apologise for not letting you know earlier but I had trouble finding the
password for his phone.
Kind regards,
Juel Briggs
On Thu, 5 Oct 2023, 02:07 Ivan Krylov В Wed, 4 Oct 2023 14:32:49 +
> John Neset пишет:
>
> > No change with True or False with R_LIBCURL_SSL_REVOKE_BEST_EFFORT
>
> Judging by the screenshot, it looks like you've set an R variable
> R_LIBCURL_SSL_REVOKE_BEST_EFFORT instead of setting an environment
> variable using Sys.setenv:
>
> Sys.setenv('R_LIBCURL_SSL_REVOKE_BEST_EFFORT' = 'TRUE')
>
> (Use Sys.getenv to verify the result.)
>
> For the next time, most people on the R-help mailing list would
> probably appreciate it if you copied and pasted the text from the R
> console instead of attaching screenshots.
>
> --
> Best regards,
> Ivan
>
> __
> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
> http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>
[[alternative HTML version deleted]]
__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Re: [R] download.file strict certificate revocation check
В Wed, 4 Oct 2023 14:32:49 +
John Neset пишет:
> No change with True or False with R_LIBCURL_SSL_REVOKE_BEST_EFFORT
Judging by the screenshot, it looks like you've set an R variable
R_LIBCURL_SSL_REVOKE_BEST_EFFORT instead of setting an environment
variable using Sys.setenv:
Sys.setenv('R_LIBCURL_SSL_REVOKE_BEST_EFFORT' = 'TRUE')
(Use Sys.getenv to verify the result.)
For the next time, most people on the R-help mailing list would
probably appreciate it if you copied and pasted the text from the R
console instead of attaching screenshots.
--
Best regards,
Ivan
__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Re: [R] download.file strict certificate revocation check
В Wed, 4 Oct 2023 13:09:47 + John Neset пишет: > Trying to do this, reference FAQ- > 2.18 The Internet download functions fail. > (c) A MITM proxy (typically in enterprise environments) makes it > impossible to validate that certificates haven't been revoked. One > can switch to only best effort revocation checks via an environment > variable: see ?download.file. Here's what help(download.file) has to say: >> On Windows with ‘method = "libcurl"’, when R was linked with >> ‘libcurl’ with ‘Schannel’ enabled, the connection fails if it >> cannot be established that the certificate has not been revoked. >> Some MITM proxies present particularly in corporate environments >> do not work with this behavior. It can be changed by setting >> environment variable ‘R_LIBCURL_SSL_REVOKE_BEST_EFFORT’ to >> ‘TRUE’, with the consequence of reducing security. Does it help to Sys.setenv(...) this environment variable before downloading? If not, please provide your sessionInfo() and the full error message. -- Best regards, Ivan __ R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
[R] download.file strict certificate revocation check
What/how do I interact with the download.file with turning off the strict
certificate revocation check in regards to download & update packages?
I clearly made an attempt at this, but failed miserably.
Trying to do this, reference FAQ-
2.18 The Internet download functions fail.
(c) A MITM proxy (typically in enterprise environments) makes it impossible to
validate that certificates haven't been revoked. One can switch to only best
effort revocation checks via an environment variable: see ?download.file.
Confidentiality Notice - This communication and any atta...{{dropped:10}}
__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
[R] download.file() problems with binary files containing EOF byte in Windows
Hello,
I'm trying to get a package to pass win-builder and have been having a
bit of trouble with Windows R and binary files (in my case a small
.tar.gz used in testing). After a little debugging, I think I've
narrowed it down to download.file() truncating files to the first '1a'
byte (often used for EOF but I think a valid byte inside gzip files)
on downloads from local "file://xxx". I'm trying to figure out if this
is a known "feature" of Windows that I should just avoid or does this
seem like a bug?
For example:
#write a file starting with byte 1a (decimal 26)
writeBin(26:100,'tmp.bin',size=1)
download.file('file://tmp.bin','download.bin')
file.size('tmp.bin')
file.size('download.bin')
On Windows (session info below), I get file sizes of 75 and 0 and on
Linux I get 75 and 75.
As a more real world example, if I download.file() on a .gz file then
a remote download seems to return different size files from a local
download. For example for a gz file from a google hit about gzip
(http://commandlinefanatic.com/cgi-bin/showarticle.cgi?article=art053):
download.file('http://commandlinefanatic.com/gunzip.c.gz','gunzip.c.gz')
download.file('file://gunzip.c.gz','dl.gz')
file.size('gunzip.c.gz')
file.size('dl.gz')
I get a 4704 byte file for the remote download and 360 for the local
download in Windows (versus 4704 and 4704 on Linux). Note that the
361st byte is 1a:
readBin('gunzip.c.gz','raw',361)
The various download.file options don't seem to fix this with the same 360 bytes
for:
download.file('file://gunzip.c.gz','dl.gz',mode='wb')
file.size('dl.gz')
download.file('file://gunzip.c.gz','dl.gz',mode='wb',method='internal')
file.size('dl.gz')
It looks like the 'auto' and 'internal' methods both resolve to the
'wininet' method on Windows and mode is automatically set to 'wb' for
gz files so maybe not surprising those don't change things.
Thanks,
Scott
## Windows sessionInfo():
R version 3.5.1 (2018-07-02)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 8.1 x64 (build 9600)
Matrix products: default
locale:
[1] LC_COLLATE=English_United States.1252
[2] LC_CTYPE=English_United States.1252
[3] LC_MONETARY=English_United States.1252
[4] LC_NUMERIC=C
[5] LC_TIME=English_United States.1252
attached base packages:
[1] stats graphics grDevices utils datasets methods base
loaded via a namespace (and not attached):
[1] compiler_3.5.1
## Linux sessionInfo():
R version 3.4.4 (2018-03-15)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 16.04.5 LTS
Matrix products: default
BLAS: /usr/lib/libblas/libblas.so.3.6.0
LAPACK: /usr/lib/lapack/liblapack.so.3.6.0
locale:
[1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C
[3] LC_TIME=en_US.UTF-8LC_COLLATE=en_US.UTF-8
[5] LC_MONETARY=en_US.UTF-8LC_MESSAGES=en_US.UTF-8
[7] LC_PAPER=en_US.UTF-8 LC_NAME=C
[9] LC_ADDRESS=C LC_TELEPHONE=C
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C
attached base packages:
[1] stats graphics grDevices utils datasets methods base
loaded via a namespace (and not attached):
[1] compiler_3.4.4
__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
[R] download.file: unsuppported url scheme
I have found an interesting issue. I use R Studio for most of my development and automate many tasks to run over night. I have a https file scrape process that runs without issue from R Studio command line. When I try to run from the R console, I get an error from download.file stating unsuppported url scheme. I use both windows and linux. If I add a OS check line and execute setInternet2(T) to my script then the process works from the R console. Did I miss an internet setting in the installation on my windows machines setting internet2 to T? I have a work around, but I found this interesting. Have a great day all! -- *Joe W. Byers* [[alternative HTML version deleted]] __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] download.file error - corrupt: Can't read SAT; charset=binary'
Tal, please quote the whole message! I had to look for the original one now. And then I found that code was not reproducible for me: exe_URL = 'http://pandoc.googlecode.com/files/pandoc-1.11.1.msi' exe_filename - file.path(tempdir(), file.name.from.url(exe_URL)) Error in file.path(tempdir(), file.name.from.url(exe_URL)) : could not find function file.name.from.url and I do not know such a function. From its name I can guess basename(exe_URL) is the base function that does the trick. So please specify full and reproducible code that shows the error, including and does not contain any additional quirks like the mode=w Anyway, the installer works for me downloaded with the method I indicated before. Best, Uwe Ligges On 22.08.2013 07:23, Tal Galili wrote: Dear Uwe, My apologies - my original code had mode=wb, which also *does not* work and produce the errors I've mentioned in the previous e-mail. The code I pasted had mode =w, since that is one of the versions I've played with (which also does not work). With regards, Tal On Wed, Aug 21, 2013 at 11:45 PM, Uwe Ligges lig...@statistik.tu-dortmund.de wrote: Uwe Contact Details:--- Contact me: tal.gal...@gmail.com | Read me: www.talgalili.com (Hebrew) | www.biostatistics.co.il (Hebrew) | www.r-statistics.com (English) -- [[alternative HTML version deleted]] __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] download.file error - corrupt: Can't read SAT; charset=binary'
Dear Uwe, Here is the updated code and error massage I get: exe_URL = 'http://pandoc.googlecode.com/files/pandoc-1.11.1.msi' exe_filename - file.path(tempdir(), basename(exe_URL)) download.file(exe_URL, destfile = exe_filename, mode = wb, method= internal) # ### Error massage: trying URL 'http://pandoc.googlecode.com/files/pandoc-1.11.1.msi' Content type 'Composite Document File V2 Document, corrupt: Can't read SAT; charset=binary' length 5799936 bytes (5.5 Mb) opened URL downloaded 5.5 Mb 1) I am very sorry about the chipped previous massage (it is a side effect of some form of reply in gmail, which I have not noticed before). 2) I'm sorry about using the wrong function name - it comes with the installr package, since I have it running on startup, I was blind to see the code was not standard. Also, now that I know such a function exists in base R, I will move to using it. Best, Tal Contact Details:--- Contact me: tal.gal...@gmail.com | Read me: www.talgalili.com (Hebrew) | www.biostatistics.co.il (Hebrew) | www.r-statistics.com (English) -- On Thu, Aug 22, 2013 at 12:12 PM, Uwe Ligges lig...@statistik.tu-dortmund.de wrote: Tal, please quote the whole message! I had to look for the original one now. And then I found that code was not reproducible for me: exe_URL = 'http://pandoc.googlecode.com/**files/pandoc-1.11.1.msihttp://pandoc.googlecode.com/files/pandoc-1.11.1.msi ' exe_filename - file.path(tempdir(), file.name.from.url(exe_URL)) Error in file.path(tempdir(), file.name.from.url(exe_URL)) : could not find function file.name.from.url and I do not know such a function. From its name I can guess basename(exe_URL) is the base function that does the trick. So please specify full and reproducible code that shows the error, including and does not contain any additional quirks like the mode=w Anyway, the installer works for me downloaded with the method I indicated before. Best, Uwe Ligges On 22.08.2013 07:23, Tal Galili wrote: Dear Uwe, My apologies - my original code had mode=wb, which also *does not* work and produce the errors I've mentioned in the previous e-mail. The code I pasted had mode =w, since that is one of the versions I've played with (which also does not work). With regards, Tal On Wed, Aug 21, 2013 at 11:45 PM, Uwe Ligges lig...@statistik.tu-dortmund.**de lig...@statistik.tu-dortmund.de wrote: Uwe Contact Details:--**--**--- Contact me: tal.gal...@gmail.com | Read me: www.talgalili.com (Hebrew) | www.biostatistics.co.il (Hebrew) | www.r-statistics.com (English) --**--** --** [[alternative HTML version deleted]] __** R-help@r-project.org mailing list https://stat.ethz.ch/mailman/**listinfo/r-helphttps://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/** posting-guide.html http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. [[alternative HTML version deleted]] __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] download.file error - corrupt: Can't read SAT; charset=binary'
Quick update - I think most of the problem is resolved. After more checks (following Uwe comments): 1) I see that the installer file runs properly when I set mode='wb', and fails when mode='w' (so that solves why some of the time the installer didn't run) 2) I still get the error massage mentioned before (e.g: Composite Document File V2 Document, corrupt: Can't read SAT, etc...). I don't know what it means, or if it is important (since the installer now runs properly). Tal Contact Details:--- Contact me: tal.gal...@gmail.com | Read me: www.talgalili.com (Hebrew) | www.biostatistics.co.il (Hebrew) | www.r-statistics.com (English) -- On Thu, Aug 22, 2013 at 8:52 PM, Tal Galili tal.gal...@gmail.com wrote: Dear Uwe, Here is the updated code and error massage I get: exe_URL = 'http://pandoc.googlecode.com/files/pandoc-1.11.1.msi' exe_filename - file.path(tempdir(), basename(exe_URL)) download.file(exe_URL, destfile = exe_filename, mode = wb, method= internal) # ### Error massage: trying URL 'http://pandoc.googlecode.com/files/pandoc-1.11.1.msi' Content type 'Composite Document File V2 Document, corrupt: Can't read SAT; charset=binary' length 5799936 bytes (5.5 Mb) opened URL downloaded 5.5 Mb 1) I am very sorry about the chipped previous massage (it is a side effect of some form of reply in gmail, which I have not noticed before). 2) I'm sorry about using the wrong function name - it comes with the installr package, since I have it running on startup, I was blind to see the code was not standard. Also, now that I know such a function exists in base R, I will move to using it. Best, Tal Contact Details:--- Contact me: tal.gal...@gmail.com | Read me: www.talgalili.com (Hebrew) | www.biostatistics.co.il (Hebrew) | www.r-statistics.com (English) -- On Thu, Aug 22, 2013 at 12:12 PM, Uwe Ligges lig...@statistik.tu-dortmund.de wrote: Tal, please quote the whole message! I had to look for the original one now. And then I found that code was not reproducible for me: exe_URL = 'http://pandoc.googlecode.com/**files/pandoc-1.11.1.msihttp://pandoc.googlecode.com/files/pandoc-1.11.1.msi ' exe_filename - file.path(tempdir(), file.name.from.url(exe_URL)) Error in file.path(tempdir(), file.name.from.url(exe_URL)) : could not find function file.name.from.url and I do not know such a function. From its name I can guess basename(exe_URL) is the base function that does the trick. So please specify full and reproducible code that shows the error, including and does not contain any additional quirks like the mode=w Anyway, the installer works for me downloaded with the method I indicated before. Best, Uwe Ligges On 22.08.2013 07:23, Tal Galili wrote: Dear Uwe, My apologies - my original code had mode=wb, which also *does not* work and produce the errors I've mentioned in the previous e-mail. The code I pasted had mode =w, since that is one of the versions I've played with (which also does not work). With regards, Tal On Wed, Aug 21, 2013 at 11:45 PM, Uwe Ligges lig...@statistik.tu-dortmund.**de lig...@statistik.tu-dortmund.de wrote: Uwe Contact Details:--**--**--- Contact me: tal.gal...@gmail.com | Read me: www.talgalili.com (Hebrew) | www.biostatistics.co.il (Hebrew) | www.r-statistics.com (English) --**--** --** [[alternative HTML version deleted]] __** R-help@r-project.org mailing list https://stat.ethz.ch/mailman/**listinfo/r-helphttps://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/** posting-guide.html http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. [[alternative HTML version deleted]] __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] download.file error - corrupt: Can't read SAT; charset=binary'
On 22.08.2013 19:52, Tal Galili wrote: Dear Uwe, Here is the updated code and error massage I get: exe_URL = 'http://pandoc.googlecode.com/files/pandoc-1.11.1.msi' exe_filename - file.path(tempdir(), basename(exe_URL)) download.file(exe_URL, destfile = exe_filename, mode = wb, method= internal) # ### Error massage: trying URL 'http://pandoc.googlecode.com/files/pandoc-1.11.1.msi' Content type 'Composite Document File V2 Document, corrupt: Can't read SAT; charset=binary' length 5799936 bytes (5.5 Mb) opened URL downloaded 5.5 Mb I still don't see a problem when executing the installer, it is just a message that the content type cannot be read appropriately. Best, Uwe Ligges 1) I am very sorry about the chipped previous massage (it is a side effect of some form of reply in gmail, which I have not noticed before). 2) I'm sorry about using the wrong function name - it comes with the installr package, since I have it running on startup, I was blind to see the code was not standard. Also, now that I know such a function exists in base R, I will move to using it. Best, Tal Contact Details:--- Contact me: tal.gal...@gmail.com mailto:tal.gal...@gmail.com | Read me: www.talgalili.com http://www.talgalili.com (Hebrew) | www.biostatistics.co.il http://www.biostatistics.co.il (Hebrew) | www.r-statistics.com http://www.r-statistics.com (English) -- On Thu, Aug 22, 2013 at 12:12 PM, Uwe Ligges lig...@statistik.tu-dortmund.de mailto:lig...@statistik.tu-dortmund.de wrote: Tal, please quote the whole message! I had to look for the original one now. And then I found that code was not reproducible for me: exe_URL = 'http://pandoc.googlecode.com/__files/pandoc-1.11.1.msi http://pandoc.googlecode.com/files/pandoc-1.11.1.msi' exe_filename - file.path(tempdir(), file.name.from.url(exe_URL)) Error in file.path(tempdir(), file.name.from.url(exe_URL)) : could not find function file.name.from.url and I do not know such a function. From its name I can guess basename(exe_URL) is the base function that does the trick. So please specify full and reproducible code that shows the error, including and does not contain any additional quirks like the mode=w Anyway, the installer works for me downloaded with the method I indicated before. Best, Uwe Ligges On 22.08.2013 07:23, Tal Galili wrote: Dear Uwe, My apologies - my original code had mode=wb, which also *does not* work and produce the errors I've mentioned in the previous e-mail. The code I pasted had mode =w, since that is one of the versions I've played with (which also does not work). With regards, Tal On Wed, Aug 21, 2013 at 11:45 PM, Uwe Ligges lig...@statistik.tu-dortmund.__de mailto:lig...@statistik.tu-dortmund.de wrote: Uwe Contact Details:--__--__--- Contact me: tal.gal...@gmail.com mailto:tal.gal...@gmail.com | Read me: www.talgalili.com http://www.talgalili.com (Hebrew) | www.biostatistics.co.il http://www.biostatistics.co.il (Hebrew) | www.r-statistics.com http://www.r-statistics.com (English) --__--__--__ [[alternative HTML version deleted]] R-help@r-project.org mailto:R-help@r-project.org mailing list https://stat.ethz.ch/mailman/__listinfo/r-help https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/__posting-guide.html http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
[R] download.file error - corrupt: Can't read SAT; charset=binary'
Dear R-help mailing list memebers, I encountered the following error, and I'd be happy for ideas on how to fix it: # using R 3.0.1 on Windows 7: exe_URL = 'http://pandoc.googlecode.com/files/pandoc-1.11.1.msi' exe_filename - file.path(tempdir(), file.name.from.url(exe_URL)) download.file(exe_URL, destfile = exe_filename, mode = w, method= internal) ### output error : trying URL 'http://pandoc.googlecode.com/files/pandoc-1.11.1.msi' Content type 'Composite Document File V2 Document, corrupt: Can't read SAT; charset=binary' length 5799936 bytes (5.5 Mb) opened URL downloaded 5.5 Mb ### another error: And then when I try to execute the file (it is a windows installer), I get the error: This installation package could not be opened. Contact the application vendor to verify that this is a valid Windows installer package. Downloading the file manually and running it does work. Any suggestions? Thanks. Contact Details:--- Contact me: tal.gal...@gmail.com | Read me: www.talgalili.com (Hebrew) | www.biostatistics.co.il (Hebrew) | www.r-statistics.com (English) -- [[alternative HTML version deleted]] __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] download.file error - corrupt: Can't read SAT; charset=binary'
On 21.08.2013 22:35, Tal Galili wrote: Dear R-help mailing list memebers, I encountered the following error, and I'd be happy for ideas on how to fix it: # using R 3.0.1 on Windows 7: exe_URL = 'http://pandoc.googlecode.com/files/pandoc-1.11.1.msi' exe_filename - file.path(tempdir(), file.name.from.url(exe_URL)) download.file(exe_URL, destfile = exe_filename, mode = w, method= internal) ### output error : trying URL 'http://pandoc.googlecode.com/files/pandoc-1.11.1.msi' Content type 'Composite Document File V2 Document, corrupt: Can't read SAT; charset=binary' length 5799936 bytes (5.5 Mb) opened URL downloaded 5.5 Mb ### another error: And then when I try to execute the file (it is a windows installer), I get the error: This installation package could not be opened. Contact the application vendor to verify that this is a valid Windows installer package. Downloading the file manually and running it does work. Any suggestions? Use mode=wb, it is a binary file! Uwe Ligges Thanks. Contact Details:--- Contact me: tal.gal...@gmail.com | Read me: www.talgalili.com (Hebrew) | www.biostatistics.co.il (Hebrew) | www.r-statistics.com (English) -- [[alternative HTML version deleted]] __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] download.file error - corrupt: Can't read SAT; charset=binary'
Dear Uwe, My apologies - my original code had mode=wb, which also *does not* work and produce the errors I've mentioned in the previous e-mail. The code I pasted had mode =w, since that is one of the versions I've played with (which also does not work). With regards, Tal On Wed, Aug 21, 2013 at 11:45 PM, Uwe Ligges lig...@statistik.tu-dortmund.de wrote: Uwe Contact Details:--- Contact me: tal.gal...@gmail.com | Read me: www.talgalili.com (Hebrew) | www.biostatistics.co.il (Hebrew) | www.r-statistics.com (English) -- [[alternative HTML version deleted]] __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
[R] download.file
I am downloading say 100 files from ucsc website and storing it into dest
folder.
download.file function create a file in destination folder even if the file
is not present which is something I dont want.
So I wrote if condition to remove the file if the download function has non
zero value.
Now it exits when there is an error or file not present. How can I use
try and if condition together so that the program does not exit on
error and delete the created file in destination folder.
for (i in 1: 100)
{
fileUrl = ucscfilenames[i]
if (download.file(fileUrl, destFile, 'wget' , quiet = TRUE) != 0)
{
file.remove(destFile)
}
}
thanks
--
-
Mary Kindall
Yorktown Heights, NY
USA
[[alternative HTML version deleted]]
__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Re: [R] download.file
On Nov 8, 2011, at 6:32 PM, Mary Kindall wrote:
I am downloading say 100 files from ucsc website and storing it into
dest
folder.
download.file function create a file in destination folder even if
the file
is not present which is something I dont want.
So I wrote if condition to remove the file if the download function
has non
zero value.
Now it exits when there is an error or file not present. How can I use
try and if condition together so that the program does not exit on
error and delete the created file in destination folder.
for (i in 1: 100)
{
fileUrl = ucscfilenames[i]
if (download.file(fileUrl, destFile, 'wget' , quiet = TRUE) != 0)
{
file.remove(destFile)
}
}
I do not see where destFile gets its values. I don't think you can
just give it a path. And why would you want to remove a file that was
never successfully created, anyway?
Wouldn't it be something like (pseudo-) :
define a vector of destination file names
if( try(download.file(fileUrl, destFile[i], 'wget' , quiet = TRUE) ) ) {
write to error log } else {
write to success log }
--
David Winsemius, MD
West Hartford, CT
__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Re: [R] download.file
See downloadFile() of the R.utils package (on CRAN);
it allows you to download to a file to a given path (without have to
replicate the filename), and it also have an option (the default) to
drop zero-size files that are (sometimes) created when download.file()
tries to download a non-existing file. Moreover, the method downloads
files in binary mode by default (download in text mode is a common
cause for corrupt files). See help(downloadFile,
package=R.utils) for more bells and whistles.
/Henrik
On Tue, Nov 8, 2011 at 3:32 PM, Mary Kindall mary.kind...@gmail.com wrote:
I am downloading say 100 files from ucsc website and storing it into dest
folder.
download.file function create a file in destination folder even if the file
is not present which is something I dont want.
So I wrote if condition to remove the file if the download function has non
zero value.
Now it exits when there is an error or file not present. How can I use
try and if condition together so that the program does not exit on
error and delete the created file in destination folder.
for (i in 1: 100)
{
fileUrl = ucscfilenames[i]
if (download.file(fileUrl, destFile, 'wget' , quiet = TRUE) != 0)
{
file.remove(destFile)
}
}
thanks
--
-
Mary Kindall
Yorktown Heights, NY
USA
[[alternative HTML version deleted]]
__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
[R] Download.file problem
Strange problem with download.file . for non existent URL an empty file is created but I am not able to delete the without shutting down R Example: download.file(http://test.com/test.txt,test.txt;) trying URL 'http://test.com/test.txt' Error in download.file(http://test.com/test.txt;, test.txt) : cannot open URL 'http://test.com/test.txt' In addition: Warning message: In download.file(http://test.com/test.txt;, test.txt) : cannot open: HTTP status was '404 Not Found' If you go to working directory through windows explorer, you can see the empty file test.txt but try deleteting the file and it says that the file is locked. I tried closeAllConnections() but of no use. Any suggestions? Thanks, S __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
[R] download.file() help! setting the proxy for user /passw0rd
I would like to download climate data files from PCMDI website using R. I
tried this line below and I was not able to get the file mainly due to user
name and password requirements. I am looking for help for setting up the
user and password within R (or somewhere). I have read the FAQ but
unfortunately I am a newbie on R and couldnt figure out how to do it. Many
thanks in advance
download.file('ftp://ftp-esg.ucllnl.org/ipcc/20c3m/land/mo/pr//run1/pr_A1_1.nc',
destfile=clt_A1.nc, method=wget, quiet = FALSE, mode = w,cacheOK=TRUE)
--2009-07-22 19:20:12--
ftp://ftp-esg.ucllnl.org/ipcc/20c3m/land/mo/pr//run1/pr_A1_1.nc
= `clt_A1.nc'
Resolving ftp-esg.ucllnl.org... 192.12.137.5
Connecting to ftp-esg.ucllnl.org|192.12.137.5|:21... connected.
Logging in as anonymous ...
Login incorrect.
Warning message:
In
download.file(ftp://ftp-esg.ucllnl.org/ipcc/20c3m/land/mo/pr//run1/pr_A1_1.nc;,
:
download had nonzero exit status
--
View this message in context:
http://www.nabble.com/download.file%28%29-help%21-setting-the-proxy-for-user--passw0rd-tp24610964p24610964.html
Sent from the R help mailing list archive at Nabble.com.
__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Re: [R] download.file error
Hi Chib whether there is a way of handling the error you get in the download.file() function I think the function try will do this for you. That is, handle the error. Have a look at ?try. Regards John Seers --- -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of CHIB CO Sent: 01 April 2008 15:28 To: jim holtman Cc: r-help@r-project.org Subject: Re: [R] download.file error Hi Jim, I just wanted to know if there is a function in R that can tell you whether a file on the internet exists before you attempt to download it or whether there is a way of handling the error you get in the download.file() function without having it break a for loop with the download error when it is part of that loop. Kind Regards Chib __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] download.file error
Hi Jim, I just wanted to know if there is a function in R that can tell you whether a file on the internet exists before you attempt to download it or whether there is a way of handling the error you get in the download.file() function without having it break a for loop with the download error when it is part of that loop. Kind Regards Chib Date: Mon, 31 Mar 2008 18:33:35 -0500 From: [EMAIL PROTECTED] To: [EMAIL PROTECTED] Subject: Re: [R] download.file error CC: r-help@r-project.org ?try On Mon, Mar 31, 2008 at 6:26 PM, CHIB CO [EMAIL PROTECTED] wrote: Dear all, I am looking for a way to work out if a file on the internet exists before attempting to download it using the function download.file(). For example, using a url that does not exist url - http://finance.yahoo.com/ftse.csv; destfile - tempfile() download.file(url = url, destfile = destfile) # gives the following response ... trying URL 'http://finance.yahoo.com/ftse.csv' Error in download.file(url = url, destfile = destfile) : cannot open URL 'http://finance.yahoo.com/ftse.csv' In addition: Warning message: In download.file(url = url, destfile = destfile) : cannot open: HTTP status was '404 Not Found' When I am using the download.file() function in a loop over multiple URLs, th! e above error will cause the loop to terminate, so I want to avoid this by checking if the file exists first then wrapping the subsequent functions in an if() statment. The original fault came from the function get.hist.quote() in the tseries package. I was trying to iterate over various stocks, but some stocks listed in the yahoo website do not have any downloadable data associated with them, which causes the loop to terminate. The workhorse function of get.hist.quote() is the download.file() function. Kind Regards Chib _ Win 100's of Virgin Experience days with BigSnapSearch.com [[alternative HTML version deleted]]__ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contain! ed, reproducible code. -- Jim Holtman Cincinnati, OH +1 513 646 9390 What is the problem you are trying to solve? _ Welcome to the next generation of Windows Live [[alternative HTML version deleted]] __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] download.file error
?file.exists On 4/1/08, CHIB CO [EMAIL PROTECTED] wrote: Hi Jim, I just wanted to know if there is a function in R that can tell you whether a file on the internet exists before you attempt to download it or whether there is a way of handling the error you get in the download.file() function without having it break a for loop with the download error when it is part of that loop. Kind Regards Chib Date: Mon, 31 Mar 2008 18:33:35 -0500 From: [EMAIL PROTECTED] To: [EMAIL PROTECTED] Subject: Re: [R] download.file error CC: r-help@r-project.org ?try On Mon, Mar 31, 2008 at 6:26 PM, CHIB CO [EMAIL PROTECTED] wrote: Dear all, I am looking for a way to work out if a file on the internet exists before attempting to download it using the function download.file(). For example, using a url that does not exist url - http://finance.yahoo.com/ftse.csv; destfile - tempfile() download.file(url = url, destfile = destfile) # gives the following response ... trying URL 'http://finance.yahoo.com/ftse.csv' Error in download.file(url = url, destfile = destfile) : cannot open URL 'http://finance.yahoo.com/ftse.csv' In addition: Warning message: In download.file(url = url, destfile = destfile) : cannot open: HTTP status was '404 Not Found' When I am using the download.file() function in a loop over multiple URLs, the above error will cause the loop to terminate, so I want to avoid this by checking if the file exists first then wrapping the subsequent functions in an if() statment. The original fault came from the function get.hist.quote() in the tseries package. I was trying to iterate over various stocks, but some stocks listed in the yahoo website do not have any downloadable data associated with them, which causes the loop to terminate. The workhorse function of get.hist.quote() is the download.file() function. Kind Regards Chib _ Win 100's of Virgin Experience days with BigSnapSearch.com [[alternative HTML version deleted]] __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. -- Jim Holtman Cincinnati, OH +1 513 646 9390 What is the problem you are trying to solve? Have you played Fishticuffs? Get fish-slapping on Messenger -- Jim Holtman Cincinnati, OH +1 513 646 9390 What is the problem you are trying to solve? __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] download.file error
Hi John Seers, thanks for the tip, it works. Kind Regards Chib Subject: RE: [R] download.file error Date: Tue, 1 Apr 2008 15:39:01 +0100 From: [EMAIL PROTECTED] To: [EMAIL PROTECTED]; [EMAIL PROTECTED] CC: r-help@r-project.org Hi Chib whether there is a way of handling the error you get in the download.file() function I think the function try will do this for you. That is, handle the error. Have a look at ?try. Regards John Seers --- -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of CHIB CO Sent: 01 April 2008 15:28 To: jim holtman Cc: r-help@r-project.org Subject: Re: [R] download.file error Hi Jim, I just wanted to know if there is a function in R that can tell you whether a file on the internet exists before you attempt to download it or whether there is a way of handling the error you get in the download.file() function without having it break a for loop with the download error when it is part of that loop. Kind Regards Chib _ Win 100s of Virgin Experience days with BigSnapSearch.com [[alternative HTML version deleted]] __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
[R] download.file error
Dear all, I am looking for a way to work out if a file on the internet exists before attempting to download it using the function download.file(). For example, using a url that does not exist url - http://finance.yahoo.com/ftse.csv; destfile - tempfile() download.file(url = url, destfile = destfile) # gives the following response ... trying URL 'http://finance.yahoo.com/ftse.csv' Error in download.file(url = url, destfile = destfile) : cannot open URL 'http://finance.yahoo.com/ftse.csv' In addition: Warning message: In download.file(url = url, destfile = destfile) : cannot open: HTTP status was '404 Not Found' When I am using the download.file() function in a loop over multiple URLs, the above error will cause the loop to terminate, so I want to avoid this by checking if the file exists first then wrapping the subsequent functions in an if() statment. The original fault came from the function get.hist.quote() in the tseries package. I was trying to iterate over various stocks, but some stocks listed in the yahoo website do not have any downloadable data associated with them, which causes the loop to terminate. The workhorse function of get.hist.quote() is the download.file() function. Kind Regards Chib _ Win 100s of Virgin Experience days with BigSnapSearch.com [[alternative HTML version deleted]] __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] download.file error
?try On Mon, Mar 31, 2008 at 6:26 PM, CHIB CO [EMAIL PROTECTED] wrote: Dear all, I am looking for a way to work out if a file on the internet exists before attempting to download it using the function download.file(). For example, using a url that does not exist url - http://finance.yahoo.com/ftse.csv; destfile - tempfile() download.file(url = url, destfile = destfile) # gives the following response ... trying URL 'http://finance.yahoo.com/ftse.csv' Error in download.file(url = url, destfile = destfile) : cannot open URL 'http://finance.yahoo.com/ftse.csv' In addition: Warning message: In download.file(url = url, destfile = destfile) : cannot open: HTTP status was '404 Not Found' When I am using the download.file() function in a loop over multiple URLs, the above error will cause the loop to terminate, so I want to avoid this by checking if the file exists first then wrapping the subsequent functions in an if() statment. The original fault came from the function get.hist.quote() in the tseries package. I was trying to iterate over various stocks, but some stocks listed in the yahoo website do not have any downloadable data associated with them, which causes the loop to terminate. The workhorse function of get.hist.quote() is the download.file() function. Kind Regards Chib _ Win 100's of Virgin Experience days with BigSnapSearch.com [[alternative HTML version deleted]] __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. -- Jim Holtman Cincinnati, OH +1 513 646 9390 What is the problem you are trying to solve? __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
[R] download.file()
Hi,
I wanted to download a file and did the following:
-
fileLink -
'ftp://ftp.ncbi.nih.gov/pub/geo/DATA/supplementary/series/GSE1000/GSE1000_RAW.tar'
download.file(fileLink,'/geoDat')
trying URL
'ftp://ftp.ncbi.nih.gov/pub/geo/DATA/supplementary/series/GSE1000/GSE1000_RAW.tar'
ftp data connection made, file length 35307520 bytes
opened URL
downloaded 34480Kb
However, when I look in the destination directory ('/geoDat'), the file is not
there (I also tried giving it the absolute path).
What am I doing wrong?
thanks!
Be a better friend, newshound, and
[[alternative HTML version deleted]]
__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Re: [R] download.file()
On Tue, Mar 18, 2008 at 8:46 AM, Paul Evans [EMAIL PROTECTED] wrote:
Hi,
I wanted to download a file and did the following:
-
fileLink -
'ftp://ftp.ncbi.nih.gov/pub/geo/DATA/supplementary/series/GSE1000/GSE1000_RAW.tar'
download.file(fileLink,'/geoDat')
trying URL
'ftp://ftp.ncbi.nih.gov/pub/geo/DATA/supplementary/series/GSE1000/GSE1000_RAW.tar'
ftp data connection made, file length 35307520 bytes
opened URL
downloaded 34480Kb
However, when I look in the destination directory ('/geoDat'), the file is
not there (I also tried giving it the absolute path).
What am I doing wrong?
I'm quite sure the 'destfile' argument of download.file() is a file
not a directory. Example:
download.file(http://www.r-project.org/index.html;, destfile=foo.html)
file.info(foo.html)
size isdir mode mtime ctime
foo.html 785 FALSE 666 2008-03-18 08:54:19 2008-03-18 08:54:11
atime exe
foo.html 2008-03-18 08:54:19 no
So you probably saved the downloaded file as 'geoDat' in the root
directory '/'.
/Henrik
thanks!
Be a better friend, newshound, and
[[alternative HTML version deleted]]
__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Re: [R] download.file()
On Tuesday 18 March 2008 04:46:12 pm Paul Evans wrote: PE download.file(fileLink,'/geoDat') I would have expected download.file(fileLink,/geoDat/yourfilename) note that you need your complete path (or use setwd) -- Microeconomics University of Erfurt signature.asc Description: This is a digitally signed message part. __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] download.file()
On Tue, Mar 18, 2008 at 8:57 AM, Henrik Bengtsson [EMAIL PROTECTED] wrote:
On Tue, Mar 18, 2008 at 8:46 AM, Paul Evans [EMAIL PROTECTED] wrote:
Hi,
I wanted to download a file and did the following:
-
fileLink -
'ftp://ftp.ncbi.nih.gov/pub/geo/DATA/supplementary/series/GSE1000/GSE1000_RAW.tar'
download.file(fileLink,'/geoDat')
trying URL
'ftp://ftp.ncbi.nih.gov/pub/geo/DATA/supplementary/series/GSE1000/GSE1000_RAW.tar'
ftp data connection made, file length 35307520 bytes
opened URL
downloaded 34480Kb
However, when I look in the destination directory ('/geoDat'), the file
is not there (I also tried giving it the absolute path).
What am I doing wrong?
I'm quite sure the 'destfile' argument of download.file() is a file
not a directory. Example:
download.file(http://www.r-project.org/index.html;, destfile=foo.html)
file.info(foo.html)
size isdir mode mtime ctime
foo.html 785 FALSE 666 2008-03-18 08:54:19 2008-03-18 08:54:11
atime exe
foo.html 2008-03-18 08:54:19 no
So you probably saved the downloaded file as 'geoDat' in the root
directory '/'.
Also, you want to download the file in a binary fashion, i.e. use
argument mode=wb, otherwise your binary tar file will be corrupt.
R-core: I'd suggest to replace the default to mode=wb for file transfers.
/Henrik
/Henrik
thanks!
Be a better friend, newshound, and
[[alternative HTML version deleted]]
__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Re: [R] download.file()
Aaah...I see now. Thanks !!
- Original Message
From: Henrik Bengtsson [EMAIL PROTECTED]
To: Paul Evans [EMAIL PROTECTED]
Cc: [EMAIL PROTECTED]
Sent: Tuesday, March 18, 2008 11:59:45 AM
Subject: Re: [R] download.file()
On Tue, Mar 18, 2008 at 8:57 AM, Henrik Bengtsson [EMAIL PROTECTED] wrote:
On Tue, Mar 18, 2008 at 8:46 AM, Paul Evans [EMAIL PROTECTED] wrote:
Hi,
I wanted to download a file and did the following:
-
fileLink -
'ftp://ftp.ncbi.nih.gov/pub/geo/DATA/supplementary/series/GSE1000/GSE1000_RAW.tar'
download.file(fileLink,'/geoDat')
trying URL
'ftp://ftp.ncbi.nih.gov/pub/geo/DATA/supplementary/series/GSE1000/GSE1000_RAW.tar'
ftp data connection made, file length 35307520 bytes
opened URL
downloaded 34480Kb
However, when I look in the destination directory ('/geoDat'), the file
is not there (I also tried giving it the absolute path).
What am I doing wrong?
I'm quite sure the 'destfile' argument of download.file() is a file
not a directory. Example:
download.file(http://www.r-project.org/index.html;, destfile=foo.html)
file.info(foo.html)
size isdir mode mtime ctime
foo.html 785 FALSE 666 2008-03-18 08:54:19 2008-03-18 08:54:11
atime exe
foo.html 2008-03-18 08:54:19 no
So you probably saved the downloaded file as 'geoDat' in the root
directory '/'.
Also, you want to download the file in a binary fashion, i.e. use
argument mode=wb, otherwise your binary tar file will be corrupt.
R-core: I'd suggest to replace the default to mode=wb for file transfers.
/Henrik
/Henrik
thanks!
Be a better friend, newshound, and
[[alternative HTML version deleted]]
__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Looking for last minute shopping deals?
[[alternative HTML version deleted]]
__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Re: [R] download.file()
/geodat *is* an absoute path!
The second argument of download.file is called 'destfile', so what makes
you think it is a desination *directory*?
Your command works for me, using an account that has write permission in /
, and not otherwise. (It seems very unsafe that you would be using such
an account unless this is a pre-Vista Windows machine.)
On Tue, 18 Mar 2008, Paul Evans wrote:
Hi,
I wanted to download a file and did the following:
-
fileLink -
'ftp://ftp.ncbi.nih.gov/pub/geo/DATA/supplementary/series/GSE1000/GSE1000_RAW.tar'
download.file(fileLink,'/geoDat')
trying URL
'ftp://ftp.ncbi.nih.gov/pub/geo/DATA/supplementary/series/GSE1000/GSE1000_RAW.tar'
ftp data connection made, file length 35307520 bytes
opened URL
downloaded 34480Kb
However, when I look in the destination directory ('/geoDat'), the file
is not there (I also tried giving it the absolute path).
What am I doing wrong?
Not reading the help page?
thanks!
Be a better friend, newshound, and
[[alternative HTML version deleted]]
__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
--
Brian D. Ripley, [EMAIL PROTECTED]
Professor of Applied Statistics, http://www.stats.ox.ac.uk/~ripley/
University of Oxford, Tel: +44 1865 272861 (self)
1 South Parks Road, +44 1865 272866 (PA)
Oxford OX1 3TG, UKFax: +44 1865 272595
__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Re: [R] download.file()
On Tuesday 18 March 2008 04:59:45 pm Henrik Bengtsson wrote: HB Also, you want to download the file in a binary fashion, i.e. use HB argument mode=wb, otherwise your binary tar file will be corrupt. HB HB R-core: I'd suggest to replace the default to mode=wb for file HB transfers. I had no corrupt file and I did not specify wb (R-2.6.2 on Fedora 8) Stefan -- Microeconomics University of Erfurt signature.asc Description: This is a digitally signed message part. __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] download.file()
On Tue, Mar 18, 2008 at 9:10 AM, Prof Brian Ripley
[EMAIL PROTECTED] wrote:
/geodat *is* an absoute path!
The second argument of download.file is called 'destfile', so what makes
you think it is a desination *directory*?
The help file might be the source of confusion:
destfile: A character string with the name where the downloaded file
is saved. Tilde-expansion is performed.
...especially the word where. Better with A character string
specifying the name [pathname?] of the saved file is better.
/Henrik
Your command works for me, using an account that has write permission in /
, and not otherwise. (It seems very unsafe that you would be using such
an account unless this is a pre-Vista Windows machine.)
On Tue, 18 Mar 2008, Paul Evans wrote:
Hi,
I wanted to download a file and did the following:
-
fileLink -
'ftp://ftp.ncbi.nih.gov/pub/geo/DATA/supplementary/series/GSE1000/GSE1000_RAW.tar'
download.file(fileLink,'/geoDat')
trying URL
'ftp://ftp.ncbi.nih.gov/pub/geo/DATA/supplementary/series/GSE1000/GSE1000_RAW.tar'
ftp data connection made, file length 35307520 bytes
opened URL
downloaded 34480Kb
However, when I look in the destination directory ('/geoDat'), the file
is not there (I also tried giving it the absolute path).
What am I doing wrong?
Not reading the help page?
thanks!
Be a better friend, newshound, and
[[alternative HTML version deleted]]
__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
--
Brian D. Ripley, [EMAIL PROTECTED]
Professor of Applied Statistics, http://www.stats.ox.ac.uk/~ripley/
University of Oxford, Tel: +44 1865 272861 (self)
1 South Parks Road, +44 1865 272866 (PA)
Oxford OX1 3TG, UKFax: +44 1865 272595
__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Re: [R] download.file()
Thanks Henrik.
Also, is there a method/package (in R) with which I can unpack a tar file
programatically?
thanks again!
- Original Message
From: Henrik Bengtsson [EMAIL PROTECTED]
To: Prof Brian Ripley [EMAIL PROTECTED]
Cc: Paul Evans [EMAIL PROTECTED]; [EMAIL PROTECTED]
Sent: Tuesday, March 18, 2008 12:38:14 PM
Subject: Re: [R] download.file()
On Tue, Mar 18, 2008 at 9:10 AM, Prof Brian Ripley
[EMAIL PROTECTED] wrote:
/geodat *is* an absoute path!
The second argument of download.file is called 'destfile', so what makes
you think it is a desination *directory*?
The help file might be the source of confusion:
destfile: A character string with the name where the downloaded file
is saved. Tilde-expansion is performed.
especially the word where. Better with A character string
specifying the name [pathname?] of the saved file is better.
/Henrik
Your command works for me, using an account that has write permission in /
, and not otherwise. (It seems very unsafe that you would be using such
an account unless this is a pre-Vista Windows machine.)
On Tue, 18 Mar 2008, Paul Evans wrote:
Hi,
I wanted to download a file and did the following:
-
fileLink -
'ftp://ftp.ncbi.nih.gov/pub/geo/DATA/supplementary/series/GSE1000/GSE1000_RAW.tar'
download.file(fileLink,'/geoDat')
trying URL
'ftp://ftp.ncbi.nih.gov/pub/geo/DATA/supplementary/series/GSE1000/GSE1000_RAW.tar'
ftp data connection made, file length 35307520 bytes
opened URL
downloaded 34480Kb
However, when I look in the destination directory ('/geoDat'), the file
is not there (I also tried giving it the absolute path).
What am I doing wrong?
Not reading the help page?
thanks!
Be a better friend, newshound, and
[[alternative HTML version deleted]]
__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
--
Brian D. Ripley, [EMAIL PROTECTED]
Professor of Applied Statistics, http://www.stats.ox.ac.uk/~ripley/
University of Oxford, Tel: +44 1865 272861 (self)
1 South Parks Road, +44 1865 272866 (PA)
Oxford OX1 3TG, UKFax: +44 1865 272595
__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Looking for last minute shopping deals?
[[alternative HTML version deleted]]
__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Re: [R] download.file()
On Tuesday 18 March 2008 06:32:18 pm Paul Evans wrote: PE Also, is there a method/package (in R) with which I can unpack a tar file PE programatically? I guesses that question when I saw your tar file. Not that you not only need to un-tar but also to un-zip then several other files in that tarball. Counter-question: is it necessary that you do all that from within R? You could manually download, untar/unzip and then read the data that will be your next problem: the files have different names and some other information then just a table ... So unless you want to extract a lot of separate tables I think it is easier to do it manually. Stefan -- Microeconomics University of Erfurt __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] download.file()
Stefan, Correct. However I have a large number of tar files to download (with several zip files in each) - so doing it manually is going to be somewhat difficult. I may have to package this code for other users and if possible, would only like to use R for these operations. thanks for the reply! - Original Message From: Stefan Grosse [EMAIL PROTECTED] To: r-help@r-project.org; Paul Evans [EMAIL PROTECTED] Sent: Tuesday, March 18, 2008 1:57:46 PM Subject: Re: [R] download.file() On Tuesday 18 March 2008 06:32:18 pm Paul Evans wrote: PE Also, is there a method/package (in R) with which I can unpack a tar file PE programatically? I guesses that question when I saw your tar file. Not that you not only need to un-tar but also to un-zip then several other files in that tarball. Counter-question: is it necessary that you do all that from within R? You could manually download, untar/unzip and then read the data that will be your next problem: the files have different names and some other information then just a table ... So unless you want to extract a lot of separate tables I think it is easier to do it manually. Stefan -- Microeconomics University of Erfurt Be a better friend, newshound, and [[alternative HTML version deleted]] __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] download.file not working
So, if you download the same file using a web browser it works, ehe? Then look at the file size and you'll most likely find that it is different from when you use R code. Conclusion? Something is different and it works in the web browser so you might tell R to do something different from what you would expect. Hint, look at the set of arguments download.file() takes. That's my $0.02 Henrik After using the argument mode=wb, the downloaded file is OK. Thank you very much for your comments, Henrik. I've got no real idea of what the mode=wb argument means, so if someone could give an explanation, or a reference, I'd be most grateful. Best Gustaf -- Gustaf Rydevik, M.Sci. tel: +46(0)703 051 451 address:Essingetorget 40,112 66 Stockholm, SE skype:gustaf_rydevik __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
[R] download.file not working
Hi all, I'm trying to download a file from the net, and the download.file command leaves it corrupted, i.e excel cannot open it. It seems as if it's going ok, however. Does anyone have an idea of what's going on? regards, Gustaf Rydevik download.file(url=http://www.who.int/entity/whosis/whostat2006_demographics.xls,destfile=whodem.xls;) trying URL 'http://www.who.int/entity/whosis/whostat2006_demographics.xls' Content type 'application/vnd.ms-excel' length 77824 bytes opened URL downloaded 76Kb sessionInfo() R version 2.5.0 (2007-04-23) i386-pc-mingw32 locale: LC_COLLATE=Swedish_Sweden.1252;LC_CTYPE=Swedish_Sweden.1252;LC_MONETARY=Swedish_Sweden.1252;LC_NUMERIC=C;LC_TIME=Swedish_Sweden.1252 attached base packages: [1] stats graphics grDevices utils datasets methods [7] base other attached packages: RWinEdt 1.7-5 Sys.getlocale() [1] LC_COLLATE=Swedish_Sweden.1252;LC_CTYPE=Swedish_Sweden.1252;LC_MONETARY=Swedish_Sweden.1252;LC_NUMERIC=C;LC_TIME=Swedish_Sweden.1252 -- Gustaf Rydevik, M.Sci. tel: +46(0)703 051 451 address:Essingetorget 40,112 66 Stockholm, SE skype:gustaf_rydevik __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] download.file used incorrectly (was not working)
On Wed, 10 Oct 2007, Gustaf Rydevik wrote: Hi all, I'm trying to download a file from the net, and the download.file command leaves it corrupted, i.e excel cannot open it. It seems as if it's going ok, however. Does anyone have an idea of what's going on? That is a binary file. You needed mode=wb. The R posting guide did ask you to update your R before posting, so better late than never. regards, Gustaf Rydevik download.file(url=http://www.who.int/entity/whosis/whostat2006_demographics.xls,destfile=whodem.xls;) trying URL 'http://www.who.int/entity/whosis/whostat2006_demographics.xls' Content type 'application/vnd.ms-excel' length 77824 bytes opened URL downloaded 76Kb sessionInfo() R version 2.5.0 (2007-04-23) i386-pc-mingw32 locale: LC_COLLATE=Swedish_Sweden.1252;LC_CTYPE=Swedish_Sweden.1252;LC_MONETARY=Swedish_Sweden.1252;LC_NUMERIC=C;LC_TIME=Swedish_Sweden.1252 attached base packages: [1] stats graphics grDevices utils datasets methods [7] base other attached packages: RWinEdt 1.7-5 Sys.getlocale() [1] LC_COLLATE=Swedish_Sweden.1252;LC_CTYPE=Swedish_Sweden.1252;LC_MONETARY=Swedish_Sweden.1252;LC_NUMERIC=C;LC_TIME=Swedish_Sweden.1252 -- Brian D. Ripley, [EMAIL PROTECTED] Professor of Applied Statistics, http://www.stats.ox.ac.uk/~ripley/ University of Oxford, Tel: +44 1865 272861 (self) 1 South Parks Road, +44 1865 272866 (PA) Oxford OX1 3TG, UKFax: +44 1865 272595 __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
