[R] UNC windows path beginning with backslashes: normalizePath bug??
Hi, Back in June I posted the message below, but had no replies. I've made a little progress since then so this is to update anyone interested (!) and to ask for comments. Brief problem statement: Under Windows, some parts of R don't handle UNC paths beginning with backslashes. Specifically a) Sys.glob() fails to find some files breaking (e.g.) Rcmdr plugins Sys.glob(file.path(.libPaths(), */etc/menus.txt)) fails to find files which are there b) update.packages(ask='graphics') fails when copying the updates into the destination folders In Renviron.site I define the site library with forward slashes, not backslashes thus... R_LIBS_SITE=//campden/shares/workgroup/stats/R/library/%v ... but the startup process seems to replace them with forward slashes. I guess because .libPaths with a 'new' argument calls normalizePath which changes leading slashes to backslashes, even with winslash=/ normalizePath(//campden/shares/workgroup/stats/R/library, winslash=/) [1] campden/shares/workgroup/Stats/R/library I've corrected (??) this by inserting a line into Rprofile.site assign(.lib.loc, gsub(\\, /, .libPaths(), fixed=TRUE), env=environment(.libPaths)) That seems to fix problem (a) above, which was affecting a number of users. But have I broken anything else? I'm still experiencing problem (b). I'm the only person on site who updates packages so I've mapped a drive letter (L:) and in my own .Rprofile I have a line assign(.lib.loc, sub(//campden/shares/workgroup/Stats, L:, .libPaths(), ignore.case = TRUE), env=environment(.libPaths)) So that's OK as far as it goes, but it's all a bit messy! If .libPaths is called with a 'new' argument it will breaks things again. normalizePath seems to produce paths that don't work with Sys.glob. I have the feeling I'm being silly and making hard work of all this. Any comments? Suggestions? Best regards, and thanks in advance/ Keith Jewell Keith Jewell k.jew...@campden.co.uk wrote in message news:... Hi, Back in 2010 I had a problem with 'update.packages()', which I worked around by mapping a drive letter to a UNC path [described in http://finzi.psych.upenn.edu/Rhelp10/2010-February/229820.html but my current workaround is assign(.lib.loc, sub(Server02/stats, L:, .libPaths(), ignore.case = TRUE), env=environment(.libPaths))]. More recently a colleague had problems using the 'FactoMineR' plug in for the Rcmdr package; a) directly loading 'RcmdrPlugin.FactoMineR' opened and crashed R Commander; b) opening R Commander without FactoMiner, the Tools option 'Load Rcmdr plug-in(s)...' was greyed out. It transpired that in .libPaths() the path to the library holding 'RcmdrPlugin.FactoMineR' was specified as a UNC address: Server02/stats/R/library/2.13. Mapping a virtual drive letter (e.g. L:) and specifying the path in .libPaths() as a 'local file system' (LFS) address L:/R/library/2.13 fixed the problem. I contacted Professor Fox (maintainer of Rcmdr) who told me that Rcmdr finds plug-in packages via the command plugins - unlist(lapply(.libPaths(), function(x) Sys.glob(file.path(x, */etc/menus.txt Because file.path and Sys.glob are both vectorised I think (but am not certain) that this could be simplified to: plugins - Sys.glob(file.path(.libPaths(), */etc/menus.txt)) but that's by the way, the problem seems to lie in Sys.glob under Windows operating systems. I note that 'help(Sys.glob)' on my Windows system differs from http://finzi.psych.upenn.edu/R/library/base/html/Sys.glob.html. The latter says For precise details, see your system's documentation on the glob system call. There is a POSIX 1003.2 standard snip The rest of these details are indicative (and based on the POSIX standard). On Windows The glob system call is not part of Windows, and we supply a partial emulation. snip An attempt is made to handle UNC paths starting with a double backslash which doesn't really inspire confidence. This was discussed in a 2009 R-devel thread starting here https://stat.ethz.ch/pipermail/r-devel/2009-June/053879.html, but the patch proposed in that thread seems not to have been implemented (??). Trying to avoid Sys.glob in the Rcmdr application I came up with this: list.files(path=file.path(list.files(path=.libPaths(), full.names=TRUE), etc), pattern=^menus\\.txt$, full.names=TRUE) It seems to give identical results to Sys.glob for mapped drives, works with UNC paths in Windows, and seems quite fast. So my questions relate to diagnosis, prognosis, and prescription (cure?). 1) Diagnosis: Am I correct that my problem(s) originate in the partial emulation of glob in Windows. 2) Prognosis: If so, is there any likelihood that the emulation will improve in the near future? 3) Prescription: If not: a) is assign(.lib.loc, sub(Server02/stats, L:, .libPaths(), ignore.case = TRUE), env=environment(.libPaths)) a reasonable workaround in a
Re: [R] UNC Windows path beginning with backslashes: normalizePath bug??
Thanks Uwe. I'm aware (and have been forcefully reminded) that using a mapped drive avoids these problems. But there is no single drive letter which I can use site-wide, so I have problems with things like R_LIBS_SITE. As I've outlined I'm exploring a range of solutions, including mapping a drive where I can. I posted in the hope of learning from and perhaps helping those with similar problems. I hope that it is permissible to discuss non-canonical use of R on this list, I certainly did not intend disrespect for the R developers (or to make typing errors). Best regards Keith Jewell Uwe Ligges lig...@statistik.tu-dortmund.de wrote in message news:4e44091e.7090...@statistik.tu-dortmund.de... This is extremely tricky since Windows does not always accept // rather than \\. Additionally, there is not implemented system call in Windows, hence ?Sys.glob tells us a partial emulation is provided and An attempt is made to handle UNC paths starting with a double backslash. As you have seenm this does not work everywhere, therefore it is advisable to run R from mapped drives - as I am doing in the network of our university for 13 years without problems now. Best, Uwe Ligges On 11.08.2011 18:29, Keith Jewell wrote: Hi, Back in June I posted the message below, but had no replies. I've made a little progress since then so this is to update anyone interested (!) and to ask for comments. Brief problem statement: Under Windows, some parts of R don't handle UNC paths beginning with backslashes. Specifically a) Sys.glob() fails to find some files breaking (e.g.) Rcmdr plugins Sys.glob(file.path(.libPaths(), */etc/menus.txt)) fails to find files which are there b) update.packages(ask='graphics') fails when copying the updates into the destination folders In Renviron.site I define the site library with forward slashes, not backslashes thus... R_LIBS_SITE=//campden/shares/workgroup/stats/R/library/%v ... but the startup process seems to replace them with forward slashes. I guess because .libPaths with a 'new' argument calls normalizePath which changes leading slashes to backslashes, even with winslash=/ normalizePath(//campden/shares/workgroup/stats/R/library, winslash=/) [1] campden/shares/workgroup/Stats/R/library I've corrected (??) this by inserting a line into Rprofile.site assign(.lib.loc, gsub(\\, /, .libPaths(), fixed=TRUE), env=environment(.libPaths)) That seems to fix problem (a) above, which was affecting a number of users. But have I broken anything else? I'm still experiencing problem (b). I'm the only person on site who updates packages so I've mapped a drive letter (L:) and in my own .Rprofile I have a line assign(.lib.loc, sub(//campden/shares/workgroup/Stats, L:, .libPaths(), ignore.case = TRUE), env=environment(.libPaths)) So that's OK as far as it goes, but it's all a bit messy! If .libPaths is called with a 'new' argument it will breaks things again. normalizePath seems to produce paths that don't work with Sys.glob. I have the feeling I'm being silly and making hard work of all this. Any comments? Suggestions? Best regards, and thanks in advance/ Keith Jewell Keith Jewellk.jew...@campden.co.uk wrote in message news:... Hi, Back in 2010 I had a problem with 'update.packages()', which I worked around by mapping a drive letter to a UNC path [described in http://finzi.psych.upenn.edu/Rhelp10/2010-February/229820.html but my current workaround is assign(.lib.loc, sub(Server02/stats, L:, .libPaths(), ignore.case = TRUE), env=environment(.libPaths))]. More recently a colleague had problems using the 'FactoMineR' plug in for the Rcmdr package; a) directly loading 'RcmdrPlugin.FactoMineR' opened and crashed R Commander; b) opening R Commander without FactoMiner, the Tools option 'Load Rcmdr plug-in(s)...' was greyed out. It transpired that in .libPaths() the path to the library holding 'RcmdrPlugin.FactoMineR' was specified as a UNC address: Server02/stats/R/library/2.13. Mapping a virtual drive letter (e.g. L:) and specifying the path in .libPaths() as a 'local file system' (LFS) addressL:/R/library/2.13 fixed the problem. I contacted Professor Fox (maintainer of Rcmdr) who told me that Rcmdr finds plug-in packages via the command plugins- unlist(lapply(.libPaths(), function(x) Sys.glob(file.path(x, */etc/menus.txt Because file.path and Sys.glob are both vectorised I think (but am not certain) that this could be simplified to: plugins- Sys.glob(file.path(.libPaths(), */etc/menus.txt)) but that's by the way, the problem seems to lie in Sys.glob under Windows operating systems. I note that 'help(Sys.glob)' on my Windows system differs from http://finzi.psych.upenn.edu/R/library/base/html/Sys.glob.html. The latter says For precise details, see your system's documentation on the glob system call. There is a POSIX 1003.2 standardsnip
[R] UNC windows path beginning with backslashes: normalizePath bug??
Hi, Back in June I posted the message below, but had no replies. I've made a little progress since then so this is to update anyone interested (!) and to ask for comments. Brief problem statement: Under Windows, some parts of R don't handle UNC paths beginning with backslashes. Specifically a) Sys.glob() fails to find some files breaking (e.g.) Rcmdr plugins Sys.glob(file.path(.libPaths(), */etc/menus.txt)) fails to find files which are there b) update.packages(ask='graphics') fails when copying the updates into the destination folders In Renviron.site I define the site library with forward slashes, not backslashes thus... R_LIBS_SITE=//campden/shares/workgroup/stats/R/library/%v ... but the startup process seems to replace them with forward slashes. I guess because .libPaths with a 'new' argument calls normalizePath which changes leading slashes to backslashes, even with winslash=/ normalizePath(//campden/shares/workgroup/stats/R/library, winslash=/) [1] campden/shares/workgroup/Stats/R/library I've corrected (??) this by inserting a line into Rprofile.site assign(.lib.loc, gsub(\\, /, .libPaths(), fixed=TRUE), env=environment(.libPaths)) That seems to fix problem (a) above, which was affecting a number of users. But have I broken anything else? I'm still experiencing problem (b). I'm the only person on site who updates packages so I've mapped a drive letter (L:) and in my own .Rprofile I have a line assign(.lib.loc, sub(//campden/shares/workgroup/Stats, L:, .libPaths(), ignore.case = TRUE), env=environment(.libPaths)) So that's OK as far as it goes, but it's all a bit messy! If .libPaths is called with a 'new' argument it will breaks things again. normalizePath seems to produce paths that don't work with Sys.glob. I have the feeling I'm being silly and making hard work of all this. Any comments? Suggestions? Best regards, and thanks in advance/ Keith Jewell Keith Jewell k.jew...@campden.co.uk wrote in message news:... Hi, Back in 2010 I had a problem with 'update.packages()', which I worked around by mapping a drive letter to a UNC path [described in http://finzi.psych.upenn.edu/Rhelp10/2010-February/229820.html but my current workaround is assign(.lib.loc, sub(Server02/stats, L:, .libPaths(), ignore.case = TRUE), env=environment(.libPaths))]. More recently a colleague had problems using the 'FactoMineR' plug in for the Rcmdr package; a) directly loading 'RcmdrPlugin.FactoMineR' opened and crashed R Commander; b) opening R Commander without FactoMiner, the Tools option 'Load Rcmdr plug-in(s)...' was greyed out. It transpired that in .libPaths() the path to the library holding 'RcmdrPlugin.FactoMineR' was specified as a UNC address: Server02/stats/R/library/2.13. Mapping a virtual drive letter (e.g. L:) and specifying the path in .libPaths() as a 'local file system' (LFS) address L:/R/library/2.13 fixed the problem. I contacted Professor Fox (maintainer of Rcmdr) who told me that Rcmdr finds plug-in packages via the command plugins - unlist(lapply(.libPaths(), function(x) Sys.glob(file.path(x, */etc/menus.txt Because file.path and Sys.glob are both vectorised I think (but am not certain) that this could be simplified to: plugins - Sys.glob(file.path(.libPaths(), */etc/menus.txt)) but that's by the way, the problem seems to lie in Sys.glob under Windows operating systems. I note that 'help(Sys.glob)' on my Windows system differs from http://finzi.psych.upenn.edu/R/library/base/html/Sys.glob.html. The latter says For precise details, see your system's documentation on the glob system call. There is a POSIX 1003.2 standard snip The rest of these details are indicative (and based on the POSIX standard). On Windows The glob system call is not part of Windows, and we supply a partial emulation. snip An attempt is made to handle UNC paths starting with a double backslash which doesn't really inspire confidence. This was discussed in a 2009 R-devel thread starting here https://stat.ethz.ch/pipermail/r-devel/2009-June/053879.html, but the patch proposed in that thread seems not to have been implemented (??). Trying to avoid Sys.glob in the Rcmdr application I came up with this: list.files(path=file.path(list.files(path=.libPaths(), full.names=TRUE), etc), pattern=^menus\\.txt$, full.names=TRUE) It seems to give identical results to Sys.glob for mapped drives, works with UNC paths in Windows, and seems quite fast. So my questions relate to diagnosis, prognosis, and prescription (cure?). 1) Diagnosis: Am I correct that my problem(s) originate in the partial emulation of glob in Windows. 2) Prognosis: If so, is there any likelihood that the emulation will improve in the near future? 3) Prescription: If not: a) is assign(.lib.loc, sub(Server02/stats, L:, .libPaths(), ignore.case = TRUE), env=environment(.libPaths)) a reasonable workaround in a
Re: [R] UNC windows path beginning with backslashes: normalizePath bug??
This is extremely tricky since Windows does not always accept // rather than \\. Additionally, there is not implemented system call in Windows, hence ?Sys.glob tells us a partial emulation is provided and An attempt is made to handle UNC paths starting with a double backslash. As you have seenm this does not work everywhere, therefore it is advisable to run R from mapped drives - as I am doing in the network of our university for 13 years without problems now. Best, Uwe Ligges On 11.08.2011 18:29, Keith Jewell wrote: Hi, Back in June I posted the message below, but had no replies. I've made a little progress since then so this is to update anyone interested (!) and to ask for comments. Brief problem statement: Under Windows, some parts of R don't handle UNC paths beginning with backslashes. Specifically a) Sys.glob() fails to find some files breaking (e.g.) Rcmdr plugins Sys.glob(file.path(.libPaths(), */etc/menus.txt)) fails to find files which are there b) update.packages(ask='graphics') fails when copying the updates into the destination folders In Renviron.site I define the site library with forward slashes, not backslashes thus... R_LIBS_SITE=//campden/shares/workgroup/stats/R/library/%v ... but the startup process seems to replace them with forward slashes. I guess because .libPaths with a 'new' argument calls normalizePath which changes leading slashes to backslashes, even with winslash=/ normalizePath(//campden/shares/workgroup/stats/R/library, winslash=/) [1] campden/shares/workgroup/Stats/R/library I've corrected (??) this by inserting a line into Rprofile.site assign(.lib.loc, gsub(\\, /, .libPaths(), fixed=TRUE), env=environment(.libPaths)) That seems to fix problem (a) above, which was affecting a number of users. But have I broken anything else? I'm still experiencing problem (b). I'm the only person on site who updates packages so I've mapped a drive letter (L:) and in my own .Rprofile I have a line assign(.lib.loc, sub(//campden/shares/workgroup/Stats, L:, .libPaths(), ignore.case = TRUE), env=environment(.libPaths)) So that's OK as far as it goes, but it's all a bit messy! If .libPaths is called with a 'new' argument it will breaks things again. normalizePath seems to produce paths that don't work with Sys.glob. I have the feeling I'm being silly and making hard work of all this. Any comments? Suggestions? Best regards, and thanks in advance/ Keith Jewell Keith Jewellk.jew...@campden.co.uk wrote in message news:... Hi, Back in 2010 I had a problem with 'update.packages()', which I worked around by mapping a drive letter to a UNC path [described in http://finzi.psych.upenn.edu/Rhelp10/2010-February/229820.html but my current workaround is assign(.lib.loc, sub(Server02/stats, L:, .libPaths(), ignore.case = TRUE), env=environment(.libPaths))]. More recently a colleague had problems using the 'FactoMineR' plug in for the Rcmdr package; a) directly loading 'RcmdrPlugin.FactoMineR' opened and crashed R Commander; b) opening R Commander without FactoMiner, the Tools option 'Load Rcmdr plug-in(s)...' was greyed out. It transpired that in .libPaths() the path to the library holding 'RcmdrPlugin.FactoMineR' was specified as a UNC address: Server02/stats/R/library/2.13. Mapping a virtual drive letter (e.g. L:) and specifying the path in .libPaths() as a 'local file system' (LFS) addressL:/R/library/2.13 fixed the problem. I contacted Professor Fox (maintainer of Rcmdr) who told me that Rcmdr finds plug-in packages via the command plugins- unlist(lapply(.libPaths(), function(x) Sys.glob(file.path(x, */etc/menus.txt Because file.path and Sys.glob are both vectorised I think (but am not certain) that this could be simplified to: plugins- Sys.glob(file.path(.libPaths(), */etc/menus.txt)) but that's by the way, the problem seems to lie in Sys.glob under Windows operating systems. I note that 'help(Sys.glob)' on my Windows system differs from http://finzi.psych.upenn.edu/R/library/base/html/Sys.glob.html. The latter says For precise details, see your system's documentation on the glob system call. There is a POSIX 1003.2 standardsnip The rest of these details are indicative (and based on the POSIX standard). On Windows The glob system call is not part of Windows, and we supply a partial emulation.snip An attempt is made to handle UNC paths starting with a double backslash which doesn't really inspire confidence. This was discussed in a 2009 R-devel thread starting here https://stat.ethz.ch/pipermail/r-devel/2009-June/053879.html, but the patch proposed in that thread seems not to have been implemented (??). Trying to avoid Sys.glob in the Rcmdr application I came up with this: list.files(path=file.path(list.files(path=.libPaths(), full.names=TRUE), etc), pattern=^menus\\.txt$, full.names=TRUE) It seems to give identical results to Sys.glob for mapped drives, works with UNC paths in Windows, and seems quite fast.
