On 11/3/2009 9:29 AM, Johannes Graumann wrote:
Hi,
I'm dealing which huge files I would like to index. On a linux system "grep
-buo <PATTERN> <FILENAME>" hands me the byte offsets for "PATTERN" very
quickly and I am looking to emulate that speed and ease with native R tools
- for portability and elegance. "gregexpr" should be able to do that but I
fail to combine it with "scan" or an equivalent to parse the whole file
without having to read it all into memory.
I think you are going to have to write this yourself. R doesn't have
very many stream oriented functions: almost everything is aimed at
having the whole thing in memory.
You will also have trouble with the byte offsets. The semantics of the
-u option to grep are quite strange (at least according to the man page
on Cygwin).
What I'd do given your problem is use readLines to read the file, then
post-process the result of gregexpr to give line and byte offset pairs
for each match; those are more useful in R than the rather bizarre "byte
offsets" that grep -buo will give. But for a huge file you'll probably
have to do this in blocks, as the whole file may be too big.
Duncan Murdoch
I'd be grateful for any hints on how to do this without a "pipe("grep -buo
<PATTERN> <FILENAME>")".
Thanks, Joh
______________________________________________
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
______________________________________________
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
