Re: [R] Regex Split?
Dear Bert,
Thank you for the suggestion. Indeed, there are various solutions and
workarounds. However, there is still a bug in strsplit.
2.) gsub
I would try to avoid gsub on a Wikipedia-sized corpus: using strsplit
directly should be far more efficient.
3.) Punctuation marks
Abbreviations and "word1-word2" may be a problem:
gsub("(?[[:punct:]])", "\\1 ", "A.B.C.", perl=T)
# "A. B. C. "
I do not yet have an intuition if the spaces in "A. B. C. " would
adversely affect the language model. But this goes off-topic.
Sincerely,
Leonard
On 5/6/2023 1:35 AM, Bert Gunter wrote:
> Primarily for my own amusement, here is a way to do what I think you
> wanted without look-aheads/behinds
>
> strsplit(gsub("([[:punct:]])"," \\1 ","a bc,def, adef,x; ,,gh"), " +")
> [[1]]
> [1] "a" "bc" "," "def" "," "adef" "," "x" ";"
> [10] "," "," "gh"
>
> I certainly would *not* claim that it is in any way superior to
> anything that has already been suggested -- indeed, probably the
> contrary. But it's simple (as am I).
>
> Cheers,
> Bert
>
> On Fri, May 5, 2023 at 2:54 PM Leonard Mada via R-help
> wrote:
>
> Dear Avi,
>
> Punctuation marks are used in various NLP language models. Preserving
> the "," is therefore useful in such scenarios and Regex are useful to
> accomplish this (especially if you have sufficient experience with
> such
> expressions).
>
> I observed only an odd behaviour using strsplit: the example
> string is
> constructed; but it is always wise to test a Regex expression against
> various scenarios. It is usually hard to predict what special
> cases will
> occur in a specific corpus.
>
> strsplit("a bc,def, adef ,,gh", " |(?=,)|(?<=,)(?![ ])", perl=T)
> # "a" "bc" "," "def" "," "" "adef" "," "," "gh"
>
> stringi::stri_split("a bc,def, adef ,,gh", regex="
> |(?=,)|(?<=,)(?![ ])")
> # "a" "bc" "," "def" "," "adef" "" "," "," "gh"
>
> stringi::stri_split("a bc,def, adef ,,gh", regex=" |(? )(?=,)|(?<=,)(?![ ])")
> # "a" "bc" "," "def" "," "adef" "," "," "gh"
>
> # Expected:
> # "a" "bc" "," "def" "," "adef" "," "," "gh"
> # see 2nd instance of stringi::stri_split
>
>
> Sincerely,
>
>
> Leonard
>
>
> On 5/5/2023 11:20 PM, avi.e.gr...@gmail.com wrote:
> > Leonard,
> >
> > It can be helpful to spell out your intent in English or some of
> us have to go back to the documentation to remember what some of
> the operators do.
> >
> > Your text being searched seems to be an example of items between
> comas with an optional space after some commas and in one case,
> nothing between commas.
> >
> > So what is your goal for the example, and in general? You
> mention a bit unclearly at the end some of what you expect and I
> think it would be clearer if you also showed exactly the output
> you would want.
> >
> > I saw some other replies that addressed what you wanted and am
> going to reply in another direction.
> >
> > Why do things the hard way using things like lookahead or look
> behind? Would several steps get you the result way more clearly?
> >
> > For the sake of argument, you either want what reading in a CSV
> file would supply, or something else. Since you are not simply
> splitting on commas, it sounds like something else. But what
> exactly else? Something as simple as this on just a comma produces
> results including empty strings and embedded leading or trailing
> spaces:
> >
> > strsplit("a bc,def, adef ,,gh", ",")
> > [[1]]
> > [1] "a bc" "def" " adef " "" "gh"
> >
> > That can of course be handled by, for example, trimming the
> result after unlisting the odd way strsplit returns results:
> >
> > library("stringr")
> > str_squish(unlist(strsplit("a bc,def, adef ,,gh", ",")))
> >
> > [1] "a bc" "def" "adef" "" "gh"
> >
> > Now do you want the empty string to be something else, such as
> an NA? That can be done too with another step.
> >
> > And a completely different variant can be used to read in your
> one-line CSV as text using standard overkill tools:
> >
> >> read.table(text="a bc,def, adef ,,gh", sep=",")
> > V1 V2 V3 V4 V5
> > 1 a bc def adef NA gh
> >
> > The above is a vector of texts. But if you simply want to
> reassemble your initial string cleaned up a bit, you can use paste
> to put back commas, as in a variation of the earlier example:
> >
> >> paste(str_squish(unlist(strsplit("a bc,def, adef ,,gh", ","))),
> collapse=",")
> > [1] "a bc,def,adef,,gh"
> >
> > So my question is whether using advanced methods is really
> necessary for your case, or even particularly efficient. If
> efficiency matters, often, it is better to use tools
Re: [R] Regex Split?
Primarily for my own amusement, here is a way to do what I think you wanted
without look-aheads/behinds
strsplit(gsub("([[:punct:]])"," \\1 ","a bc,def, adef,x; ,,gh"), " +")
[[1]]
[1] "a""bc" ",""def" ",""adef" ",""x"";"
[10] ","",""gh"
I certainly would *not* claim that it is in any way superior to anything
that has already been suggested -- indeed, probably the contrary. But it's
simple (as am I).
Cheers,
Bert
On Fri, May 5, 2023 at 2:54 PM Leonard Mada via R-help
wrote:
> Dear Avi,
>
> Punctuation marks are used in various NLP language models. Preserving
> the "," is therefore useful in such scenarios and Regex are useful to
> accomplish this (especially if you have sufficient experience with such
> expressions).
>
> I observed only an odd behaviour using strsplit: the example string is
> constructed; but it is always wise to test a Regex expression against
> various scenarios. It is usually hard to predict what special cases will
> occur in a specific corpus.
>
> strsplit("a bc,def, adef ,,gh", " |(?=,)|(?<=,)(?![ ])", perl=T)
> # "a" "bc" "," "def" "," "" "adef" "," "," "gh"
>
> stringi::stri_split("a bc,def, adef ,,gh", regex=" |(?=,)|(?<=,)(?![ ])")
> # "a""bc" ",""def" ",""adef" "" ",""," "gh"
>
> stringi::stri_split("a bc,def, adef ,,gh", regex=" |(? )(?=,)|(?<=,)(?![ ])")
> # "a""bc" ",""def" ",""adef" ","",""gh"
>
> # Expected:
> # "a" "bc" "," "def" "," "adef" "," "," "gh"
> # see 2nd instance of stringi::stri_split
>
>
> Sincerely,
>
>
> Leonard
>
>
> On 5/5/2023 11:20 PM, avi.e.gr...@gmail.com wrote:
> > Leonard,
> >
> > It can be helpful to spell out your intent in English or some of us have
> to go back to the documentation to remember what some of the operators do.
> >
> > Your text being searched seems to be an example of items between comas
> with an optional space after some commas and in one case, nothing between
> commas.
> >
> > So what is your goal for the example, and in general? You mention a bit
> unclearly at the end some of what you expect and I think it would be
> clearer if you also showed exactly the output you would want.
> >
> > I saw some other replies that addressed what you wanted and am going to
> reply in another direction.
> >
> > Why do things the hard way using things like lookahead or look behind?
> Would several steps get you the result way more clearly?
> >
> > For the sake of argument, you either want what reading in a CSV file
> would supply, or something else. Since you are not simply splitting on
> commas, it sounds like something else. But what exactly else? Something as
> simple as this on just a comma produces results including empty strings and
> embedded leading or trailing spaces:
> >
> > strsplit("a bc,def, adef ,,gh", ",")
> > [[1]]
> > [1] "a bc" "def"" adef " "" "gh"
> >
> > That can of course be handled by, for example, trimming the result after
> unlisting the odd way strsplit returns results:
> >
> > library("stringr")
> > str_squish(unlist(strsplit("a bc,def, adef ,,gh", ",")))
> >
> > [1] "a bc" "def" "adef" "" "gh"
> >
> > Now do you want the empty string to be something else, such as an NA?
> That can be done too with another step.
> >
> > And a completely different variant can be used to read in your one-line
> CSV as text using standard overkill tools:
> >
> >> read.table(text="a bc,def, adef ,,gh", sep=",")
> > V1 V2 V3 V4 V5
> > 1 a bc def adef NA gh
> >
> > The above is a vector of texts. But if you simply want to reassemble
> your initial string cleaned up a bit, you can use paste to put back commas,
> as in a variation of the earlier example:
> >
> >> paste(str_squish(unlist(strsplit("a bc,def, adef ,,gh", ","))),
> collapse=",")
> > [1] "a bc,def,adef,,gh"
> >
> > So my question is whether using advanced methods is really necessary for
> your case, or even particularly efficient. If efficiency matters, often, it
> is better to use tools without regular expressions such as paste0() when
> they meet your needs.
> >
> > Of course, unless I know what you are actually trying to do, my remarks
> may be not useful.
> >
> >
> >
> > -Original Message-
> > From: R-help On Behalf Of Leonard Mada
> via R-help
> > Sent: Thursday, May 4, 2023 5:00 PM
> > To: R-help Mailing List
> > Subject: [R] Regex Split?
> >
> > Dear R-Users,
> >
> > I tried the following 3 Regex expressions in R 4.3:
> > strsplit("a bc,def, adef ,,gh", " |(?=,)|(?<=,)(?![ ])", perl=T)
> > # "a""bc" ",""def" ",""" "adef" ",""," "gh"
> >
> > strsplit("a bc,def, adef ,,gh", " |(? > # "a""bc" ",""def" ",""" "adef" ",""," "gh"
> >
> > strsplit("a bc,def, adef ,,gh", " |(? > # "a""bc" ",""def" ",""" "adef" ",""," "gh"
> >
> >
> > Is this correct?
> >
> >
> > I feel that:
> > - none should return (after "def"): ",", "";
> > - the first one could also return "", "," (but
Re: [R] Regex Split?
Dear Avi,
Punctuation marks are used in various NLP language models. Preserving
the "," is therefore useful in such scenarios and Regex are useful to
accomplish this (especially if you have sufficient experience with such
expressions).
I observed only an odd behaviour using strsplit: the example string is
constructed; but it is always wise to test a Regex expression against
various scenarios. It is usually hard to predict what special cases will
occur in a specific corpus.
strsplit("a bc,def, adef ,,gh", " |(?=,)|(?<=,)(?![ ])", perl=T)
# "a" "bc" "," "def" "," "" "adef" "," "," "gh"
stringi::stri_split("a bc,def, adef ,,gh", regex=" |(?=,)|(?<=,)(?![ ])")
# "a" "bc" "," "def" "," "adef" "" "," "," "gh"
stringi::stri_split("a bc,def, adef ,,gh", regex=" |(?)(?=,)|(?<=,)(?![ ])")
# "a" "bc" "," "def" "," "adef" "," "," "gh"
# Expected:
# "a" "bc" "," "def" "," "adef" "," "," "gh"
# see 2nd instance of stringi::stri_split
Sincerely,
Leonard
On 5/5/2023 11:20 PM, avi.e.gr...@gmail.com wrote:
Leonard,
It can be helpful to spell out your intent in English or some of us have to go
back to the documentation to remember what some of the operators do.
Your text being searched seems to be an example of items between comas with an
optional space after some commas and in one case, nothing between commas.
So what is your goal for the example, and in general? You mention a bit
unclearly at the end some of what you expect and I think it would be clearer if
you also showed exactly the output you would want.
I saw some other replies that addressed what you wanted and am going to reply
in another direction.
Why do things the hard way using things like lookahead or look behind? Would
several steps get you the result way more clearly?
For the sake of argument, you either want what reading in a CSV file would
supply, or something else. Since you are not simply splitting on commas, it
sounds like something else. But what exactly else? Something as simple as this
on just a comma produces results including empty strings and embedded leading
or trailing spaces:
strsplit("a bc,def, adef ,,gh", ",")
[[1]]
[1] "a bc" "def"" adef " "" "gh"
That can of course be handled by, for example, trimming the result after
unlisting the odd way strsplit returns results:
library("stringr")
str_squish(unlist(strsplit("a bc,def, adef ,,gh", ",")))
[1] "a bc" "def" "adef" "" "gh"
Now do you want the empty string to be something else, such as an NA? That can
be done too with another step.
And a completely different variant can be used to read in your one-line CSV as
text using standard overkill tools:
read.table(text="a bc,def, adef ,,gh", sep=",")
V1 V2 V3 V4 V5
1 a bc def adef NA gh
The above is a vector of texts. But if you simply want to reassemble your
initial string cleaned up a bit, you can use paste to put back commas, as in a
variation of the earlier example:
paste(str_squish(unlist(strsplit("a bc,def, adef ,,gh", ","))), collapse=",")
[1] "a bc,def,adef,,gh"
So my question is whether using advanced methods is really necessary for your
case, or even particularly efficient. If efficiency matters, often, it is
better to use tools without regular expressions such as paste0() when they meet
your needs.
Of course, unless I know what you are actually trying to do, my remarks may be
not useful.
-Original Message-
From: R-help On Behalf Of Leonard Mada via R-help
Sent: Thursday, May 4, 2023 5:00 PM
To: R-help Mailing List
Subject: [R] Regex Split?
Dear R-Users,
I tried the following 3 Regex expressions in R 4.3:
strsplit("a bc,def, adef ,,gh", " |(?=,)|(?<=,)(?![ ])", perl=T)
# "a""bc" ",""def" ",""" "adef" ",""," "gh"
strsplit("a bc,def, adef ,,gh", " |(?https://eu01.z.antigena.com/l/boS91wizs77ZHrpn6fDgE-TZu7JxUnjyNg_9mZDUsLWLylcL-dhQytfeUHheLHZnKJw-VwwfCd_W4XdAukyKenqYPFzSJmP5FrWmF_wepejCrBByUVa66jUF7wKGiA8LnqB49ZUVq-urjKs272Rl-mj-SE1q7--Xj1UXRol3
PLEASE do read the posting guide
https://eu01.z.antigena.com/l/rUS82cEKjOa3tTqQ7yTAXLpuOWG1NttoMdEKDQkk3EZhrLW63rsvJ77vuFxoc44Nwo7BGuQyBzF3bNlYLccamhXBk0shpe_1ZhOeonqIbTm59I58PKOPwwqUt6gLF2fLg3OmstDk7ueraKARO4qpUToOguMdYKyE2_LZnBk7QR
and provide commented, minimal, self-contained, reproducible code.
__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Re: [R] Regex Split?
Leonard,
It can be helpful to spell out your intent in English or some of us have to go
back to the documentation to remember what some of the operators do.
Your text being searched seems to be an example of items between comas with an
optional space after some commas and in one case, nothing between commas.
So what is your goal for the example, and in general? You mention a bit
unclearly at the end some of what you expect and I think it would be clearer if
you also showed exactly the output you would want.
I saw some other replies that addressed what you wanted and am going to reply
in another direction.
Why do things the hard way using things like lookahead or look behind? Would
several steps get you the result way more clearly?
For the sake of argument, you either want what reading in a CSV file would
supply, or something else. Since you are not simply splitting on commas, it
sounds like something else. But what exactly else? Something as simple as this
on just a comma produces results including empty strings and embedded leading
or trailing spaces:
strsplit("a bc,def, adef ,,gh", ",")
[[1]]
[1] "a bc" "def"" adef " "" "gh"
That can of course be handled by, for example, trimming the result after
unlisting the odd way strsplit returns results:
library("stringr")
str_squish(unlist(strsplit("a bc,def, adef ,,gh", ",")))
[1] "a bc" "def" "adef" "" "gh"
Now do you want the empty string to be something else, such as an NA? That can
be done too with another step.
And a completely different variant can be used to read in your one-line CSV as
text using standard overkill tools:
> read.table(text="a bc,def, adef ,,gh", sep=",")
V1 V2 V3 V4 V5
1 a bc def adef NA gh
The above is a vector of texts. But if you simply want to reassemble your
initial string cleaned up a bit, you can use paste to put back commas, as in a
variation of the earlier example:
> paste(str_squish(unlist(strsplit("a bc,def, adef ,,gh", ","))), collapse=",")
[1] "a bc,def,adef,,gh"
So my question is whether using advanced methods is really necessary for your
case, or even particularly efficient. If efficiency matters, often, it is
better to use tools without regular expressions such as paste0() when they meet
your needs.
Of course, unless I know what you are actually trying to do, my remarks may be
not useful.
-Original Message-
From: R-help On Behalf Of Leonard Mada via R-help
Sent: Thursday, May 4, 2023 5:00 PM
To: R-help Mailing List
Subject: [R] Regex Split?
Dear R-Users,
I tried the following 3 Regex expressions in R 4.3:
strsplit("a bc,def, adef ,,gh", " |(?=,)|(?<=,)(?![ ])", perl=T)
# "a""bc" ",""def" ",""" "adef" ",""," "gh"
strsplit("a bc,def, adef ,,gh", " |(?https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Re: [R] Regex Split?
Dear Bill,
Indeed, there are other cases as well - as documented.
Various Regex sites give the warning to avoid the legacy syntax
"[[:<:]]", so this is the alternative syntax:
strsplit(split="\\b(?=\\w)", "One, two; three!", perl=TRUE)
# "O" "n" "e" ", " "t" "w" "o" "; " "t" "h" "r" "e" "e" "!"
gsub("\\b(?=\\w)", "#", "One, two; three!", perl=TRUE)
# "#One, #two; #three!"
Sincerely,
Leonard
On 5/5/2023 6:19 PM, Bill Dunlap wrote:
> https://eu01.z.antigena.com/l/BgIBOxsm88PwDTBiTTrQ784MFk2oGZVOA3RMHiarAZuyoEemKrcnpfJeD8X0FgxRDG33qHZho~NriRCbhv9_Ffr3EOfqn2vpaNUAlCDjQ8nOyVUgPM2iGnHi-qpN54kl1YVO_gHimn0m2ZJ68ntGtysras~0mRMDuAgwbTXsQcQ~
>
> (from 2016, still labelled 'UNCONFIRMED") contains some other examples
> of strsplit misbehaving when using 0-length perl look-behinds. E.g.,
>
> > strsplit(split="[[:<:]]", "One, two; three!", perl=TRUE)[[1]]
> [1] "O" "n" "e" ", " "t" "w" "o" "; " "t" "h" "r" "e" "e" "!"
> > gsub(pattern="[[:<:]]", "#", "One, two; three!", perl=TRUE)
> [1] "#One, #two; #three!"
>
> The bug report includes the comment
> It may be possible that strsplit is not using the startoffset argument
> to pcre_exec
>
>pcre/pcre/doc/html/pcreapi.html
> A non-zero starting offset is useful when searching for another match
> in the same subject by calling pcre_exec() again after a previous
> success. Setting startoffset differs from just passing over a
> shortened string and setting PCRE_NOTBOL in the case of a pattern that
> begins with any kind of lookbehind.
>
> or it could be something else.
>
>
> On Fri, May 5, 2023 at 3:25 AM Ivan Krylov wrote:
>
> On Thu, 4 May 2023 23:59:33 +0300
> Leonard Mada via R-help wrote:
>
> > strsplit("a bc,def, adef ,,gh", " |(?=,)|(?<=,)(?![ ])", perl=T)
> > # "a" "bc" "," "def" "," "" "adef" "," "," "gh"
> >
> > strsplit("a bc,def, adef ,,gh", " |(? perl=T)
> > # "a" "bc" "," "def" "," "" "adef" "," "," "gh"
> >
> > strsplit("a bc,def, adef ,,gh", " |(? > perl=T)
> > # "a" "bc" "," "def" "," "" "adef" "," "," "gh"
> >
> >
> > Is this correct?
>
> Perl seems to return the results you expect:
>
> $ perl -E '
> say("$_:\n ", join " ", map qq["$_"], split $_, q[a bc,def, adef
> ,,gh])
> for (
> qr[ |(?=,)|(?<=,)(?![ ])],
> qr[ |(? qr[ |(? )'
> (?^u: |(?=,)|(?<=,)(?![ ])):
> "a" "bc" "," "def" "," "adef" "," "," "gh"
> (?^u: |(? "a" "bc" "," "def" "," "adef" "," "," "gh"
> (?^u: |(? "a" "bc" "," "def" "," "adef" "," "," "gh"
>
> The same thing happens when I ask R to replace the separators instead
> of splitting by them:
>
> sapply(setNames(nm = c(
> " |(?=,)|(?<=,)(?![ ])",
> " |(? " |(? ), gsub, '[]', "a bc,def, adef ,,gh", perl = TRUE)
> # |(?=,)|(?<=,)(?![ ]) |(? )(?=,)|(?<=,)(?![ ])
> # "a[]bc[],[]def[],[]adef[],[],[]gh"
> "a[]bc[],[]def[],[]adef[],[],[]gh"
> # |(? # "a[]bc[],[]def[],[]adef[],[],[]gh"
>
> I think that something strange happens when the delimeter pattern
> matches more than once in the same place:
>
> gsub(
> '(?=<--)|(?<=-->)', '[]', 'split here --><-- split here',
> perl = TRUE
> )
> # [1] "split here -->[]<-- split here"
>
> (Both Perl's split() and s///g agree with R's gsub() here, although I
> would have accepted "split here -->[][]<-- split here" too.)
>
> On the other hand, the following doesn't look right:
>
> strsplit(
> 'split here --><-- split here', '(?=<--)|(?<=-->)',
> perl = TRUE
> )
> # [[1]]
> # [1] "split here -->" "<" "-- split here"
>
> The "<" is definitely not followed by "<--", and the rightmost "--" is
> definitely not preceded by "-->".
>
> Perhaps strsplit() incorrectly advances the match position after one
> match?
>
> --
> Best regards,
> Ivan
>
> __
> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
>
> https://eu01.z.antigena.com/l/WZma5cGVT7M3Pi1uuAoPo_edV2O7qj81C7uavPIJ3LEMXNUs9d2H6DCGBB12hJA-6tmSLDAJFSwSMeHfx9~UdkUSOMRYZx7tgL1P4G1w4VXdaEBqiHCYYXMGh59CijZYZiIc53dOO~~YTK7T17MIVg-A4Mj5av2VVOt4XNt
>
> PLEASE do read the posting guide
>
> https://eu01.z.antigena.com/l/boS91wizs77ZHW7jjYQJGhwKWDd7jhs-Bz84RKSuLO6Wr42WQEw~jCTfuUJGa_hsJ~G48rDp4Yd3YqBk~W12~24~eoBAwV8FTFmlNLCyjnyym8S-Ebcq0yz2IaH5TEYHyBIe7Z52GHo7s2sQIpyl93Js_4_UaWCcc2uXHZs1
>
> and provide commented, minimal, self-contained, reproducible code.
>
[[alternative HTML version deleted]]
__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented,
Re: [R] Regex Split?
> Bill Dunlap on Fri, 5 May 2023 08:19:21 -0700 writes: https://bugs.r-project.org/show_bug.cgi?id=16745 (from 2016, still labelled 'UNCONFIRMED") contains some other examples of strsplit misbehaving when using 0-length perl look-behinds. E.g., Thank you, Bill -- yes, uhmm, ... a bit embarrassing. I've finally changed the R bugzilla report's state to "CONFIRMED" now, and also added the "HELPWANTED" keyword. I think we (R Core) should be sorry to have (forgotten / not cared about) the issue completely. It's not hard to at least agree that the current behavior is buggy, e.g., in the example you show here : >> strsplit(split="[[:<:]]", "One, two; three!", perl=TRUE)[[1]] > [1] "O" "n" "e" ", " "t" "w" "o" "; " "t" "h" "r" "e" "e" "!" >> gsub(pattern="[[:<:]]", "#", "One, two; three!", perl=TRUE) > [1] "#One, #two; #three!" [...] [...] Maybe this should be continued either on Bugzilla (i.e., the URL above), or if needed, additionally on R-devel. Yes, I also added that we'd grateful for (tested) patches and/or code reviewers. Martin -- Martin Maechler ETH Zurich and R Core team __ R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Regex Split?
https://bugs.r-project.org/show_bug.cgi?id=16745 (from 2016, still labelled
'UNCONFIRMED") contains some other examples of strsplit misbehaving when
using 0-length perl look-behinds. E.g.,
> strsplit(split="[[:<:]]", "One, two; three!", perl=TRUE)[[1]]
[1] "O" "n" "e" ", " "t" "w" "o" "; " "t" "h" "r" "e" "e" "!"
> gsub(pattern="[[:<:]]", "#", "One, two; three!", perl=TRUE)
[1] "#One, #two; #three!"
The bug report includes the comment
It may be possible that strsplit is not using the startoffset argument
to pcre_exec
pcre/pcre/doc/html/pcreapi.html
A non-zero starting offset is useful when searching for another match
in the same subject by calling pcre_exec() again after a previous
success. Setting startoffset differs from just passing over a
shortened string and setting PCRE_NOTBOL in the case of a pattern that
begins with any kind of lookbehind.
or it could be something else.
On Fri, May 5, 2023 at 3:25 AM Ivan Krylov wrote:
> On Thu, 4 May 2023 23:59:33 +0300
> Leonard Mada via R-help wrote:
>
> > strsplit("a bc,def, adef ,,gh", " |(?=,)|(?<=,)(?![ ])", perl=T)
> > # "a""bc" ",""def" ",""" "adef" ",""," "gh"
> >
> > strsplit("a bc,def, adef ,,gh", " |(? > # "a""bc" ",""def" ",""" "adef" ",""," "gh"
> >
> > strsplit("a bc,def, adef ,,gh", " |(? > perl=T)
> > # "a""bc" ",""def" ",""" "adef" ",""," "gh"
> >
> >
> > Is this correct?
>
> Perl seems to return the results you expect:
>
> $ perl -E '
> say("$_:\n ", join " ", map qq["$_"], split $_, q[a bc,def, adef ,,gh])
> for (
> qr[ |(?=,)|(?<=,)(?![ ])],
> qr[ |(? qr[ |(? )'
> (?^u: |(?=,)|(?<=,)(?![ ])):
> "a" "bc" "," "def" "," "adef" "," "," "gh"
> (?^u: |(? "a" "bc" "," "def" "," "adef" "," "," "gh"
> (?^u: |(? "a" "bc" "," "def" "," "adef" "," "," "gh"
>
> The same thing happens when I ask R to replace the separators instead
> of splitting by them:
>
> sapply(setNames(nm = c(
> " |(?=,)|(?<=,)(?![ ])",
> " |(? " |(? ), gsub, '[]', "a bc,def, adef ,,gh", perl = TRUE)
> # |(?=,)|(?<=,)(?![ ]) |(? # "a[]bc[],[]def[],[]adef[],[],[]gh" "a[]bc[],[]def[],[]adef[],[],[]gh"
> #|(? # "a[]bc[],[]def[],[]adef[],[],[]gh"
>
> I think that something strange happens when the delimeter pattern
> matches more than once in the same place:
>
> gsub(
> '(?=<--)|(?<=-->)', '[]', 'split here --><-- split here',
> perl = TRUE
> )
> # [1] "split here -->[]<-- split here"
>
> (Both Perl's split() and s///g agree with R's gsub() here, although I
> would have accepted "split here -->[][]<-- split here" too.)
>
> On the other hand, the following doesn't look right:
>
> strsplit(
> 'split here --><-- split here', '(?=<--)|(?<=-->)',
> perl = TRUE
> )
> # [[1]]
> # [1] "split here -->" "<" "-- split here"
>
> The "<" is definitely not followed by "<--", and the rightmost "--" is
> definitely not preceded by "-->".
>
> Perhaps strsplit() incorrectly advances the match position after one
> match?
>
> --
> Best regards,
> Ivan
>
> __
> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
> http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>
[[alternative HTML version deleted]]
__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Re: [ESS] Emacs, ESS and Rmarkdown: Is this the way compile is supposed to work?
Hi Kevin,
First of all, sorry for the late reply, I was OOO for a couple of weeks and
I didn't have my laptop with me.
Anyway, I was trying to follow up with your init file but it has a couple
of things that, in my opinion, might be redundant or not necessary, and I
am not sure if silly mistakes there could not be conflicting with the
proper function of poly-R and markdown.
For starters, could you comment all the part of the init file that relates
to RMarkdown (what you shared above) and add instead the following few
lines:
===
(require 'polymode)
(require 'poly-R)
(require 'poly-markdown)
;; MARKDOWN
(add-to-list 'auto-mode-alist '("\\.md" . poly-markdown-mode))
;; R modes
(add-to-list 'auto-mode-alist '("\\.Rnw" . poly-noweb+r-mode))
(add-to-list 'auto-mode-alist '("\\.Rmd" . poly-markdown+r-mode))
===
Then, try opening and working with an ".Rmd" file but please, also test
with an ".md" file, plain markdown. If both work, congratulations.
Otherwise, let me know what kind of errors you get with each type of file.
We want to identify first if the problem is Rmd or markdown as a whole.
Long time ago I got the problem that one of my Linux systems did not have
any markdown parser installed and that was giving me problems, of course
external to emacs but hard to identify.
Let me know and, since I'm back in the office, I'll try to reply faster
this time.
All the best
Manuel
On Thu, Apr 27, 2023 at 7:32 PM Kevin Zembower wrote:
> Hi, Manuel, thanks for your advice and help.
>
> Here's the section of my ~/.emacs.d/init.el file that I think pertains
> to R and Rmarkdown:
> ===
> ;; Next two sections for editing manpages in markdown (duplicates
> ;; sections below it, regarding RMarkdown in ESS?)
> (autoload 'markdown-mode "markdown-mode"
> "Major mode for editing Markdown files" t)
> (add-to-list 'auto-mode-alist
>
> '("\\.\\(?:md\\|markdown\\|mkd\\|mdown\\|mkdn\\|mdwn\\)\\'" .
> markdown-mode))
>
> (autoload 'gfm-mode "markdown-mode"
> "Major mode for editing GitHub Flavored Markdown files" t)
> (add-to-list 'auto-mode-alist '("README\\.md\\'" . gfm-mode))
>
> ;; For working in Markdown (RMarkdown in ESS):
> (setq load-path (append load-path (list
> "/usr/share/emacs23/site-lisp/emacs-goodies-el")))
> ;; (autoload 'markdown-mode "markdown-mode"
> ;;"Major mode for editing Markdown files" t)
> ;; (add-to-list 'auto-mode-alist '("\\.text\\'" . markdown-mode))
> ;; (add-to-list 'auto-mode-alist '("\\.markdown\\'" . markdown-mode))
> ;; (add-to-list 'auto-mode-alist '("\\.md\\'" . markdown-mode))
>
> ;; For working with RMarkdown (polymode):
> ;; Commented out, to update to poly-R
> (setq load-path (append '("~/.emacs.d/polymode/"
> "~/.emacs.d/polymode/modes") load-path))
> (require 'poly-R)
> ;; Below from email at
> https://stat.ethz.ch/pipermail/ess-help/2023-April/013190.html
> (define-polymode poly-gfm+r-mode poly-markdown+r-mode
>:lighter " PM-Rmd(gfm)"
>:hostmode 'poly-gfm-hostmode)
> (require 'poly-markdown)
> (add-to-list 'auto-mode-alist '("\\.Rmd" . poly-markdown+r-mode))
> ;; associate the new polymode to Rmd files:
> (add-to-list 'auto-mode-alist
>'("\\.[rR]md\\'" . poly-gfm+r-mode))
> ;; uses braces around code block language strings:
> ;; (setq markdown-code-block-braces t)
> =
>
> I've also attached the whole file to this message.
>
> Thanks, again, for any advice or guidance you can offer.
>
> -Kevin
>
>
> On 4/24/23 02:45, Manuel Teodoro wrote:
> > Hi Kevin,
> >
> > I also get sometimes the problem with "cl" library but it is never a
> > critical error that stops me from using Rmd.
> >
> > I think the main question is, what is in your emacs configuration file?
> > Probably the problem comes from there.
> >
> > For example, for R markdown I have simply the following lines, maybe you
> > can try them and see if the error persist.
> >
> > ;; R markdown
> > (use-package polymode)
> > (use-package poly-R)
> > (use-package poly-markdown)
> >
> > ;; MARKDOWN
> > (add-to-list 'auto-mode-alist '("\\.md" . poly-markdown-mode))
> >
> > ;; R modes
> > (add-to-list 'auto-mode-alist '("\\.Snw" . poly-noweb+r-mode))
> > (add-to-list 'auto-mode-alist '("\\.Rnw" . poly-noweb+r-mode))
> > (add-to-list 'auto-mode-alist '("\\.Rmd" . poly-markdown+r-mode))
> > ;
> >
> > I just noticed, this is just in case you have installed use-package. If
> > not, change the first 3 lines for something like
> >
> > (require 'polymode)
> > (require 'poly-R)
> > (require 'poly-markdown)
> >
> > Although probably require could also give you trouble, in which case
> > simply remove the 3 lines. The most important are the following lines
> > that are telling emacs to recognize *.Rmd files as markdown.
> >
> > If you already have this lines then it would be helpful to see your
>
> > whole config file because maybe something there is conflicting with
>
> > markdown or
Re: [R] rgl not plotting properly
在 5/5/23 21:08, Duncan Murdoch 写道:
On 05/05/2023 7:41 a.m., Jiayue Wang wrote:
在 5/5/23 19:15, Duncan Murdoch 写道:
On 05/05/2023 6:48 a.m., Jiayue Wang wrote:
Thanks Duncan. glxgears works at the terminal; tcltk::tktoplevel()
outputs the following:
$ID
[1] ".1"
$env
attr(,"class")
[1] "tkwin"
Jiayue
But no window opens for tcltk::tktoplevel()? Sounds like R isn't seeing
your DISPLAY variable. Does Sys.getenv("DISPLAY") show the same thing
in R as you see when printing that environment variable (using e.g.
`echo $DISPLAY`) in the terminal?
Oh yes, a window did open for tcltk::tktoplevel(), I forgot to mention
it, sorry.
With Sys.getenv("DISPLAY"), I got
[1] ":0"
In that case, it sounds like it's purely an rgl issue, but I don't
really have anything else to suggest about how to fix it. There is a
workaround: Run the `rglwidget()` function whenever you want to see a
plot, and it should open in your browser.
Thanks all the same, Duncan!
Jiayue
__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Re: [R] rgl not plotting properly
On 05/05/2023 7:41 a.m., Jiayue Wang wrote:
在 5/5/23 19:15, Duncan Murdoch 写道:
On 05/05/2023 6:48 a.m., Jiayue Wang wrote:
Thanks Duncan. glxgears works at the terminal; tcltk::tktoplevel()
outputs the following:
$ID
[1] ".1"
$env
attr(,"class")
[1] "tkwin"
Jiayue
But no window opens for tcltk::tktoplevel()? Sounds like R isn't seeing
your DISPLAY variable. Does Sys.getenv("DISPLAY") show the same thing
in R as you see when printing that environment variable (using e.g.
`echo $DISPLAY`) in the terminal?
Oh yes, a window did open for tcltk::tktoplevel(), I forgot to mention
it, sorry.
With Sys.getenv("DISPLAY"), I got
[1] ":0"
In that case, it sounds like it's purely an rgl issue, but I don't
really have anything else to suggest about how to fix it. There is a
workaround: Run the `rglwidget()` function whenever you want to see a
plot, and it should open in your browser.
Duncan Murdoch
__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Re: [R] rgl not plotting properly
在 5/5/23 19:15, Duncan Murdoch 写道:
On 05/05/2023 6:48 a.m., Jiayue Wang wrote:
Thanks Duncan. glxgears works at the terminal; tcltk::tktoplevel()
outputs the following:
$ID
[1] ".1"
$env
attr(,"class")
[1] "tkwin"
Jiayue
But no window opens for tcltk::tktoplevel()? Sounds like R isn't seeing
your DISPLAY variable. Does Sys.getenv("DISPLAY") show the same thing
in R as you see when printing that environment variable (using e.g.
`echo $DISPLAY`) in the terminal?
Oh yes, a window did open for tcltk::tktoplevel(), I forgot to mention
it, sorry.
With Sys.getenv("DISPLAY"), I got
[1] ":0"
Jiayue
Duncan Murdoch
在 5/5/23 16:45, Duncan Murdoch 写道:
On 05/05/2023 2:24 a.m., Jiayue Wang wrote:
Hi
I can't get rgl working. I installed rgl, loaded it, but it seems all
plot3d codes have failed:
remotes::install_github("dmurdoch/rgl")
library(rgl)
open3d()
x <- sort(rnorm(1000))
y <- rnorm(1000)
z <- rnorm(1000) + atan2(x, y)
plot3d(x, y, z, col = rainbow(1000))
I see no plotting at all. Am I missing some important system library or
whatever? I'm using Linux Mint 23. Thanks!
If you install the version from CRAN, do you get the same results?
rgl needs X11 on your system. Do other programs using it work, e.g. try
running "glxgears" at the terminal, and try running
"tcltk::tktoplevel()" in R.
BTW, this list isn't for contributed package support, so if it turns out
that only rgl is failing, it may be best to move the discussion to
https://github.com/dmurdoch/rgl/issues .
Duncan Murdoch
__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Re: [R] rgl not plotting properly
On 05/05/2023 6:48 a.m., Jiayue Wang wrote:
Thanks Duncan. glxgears works at the terminal; tcltk::tktoplevel()
outputs the following:
$ID
[1] ".1"
$env
attr(,"class")
[1] "tkwin"
Jiayue
But no window opens for tcltk::tktoplevel()? Sounds like R isn't seeing
your DISPLAY variable. Does Sys.getenv("DISPLAY") show the same thing
in R as you see when printing that environment variable (using e.g.
`echo $DISPLAY`) in the terminal?
Duncan Murdoch
在 5/5/23 16:45, Duncan Murdoch 写道:
On 05/05/2023 2:24 a.m., Jiayue Wang wrote:
Hi
I can't get rgl working. I installed rgl, loaded it, but it seems all
plot3d codes have failed:
remotes::install_github("dmurdoch/rgl")
library(rgl)
open3d()
x <- sort(rnorm(1000))
y <- rnorm(1000)
z <- rnorm(1000) + atan2(x, y)
plot3d(x, y, z, col = rainbow(1000))
I see no plotting at all. Am I missing some important system library or
whatever? I'm using Linux Mint 23. Thanks!
If you install the version from CRAN, do you get the same results?
rgl needs X11 on your system. Do other programs using it work, e.g. try
running "glxgears" at the terminal, and try running
"tcltk::tktoplevel()" in R.
BTW, this list isn't for contributed package support, so if it turns out
that only rgl is failing, it may be best to move the discussion to
https://github.com/dmurdoch/rgl/issues .
Duncan Murdoch
__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Re: [R] Regex Split?
If you only want the character strings, this seems a little simpler:
> strsplit("a bc,def, adef ,,gh", "[ ,]+", perl=T)
[[1]]
[1] "a""bc" "def" "adef" "gh"
If you need delimeters (the commas) you could then add them back in again
afterwards.
Tim
--
Message: 2
Date: Thu, 4 May 2023 23:59:33 +0300
From: Leonard Mada
To: R-help Mailing List
Subject: [R] Regex Split?
Message-ID: <7b1cdbe7-0086-24b4-9da6-369296ead...@syonic.eu>
Content-Type: text/plain; charset="utf-8"; Format="flowed"
Dear R-Users,
I tried the following 3 Regex expressions in R 4.3:
strsplit("a bc,def, adef ,,gh", " |(?=,)|(?<=,)(?![ ])", perl=T)
# "a" "bc" "," "def" "," "" "adef" "," "," "gh"
strsplit("a bc,def, adef ,,gh", " |(?https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Re: [R] rgl not plotting properly
Thanks Duncan. glxgears works at the terminal; tcltk::tktoplevel()
outputs the following:
$ID
[1] ".1"
$env
attr(,"class")
[1] "tkwin"
Jiayue
在 5/5/23 16:45, Duncan Murdoch 写道:
On 05/05/2023 2:24 a.m., Jiayue Wang wrote:
Hi
I can't get rgl working. I installed rgl, loaded it, but it seems all
plot3d codes have failed:
remotes::install_github("dmurdoch/rgl")
library(rgl)
open3d()
x <- sort(rnorm(1000))
y <- rnorm(1000)
z <- rnorm(1000) + atan2(x, y)
plot3d(x, y, z, col = rainbow(1000))
I see no plotting at all. Am I missing some important system library or
whatever? I'm using Linux Mint 23. Thanks!
If you install the version from CRAN, do you get the same results?
rgl needs X11 on your system. Do other programs using it work, e.g. try
running "glxgears" at the terminal, and try running
"tcltk::tktoplevel()" in R.
BTW, this list isn't for contributed package support, so if it turns out
that only rgl is failing, it may be best to move the discussion to
https://github.com/dmurdoch/rgl/issues .
Duncan Murdoch
__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Re: [R] Regex Split?
On Thu, 4 May 2023 23:59:33 +0300
Leonard Mada via R-help wrote:
> strsplit("a bc,def, adef ,,gh", " |(?=,)|(?<=,)(?![ ])", perl=T)
> # "a" "bc" "," "def" "," "" "adef" "," "," "gh"
>
> strsplit("a bc,def, adef ,,gh", " |(? # "a" "bc" "," "def" "," "" "adef" "," "," "gh"
>
> strsplit("a bc,def, adef ,,gh", " |(? perl=T)
> # "a" "bc" "," "def" "," "" "adef" "," "," "gh"
>
>
> Is this correct?
Perl seems to return the results you expect:
$ perl -E '
say("$_:\n ", join " ", map qq["$_"], split $_, q[a bc,def, adef ,,gh])
for (
qr[ |(?=,)|(?<=,)(?![ ])],
qr[ |(?)', '[]', 'split here --><-- split here',
perl = TRUE
)
# [1] "split here -->[]<-- split here"
(Both Perl's split() and s///g agree with R's gsub() here, although I
would have accepted "split here -->[][]<-- split here" too.)
On the other hand, the following doesn't look right:
strsplit(
'split here --><-- split here', '(?=<--)|(?<=-->)',
perl = TRUE
)
# [[1]]
# [1] "split here -->" "<" "-- split here"
The "<" is definitely not followed by "<--", and the rightmost "--" is
definitely not preceded by "-->".
Perhaps strsplit() incorrectly advances the match position after one
match?
--
Best regards,
Ivan
__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Re: [R] rgl not plotting properly
On 05/05/2023 2:24 a.m., Jiayue Wang wrote:
Hi
I can't get rgl working. I installed rgl, loaded it, but it seems all
plot3d codes have failed:
remotes::install_github("dmurdoch/rgl")
library(rgl)
open3d()
x <- sort(rnorm(1000))
y <- rnorm(1000)
z <- rnorm(1000) + atan2(x, y)
plot3d(x, y, z, col = rainbow(1000))
I see no plotting at all. Am I missing some important system library or
whatever? I'm using Linux Mint 23. Thanks!
If you install the version from CRAN, do you get the same results?
rgl needs X11 on your system. Do other programs using it work, e.g. try
running "glxgears" at the terminal, and try running
"tcltk::tktoplevel()" in R.
BTW, this list isn't for contributed package support, so if it turns out
that only rgl is failing, it may be best to move the discussion to
https://github.com/dmurdoch/rgl/issues .
Duncan Murdoch
__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
[R] rgl not plotting properly
Hi
I can't get rgl working. I installed rgl, loaded it, but it seems all
plot3d codes have failed:
remotes::install_github("dmurdoch/rgl")
library(rgl)
open3d()
x <- sort(rnorm(1000))
y <- rnorm(1000)
z <- rnorm(1000) + atan2(x, y)
plot3d(x, y, z, col = rainbow(1000))
I see no plotting at all. Am I missing some important system library or
whatever? I'm using Linux Mint 23. Thanks!
Thanks,
Jiayue
__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
