Hi Gabriele,
you wrote:
>>> parse {This was really surprising, "but quotes are taken into
consideration" by parse.} none
>== ["This" "was" "really" "surprising" "but quotes are taken into
consideration" "by" "parse."]
The problem is how they are taken into consideration. When you use 'none as
a rule then parse uses a leading quote character as a control symbol that
suppresses the use of of spaces as token delimiters, until another quote
character is encountered!
That means that the substring that is surrounded by quotes will not be
split along its spaces. That substring is then returned by parse as a
string, surrounded by two quotes. If you are trying to retrieve a single
quote character that is part of your input stream, then instead you will
get a long quoted string surrounded by two quotes.
>> parse { abc "def ghi } none
== ["abc" "def ghi "]
Here the single quote "def caused REBOL to stop using the space between def
and ghi as a delimiter. Instead "def ghi " was returned as one long string,
surrounded by two quotes. The one original quote character was lost.
Parse behaves differently if you use the all refinement:
>> parse/all {abc "def ghi} none
== [{abc "def ghi}]
Now the single quote was preserved, however the tokens in the string were
not separated along the spaces! Parse simply returned a block containing
one long string, albeit preserving the single quote character.
If you use the all refinement and yet force parse to use the space
character as delimiters:
>> parse/all {abc "def ghi} " "
== ["abc" "def ghi"]
we again get the same results as we did when we used parse without the all
refinement.
The special function of the leading quote character - turning off the space
delimiter - appears to be suspended, when the delimiter character is not a
space:
>> parse/all { abc : "def : ghi } ":"
== [" abc " { "def } " ghi "]
This is the result we wanted all along, where the leading quote in front of
def is preserved as a leading quote.
I don't see why a leading quote has a special status only with respect to
spaces. What's so special about spaces? If a leading quote suppresses the
recognition of the space as a token delimiter, then it should do the same
job for any delimiting character I may be using.
Anyway, in order to preserve single quotes in a situation where spaces
would be the natural token delimiters, one should probably first add some
other delimiting character and then use parse/all string delim-char to
generate a quote preserving block with parse.
Here's another oddity:
>> parse/all {abc:"def:ghi} ":"
== ["abc" "def:ghi"]
Here I removed the spaces surrounding the delimiting colon in the string
and as a result the single quote character in front of def:ghi prevented
the colon from being identified a token delimiter. (Previously I used { abc
: "def : ghi } ":", where spaces surrounded the colon).
Elan
