Re: RFC 145 (alternate approach)
I'd suggest also, that (?[) (with no specified brackets) have the
default meaning
of the "four standard brackets" :
(?['('=')','{'='}','['=']',''='')
Note also the subtle syntax change. We are either dealing with strings
or with patterns. The consensus seems to be against patterns (I can
understand that). Given that, we need to quote the right hand side of
the = operator I think. The quotes on the left side would be optional,
I think.
Richard Proctor wrote:
On Tue 05 Sep, Nathan Wiger wrote:
Eric Roode wrote:
Now *that* sounds cool, I like it!
What if the RFC only suggested the addition of two new constructs, (?[)
and (?]), which did nested matches. The rest would be bound by standard
regex constructs and your imagination!
That is, the ?] simply takes whatever the closest ?[ matched and
reverses it, verbatim, including ordering, case, and number of
characters. The only trick would be a way to get what "reverses it"
means correct.
No ?] should match the closest ?[ it should nest the ?[s bound by any
brackets in the regex and act accordingly.
Also this does not work as a definition of simple bracket matching as you
need ( to match ) not ( to match (. A ?[ list should specify for each
element what the matching element is perhaps
(?[( = ),{ = }, 01 = 10)
sort of hashish in style.
Perhaps the brackets could be defined as a hash allowing (?[%Hash)
Richard
--
[EMAIL PROTECTED]
--
David Corbin
Mach Turtle Technologies, Inc.
http://www.machturtle.com
[EMAIL PROTECTED]
Re: RFC 145 (alternate approach)
At 09:05 AM 9/6/00 -0400, David Corbin wrote:
I'd suggest also, that (?[) (with no specified brackets) have the
default meaning
of the "four standard brackets" :
(?['('=')','{'='}','['=']',''='')
Note also the subtle syntax change. We are either dealing with strings
or with patterns. The consensus seems to be against patterns (I can
understand that). Given that, we need to quote the right hand side of
the = operator I think. The quotes on the left side would be optional,
I think.
It would be useful (and increasingly more common) to be able to match
qr|\s*(\w+)([^]*)| to qr|\s*/\1\s*|, and handle the case where those
can nest as well. Something like
listmatch this with
list
/list not this but
/list this.
Richard Proctor wrote:
On Tue 05 Sep, Nathan Wiger wrote:
Eric Roode wrote:
Now *that* sounds cool, I like it!
What if the RFC only suggested the addition of two new constructs, (?[)
and (?]), which did nested matches. The rest would be bound by standard
regex constructs and your imagination!
That is, the ?] simply takes whatever the closest ?[ matched and
reverses it, verbatim, including ordering, case, and number of
characters. The only trick would be a way to get what "reverses it"
means correct.
No ?] should match the closest ?[ it should nest the ?[s bound by any
brackets in the regex and act accordingly.
Also this does not work as a definition of simple bracket matching as you
need ( to match ) not ( to match (. A ?[ list should specify for each
element what the matching element is perhaps
(?[( = ),{ = }, 01 = 10)
sort of hashish in style.
Perhaps the brackets could be defined as a hash allowing (?[%Hash)
Richard
--
[EMAIL PROTECTED]
--
David Corbin
Mach Turtle Technologies, Inc.
http://www.machturtle.com
[EMAIL PROTECTED]
XML/HTML-specific ? and ? operators? (was Re: RFC 145 (alternate approach))
It would be useful (and increasingly more common) to be able to match qr|\s*(\w+)([^]*)| to qr|\s*/\1\s*|, and handle the case where those can nest as well. Something like listmatch this with list /list not this but /list this. I suspect this is going to need a ?[ and ?] of its own. I've been thinking about this since your email on the subject yesterday, and I don't see how either RFC 145 or this alternative method could support it, since there are two tags - and / - which are paired asymmetrically, and neither approach gives any credence to what's contained inside the tag. So tag would be matched itself as " matches ". What if we added special XML/HTML-parsing ? and ? operators? Unfortunately, as Richard notes, ? is already taken, but I will use it for the examples to make things symmetrical. ? = opening tag (with name specified) ? = closing tag (matches based on nesting) Your example would simply be: /(?list)[\s\w]*(?list)[\s\w]*(?)[\s\w]*(?)/; What makes me nervous about this is that ? and ? seem special-case. They are, but then again XML and HTML are also pervasive. So a special-case for something like this might not be any stranger than having a special-case for sin() and cos() - they're extremely important operations. The other thing that this doesn't handle is tags with no closing counterpart, like: br Perhaps for these the easiest thing is to tell people not to use ? and ?: /(?p)[\s*\w](?:br)(?)/; Would match p Some stuffbr /p Finally, tags which take arguments: div align="center"Stuff/div Would require some type of "this is optional" syntax: /(?div\s*\w*)Stuff(?)/ Perhaps only the first word specified is taken as the tag name? This is the XML/HTML spec anyways. -Nate
Re: XML/HTML-specific ? and ? operators? (was Re: RFC 145 (alternate approach))
Nathan Wiger wrote: It would be useful (and increasingly more common) to be able to match qr|\s*(\w+)([^]*)| to qr|\s*/\1\s*|, and handle the case where those can nest as well. Something like listmatch this with list /list not this but /list this. I suspect this is going to need a ?[ and ?] of its own. I've been thinking about this since your email on the subject yesterday, and I don't see how either RFC 145 or this alternative method could support it, since there are two tags - and / - which are paired asymmetrically, and neither approach gives any credence to what's contained inside the tag. So tag would be matched itself as " matches ". Actually, in one of my responses I did outline a syntax which would handle this with reasonably ease, I think. If the contents of (?[) is considered a pattern, then you can define a matching pattern. Consider either of these. m:(?[list]).*?(?]/list): or m:(?['list' = '/list').*(?]):# really ought to include (?i:) in there, but left out for readablity or more generically m:(?['\w+' = '/\1').*(?]): I'll grant you it's not the simplest syntax, but it's a lot simpler than using the 5.6 method... :) What if we added special XML/HTML-parsing ? and ? operators? Unfortunately, as Richard notes, ? is already taken, but I will use it for the examples to make things symmetrical. ? = opening tag (with name specified) ? = closing tag (matches based on nesting) Your example would simply be: /(?list)[\s\w]*(?list)[\s\w]*(?)[\s\w]*(?)/; What makes me nervous about this is that ? and ? seem special-case. They are, but then again XML and HTML are also pervasive. So a special-case for something like this might not be any stranger than having a special-case for sin() and cos() - they're extremely important operations. The other thing that this doesn't handle is tags with no closing counterpart, like: br Perhaps for these the easiest thing is to tell people not to use ? and ?: /(?p)[\s*\w](?:br)(?)/; Would match p Some stuffbr /p Finally, tags which take arguments: div align="center"Stuff/div Would require some type of "this is optional" syntax: /(?div\s*\w*)Stuff(?)/ Perhaps only the first word specified is taken as the tag name? This is the XML/HTML spec anyways. -Nate -- David Corbin Mach Turtle Technologies, Inc. http://www.machturtle.com [EMAIL PROTECTED]
Re: RFC 145 (alternate approach)
On Tue 05 Sep, Nathan Wiger wrote:
"normal" "reversed"
-- ---
103301
99aa99
(( ))
+ +
{{[!_ _!]}}
{__A1( )A1__}
That is, when a bracket is encountered, the "reverse" of that is
automatically interpreted as its closing counterpart. This is the same
reason why qq// and qq() and qq{} all work without special notation.
So we can replace @^g and @^G with simple precendence rules, the same
that are actually invoked automatically throughout Perl already.
(?[( = ),{ = }, 01 = 10)
sort of hashish in style.
I actually think this is redundant, for the reasons I mentioned above.
I'm not striking it down outright, but it seems simple rules could make
all this unnecessary.
I dont think you will ever come up with a set of rules that will satisfy
everybody all the time. what about html comments !-- , -- are they
brackets? What about people doing 66/99 pairs? The best you could
achieve is a set of default rules as you have suggested AND a way
of overriding them with an explicit hash of what is the closing
bracket for each opening bracket.
The two methods depend on what follows the (?[ is it a hash or not.
For the "Default" method the list of brackets could be as has been
suggested a regex, or perhaps a simple comma separated list. For this
you should define what is the "reverse" of each character, at
least for latin-1, what do you do about the full utf-8...? An \X type
construct that covers all the common brackets might be a usefull addition
({?
If there is a hash, the hash supplies both the brackets to be matched and
their corresponding close brackets
Richard
--
[EMAIL PROTECTED]
Re: XML/HTML-specific ? and ? operators? (was Re: RFC 145 (alternate approach))
- Original Message - From: "Jonathan Scott Duff" [EMAIL PROTECTED] Subject: Re: XML/HTML-specific ? and ? operators? (was Re: RFC 145 (alternate approach)) On Wed, Sep 06, 2000 at 08:40:37AM -0700, Nathan Wiger wrote: What if we added special XML/HTML-parsing ? and ? operators? What if we just provided deep enough hooks into the RE engine that specialized parsing constructs like these could easily be added by those who need them? -Scott Ok, I've avoided this thread for a while, but I'll make my comment now. I've played with several ideas of reg-ex extensions that would allow arbitrary "parsing". My first goal was to be able to parse perl-like text, then later a simple nested parentheses, then later nested xml as with this thread. I have been able to solve these problems using perl5.6's recursive reg-ex's, and inserted procedure code. Unfortunately this isn't very safe, nor is it 'pretty' to figure out by a non-perl-guru. What's more, what I'm attempting to do with these nested parens and xml is to _parse_ the data.. Well, guess what guys, we've had decades of research into the area of parsing, and we came out with yacc and lex. My point is that I think we're approaching this the wrong way. We're trying to apply more and more parser power into what classically has been the lexer / tokenizer, namely our beloved regular-expression engine. A great deal of string processing is possible with perls enhanced NFA engine, but at some point we're looking at perl code that is inside out: all code embedded within a reg-ex. That, boys and girls, is a parser, and I'm not convinced it's the right approach for rapid design, and definately not for large-scale robust design. As for XML, we already have lovely c-modules that take of that.. You even get your choice. Call per tag, or generate a tree (where you can search for sub-trees). What else could you want? (Ok, stupid question, but you could still accomplish it via a customized parser). My suggestion, therefore would be to discuss a method of encorportating more powerful and convinient parsing within _perl_; not necessarily directly within the reg-ex engine, and most likely not within a reg-ex statement. I know we have Yacc and Parser modules. But try this out for size: Perl's very name is about extraction and reporting. Reg-ex's are fundamental to this, but for complex jobs, so is parsing. After I think about this some more, I'm going to make an RFC for it. If anyone has any hardened opinions on the matter, I'd like to hear from you while my brain churns. -Michael
Re: XML/HTML-specific ? and ? operators? (was Re: RFC 145 (alternate approach))
I am working on an RFC
to allow boolean logic ( and || and !) to apply a number of patterns to
the same substring to allow easier mining of information out of such
constructs.
What, you don't like: :-)
$pattern = $conjunction eq "AND"
? join('' = map { "(?=.*$_)" } @patterns)
| join("|" =@patterns);
--tom
Re: XML/HTML-specific ? and ? operators? (was Re: RFC 145 (alternate approach))
Jonathan Scott Duff wrote: On Wed, Sep 06, 2000 at 08:40:37AM -0700, Nathan Wiger wrote: What if we added special XML/HTML-parsing ? and ? operators? What if we just provided deep enough hooks into the RE engine that specialized parsing constructs like these could easily be added by those who need them? In principle, that's a very Perlish thing to do... -Scott -- Jonathan Scott Duff [EMAIL PROTECTED] -- David Corbin Mach Turtle Technologies, Inc. http://www.machturtle.com [EMAIL PROTECTED]
Re: XML/HTML-specific ? and ? operators? (was Re: RFC 145 (alternate approach))
...My point is that I think we're approaching this
the wrong way. We're trying to apply more and more parser power into what
classically has been the lexer / tokenizer, namely our beloved
regular-expression engine.
I've been thinking the same thing. It seems to me that the attempts to
shoehorn parsers into regex syntax have either been unsuccessful
(yielding an underpowered extension) or illegible or both.
An approach that appears to have been more successful is to find ways
to integrate regexes *into* parser code more effectively. Damian
Conway's Parse::RecDescent module does this, and so does SNOBOL.
In SNOBOL, if you want to write a pattern that matches balanced
parenteses, it's easy and straightforward and legible:
parenstring = '(' *parenstring ')'
| *parenstring *parenstring
| span('()')
(span('()') is like [^()]* in Perl.)
The solution in Parse::RecDescent is similar.
Compare this with the solutions that work now:
# man page solution
$re = qr{
\(
(?:
(? [^()]+ )# Non-parens without backtracking
|
(??{ $re }) # Group with matching parens
)*
\)
}x;
This is not exactly the same, but I tried a direct translation:
$re = qr{ \( (??{$re}) \)
| (??{$re}) (??{$re})
| (? [^()]+)
}x;
and it looks worse and dumps core.
This works:
qr{
^
(?{ local $d=0 })
(?:
\(
(?{$d++})
|
\)
(?{$d--})
(?
(?{$d0})
(?!)
)
|
(? [^()]* )
)*
(?
(?{$d!=0})
(?!)
)
$
}x;
but it's rather difficult to take seriously.
The solution proposed in the recent RFC 145:
/([^\m]*)(\m)(.*?)(\M)([^\m\M]*)/g
is not a lot better. David Corbin's alternative looks about the same.
On a different topic from the same barrel, we just got a proposal that
([23,39]) should match only numbers between 23 and 39. It seems to me
that rather than trying to shoehorn one special-purpose syntax after
another into the regex language, which is already overloaded, that it
would be better to try to integrate regex matching better with Perl
itself. Then you could use regular Perl code to control things like
numeric ranges.
Note that at present, you can get the effect of [(23,39)] by writing
this:
(\d+)(?(?{$1 23 || $1 39})(?!))
which isn't pleasant to look at, but I think it points in the right
direction, because it is a lot more flexible than [(23,39)]. If you
need to fix it to match 23.2 but not 39.5, it is straightforward to do
that:
(\d+(\.\d*)?)(?(?{$1 23 || $1 39})(?!))
The [(23,39)] notation, however, is doomed.All you can do is
propose Yet Another Extension for Perl 7.
The big problem with
(\d+)(?(?{$1 23 || $1 39})(?!))
is that it is hard to read and understand.
The real problem here is that regexes are single strings. When you
try to compress a programming language into a single string this way,
you end up with something that looks like Befunge or TECO. We are
going in the same direction here.
Suppose there were an alternative syntax for regexes that did *not*
require that everything be compressed into a single string? Rather
than trying to pack all of Perl into the regex syntax, bit by bit,
using ever longer and more bizarre punctuation sequences, I think a
better solution would be to try to expose the parts of the regex
engine that we are trying to control.
I have some ideas about how to do this, and I will try to write up an
RFC this week.
Re: XML/HTML-specific ? and ? operators? (was Re: RFC 145 (alternate approach))
On Wed, Sep 06, 2000 at 03:47:57PM -0700, Randal L. Schwartz wrote: "Mark-Jason" == Mark-Jason Dominus [EMAIL PROTECTED] writes: Mark-Jason I have some ideas about how to do this, and I will try to Mark-Jason write up an RFC this week. "You want Icon, you know where to find it..." :) Hey, it's one of the few languages we haven't yet stolen a neat feature or few from... (I don't really count the few regex thingies as full-fledged stealing, more like an experimental sleight-of-hand.) But yes, a way that allows programmatic backtracking sort of "inside out" from a regex would be nice. -- $jhi++; # http://www.iki.fi/jhi/ # There is this special biologist word we use for 'stable'. # It is 'dead'. -- Jack Cohen
RFC 197 (v1) Numberic Value Ranges In Regular Expressions
This and other RFCs are available on the web at http://dev.perl.org/rfc/ =head1 TITLE Numberic Value Ranges In Regular Expressions =head1 VERSION Maintainer: David Nicol [EMAIL PROTECTED] Date: 5 september 2000 Mailing List: [EMAIL PROTECTED] Version: 1 Number: 197 Status: Developing =head1 ABSTRACT round and square bratches mated around two optional comma separated numbers match iff a gobbled number is within the described range. =head1 DESCRIPTION =head2 the syntax of the numeric range regex element Given a passage of regex text matching ($B1,$N1,$N2,$B2) = /(\[|\()(\-?\d*\.?\d*),(\-?\d*\.?\d*)(\]|\))/ and ($N1 = $N2 or $N1 eq '' or $N2 eq '') we've got something we hereinafter call a "range." =head2 what the range matches A range matches, in the target string, a passage C(\-?\d*\.?\d*) also known as a "number" if and only if the number is within the range. In the normal agebraic sense. =head2 "within the range" Square bracket means, that end of the range may include the range specifying number, and round parenthesis means, that end of the range includes numbers ov value up to (or down to) the number but not equal to it. =head2 infinity in the event that one or the other of the range specifying numbers is the empty string, that end of the range is unbounded. In the further event that we have defined infinity and negative infinity on our numbers, the square/round distinction will come into play. =head1 COMPATIBILITY To disambiguate ranges from character sets indluding digits, commas, and parentheses, either put a backslash on the right parentheses, or the comma, or arrange things so the left hand side of the comma is greater than the right hand side, that way this special case will not apply: /(37.3,200)/; # matches any number x, 37.3 x 200 /([37,))/; # matches and saves any number = 37. /(37\,200)/;# matches and saves the literal text '37,200' /[-35,9)]/; # matches any number x, -35 = x 9; followed by a ] /[3-5,9)]/; # matches a string containing any of 3,4,5,,,9 or ) =head1 IMPLEMENTATION When applying regular expressions to numeric data, ranges may optimize away all of the digit lookahead we must currently indulge in to implement them in perl5. If we have infinity defined, we'll have to recognize it in strings. =head1 BUT WAIT THERE'S MORE It is possible that the syntax described in this document may help slice multidimensional containers. (RFC 191) =head1 REFERENCES high school algebra
