Re: [cmake-developers] slow regex implementation in RegularExpression

2011-11-29 Thread David Cole
On Tue, Nov 29, 2011 at 3:44 PM, James Bigler wrote: > On Tue, Nov 29, 2011 at 1:05 PM, Bill Hoffman > wrote: >> >> On 11/29/2011 2:41 PM, Alexandru Ciobanu wrote: >>> >>> Hi, >>> >>> We found a workaround that does not require any source code >>> modifications. I added the description to the bug

Re: [cmake-developers] slow regex implementation in RegularExpression

2011-11-29 Thread James Bigler
On Tue, Nov 29, 2011 at 1:05 PM, Bill Hoffman wrote: > On 11/29/2011 2:41 PM, Alexandru Ciobanu wrote: > >> Hi, >> >> We found a workaround that does not require any source code >> modifications. I added the description to the bug report: >> http://public.kitware.com/Bug/**view.php?id=12381#c27872

Re: [cmake-developers] slow regex implementation in RegularExpression

2011-11-29 Thread Bill Hoffman
On 11/29/2011 2:41 PM, Alexandru Ciobanu wrote: Hi, We found a workaround that does not require any source code modifications. I added the description to the bug report: http://public.kitware.com/Bug/view.php?id=12381#c27872 Good, glad you are up and running again. ... Since it is similar to

Re: [cmake-developers] slow regex implementation in RegularExpression

2011-11-29 Thread Alexandru Ciobanu
Hi, We found a workaround that does not require any source code modifications. I added the description to the bug report: http://public.kitware.com/Bug/view.php?id=12381#c27872 In short, we filter out the longer lines in the build output, so that CTest does not spend hours regex-matching

Re: [cmake-developers] slow regex implementation in RegularExpression

2011-11-24 Thread Alexander Neundorf
On Wednesday 23 November 2011, David Cole wrote: > On Wed, Nov 23, 2011 at 2:09 PM, David Cole wrote: > > On Wed, Nov 23, 2011 at 2:03 PM, Bill Hoffman wrote: > >> On 11/23/2011 12:51 PM, Brad King wrote: > >>> On 11/23/2011 12:48 PM, Brad King wrote: > On 11/23/2011 12:43 PM, Brad King wro

Re: [cmake-developers] slow regex implementation in RegularExpression

2011-11-24 Thread Rolf Eike Beer
> On 11/24/2011 12:34 AM, Brad King wrote: >> On 11/23/2011 5:43 PM, Brad King wrote: >>> On 11/23/2011 12:44 PM, Brad King wrote: However, the above does not need to stand in the way of solving the problem you're addressing. We can simply set that goal aside for now by not exposing TRE in th

Re: [cmake-developers] slow regex implementation in RegularExpression

2011-11-23 Thread Michael Wild
On 11/24/2011 12:34 AM, Brad King wrote: > On 11/23/2011 5:43 PM, Brad King wrote: >> On 11/23/2011 12:44 PM, Brad King wrote: >>> However, the above does not need to stand in the way of solving the >>> problem you're addressing. We can simply set that goal aside for >>> now by not exposing TRE in

Re: [cmake-developers] slow regex implementation in RegularExpression

2011-11-23 Thread Brad King
On 11/23/2011 5:43 PM, Brad King wrote: > On 11/23/2011 12:44 PM, Brad King wrote: >> However, the above does not need to stand in the way of solving the >> problem you're addressing. We can simply set that goal aside for >> now by not exposing TRE in the CMake language anywhere. Use it >> just f

Re: [cmake-developers] slow regex implementation in RegularExpression

2011-11-23 Thread Brad King
On 11/23/2011 4:08 PM, Sean McBride wrote: > How about Alex's suggestion to use the new TRE code only where it > gives the biggest win? (See his 12:20 message, part #3)... I agreed with that suggestion in my 12:44 post: http://www.cmake.org/pipermail/cmake-developers/2011-November/002633.html

Re: [cmake-developers] slow regex implementation in RegularExpression

2011-11-23 Thread Bill Hoffman
On Wed, Nov 23, 2011 at 3:24 PM, Sean McBride wrote: > On Wed, 23 Nov 2011 14:03:20 -0500, Bill Hoffman said: > >>For 99% of folks the current regex is just fine. > > AFAICT, this performance bug affects 100% of Xcode generator users.  Even > looking at CMake's dashboard, you can see the differen

Re: [cmake-developers] slow regex implementation in RegularExpression

2011-11-23 Thread David Cole
On Wed, Nov 23, 2011 at 2:09 PM, David Cole wrote: > On Wed, Nov 23, 2011 at 2:03 PM, Bill Hoffman > wrote: >> On 11/23/2011 12:51 PM, Brad King wrote: >>> >>> On 11/23/2011 12:48 PM, Brad King wrote: On 11/23/2011 12:43 PM, Brad King wrote: > > On 11/23/2011 12:34 PM, Alexandr

Re: [cmake-developers] slow regex implementation in RegularExpression

2011-11-23 Thread Marcus D. Hanwell
On Wed, Nov 23, 2011 at 2:03 PM, Bill Hoffman wrote: > On 11/23/2011 12:51 PM, Brad King wrote: >> >> On 11/23/2011 12:48 PM, Brad King wrote: >>> >>> On 11/23/2011 12:43 PM, Brad King wrote: On 11/23/2011 12:34 PM, Alexandru Ciobanu wrote: > > The regex in question is: >    

Re: [cmake-developers] slow regex implementation in RegularExpression

2011-11-23 Thread David Cole
On Wed, Nov 23, 2011 at 2:03 PM, Bill Hoffman wrote: > On 11/23/2011 12:51 PM, Brad King wrote: >> >> On 11/23/2011 12:48 PM, Brad King wrote: >>> >>> On 11/23/2011 12:43 PM, Brad King wrote: On 11/23/2011 12:34 PM, Alexandru Ciobanu wrote: > > The regex in question is: >    

Re: [cmake-developers] slow regex implementation in RegularExpression

2011-11-23 Thread Bill Hoffman
On 11/23/2011 12:51 PM, Brad King wrote: On 11/23/2011 12:48 PM, Brad King wrote: On 11/23/2011 12:43 PM, Brad King wrote: On 11/23/2011 12:34 PM, Alexandru Ciobanu wrote: The regex in question is: ^[^][:/*?]+\$ "To include a literal ] in the list, make it either the first item" It

Re: [cmake-developers] slow regex implementation in RegularExpression

2011-11-23 Thread Brad King
On 11/23/2011 12:48 PM, Brad King wrote: > On 11/23/2011 12:43 PM, Brad King wrote: >> On 11/23/2011 12:34 PM, Alexandru Ciobanu wrote: >>> The regex in question is: >>> ^[^][:/*?]+\$ > > "To include a literal ] in the list, make it either the first item" It must be the "[:" in this regex th

Re: [cmake-developers] slow regex implementation in RegularExpression

2011-11-23 Thread Brad King
On 11/23/2011 12:43 PM, Brad King wrote: > On 11/23/2011 12:34 PM, Alexandru Ciobanu wrote: >> The regex in question is: >> ^[^][:/*?]+\$ >> >> And it appears at this location in the ITK source tree: >> CMake/ExternalData.cmake:347 >> >> And the expression is correct, because you're allow

Re: [cmake-developers] slow regex implementation in RegularExpression

2011-11-23 Thread Brad King
On 11/23/2011 12:20 PM, Alexandru Ciobanu wrote: > The solution is to use the old regex code everywhere, except for the > very specific place where it causes problems. I maintain that the proper long-term solution is to use a policy to fully replace the old impl with TRE. Exposing it through a se

Re: [cmake-developers] slow regex implementation in RegularExpression

2011-11-23 Thread Brad King
On 11/23/2011 12:34 PM, Alexandru Ciobanu wrote: > The regex in question is: > ^[^][:/*?]+\$ > > And it appears at this location in the ITK source tree: > CMake/ExternalData.cmake:347 > > And the expression is correct, because you're allowed to have the ] > metacharacter inside a [^xyz] c

Re: [cmake-developers] slow regex implementation in RegularExpression

2011-11-23 Thread Alexandru Ciobanu
On 2011-11-23, at 12:24 PM, Brad King wrote: > On 11/23/2011 12:20 PM, Alexandru Ciobanu wrote: >> to compile ITK using CMake+TRE. And there was at least one regex that >> TRE refused to compile. > > What was it, and where in the ITK code is it? The regex in question is: ^[^][:/*?]+\$ And

Re: [cmake-developers] slow regex implementation in RegularExpression

2011-11-23 Thread Brad King
On 11/23/2011 12:20 PM, Alexandru Ciobanu wrote: > to compile ITK using CMake+TRE. And there was at least one regex that > TRE refused to compile. What was it, and where in the ITK code is it? Thanks, -Brad -- Powered by www.kitware.com Visit other Kitware open-source projects at http://www.k

Re: [cmake-developers] slow regex implementation in RegularExpression

2011-11-23 Thread Alexandru Ciobanu
Hi Bill, On 2011-11-23, at 10:36 AM, Bill Hoffman wrote: > I am pretty sure the last time we talked about adding a new regex we talked > about requiring explicit requests. I think this would be a much safer > approach. I am really scared that this regex will not be compatible with the > old

Re: [cmake-developers] slow regex implementation in RegularExpression

2011-11-23 Thread Brad King
On 11/23/2011 12:06 PM, Bill Hoffman wrote: > On 11/23/2011 11:43 AM, James Bigler wrote: >> Why can't this be solved with a policy? One problem of using an >> explicit TRE command is that if you want to write code that *could* be >> used in an older version of CMake you won't be able to use it. >

Re: [cmake-developers] slow regex implementation in RegularExpression

2011-11-23 Thread Bill Hoffman
On 11/23/2011 11:43 AM, James Bigler wrote: Why can't this be solved with a policy? One problem of using an explicit TRE command is that if you want to write code that *could* be used in an older version of CMake you won't be able to use it. It could be, but that will not come without pain.

Re: [cmake-developers] slow regex implementation in RegularExpression

2011-11-23 Thread James Bigler
On Wed, Nov 23, 2011 at 8:36 AM, Bill Hoffman wrote: > On 11/22/2011 4:39 PM, Brad King wrote: > > It is tempting to always require explicit requests for new TRE behavior, >> such as using "TRE" instead of "REGEX" in keyword locations, but one >> advantage of using a policy is that over time the

Re: [cmake-developers] slow regex implementation in RegularExpression

2011-11-23 Thread Bill Hoffman
On 11/22/2011 4:39 PM, Brad King wrote: It is tempting to always require explicit requests for new TRE behavior, such as using "TRE" instead of "REGEX" in keyword locations, but one advantage of using a policy is that over time the old behavior will disappear completely from usage. I am pretty

Re: [cmake-developers] slow regex implementation in RegularExpression

2011-11-22 Thread Alexandru Ciobanu
> >> I also added the TRE library to Utilities/cmtre, and added >> CMAKE_USE_SYSTEM_TRE. >> >> I added the patch to the bug tracker: >> http://public.kitware.com/Bug/view.php?id=12381 > > Please add a note there indicating the CMake version (git commit sha1) > on which the patch was based. Oth

Re: [cmake-developers] slow regex implementation in RegularExpression

2011-11-22 Thread Brad King
On 11/22/2011 1:50 PM, Alexandru Ciobanu wrote: As Brad King suggested, instead of changing the files in Source/kwsys/, > I created a wrapper class and made all the calls go through it. Thanks. I also added the TRE library to Utilities/cmtre, and added CMAKE_USE_SYSTEM_TRE. I added the patch

Re: [cmake-developers] slow regex implementation in RegularExpression

2011-11-22 Thread Alexandru Ciobanu
On 2011-11-17, at 3:59 PM, Brad King wrote: > On 11/17/2011 3:19 PM, Alexandru Ciobanu wrote: > > I was able to make CMake use TRE, by changing the > > RegularExpression.{cxx,hxx.in} files. > > Those are down in Source/kwsys which is a directory shared by > projects other than just CMake. We ca

Re: [cmake-developers] slow regex implementation in RegularExpression

2011-11-17 Thread Alexander Neundorf
On Thursday 17 November 2011, Alexandru Ciobanu wrote: > On 2011-11-17, at 3:26 PM, Alexander Neundorf wrote: > >> [ Regular expressions syntax ] > >> In terms of regular expressions syntax, the only difference that I've > >> seen is that TRE treats the curly brackets "{" and "}" as special > >> ch

Re: [cmake-developers] slow regex implementation in RegularExpression

2011-11-17 Thread Brad King
On 11/17/2011 4:28 PM, Sean McBride wrote: Has using the POSIX regex.h APIs been ruled out? Windows? -Brad -- Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the CMake FAQ at:

Re: [cmake-developers] slow regex implementation in RegularExpression

2011-11-17 Thread Brad King
On 11/17/2011 3:19 PM, Alexandru Ciobanu wrote: > I was able to make CMake use TRE, by changing the > RegularExpression.{cxx,hxx.in} files. Those are down in Source/kwsys which is a directory shared by projects other than just CMake. We cannot touch the files there. Instead you will need to re-f

Re: [cmake-developers] slow regex implementation in RegularExpression

2011-11-17 Thread Brad King
On 11/17/2011 3:33 PM, Alexandru Ciobanu wrote: On 2011-11-17, at 3:26 PM, Alexander Neundorf wrote: Not sure how to achieve this. A policy ? Actually it is very easy to make it transparent and thus not need to modify any .cmake files. We just need to escape the curly brackets: { ->

Re: [cmake-developers] slow regex implementation in RegularExpression

2011-11-17 Thread Alexandru Ciobanu
On 2011-11-17, at 3:26 PM, Alexander Neundorf wrote: >> [ Regular expressions syntax ] >> In terms of regular expressions syntax, the only difference that I've seen >> is that TRE treats the curly brackets "{" and "}" as special characters, >> because it uses them for its "approximate matching".

Re: [cmake-developers] slow regex implementation in RegularExpression

2011-11-17 Thread Alexander Neundorf
On Thursday 17 November 2011, Alexandru Ciobanu wrote: > Hi everyone, > > [ CMake + TRE ] > I was able to make CMake use TRE, by changing the > RegularExpression.{cxx,hxx.in} files. > > I ran the CMake tests, and 100% pass. See the attached log file. > (NOTE: Bootstrap, complex, complexOne were i

Re: [cmake-developers] slow regex implementation in RegularExpression

2011-11-17 Thread Alexandru Ciobanu
Hi everyone,[ CMake + TRE ]I was able to make CMake use TRE, by changing the RegularExpression.{cxx,hxx.in} files.I ran the CMake tests, and 100% pass. See the attached log file. (NOTE: Bootstrap, complex, complexOne were initially not aware of TRE dependency, but I fixed that easily).[ Impact of u

Re: [cmake-developers] slow regex implementation in RegularExpression

2011-11-16 Thread David Cole
The "setenv" output will be gone (by default) as of CMake 2.8.7 because of this recent commit: http://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=5a94d099ddbd8f3d4b850957faa8c11f619c6f18 Does using ctest from 'next' or 'master' already show any improvement over 2.8.6? On Wed, Nov 16, 2011 at

Re: [cmake-developers] slow regex implementation in RegularExpression

2011-11-16 Thread Bill Hoffman
On 11/16/2011 4:11 PM, Sean McBride wrote: The downside is that this solution would be fragile. xcodebuild's output is not guaranteed to be the same forever, it's not like a public API. Already today, gcc and clang output pretty differently. I'm a little worried this would bite us. Besides,

Re: [cmake-developers] slow regex implementation in RegularExpression

2011-11-16 Thread Bill Hoffman
A little off topic, but I am wondering if the ctest performance issue for xcode could be fixed without changing the regex. The problem with xcode is that it spits out very verbose output. I am wondering if some short circuit stuff could be put in place. Maybe do a string compare of the first

Re: [cmake-developers] slow regex implementation in RegularExpression

2011-11-16 Thread Brad King
On 11/16/2011 2:12 PM, Alexandru Ciobanu wrote: This is how I run the tests (pseudocode): recomp() repeat 1000 times: regexec() Thanks for the explanation. TRex -- 680ms PCRE -- 610ms ( with pcre_exec() ) PCRE -- 990ms ( with pcre_dfa_exec() ) re2 --

Re: [cmake-developers] slow regex implementation in RegularExpression

2011-11-16 Thread Alexander Neundorf
On Wednesday 16 November 2011, Alexandru Ciobanu wrote: > Hi Brad, ... > Advantages of TRE: > - API very similar to standard regex.h (i.e. easy to integrate with > CMake) - supports wide characters > - compiles on many platforms Windows, AIX, HP-UX, you name it. > > What do you think about TRE

Re: [cmake-developers] slow regex implementation in RegularExpression

2011-11-16 Thread Alexandru Ciobanu
Hi Brad, [1] > On 11/16/2011 12:44 PM, Alexandru Ciobanu wrote: >> For each library the steps are: >> - regcomp() the regular expression >> - regexec() the expression on the string > > Can you time each of these steps separately for each library? I would not > be surprised if the compilation ti

Re: [cmake-developers] slow regex implementation in RegularExpression

2011-11-16 Thread Brad King
On 11/16/2011 12:44 PM, Alexandru Ciobanu wrote: For each library the steps are: - regcomp() the regular expression - regexec() the expression on the string Can you time each of these steps separately for each library? I would not be surprised if the compilation time is the bottleneck. The ev

Re: [cmake-developers] slow regex implementation in RegularExpression

2011-11-16 Thread Pau Garcia i Quiles
On Wed, Nov 16, 2011 at 6:44 PM, Alexandru Ciobanu wrote: > As it can be seen re2 and the standard regex.h are orders of magnitude > faster in executing this particular regular expression. > The difference between PCRE and re2 is also confirmed by this study: >     http://swtch.com/~rsc/regexp/re

Re: [cmake-developers] slow regex implementation in RegularExpression

2011-11-16 Thread Alexander Neundorf
On Wednesday 16 November 2011, Alexandru Ciobanu wrote: > Hi, > > I was successful in making CMake work with PCRE. As expected, it was > straightforward. > > The problem is that PCRE is also slow. So, I tested the same string and > regex with multiple different libraries in order to assess perfor

Re: [cmake-developers] slow regex implementation in RegularExpression

2011-11-16 Thread Alexandru Ciobanu
Hi, I was successful in making CMake work with PCRE. As expected, it was straightforward. The problem is that PCRE is also slow. So, I tested the same string and regex with multiple different libraries in order to assess performance. The regular expression in question is: ([^:]+): warni

Re: [cmake-developers] slow regex implementation in RegularExpression

2011-11-15 Thread Pau Garcia i Quiles
Hi, If it's of any help, I used the pcrecpp library by Google (it's part of PCRE). With pcrecpp, most operations were only 1-3 lines long. The only problem I found is PCRE provided no way to get the previous/next match, which CMake needs. On Tue, Nov 15, 2011 at 4:25 PM, Alexandru Ciobanu wrot

Re: [cmake-developers] slow regex implementation in RegularExpression

2011-11-15 Thread Alexandru Ciobanu
Hi Bill and Pau, I am currently working on adding PCRE to CMake. Chances are very hight that it will work, given the very similar comp()/exec() API calls in both implementations. I'll let you know about the results soon. Alex On 2011-11-14, at 10:31 PM, Bill Hoffman wrote: > On 11/14/2011 6

Re: [cmake-developers] slow regex implementation in RegularExpression

2011-11-14 Thread Bill Hoffman
On 11/14/2011 6:08 PM, Pau Garcia i Quiles wrote: Bill, I think the current incarnation of regexps in CMake should be kept for compatibility reasons. Yes, of course. Adding PCRE is not difficult, just time consuming. The implementation I'd do would be an additional abstraction layer: - For t

Re: [cmake-developers] slow regex implementation in RegularExpression

2011-11-14 Thread Pau Garcia i Quiles
Bill, I think the current incarnation of regexps in CMake should be kept for compatibility reasons. Adding PCRE is not difficult, just time consuming. The implementation I'd do would be an additional abstraction layer: - For the current BRE implementation, it would be a 1:1 call match - For the P

Re: [cmake-developers] slow regex implementation in RegularExpression

2011-11-14 Thread Bill Hoffman
Sorry for the top post... However, if the issue with ctest being slow can be fixed by using PCRE in CMake, that is good news. We can just link in the library, and replace that small part of CMake internal code that has the performance problem. This should not break backwards compatibility.

Re: [cmake-developers] slow regex implementation in RegularExpression

2011-11-14 Thread Pau Garcia i Quiles
Hi, Check this: A wish a day 11: Perl Compatible Regular Expressions in CMake http://www.elpauer.org/?p=684 Unfortunately the student turned out to be a total fraud: he knew nothing about CMake, regular expressions (much less PCRE!), git, and could barely manage with C/C++. After months of expla