Re: [HACKERS] Have we out-grown Flex?

2012-05-03 Thread james
I haven't tried quex, but I have tried lemon (which can be broken out of SQLite) and re2c and ragel. I like ragel and lemon, but the combination supports a push-parser style from memory, and many tools are inconvenient unless you are prepared to suck in a whole message before parsing, or let

Re: [HACKERS] Have we out-grown Flex?

2012-05-03 Thread james
Doesn't that imply that a plan cache might be worthwhile? But no matter: didn't the OP really have issue with packaging and Windows support - and there are a lot of Windows users, and in general there are many Windows devs: making it easier for them to contribute has to be good doesn't it?

Re: [HACKERS] Have we out-grown Flex?

2012-05-03 Thread Daniel Farina
On Thu, May 3, 2012 at 12:51 PM, james ja...@mansionfamily.plus.com wrote: I haven't tried quex, but I have tried lemon (which can be broken out of SQLite) and re2c and ragel. I like ragel and lemon, but the combination supports a push-parser style from memory, and many tools are inconvenient

Re: [HACKERS] Have we out-grown Flex?

2012-05-03 Thread james
I believe there are tools that are significantly faster than flex. I believe re2c generates code that is faster. But the key thing is to test, probably, or perhaps ask around. I'm out of touch, but from memory flex wasn't the be-all and end-all. Lemon is definitely easy to maintain/port

Re: [HACKERS] Have we out-grown Flex?

2012-05-03 Thread Jeff Janes
On Thu, May 3, 2012 at 12:53 PM, james ja...@mansionfamily.plus.com wrote: Doesn't that imply that a plan cache might be worthwhile? PG has one if you use prepared statements. Some people are either unable or unwilling to use prepared statements, though. Cheers, Jeff -- Sent via

Re: [HACKERS] Have we out-grown Flex?

2012-05-02 Thread Peter Geoghegan
On 2 May 2012 04:24, Robert Haas robertmh...@gmail.com wrote: I think Tom's question of whether the parser or lexer is the problem is something that ought to be investigated.  Personally, I suspect that our tendency to use lists everywhere, for everything, an artifact of our proud LISP

Re: [HACKERS] Have we out-grown Flex?

2012-05-02 Thread Peter Geoghegan
On 2 May 2012 04:57, Tom Lane t...@sss.pgh.pa.us wrote: FWIW, I think only developers not packagers would really be taking such a hit.  I assume we'd continue to ship prebuilt lexer output in tarballs, so there'd seldom be a reason for a packager to need to run the tool.  Given the extremely

Re: [HACKERS] Have we out-grown Flex?

2012-05-02 Thread Jeff Janes
On Tue, May 1, 2012 at 5:53 PM, Peter Geoghegan pe...@2ndquadrant.com wrote: Quite apart from the practical difficulties that we have with Flex (the fact that the authors are non-responsive and possibly retired, that annoying compiler warning, and the fact that we are forced to maintain our

Re: [HACKERS] Have we out-grown Flex?

2012-05-02 Thread Peter Geoghegan
On 2 May 2012 17:20, Jeff Janes jeff.ja...@gmail.com wrote: For -S -M simple, the time spent planning is 5 times more than the time spent parsing.  It may be worthwhile to reduce the time spent parsing, but if the goal is parity with MySQL it probably isn't the place to start. Could you

Re: [HACKERS] Have we out-grown Flex?

2012-05-02 Thread Jeff Janes
On Wed, May 2, 2012 at 9:31 AM, Peter Geoghegan pe...@2ndquadrant.com wrote: On 2 May 2012 17:20, Jeff Janes jeff.ja...@gmail.com wrote: For -S -M simple, the time spent planning is 5 times more than the time spent parsing.  It may be worthwhile to reduce the time spent parsing, but if the

Re: [HACKERS] Have we out-grown Flex?

2012-05-02 Thread Magnus Hagander
On Wed, May 2, 2012 at 3:33 PM, Peter Geoghegan pe...@2ndquadrant.com wrote: On 2 May 2012 04:57, Tom Lane t...@sss.pgh.pa.us wrote: FWIW, I think only developers not packagers would really be taking such a hit.  I assume we'd continue to ship prebuilt lexer output in tarballs, so there'd

Re: [HACKERS] Have we out-grown Flex?

2012-05-02 Thread Bruce Momjian
On Wed, May 02, 2012 at 10:37:58AM -0700, Jeff Janes wrote: I could try to clean up and post the patch that implements this if you want. The second method was just to do --enable-profiling on a stock build and look at the call graph section of gprof output. It attributed 50% to

[HACKERS] Have we out-grown Flex?

2012-05-01 Thread Peter Geoghegan
Quite apart from the practical difficulties that we have with Flex (the fact that the authors are non-responsive and possibly retired, that annoying compiler warning, and the fact that we are forced to maintain our own Windows binaries of 2.5.35), it has some notable disadvantages. I am aware that

Re: [HACKERS] Have we out-grown Flex?

2012-05-01 Thread Tom Lane
Peter Geoghegan pe...@2ndquadrant.com writes: ... I have heard people complain about Postgres parser overhead for pgbench -S style use-cases where it is unreasonably high, and I've heard them do so more than once. I was under the impression that those cycles were mostly in bison not flex...

Re: [HACKERS] Have we out-grown Flex?

2012-05-01 Thread Tom Lane
Peter Geoghegan pe...@2ndquadrant.com writes: Quite apart from the practical difficulties that we have with Flex (the fact that the authors are non-responsive and possibly retired, btw, as far as that goes, a look into their sourceforge SCM shows evidence of continued activity. I dunno why

Re: [HACKERS] Have we out-grown Flex?

2012-05-01 Thread Robert Haas
On Tue, May 1, 2012 at 8:53 PM, Peter Geoghegan pe...@2ndquadrant.com wrote: I'm certainly not suggesting that this isn't something that, in order to be adopted, will have to be carefully considered from many angles, of which the performance of the resulting lexer is only one, but at the same

Re: [HACKERS] Have we out-grown Flex?

2012-05-01 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: One fairly major obstacle to using quex for anything is that it doesn't appear to be packaged on either Fedora or Ubuntu (I tried apt-cache search quex on my Ubuntu laptop, and yum install quex on an F16 machine). 'port install quex' on my MacOS X