Re: [HACKERS] Have we out-grown Flex?

2012-05-03 Thread Jeff Janes
On Thu, May 3, 2012 at 12:53 PM, james wrote: > Doesn't that imply that a plan cache might be worthwhile? PG has one if you use prepared statements. Some people are either unable or unwilling to use prepared statements, though. Cheers, Jeff -- Sent via pgsql-hackers mailing list (pgsql-hacke

Re: [HACKERS] Have we out-grown Flex?

2012-05-03 Thread james
I believe there are tools that are significantly faster than flex. I believe re2c generates code that is faster. But the key thing is to test, probably, or perhaps ask around. I'm out of touch, but from memory flex wasn't the be-all and end-all. Lemon is definitely easy to maintain/port and

Re: [HACKERS] Have we out-grown Flex?

2012-05-03 Thread Daniel Farina
On Thu, May 3, 2012 at 12:51 PM, james wrote: > I haven't tried quex, but I have tried lemon (which can be broken out of > SQLite) and re2c and ragel. > > I like ragel and lemon, but the combination supports a push-parser style > from memory, and many tools are inconvenient unless you are prepared

Re: [HACKERS] Have we out-grown Flex?

2012-05-03 Thread james
Doesn't that imply that a plan cache might be worthwhile? But no matter: didn't the OP really have issue with packaging and Windows support - and there are a lot of Windows users, and in general there are many Windows devs: making it easier for them to contribute has to be good doesn't it? -

Re: [HACKERS] Have we out-grown Flex?

2012-05-03 Thread james
I haven't tried quex, but I have tried lemon (which can be broken out of SQLite) and re2c and ragel. I like ragel and lemon, but the combination supports a push-parser style from memory, and many tools are inconvenient unless you are prepared to suck in a whole message before parsing, or let t

Re: [HACKERS] Have we out-grown Flex?

2012-05-02 Thread Bruce Momjian
On Wed, May 02, 2012 at 10:37:58AM -0700, Jeff Janes wrote: > I could try to clean up and post the patch that implements this if you want. > > The second method was just to do --enable-profiling on a stock build > and look at the call graph section of gprof output. It attributed 50% > to pg_plan_

Re: [HACKERS] Have we out-grown Flex?

2012-05-02 Thread Magnus Hagander
On Wed, May 2, 2012 at 3:33 PM, Peter Geoghegan wrote: > On 2 May 2012 04:57, Tom Lane wrote: >> FWIW, I think only developers not packagers would really be taking such >> a hit.  I assume we'd continue to ship prebuilt lexer output in >> tarballs, so there'd seldom be a reason for a packager to

Re: [HACKERS] Have we out-grown Flex?

2012-05-02 Thread Jeff Janes
On Wed, May 2, 2012 at 9:31 AM, Peter Geoghegan wrote: > On 2 May 2012 17:20, Jeff Janes wrote: >> For -S -M simple, the time spent planning is 5 times more than the >> time spent parsing.  It may be worthwhile to reduce the time spent >> parsing, but if the goal is parity with MySQL it probably

Re: [HACKERS] Have we out-grown Flex?

2012-05-02 Thread Peter Geoghegan
On 2 May 2012 17:20, Jeff Janes wrote: > For -S -M simple, the time spent planning is 5 times more than the > time spent parsing.  It may be worthwhile to reduce the time spent > parsing, but if the goal is parity with MySQL it probably isn't the > place to start. Could you please share your figu

Re: [HACKERS] Have we out-grown Flex?

2012-05-02 Thread Jeff Janes
On Tue, May 1, 2012 at 5:53 PM, Peter Geoghegan wrote: > Quite apart from the practical difficulties that we have with Flex > (the fact that the authors are non-responsive and possibly retired, > that annoying compiler warning, and the fact that we are forced to > maintain our own Windows binaries

Re: [HACKERS] Have we out-grown Flex?

2012-05-02 Thread Peter Geoghegan
On 2 May 2012 04:57, Tom Lane wrote: > FWIW, I think only developers not packagers would really be taking such > a hit.  I assume we'd continue to ship prebuilt lexer output in > tarballs, so there'd seldom be a reason for a packager to need to run > the tool.  Given the extremely slow rate of chu

Re: [HACKERS] Have we out-grown Flex?

2012-05-02 Thread Peter Geoghegan
On 2 May 2012 04:24, Robert Haas wrote: > I think Tom's question of whether the parser or lexer is the problem > is something that ought to be investigated.  Personally, I suspect > that our tendency to use lists everywhere, for everything, an artifact > of our proud LISP heritage, may be costing

Re: [HACKERS] Have we out-grown Flex?

2012-05-01 Thread Tom Lane
Robert Haas writes: > One fairly major obstacle to using quex for anything is that it > doesn't appear to be packaged on either Fedora or Ubuntu (I tried > apt-cache search quex on my Ubuntu laptop, and yum install quex on an > F16 machine). 'port install quex' on my MacOS X doesn't find it > eit

Re: [HACKERS] Have we out-grown Flex?

2012-05-01 Thread Robert Haas
On Tue, May 1, 2012 at 8:53 PM, Peter Geoghegan wrote: > I'm certainly not suggesting that this isn't something that, in order > to be adopted, will have to be carefully considered from many angles, > of which the performance of the resulting lexer is only one, but at > the same time I dare say th

Re: [HACKERS] Have we out-grown Flex?

2012-05-01 Thread Tom Lane
Peter Geoghegan writes: > Quite apart from the practical difficulties that we have with Flex > (the fact that the authors are non-responsive and possibly retired, btw, as far as that goes, a look into their sourceforge SCM shows evidence of continued activity. I dunno why it's been so long since

Re: [HACKERS] Have we out-grown Flex?

2012-05-01 Thread Tom Lane
Peter Geoghegan writes: > ... I have heard people complain about Postgres > parser overhead for "pgbench -S" style use-cases where it is > unreasonably high, and I've heard them do so more than once. I was under the impression that those cycles were mostly in bison not flex...

[HACKERS] Have we out-grown Flex?

2012-05-01 Thread Peter Geoghegan
Quite apart from the practical difficulties that we have with Flex (the fact that the authors are non-responsive and possibly retired, that annoying compiler warning, and the fact that we are forced to maintain our own Windows binaries of 2.5.35), it has some notable disadvantages. I am aware that