Re: [HACKERS] Upgrading our minimum required flex version for 8.5

2009-07-23 Thread Dave Page
On Thu, Jul 23, 2009 at 2:57 AM, Greg Smithgsm...@gregsmith.com wrote: I got bit by this tonight as part of testing a patch on CentOS 5, which like RHEL 5 still ships flex 2.5.4.  I just wrote a little guide on how to grab a source RPM from a Fedora version and install it to work around that

Re: [HACKERS] Upgrading our minimum required flex version for 8.5

2009-07-23 Thread Magnus Hagander
On Thu, Jul 23, 2009 at 10:52, Dave Pagedp...@pgadmin.org wrote: On Thu, Jul 23, 2009 at 2:57 AM, Greg Smithgsm...@gregsmith.com wrote: I got bit by this tonight as part of testing a patch on CentOS 5, which like RHEL 5 still ships flex 2.5.4.  I just wrote a little guide on how to grab a

Re: [HACKERS] Upgrading our minimum required flex version for 8.5

2009-07-23 Thread Andrew Dunstan
Magnus Hagander wrote: On Thu, Jul 23, 2009 at 10:52, Dave Pagedp...@pgadmin.org wrote: On Thu, Jul 23, 2009 at 2:57 AM, Greg Smithgsm...@gregsmith.com wrote: I got bit by this tonight as part of testing a patch on CentOS 5, which like RHEL 5 still ships flex 2.5.4. I just wrote a

Re: [HACKERS] Upgrading our minimum required flex version for 8.5

2009-07-23 Thread Tom Lane
Andrew Dunstan and...@dunslane.net writes: Magnus Hagander wrote: Much as I dislike it, we may need to revisit the idea about putting the flex output files in CVS... Why? This only affects developers building from a CVS pull. You don't need any flex at all to build from a tarball. If

Re: [HACKERS] Upgrading our minimum required flex version for 8.5

2009-07-23 Thread Andrew Dunstan
Tom Lane wrote: Andrew Dunstan and...@dunslane.net writes: Magnus Hagander wrote: Much as I dislike it, we may need to revisit the idea about putting the flex output files in CVS... Why? This only affects developers building from a CVS pull. You don't need any flex at

Re: [HACKERS] Upgrading our minimum required flex version for 8.5

2009-07-23 Thread Tom Lane
Andrew Dunstan and...@dunslane.net writes: Tom Lane wrote: I wonder if it would be helpful to have a buildfarm option whereby it would fetch the latest nightly-snapshot tarball and use that instead of a CVS pull. That is certainly doable. It would be in effect a forced run, because we

Re: [HACKERS] Upgrading our minimum required flex version for 8.5

2009-07-22 Thread Magnus Hagander
On Tue, Jul 14, 2009 at 05:10, Andrew Dunstanand...@dunslane.net wrote: Tom Lane wrote: Maybe for the time being we need to think about keeping scan.c in CVS. It's not like scan.l gets updated all that often. We could if we had to, though it amounts to saying that Windows-based developers

Re: [HACKERS] Upgrading our minimum required flex version for 8.5

2009-07-22 Thread Dave Page
On Wed, Jul 22, 2009 at 12:42 PM, Magnus Hagandermag...@hagander.net wrote: FWIW, it seems the version that Andrew put up doesn't work in one of my test environments, and also not in at last one of Dave's. I will test it in my second test environment later today to be sure. It doesn't work in

Re: [HACKERS] Upgrading our minimum required flex version for 8.5

2009-07-22 Thread Andrew Dunstan
Dave Page wrote: On Wed, Jul 22, 2009 at 12:42 PM, Magnus Hagandermag...@hagander.net wrote: FWIW, it seems the version that Andrew put up doesn't work in one of my test environments, and also not in at last one of Dave's. I will test it in my second test environment later today to be

Re: [HACKERS] Upgrading our minimum required flex version for 8.5

2009-07-22 Thread Tom Lane
Dave Page dp...@pgadmin.org writes: It doesn't work in any of my bf animals, or build machines :-( ?? narwhal seems to have gone green. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription:

Re: [HACKERS] Upgrading our minimum required flex version for 8.5

2009-07-22 Thread Dave Page
On Wed, Jul 22, 2009 at 2:16 PM, Tom Lanet...@sss.pgh.pa.us wrote: Dave Page dp...@pgadmin.org writes: It doesn't work in any of my bf animals, or build machines :-( ?? narwhal seems to have gone green. Narwhal is mingw/msys. The misbehaving flex is the one provided by Andrew for use with

Re: [HACKERS] Upgrading our minimum required flex version for 8.5

2009-07-22 Thread Andrew Dunstan
Tom Lane wrote: Dave Page dp...@pgadmin.org writes: It doesn't work in any of my bf animals, or build machines :-( ?? narwhal seems to have gone green. Yeah, the problem is with MSVC, Narwal is a Mingw box. cheers andrew -- Sent via pgsql-hackers mailing list

Re: [HACKERS] Upgrading our minimum required flex version for 8.5

2009-07-22 Thread Greg Smith
I got bit by this tonight as part of testing a patch on CentOS 5, which like RHEL 5 still ships flex 2.5.4. I just wrote a little guide on how to grab a source RPM from a Fedora version and install it to work around that problem:

Re: [HACKERS] Upgrading our minimum required flex version for 8.5

2009-07-14 Thread Chuck McDevitt
-Original Message- From: pgsql-hackers-ow...@postgresql.org [mailto:pgsql-hackers- ow...@postgresql.org] On Behalf Of Tom Lane Sent: Monday, July 13, 2009 7:43 PM To: Andrew Dunstan Cc: pgsql-hackers@postgresql.org Subject: Re: [HACKERS] Upgrading our minimum required flex version

Re: [HACKERS] Upgrading our minimum required flex version for 8.5

2009-07-14 Thread Andrew Dunstan
Chuck McDevitt wrote: Flex 2.5.33 and bison 2.3 are available from mingw for windows. http://sourceforge.net/projects/mingw/files/ Since mingw programs don't need Cygwin installed, these should probably be OK for most Windows people. But if really needed, flex 2.5.33 could be ported (m4

Re: [HACKERS] Upgrading our minimum required flex version for 8.5

2009-07-14 Thread Andrew Dunstan
Andrew Dunstan wrote: Chuck McDevitt wrote: Flex 2.5.33 and bison 2.3 are available from mingw for windows. http://sourceforge.net/projects/mingw/files/ Since mingw programs don't need Cygwin installed, these should probably be OK for most Windows people. But if really needed, flex

Re: [HACKERS] Upgrading our minimum required flex version for 8.5

2009-07-14 Thread Tom Lane
Andrew Dunstan and...@dunslane.net writes: What I have done is built a version of flex 2.5.35 with Cygwin, and bundled the .exe with the Cygwin DLL (the only other thing it should need, for our purposes) in a zip file. It can currently be fetched from

Re: [HACKERS] Upgrading our minimum required flex version for 8.5

2009-07-13 Thread Kevin Grittner
Tom Lane t...@sss.pgh.pa.us wrote: Now this disappears into the noise as soon as you include parse analysis (let alone planning and execution) No need to go for silly options to avoid a performance issue at that level. Time wasted dealing with subsequent silliness would almost certainly

Re: [HACKERS] Upgrading our minimum required flex version for 8.5

2009-07-13 Thread Andrew Dunstan
Andrew Dunstan wrote: Tom Lane wrote: I see that a good-sized fraction of the buildfarm is still on flex 2.5.4 and will therefore go red when this goes in. Should I hold off a bit longer, or is committing the best way to get their attention? Probably the latter. I will update my

Re: [HACKERS] Upgrading our minimum required flex version for 8.5

2009-07-13 Thread Andrew Dunstan
Andrew Dunstan wrote: OK, the fly in this ointment turns out to be MSVC. The latest flex from GnuWin32 is 2.5.4a, and building 2.5.35 for Windows is turning out to be quite a pain. Luckily, MinGW has a pre-built modified 2.5.33 available, and I have installed this (also needed msys-regex),

Re: [HACKERS] Upgrading our minimum required flex version for 8.5

2009-07-13 Thread Tom Lane
Andrew Dunstan and...@dunslane.net writes: Well, it looks like there's a reason GnuWin32 hasn't advanced beyond 2.5.4a - after that the flex developers proceeded to make flex use a filter chain methodology that requires the use of fork(). Making it run on Windows without the support of

Re: [HACKERS] Upgrading our minimum required flex version for 8.5

2009-07-13 Thread Andrew Dunstan
Tom Lane wrote: Andrew Dunstan and...@dunslane.net writes: Well, it looks like there's a reason GnuWin32 hasn't advanced beyond 2.5.4a - after that the flex developers proceeded to make flex use a filter chain methodology that requires the use of fork(). Making it run on Windows without

Re: [HACKERS] Upgrading our minimum required flex version for 8.5

2009-07-12 Thread Pavel Stehule
2009/7/12 Tom Lane t...@sss.pgh.pa.us: Andrew Dunstan and...@dunslane.net writes: Tom Lane wrote: Andrew Dunstan and...@dunslane.net writes: I think it would need to be benchmarked. My faint recollection is that the re-entrant lexers are slower. The flex documentation states in so many

Re: [HACKERS] Upgrading our minimum required flex version for 8.5

2009-07-12 Thread Andrew Dunstan
Tom Lane wrote: As best I can tell after some casual testing on a couple of machines, the actual bottom line is that raw_parser (ie, the bison and flex processing) is going to be a couple of percent slower with a reentrant grammar and lexer, for typical queries involving a lot of short tokens.

Re: [HACKERS] Upgrading our minimum required flex version for 8.5

2009-07-12 Thread Tom Lane
Pavel Stehule pavel.steh...@gmail.com writes: 2009/7/12 Tom Lane t...@sss.pgh.pa.us: If we're going to go for reentrancy I think we should fix both components. when we don't use reentrant grammar, then we cannot use main sql parser in SQL? It wouldn't be a problem for the immediate

Re: [HACKERS] Upgrading our minimum required flex version for 8.5

2009-07-12 Thread Pavel Stehule
2009/7/12 Tom Lane t...@sss.pgh.pa.us: Pavel Stehule pavel.steh...@gmail.com writes: 2009/7/12 Tom Lane t...@sss.pgh.pa.us: If we're going to go for reentrancy I think we should fix both components. when we don't use reentrant grammar, then we cannot use main sql parser in SQL? It

Re: [HACKERS] Upgrading our minimum required flex version for 8.5

2009-07-12 Thread Tom Lane
Andrew Dunstan and...@dunslane.net writes: If we're going to have a reentrant lexer, I think we should go the whole nine yards. I agree that a couple of percent slowdown on just the lexing and parsing will be lost in the noise. So +1 from me. I found a couple of places where a few cycles

Re: [HACKERS] Upgrading our minimum required flex version for 8.5

2009-07-12 Thread Andrew Dunstan
Tom Lane wrote: I see that a good-sized fraction of the buildfarm is still on flex 2.5.4 and will therefore go red when this goes in. Should I hold off a bit longer, or is committing the best way to get their attention? Probably the latter. I will update my various members. I see that

Re: [HACKERS] Upgrading our minimum required flex version for 8.5

2009-07-12 Thread Josh Berkus
This is ready to go except for changing the minimum flex version test in configure (and associated documentation). I see that a good-sized fraction of the buildfarm is still on flex 2.5.4 and will therefore go red when this goes in. Should I hold off a bit longer, or is committing the best

Re: [HACKERS] Upgrading our minimum required flex version for 8.5

2009-07-12 Thread Andrew Dunstan
Josh Berkus wrote: This is ready to go except for changing the minimum flex version test in configure (and associated documentation). I see that a good-sized fraction of the buildfarm is still on flex 2.5.4 and will therefore go red when this goes in. Should I hold off a bit longer, or is

Re: [HACKERS] Upgrading our minimum required flex version for 8.5

2009-07-12 Thread Tom Lane
Andrew Dunstan and...@dunslane.net writes: Josh Berkus wrote: Oh, I didn't think about the Flex version. That's going to be a far more widespread problem. OSX 10.5, for example, still ships with 2.5.33. I suspect that a *lot* of OSes won't have anything up-to-date. That's the version

Re: [HACKERS] Upgrading our minimum required flex version for 8.5

2009-07-11 Thread Tom Lane
Andrew Dunstan and...@dunslane.net writes: Tom Lane wrote: Andrew Dunstan and...@dunslane.net writes: I think it would need to be benchmarked. My faint recollection is that the re-entrant lexers are slower. The flex documentation states in so many words: The option `--reentrant' does

Re: [HACKERS] Upgrading our minimum required flex version for 8.5

2009-07-03 Thread Zdenek Kotala
Tom Lane píše v čt 02. 07. 2009 v 13:13 -0400: Actually, most of the buildfarm members show which flex version they are running in the configure output. A quick look shows that of the 45 members that have reported on HEAD in the past 2 days, 22 are running 2.5.4, which is a lot higher

[HACKERS] Upgrading our minimum required flex version for 8.5

2009-07-02 Thread Tom Lane
I'd like to return to the project I suggested here: http://archives.postgresql.org/message-id/18653.1239741...@sss.pgh.pa.us of getting rid of plpgsql's private lexer and having it use the core lexer instead. This will require making the core lexer re-entrant, which is not possible with our

Re: [HACKERS] Upgrading our minimum required flex version for 8.5

2009-07-02 Thread Andrew Dunstan
Tom Lane wrote: I'd like to return to the project I suggested here: http://archives.postgresql.org/message-id/18653.1239741...@sss.pgh.pa.us of getting rid of plpgsql's private lexer and having it use the core lexer instead. This will require making the core lexer re-entrant, which is not

Re: [HACKERS] Upgrading our minimum required flex version for 8.5

2009-07-02 Thread Kevin Grittner
Tom Lane t...@sss.pgh.pa.us wrote: so the new minimum supported version would probably be 2.5.33 (2.5.31 was kinda broken for other reasons, and there was no 2.5.32). Since 2.5.33 is now over three years old, this does not seem like an onerous requirement, but I thought I'd better ask if

Re: [HACKERS] Upgrading our minimum required flex version for 8.5

2009-07-02 Thread Kevin Grittner
Kevin Grittner kevin.gritt...@wicourts.gov wrote: Tom Lane t...@sss.pgh.pa.us wrote: so the new minimum supported version would probably be 2.5.33 We still have five SuSE Enterprise 9 boxes in use as database servers; while we've got someone tasked with replacing them, I wonder if that

Re: [HACKERS] Upgrading our minimum required flex version for 8.5

2009-07-02 Thread Tom Lane
Andrew Dunstan and...@dunslane.net writes: I think it would need to be benchmarked. My faint recollection is that the re-entrant lexers are slower. The flex documentation states in so many words: The option `--reentrant' does not affect the performance of the scanner. Do you feel a

Re: [HACKERS] Upgrading our minimum required flex version for 8.5

2009-07-02 Thread Andrew Dunstan
Tom Lane wrote: Andrew Dunstan and...@dunslane.net writes: I think it would need to be benchmarked. My faint recollection is that the re-entrant lexers are slower. The flex documentation states in so many words: The option `--reentrant' does not affect the performance of

Re: [HACKERS] Upgrading our minimum required flex version for 8.5

2009-07-02 Thread Alvaro Herrera
Kevin Grittner wrote: Tom Lane t...@sss.pgh.pa.us wrote: so the new minimum supported version would probably be 2.5.33 (2.5.31 was kinda broken for other reasons, and there was no 2.5.32). Since 2.5.33 is now over three years old, this does not seem like an onerous requirement,

Re: [HACKERS] Upgrading our minimum required flex version for 8.5

2009-07-02 Thread Tom Lane
Kevin Grittner kevin.gritt...@wicourts.gov writes: kgri...@ccdev-db:/etc/init.d flex --version flex 2.5.31 Just how broken is the version we're using? You might want to take that up with SuSE. The flex NEWS file just cites numerous bug and security fixes from .31 to .33. Our CVS logs show

Re: [HACKERS] Upgrading our minimum required flex version for 8.5

2009-07-02 Thread Tom Lane
Alvaro Herrera alvhe...@commandprompt.com writes: Kevin Grittner wrote: Tom Lane t...@sss.pgh.pa.us wrote: Since 2.5.33 is now over three years old, this does not seem like an onerous requirement, but I thought I'd better ask if anyone has an objection? You'd be causing problems for SuSE

Re: [HACKERS] Upgrading our minimum required flex version for 8.5

2009-07-02 Thread Andrew Dunstan
Tom Lane wrote: Alvaro Herrera alvhe...@commandprompt.com writes: Kevin Grittner wrote: Tom Lane t...@sss.pgh.pa.us wrote: Since 2.5.33 is now over three years old, this does not seem like an onerous requirement, but I thought I'd better ask if anyone has an objection?

Re: [HACKERS] Upgrading our minimum required flex version for 8.5

2009-07-02 Thread Kevin Grittner
Alvaro Herrera alvhe...@commandprompt.com wrote: Keep in mind that if you build from a tarball, the lexer has already been run elsewhere and you don't need flex. Does that hold for the daily snapshots? If so, I should be good. My workstation is OK on flex version. (I run kubuntu on my

Re: [HACKERS] Upgrading our minimum required flex version for 8.5

2009-07-02 Thread Tom Lane
Andrew Dunstan and...@dunslane.net writes: Tom Lane wrote: Right, this will only affect people doing development or otherwise building from a CVS pull. Of course, that includes the whole buildfarm. We might need to ask some people to upgrade there. Yes. What I was thinking of doing was

Re: [HACKERS] Upgrading our minimum required flex version for 8.5

2009-07-02 Thread Alvaro Herrera
Kevin Grittner wrote: Alvaro Herrera alvhe...@commandprompt.com wrote: Keep in mind that if you build from a tarball, the lexer has already been run elsewhere and you don't need flex. Does that hold for the daily snapshots? If so, I should be good. Yes, the snapshots have the

Re: [HACKERS] Upgrading our minimum required flex version for 8.5

2009-07-02 Thread Greg Stark
Tom Lane wrote: Right, this will only affect people doing development or otherwise building from a CVS pull. Someone doing a CVS pull already needs a specific recent version of autoconf anyways. How old is this version of flex compared to the version of autoconf we require? -- greg

Re: [HACKERS] Upgrading our minimum required flex version for 8.5

2009-07-02 Thread Robert Haas
On Thu, Jul 2, 2009 at 11:39 AM, Greg Starkgsst...@mit.edu wrote: Tom Lane wrote: Right, this will only affect people doing development or otherwise building from a CVS pull. Someone doing a CVS pull already needs a specific recent version of autoconf anyways. How old is this version of flex

Re: [HACKERS] Upgrading our minimum required flex version for 8.5

2009-07-02 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Thu, Jul 2, 2009 at 11:39 AM, Greg Starkgsst...@mit.edu wrote: Someone doing a CVS pull already needs a specific recent version of autoconf anyways. How old is this version of flex compared to the version of autoconf we require? Really? configure

Re: [HACKERS] Upgrading our minimum required flex version for 8.5

2009-07-02 Thread Kevin Grittner
Alvaro Herrera alvhe...@commandprompt.com wrote: Yes, the snapshots have the derived files too. Then I take it back -- the new flex versions would have little or no impact on me. Worst case, I might need to download a snapshot to apply my patch for testing on the big machines. If I

Re: [HACKERS] Upgrading our minimum required flex version for 8.5

2009-07-02 Thread Tom Lane
Kevin Grittner kevin.gritt...@wicourts.gov writes: Then I take it back -- the new flex versions would have little or no impact on me. Worst case, I might need to download a snapshot to apply my patch for testing on the big machines. If I understood what make options I could use on my machine

Re: [HACKERS] Upgrading our minimum required flex version for 8.5

2009-07-02 Thread Tom Lane
I wrote: Yes. What I was thinking of doing was committing a configure change to reject flex 2.5.31, and waiting to see how much of the buildfarm goes red. Actually, most of the buildfarm members show which flex version they are running in the configure output. A quick look shows that of the

Re: [HACKERS] Upgrading our minimum required flex version for 8.5

2009-07-02 Thread Kevin Grittner
Tom Lane t...@sss.pgh.pa.us wrote: Peter or Marc could clue you in a bit better, but I think it's as simple as saying make dist at the top level of a modified source tree. This gets you a source tarball the same way the release tarballs are made. I seem to be able to sneak up on it from

Re: [HACKERS] Upgrading our minimum required flex version for 8.5

2009-07-02 Thread Tom Lane
Kevin Grittner kevin.gritt...@wicourts.gov writes: I seem to be able to sneak up on it from behind by doing a regular make and then make distclean and copying the results. Perhaps someone knows off-hand what I'm missing that prevents make dist from working. The attempt ends with: jade -D

Re: [HACKERS] Upgrading our minimum required flex version for 8.5

2009-07-02 Thread Kevin Grittner
Tom Lane t...@sss.pgh.pa.us wrote: Kevin Grittner kevin.gritt...@wicourts.gov writes: jade -D . -d stylesheet.dsl -i output-text -t sgml -V nochunks tempfile_HISTORY.sgml HISTORY.html /bin/sh: jade: not found Looks like you're missing openjade. However, that's only used to produce

Re: [HACKERS] Upgrading our minimum required flex version for 8.5

2009-07-02 Thread Peter Eisentraut
On Thursday 02 July 2009 19:46:04 Tom Lane wrote: Kevin Grittner kevin.gritt...@wicourts.gov writes: Then I take it back -- the new flex versions would have little or no impact on me. Worst case, I might need to download a snapshot to apply my patch for testing on the big machines. If I

Re: [HACKERS] Upgrading our minimum required flex version for 8.5

2009-07-02 Thread Jaime Casanova
On Thu, Jul 2, 2009 at 12:13 PM, Tom Lanet...@sss.pgh.pa.us wrote: However I'm a bit worried about the situation for Windows --- does anyone know whether a newer flex is readily available for Windows? MSYS Suplementary Tools (for mingw) includes flex-2.5.33

Re: [HACKERS] Upgrading our minimum required flex version for 8.5

2009-07-02 Thread Kevin Grittner
Peter Eisentraut pete...@gmx.net wrote: make distprep is what prepares those files. Perfect! Flex files built. No errors. Thanks much! -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription:

Re: [HACKERS] Upgrading our minimum required flex version for 8.5

2009-07-02 Thread Zdenek Kotala
Tom Lane píše v čt 02. 07. 2009 v 11:32 -0400: Andrew Dunstan and...@dunslane.net writes: Tom Lane wrote: Right, this will only affect people doing development or otherwise building from a CVS pull. Of course, that includes the whole buildfarm. We might need to ask some people to

Re: [HACKERS] Upgrading our minimum required flex version for 8.5

2009-07-02 Thread Zdenek Kotala
Tom Lane píše v čt 02. 07. 2009 v 13:13 -0400: I wrote: Yes. What I was thinking of doing was committing a configure change to reject flex 2.5.31, and waiting to see how much of the buildfarm goes red. Actually, most of the buildfarm members show which flex version they are running