Re: [HACKERS] Large C files

2011-10-19 Thread Alvaro Herrera
Excerpts from David Fetter's message of lun oct 17 03:00:19 -0300 2011: On Fri, Oct 14, 2011 at 07:36:32PM +0100, Peter Geoghegan wrote: This evening, David Fetter described a problem to me that he was having building the Twitter FDW. It transpired that it was down to its dependence on an

Re: [HACKERS] Large C files

2011-10-17 Thread David Fetter
On Fri, Oct 14, 2011 at 07:36:32PM +0100, Peter Geoghegan wrote: This evening, David Fetter described a problem to me that he was having building the Twitter FDW. It transpired that it was down to its dependence on an #include that was recently judged to be redundant.This seems like another

Re: [HACKERS] Large C files

2011-10-14 Thread Peter Geoghegan
This evening, David Fetter described a problem to me that he was having building the Twitter FDW. It transpired that it was down to its dependence on an #include that was recently judged to be redundant.This seems like another reason to avoid using pgrminclude - even if we can be certain that the

Re: [HACKERS] Large C files

2011-10-14 Thread Alvaro Herrera
Excerpts from Peter Geoghegan's message of vie oct 14 15:36:32 -0300 2011: This evening, David Fetter described a problem to me that he was having building the Twitter FDW. It transpired that it was down to its dependence on an #include that was recently judged to be redundant.This seems like

Re: [HACKERS] Large C files

2011-09-24 Thread Hannu Krosing
On Wed, 2011-09-07 at 01:22 +0200, Jan Urbański wrote: On 07/09/11 01:13, Peter Geoghegan wrote: On 6 September 2011 08:29, Peter Eisentraut pete...@gmx.net wrote: I was thinking about splitting up plpython.c, but it's not even on that list. ;-) IIRC the obesity of that file is

Re: [HACKERS] Large C files

2011-09-24 Thread Bruce Momjian
Robert Haas wrote: On Fri, Sep 9, 2011 at 11:28 PM, Peter Geoghegan pe...@2ndquadrant.com wrote: It's very difficult or impossible to anticipate how effective the tool will be in practice, but when you consider that it works and does not produce false positives for the first 3 real-world

Re: [HACKERS] Large C files

2011-09-24 Thread Tom Lane
Bruce Momjian br...@momjian.us writes: Robert Haas wrote: I'm also curious to see how much more fallout we're going to see from that run. We had a few glitches when it was first done, but it didn't seem like they were really all that bad. It might be that we'd be better off running

Re: [HACKERS] Large C files

2011-09-24 Thread Bruce Momjian
Tom Lane wrote: Actually, I believe that the *main* problem with pgrminclude is that it fails to account for combinations of build options other than those that Bruce uses. In the previous go-round, the reason we were still squashing bugs months later is that it took that long for people to

Re: [HACKERS] Large C files

2011-09-24 Thread Tom Lane
Bruce Momjian br...@momjian.us writes: Tom Lane wrote: Actually, I believe that the *main* problem with pgrminclude is that it fails to account for combinations of build options other than those that Bruce uses. In the previous go-round, the reason we were still squashing bugs months later

Re: [HACKERS] Large C files

2011-09-24 Thread Peter Geoghegan
On 24 September 2011 16:41, Tom Lane t...@sss.pgh.pa.us wrote: Frankly, with the tool in its current state I'd rather not run it at all, ever.  The value per man-hour expended is too low.  The mess it made out of the xlog-related includes this time around makes me question whether it's even a

Re: [HACKERS] Large C files

2011-09-24 Thread Andrew Dunstan
On 09/24/2011 01:10 PM, Peter Geoghegan wrote: On 24 September 2011 16:41, Tom Lanet...@sss.pgh.pa.us wrote: Frankly, with the tool in its current state I'd rather not run it at all, ever. The value per man-hour expended is too low. The mess it made out of the xlog-related includes this

Re: [HACKERS] Large C files

2011-09-24 Thread Bruce Momjian
Tom Lane wrote: Bruce Momjian br...@momjian.us writes: Tom Lane wrote: Actually, I believe that the *main* problem with pgrminclude is that it fails to account for combinations of build options other than those that Bruce uses. In the previous go-round, the reason we were still

Re: [HACKERS] Large C files

2011-09-23 Thread Robert Haas
On Fri, Sep 9, 2011 at 11:28 PM, Peter Geoghegan pe...@2ndquadrant.com wrote: It's very difficult or impossible to anticipate how effective the tool will be in practice, but when you consider that it works and does not produce false positives for the first 3 real-world cases tested, it seems

Re: [HACKERS] Large C files

2011-09-23 Thread Peter Geoghegan
On 23 September 2011 15:46, Robert Haas robertmh...@gmail.com wrote: I'm not opposed to adding something like this, but I think it needs to either be tied into the actual running of the script, or have a lot more documentation than it does now, or both.  I am possibly stupid, but I can't

Re: [HACKERS] Large C files

2011-09-09 Thread Greg Stark
On Fri, Sep 9, 2011 at 6:57 AM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: In particular, I'd like to know what boundaries it is envisaged that the code should be refactored along to increase its conceptual integrity, or to better separate concerns. I assume that that's the

Re: [HACKERS] Large C files

2011-09-09 Thread Peter Geoghegan
I've produced the refinement of my little shell script anticipated by my last e-mail (using sed to remove irrelevant variations in __func__.12345 type symbol names). I decided to test it for: 1. Detecting behavioural changes when removing existing pgrminclude ignore files (Basically headers that

Re: [HACKERS] Large C files

2011-09-08 Thread Simon Riggs
On Wed, Sep 7, 2011 at 9:02 PM, Tom Lane t...@sss.pgh.pa.us wrote: Simon Riggs si...@2ndquadrant.com writes: Please lets not waste effort on refactoring efforts in mid dev cycle. Say what?  When else would you have us do it? When else would you have us develop? Major changes happen at start

Re: [HACKERS] Large C files

2011-09-08 Thread Tom Lane
Simon Riggs si...@2ndquadrant.com writes: On Wed, Sep 7, 2011 at 9:02 PM, Tom Lane t...@sss.pgh.pa.us wrote: Simon Riggs si...@2ndquadrant.com writes: Please lets not waste effort on refactoring efforts in mid dev cycle. Say what?  When else would you have us do it? When else would you have

Re: [HACKERS] Large C files

2011-09-08 Thread Bruce Momjian
Tom Lane wrote: Simon Riggs si...@2ndquadrant.com writes: On Wed, Sep 7, 2011 at 9:02 PM, Tom Lane t...@sss.pgh.pa.us wrote: Simon Riggs si...@2ndquadrant.com writes: Please lets not waste effort on refactoring efforts in mid dev cycle. Say what? ?When else would you have us do it?

Re: [HACKERS] Large C files

2011-09-08 Thread Robert Haas
On Thu, Sep 8, 2011 at 10:29 AM, Bruce Momjian br...@momjian.us wrote: Tom Lane wrote: Simon Riggs si...@2ndquadrant.com writes: On Wed, Sep 7, 2011 at 9:02 PM, Tom Lane t...@sss.pgh.pa.us wrote: Simon Riggs si...@2ndquadrant.com writes: Please lets not waste effort on refactoring efforts

Re: [HACKERS] Large C files

2011-09-08 Thread Simon Riggs
On Thu, Sep 8, 2011 at 3:25 PM, Tom Lane t...@sss.pgh.pa.us wrote: Simon Riggs si...@2ndquadrant.com writes: On Wed, Sep 7, 2011 at 9:02 PM, Tom Lane t...@sss.pgh.pa.us wrote: Simon Riggs si...@2ndquadrant.com writes: Please lets not waste effort on refactoring efforts in mid dev cycle. Say

Re: [HACKERS] Large C files

2011-09-08 Thread Bruce Momjian
Simon Riggs wrote: On Thu, Sep 8, 2011 at 3:25 PM, Tom Lane t...@sss.pgh.pa.us wrote: Simon Riggs si...@2ndquadrant.com writes: On Wed, Sep 7, 2011 at 9:02 PM, Tom Lane t...@sss.pgh.pa.us wrote: Simon Riggs si...@2ndquadrant.com writes: Please lets not waste effort on refactoring efforts

Re: [HACKERS] Large C files

2011-09-08 Thread Tom Lane
Simon Riggs si...@2ndquadrant.com writes: You clearly have the bit between your teeth on this. Personally, I'm neither intending to break up xlog.c right now, nor asking you to do it. I'm just objecting to your claim that there should be some project-policy restriction on when refactoring gets

Re: [HACKERS] Large C files

2011-09-08 Thread Peter Geoghegan
On 8 September 2011 15:43, Robert Haas robertmh...@gmail.com wrote: I wouldn't be too enthusiastic about starting a project like this in January, but now seems fine.  A bigger problem is that I don't hear anyone volunteering to do the work. You seem to have a fairly strong opinion on the

Re: [HACKERS] Large C files

2011-09-08 Thread Josh Berkus
Simon, Robert, Bruce, Tom, Say what? When else would you have us do it? When else would you have us develop? In my eyes that sort of activity *is* development. I find the distinction you are drawing entirely artificial, and more calculated to make sure refactoring never happens

Re: [HACKERS] Large C files

2011-09-08 Thread Robert Haas
On Thu, Sep 8, 2011 at 4:45 PM, Peter Geoghegan pe...@2ndquadrant.com wrote: On 8 September 2011 15:43, Robert Haas robertmh...@gmail.com wrote: I wouldn't be too enthusiastic about starting a project like this in January, but now seems fine.  A bigger problem is that I don't hear anyone

Re: [HACKERS] Large C files

2011-09-08 Thread Heikki Linnakangas
On 08.09.2011 23:45, Peter Geoghegan wrote: On 8 September 2011 15:43, Robert Haasrobertmh...@gmail.com wrote: I wouldn't be too enthusiastic about starting a project like this in January, but now seems fine. A bigger problem is that I don't hear anyone volunteering to do the work. You seem

Re: [HACKERS] Large C files

2011-09-07 Thread Robert Haas
On Tue, Sep 6, 2011 at 9:14 PM, Peter Geoghegan pe...@2ndquadrant.com wrote: On 7 September 2011 01:18, Bruce Momjian br...@momjian.us wrote: I am confused how moving a function from one C file to another could cause breakage? I'm really concerned about silent breakage, however unlikely -

Re: [HACKERS] Large C files

2011-09-07 Thread Bruce Momjian
Robert Haas wrote: IMHO, when manipulating code at this sort of macro scale, it's good to be paranoid and exhaustive, particularly when it doesn't cost you anything anyway. Unlike when writing or fixing a bit of code at a time, which we're all used to, if the compiler doesn't tell you

Re: [HACKERS] Large C files

2011-09-07 Thread Tom Lane
Bruce Momjian br...@momjian.us writes: Robert Haas wrote: I was less concerned about the breakage that might be caused by variables acquiring unintended referents - which should be unlikely anyway given reasonable variable naming conventions - and more concerned that the associated

Re: [HACKERS] Large C files

2011-09-07 Thread Simon Riggs
On Wed, Sep 7, 2011 at 7:12 PM, Tom Lane t...@sss.pgh.pa.us wrote: Bruce Momjian br...@momjian.us writes: Robert Haas wrote: I was less concerned about the breakage that might be caused by variables acquiring unintended referents - which should be unlikely anyway given reasonable variable

Re: [HACKERS] Large C files

2011-09-07 Thread Tom Lane
Simon Riggs si...@2ndquadrant.com writes: Please lets not waste effort on refactoring efforts in mid dev cycle. Say what? When else would you have us do it? regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your

Re: [HACKERS] Large C files

2011-09-06 Thread Peter Eisentraut
On lör, 2011-09-03 at 19:18 -0400, Bruce Momjian wrote: FYI, here are all the C files with over 6k lines: - 45133 ./interfaces/ecpg/preproc/preproc.c - 33651 ./backend/parser/gram.c - 17551 ./backend/parser/scan.c 14209 ./bin/pg_dump/pg_dump.c 10590 ./backend/access/transam/xlog.c

Re: [HACKERS] Large C files

2011-09-06 Thread Bruce Momjian
Peter Eisentraut wrote: On l?r, 2011-09-03 at 19:18 -0400, Bruce Momjian wrote: FYI, here are all the C files with over 6k lines: - 45133 ./interfaces/ecpg/preproc/preproc.c - 33651 ./backend/parser/gram.c - 17551 ./backend/parser/scan.c 14209 ./bin/pg_dump/pg_dump.c 10590

Re: [HACKERS] Large C files

2011-09-06 Thread Simon Riggs
On Tue, Sep 6, 2011 at 3:05 PM, Bruce Momjian br...@momjian.us wrote: Peter Eisentraut wrote: On l?r, 2011-09-03 at 19:18 -0400, Bruce Momjian wrote: FYI, here are all the C files with over 6k lines: -  45133 ./interfaces/ecpg/preproc/preproc.c -  33651 ./backend/parser/gram.c -  17551

Re: [HACKERS] Large C files

2011-09-06 Thread Robert Haas
On Tue, Sep 6, 2011 at 3:56 PM, Simon Riggs si...@2ndquadrant.com wrote: The only way I would entertain thoughts of major refactoring would be if comprehensive tests were contributed, so we could verify everything still works afterwards. That sounds like a really good idea. Mind you, I don't

Re: [HACKERS] Large C files

2011-09-06 Thread Peter Geoghegan
On 6 September 2011 21:07, Robert Haas robertmh...@gmail.com wrote: On Tue, Sep 6, 2011 at 3:56 PM, Simon Riggs si...@2ndquadrant.com wrote: The only way I would entertain thoughts of major refactoring would be if comprehensive tests were contributed, so we could verify everything still works

Re: [HACKERS] Large C files

2011-09-06 Thread Jan Urbański
On 07/09/11 01:13, Peter Geoghegan wrote: On 6 September 2011 08:29, Peter Eisentraut pete...@gmx.net wrote: I was thinking about splitting up plpython.c, but it's not even on that list. ;-) IIRC the obesity of that file is something that Jan Urbański intends to fix, or is at least

Re: [HACKERS] Large C files

2011-09-06 Thread Peter Geoghegan
On 7 September 2011 00:13, Peter Geoghegan pe...@2ndquadrant.com wrote: * Within TUs, we unshadow a previously shadowed variable, so we link to a global variable rather than one local to the original/other new file. Unlikely to be a problem. Here's what I get when I compile xlog.c in the usual

Re: [HACKERS] Large C files

2011-09-06 Thread Bruce Momjian
Peter Geoghegan wrote: On 7 September 2011 00:13, Peter Geoghegan pe...@2ndquadrant.com wrote: * Within TUs, we unshadow a previously shadowed variable, so we link to a global variable rather than one local to the original/other new file. Unlikely to be a problem. Here's what I get when I

Re: [HACKERS] Large C files

2011-09-06 Thread Peter Geoghegan
On 7 September 2011 01:18, Bruce Momjian br...@momjian.us wrote: I am confused how moving a function from one C file to another could cause breakage? I'm really concerned about silent breakage, however unlikely - that is also Simon and Robert's concern, and rightly so. If it's possible in

Re: [HACKERS] Large C files

2011-09-05 Thread Alvaro Herrera
Excerpts from Bruce Momjian's message of sáb sep 03 20:18:47 -0300 2011: FYI, here are all the C files with over 6k lines: - 45133 ./interfaces/ecpg/preproc/preproc.c - 33651 ./backend/parser/gram.c - 17551 ./backend/parser/scan.c 14209 ./bin/pg_dump/pg_dump.c 10590

Re: [HACKERS] Large C files

2011-09-05 Thread Robert Haas
On Mon, Sep 5, 2011 at 6:56 PM, Alvaro Herrera alvhe...@commandprompt.com wrote: Excerpts from Bruce Momjian's message of sáb sep 03 20:18:47 -0300 2011: FYI, here are all the C files with over 6k lines: -  45133 ./interfaces/ecpg/preproc/preproc.c -  33651 ./backend/parser/gram.c -  17551

[HACKERS] Large C files

2011-09-03 Thread Bruce Momjian
FYI, here are all the C files with over 6k lines: - 45133 ./interfaces/ecpg/preproc/preproc.c - 33651 ./backend/parser/gram.c - 17551 ./backend/parser/scan.c 14209 ./bin/pg_dump/pg_dump.c 10590 ./backend/access/transam/xlog.c 9764 ./backend/commands/tablecmds.c 8681