Re: [HACKERS] Japanies translation breaks solaris build
Takahiro Itagaki píše v pá 14. 05. 2010 v 19:38 +0900: > Zdenek Kotala wrote: > > > http://www.pgbuildfarm.org/cgi-bin/show_log.pl?nm=comet_moth&dt=2010-05-13%2021:06:01 > > The problem is that it contains mix of DOS/Unix end of lines. > > I removed two CRs in ja.po. Thanks. Gothic moth is green again http://www.pgbuildfarm.org/cgi-bin/show_history.pl?nm=gothic_moth&br=HEAD The rest solaris machine will recover during a night. Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Japanies translation breaks solaris build
http://www.pgbuildfarm.org/cgi-bin/show_log.pl?nm=comet_moth&dt=2010-05-13%2021:06:01 msgfmt -o po/ja.mo po/ja.po WARNING: the string after closing " is ignored at line number 11. Error, No space after directive at line number 2008. ERROR: Exiting... gmake[2]: *** [po/ja.mo] Error 2 The problem is that it contains mix of DOS/Unix end of lines. Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] gothic_moth, codlin_moth failures on REL8_2_STABLE
Tom Lane píše v čt 11. 03. 2010 v 11:37 -0500: > Zdenek Kotala writes: > > "-xO4 -xalias_level=basic" generates problem. > > "-xO3 -xalias_level=basic" works fine > > "-xO5" works fine > > > As documentation say: > > > Cite from Sun studio compiler guide: > > http://docs.sun.com/app/docs/doc/819-5265/bjapp?a=view > > > xalias_level=basic > > -- > > If you use the -xalias_level=basic option, the compiler assumes that > > memory references that involve different C basic types do not alias each > > other. The compiler also assumes that references to all other types can > > alias each other as well as any C basic type. The compiler assumes that > > references using char * can alias any other type. > > > For example, at the -xalias_level=basic level, the compiler assumes that > > a pointer variable of type int * is not going to access a float object. > > Therefore it is safe for the compiler to perform optimizations that > > assume a pointer of type float * will not alias the same memory that is > > referenced with a pointer of type int *. > > I think you need to turn that off. On gcc we use -fno-strict-aliasing > which disables the type of compiler assumption that this is talking about. > I'm not sure exactly how that might create the specific failure we are > seeing here, but I can point you to lots and lots of places in the > sources where such an assumption would break things. Reconfigured and both animal are green. Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] gothic_moth, codlin_moth failures on REL8_2_STABLE
Dne 11.03.10 17:37, Tom Lane napsal(a): Zdenek Kotala writes: "-xO4 -xalias_level=basic" generates problem. "-xO3 -xalias_level=basic" works fine "-xO5" works fine As documentation say: Cite from Sun studio compiler guide: http://docs.sun.com/app/docs/doc/819-5265/bjapp?a=view xalias_level=basic -- If you use the -xalias_level=basic option, the compiler assumes that memory references that involve different C basic types do not alias each other. The compiler also assumes that references to all other types can alias each other as well as any C basic type. The compiler assumes that references using char * can alias any other type. For example, at the -xalias_level=basic level, the compiler assumes that a pointer variable of type int * is not going to access a float object. Therefore it is safe for the compiler to perform optimizations that assume a pointer of type float * will not alias the same memory that is referenced with a pointer of type int *. I think you need to turn that off. On gcc we use -fno-strict-aliasing which disables the type of compiler assumption that this is talking about. I'm not sure exactly how that might create the specific failure we are seeing here, but I can point you to lots and lots of places in the sources where such an assumption would break things. OK. I first try to update compiler to latest version to see if it helps and finally I will remove aliasing. Thanks Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] gothic_moth, codlin_moth failures on REL8_2_STABLE
Dne 11.03.10 16:24, Greg Stark napsal(a): On Wed, Mar 10, 2010 at 11:37 PM, Tom Lane wrote: My conclusion is that this is probably a compiler bug. Both buildfarm animals appear to be using Sun Studio, although on different architectures which weakens the compiler-bug theory a bit. Even though we are not seeing the failure in later PG branches, it would probably be worth looking into more closely, because if it's biting 8.2 it could start biting newer code as well. But without access to a machine showing the problem it is difficult to do much. Could be this: http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6750087 It's fixed in patch 124861-11 which came out Feb 23, 2009. Is this patch missing on both gothic-moth and codlin-moth? It seems as a our case. See compiler versions: Ghost: -bash-3.2$ cc -V cc: Sun C 5.9 SunOS_sparc Patch 124867-09 2008/11/25 Codlin -bash-4.0$ cc -V cc: Sun C 5.9 SunOS_i386 Patch 124868-10 2009/04/30 I should apply patch on Ghost, but Codlin have to wait, because I don't have a control on compiler version. I try to find update SS12 somewhere on the disk/network. The patch which you refer does not fix cc itself but some others binaries/libs which cc uses. I try to update Ghost and we will see what happen. I suppose it's possible to have a configure test to check for whether this patch is present but I'm not sure how much it's worthwhile given that it'll only help people who happen to recompile their 8.2 server after the next Postgres patch. And I'm not sure we can check for patches without assuming the CC is the OS-shipped cc. Does cc itself have an option to list which patches it has applied to it? cc is not shipped with solaris you have to install it separately. And bug appear only when you use high optimization (see my email). You can see patch version when you run cc -V but you see only compiler version. Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] gothic_moth, codlin_moth failures on REL8_2_STABLE
Hi Tom, I'm sorry that I did not look on it early. I played with it and there are some facts. gothic(sparc) and codlin(x86) uses Sun Studio 12 nad I setup them to use very high optimization. Gothic: --- -xalias_level=basic -xarch=native -xdepend -xmemalign=8s -xO5 -xprefetch=auto,explicit Codlin: --- -xalias_level=basic -xarch=native -xdepend -xO4 -xprefetch=auto,explicit -xO5 is highest optimization, -xO4 is little bit worse A play with flags and found that "-xO4 -xalias_level=basic" generates problem. "-xO3 -xalias_level=basic" works fine "-xO5" works fine As documentation say: Cite from Sun studio compiler guide: http://docs.sun.com/app/docs/doc/819-5265/bjapp?a=view xalias_level=basic -- If you use the -xalias_level=basic option, the compiler assumes that memory references that involve different C basic types do not alias each other. The compiler also assumes that references to all other types can alias each other as well as any C basic type. The compiler assumes that references using char * can alias any other type. For example, at the -xalias_level=basic level, the compiler assumes that a pointer variable of type int * is not going to access a float object. Therefore it is safe for the compiler to perform optimizations that assume a pointer of type float * will not alias the same memory that is referenced with a pointer of type int *. -x04 - Preforms automatic inlining of functions contained in the same file in addition to performing -xO3 optimizations. This automatic inlining usually improves execution speed, but sometimes makes it worse. In general, this level results in increased code size. I redefined bitfields to char in HLWORD and it works. Your guess is correct. But question still where is the place when bitfields works bad. Any idea where I should look? IIRC, I had this problem also on head, when I tried to fix tsearch regression test for Czech locale. This problem appears and disappears. Zdenek Dne 11.03.10 00:37, Tom Lane napsal(a): Since the buildfarm is mostly green these days, I took some time to look into the few remaining consistent failures. One is that gothic_moth and codlin_moth fail on contrib/tsearch2 in the 8.2 branch, with a regression diff like this: *** 2453,2459 Sea view wowfoo bar qq http://www.google.com/foo.bar.html"; target="_blank">YES ! ff-bg document.write(15); --- 2453,2459 Sea view wowfoo bar qq http://www.google.com/foo.bar.html"; target="_blank">YES ! ff-bgff-bg document.write(15); These animals are not testing any branches older than 8.2. The same test appears in newer branches and passes, but the code involved got migrated to core and probably changed around a bit. I traced through this test on my own machine and determined that the way it's supposed to work is like this: the tsearch parser breaks the string into a series of tokens that include these: ff-bg compound word ff compound word element - punctuation bg compound word element The highlight function is then supposed to set skip = 1 on the compound word, causing it to be skipped when genhl() reconstructs the text. The failure looks to me like the compound word is not getting skipped. Both the setting and the testing of the flag seem to be absolutely straightforward portable code; although the "skip" struct field is a bitfield, which is a C feature we don't use very heavily. My conclusion is that this is probably a compiler bug. Both buildfarm animals appear to be using Sun Studio, although on different architectures which weakens the compiler-bug theory a bit. Even though we are not seeing the failure in later PG branches, it would probably be worth looking into more closely, because if it's biting 8.2 it could start biting newer code as well. But without access to a machine showing the problem it is difficult to do much. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] psql with GSS can crash
Magnus Hagander píše v po 01. 03. 2010 v 16:55 +0100: > 2010/3/1 Zdenek Kotala : > > Magnus Hagander píše v čt 25. 02. 2010 v 15:17 +0100: > >> On Thu, Feb 25, 2010 at 15:04, Zdenek Kotala wrote: > >> > Hi all, > >> > > >> > I got following stack: > >> > > >> > fd7ffed14b70 strlen () + 40 > >> > fd7ffed71665 snprintf () + e5 > >> > fd7fff36d088 pg_GSS_startup () + 88 > >> > fd7fff36d43a pg_fe_sendauth () + 15a > >> > fd7fff36e557 PQconnectPoll () + 3b7 > >> > fd7fff36e152 connectDBComplete () + a2 > >> > fd7fff36dc32 PQsetdbLogin () + 1b2 > >> > 0041e96d main () + 30d > >> > 0041302c () > >> > > >> > It seems that connection is not fully configured and krbsrvname or > >> > pghost is > >> > not filled. Following code in fe-auth.c pg_GSS_startup() causes a crash: > >> > > >> >440 maxlen = NI_MAXHOST + strlen(conn->krbsrvname) + 2; > >> >441 temp_gbuf.value = (char *) malloc(maxlen); > >> >442 snprintf(temp_gbuf.value, maxlen, "%...@%s", > >> >443 conn->krbsrvname, conn->pghost); > >> >444 temp_gbuf.length = strlen(temp_gbuf.value); > >> > > >> > And following code in fe-connect.c fillPGconn() fill NULL value. > >> > > >> >571 tmp = conninfo_getval(connOptions, "krbsrvname"); > >> >572 conn->krbsrvname = tmp ? strdup(tmp) : NULL; > >> > > >> > I think that pg_GSS_startup should sanity the input. > >> > >> How did you get NULL in there? :-) > >> There's a default set for that one that's PG_KRB_SRVNAM, so it really > >> should never come out as NULL, I think... > > > > Yeah, you are right. conn->krbsrvname is "postgres" and conn->pghost is > > null > > Ah, good. We should defentd against that then. > > > >> As for pghost, that certainly seems to be a bug. We check that one in > >> krb5 and SSPI, but for some reason we seem to be missing it in GSSAPI. > > > > Yes. The check should be in GSSAPI too. > > > > However what I see in pg_hba.conf is following line: > > > > local all all gss > > > > Gss is used on local unix socket which probably cause a problem that > > conn->pghost is not filled when psql tries to connect. > > So there are really two errors - because we should disallow that. > > See attached patch - can you confirm it removes the crash with just > the client side applied, and then that it properly rejects GSS with > the server side applied as well? I tested it, but I cannot reproduce crash because I cannot setup illegal combination now ;-). I think it is OK. Thanks Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] psql with GSS can crash
Magnus Hagander píše v čt 25. 02. 2010 v 15:17 +0100: > On Thu, Feb 25, 2010 at 15:04, Zdenek Kotala wrote: > > Hi all, > > > > I got following stack: > > > > fd7ffed14b70 strlen () + 40 > > fd7ffed71665 snprintf () + e5 > > fd7fff36d088 pg_GSS_startup () + 88 > > fd7fff36d43a pg_fe_sendauth () + 15a > > fd7fff36e557 PQconnectPoll () + 3b7 > > fd7fff36e152 connectDBComplete () + a2 > > fd7fff36dc32 PQsetdbLogin () + 1b2 > > 0041e96d main () + 30d > > 0041302c () > > > > It seems that connection is not fully configured and krbsrvname or pghost is > > not filled. Following code in fe-auth.c pg_GSS_startup() causes a crash: > > > >440 maxlen = NI_MAXHOST + strlen(conn->krbsrvname) + 2; > >441 temp_gbuf.value = (char *) malloc(maxlen); > >442 snprintf(temp_gbuf.value, maxlen, "%...@%s", > >443 conn->krbsrvname, conn->pghost); > >444 temp_gbuf.length = strlen(temp_gbuf.value); > > > > And following code in fe-connect.c fillPGconn() fill NULL value. > > > >571 tmp = conninfo_getval(connOptions, "krbsrvname"); > >572 conn->krbsrvname = tmp ? strdup(tmp) : NULL; > > > > I think that pg_GSS_startup should sanity the input. > > How did you get NULL in there? :-) > There's a default set for that one that's PG_KRB_SRVNAM, so it really > should never come out as NULL, I think... Yeah, you are right. conn->krbsrvname is "postgres" and conn->pghost is null > As for pghost, that certainly seems to be a bug. We check that one in > krb5 and SSPI, but for some reason we seem to be missing it in GSSAPI. Yes. The check should be in GSSAPI too. However what I see in pg_hba.conf is following line: local all all gss Gss is used on local unix socket which probably cause a problem that conn->pghost is not filled when psql tries to connect. thanks Zdenek Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] psql with GSS can crash
Hi all, I got following stack: fd7ffed14b70 strlen () + 40 fd7ffed71665 snprintf () + e5 fd7fff36d088 pg_GSS_startup () + 88 fd7fff36d43a pg_fe_sendauth () + 15a fd7fff36e557 PQconnectPoll () + 3b7 fd7fff36e152 connectDBComplete () + a2 fd7fff36dc32 PQsetdbLogin () + 1b2 0041e96d main () + 30d 0041302c () It seems that connection is not fully configured and krbsrvname or pghost is not filled. Following code in fe-auth.c pg_GSS_startup() causes a crash: 440 maxlen = NI_MAXHOST + strlen(conn->krbsrvname) + 2; 441 temp_gbuf.value = (char *) malloc(maxlen); 442 snprintf(temp_gbuf.value, maxlen, "%...@%s", 443 conn->krbsrvname, conn->pghost); 444 temp_gbuf.length = strlen(temp_gbuf.value); And following code in fe-connect.c fillPGconn() fill NULL value. 571 tmp = conninfo_getval(connOptions, "krbsrvname"); 572 conn->krbsrvname = tmp ? strdup(tmp) : NULL; I think that pg_GSS_startup should sanity the input. Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] codlin_month is up and complain - PL/Python crash
Dne 17.02.10 18:39, Peter Eisentraut napsal(a): On ons, 2010-02-17 at 11:26 -0500, Tom Lane wrote: But the behavior gcc appears to exhibit is that it won't warn about variables that are only assigned once before the PG_TRY is entered, and that seems reasonable to me since such a variable ought to have the correct value either way. FWIW, this is a Sun Studio build that is complaining here. Yes It is SS12. I add volatile keyword and problem disappears. The code difference is following: < PLy_spi_execute+0x742: 83 ec 0c subl $0xc,%esp < PLy_spi_execute+0x745: ff b5 b8 f9 ff ff pushl 0xf9b8(%ebp) < PLy_spi_execute+0x74b: e8 fc ff ff ff call MemoryContextSwitch > PLy_spi_execute+0x742: 8b 85 cc f9 ff ff movl 0xf9cc(%ebp),%eax > PLy_spi_execute+0x748: 83 ec 0c subl $0xc,%esp > PLy_spi_execute+0x74b: 50 pushl %eax > PLy_spi_execute+0x74c: e8 fc ff ff ff call MemoryContextSwitch Good to mention that SS inline PLy_spi_execute_query inside PLy_spi_execute(), because it is only one caller. Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] codlin_month is up and complain - PL/Python crash
I revived codlin_month and it falls during PL/Python test: http://www.pgbuildfarm.org/cgi-bin/show_log.pl?nm=codlin_moth&dt=2010-02-16%2015:09:05 TRAP: BadArgument("!(((context) != 0 && (Node*)((context)))->type) == T_AllocSetContext", File: "mcxt.c", Line: 641) feaf5005 _lwp_kill (1, 6, 80459c8, fea9bbde) + 15 fea9bbea raise(6, 0, 8045a18, fea725aa) + 22 fea725ca abort(8046670, 8361f80, 8045a48, 8719ccf, 89021f0, 89021e4) + f2 086d07c0 ExceptionalCondition (89021f0, 89021e4, 89021dc, 281) + 58 08719ccf MemoryContextSwitchTo (89264ac, 0, 0, 8045a7c) + 47 fec21990 PLy_spi_execute (0, 8b141cc, 80460f8, fe84abde) + 750 fe84ad6e PyCFunction_Call (8b0ff6c, 8b141cc, 0, fe8a8d92) + 19e fe8a91a0 call_function (80461bc, 1, 610f2d31, fe8a3206) + 41c fe8a6221 PyEval_EvalFrameEx (8b5798c, 0, 8b0cbdc, 0) + 3029 fe8a9310 fast_function (8b05144, 80462fc, 0, 0, 0, fe91c63c) + 108 fe8a8e72 call_function (80462fc, 0, 80462d8, fe8a3206) + ee fe8a6221 PyEval_EvalFrameEx (8b576a4, 0, 8b0cbdc, 8b0cbdc) + 3029 fe8a7cd0 PyEval_EvalCodeEx (8ab4770, 8b0cbdc, 8b0cbdc, 0, 0, 0) + 91c fe8a3102 PyEval_EvalCode (8ab4770, 8b0cbdc, 8b0cbdc, fec17831) + 32 fec1799c PLy_function_handler (8046980, 8b5d508, 8046880, fec1480f) + 17c fec14b92 plpython_call_handler (8046980, 8046bb0, 8046be8, 8323774) + 3aa 08324393 ExecEvalFunc (8a033b0, 8a0329c, 8a0390c, 8a039b8) + e33 0832b1bc ExecProject (8a03920, 8046c6c, 2, 8977abc) + 834 08348785 ExecResult (8a03210, 8a03184, 0, 1) + 9d 0831f66f ExecProcNode (8a03210, 1, 8a037ec, 8731314) + 227 0831a186 ExecutorRun (8a02d7c, 1, 0, 8719ad4) + 2de 084d7778 PortalRun (898effc, 7fff, 1, 8977b38, 8977b38) + 450 084ceae9 exec_simple_query (8976984, 0, 80473b8, 84d5185) + ba9 084d51a2 PostgresMain (2, 8973b4c, 897398c, 893d00c, 893d008, 130d7661) + 7fa 0844aded BackendRun (898c3d0) + 1cd 084440f3 ServerLoop (1, 89561d4, 3, fea7bb7e, 5c54, feb83cd8) + 973 08443004 PostmasterMain (3) + 119c 0837db12 main (3, 8047b14, 8047b24, 80fa21f) + 1ea 080fa27d _start (3, 8047be8, 8047fb0, 8047fb0, 0, 8047c35) + 7d It seems that problem is with compiler aggressive optimization. I change it to lower level and now it works fine. Interesting is that MemoryContext corruption only appears with PL/Python. Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] buildfarm breakage
Andrew Dunstan píše v po 08. 02. 2010 v 20:07 -0500: > > Our Solaris *moth members seem to have stopped building. Have we lost them? Hi Andrew, The answer is not simple. Yes, we lost Solaris 8 and 9 machines which was reinstalled and now they are used for different purpose. It was planned before the April and I announced it long time ago. It unfortunately happed and timing looks strange. And I did not find replacement. I have replacement for nevada/x86 machine already, but I need to setup it which is one item in my very long TODO list :(. Solaris 10 Sparc/x86 and nevada sparc are covered at this moment. Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Deadlock in vacuum (check fails)
I found following strange error on gothic moth: == pgsql.22658/src/test/regress/regression.diffs === *** /zfs_data/home/postgres/buildfarm/gothic_moth/HEAD/pgsql.22658/src/test/regress/expected/vacuum.out Tue Jan 12 22:06:13 2010 --- /zfs_data/home/postgres/buildfarm/gothic_moth/HEAD/pgsql.22658/src/test/regress/results/vacuum.out Tue Jan 12 22:35:25 2010 *** *** 94,99 --- 94,103 -- only non-system tables should be changed VACUUM FULL pg_am; VACUUM FULL pg_class; + ERROR: deadlock detected + DETAIL: Process 5913 waits for AccessExclusiveLock on relation 2662 of database 16384; blocked by process 5915. + Process 5915 waits for AccessShareLock on relation 1259 of database 16384; blocked by process 5913. + HINT: See server log for query details. VACUUM FULL pg_database; VACUUM FULL vaccluster; VACUUM FULL vacid; Detail here: http://www.pgbuildfarm.org/cgi-bin/show_log.pl?nm=gothic_moth&dt=2010-01-12%2021:06:01 Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] pg_migrator issues
Dne 4.01.10 19:28, Alvaro Herrera napsal(a): Bruce Momjian escribió: I considered that but realize that pg_migrator has to read pg_controldata in both the old and new servers, meaning it would need access to both C structures, and considering they both have the same structure names, that would require some odd C tricks. Add to that you don't know which version of Postgres you are migrating from/to during compile and the idea of using C becomes even less attractive. However, keep in mind that this might not be the last time on which we will want to read something from a C struct, so perhaps it would be good to bite the bullet and write the odd tricks. Does it already have access (at compile time) to the old and new source trees? I have some proof of concept when each control data struct version version have one header file like pg_control_843.h and structure like ControlFileData has name ControlFileData_843. The main pg_control.h defines types without version like typedef ControlFileData_843 ControlFileData; I planed to do it for 8.5 but unfortunately no time :( commit fest is too close. Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [patch] executor and slru dtrace probes
Greg Smith píše v út 15. 12. 2009 v 12:10 -0500: > Please send that updated version, and let's keep working on this into > the next CommitFest, where it will be in the front of the queue rather > than how it ended up at the tail of this one just based on its > submission date. You're not really getting a fair chunk of time here > between your review and the end here because of problems lining up > reviewer time, that shouldn't happen next time. Make sense. Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [patch] executor and slru dtrace probes
Bernd Helmle píše v po 14. 12. 2009 v 20:42 +0100: > > --On 14. Dezember 2009 20:33:12 +0100 Bernd Helmle > wrote: > > >> Since the author has pretty much admitted he didn't fix any of the > >> issues that were raised by the last committer review, I'm a little > >> confused about why you're asking for another one. > > > > It wasn't clear to me what Zdenek meant with "If I think about it". > > Oh, and i was under the opinion the last discussions were about executor > probes only (note the patch is split up into two parts now, SLRU and > executor probes). The latter won't be fixed, but it seems the SLRU part at > least is ready. > I would like to add SimpleLruInit probe. I'm busy with PG packaging, but I hope that I will send updated version tomorrow. Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCH] dtrace probes for memory manager
Tom Lane píše v pá 11. 12. 2009 v 15:11 -0500: > Zdenek Kotala writes: > > I thought about it. I think we can use GUC variable (e.g. dtraced_alloc) > > and hook switch pointers to dtraced AsetFunctions. The problem is how to > > distribute to all backend. > > You set the GUC in postgresql.conf. No big deal. > > If we go this route it would be nice to think about making a facility > that has some usefulness for non-DTrace platforms too. Do you mean general facility for switching memory allocator? Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCH] dtrace probes for memory manager
Tom Lane píše v pá 11. 12. 2009 v 14:38 -0500: > Robert Haas writes: > > I thought we had an idea of using the AllocSet dispatch mechanism to > > make this zero-overhead in the case where the probes are not enabled. > > What happened to that notion? > > I must have missed that discussion, but +1 --- should be possible to get > to zero-overhead-when-off that way. The trick is to figure out > what/where enables the alternate implementation. The current design > assumes that the callers of FooContextCreate choose the implementation, > but we don't want that here. I thought about it. I think we can use GUC variable (e.g. dtraced_alloc) and hook switch pointers to dtraced AsetFunctions. The problem is how to distribute to all backend. Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCH] dtrace probes for memory manager
Tom Lane píše v pá 11. 12. 2009 v 13:56 -0500: > Robert Haas writes: > > As far as I am concerned that is way too much, particularly > > considering that your test case isn't designed to be particularly > > memory-allocation intensive, and if it is up to me I will reject this. > > Even a quarter-percent slowdown for a feature that will be used only > > by a small fraction of users only a small fraction of time time seems > > totally unacceptable to me. > > It seems to me that anyone who really needs this can instrument the > alloc functions anyway --- isn't one of the features of DTrace supposed > to be that you can monitor calls to a particular function without any > prearranged code support? Or is that one of the things like "zero > overhead" that turns out to be more marketing-speak than reality? There are several types of probes. For example for PID provider probes you can monitor all entry and return point from global function. And also you can put probe on each asm instruction in the function. These probes have zero overhead, because dtrace understand ABI and know where args are. Unfortunately user defined probes has small overhead which is price for universal solution which works with all compilers and linkers. > Anyway I concur with Robert's opinion that the use-case is far too small > to justify incurring a measurable overhead for everybody. OK. > There might > be some small argument for putting these in under an extra #ifdef, but > they wouldn't get into any regular production build. unfortunately #ifdef solution kills main dtrace goal - without recompilation :(. Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCH] dtrace probes for memory manager
Robert Haas píše v pá 11. 12. 2009 v 12:55 -0500: > On Fri, Dec 11, 2009 at 12:55 PM, Zdenek Kotala wrote: > > Bernd Helmle píše v pá 11. 12. 2009 v 17:13 +0100: > >> > >> --On 11. Dezember 2009 11:28:54 -0300 Alvaro Herrera > >> wrote: > >> > >> >> > >> >> without compiled probes: AVG(2531.68) > >> >> with compiled probes: AVG(2511.97) > >> > > >> > Were the probes enabled? > >> > >> Hmm, they were just compiled in, i didn't anything to instrument them with > >> dtrace. > >> > >> I've just started a pgbench/dtrace run with instrumented probes aset_alloc, > >> aset_free and aset_realloc which just counts the calls to them during > >> pgbench, the first run gives me > >> > >> tps = 1035.045523 (excluding connections establishing) > >> > >> Ideas? > > > > When probes are activated they have performance impact. It is normal. > > Important is that you can use it when you need it on production system. > > No recompilation, no extra binary, no downtime and it is safe. > > Performance impact depends on Dscript > > Yeah. The problem here is the impact when the probes aren't enabled. > > I thought we had an idea of using the AllocSet dispatch mechanism to > make this zero-overhead in the case where the probes are not enabled. > What happened to that notion? We know that performance impact is less then 1% probably less then 0.6%. The question is if it is acceptable or not. I personally think that it is acceptable. However if not, I will start work on backup solution with dtraced AllocSet and some switching mechanism. But it needs little discussion about design. And first we need decision about current performance impact. Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCH] dtrace probes for memory manager
Robert Haas píše v čt 10. 12. 2009 v 23:55 -0500: > On Wed, Dec 9, 2009 at 9:04 AM, Zdenek Kotala wrote: > > > > But in normal situation database does also other thing and palloc is > > only one part of code path. It is why I run second test and use sun > > studio profiling tools (collect/analyzer) to determine how much CPU > > ticks cost the probes during pg_bench run. And results are much better. > > AllocSet alloc function takes about 4-5% and probes assembler code takes > > 0.1-0.2% on 64bit. I did not test 32bit but my expectation is that it > > should be about 0.3-0.4%. > > There's not really enough detail here to determine what you tested and > what the results were, and I don't think this patch has any chance at > all of getting committed without that. Please clarify. > > If there's some real-world test where this probe costs 0.3%-0.4%, I > think that is sufficient grounds for rejecting this patch. I > understand the desire of people to be able to use dtrace, but our > performance is too hard-won for me to want to give any measurable of > it up for tracing and instrumentation hooks that will only be used by > a small number of users in a small number of situations. > As I mentioned I run pg_bench -c10 -t1000 and collect data from backends. collect and analyzer is similar tool to gprof. Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCH] dtrace probes for memory manager
Bernd Helmle píše v pá 11. 12. 2009 v 17:13 +0100: > > --On 11. Dezember 2009 11:28:54 -0300 Alvaro Herrera > wrote: > > >> > >> without compiled probes: AVG(2531.68) > >> with compiled probes: AVG(2511.97) > > > > Were the probes enabled? > > Hmm, they were just compiled in, i didn't anything to instrument them with > dtrace. > > I've just started a pgbench/dtrace run with instrumented probes aset_alloc, > aset_free and aset_realloc which just counts the calls to them during > pgbench, the first run gives me > > tps = 1035.045523 (excluding connections establishing) > > Ideas? When probes are activated they have performance impact. It is normal. Important is that you can use it when you need it on production system. No recompilation, no extra binary, no downtime and it is safe. Performance impact depends on Dscript Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCH] dtrace probes for memory manager
Frank Ch. Eigler píše v čt 10. 12. 2009 v 14:11 -0500: > Zdenek Kotala writes: > > > [...] > > + header = (StandardChunkHeader *) > > + ((char *) ret - STANDARDCHUNKHEADERSIZE); > > + > > +// TRACE_POSTGRESQL_MCXT_ALLOC(context->name, context, size, header->size, > > true); > > + > > [...] > > If the dormant overhead of these probes is measured or suspected to be > excessive, consider using the dtrace-generated per-probe foo_ENABLED() > conditional, or a postgres configuration global thusly: TRACE_POSTGRESQL_MCXT_ALLOC and TRACE_POSTGRESQL_ASET_ALLOC are duplicated probes. Have them both make sense but from performance point of view to have one of them is acceptable. foo_enable() is good to use when number of argument and their evaluation cost too much. In this case it does no seem to be much useful. See ASM code: AllocSetAlloc+0x17: xorq %rax,%rax AllocSetAlloc+0x1a: nop AllocSetAlloc+0x1b: nop AllocSetAlloc+0x1c: testl %eax,%eax AllocSetAlloc+0x1e: je +0xb AllocSetAlloc+0x20: movq %r13,%rdi AllocSetAlloc+0x23: movq %r14,%rsi AllocSetAlloc+0x26: nop AllocSetAlloc+0x27: nop AllocSetAlloc+0x28: nop AllocSetAlloc+0x29: nop AllocSetAlloc+0x2a: nop >if (__builtin_expect(TRACE_POSTGRESQL_MCXT_ALLOC_ENABLED(), 0)) > TRACE_POSTGRESQL_MCXT_ALLOC(...); > > so that the whole instrumentation parameter setup/call can be placed > out of the hot line with gcc -freorder-blocks. compiler specific construct is not good way. Do not forget that also other compiler exists. Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [patch] executor and slru dtrace probes
Dne 10.12.09 15:51, Bernd Helmle napsal(a): --On 8. Dezember 2009 11:10:44 +0100 Zdenek Kotala wrote: If you think that it is better I could split patch into two separate patches and both can be reviewed separately. I split up this patch into two separate patches: one for SLRU and one for the executor probes. I've done some documentation on the SLRU part, maybe you can look over it (to make sure i didn't break anything). I left the executor probes out of the documentation for now, since it seems not to be clear how they would manifest. Out of curiosity: Why do we want to pass the SlruCtl pointer down to the probes? I don't understand what those probes are going to do with those pointers, can you explain? You need to determine which SLRU is used. Because SLRUs are initialized during startup pointer should be same in all backends. If I think more about it. Maybe it could be goot to add probe also into SimpleLruInit to catch name of SLRUs. Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [patch] pg_ctl init extension
Robert Haas píše v st 09. 12. 2009 v 08:56 -0500: > On Dec 9, 2009, at 8:32 AM, Zdenek Kotala wrote: > > > > Ahh, It is somethink what I want to do, but it is not ready yet in > > this > > patch, but I already documented it. Idea is to install initdb and > > postgres into libexecdir and packager can select if libexecdir will be > > equal bindir or not. > > > > The paragraph should be removed at this moment. Shell I send modified > > patch or does committer remove it before commit? > > I think Peter claimed this one but as far as I am concerned, I would > always rather have an updated patch. OK, here it is. Thanks Zdenek diff -r ab32ed8164e7 doc/src/sgml/config.sgml --- a/doc/src/sgml/config.sgml Mon Dec 07 15:10:09 2009 +0100 +++ b/doc/src/sgml/config.sgml Wed Dec 09 15:17:54 2009 +0100 @@ -54,9 +54,10 @@ One way to set these parameters is to edit the file postgresql.confpostgresql.conf, -which is normally kept in the data directory. (initdb -installs a default copy there.) An example of what this file might look -like is: +which is normally kept in the data directory. +(database cluster initialization +process installs a default copy there.) +An example of what this file might look like is: # This is a comment log_connections = yes @@ -365,8 +366,8 @@ Determines the maximum number of concurrent connections to the database server. The default is typically 100 connections, but might be less if your kernel settings will not support it (as -determined during initdb). This parameter can -only be set at server start. +determined during database cluster +initialization). This parameter can only be set at server start. @@ -747,7 +748,8 @@ Sets the amount of memory the database server uses for shared memory buffers. The default is typically 32 megabytes (32MB), but might be less if your kernel settings will -not support it (as determined during initdb). +not support it (as determined during database +cluster initialization). This setting must be at least 128 kilobytes. (Non-default values of BLCKSZ change the minimum.) However, settings significantly higher than the minimum are usually needed @@ -4267,10 +4269,10 @@ keywords US, NonEuro, and NonEuropean are synonyms for MDY. See for more information. The -built-in default is ISO, MDY, but -initdb will initialize the -configuration file with a setting that corresponds to the -behavior of the chosen lc_time locale. +built-in default is ISO, MDY, but +database cluster initialization +will initialize the configuration file with a setting that corresponds +to the behavior of the chosen lc_time locale. @@ -4476,9 +4478,9 @@ specifying the configuration. See for further information. The built-in default is pg_catalog.simple, but -initdb will initialize the -configuration file with a setting that corresponds to the -chosen lc_ctype locale, if a configuration +database cluster initialization +will initialize the configuration file with a setting that corresponds +to the chosen lc_ctype locale, if a configuration matching that locale can be identified. @@ -5240,8 +5242,9 @@ Allows modification of the structure of system tables. -This is used by initdb. -This parameter can only be set at server start. +This is used by database cluster +initialization process. This parameter can only be set at server +start. diff -r ab32ed8164e7 doc/src/sgml/manage-ag.sgml --- a/doc/src/sgml/manage-ag.sgml Mon Dec 07 15:10:09 2009 +0100 +++ b/doc/src/sgml/manage-ag.sgml Wed Dec 09 15:17:54 2009 +0100 @@ -107,10 +107,9 @@ Since you need to be connected to the database server in order to execute the CREATE DATABASE command, the question remains how the first database at any given - site can be created. The first database is always created by the - initdb command when the data storage area is - initialized. (See .) This - database is called + site can be created. The first database is always created when + the data storage area is initialized. (See .) + This database is called postgres.postgres So to create the first ordinary database you can connect to postgres. @@ -119,8 +118,8 @@ A second database, template1,template1 - is also created by - initdb. Whenever a new database is created within the + is also created during database cluster + initialization. Whenever a new database is created within the cluster, template1 is essentially
Re: [HACKERS] [PATCH] dtrace probes for memory manager
Bernd Helmle píše v út 08. 12. 2009 v 22:06 +0100: > > --On 8. Dezember 2009 15:51:52 -0500 Greg Smith > wrote: > > > Try this instead, which will give you a test where checkpoints have a > > minimal impact, but lots of memory will be thrown around: > > > > pgbench -i -s 10 > > pgbench -S -c 10 -T 600 > > Thanks for the input, will try > I modified probes to reduce overhead. Prototype patch is attached. Main point is to remove mcxt_alloc probe and keep only aset_alloc. I did also some testing with interesting results. At first I prepare special C store function (attached) which do only allocation and deallocation and I measured how long it takes: On 32bit the memory allocation is slow down 8.4% and on 64bit it is only 4.6%. Good to mention that I call palloc and pfree but in standard behavior pfree is not much used and memory is freed when context is destroyed. It means that we should think about 4.2% and 2.3% instead. But in normal situation database does also other thing and palloc is only one part of code path. It is why I run second test and use sun studio profiling tools (collect/analyzer) to determine how much CPU ticks cost the probes during pg_bench run. And results are much better. AllocSet alloc function takes about 4-5% and probes assembler code takes 0.1-0.2% on 64bit. I did not test 32bit but my expectation is that it should be about 0.3-0.4%. Zdenek diff -r 68b8827f4738 src/backend/utils/mmgr/aset.c --- a/src/backend/utils/mmgr/aset.c Fri Nov 13 11:17:04 2009 + +++ b/src/backend/utils/mmgr/aset.c Wed Dec 09 14:41:34 2009 +0100 @@ -64,6 +64,7 @@ #include "postgres.h" +#include "pg_trace.h" #include "utils/memutils.h" /* Define this to detail debug alloc information */ @@ -463,6 +464,8 @@ AssertArg(AllocSetIsValid(set)); + TRACE_POSTGRESQL_ASET_RESET(context); + /* Nothing to do if no pallocs since startup or last reset */ if (set->isReset) return; @@ -495,6 +498,8 @@ #endif block->freeptr = datastart; block->next = NULL; + TRACE_POSTGRESQL_ASET_BLOCK_RESET(context, block, +block->endptr - ((char *) block)); } else { @@ -503,6 +508,8 @@ /* Wipe freed memory for debugging purposes */ memset(block, 0x7F, block->freeptr - ((char *) block)); #endif + TRACE_POSTGRESQL_ASET_BLOCK_FREE(context, block, +block->endptr - ((char *) block)); free(block); } block = next; @@ -529,6 +536,7 @@ AllocBlock block = set->blocks; AssertArg(AllocSetIsValid(set)); + TRACE_POSTGRESQL_ASET_DELETE(context); #ifdef MEMORY_CONTEXT_CHECKING /* Check for corruption and leaks before freeing */ @@ -548,6 +556,8 @@ /* Wipe freed memory for debugging purposes */ memset(block, 0x7F, block->freeptr - ((char *) block)); #endif + TRACE_POSTGRESQL_ASET_BLOCK_FREE(context, block, + block->endptr - ((char *) block)); free(block); block = next; } @@ -570,6 +580,7 @@ AssertArg(AllocSetIsValid(set)); + TRACE_POSTGRESQL_ASET_ALLOC(context, size); /* * If requested size exceeds maximum for chunks, allocate an entire block * for this request. @@ -623,6 +634,7 @@ set->isReset = false; AllocAllocInfo(set, chunk); + TRACE_POSTGRESQL_ASET_BLOCK_NEW(context, block, blksize); return AllocChunkGetPointer(chunk); } @@ -771,6 +783,7 @@ errdetail("Failed on request of size %lu.", (unsigned long) size))); } + TRACE_POSTGRESQL_ASET_BLOCK_NEW(context, block, blksize); block->aset = set; block->freeptr = ((char *) block) + ALLOC_BLOCKHDRSZ; @@ -830,6 +843,7 @@ AllocChunk chunk = AllocPointerGetChunk(pointer); AllocFreeInfo(set, chunk); + TRACE_POSTGRESQL_ASET_FREE(context, pointer); #ifdef MEMORY_CONTEXT_CHECKING /* Test for someone scribbling on unused space in chunk */ @@ -866,6 +880,9 @@ set->blocks = block->next; else prevblock->next = block->next; + + TRACE_POSTGRESQL_ASET_BLOCK_FREE(context, block, + block->endptr - ((char *) block)); #ifdef CLOBBER_FREED_MEMORY /* Wipe freed memory for debugging purposes */ memset(block, 0x7F, block->freeptr - ((char *) block)); @@ -905,6 +922,8 @@ AllocChunk chunk = AllocPointerGetChunk(pointer); Size oldsize = chunk->size; + TRACE_POSTGRESQL_ASET_REALLOC(context, pointer, size, oldsize); + #ifdef MEMORY_CONTEXT_CHECKING /* Test for someone scribbling on unused space in chunk */ if (chunk->requested_size < oldsize) @@ -948,6 +967,7 @@ */ AllocBlock block = set->blocks; AllocBlock prevblock = NULL; + AllocBlock newblock; Size chksize; Size blksize; @@ -967,7 +987,11 @@ /* Do the realloc */ chksize = MAXALIGN(size); blksize = chksize + ALLOC_BLOCKHDRSZ + ALLOC_CHUNKHDRSZ; - block = (AllocBlock) realloc(block, blksize); + newblock = (AllocBlock) realloc(block, blksize); + + TRACE_POSTGRESQL_ASET_BLOCK_REALLOC(context, block, newblock, blksize); + block = newblock; + if (block == NULL) { MemoryContextStats(TopMemoryContext); diff
Re: [HACKERS] [patch] pg_ctl init extension
Greg Smith píše v út 08. 12. 2009 v 22:44 -0500: > Zdenek Kotala wrote: > > thanks for your useful comments. I attached new doc patch version. I > > removed example changes and add link to create database cluster (I hope) > > everywhere. Please, look on it and let me know if there is still > > something what should be changed. > > > That looks much better. There's only one bit that sticks out oddly now: > > + Note: The initdb might be invoked by > + pg_ctl initdb and initdb cannot be > in > + default path on a PostgreSQL installations. > > > > What is that supposed to mean exactly? Ahh, It is somethink what I want to do, but it is not ready yet in this patch, but I already documented it. Idea is to install initdb and postgres into libexecdir and packager can select if libexecdir will be equal bindir or not. The paragraph should be removed at this moment. Shell I send modified patch or does committer remove it before commit? thanks Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [patch] executor and slru dtrace probes
Dne 8.12.09 00:27, Bernd Helmle napsal(a): --On 13. November 2009 23:29:41 +0100 Zdenek Kotala wrote: t contains two DTrace probe groups. One is related to monitoring SLRU and second is about executor nodes. I merged it with the head. Original end of mail thread is here: http://archives.postgresql.org/pgsql-hackers/2009-04/msg00148.php I've started to review this. It seems to me the attached patch wasn't adjusted or discussed again to address Tom's complaints? At least the executor probes contained here hold still the same issues mentioned by Tom in the discussion linked here. I did not make any change. I only revival patch and merge it with head. I think that SLRU probes are OK and acceptable. Tom's issues with executor probes are still there and I expect discussion about them. IIRC Theo uses these probes in production. If you think that it is better I could split patch into two separate patches and both can be reviewed separately. thanks Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [patch] pg_ctl init extension
Greg, thanks for your useful comments. I attached new doc patch version. I removed example changes and add link to create database cluster (I hope) everywhere. Please, look on it and let me know if there is still something what should be changed. Thanks Zdenek Greg Smith píše v ne 06. 12. 2009 v 22:29 -0500: > I just noticed that there was an updated patch here that never made its > way onto the CommitFest app listing. I just fixed that and took a quick > look at it. I was in favor of this code change, but I have to say even > I don't really like how it ended up getting documented--and I'm sure > there are others would be downright hostile toward it. > > The biggest problem is that all of the places that used to say > "" when talking about creating a cluster now just say > "database cluster initialization"--with no link to a section covering > that topic. That's not a good forward step. The part I'm more > favorable toward that I expect other people to really cringe at is that > the examples showing how to manually run initdb have all been switched > to use "pg_ctl initdb" too. > > If we're going to have a smooth transition toward supporting both styles > of working in the next release, I think what needs to happen to the > documentation here is adding a very clear section saying that "initdb" > and "pg_ctl initdb" are the same thing, and noting why both forms > exist. Maybe a short note in both pg_ctl and initdb pointing at each > other; not sure yet what's best. Then update all the current places > that say "initdb" that have been rewritten in this doc patch to > "database cluster initialization" to reference things appropriate still. > > Going as far as making all the examples exclusively use the pg_ctl form > right now is probably more than the community at large wants to handle > just yet I suspect. At best, maybe we could make some or all of those > either use both forms, or link them to the new discussion of > alternatives section. > > I'm glad we made some progress (and are basically at code complete now) > on this patch this round. Given that this patch doesn't have a large > amount of support, I think that the remaining work here is fine-tuning > the documentation to cover the new option available without introducing > and abrupt change people won't like. I'm going to mark this "returned > with feedback" for now since I think that's going to take a bit more > work than we really have time for right now, particularly given the > limited number of people who care about this change. Zdenek, once this > CommitFest clears out, I can help out with getting the documentation > parts here smoothed over so this is in proper shape to commit during the > next one; I don't think there's anything left you need to do. > diff -r ab32ed8164e7 doc/src/sgml/config.sgml --- a/doc/src/sgml/config.sgml Mon Dec 07 15:10:09 2009 +0100 +++ b/doc/src/sgml/config.sgml Mon Dec 07 17:12:32 2009 +0100 @@ -54,9 +54,10 @@ One way to set these parameters is to edit the file postgresql.confpostgresql.conf, -which is normally kept in the data directory. (initdb -installs a default copy there.) An example of what this file might look -like is: +which is normally kept in the data directory. +(database cluster initialization +process installs a default copy there.) +An example of what this file might look like is: # This is a comment log_connections = yes @@ -365,8 +366,8 @@ Determines the maximum number of concurrent connections to the database server. The default is typically 100 connections, but might be less if your kernel settings will not support it (as -determined during initdb). This parameter can -only be set at server start. +determined during database cluster +initialization). This parameter can only be set at server start. @@ -747,7 +748,8 @@ Sets the amount of memory the database server uses for shared memory buffers. The default is typically 32 megabytes (32MB), but might be less if your kernel settings will -not support it (as determined during initdb). +not support it (as determined during database +cluster initialization). This setting must be at least 128 kilobytes. (Non-default values of BLCKSZ change the minimum.) However, settings significantly higher than the minimum are usually needed @@ -4267,10 +4269,10 @@ keywords US, NonEuro, and NonEuropean are synonyms for MDY. See for more information. The -built-in default is ISO, MDY, but -initdb will initialize the -configuration file with a setting that corresponds to the -behavior of the chosen lc_time locale. +built-in default is ISO, MDY, but +database cluster initialization +will initialize the configuration file w
Re: [HACKERS] [patch] pg_ctl init extension
I attached updated patch and doc patch. Zdenek Peter Eisentraut píše v so 21. 11. 2009 v 13:19 +0200: > On lör, 2009-11-14 at 14:50 +0100, Zdenek Kotala wrote: > > Peter Eisentraut píše v so 14. 11. 2009 v 10:41 +0200: > > > On tor, 2009-09-17 at 21:43 +0200, Zdenek Kotala wrote: > > > > Attached patch extends pg_ctl command with init option. > > > > > > > > pg_ctl -D /var/lib/postgres [-s] init > > > > > > > > This should replace usage of initdb command which has problematic name > > > > as we already discussed several times. Initdb binary will be still > > > > there, but it can be renamed and move into execlib dir in the future. > > > > > > > > Patch does not contains documentation changes. They will depends on > > > > decision which database initialization method will be preferred. > > > > > > OK, let's see. The patch is pretty straightforward, but does anyone > > > else actually want this? Comments? > > > > > > > Maybe we could ask on general where is more admins. I will send voting > > email. > > I think this is over now. There was some support, some "don't care, but > could make sense", and no one violently objecting, so please finish the > patch up with documentation, and it can go in as far as I'm concerned. > > Someone was proposing that pg_ctl initdb be an alias to pg_ctl init. > Perhaps you could add that. > diff -r 2d87758e836b src/bin/pg_ctl/pg_ctl.c --- a/src/bin/pg_ctl/pg_ctl.c Sun Nov 22 22:06:30 2009 + +++ b/src/bin/pg_ctl/pg_ctl.c Fri Dec 04 22:13:28 2009 +0100 @@ -57,6 +57,7 @@ typedef enum { NO_COMMAND = 0, + INIT_COMMAND, START_COMMAND, STOP_COMMAND, RESTART_COMMAND, @@ -100,6 +101,7 @@ static void do_help(void); static void set_mode(char *modeopt); static void set_sig(char *signame); +static void do_init(void); static void do_start(void); static void do_stop(void); static void do_restart(void); @@ -615,6 +617,38 @@ } static void +do_init(void) +{ + char pg_initdb[MAXPGPATH]; + char cmd[MAXPGPATH]; + int ret; + + if ((ret = find_other_exec(argv0, "initdb", "initdb (PostgreSQL) " PG_VERSION "\n", + pg_initdb)) < 0) + { + write_stderr(_("%s: could not find initdb\n"), + progname); + exit(1); + } + + if (post_opts == NULL) + post_opts = ""; + + if (!silent_mode) + snprintf(cmd, MAXPGPATH, SYSTEMQUOTE "\"%s\" %s%s" SYSTEMQUOTE, + pg_initdb, pgdata_opt, post_opts); + else + snprintf(cmd, MAXPGPATH, SYSTEMQUOTE "\"%s\" %s%s > \"%s\"" SYSTEMQUOTE, + pg_initdb, pgdata_opt, post_opts, DEVNULL); + + if ( system(cmd) != 0 ) + { + write_stderr(_("%s: database initialization failed.\n"), progname); + exit(1); + } +} + +static void do_start(void) { pgpid_t pid; @@ -1536,6 +1570,7 @@ printf(_("%s is a utility to start, stop, restart, reload configuration files,\n" "report the status of a PostgreSQL server, or signal a PostgreSQL process.\n\n"), progname); printf(_("Usage:\n")); + printf(_(" %s init[db] [-D DATADIR] [-s] [-o \"OPTIONS\"]\n"), progname); printf(_(" %s start [-w] [-t SECS] [-D DATADIR] [-s] [-l FILENAME] [-o \"OPTIONS\"]\n"), progname); printf(_(" %s stop[-W] [-t SECS] [-D DATADIR] [-s] [-m SHUTDOWN-MODE]\n"), progname); printf(_(" %s restart [-w] [-t SECS] [-D DATADIR] [-s] [-m SHUTDOWN-MODE]\n" @@ -1568,7 +1603,7 @@ #endif printf(_(" -l, --log FILENAME write (or append) server log to FILENAME\n")); printf(_(" -o OPTIONS command line options to pass to postgres\n" - " (PostgreSQL server executable)\n")); + " (PostgreSQL server executable) or initdb\n")); printf(_(" -p PATH-TO-POSTGRESnormally not necessary\n")); printf(_("\nOptions for stop or restart:\n")); printf(_(" -m SHUTDOWN-MODE can be \"smart\", \"fast\", or \"immediate\"\n")); @@ -1825,7 +1860,11 @@ exit(1); } - if (strcmp(argv[optind], "start") == 0) + if (strcmp(argv[optind], "init") == 0) +ctl_command = INIT_COMMAND; + else if (strcmp(argv[optind], "initdb") == 0) +ctl_command = INIT_COMMAND; + else if (strcmp(argv[optind], "start") == 0) ctl_command = START_COMMAND; else if (strcmp(argv[optind], "stop") == 0) ctl_command = STOP_COMMAND; @@ -1922,6 +1961,9 @@ switch (ctl_command) { + case INIT_COMMAND: + do_init(); + break; case STATUS_COMMAND: do_status(); br
Re: [HACKERS] [PATCH] Add solaris path for docbook COLLATEINDEX
Peter Eisentraut píše v po 30. 11. 2009 v 21:27 +0200: > On mån, 2009-11-30 at 19:53 +0100, Zdenek Kotala wrote: > > Bruce Momjian píše v po 30. 11. 2009 v 12:32 -0500: > > > I am not happy looking in a directory _above_ a specified directory by > > > default: > > > > > >[$DOCBOOKSTYLE/bin $DOCBOOKSTYLE/.. $PATH]) > > > > > > That seems possibly unsafe. I suggest you just add it to the PATH for > > > Solaris builds. > > > > I'm not sure if it is unsafer that searching in $PATH. Anyway I don't > > think that your proposed solution is good way. I'm able to do it but how > > many other people can? Who know where collateindex.pl stays on Solaris? > > configure is here to make live easier for people. If we know how to do > > it automatically we should do it. > > Note that $DOCBOOKSTYLE/bin corresponds to the location of the file in > an unpacked source archive and $PATH is of course where programs > normally go. The $DOCBOOKSTYLE/.. location makes no sense at all, > because that basically says that the installer intentionally moved the > file, but to a completely nonstandard location. I'm not sgml//docbook guru. Do you think that Solaris location of collateindex.pl is wrong? Does exist any recommendation for this? I could log a bug, but I need some link with recommendation. > > If you think that $DOCBOOKSTYLE/.. is not good than I propose to > > use /usr/share/sgml/docbook/ directly. > > That would be the less ugly solution. Patch attached. thanks Zdenek diff -r 2d87758e836b config/docbook.m4 --- a/config/docbook.m4 Sun Nov 22 22:06:30 2009 + +++ b/config/docbook.m4 Mon Nov 30 20:55:15 2009 +0100 @@ -93,7 +93,7 @@ [AC_REQUIRE([PGAC_PATH_DOCBOOK_STYLESHEETS])dnl if test -n "$DOCBOOKSTYLE"; then AC_PATH_PROGS(COLLATEINDEX, collateindex.pl, [], -[$DOCBOOKSTYLE/bin $PATH]) +[$DOCBOOKSTYLE/bin /usr/share/sgml/docbook $PATH]) else AC_PATH_PROGS(COLLATEINDEX, collateindex.pl) fi])# PGAC_PATH_COLLATEINDEX -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCH] Add solaris path for docbook COLLATEINDEX
Bruce Momjian píše v po 30. 11. 2009 v 12:32 -0500: > Zdenek Kotala wrote: > > collateindex.pl is stored in /usr/share/sgml/docbook/. Attached fix > > modify docbook.m4 to find correct path. > > > > It would be nice also backported the fix back at least to 8.2. > > I am not happy looking in a directory _above_ a specified directory by > default: > >[$DOCBOOKSTYLE/bin $DOCBOOKSTYLE/.. $PATH]) > > That seems possibly unsafe. I suggest you just add it to the PATH for > Solaris builds. I'm not sure if it is unsafer that searching in $PATH. Anyway I don't think that your proposed solution is good way. I'm able to do it but how many other people can? Who know where collateindex.pl stays on Solaris? configure is here to make live easier for people. If we know how to do it automatically we should do it. If you think that $DOCBOOKSTYLE/.. is not good than I propose to use /usr/share/sgml/docbook/ directly. Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] [PATCH] Add solaris path for docbook COLLATEINDEX
collateindex.pl is stored in /usr/share/sgml/docbook/. Attached fix modify docbook.m4 to find correct path. It would be nice also backported the fix back at least to 8.2. Thanks Zdenek diff -r 2d87758e836b config/docbook.m4 --- a/config/docbook.m4 Sun Nov 22 22:06:30 2009 + +++ b/config/docbook.m4 Wed Nov 25 21:27:09 2009 +0100 @@ -93,7 +93,7 @@ [AC_REQUIRE([PGAC_PATH_DOCBOOK_STYLESHEETS])dnl if test -n "$DOCBOOKSTYLE"; then AC_PATH_PROGS(COLLATEINDEX, collateindex.pl, [], -[$DOCBOOKSTYLE/bin $PATH]) +[$DOCBOOKSTYLE/bin $DOCBOOKSTYLE/.. $PATH]) else AC_PATH_PROGS(COLLATEINDEX, collateindex.pl) fi])# PGAC_PATH_COLLATEINDEX -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] DTrace compiler warnings
Hmm, const is also problem on solaris. dtrace generates probe.h file and eats const. It generates following noise on solaris build: "postgres.c", line 554: warning: argument #1 is incompatible with prototype: prototype: pointer to char : "../../../src/include/utils/probes.h", line 582 argument : pointer to const char IIRC, it was discussed. I cc Robert. He should know answer why const is ignored. Zdenek Bernd Helmle píše v so 14. 11. 2009 v 14:54 +0100: > I'm just seeing these kind of compiler warnings in current HEAD: > > pgstat.c: In function ‘pgstat_report_activity’: > pgstat.c:2276: warning: passing argument 1 of > ‘__dtrace_probe$postgresql$statement__status$v1$63686172202a’ discards > qualifiers from pointer target type > > There are more of them (e.g. postgres.c), all passing a const char pointer. > > Platform is Snow Leopard, 10.6.2, gcc 4.2.1 > > -- > Thanks > > Bernd > -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [patch] pg_ctl init extension
Peter Eisentraut píše v so 14. 11. 2009 v 10:41 +0200: > On tor, 2009-09-17 at 21:43 +0200, Zdenek Kotala wrote: > > Attached patch extends pg_ctl command with init option. > > > > pg_ctl -D /var/lib/postgres [-s] init > > > > This should replace usage of initdb command which has problematic name > > as we already discussed several times. Initdb binary will be still > > there, but it can be renamed and move into execlib dir in the future. > > > > Patch does not contains documentation changes. They will depends on > > decision which database initialization method will be preferred. > > OK, let's see. The patch is pretty straightforward, but does anyone > else actually want this? Comments? > Maybe we could ask on general where is more admins. I will send voting email. thanks for looking on this Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] [patch] executor and slru dtrace probes
I attached patch which was already sent on february/april, but it was lost in time. It is originally from Robert Lor and Theo Schlossnagle. It contains two DTrace probe groups. One is related to monitoring SLRU and second is about executor nodes. I merged it with the head. Original end of mail thread is here: http://archives.postgresql.org/pgsql-hackers/2009-04/msg00148.php Zdenek diff -r 68b8827f4738 src/backend/access/transam/slru.c --- a/src/backend/access/transam/slru.c Fri Nov 13 11:17:04 2009 + +++ b/src/backend/access/transam/slru.c Fri Nov 13 23:29:14 2009 +0100 @@ -57,6 +57,7 @@ #include "storage/fd.h" #include "storage/shmem.h" #include "miscadmin.h" +#include "pg_trace.h" /* @@ -372,6 +373,7 @@ { SlruShared shared = ctl->shared; + TRACE_POSTGRESQL_SLRU_READPAGE_START((uintptr_t)ctl, pageno, write_ok, xid); /* Outer loop handles restart if we must wait for someone else's I/O */ for (;;) { @@ -399,6 +401,7 @@ } /* Otherwise, it's ready to use */ SlruRecentlyUsed(shared, slotno); + TRACE_POSTGRESQL_SLRU_READPAGE_DONE(slotno); return slotno; } @@ -446,6 +449,7 @@ SlruReportIOError(ctl, pageno, xid); SlruRecentlyUsed(shared, slotno); + TRACE_POSTGRESQL_SLRU_READPAGE_DONE(slotno); return slotno; } } @@ -470,6 +474,8 @@ SlruShared shared = ctl->shared; int slotno; + TRACE_POSTGRESQL_SLRU_READPAGE_READONLY((uintptr_t)ctl, pageno, xid); + /* Try to find the page while holding only shared lock */ LWLockAcquire(shared->ControlLock, LW_SHARED); @@ -511,6 +517,8 @@ int pageno = shared->page_number[slotno]; bool ok; + TRACE_POSTGRESQL_SLRU_WRITEPAGE_START((uintptr_t)ctl, pageno, slotno); + /* If a write is in progress, wait for it to finish */ while (shared->page_status[slotno] == SLRU_PAGE_WRITE_IN_PROGRESS && shared->page_number[slotno] == pageno) @@ -525,7 +533,10 @@ if (!shared->page_dirty[slotno] || shared->page_status[slotno] != SLRU_PAGE_VALID || shared->page_number[slotno] != pageno) + { + TRACE_POSTGRESQL_SLRU_WRITEPAGE_DONE(); return; + } /* * Mark the slot write-busy, and clear the dirtybit. After this point, a @@ -569,6 +580,8 @@ /* Now it's okay to ereport if we failed */ if (!ok) SlruReportIOError(ctl, pageno, InvalidTransactionId); + + TRACE_POSTGRESQL_SLRU_WRITEPAGE_DONE(); } /* @@ -593,6 +606,8 @@ SlruFileName(ctl, path, segno); + TRACE_POSTGRESQL_SLRU_READPAGE_PHYSICAL_START((uintptr_t)ctl, path, pageno, slotno); + /* * In a crash-and-restart situation, it's possible for us to receive * commands to set the commit status of transactions whose bits are in @@ -607,6 +622,7 @@ { slru_errcause = SLRU_OPEN_FAILED; slru_errno = errno; + TRACE_POSTGRESQL_SLRU_READPAGE_PHYSICAL_DONE(false, slru_errcause, slru_errno); return false; } @@ -614,6 +630,7 @@ (errmsg("file \"%s\" doesn't exist, reading as zeroes", path))); MemSet(shared->page_buffer[slotno], 0, BLCKSZ); + TRACE_POSTGRESQL_SLRU_READPAGE_PHYSICAL_DONE(true, -1, -1); return true; } @@ -622,6 +639,7 @@ slru_errcause = SLRU_SEEK_FAILED; slru_errno = errno; close(fd); + TRACE_POSTGRESQL_SLRU_READPAGE_PHYSICAL_DONE(false, slru_errcause, slru_errno); return false; } @@ -631,6 +649,7 @@ slru_errcause = SLRU_READ_FAILED; slru_errno = errno; close(fd); + TRACE_POSTGRESQL_SLRU_READPAGE_PHYSICAL_DONE(false, slru_errcause, slru_errno); return false; } @@ -638,9 +657,12 @@ { slru_errcause = SLRU_CLOSE_FAILED; slru_errno = errno; + TRACE_POSTGRESQL_SLRU_READPAGE_PHYSICAL_DONE(false, slru_errcause, slru_errno); return false; } + TRACE_POSTGRESQL_SLRU_READPAGE_PHYSICAL_DONE(true, -1, -1); + return true; } @@ -668,6 +690,8 @@ char path[MAXPGPATH]; int fd = -1; + TRACE_POSTGRESQL_SLRU_WRITEPAGE_PHYSICAL_START((uintptr_t)ctl, pageno, slotno); + /* * Honor the write-WAL-before-data rule, if appropriate, so that we do not * write out data before associated WAL records. This is the same action @@ -753,6 +777,7 @@ { slru_errcause = SLRU_OPEN_FAILED; slru_errno = errno; + TRACE_POSTGRESQL_SLRU_WRITEPAGE_PHYSICAL_DONE(false, slru_errcause, slru_errno); return false; } @@ -781,6 +806,7 @@ slru_errno = errno; if (!fdata) close(fd); + TRACE_POSTGRESQL_SLRU_WRITEPAGE_PHYSICAL_DONE(false, slru_errcause, slru_errno); return false; } @@ -794,6 +820,7 @@ slru_errno = errno; if (!fdata) close(fd); + TRACE_POSTGRESQL_SLRU_WRITEPAGE_PHYSICAL_DONE(false, slru_errcause, slru_errno); return false; } @@ -808,6 +835,7 @@ slru_errcause = SLRU_FSYNC_FAILED; slru_errno = errno; close(fd); + TRACE_POSTGRESQL_SLRU_WRITEPAGE_PHYSICAL_DONE(false, slru_errcause, slru_errno); return false; } @@ -815,10 +843,12 @@ { slru_errcause = SLRU_CLOSE_FAILED; slru_errno = errno; + TRACE_POSTGRESQL_SLRU_WRITEPAGE_PHYSICAL_DONE(false, slru_errca
Re: [HACKERS] [PATCH] dtrace probes for memory manager
Alvaro Herrera píše v pá 13. 11. 2009 v 18:34 -0300: > Zdenek Kotala wrote: > > Attached patch contains new dtrace probes for memory management. Main > > purpose is to analyze memory footprint - for example how many memory > > needs transaction, peak memory per context, when memory block is reused > > or when it is allocate by malloc and so on. > > Having had to instrument these to figure out some problems, I'd give > this patch a +1. However, the performance argument is compelling. As a > compromise, maybe we could have a #define that needs to be turned on at > compile time to enable these probes; so a regular dtrace-enabled build > would not have them, but if you really needed to analyze memory > allocations, you could recompile to turn them on. The another idea is to have two AllocSet functions set. One without and one with dtrace probes. And switch it by GUC flag for example. From mcxt I create and delete context is important, rest can be taken from alloc set probes. Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCH] dtrace probes for memory manager
Alvaro Herrera píše v pá 13. 11. 2009 v 18:34 -0300: > Zdenek Kotala wrote: > > Attached patch contains new dtrace probes for memory management. Main > > purpose is to analyze memory footprint - for example how many memory > > needs transaction, peak memory per context, when memory block is reused > > or when it is allocate by malloc and so on. > > Having had to instrument these to figure out some problems, I'd give > this patch a +1. However, the performance argument is compelling. As a > compromise, maybe we could have a #define that needs to be turned on at > compile time to enable these probes; so a regular dtrace-enabled build > would not have them, but if you really needed to analyze memory > allocations, you could recompile to turn them on. But point of dtrace probes is that they are here without recompilation :(. Do we have any test which we could use for performance penalty testing? I don't think that overhead is significant. Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCH] dtrace probes for memory manager
Tom Lane píše v pá 13. 11. 2009 v 16:06 -0500: > Zdenek Kotala writes: > > Attached patch contains new dtrace probes for memory management. > > This is a bad idea and I want to reject it outright. No ordinary user > is really going to care about those details, and palloc is a > sufficiently hot hot-spot that even the allegedly negligible overhead > of an inactive dtrace probe is going to cost us. I don't think that impact is too big here. User space DTrace probes are implemented like function call. When probe is inactive call is replaced by nop. Only what stay in code is arguments preparations. there is asm code 32bit intel from palloc: MemoryContextAlloc+0xae:pushl $0x0 MemoryContextAlloc+0xb0:pushl -0x8(%ebx) MemoryContextAlloc+0xb3:pushl 0xc(%ebp) MemoryContextAlloc+0xb6:movl 0x8(%ebp),%eax MemoryContextAlloc+0xb9:pushl %eax MemoryContextAlloc+0xba:pushl 0x14(%eax) MemoryContextAlloc+0xbd:nop MemoryContextAlloc+0xbe:nop MemoryContextAlloc+0xbf:nop MemoryContextAlloc+0xc0:nop MemoryContextAlloc+0xc1:nop MemoryContextAlloc+0xc2:addl $0x20,%esp You can see is that overhead depends on number of argument. I used five arguments now but two can be enough. Only dtrace script will be complicated in some cases after that. By my opinion if you compare whole palloc code path, Dtrace probes overhead is minimal. Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] [PATCH] dtrace probes for memory manager
Attached patch contains new dtrace probes for memory management. Main purpose is to analyze memory footprint - for example how many memory needs transaction, peak memory per context, when memory block is reused or when it is allocate by malloc and so on. There are three groups of probes: 1) general memory context operation: mcxt-alloc mcxt-create mcxt-delete mcxt-free mcxt-realloc mcxt-reset 2) AllocSet operations (called from mcxt) aset-alloc aset-delete aset-free aset-realloc aset-reset 3) AllocSet Block operations. aset-block-free aset-block-new aset-block-realloc aset-block-reset Zdenek diff -r 68b8827f4738 src/backend/utils/mmgr/aset.c --- a/src/backend/utils/mmgr/aset.c Fri Nov 13 11:17:04 2009 + +++ b/src/backend/utils/mmgr/aset.c Fri Nov 13 21:24:56 2009 +0100 @@ -64,6 +64,7 @@ #include "postgres.h" +#include "pg_trace.h" #include "utils/memutils.h" /* Define this to detail debug alloc information */ @@ -463,6 +464,8 @@ AssertArg(AllocSetIsValid(set)); + TRACE_POSTGRESQL_ASET_RESET(context); + /* Nothing to do if no pallocs since startup or last reset */ if (set->isReset) return; @@ -495,6 +498,8 @@ #endif block->freeptr = datastart; block->next = NULL; + TRACE_POSTGRESQL_ASET_BLOCK_RESET(context, block, +block->endptr - ((char *) block)); } else { @@ -503,6 +508,8 @@ /* Wipe freed memory for debugging purposes */ memset(block, 0x7F, block->freeptr - ((char *) block)); #endif + TRACE_POSTGRESQL_ASET_BLOCK_FREE(context, block, +block->endptr - ((char *) block)); free(block); } block = next; @@ -529,6 +536,7 @@ AllocBlock block = set->blocks; AssertArg(AllocSetIsValid(set)); + TRACE_POSTGRESQL_ASET_DELETE(context); #ifdef MEMORY_CONTEXT_CHECKING /* Check for corruption and leaks before freeing */ @@ -548,6 +556,8 @@ /* Wipe freed memory for debugging purposes */ memset(block, 0x7F, block->freeptr - ((char *) block)); #endif + TRACE_POSTGRESQL_ASET_BLOCK_FREE(context, block, + block->endptr - ((char *) block)); free(block); block = next; } @@ -570,6 +580,7 @@ AssertArg(AllocSetIsValid(set)); + TRACE_POSTGRESQL_ASET_ALLOC(context, size); /* * If requested size exceeds maximum for chunks, allocate an entire block * for this request. @@ -623,6 +634,7 @@ set->isReset = false; AllocAllocInfo(set, chunk); + TRACE_POSTGRESQL_ASET_BLOCK_NEW(context, block, blksize); return AllocChunkGetPointer(chunk); } @@ -771,6 +783,7 @@ errdetail("Failed on request of size %lu.", (unsigned long) size))); } + TRACE_POSTGRESQL_ASET_BLOCK_NEW(context, block, blksize); block->aset = set; block->freeptr = ((char *) block) + ALLOC_BLOCKHDRSZ; @@ -830,6 +843,7 @@ AllocChunk chunk = AllocPointerGetChunk(pointer); AllocFreeInfo(set, chunk); + TRACE_POSTGRESQL_ASET_FREE(context, pointer); #ifdef MEMORY_CONTEXT_CHECKING /* Test for someone scribbling on unused space in chunk */ @@ -866,6 +880,9 @@ set->blocks = block->next; else prevblock->next = block->next; + + TRACE_POSTGRESQL_ASET_BLOCK_FREE(context, block, + block->endptr - ((char *) block)); #ifdef CLOBBER_FREED_MEMORY /* Wipe freed memory for debugging purposes */ memset(block, 0x7F, block->freeptr - ((char *) block)); @@ -905,6 +922,8 @@ AllocChunk chunk = AllocPointerGetChunk(pointer); Size oldsize = chunk->size; + TRACE_POSTGRESQL_ASET_REALLOC(context, pointer, size, oldsize); + #ifdef MEMORY_CONTEXT_CHECKING /* Test for someone scribbling on unused space in chunk */ if (chunk->requested_size < oldsize) @@ -948,6 +967,7 @@ */ AllocBlock block = set->blocks; AllocBlock prevblock = NULL; + AllocBlock newblock; Size chksize; Size blksize; @@ -967,7 +987,11 @@ /* Do the realloc */ chksize = MAXALIGN(size); blksize = chksize + ALLOC_BLOCKHDRSZ + ALLOC_CHUNKHDRSZ; - block = (AllocBlock) realloc(block, blksize); + newblock = (AllocBlock) realloc(block, blksize); + + TRACE_POSTGRESQL_ASET_BLOCK_REALLOC(context, block, newblock, blksize); + block = newblock; + if (block == NULL) { MemoryContextStats(TopMemoryContext); diff -r 68b8827f4738 src/backend/utils/mmgr/mcxt.c --- a/src/backend/utils/mmgr/mcxt.c Fri Nov 13 11:17:04 2009 + +++ b/src/backend/utils/mmgr/mcxt.c Fri Nov 13 21:24:56 2009 +0100 @@ -22,7 +22,7 @@ #include "postgres.h" #include "utils/memutils.h" - +#include "pg_trace.h" /* * GLOBAL MEMORY * @@ -123,6 +123,8 @@ { AssertArg(MemoryContextIsValid(context)); + TRACE_POSTGRESQL_MCXT_RESET(context->name, context); + /* save a function call in common case where there are no children */ if (context->firstchild != NULL) MemoryContextResetChildren(context); @@ -168,6 +170,8 @@ MemoryContextDeleteChildren(context); + TRACE_POSTGRESQL_MCXT_DELETE(context->name, context, context->par
[HACKERS] [Patch] Fix enum type mismatch
Attached patch fixed following warning: "../../../src/include/nodes/parsenodes.h", line 487: warning: enumerator value overflows INT_MAX (2147483647) The reason is clear, enum is int not unsigned. It is short fix, but I'm thinking about enum conversion to #define. We use e.g. in the same file. 60 typedef uint32 AclMode; /* a bitmask of privilege bits */ 61 62 #define ACL_INSERT (1<<0) /* for relations */ 63 #define ACL_SELECT (1<<1) 64 #define ACL_UPDATE (1<<2) 65 #define ACL_DELETE (1<<3) 66 #define ACL_TRUNCATE(1<<4) 67 #define ACL_REFERENCES (1<<5) 68 #define ACL_TRIGGER (1<<6) Zdenek diff -r 68b8827f4738 src/include/nodes/parsenodes.h --- a/src/include/nodes/parsenodes.h Fri Nov 13 11:17:04 2009 + +++ b/src/include/nodes/parsenodes.h Fri Nov 13 16:21:38 2009 +0100 @@ -484,7 +484,7 @@ CREATE_TABLE_LIKE_INDEXES = 1 << 2, CREATE_TABLE_LIKE_STORAGE = 1 << 3, CREATE_TABLE_LIKE_COMMENTS = 1 << 4, - CREATE_TABLE_LIKE_ALL = 0x + CREATE_TABLE_LIKE_ALL = 0x7FFF /* enum is int */ } CreateStmtLikeOption; /* -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] postgres 8.3.8 and Solaris 10_x86 64 bit problems?
u235sentinel píše v út 20. 10. 2009 v 12:22 -0600: > Now I'm running and will add a few more things in like readline. The > goal is to build plr and plperl and load them into postgres. Hmm, I'm afraid that 64bit plperl is a problem. Solaris 10 is not shipped with 64bit perl :(. Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] postgres 8.3.8 and Solaris 10_x86 64 bit problems?
Andrew Chernow píše v po 19. 10. 2009 v 14:14 -0400: > > # ./pg_ctl > > ld.so.1: pg_ctl: fatal: relocation error: R_AMD64_32: file > > /usr/local/postgres64/lib/libpq.so.5: symbol (unknown): value > > 0xfd7fff1cf210 does not fit > > Killed > > > > "symbol (unknown)". Can you turn on debugging symbols? Knowing the > symbol may point to a library that was not compiled properly. Also you can try to run LD_DEBUG=basic ./pg_ctl and also elfdump | grep R_AMD64_32 it should show when symbols came from. By theway what S10 version do you use? ld -V and uname -a also helps. > > So I run 'ldd pg_ctl' to see if everything is linking ok. > > > > And I'm wondering if there is a problem with libpq.so.5 as mentioned in > > the original error > > > > # file /usr/local/postgres64/lib/libpq.so.5 > > > > > > /usr/local/postgres64/lib/libpq.so.5: ELF 64-bit LSB dynamic lib AMD64 > > Version 1 [SSE CMOV], dynamically linked, not stripped > > > > Ok. So looking good. Maybe there is a library or header libpq needs > > that I'm missing in 64 bit? > > > > # ldd /usr/local/postgres64/lib/libpq.so.5 > > Are you sure that all pg_ctl referenced libraries and all libpq.so > referenced libraries were built as 64-bit using PIC? Are you linking > with any static library that may contain 32-bit objects? That error is > most commonly PIC or arch-mismatch. > Agree, I went through linker bugs and missing PIC is often root cause of this problem. See http://bugs.opensolaris.org/bugdatabase/view_bug.do?bug_id=6261066 Problem was that ./configure badly setup PIC switch on amd64 platform. Please, could you compile pure postgreSQL without other own libraries like readline and openssl? It should help to find which library is culprit. Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] postgres 8.3.8 and Solaris 10_x86 64 bit problems?
Andrew Chernow píše v ne 18. 10. 2009 v 21:09 -0400: > > I'm curious if this is a lost hope. My boss is recommending we flatten > > the Sun box and install redhat linux (which I'm fine with). I'd rather > > not as threading in Solaris is better. > > Maybe solaris threads were better 10-15 years ago, but I'm not convinced that > is > still the case. Any data supporting that argument, solaris 10 threads vs. > linux > 2.6.11+ kernel (p)threads? I can point on this article: http://tweakers.net/reviews/649/all/database-test-sun-ultrasparc-t1-vs-punt-amd-opteron.html Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] postgres 8.3.8 and Solaris 10_x86 64 bit problems?
u235sentinel píše v ne 18. 10. 2009 v 17:50 -0600: > Are you sure about this? When I try to build and don't have openssl in > the lib/include path it claims it needs it. As I'm building 64 bit I > can now build postgres in 64 bit with openssl 98k just fine. However > when I run it I'm getting the same error message. If you want to link against to builtin OpenSSL you need following setup: ./configure ... --with-openssl --with-includes=/usr/sfw/include --with-libs=/usr/lib/amd64:/usr/sfw/lib/amd64 and important is: LD_OPTIONS="-R/usr/sfw/lib/amd64 -L/usr/sfw/lib/amd64" Or if you don't need own compilation, you can use built-in PostgreSQL 8.3. It is located in /usr/postgres/8.3/bin or /usr/postgres/8.3/bin/64. See man postgres_83 for details. Also you need to apply last patch 138827-05: http://sunsolve.sun.com/search/document.do?assetkey=1-21-138827-05-1 Or if you still needs own compilation try to compile openssl 98k with Sun Studio. Or if you cannot compile it with Sun Studio, you can try -mimpure-text gcc switch to compile OpenSSL. It is workaround for some kind of linking issues. Let me know it it helps Zdenek > I'm curious if this is a lost hope. My boss is recommending we flatten > the Sun box and install redhat linux (which I'm fine with). I'd rather > not as threading in Solaris is better. > > Thoughts? > > thanks > > > Zdenek Kotala wrote: > > You can look on > > http://www.pgbuildfarm.org/cgi-bin/show_log.pl?nm=ghost_moth&dt=2009-10-07%2021:06:00 > > > > > > > > How it is built. You also does not needed own version of Openssl. All > > security fixes are backported. It is located in /usr/sfw/lib or > > /usr/sfw/lib/64 > > > > Sometimes are problem with gcc and solaris linker. IIRC, I had problem > > with PLPerl compilation. > > > > Zdenek > > > > Dne 8.10.09 03:48, u235sentinel napsal(a): > >> So I compiled postgres with Solaris 10 and have problems running it. > >> > >> # ./pg_ctl > >> ld.so.1: pg_ctl: fatal: relocation error: R_AMD64_32: file > >> /usr/local/postgres64/lib/libpq.so.5: symbol (unknown): value > >> 0xfd7fff1cf210 does not fit > >> Killed > >> > >> # ldd pg_ctl > >> libpq.so.5 =>/usr/local/postgres64/lib/libpq.so.5 > >> libm.so.2 => /usr/lib/64/libm.so.2 > >> libxml2.so.2 => /usr/lib/64/libxml2.so.2 > >> libz.so.1 => /usr/lib/64/libz.so.1 > >> libreadline.so.6 => /usr/local/lib/libreadline.so.6 > >> libcurses.so.1 =>/usr/lib/64/libcurses.so.1 > >> librt.so.1 =>/usr/lib/64/librt.so.1 > >> libsocket.so.1 =>/usr/lib/64/libsocket.so.1 > >> libc.so.1 => /usr/lib/64/libc.so.1 > >> libpthread.so.1 => /usr/lib/64/libpthread.so.1 > >> libnsl.so.1 => /lib/64/libnsl.so.1 > >> libgcc_s.so.1 => /usr/sfw/lib/amd64/libgcc_s.so.1 > >> libaio.so.1 => /lib/64/libaio.so.1 > >> libmd.so.1 =>/lib/64/libmd.so.1 > >> libmp.so.2 =>/lib/64/libmp.so.2 > >> libscf.so.1 => /lib/64/libscf.so.1 > >> libdoor.so.1 => /lib/64/libdoor.so.1 > >> libuutil.so.1 => /lib/64/libuutil.so.1 > >> libgen.so.1 => /lib/64/libgen.so.1 > >> > >> # file /usr/local/postgres64/lib/libpq.so.5 > >> /usr/local/postgres64/lib/libpq.so.5: ELF 64-bit LSB dynamic lib > >> AMD64 Version 1 [SSE CMOV], dynamically linked, not stripped > >> > >> > >> What am I missing??? > >> > >> Here's my environment. > >> > >> Solaris 10 x86_64 with postgres 8.3.8 and openssl 98k using gcc > >> version 3.4.3 (csl-sol210-3_4-branch+sol_rpath) > >> , sunstudio12.1 and GNU Make 3.80 > >> > >> I've even monkied with LD_LIBRARY_PATH but getting the same issues. > >> Seems when I don't compile in openssl everything is fine. > >> > >> Thanks! > >> > > > > > > -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] postgres 8.3.8 and Solaris 10_x86 64 bit problems?
You can look on http://www.pgbuildfarm.org/cgi-bin/show_log.pl?nm=ghost_moth&dt=2009-10-07%2021:06:00 How it is built. You also does not needed own version of Openssl. All security fixes are backported. It is located in /usr/sfw/lib or /usr/sfw/lib/64 Sometimes are problem with gcc and solaris linker. IIRC, I had problem with PLPerl compilation. Zdenek Dne 8.10.09 03:48, u235sentinel napsal(a): So I compiled postgres with Solaris 10 and have problems running it. # ./pg_ctl ld.so.1: pg_ctl: fatal: relocation error: R_AMD64_32: file /usr/local/postgres64/lib/libpq.so.5: symbol (unknown): value 0xfd7fff1cf210 does not fit Killed # ldd pg_ctl libpq.so.5 =>/usr/local/postgres64/lib/libpq.so.5 libm.so.2 => /usr/lib/64/libm.so.2 libxml2.so.2 => /usr/lib/64/libxml2.so.2 libz.so.1 => /usr/lib/64/libz.so.1 libreadline.so.6 => /usr/local/lib/libreadline.so.6 libcurses.so.1 =>/usr/lib/64/libcurses.so.1 librt.so.1 =>/usr/lib/64/librt.so.1 libsocket.so.1 =>/usr/lib/64/libsocket.so.1 libc.so.1 => /usr/lib/64/libc.so.1 libpthread.so.1 => /usr/lib/64/libpthread.so.1 libnsl.so.1 => /lib/64/libnsl.so.1 libgcc_s.so.1 => /usr/sfw/lib/amd64/libgcc_s.so.1 libaio.so.1 => /lib/64/libaio.so.1 libmd.so.1 =>/lib/64/libmd.so.1 libmp.so.2 =>/lib/64/libmp.so.2 libscf.so.1 => /lib/64/libscf.so.1 libdoor.so.1 => /lib/64/libdoor.so.1 libuutil.so.1 => /lib/64/libuutil.so.1 libgen.so.1 => /lib/64/libgen.so.1 # file /usr/local/postgres64/lib/libpq.so.5 /usr/local/postgres64/lib/libpq.so.5: ELF 64-bit LSB dynamic lib AMD64 Version 1 [SSE CMOV], dynamically linked, not stripped What am I missing??? Here's my environment. Solaris 10 x86_64 with postgres 8.3.8 and openssl 98k using gcc version 3.4.3 (csl-sol210-3_4-branch+sol_rpath) , sunstudio12.1 and GNU Make 3.80 I've even monkied with LD_LIBRARY_PATH but getting the same issues. Seems when I don't compile in openssl everything is fine. Thanks! -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] hstore crasesh on 64bit Sparc
Tom Lane píše v čt 01. 10. 2009 v 12:28 -0400: > Zdenek Kotala writes: > > I'm looking why cometh_month fails and it is problem with last hstore > > putback: > > I think this is the same 64-bit problem we fixed last night --- wait for > the next rebuild before worrying. Correct, last build seems OK. There only remains issue with plpython UTF8 test which need some investigation. thanks Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] hstore crasesh on 64bit Sparc
I'm looking why cometh_month fails and it is problem with last hstore putback: http://www.pgbuildfarm.org/cgi-bin/show_log.pl?nm=comet_moth&dt=2009-09-30%2021:06:00 Stack trace is following: 6f013828 hstore_hash (7fffa050, 1395298, 1, 41, 7370616365414143, 0) + c0 0001005f396c FunctionCall1 (100b9ca10, 1395238, 7fffa4cf, 100ba0b48, 43, 100ba0b44) + 4c 0001002e8264 TupleHashTableHash (7fffa6f0, 8, 100b9f9e8, 1, 7fffa6ef, 0) + 15c 0001005fa734 hash_search (100ba20f8, 7fffa6f0, 1, 7fffa6ef, 0, 100ba4191) + 2c 0001002e7e74 LookupTupleHashEntry (100ba00e8, 100b9ae90, 7fffa7ef, 0, 1, 100b96b40) + 114 000100308420 lookup_hash_entry (100b9a978, 100b9b588, 0, 0, 0, 0) + 118 status: process terminated by SIGSEGV (Segmentation Fault), addr=500ba410c It is not assert problem but Segmentation Fault. 32bit compilation works fine (ghost moth). Any idea? Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [patch] pg_ctl init extension
Bernd Helmle píše v čt 17. 09. 2009 v 23:26 +0200: > > --On 17. September 2009 23:00:12 +0300 Peter Eisentraut > wrote: > > > f the name is a problem, why not change the name? What you are > > proposing above is effectively a very elaborate name change, so why not > > go for a simpler one? > > I don't like the solution by using -o "" to push down > command line options to initdb. I always had the opinion that this was (and > is) a valid workaround for postgres itself, but it doesn't feel correct to > expose that further to initdb and its purposes. hmm, I could modify it that all args after init keyword will be pass to the initdb like this: pg_ctl -D /tmp/db init [] initdb [] and "pg_ctl -h init" will show help which commands are allowed. Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [patch] pg_ctl init extension
Peter Eisentraut píše v čt 17. 09. 2009 v 23:00 +0300: > On tor, 2009-09-17 at 21:43 +0200, Zdenek Kotala wrote: > > Attached patch extends pg_ctl command with init option. > > > > pg_ctl -D /var/lib/postgres [-s] init > > > > This should replace usage of initdb command which has problematic name > > as we already discussed several times. Initdb binary will be still > > there, but it can be renamed and move into execlib dir in the future. > > If the name is a problem, why not change the name? What you are > proposing above is effectively a very elaborate name change, so why not > go for a simpler one? The idea is to have one command for server control. By my opinion init logically belongs to group of command like start/stop. It is also possible to add parameter for init+start in one command and so on. If you look on ZFS you have only two commands to manage everything, it is easy and you can start to use ZFS very quickly. I think this patch increase usability/adoption od postgreSQL for newbies. And second big advantage is that it would be possible easily extend pg_ctl to cope with different postgresql versions (e.g. "pg_ctl -v 8.2 -D . init"). There is no reason why pg_ctl couldn't start different postgreSQL version depends on PG_VERSION in data directory. Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] [patch] pg_ctl init extension
Attached patch extends pg_ctl command with init option. pg_ctl -D /var/lib/postgres [-s] init This should replace usage of initdb command which has problematic name as we already discussed several times. Initdb binary will be still there, but it can be renamed and move into execlib dir in the future. Patch does not contains documentation changes. They will depends on decision which database initialization method will be preferred. Zdenek *** pgsql_init.8d83e5030d44/src/bin/pg_ctl/pg_ctl.c 2009-09-17 21:42:20.865268360 +0200 --- /export/home/zk200664/work/mercurial/pgsql_init/src/bin/pg_ctl/pg_ctl.c 2009-09-17 21:15:04.630265322 +0200 *** *** 57,62 --- 57,63 typedef enum { NO_COMMAND = 0, + INIT_COMMAND, START_COMMAND, STOP_COMMAND, RESTART_COMMAND, *** *** 100,105 --- 101,107 static void do_help(void); static void set_mode(char *modeopt); static void set_sig(char *signame); + static void do_init(void); static void do_start(void); static void do_stop(void); static void do_restart(void); *** *** 615,620 --- 617,655 } static void + do_init(void) + { + char pg_initdb[MAXPGPATH]; + char cmd[MAXPGPATH]; + int ret; + + if ((ret = find_other_exec(argv0, "initdb", "initdb (PostgreSQL) " PG_VERSION "\n", + pg_initdb)) < 0) + + { + write_stderr(_("%s: could not find initdb\n"), + progname); + exit(1); + } + + if (post_opts == NULL) + post_opts = ""; + + if (!silent_mode) + snprintf(cmd, MAXPGPATH, SYSTEMQUOTE "\"%s\" %s%s" SYSTEMQUOTE, + pg_initdb, pgdata_opt, post_opts); + else + snprintf(cmd, MAXPGPATH, SYSTEMQUOTE "\"%s\" %s%s > \"%s\"" SYSTEMQUOTE, + pg_initdb, pgdata_opt, post_opts, DEVNULL); + + if ( system(cmd) != 0 ) + { + write_stderr(_("%s: database initialization failed.\n"), progname); + exit(1); + } + } + + static void do_start(void) { pgpid_t pid; *** *** 694,700 progname, exitcode); exit(1); } - if (old_pid != 0) { pg_usleep(100); --- 729,734 *** *** 1535,1540 --- 1569,1575 printf(_("%s is a utility to start, stop, restart, reload configuration files,\n" "report the status of a PostgreSQL server, or signal a PostgreSQL process.\n\n"), progname); printf(_("Usage:\n")); + printf(_(" %s init[-D DATADIR] [-s] [-o \"OPTIONS\"]\n"), progname); printf(_(" %s start [-w] [-t SECS] [-D DATADIR] [-s] [-l FILENAME] [-o \"OPTIONS\"]\n"), progname); printf(_(" %s stop[-W] [-t SECS] [-D DATADIR] [-s] [-m SHUTDOWN-MODE]\n"), progname); printf(_(" %s restart [-w] [-t SECS] [-D DATADIR] [-s] [-m SHUTDOWN-MODE]\n" *** *** 1567,1573 #endif printf(_(" -l, --log FILENAME write (or append) server log to FILENAME\n")); printf(_(" -o OPTIONS command line options to pass to postgres\n" ! " (PostgreSQL server executable)\n")); printf(_(" -p PATH-TO-POSTGRESnormally not necessary\n")); printf(_("\nOptions for stop or restart:\n")); printf(_(" -m SHUTDOWN-MODE can be \"smart\", \"fast\", or \"immediate\"\n")); --- 1602,1608 #endif printf(_(" -l, --log FILENAME write (or append) server log to FILENAME\n")); printf(_(" -o OPTIONS command line options to pass to postgres\n" ! " (PostgreSQL server executable) or initdb\n")); printf(_(" -p PATH-TO-POSTGRESnormally not necessary\n")); printf(_("\nOptions for stop or restart:\n")); printf(_(" -m SHUTDOWN-MODE can be \"smart\", \"fast\", or \"immediate\"\n")); *** *** 1824,1830 exit(1); } ! if (strcmp(argv[optind], "start") == 0) ctl_command = START_COMMAND; else if (strcmp(argv[optind], "stop") == 0) ctl_command = STOP_COMMAND; --- 1859,1867 exit(1); } ! if (strcmp(argv[optind], "init") == 0) ! ctl_command = INIT_COMMAND; ! else if (strcmp(argv[optind], "start") == 0) ctl_command = START_COMMAND; else if (strcmp(argv[optind], "stop") == 0) ctl_command = STOP_COMMAND; *** *** 1921,1926 --- 1958,1966 switch (ctl_command) { + case INIT_COMMAND: + do_init(); + break; case STATUS_COMMAND: do_status(); break; -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Non-Solaris dtrace support is disabled in 8.4!!!?
Dne 5.09.09 00:09, Josh Berkus napsal(a): On 9/4/09 2:49 PM, Tom Lane wrote: Wasn't anybody paying attention? Apparently nobody is using it on other platforms. I know I'm not. I think that buildfarm members can enable it for supported platform. I already did it for my machine. Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Gothic moth fails on Tsearch2 contrib module check (PG8.2)
You can see strange error on gothic moth (Solaris Nevada, SunStudio 12, Sparc): http://www.pgbuildfarm.org/cgi-bin/show_log.pl?nm=gothic_moth&dt=2009-09-08%2019:06:01 = pgsql.19404/contrib/tsearch2/regression.diffs === *** ./expected/tsearch2.out Tue Sep 8 21:08:08 2009 --- ./results/tsearch2.out Tue Sep 8 21:38:03 2009 *** *** 2453,2459 Sea view wow foo bar qq http://www.google.com/foo.bar.html"; target="_blank">YES ! ff-bg document.write(15); --- 2453,2459 Sea view wow foo bar qq http://www.google.com/foo.bar.html"; target="_blank">YES ! ff-bgff-bg document.write(15); == ff-bg is doubled in the output and space is missing. It smells like compiler bug. I will look closer on it next week, but can somebody point me place in the code where I shall start? Thanks Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] set_client_encoding is broken
Tom Lane píše v po 31. 08. 2009 v 11:00 -0400: > 3. Push the startup-packet GUC processing (approx. lines 3340..3395 of > postgres.c, as of CVS HEAD) into InitPostgres, so it can be run during > the startup transaction. This is not too unclean, though it would > mean exporting process_postgres_switches() from postgres.c; I guess > the main thing I don't like about it is that InitPostgres has enough > weird responsibilities already. > > I'm leaning to the third choice, but I wonder if anyone has any > comments > or better ideas. It seems to me that 3 is OK. Another possibility is that InitPostgres can only fill up rel cache and GUC processing can stay on the same place. But in general, this problem can affect any other GUC variable which has assign hook and needs to lookup. I don't know how it works before, but I'm afraid that user can get error message in server encoding before it is correctly set. Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] set_client_encoding is broken
If you look on gothic_moth and comet_moth http://www.pgbuildfarm.org/cgi-bin/show_log.pl?nm=gothic_moth&dt=2009-08-30%2020:06:00 http://www.pgbuildfarm.org/cgi-bin/show_log.pl?nm=comet_moth&dt=2009-08-29%2021:06:00 you can see following error: ../../src/test/regress/pg_regress --inputdir=. --psqldir=/zfs_data/home/postgres/buildfarm/gothic_moth/HEAD/inst/bin --dbname=contrib_regression --multibyte=UTF8 --no-locale unaccent (using postmaster on Unix socket, default port) == dropping database "contrib_regression" == psql: FATAL: invalid value for parameter "client_encoding": "UTF8" command failed: "/zfs_data/home/postgres/buildfarm/gothic_moth/HEAD/inst/bin/psql" -X -c "DROP DATABASE IF EXISTS \"contrib_regression\"" "postgres" gmake[1]: *** [installcheck] Error 2 gmake[1]: Leaving directory `/zfs_data/home/postgres/buildfarm/gothic_moth/HEAD/pgsql.4092/contrib/unaccent' [4a9ae815.696e:1] LOG: connection received: host=[local] [4a9ae815.696e:2] LOG: connection authorized: user=postgres database=postgres [4a9ae815.696e:3] LOG: conversion between UTF8 and LATIN2 is not supported [4a9ae815.696e:4] FATAL: invalid value for parameter "client_encoding": "UTF8" The assign_client_encoding->SetClientEncoding fails to find conversion function. http://doxygen.postgresql.org/backend_2commands_2variable_8c.html#7f2d0624e7b7fb46644c5ce284e6479c http://doxygen.postgresql.org/mbutils_8c.html#8eeff4ecab443ba7073c426fcd4bc4d6 I guess that flat auth file patch http://archives.postgresql.org/pgsql-committers/2009-08/msg00301.php is culprint. It seems that backend does not have loaded pg_encoding yet when SetClientEncoding is processed. Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] SIGUSR1 pingpong between master na autovacum launcher causes crash
Zdenek Kotala píše v po 24. 08. 2009 v 13:47 +0200: > I tested Alvaro's patch and it works, because it does not lead to stack > consumption, but it shows another bug in StartAutovacuumWorker() code. > When fork fails bn structure is freed but > ReleasePostmasterChildSlot() should be called as well. See error: > > 2009-08-24 11:50:20.360 CEST 3468 FATAL: no free slots in PMChildFlags array > > and comment in source code: > > /* Out of slots ... should never happen, else postmaster.c messed up */ > Just a confirmation that Alvaro's patch+ReleasePostmasterChildSlot() fix solves the problem and PostgreSQL survives well during a memory shortages. Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] SIGUSR1 pingpong between master na autovacum launcher causes crash
Tom Lane píše v so 22. 08. 2009 v 09:56 -0400: > Zdenek Kotala writes: > > There are most important records from yesterdays issues. > > Messages: > > - > > Aug 20 11:14:54 genunix: [ID 470503 kern.warning] WARNING: Sorry, no swap > > space to grow stack for pid 507 (postgres) > > Hmm, that seems to confirm the idea that something had run the machine > out of memory/swap space, which would explain the repeated ENOMEM fork > failures. But we're still no closer to understanding how come the > delay in the avlauncher didn't do what it was supposed to. I found hungry process which eats up all memory and fortunately it is not postgres :-). I run also following dtrace script: dtrace -n 'syscall::kill:entry / execname=="postgres"/ { printf("%i % s, %i->%i : %i", timestamp, execname, pid, arg0, arg1); }' and it show following (little bit modified) output: CPU Timestamp[ns] diff[ms]caller callee sig 0 275074552090899,96 28604 -> 28608 16 3 2750745100280460100,23 28608 -> 28604 16 1 2750746000144690899,86 28604 -> 28608 16 3 2750746100380940100,24 28608 -> 28604 16 2 2750747000135380899,75 28604 -> 28608 16 3 2750747100171650100,04 28608 -> 28604 16 0 2750748000101050899,93 28604 -> 28608 16 3 2750748100331900100,23 28608 -> 28604 16 1 2750749000148550899,82 28604 -> 28608 16 3 2750749100386640100,24 28608 -> 28604 16 2 275075095040899,71 28604 -> 28608 16 3 2750750100127780100,03 28608 -> 28604 16 We can see there that AVlauncher really wait 100ms, but it is not enough when system is under stress. I tested Alvaro's patch and it works, because it does not lead to stack consumption, but it shows another bug in StartAutovacuumWorker() code. When fork fails bn structure is freed but ReleasePostmasterChildSlot() should be called as well. See error: 2009-08-24 11:50:20.360 CEST 3468 FATAL: no free slots in PMChildFlags array and comment in source code: /* Out of slots ... should never happen, else postmaster.c messed up */ I think that Alvaro's patch is good and it fix a crash problem. I also think that AVlauncher could wait little bit more then 100ms. When system cannot fork, I don't see any reason why hurry to repeat a fork operation. Maybe 1s is good compromise. Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] SIGUSR1 pingpong between master na autovacum launcher causes crash
Tom Lane píše v pá 21. 08. 2009 v 18:06 -0400: > Maybe, but I think we need to understand exactly what happened first. I try to mine more data from the system to reconstruct what happen. Unfortunately, default postgresql log configuration does not have timestamp. The postgresql had no load, system was under load which could cause some slow system response. ... and ... bingo I'm able to reproduce it. I will look on it deeply with dtrace on Monday (I'm leaving now for a weekend). There are most important records from yesterdays issues. Messages: - Aug 20 11:14:54 genunix: [ID 470503 kern.warning] WARNING: Sorry, no swap space to grow stack for pid 507 (postgres) svc log: [ Aug 20 10:53:27 Stopping because service disabled. ] [ Aug 20 10:53:27 Executing stop method ("/lib/svc/method/postgres_84 stop"). ] waiting for server to shut down. done server stopped [ Aug 20 10:53:31 Method "stop" exited with status 0. ] [ Aug 20 10:54:17 Executing start method ("/lib/svc/method/postgres_84 start"). ] server starting [ Aug 20 10:54:19 Method "start" exited with status 0. ] [ Aug 20 11:14:58 Stopping because process dumped core. ] [ Aug 20 11:16:05 Executing stop method ("/lib/svc/method/postgres_84 stop"). ] pg_ctl: could not send stop signal (PID: 507): No such process [ Aug 20 11:16:12 Method "stop" exited with status 0. ] [ Aug 20 11:17:17 Executing start method ("/lib/svc/method/postgres_84 start"). ] ld.so.1: sh: fatal: mmap anon failed: Resource temporarily unavailable [ Aug 20 11:17:28 Method "start" failed due to signal KILL. ] [ Aug 20 11:18:39 Executing start method ("/lib/svc/method/postgres_84 start"). ] ld.so.1: sh: fatal: mmap anon failed: Resource temporarily unavailable [ Aug 20 11:18:50 Method "start" failed due to signal KILL. ] [ Aug 20 11:19:58 Executing start method ("/lib/svc/method/postgres_84 start"). ] ld.so.1: sh: fatal: mmap anon failed: Resource temporarily unavailable postgresql.log: --- LOG: received smart shutdown request LOG: autovacuum launcher shutting down LOG: shutting down LOG: database system is shut down LOG: database system was shut down at 2009-08-20 10:53:30 CEST LOG: database system is ready to accept connections LOG: autovacuum launcher started LOG: could not fork autovacuum worker process: Not enough space ... TopMemoryContext: 31192 total in 4 blocks; 6056 free (4 chunks); 25136 used TopTransactionContext: 8192 total in 1 blocks; 7792 free (1 chunks); 400 used TransactionAbortContext: 32768 total in 1 blocks; 32752 free (0 chunks); 16 used Portal hash: 8192 total in 1 blocks; 3912 free (0 chunks); 4280 used PortalMemory: 0 total in 0 blocks; 0 free (0 chunks); 0 used Relcache by OID: 8192 total in 1 blocks; 4936 free (0 chunks); 3256 used CacheMemoryContext: 274480 total in 18 blocks; 17072 free (2 chunks); 257408 used ERROR: out of memory DETAIL: Failed on request of size 8. LOG: could not fork autovacuum worker process: Not enough space ... LOG: could not fork autovacuum worker process: Not enough space WARNING: worker took too long to start; cancelled LOG: could not fork autovacuum worker process: Not enough space ... LOG: could not fork autovacuum worker process: Not enough space WARNING: worker took too long to start; cancelled 2009-08-21 13:30:48.701 CEST 416 LOG: database system was interrupted; last known up at 2009-08-20 10:54:21 CEST 2009-08-21 13:30:48.963 CEST 416 LOG: database system was not properly shut down; automatic recovery in progress -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] SIGUSR1 pingpong between master na autovacum launcher causes crash
Alvaro Herrera píše v pá 21. 08. 2009 v 17:48 -0400: > Tom Lane wrote: > > > I'd still like to have some fork-rate-limiting behavior in there > > somewhere. However, it might make sense for the avlauncher to do that > > rather than the postmaster. Does that idea seem more implementable? > > > Does it just need a longer delay? I see two problem with delay. First is how big it should be? It depends on machine speed and load. And second is that it can be interrupted by signal. Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] SIGUSR1 pingpong between master na autovacum launcher causes crash
Alvaro Herrera píše v pá 21. 08. 2009 v 15:40 -0400: > Zdenek Kotala wrote: > > > The problem what I see here is that StartAutovacuumWorker() fails and > > send SIGUSR1 to the postmaster, but it send it too quickly and signal > > handler is still active. When signal mask is unblocked in > > sigusr1_handler() than signal handler is run again... > > > > The reason why StartAutovacuumWorker() is interesting. Log says: > > > > LOG: could not fork autovacuum worker process: Not enough space > > Does this mean that the machine is out of swap space? It is ENOMEM error. But it is strange. Machine has 4GB RAM and it was freshly installed PG84 without any data and with default configuration. It was not under load. I did not find any clue what happend with memory on this system. The question is if out of memory was a source or result of the pinpong. > > > It is strange and I don't understand it. May be too many nested signal > > handlers call could cause it. > > > > Strange also is that 100ms is not enough to protect this situation, but > > I think that sleep could interrupted by signal. > > > > My suggestion is to set for example gotUSR1=true in sigusr1_handler() > > and in the server loop check if we got a USR1 signal. It avoids any > > problems with signal handler which is not currently POSIX compliant > > anyway. > > What 100ms? The pg_usleep call you see in ServerLoop is only there > during shutdown; normally it would be the select() call that would be > blocking the process. I mean AutoVacLauncherMain() http://doxygen.postgresql.org/autovacuum_8c.html#19ef1013e6110a4536ed92a454aba8c9 line 656 > If sigusr1_handler needs rewriting, don't all the other sighandler as > well? Note that the process is supposed to be running with signals > blocked all the time except during those sleep/select calls, which is > what (according to comments) let the sighandlers do nontrivial tasks. Comments says that it is OK. POSIX says that is not OK and my instinct say to trust the POSIX standard. Especially I do not see any reason why we need do this in signal handler. avl_sigterm_handler and so on are good example how it should be implemented in postmaster as well. The core shows that it is not good idea to have complicated signal handler. Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] SIGUSR1 pingpong between master na autovacum launcher causes crash
I found following core file of PG 8.4.0 on my system (Solaris Nevada b119): fe8ae42d _dowrite (85bf6e8, 3a, 8035e3c, 80350e8) + 8d fe8ae743 _ndoprnt (85bf6e8, 8035ec8, 8035e3c, 0) + 2ba fe8b322d vsnprintf (85bfaf0, 3ff, 85bf6e8, 8035ec8, 0, 0) + 65 082194ea appendStringInfoVA (8035e9c, 85bf6e8, 8035ec8) + 4a 083ca5d3 errmsg (849c340, 0) + 103 0829272d StartAutoVacWorker (fe97f000, 32, 85b82b0, 8035ef4, 82a1496, c) + 3d 082a1901 StartAutovacuumWorker (c, 8035f08, fe8ed28f, 10, 0, 8035fbc) + 71 082a1496 sigusr1_handler (10, 0, 8035fbc) + 186 fe8ed28f __sighndlr (10, 0, 8035fbc, 82a1310) + f fe8e031f call_user_handler (10) + 2af fe8e054f sigacthandler (10, 0, 8035fbc) + df --- called from signal handler with signal 16 (SIGUSR1) --- fe8f37f6 __systemcall (3, fec32b88, 0, fe8e0b46) + 6 fe8e0c71 thr_sigsetmask (3, 85abd50, 0, fe8e0d18) + 139 fe8e0d3f sigprocmask (3, 85abd50, 0) + 31 082a14a4 sigusr1_handler (10, 0, 8036340) + 194 fe8ed28f __sighndlr (10, 0, 8036340, 82a1310) + f fe8e031f call_user_handler (10) + 2af fe8e054f sigacthandler (10, 0, 8036340) + df ... 80x same sighandler stack --- called from signal handler with signal 16 (SIGUSR1) --- fe8f37f6 __systemcall (3, fec32b88, 0, fe8e0b46) + 6 fe8e0c71 thr_sigsetmask (3, 85abd50, 0, fe8e0d18) + 139 fe8e0d3f sigprocmask (3, 85abd50, 0) + 31 082a14a4 sigusr1_handler (10, 0, 80478fc) + 194 fe8ed28f __sighndlr (10, 0, 80478fc, 82a1310) + f fe8e031f call_user_handler (10) + 2af fe8e054f sigacthandler (10, 0, 80478fc) + df --- called from signal handler with signal 16 (SIGUSR1) --- fe8f1867 __pollsys (8047b50, 2, 8047c04, 0) + 7 fe89ce61 pselect (6, 8047c44, 0, 0, 8047c04, 0) + 199 fe89d236 select (6, 8047c44, 0, 0, 8047c38, 0) + 78 0829dc20 ServerLoop (feffb804, bd26003b, 41b21fcb, 85c1de0, 1, 0) + c0 0829d5d0 PostmasterMain (3, 85b72c8) + dd0 08227abf main (3, 85b72c8, 8047df0, 8047d9c) + 22f 080b893d _start (3, 8047e80, 8047ea5, 8047ea8, 0, 8047ec2) + 7d The problem what I see here is that StartAutovacuumWorker() fails and send SIGUSR1 to the postmaster, but it send it too quickly and signal handler is still active. When signal mask is unblocked in sigusr1_handler() than signal handler is run again... The reason why StartAutovacuumWorker() is interesting. Log says: LOG: could not fork autovacuum worker process: Not enough space It is strange and I don't understand it. May be too many nested signal handlers call could cause it. Strange also is that 100ms is not enough to protect this situation, but I think that sleep could interrupted by signal. My suggestion is to set for example gotUSR1=true in sigusr1_handler() and in the server loop check if we got a USR1 signal. It avoids any problems with signal handler which is not currently POSIX compliant anyway. any other ideas? Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] uuid contrib don't compile in OpenSolaris
Dne 24.07.09 19:23, Emanuel Calvo Franco napsal(a): Hi all, I have some issues to compile uuid contrib of 8.4 version. Touching something i see that the gmake don't find uuid.h. (pfexec gmake -d) Touching more, i add uuid.h into the uuid directory and i had a error message: missing separator. Did you use gmake the second time? What was the exact make and compiler error message? Every time I execute the gmake, the error is the same (uuid.h not found). WHen i copy uuid.h in the current dir of uuid contrib, the error changes to missing separator. As I say, looking in the web i found that maybe is a bad character (a hidden tab or something like this). This occurs only in opensolaris 200906 (i didn't test on previous versions). Hi, problem is that PostgreSQL uses ossp UUID which is not integrated in OpenSolaris. Solaris has own uuid implementation which seems to me similar but not same. Try man uuid_generate or less /usr/uuid/uuid.h It should be easy to port it but it needs a time :(. Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] compilation with libeditpreferred is broken
I attached conservative version of patch which only reorder #define to avoid cross including half from readline and half from editline. Zdenek Tom Lane píše v čt 06. 08. 2009 v 18:13 -0400: > Zdenek Kotala writes: > > It seems to me that editline never distributed history.h file and > > HAVE_EDITLINE_HISTORY_H is nonsense. But I'm not sure. > > I wouldn't count on that, in part because there are so many versions of > editline. On an OS X machine I see > > $ ls -l /usr/include/*line* > /usr/include/editline: > total 16 > -rw-r--r-- 1 root wheel 6882 Feb 19 2008 readline.h > > /usr/include/readline: > total 16 > lrwxr-xr-x 1 root wheel 22 Jul 23 11:31 history.h@ -> > ../editline/readline.h > lrwxr-xr-x 1 root wheel 22 Jul 23 11:31 readline.h@ -> > ../editline/readline.h > > regards, tom lane diff -r dd477d7938da src/bin/psql/input.h --- a/src/bin/psql/input.h Wed Jun 03 00:38:34 2009 + +++ b/src/bin/psql/input.h Wed Aug 12 21:49:49 2009 +0200 @@ -18,17 +18,19 @@ #define USE_READLINE 1 #if defined(HAVE_READLINE_READLINE_H) #include +#if defined(HAVE_READLINE_HISTORY_H) +#include +#endif #elif defined(HAVE_EDITLINE_READLINE_H) #include +#if defined(HAVE_EDITLINE_HISTORY_H) +#include +#endif #elif defined(HAVE_READLINE_H) #include +#if defined(HAVE_HISTORY_H) +#include #endif -#if defined(HAVE_READLINE_HISTORY_H) -#include -#elif defined(HAVE_EDITLINE_HISTORY_H) -#include -#elif defined(HAVE_HISTORY_H) -#include #endif #endif -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Re: [Pg-migrator-general] Composite types break pg_migrated tables
Dne 6.08.09 04:29, Tom Lane napsal(a): Andrew Dunstan writes: preventing a clash might be fairly difficult. Yeah, I was just thinking about that. The easiest way to avoid collisions would be to make pg_dump (in --binary-upgrade mode) responsible for being sure that *every* new pg_type and pg_class row OID matches what it was in the old DB. We could stop doing that once we have all the user tables in place --- I don't believe it's necessary to preserve the OIDs of user indexes. But we need to preserve toast table OIDs, and toast table index OIDs too if those are created at the same time they are now (else we risk one of them colliding with a toast table OID we want to create later). Oh, and pg_enum rows too. It seems doable, but we're certainly not going to back-patch any such thing into 8.4 ... Another way is to use direct catalog update which I presented on PgCon. I think it should be easy to finish it (2-3weeks) for 8.4 - needs small extension of bootstrap. And of course testing, testing ... Also to remove oid in catalog and replace it with standard column (type can be oid) should make things easier. But it is for 8.5. I will send a code on Monday for people who wants to look on it. Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] compilation with libeditpreferred is broken
Dne 7.08.09 00:13, Tom Lane napsal(a): Zdenek Kotala writes: It seems to me that editline never distributed history.h file and HAVE_EDITLINE_HISTORY_H is nonsense. But I'm not sure. I wouldn't count on that, in part because there are so many versions of editline. On an OS X machine I see $ ls -l /usr/include/*line* /usr/include/editline: total 16 -rw-r--r-- 1 root wheel 6882 Feb 19 2008 readline.h /usr/include/readline: total 16 lrwxr-xr-x 1 root wheel 22 Jul 23 11:31 history.h@ -> ../editline/readline.h lrwxr-xr-x 1 root wheel 22 Jul 23 11:31 readline.h@ -> ../editline/readline.h It is only hack for application which wants to have readline and they don't detects libedit. Finally it is like: #include #include I little bit searching on Internet and it seems that when edit/history.h exists it is link to editline/readline.h. (See e.g. http://sysinf0.klabs.be/usr/include/editline/history.h?dist=;arch= ) This link is created by packager, because when you compiling libedit, make install does not create it. By my opinion HAVE_EDITLINE_HISTORY_H is overhead, but we can keep it. Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] compilation with libeditpreferred is broken
When I try compile postgresql with --libeditpreferred option, compilation fails when readline is also installed on the system. You can see error report on: http://www.pgbuildfarm.org/cgi-bin/show_log.pl?nm=dot_moth&dt=2009-08-06%2012:46:04 The main problem is in src/bin/psql/input.h where is following ifdef magic: #define USE_READLINE 1 #if defined(HAVE_READLINE_READLINE_H) #include #elif defined(HAVE_EDITLINE_READLINE_H) #include #elif defined(HAVE_READLINE_H) #include #endif #if defined(HAVE_READLINE_HISTORY_H) #include #elif defined(HAVE_EDITLINE_HISTORY_H) #include #elif defined(HAVE_HISTORY_H) #include #endif The problem is that libedit does not distribute editline/history.h and configure detects that there is readline/history.h and sets HAVE_READLINE_HISTORY_H macro. Finally input.h includes editline/readline.h and readline/history.h which causes symbol conflicts. It seems to me that editline never distributed history.h file and HAVE_EDITLINE_HISTORY_H is nonsense. But I'm not sure. I attached suggested fix, but it needs also some work in ./configure - depends if libedit/history.h existed. Anyone knows a history? I need to backported this fix for branches 8.2 - 8.4, because OpenSolaris PostgreSQL binaries build is broken now. Zdenek diff -r dd477d7938da src/bin/psql/input.h --- a/src/bin/psql/input.h Wed Jun 03 00:38:34 2009 + +++ b/src/bin/psql/input.h Thu Aug 06 22:24:58 2009 +0200 @@ -18,17 +18,16 @@ #define USE_READLINE 1 #if defined(HAVE_READLINE_READLINE_H) #include +#if defined(HAVE_READLINE_HISTORY_H) +#include +#endif #elif defined(HAVE_EDITLINE_READLINE_H) #include #elif defined(HAVE_READLINE_H) #include +#if defined(HAVE_HISTORY_H) +#include #endif -#if defined(HAVE_READLINE_HISTORY_H) -#include -#elif defined(HAVE_EDITLINE_HISTORY_H) -#include -#elif defined(HAVE_HISTORY_H) -#include #endif #endif -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] head contrib is broken (crypto)
It seems that last contrib crypto changes broke buildfarm. You can see problem e.g. here: http://www.pgbuildfarm.org/cgi-bin/show_log.pl?nm=emperor_moth&dt=2009-08-04%2020:06:01 I don't have time to look on it now. But it should be reproducible everywhere. Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] boolean in C
Grzegorz Jaskiewicz píše v čt 16. 07. 2009 v 14:59 +0100: > > > >> Why C89, and not C99 ? Virtually all compilers for last 4 years have/ > >> had C99 support. > > > > Well, I think we want to run on systems that are older than 4 years, > > too. > > > Sure, but that's probably less than 1% of all systems. > The 4 years was a guess, I think its much more than that. For example Solaris 8 is 9 years old and still is used in production. I guess HP-UX is in same situation. And so on. I not able to say how many PostgreSQL runs on them but how Tom mentioned there is no significant reason to break old platform. Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [GENERAL] pg_migrator not setting values of sequences?
Dne 14.07.09 04:37, Bruce Momjian napsal(a): Tom Lane wrote: Bruce Momjian writes: Tilmann Singer wrote: However, all of the sequences were at the initial values and not bumped up to the last used value as I would have expected. The first nextval call on any sequence in the migrated 8.4 database always returned 1. Wow, that is also surprising. I am going to have to run some tests to find the cause, but it certainly is not intended. Looks like pg_migrator neglects to include relkind 'S' in the set of tables that it needs to physically migrate. Thanks, I have fixed pg_migrator with the attached patch. Once we find the cause of the lovacuum problem, I will make a new pg_migrator release. I think you don't need transfer sequence table, because pg_dump should create DDL command (CREATE SEQUENCE) which should create new nice sequence relation. Zdenek
Re: [HACKERS] First CommitFest: July 15th
Robert Haas píše v čt 02. 07. 2009 v 22:16 -0400: > On Thu, Jul 2, 2009 at 3:42 PM, Zdenek Kotala wrote: > Also, how would we handle changes by committers, who > don't always go through the CommitFest process? I think that all head patches should go to through a new tool for recording also in case when developer is commiter itself. Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] First CommitFest: July 15th
Peter Eisentraut píše v pá 03. 07. 2009 v 09:19 +0300: > On Friday 03 July 2009 05:16:41 Robert Haas wrote: > > On Thu, Jul 2, 2009 at 3:42 PM, Zdenek Kotala wrote: > > > Josh Berkus píše v st 01. 07. 2009 v 17:21 -0700: > > >> Folks, > > >> > > >> There's been a lot of discussion/argument around how to handle the last > > >> commitfest, but there seems to be a total consensus that we want to have > > >> the first CF on July 15th. > > > > > > Can we add flags like bump catalog version, bump page layout version, > > > modify AM for each patch? It should help to track pg_upgrade changes. > > > > That's not a bad idea, and it wouldn't be hard to add various flags > > and things to the CommitFest app I wrote, but how would we maintain > > the information and keep it correct? It seems like there might be a > > danger that patch authors wouldn't know whether or not they were doing > > those things. Also, how would we handle changes by committers, who > > don't always go through the CommitFest process? > > I think this information could be computed automatically, if someone cared > enough. It shouldn't be necessary to bother every single participant in the > process with this. I think that developer is responsible for his patch. He should know what he doing. When he will send a patch and see checkbox like "modified AM?" then he should know what he modified? It is also warning for commiter that catalog version has to be bumped during a commit. I don't see any method how to check automatically. Something could be possible - like structure checker. But when meaning of data is going to be different. Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Upgrading our minimum required flex version for 8.5
Tom Lane píše v čt 02. 07. 2009 v 13:13 -0400: > > Actually, most of the buildfarm members show which flex version they are > running in the configure output. A quick look shows that of the 45 > members that have reported on HEAD in the past 2 days, 22 are running > 2.5.4, which is a lot higher than I was expecting. Most of these are > the Solaris boxen, which I imagine can be updated fairly painlessly > since there are some of them that are already running something newer. My three S10 are updated. All of them use version 2.5.35 now. S8, S9 is not yet under my control and they are not migrated yet to new location. Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] First CommitFest: July 15th
Josh Berkus píše v st 01. 07. 2009 v 17:21 -0700: > Folks, > > There's been a lot of discussion/argument around how to handle the last > commitfest, but there seems to be a total consensus that we want to have > the first CF on July 15th. Can we add flags like bump catalog version, bump page layout version, modify AM for each patch? It should help to track pg_upgrade changes. Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Upgrading our minimum required flex version for 8.5
Tom Lane píše v čt 02. 07. 2009 v 13:13 -0400: > I wrote: > > Yes. What I was thinking of doing was committing a configure change to > > reject flex < 2.5.31, and waiting to see how much of the buildfarm goes > > red. > > Actually, most of the buildfarm members show which flex version they are > running in the configure output. A quick look shows that of the 45 > members that have reported on HEAD in the past 2 days, 22 are running > 2.5.4, which is a lot higher than I was expecting. Most of these are > the Solaris boxen, which I imagine can be updated fairly painlessly > since there are some of them that are already running something newer. I hope, need to check/compile, but I don't expect any problems. Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Upgrading our minimum required flex version for 8.5
Tom Lane píše v čt 02. 07. 2009 v 11:32 -0400: > Andrew Dunstan writes: > > Tom Lane wrote: > >> Right, this will only affect people doing development or otherwise > >> building from a CVS pull. > > > Of course, that includes the whole buildfarm. We might need to ask some > > people to upgrade there. > > Yes. What I was thinking of doing was committing a configure change to > reject flex < 2.5.31, and waiting to see how much of the buildfarm goes > red. Solaris 10 has 2.5.4 version. OpenSolaris already has 2.5.33. Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] list_head naming conflict gcc 4.2/perl/solaris
Zdenek Kotala píše v po 01. 06. 2009 v 22:45 +0200: > Tom Lane píše v po 01. 06. 2009 v 16:09 -0400: > > Zdenek Kotala writes: > > > > What is , and why is it being imported by the Perl headers? > > It seems that problem is with Perl. It includes sys/mode.h. The new > change for gcc 4.2 is that mode.h includes vnode.h and it finally > sys/list.h which is generic list for kernel structures. > > > Can we do something like "#define list_head solaris_list_head" around > > the problematic #include? (Which one is that, anyway?) > > I will try it. There is following hack: > > CFLAGS="-D_SYS_VNODE_H" > > which disable vnode.h and breaks include chain. > > I need to install fresh nightly build of Solaris to test it on live > system. Perl includes sys/mode.h which is probably not necessary (it look likes that it is something for AIX), but it seems that sys/*.h headers modifications break more than only PostgreSQL. And kernel guys should fix it. Which means no change for PostgreSQL. Only who will use opensolaris devel builds 116 could have a problem. I will skip these build(s) on my buildfarm animals. Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] list_head naming conflict gcc 4.2/perl/solaris
Tom Lane píše v po 01. 06. 2009 v 16:09 -0400: > Zdenek Kotala writes: > What is , and why is it being imported by the Perl headers? It seems that problem is with Perl. It includes sys/mode.h. The new change for gcc 4.2 is that mode.h includes vnode.h and it finally sys/list.h which is generic list for kernel structures. > Can we do something like "#define list_head solaris_list_head" around > the problematic #include? (Which one is that, anyway?) I will try it. There is following hack: CFLAGS="-D_SYS_VNODE_H" which disable vnode.h and breaks include chain. I need to install fresh nightly build of Solaris to test it on live system. Thanks Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] list_head naming conflict gcc 4.2/perl/solaris
Robert Haas píše v po 01. 06. 2009 v 16:03 -0400: > On Mon, Jun 1, 2009 at 3:57 PM, Zdenek Kotala wrote: > > During integration gcc4.2 into Solaris. My colleague hit a following > > problem with PostgreSQL compilation: > > > > http://bugs.opensolaris.org/bugdatabase/view_bug.do?bug_id=6845982 > > > > cd > > /builds/sfw-fixes/usr/src/cmd/postgres/postgresql-8.2/postgresql-8.2.13/src/pl/plperl > > + /ws/onnv-tools/SUNWspro/SS12/bin/cc -xO3 -m32 -xchip=pentium -xspace -Xa > > -xildoff -xc99=all -xc99=none -xCC -KPIC -I. -I../../../src/include > > -I/builds/sfw-fixes/proto/root_i386/usr/include > > -I/builds/sfw-fixes/proto/root_i386/usr/sfw/include -I/usr/sfw/include > > -I/usr/include/kerberosv5 -I/usr/perl5/5.8.4/lib/i86pc-solaris-64int/CORE > > -c -o plperl.o plperl.c > > "/usr/include/sys/list.h", line 52: identifier redeclared: list_head > >current : function(pointer to struct list {unsigned int list_size, > > unsigned int list_offset, struct list_node {..} list_head}) returning > > pointer to void > >previous: function(pointer to struct List {enum NodeTag > > {T_TIDBitmap(902), (blah blah) > > > > At this moment there is no clear conclusion how to fix it in solaris system > > headers to satisfy all side. :( > > > > My idea is to rename list_head to pg_list_head (and other functions > > analogously) to avoid name conflict. > > > >Comments, better ideas? > > Uh... isn't it rather bad form for the operating system headers to be > defining commonly used type names like "list"? It is in solaris since 2003 for kernel structure linked list. When I look on it closer, it seems that problem is more in perl which includes some sys/ headers which is probably not good practice. :( > I'd be very surprised > if PostgreSQL is the only application that is going to break on this. At this moment I know only about PostgreSQL. How I understand it now, it appears only when source is compiling with Perl. I will look more on it tomorrow. thanks Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] list_head naming conflict gcc 4.2/perl/solaris
During integration gcc4.2 into Solaris. My colleague hit a following problem with PostgreSQL compilation: http://bugs.opensolaris.org/bugdatabase/view_bug.do?bug_id=6845982 cd /builds/sfw-fixes/usr/src/cmd/postgres/postgresql-8.2/postgresql-8.2.13/src/pl/plperl + /ws/onnv-tools/SUNWspro/SS12/bin/cc -xO3 -m32 -xchip=pentium -xspace -Xa -xildoff -xc99=all -xc99=none -xCC -KPIC -I. -I../../../src/include -I/builds/sfw-fixes/proto/root_i386/usr/include -I/builds/sfw-fixes/proto/root_i386/usr/sfw/include -I/usr/sfw/include -I/usr/include/kerberosv5 -I/usr/perl5/5.8.4/lib/i86pc-solaris-64int/CORE -c -o plperl.o plperl.c "/usr/include/sys/list.h", line 52: identifier redeclared: list_head current : function(pointer to struct list {unsigned int list_size, unsigned int list_offset, struct list_node {..} list_head}) returning pointer to void previous: function(pointer to struct List {enum NodeTag {T_TIDBitmap(902), (blah blah) At this moment there is no clear conclusion how to fix it in solaris system headers to satisfy all side. :( My idea is to rename list_head to pg_list_head (and other functions analogously) to avoid name conflict. Comments, better ideas? Thanks Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] pg_migrator and an 8.3-compatible tsvector data type
Tom Lane píše v pá 29. 05. 2009 v 11:28 -0400: > Zdenek Kotala writes: > > The biggest problem is dictionary change. I'm not sure if it happened > > but IIRC Teodor mentioned it in Ottawa. If it happened It hits down > > tsvector compatibility at all. > > No more than changing dictionary behavior in an existing installation. > What was stated when the issue came up during 8.3 development is that > you don't normally need to worry about small changes in dictionary > behavior because overall text search behavior will still be "close > enough". (I seem to recall that I'd complained that any change in > dictionary behavior would invalidate indexes based on the dictionary, > and this was the answer.) It seems to be OK then. However, pg_migrator should inform about it. Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] pg_migrator and an 8.3-compatible tsvector data type
Bruce Momjian píše v čt 28. 05. 2009 v 17:42 -0400: > Josh Berkus wrote: > > On 5/28/09 2:30 PM, Bruce Momjian wrote: > > > Because no one has responded, I am going to prevent pg_migrator from > > > working with a cluster that uses tsvector. I realize this limits > > > pg_migrator's usefulness, but I have to move on. > > > > I don't know how to fix the issue, certainly. > > > > Why was there a format change to tsvector anyway? Was there an > > important reason for this? > > The ordering of the lexems was changed: The biggest problem is dictionary change. I'm not sure if it happened but IIRC Teodor mentioned it in Ottawa. If it happened It hits down tsvector compatibility at all. Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Compiler warning cleanup - unitilized const variables, pointer type mismatch
Tom Lane píše v čt 28. 05. 2009 v 11:57 -0400: > ). > > AFAICS, Sun's compiler is just too stupid and shouldn't be emitting > this warning. Perhaps the right response is to file a bug report > against the compiler. I checked it and it is already know bug. It is new lint style check in Sun Studio 12. Unfortunately, problem is that current workflow does not allow to detect if code is dead or not in the verification phase. Next sun studio release could fix it. Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Compiler warning cleanup - unitilized const variables, pointer type mismatch
Tom Lane píše v čt 28. 05. 2009 v 11:42 -0400: > Zdenek Kotala writes: > > I attached another cleanup patch which fixes following warnings reported > > by Sun Studio: > > I'm not too impressed with any of these. The proposed added > initializers just increase future maintenance effort without solving > any real problem (since the variables are required by C standard to > initialize to zero). Agree. > The proposed signature change on psql_completion > is going to replace a warning on your system with outright failures on > other people's. I check readline and definition is still same at least from 5.0 version. I'm not still understand why it should failure on other systems. I looked on revision 1.30 of the file and there is only readline-4.2 support mentioned. Is readline 4.2 the problem? Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] libpq is not thread safe
Bruce Momjian píše v čt 28. 05. 2009 v 17:20 -0400: > > > > Done, patch attached and applied. > > I went with a because it seemed most appropriate, but it looks > very large: > > http://developer.postgresql.org/pgdocs/postgres/libpq-connect.html > > Should it be a notice? I prefer warning. It is important message and beginners usually don't know it. Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Compiler warning cleanup - unitilized const variables, pointer type mismatch
Michael Meskes píše v čt 28. 05. 2009 v 14:47 +0200: > On Thu, May 28, 2009 at 01:51:07PM +0200, Zdenek Kotala wrote: > > Problem is with YYLLOC_DEFAULT. When I look on macro definition > > > > #define YYLLOC_DEFAULT(Current, Rhs, N) \ > > Current.first_line = Rhs[1].first_line; \ > > Current.first_column = Rhs[1].first_column;\ > > Current.last_line= Rhs[N].last_line; \ > > Current.last_column = Rhs[N].last_column; > > > > It seems to me that it is OK, because 1 is used as a index which finally > > point on yyerror_range[0]. > > Wait, this is the bison definition. Well to be more precise the bison > definition in your bison version. Mine is different: I took it from documentation. I have same as you in generated code. > # define YYLLOC_DEFAULT(Current, Rhs, N)\ > do \ > if (YYID (N))\ > { \ > (Current).first_line = YYRHSLOC (Rhs, 1).first_line;\ > (Current).first_column = YYRHSLOC (Rhs, 1).first_column; \ > (Current).last_line= YYRHSLOC (Rhs, N).last_line; \ > (Current).last_column = YYRHSLOC (Rhs, N).last_column; \ > } \ > else \ > { \ > (Current).first_line = (Current).last_line = \ > YYRHSLOC (Rhs, 0).last_line;\ > (Current).first_column = (Current).last_column = \ > YYRHSLOC (Rhs, 0).last_column; \ >} \ > while (YYID (0)) > > Having said that, it doesn't really matter as we redefine the macro: > > #define YYLLOC_DEFAULT(Current, Rhs, N) \ > do { \ > if (N) \ > (Current) = (Rhs)[1]; \ > else \ > (Current) = (Rhs)[0]; \ > } while (0) > > I have to admit that those version look strikingly unsimilar to me. There is > no > reference to Rhs[N] in our macro at all. But then I have no idea whether this > is needed. Current is only int. See gramparse.h. I think we could rewrite it this way: #define YYLLOC_DEFAULT(Current, Rhs, N) \ do { \ if (N) \ (Current) = (Rhs)[1]; \ else \ (Current) = (Rhs)[N]; \ } while (0) It is same result and compiler is quite. Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Compiler warning cleanup - unitilized const variables, pointer type mismatch
Michael Meskes píše v čt 28. 05. 2009 v 13:33 +0200: > On Thu, May 28, 2009 at 11:11:20AM +0200, Zdenek Kotala wrote: > > I attached another cleanup patch which fixes following warnings reported > > by Sun Studio: > > ... > > "preproc.c", line 39569: warning: pointer expression or its operand do not > > point to the same object yyerror_range, result is undefined and > > non-portable > > ... > > Following list is still unfixed plus see my comments: > > > > "gram.c", line 28487: warning: pointer expression or its operand do not > > point to the same object yyerror_range, result is undefined and > > non-portable > > ... > > These two should be the same, both coming from bison. Both files are > auto-generated, thus it might be bison that has to be fixed to remove this > warning. yeah it is generated, but question is if generated code is valid or it is bug in bison. If it bison bug we need to care about it. There is the code: yyerror_range[1] = yylloc; /* Using YYLLOC is tempting, but would change the location of the look-ahead. YYLOC is available though. */ YYLLOC_DEFAULT (yyloc, (yyerror_range - 1), 2); *++yylsp = yyloc; Problem is with YYLLOC_DEFAULT. When I look on macro definition #define YYLLOC_DEFAULT(Current, Rhs, N) \ Current.first_line = Rhs[1].first_line; \ Current.first_column = Rhs[1].first_column;\ Current.last_line= Rhs[N].last_line; \ Current.last_column = Rhs[N].last_column; It seems to me that it is OK, because 1 is used as a index which finally point on yyerror_range[0]. > Given that I didn't find any mentioning of preproc in your patch I > suppose it just hit the wrong list though. I'm sorry copy paste error. Yeah, I did not fix preproc too. Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Compiler warning cleanup - unitilized const variables, pointer type mismatch
I attached another cleanup patch which fixes following warnings reported by Sun Studio: "zic.c", line 1534: warning: const object should have initializer: tzh0 "dynloader.c", line 7: warning: empty translation unit "pgstat.c", line 666: warning: const object should have initializer: all_zeroes "pgstat.c", line 799: warning: const object should have initializer: all_zeroes "pgstat.c", line 2552: warning: const object should have initializer: all_zeroes "preproc.c", line 39569: warning: pointer expression or its operand do not point to the same object yyerror_range, result is undefined and non-portable "tab-complete.c", line 587: warning: assignment type mismatch: pointer to function(pointer to const char, int, int) returning pointer to pointer to char "=" pointer to void Following list is still unfixed plus see my comments: "gram.c", line 28487: warning: pointer expression or its operand do not point to the same object yyerror_range, result is undefined and non-portable - This is really strange warning. The code is really strange because it points to -1 index of array, but I'm not bison guru. Maybe it is correct, but it would be good if somebody check it. "../../../src/include/pg_config.h", line 782: warning: macro redefined: _FILE_OFFSET_BITS - This probably needs some extra love in configure. "regc_lex.c", line 401: warning: loop not entered at top "regc_lex.c", line 484: warning: loop not entered at top "regc_lex.c", line 578: warning: loop not entered at top "regc_lex.c", line 610: warning: loop not entered at top "regc_lex.c", line 870: warning: loop not entered at top "regc_lex.c", line 1073: warning: loop not entered at top "postgres.c", line 3845: warning: loop not entered at top - Assert on not reached place probably confuse compiler. I'm not sure if it make sense nowadays? Most compiler should optimize this anyway and code is removed. I suppose to remove these asserts. Zdenek diff -Nrc pgsql.orig.8fc4f032818a/src/backend/port/dynloader/solaris.c pgsql.orig/src/backend/port/dynloader/solaris.c *** pgsql.orig.8fc4f032818a/src/backend/port/dynloader/solaris.c 2009-05-28 11:09:24.874020865 +0200 --- pgsql.orig/src/backend/port/dynloader/solaris.c 2009-05-28 11:09:24.893688008 +0200 *** *** 5,7 --- 5,11 * * see solaris.h */ + + /* compiler complains about empty unit, let compiler quite */ + extern int no_such_variable; + diff -Nrc pgsql.orig.8fc4f032818a/src/backend/postmaster/pgstat.c pgsql.orig/src/backend/postmaster/pgstat.c *** pgsql.orig.8fc4f032818a/src/backend/postmaster/pgstat.c 2009-05-28 11:09:24.882883806 +0200 --- pgsql.orig/src/backend/postmaster/pgstat.c 2009-05-28 11:09:24.894106321 +0200 *** *** 662,669 void pgstat_report_stat(bool force) { ! /* we assume this inits to all zeroes: */ ! static const PgStat_TableCounts all_zeroes; static TimestampTz last_report = 0; TimestampTz now; --- 662,668 void pgstat_report_stat(bool force) { ! static const PgStat_TableCounts all_zeroes = {0,0,0,0,0,0,0,0,0,0,0}; static TimestampTz last_report = 0; TimestampTz now; *** *** 795,802 static void pgstat_send_funcstats(void) { ! /* we assume this inits to all zeroes: */ ! static const PgStat_FunctionCounts all_zeroes; PgStat_MsgFuncstat msg; PgStat_BackendFunctionEntry *entry; --- 794,800 static void pgstat_send_funcstats(void) { ! static const PgStat_FunctionCounts all_zeroes = {0,{0,0},{0,0}}; PgStat_MsgFuncstat msg; PgStat_BackendFunctionEntry *entry; *** *** 2548,2555 void pgstat_send_bgwriter(void) { ! /* We assume this initializes to zeroes */ ! static const PgStat_MsgBgWriter all_zeroes; /* * This function can be called even if nothing at all has happened. In --- 2546,2552 void pgstat_send_bgwriter(void) { ! static const PgStat_MsgBgWriter all_zeroes = { {0,0},0,0,0,0,0,0,0}; /* * This function can be called even if nothing at all has happened. In diff -Nrc pgsql.orig.8fc4f032818a/src/bin/psql/tab-complete.c pgsql.orig/src/bin/psql/tab-complete.c *** pgsql.orig.8fc4f032818a/src/bin/psql/tab-complete.c 2009-05-28 11:09:24.890322342 +0200 --- pgsql.orig/src/bin/psql/tab-complete.c 2009-05-28 11:09:24.894581639 +0200 *** *** 557,563 /* Forward declaration of functions */ ! static char **psql_completion(char *text, int start, int end); static char *create_command_generator(const char *text, int state); static char *drop_command_generator(const char *text, int state); static char *complete_from_query(const char *text, int state); --- 557,563 /* Forward declaration of functions */ ! static char **psql_completion(const char *text, int start, int end); static char *create_command_generator(const char *text, int state); static char *drop_command_generator(const cha
Re: [HACKERS] problem with plural-forms
Here is output of: for FILE in `find . -name *.po`;do LC_ALL=C msgfmt -v -o /dev/null $FILE 2>> msgfmt.txt; done Zdenek Peter Eisentraut píše v st 27. 05. 2009 v 23:08 +0300: > On Monday 25 May 2009 19:11:24 Zdenek Kotala wrote: > > The problem here is (1 row) instead of (%lu row). When I run msgfmt > > without -v everything works fine but I think we should fixed it (there > > are more occurrences of this issue). > > I don't think we can find all these occurrences without the Solaris version > of > msgfmt. So please send a complete error log over all files, or better yet a > patch. Processing file "./src/bin/initdb/po/ja.po"... GNU PO file found. Generating the MO file in the GNU MO format. Processing file "./src/bin/initdb/po/ja.po"... 129 translated message(s), 0 fuzzy translation(s), 0 untranslated message(s). Processing file "./src/bin/initdb/po/sl.po"... GNU PO file found. Generating the MO file in the GNU MO format. Processing file "./src/bin/initdb/po/sl.po"... 99 translated message(s), 0 fuzzy translation(s), 0 untranslated message(s). Processing file "./src/bin/initdb/po/es.po"... GNU PO file found. Generating the MO file in the GNU MO format. Processing file "./src/bin/initdb/po/es.po"... 131 translated message(s), 0 fuzzy translation(s), 0 untranslated message(s). Processing file "./src/bin/initdb/po/ro.po"... GNU PO file found. Generating the MO file in the GNU MO format. Processing file "./src/bin/initdb/po/ro.po"... 107 translated message(s), 0 fuzzy translation(s), 1 untranslated message(s). Processing file "./src/bin/initdb/po/sk.po"... GNU PO file found. Generating the MO file in the GNU MO format. Processing file "./src/bin/initdb/po/sk.po"... 82 translated message(s), 0 fuzzy translation(s), 0 untranslated message(s). Processing file "./src/bin/initdb/po/pt_BR.po"... GNU PO file found. Generating the MO file in the GNU MO format. Processing file "./src/bin/initdb/po/pt_BR.po"... 131 translated message(s), 0 fuzzy translation(s), 0 untranslated message(s). Processing file "./src/bin/initdb/po/de.po"... GNU PO file found. Generating the MO file in the GNU MO format. Processing file "./src/bin/initdb/po/de.po"... 131 translated message(s), 0 fuzzy translation(s), 0 untranslated message(s). Processing file "./src/bin/initdb/po/cs.po"... GNU PO file found. Generating the MO file in the GNU MO format. Processing file "./src/bin/initdb/po/cs.po"... 83 translated message(s), 0 fuzzy translation(s), 0 untranslated message(s). Processing file "./src/bin/initdb/po/ta.po"... GNU PO file found. Generating the MO file in the GNU MO format. Processing file "./src/bin/initdb/po/ta.po"... 123 translated message(s), 0 fuzzy translation(s), 0 untranslated message(s). Processing file "./src/bin/initdb/po/sv.po"... GNU PO file found. Generating the MO file in the GNU MO format. Processing file "./src/bin/initdb/po/sv.po"... 126 translated message(s), 0 fuzzy translation(s), 0 untranslated message(s). Processing file "./src/bin/initdb/po/it.po"... GNU PO file found. Generating the MO file in the GNU MO format. Processing file "./src/bin/initdb/po/it.po"... 92 translated message(s), 34 fuzzy translation(s), 0 untranslated message(s). Processing file "./src/bin/initdb/po/ru.po"... GNU PO file found. Generating the MO file in the GNU MO format. Processing file "./src/bin/initdb/po/ru.po"... 128 translated message(s), 0 fuzzy translation(s), 0 untranslated message(s). Processing file "./src/bin/initdb/po/pl.po"... GNU PO file found. Generating the MO file in the GNU MO format. Processing file "./src/bin/initdb/po/pl.po"... 96 translated message(s), 0 fuzzy translation(s), 0 untranslated message(s). Processing file "./src/bin/initdb/po/zh_TW.po"... GNU PO file found. Generating the MO file in the GNU MO format. Processing file "./src/bin/initdb/po/zh_TW.po"... 96 translated message(s), 0 fuzzy translation(s), 0 untranslated message(s). Processing file "./src/bin/initdb/po/tr.po"... GNU PO file found. Generating the MO file in the GNU MO format. Processing file "./src/bin/initdb/po/tr.po"... 131 translated message(s), 0 fuzzy translation(s), 0 untranslated message(s). Processing file "./src/bin/initdb/po/zh_CN.po"... GNU PO file found. Generating the MO file in the GNU MO format. Processing file "./src/bin/initdb/po/zh_CN.po"... 96 translated message(s), 0 fuzzy translation(s), 0 untranslated message(s). Processing file "./src/bin/initdb/po/fr.po"... GNU PO file found. Generating the MO file in the GNU MO format. Processing file "./src/bin/initdb/po/fr.po"... 131 translated message(s), 0 fuzzy translation(s), 0 unt
Re: [HACKERS] problem with plural-forms
Peter Eisentraut píše v út 26. 05. 2009 v 13:39 +0300: > Of course the concrete example that you show doesn't actually take advantage > of this, so if it is important to you, please send a patch to fix it. Fix attached. I found only two problems, both in psql. I did not fix .po files. Is necessary to fix them manually or do you regenerate files? thanks Zdenek diff -r 7a4dd8cda3b6 src/bin/psql/describe.c --- a/src/bin/psql/describe.c Wed May 27 12:28:55 2009 + +++ b/src/bin/psql/describe.c Wed May 27 21:58:15 2009 +0200 @@ -2036,7 +2036,7 @@ if (conns == 0) appendPQExpBuffer(&buf, _("No connections")); else -appendPQExpBuffer(&buf, ngettext("1 connection", "%d connections", conns), conns); +appendPQExpBuffer(&buf, ngettext("%d connection", "%d connections", conns), conns); } attr[i] = pg_strdup(buf.data); diff -r 7a4dd8cda3b6 src/bin/psql/print.c --- a/src/bin/psql/print.c Wed May 27 12:28:55 2009 + +++ b/src/bin/psql/print.c Wed May 27 21:58:15 2009 +0200 @@ -2348,7 +2348,7 @@ char default_footer[100]; total_records = opt->topt.prior_records + cont.nrows; - snprintf(default_footer, 100, ngettext("(1 row)", "(%lu rows)", total_records), total_records); + snprintf(default_footer, 100, ngettext("(%lu row)", "(%lu rows)", total_records), total_records); printTableAddFooter(&cont, default_footer); } -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCH] cleanup hashindex for pg_migrator hashindex compat mode (for 8.4)
Tom Lane píše v po 25. 05. 2009 v 13:07 -0400: > Zdenek Kotala writes: > > Tom Lane píše v ne 24. 05. 2009 v 18:46 -0400: > >> In any case, the barriers to implementing 8.3-style hash indexes in 8.4 > >> are pretty huge: you'd need to duplicate not only the hash AM code, but > >> also all the hash functions, and therefore all of the hash pg_amop and > >> pg_amproc entries. > > > I'm not sure if I need duplicate functions. Generally yes but It seems > > to me that hash index does not changed functions behavior and they could > > be shared at this moment. > > No, the behavior of the hash functions themselves changed during 8.4. > Twice, even: hmm, I'm missed it. :( > So as far as I can see, you need completely separate copies of both > hash_any() and the SQL-level functions that call it. I'm not really > seeing that the proposed refactoring makes this any easier. You might > as well just copy-and-paste all that old code into a separate set of > files, and not worry about what is in access/hash.h. Yeah, in this case everything have to be duplicated which is not big deal in comparison to do same amount of work for GIN. Then I can start with GIN. The advantage of refactoring is then only nicer code. thanks Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] problem with plural-forms
Peter Eisentraut píše v út 26. 05. 2009 v 13:39 +0300: > On Monday 25 May 2009 19:11:24 Zdenek Kotala wrote: > > > > The problem here is (1 row) instead of (%lu row). When I run msgfmt > > without -v everything works fine but I think we should fixed it (there > > are more occurrences of this issue). > > GNU gettext accepts this, and in fact the GNU gettext documentation > explicitly > points out that this allowed: > > """ > In the English singular case, the number - always 1 - can be > replaced with "one": > > printf (ngettext ("One file removed", "%d files removed", n), n); > > This works because the `printf' function discards excess arguments > that are not consumed by the format string. > """ Yeah, I check also printf specification and it is allowed. > One might consider this better style (English style, not C style) in some > contexts. > > Of course the concrete example that you show doesn't actually take advantage > of this, so if it is important to you, please send a patch to fix it. It is not a big issue, because it works without -v but I prefer to fix it. I will send a patch. I also sended question to i18n solaris group if it is supported on solaris. thanks Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] problem with plural-forms
I tried to run msgfmt -v ... on solaris and I got following error: Processing file "psql-cs.po"... GNU PO file found. Generating the MO file in the GNU MO format. Processing file "psql-cs.po"... Lines 1311, 1312 (psql-cs.po): incompatible printf-format. 0 format specifier(s) in "msgid", but 1 format specifier(s) in "msgstr". ... ... Problem is in: #: print.c:2351 #, c-format msgid "(1 row)" msgid_plural "(%lu rows)" msgstr[0] "(%lu řádka)" msgstr[1] "(%lu řádky)" msgstr[2] "(%lu řádek)" The problem here is (1 row) instead of (%lu row). When I run msgfmt without -v everything works fine but I think we should fixed it (there are more occurrences of this issue). Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCH] cleanup hashindex for pg_migrator hashindex compat mode (for 8.4)
Tom Lane píše v ne 24. 05. 2009 v 18:46 -0400: > Kenneth Marshall writes: > > On Sat, May 23, 2009 at 02:52:49PM -0400, Zdenek Kotala wrote: > >>> Attached patch cleanups hash index headers to allow compile hasham for > >>> 8.3 version. It helps to improve pg_migrator with capability to migrate > >>> database with hash index without reindexing. > > > How does that work with the updated hash functions without a reindex? > > I looked at this patch and I don't see how it helps pg_migrator at all. > It's just pushing some code and function declarations around. The main important thing is move hash_any/hash_uint32 function from hash am. It should reduce amount of duplicated code necessary for old hash index implementation. Rest is only rearrangement and cleanup. > The rearrangement might be marginally nicer from a code beautification > point of view --- right now we're a bit inconsistent about whether > datatype-specific hash functions live in hashfunc.c or in the datatype's > utils/adt/ file. I personally prefer to keep it in type definition. AM should be independent on types which are installed and data type code should be self contained. > But I'm not sure that removing hashfunc.c altogether is > an appropriatera solution to that, not least because of the loss of CVS > history for the functions. I'd be inclined to leave the core hash_any() > code where it is, if not all of these functions altogether. Until we will have better version control system, hashfunc.c can stay here, but what I need is hashfunc.h. Minimalistic version of this patch is to create hashfuct.h and modified related #include in C and header files. > What does seem useful is to refactor the headers so that datatype hash > functions don't need to include all of the AM's implementation details. > But this patch seems to do both more and less than that --- I don't > think it's gotten rid of all external #includes of access/hash.h, and > in any case moving the function code is not necessary to that goal. Agree, I will prepare minimalistic version with hashfunc.h only. > In any case, the barriers to implementing 8.3-style hash indexes in 8.4 > are pretty huge: you'd need to duplicate not only the hash AM code, but > also all the hash functions, and therefore all of the hash pg_amop and > pg_amproc entries. I'm not sure if I need duplicate functions. Generally yes but It seems to me that hash index does not changed functions behavior and they could be shared at this moment. > Given the close-to-zero usefulness of hash indexes > in production installations, I don't think it's worth the trouble. It > would be much more helpful to look into supporting 8.3-compatible GIN > indexes. Agree, I wanted to quickly verify function naming collision problem and HASH index seems to me better candidate for this basic test. GIN has priority. Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCH] Compiler warning cleanup
Peter Eisentraut píše v ne 24. 05. 2009 v 00:40 +0300: > I think this is not the best way to do it, because it will confuse pgindent > and editors and such. The DATA() macros in include/catalog have this solved; > see include/catalog/genbki.sh. Attached variant with faked extern without ; which is what genbki.sh uses. Zdenek diff -Nrc pgsql.orig.9bee6bd68ed8/src/backend/utils/adt/tsquery_op.c pgsql.orig/src/backend/utils/adt/tsquery_op.c *** pgsql.orig.9bee6bd68ed8/src/backend/utils/adt/tsquery_op.c 2009-05-25 16:45:34.594747865 +0200 --- pgsql.orig/src/backend/utils/adt/tsquery_op.c 2009-05-25 16:45:34.599072981 +0200 *** *** 188,194 PG_FREE_IF_COPY(b,1); \ \ PG_RETURN_BOOL( CONDITION );\ ! } CMPFUNC(tsquery_lt, res < 0); CMPFUNC(tsquery_le, res <= 0); --- 188,196 PG_FREE_IF_COPY(b,1); \ \ PG_RETURN_BOOL( CONDITION );\ ! }\ ! /* keep compiler quiet - no extra ; */ \ ! extern int no_such_variable CMPFUNC(tsquery_lt, res < 0); CMPFUNC(tsquery_le, res <= 0); diff -Nrc pgsql.orig.9bee6bd68ed8/src/backend/utils/adt/tsvector_op.c pgsql.orig/src/backend/utils/adt/tsvector_op.c *** pgsql.orig.9bee6bd68ed8/src/backend/utils/adt/tsvector_op.c 2009-05-25 16:45:34.596082053 +0200 --- pgsql.orig/src/backend/utils/adt/tsvector_op.c 2009-05-25 16:45:34.599263998 +0200 *** *** 172,178 PG_FREE_IF_COPY(a,0);\ PG_FREE_IF_COPY(b,1);\ PG_RETURN_##ret( res action 0 ); \ ! } TSVECTORCMPFUNC(lt, <, BOOL); TSVECTORCMPFUNC(le, <=, BOOL); --- 172,180 PG_FREE_IF_COPY(a,0);\ PG_FREE_IF_COPY(b,1);\ PG_RETURN_##ret( res action 0 ); \ ! } \ ! /* keep compiler quiet - no extra ; */ \ ! extern int no_such_variable TSVECTORCMPFUNC(lt, <, BOOL); TSVECTORCMPFUNC(le, <=, BOOL); -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] psql is broken in 8.4
Heikki Linnakangas píše v čt 21. 05. 2009 v 16:53 +0300: > Zdenek Kotala wrote: > > > Another problem is with resultset. When I run for example following > > command I got this output: > > > > postgres=# select oid from pg_am; > > oid > > -- > > 403 > > 405 > > 783 > > 2742 > > > > That's odd. Work for me. Can you try to debug that, please? > We looked on it with Peter and it is problem with plural translation of rows for Czech language. I'm going to fix it. Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCH] cleanup hashindex for pg_migrator hashindex compat mode (for 8.4)
I forgot to fix contrib. Updated patch attached. Zdenek Zdenek Kotala píše v pá 22. 05. 2009 v 16:23 -0400: > Attached patch cleanups hash index headers to allow compile hasham for > 8.3 version. It helps to improve pg_migrator with capability to migrate > database with hash index without reindexing. > > I discussed this patch year ago with Alvaro when we tried to cleanup > include bloating problem. It should reduce also number of including. > > The main point is that hash functions for datatypes are now in related > data files in utils/adt directory. hash_any() and hash_uint32 it now in > utils/hashfunc.c. > > It would be nice to have this in 8.4 because it allows to test index > migration functionality. > > Thanks Zdenek > hash_02.patch.gz Description: GNU Zip compressed data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] [PATCH] Compiler warning cleanup
Attached patch fixes following sun studio compiler warning: "tsquery_op.c", line 193: warning: syntax error: empty declaration "tsquery_op.c", line 194: warning: syntax error: empty declaration "tsquery_op.c", line 195: warning: syntax error: empty declaration "tsquery_op.c", line 196: warning: syntax error: empty declaration "tsquery_op.c", line 197: warning: syntax error: empty declaration "tsquery_op.c", line 198: warning: syntax error: empty declaration "tsvector_op.c", line 177: warning: syntax error: empty declaration "tsvector_op.c", line 178: warning: syntax error: empty declaration "tsvector_op.c", line 179: warning: syntax error: empty declaration "tsvector_op.c", line 180: warning: syntax error: empty declaration "tsvector_op.c", line 181: warning: syntax error: empty declaration "tsvector_op.c", line 182: warning: syntax error: empty declaration "tsvector_op.c", line 183: warning: syntax error: empty declaration Thanks Zdenek diff -Nrc pgsql.orig.5d4d60e3a557/src/backend/utils/adt/tsquery_op.c pgsql.orig/src/backend/utils/adt/tsquery_op.c *** pgsql.orig.5d4d60e3a557/src/backend/utils/adt/tsquery_op.c 2009-05-22 17:00:22.081796110 -0400 --- pgsql.orig/src/backend/utils/adt/tsquery_op.c 2009-05-22 17:00:22.086155307 -0400 *** *** 190,201 PG_RETURN_BOOL( CONDITION );\ } ! CMPFUNC(tsquery_lt, res < 0); ! CMPFUNC(tsquery_le, res <= 0); ! CMPFUNC(tsquery_eq, res == 0); ! CMPFUNC(tsquery_ge, res >= 0); ! CMPFUNC(tsquery_gt, res > 0); ! CMPFUNC(tsquery_ne, res != 0); TSQuerySign makeTSQuerySign(TSQuery a) --- 190,201 PG_RETURN_BOOL( CONDITION );\ } ! CMPFUNC(tsquery_lt, res < 0) ! CMPFUNC(tsquery_le, res <= 0) ! CMPFUNC(tsquery_eq, res == 0) ! CMPFUNC(tsquery_ge, res >= 0) ! CMPFUNC(tsquery_gt, res > 0) ! CMPFUNC(tsquery_ne, res != 0) TSQuerySign makeTSQuerySign(TSQuery a) diff -Nrc pgsql.orig.5d4d60e3a557/src/backend/utils/adt/tsvector_op.c pgsql.orig/src/backend/utils/adt/tsvector_op.c *** pgsql.orig.5d4d60e3a557/src/backend/utils/adt/tsvector_op.c 2009-05-22 17:00:22.083116270 -0400 --- pgsql.orig/src/backend/utils/adt/tsvector_op.c 2009-05-22 17:00:22.086348815 -0400 *** *** 174,186 PG_RETURN_##ret( res action 0 ); \ } ! TSVECTORCMPFUNC(lt, <, BOOL); ! TSVECTORCMPFUNC(le, <=, BOOL); ! TSVECTORCMPFUNC(eq, ==, BOOL); ! TSVECTORCMPFUNC(ge, >=, BOOL); ! TSVECTORCMPFUNC(gt, >, BOOL); ! TSVECTORCMPFUNC(ne, !=, BOOL); ! TSVECTORCMPFUNC(cmp, +, INT32); Datum tsvector_strip(PG_FUNCTION_ARGS) --- 174,186 PG_RETURN_##ret( res action 0 ); \ } ! TSVECTORCMPFUNC(lt, <, BOOL) ! TSVECTORCMPFUNC(le, <=, BOOL) ! TSVECTORCMPFUNC(eq, ==, BOOL) ! TSVECTORCMPFUNC(ge, >=, BOOL) ! TSVECTORCMPFUNC(gt, >, BOOL) ! TSVECTORCMPFUNC(ne, !=, BOOL) ! TSVECTORCMPFUNC(cmp, +, INT32) Datum tsvector_strip(PG_FUNCTION_ARGS) -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] [PATCH] cleanup hashindex for pg_migrator hashindex compat mode (for 8.4)
Attached patch cleanups hash index headers to allow compile hasham for 8.3 version. It helps to improve pg_migrator with capability to migrate database with hash index without reindexing. I discussed this patch year ago with Alvaro when we tried to cleanup include bloating problem. It should reduce also number of including. The main point is that hash functions for datatypes are now in related data files in utils/adt directory. hash_any() and hash_uint32 it now in utils/hashfunc.c. It would be nice to have this in 8.4 because it allows to test index migration functionality. Thanks Zdenek diff -Nrc pgsql_indexcompat.5d4d60e3a557/src/backend/access/hash/hashfunc.c pgsql_indexcompat/src/backend/access/hash/hashfunc.c *** pgsql_indexcompat.5d4d60e3a557/src/backend/access/hash/hashfunc.c 2009-05-22 15:56:34.409314434 -0400 --- pgsql_indexcompat/src/backend/access/hash/hashfunc.c 1969-12-31 19:00:00.0 -0500 *** *** 1,528 - /*- - * - * hashfunc.c - * Support functions for hash access method. - * - * Portions Copyright (c) 1996-2009, PostgreSQL Global Development Group - * Portions Copyright (c) 1994, Regents of the University of California - * - * - * IDENTIFICATION - * $PostgreSQL: pgsql/src/backend/access/hash/hashfunc.c,v 1.57 2009/01/01 17:23:35 momjian Exp $ - * - * NOTES - * These functions are stored in pg_amproc. For each operator class - * defined for hash indexes, they compute the hash value of the argument. - * - * Additional hash functions appear in /utils/adt/ files for various - * specialized datatypes. - * - * It is expected that every bit of a hash function's 32-bit result is - * as random as every other; failure to ensure this is likely to lead - * to poor performance of hash joins, for example. In most cases a hash - * function should use hash_any() or its variant hash_uint32(). - *- - */ - - #include "postgres.h" - - #include "access/hash.h" - - - /* Note: this is used for both "char" and boolean datatypes */ - Datum - hashchar(PG_FUNCTION_ARGS) - { - return hash_uint32((int32) PG_GETARG_CHAR(0)); - } - - Datum - hashint2(PG_FUNCTION_ARGS) - { - return hash_uint32((int32) PG_GETARG_INT16(0)); - } - - Datum - hashint4(PG_FUNCTION_ARGS) - { - return hash_uint32(PG_GETARG_INT32(0)); - } - - Datum - hashint8(PG_FUNCTION_ARGS) - { - /* - * The idea here is to produce a hash value compatible with the values - * produced by hashint4 and hashint2 for logically equal inputs; this is - * necessary to support cross-type hash joins across these input types. - * Since all three types are signed, we can xor the high half of the int8 - * value if the sign is positive, or the complement of the high half when - * the sign is negative. - */ - #ifndef INT64_IS_BUSTED - int64 val = PG_GETARG_INT64(0); - uint32 lohalf = (uint32) val; - uint32 hihalf = (uint32) (val >> 32); - - lohalf ^= (val >= 0) ? hihalf : ~hihalf; - - return hash_uint32(lohalf); - #else - /* here if we can't count on "x >> 32" to work sanely */ - return hash_uint32((int32) PG_GETARG_INT64(0)); - #endif - } - - Datum - hashoid(PG_FUNCTION_ARGS) - { - return hash_uint32((uint32) PG_GETARG_OID(0)); - } - - Datum - hashenum(PG_FUNCTION_ARGS) - { - return hash_uint32((uint32) PG_GETARG_OID(0)); - } - - Datum - hashfloat4(PG_FUNCTION_ARGS) - { - float4 key = PG_GETARG_FLOAT4(0); - float8 key8; - - /* - * On IEEE-float machines, minus zero and zero have different bit patterns - * but should compare as equal. We must ensure that they have the same - * hash value, which is most reliably done this way: - */ - if (key == (float4) 0) - PG_RETURN_UINT32(0); - - /* - * To support cross-type hashing of float8 and float4, we want to return - * the same hash value hashfloat8 would produce for an equal float8 value. - * So, widen the value to float8 and hash that. (We must do this rather - * than have hashfloat8 try to narrow its value to float4; that could fail - * on overflow.) - */ - key8 = key; - - return hash_any((unsigned char *) &key8, sizeof(key8)); - } - - Datum - hashfloat8(PG_FUNCTION_ARGS) - { - float8 key = PG_GETARG_FLOAT8(0); - - /* - * On IEEE-float machines, minus zero and zero have different bit patterns - * but should compare as equal. We must ensure that they have the same - * hash value, which is most reliably done this way: - */ - if (key == (float8) 0) - PG_RETURN_UINT32(0); - - return hash_any((unsigned char *) &key, sizeof(key)); - } - - Datum - hashoidvector(PG_FUNCTION_ARGS) - { - oidvector *key = (oidvector *) PG_GETARG_POINTER(0); - - return hash_any((unsigned char *) key->values, key->dim1 * sizeof(Oid)); - } - - Datum - hashint2vector(PG_FUNCTION_ARGS) - { - int2vector *key = (int2vector *) PG_GETARG
[HACKERS] integer overflow in reloption.h
When I compile postgresql now I get following message: "../../../src/include/access/reloptions.h", line 45: warning: integer overflow detected: op "<<" The problem is on the following lines typedef enum relopt_kind { ... RELOPT_KIND_MAX = (1 << 31) } enum is int datatype and 1 << 31 == -2147483648. It is reason why compiler (sun studio) complains. Is possible to change it to 1 << 30 to stop compiler generates noise? Thanks Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] psql is broken in 8.4
Heikki Linnakangas píše v čt 21. 05. 2009 v 16:53 +0300: > Zdenek Kotala wrote: > > last version of psql is broken: > > > > psql (8.4beta1, server 8.3.7) > > WARNING: psql version 8.4, server version 8.3. > > Some psql features might not work. > > Type "help" for help. > > > > postgres=# \d test > > ERROR: syntax error at or near "," > > LINE 1: ...index, relhasrules, reltriggers <> 0, relhasoids, ,'', > > relta... > > ^ > > It should work. I think describeTableDetails() does not have something > > like: if (pset.sversion >= 80100) ... > > I fixed this a while ago: > > revision 1.211 > date: 2009/05/04 17:31:35; author: heikki; state: Exp; lines: +2 -2 > Fix the query used for \d against 8.2 and 8.3 servers. > > Should work with a fresh checkout. Yeah, thanks. It works with beta2. > > Another problem is with resultset. When I run for example following > > command I got this output: > > > > postgres=# select oid from pg_am; > > oid > > -- > > 403 > > 405 > > 783 > > 2742 > > > > That's odd. Work for me. Can you try to debug that, please? I will look on it. Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers