Re: [HACKERS] missing $PostgreSQL:$
On 5/17/08, Andrew Dunstan [EMAIL PROTECTED] wrote: Tom Lane wrote: Andrew Dunstan [EMAIL PROTECTED] writes: second pass. There are 130 files in this list. Offhand I'd say the vast majority should have markers. Yeah, that list looks reasonably sane. The main thing I was worried about was not plastering PostgreSQL markers on files that are simply imported from an upstream source, like the zic database and the libstemmer files. The following files have markers from upstream systems. I'm not sure that means we don't want to add $PostgreSQL$ markers, though. Thoughts? ./src/port/strlcat.c:/* $OpenBSD: strlcat.c,v 1.13 2005/08/08 08:05:37 espie Exp $ */ ./src/backend/port/darwin/system.c: * $FreeBSD: src/lib/libc/stdlib/system.c,v 1.6 2000/03/16 02:14:41 jasone Exp $ ./contrib/pgcrypto/rijndael.h:/*$OpenBSD: rijndael.h,v 1.3 2001/05/09 23:01:32 markus Exp $ */ You can add, but please keep upstream marker. -- marko -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [PATCHES] [HACKERS] TRUNCATE TABLE with IDENTITY
On Fri, 2008-05-16 at 21:50 -0400, Tom Lane wrote: Neil Conway [EMAIL PROTECTED] writes: Ugh. The fact that the RESTART IDENTITY part of TRUNCATE is non-transactional is a pretty unsightly wort. Actually, I agree. Shall we just revert that feature? The ALTER SEQUENCE part of this patch is clean and useful, but I'm less than enamored of the TRUNCATE part. Perhaps, but we should also take into account that TRUNCATE is not and never will be MVCC compliant, so its not something you'd expect to run except as a maintenance action. If we do keep it, I would suggest that we add a WARNING so that the effects are clearly recorded. -- Simon Riggs www.2ndQuadrant.com PostgreSQL Training, Services and Support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] windows builds hanging on regression checks
Something committed in about the last 24 hours or so has caused my Windows buildfarm members to hang completely while running the regression tests. It's happened for both MinGW and MSVC builds. To get the buildfarm run to complete in some fashion, I had to kill 2 psql processes, and then 2 dwwin processes. The result was this: http://www.pgbuildfarm.org/cgi-bin/show_log.pl?nm=dawn_batdt=2008-05-17%2012:28:51 The most likely candidate looks to me like this one :Implement error checking for pthreads calls in thread-safe mode. but that's just a guess. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Adding variables for segment_size, wal_segment_size and block sizes
Bernd Helmle wrote: segment_size: Reports heap segment size wal_segment_size: Reports wal segment size block_size: Available yet wal_block_size: wal block size +1. We already have block_size in GUC. -- Euler Taveira de Oliveira http://www.timbira.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [PATCHES] [HACKERS] TRUNCATE TABLE with IDENTITY
Simon Riggs [EMAIL PROTECTED] writes:
On Fri, 2008-05-16 at 21:50 -0400, Tom Lane wrote:
Actually, I agree. Shall we just revert that feature?
Perhaps, but we should also take into account that TRUNCATE is not and
never will be MVCC compliant, so its not something you'd expect to run
except as a maintenance action.
Good point. I had a couple of further thoughts this morning:
1. The case Neil is worried about is something like
BEGIN;
TRUNCATE TABLE foo RESTART IDENTITY;
COPY foo FROM ...;
COMMIT;
If the COPY fails partway through, the old table contents are restored,
but the sequences are not. However removing RESTART IDENTITY will not
remove the hazard, because there is no difference between this and
BEGIN;
TRUNCATE TABLE foo;
SELECT setval('foo_id', 1);
COPY foo FROM ...;
COMMIT;
other than the latter adding a little extra chance for pilot error in
resetting the wrong sequence. So if we revert the patch we haven't
accomplished much except to take away an opportunity to point out the
risk. I vote for leaving the patch in and rewriting the warning
to point out this risk.
2. I had first dismissed Neil's idea of transactional sequence updates
as impossible, but on second look it could be done. Suppose RESTART
IDENTITY does this for each sequence;
* obtain AccessExclusiveLock;
* assign a new relfilenode;
* insert a sequence row with all parameters copied except
last_value copies start_value;
* hold AccessExclusiveLock till commit.
IOW just like truncate-and-reload, but for a sequence. Within the
current backend, subsequent operations see the new sequence values.
If the transaction rolls back, the old sequence relfilenode is still
there and untouched.
It's slightly annoying to need to lock out other backends' nextval
operations, but for the use-case of TRUNCATE this doesn't seem
like it's really much of a problem.
I'm not sure if it'd be worth exposing this behavior as a separate
user-visible command (CREATE OR REPLACE SEQUENCE, maybe?), but it seems
worth doing to make TRUNCATE-and-reload less of a foot gun.
So what I think we should do is leave the patch there, revise the
warning per Neil's complaint, and add a TODO item to reimplement RESTART
IDENTITY transactionally.
regards, tom lane
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [PATCHES] [HACKERS] TRUNCATE TABLE with IDENTITY
On Sat, 2008-05-17 at 12:04 -0400, Tom Lane wrote: So what I think we should do is leave the patch there, revise the warning per Neil's complaint, and add a TODO item to reimplement RESTART IDENTITY transactionally. Sounds good. -- Simon Riggs www.2ndQuadrant.com PostgreSQL Training, Services and Support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] What in the world is happening on spoonbill?
Buildfarm member spoonbill's last four HEAD builds have all failed in the same utterly bizarre way. It looks like about half of the test results files got truncated at random places --- no errors, no nothing, the file just ends early. What's up with that? regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] What in the world is happening on spoonbill?
Tom Lane wrote: Buildfarm member spoonbill's last four HEAD builds have all failed in the same utterly bizarre way. It looks like about half of the test results files got truncated at random places --- no errors, no nothing, the file just ends early. What's up with that? psql is coredumping: #0 0x00112ea0 in print_aligned_text (cont=0x8a90, fout=0x74c0c8) at print.c:664 664 if (width 0 width_wrap[i] (gdb) bt #0 0x00112ea0 in print_aligned_text (cont=0x8a90, fout=0x74c0c8) at print.c:664 #1 0x00116e40 in printTable (cont=0x8a90, fout=0x74c0c8, flog=0x0) at print.c:2248 #2 0x001170e0 in printQuery (result=0x41a44800, opt=0x4, fout=0x74c0c8, flog=0x0) at print.c:2365 #3 0x00107dc0 in PrintQueryTuples (results=0x41a44800) at common.c:605 #4 0x001080b0 in PrintQueryResults (results=0x41a44800) at common.c:710 #5 0x00108508 in SendQuery (query=0x4f4cd600 select * from def_test;) at common.c:870 #6 0x0010c5f4 in MainLoop (source=0x74c030) at mainloop.c:242 #7 0x0010eb40 in main (argc=6, argv=0x91f8) at startup.c:347 which points the figner towards the psql changes ... Stefan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] What in the world is happening on spoonbill?
Stefan Kaltenbrunner [EMAIL PROTECTED] writes: psql is coredumping: Huh. I wonder why it's only happening on that one machine. Is there anything particularly unusual about datatype sizes or alignment rules on that platform? regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] What in the world is happening on spoonbill?
Tom Lane wrote: Stefan Kaltenbrunner [EMAIL PROTECTED] writes: psql is coredumping: Huh. I wonder why it's only happening on that one machine. Is there anything particularly unusual about datatype sizes or alignment rules on that platform? hmm well it is a 64bit Sparc box running OpenBSD which is a tad unusual in itself. But if i had to guess this more likely caused by the special malloc flags used on spoonbill (FGJPZ) - per your recommendations in: http://archives.postgresql.org/pgsql-hackers/2005-06/msg00828.php docs at: http://www.openbsd.org/cgi-bin/man.cgi?query=malloc.confapropos=0sektion=0manpath=OpenBSD+4.2arch=sparc64format=html Stefan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] What in the world is happening on spoonbill?
Tom Lane wrote: Stefan Kaltenbrunner [EMAIL PROTECTED] writes: psql is coredumping: Huh. I wonder why it's only happening on that one machine. Is there anything particularly unusual about datatype sizes or alignment rules on that platform? hmm actually - the windows buildfarm failures/issues andrew reported might be the same issue from looking at his report and the failure after killing psql ... Stefan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] What in the world is happening on spoonbill?
Stefan Kaltenbrunner [EMAIL PROTECTED] writes: Tom Lane wrote: Huh. I wonder why it's only happening on that one machine. But if i had to guess this more likely caused by the special malloc flags used on spoonbill (FGJPZ) - per your recommendations in: Hah, yeah, that's it. The code was definitely indexing off the end of the width_wrap[] array. It's surprising that we didn't get any more-obvious failures, like bogus output formatting. Can you modify the buildfarm's description of that machine to mention the special malloc debug flags? It'd probably stop me from asking you this question again ;-) regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] What in the world is happening on spoonbill?
Stefan Kaltenbrunner [EMAIL PROTECTED] writes: psql is coredumping: BTW, this exposes a pretty nasty omission in pg_regress: it fails to say anything about a nonzero exit code from a psql child process that's running a test. Seems like wait_for_tests() ought to complain about that. Any objections? Does anyone know how to get the child process exit status on Windows? regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] What in the world is happening on spoonbill?
Tom Lane wrote: Stefan Kaltenbrunner [EMAIL PROTECTED] writes: Tom Lane wrote: Huh. I wonder why it's only happening on that one machine. But if i had to guess this more likely caused by the special malloc flags used on spoonbill (FGJPZ) - per your recommendations in: Hah, yeah, that's it. The code was definitely indexing off the end of the width_wrap[] array. It's surprising that we didn't get any more-obvious failures, like bogus output formatting. Can you modify the buildfarm's description of that machine to mention the special malloc debug flags? It'd probably stop me from asking you this question again ;-) hmm - would take somebody with SQL-level access to do this - the script to update OS/compiler related data is only partially(ie not updating all information) working... But maybe it would be nice to have some sort of notes about this buildfarm member text field that contains this information (or other stuff like this is a VM running on bar or this is really the same hardware as animal bar just with configuration baz ? Stefan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] What in the world is happening on spoonbill?
On Sat, 17 May 2008, Tom Lane wrote: Does anyone know how to get the child process exit status on Windows? GetExitCodeProcess, if you've got the process handle handy (which I assume you do, since you most likely were calling one of the WaitFor...Object family of functions. http://msdn.microsoft.com/en-us/library/ms683189(VS.85).aspx regards, tom lane -- Then a man said: Speak to us of Expectations. He then said: If a man does not see or hear the waters of the Jordan, then he should not taste the pomegranate or ply his wares in an open market. If a man would not labour in the salt and rock quarries then he should not accept of the Earth that which he refuses to give of himself. Such a man would expect a pear of a peach tree. Such a man would expect a stone to lay an egg. Such a man would expect Sears to assemble a lawnmower. -- Kehlog Albran, The Profit -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [rfc,patch] PL/Proxy in core
On Thu, 15 May 2008, Marko Kreen wrote: On 5/15/08, Josh Berkus [EMAIL PROTECTED] wrote: Has PL/proxy been tested on other OSes? FreeBSD/Solaris/Windows? It definitely works on Linux and MacOS. I've seen ports for *BSD. I think any unix-like OS-es should work fine. I've compiled it with a minor Makefile modification on Solaris and it seems to work (it passes all the unit tests but the one involving random; we might want to rethink that test so 'passes' on platforms with different implementations of random()). However, I haven't deployed it into a production environment. Somewhat unrelated, I can see use-cases for replacing the call to random() with something that allows user defined polices for RUN ON ANY. In fact, only syscalls it does on its own are gettimeofday() and poll() [or select()], otherwise it calls either core or libpq functions. So I see no reason why it shouldnt already work on Windows. The biggest portability problem thus far has been scanner.l, which at the beginning was written for flex 2.5.33+, but 2.5.4 is still pretty widespread. But this I fixed in 2.0.3. Hmm.. Now that I think about it, in my effort to remove malloc() calls in both scanner and parser I told bison to use alloca(). Is it portability concern? Easy to fix, just need to be careful not to create memleaks. -- marko -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [rfc,patch] PL/Proxy in core
Steve, I've compiled it with a minor Makefile modification on Solaris and it seems to work (it passes all the unit tests but the one involving random; we might want to rethink that test so 'passes' on platforms with different implementations of random()). However, I haven't deployed it into a production environment. Confirmed on my copy of Nevada 70 (opensolaris) as well. Looks like we're good on OS support. Now time to start playing with the semantics ... --Josh -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] What in the world is happening on spoonbill?
I wrote: BTW, this exposes a pretty nasty omission in pg_regress: it fails to say anything about a nonzero exit code from a psql child process that's running a test. Seems like wait_for_tests() ought to complain about that. Any objections? So I coded this up, and fortunately thought to try it with ecpg's tests before committing: test preproc/define ... ok test preproc/init ... ok test preproc/type ... ok test preproc/variable ... ok test preproc/whenever ... FAILED: test process exited with exit code 1 test sql/array... ok test sql/binary ... ok test sql/code100 ... ok test sql/copystdout ... ok Apparently the exit(1) is intentional in that test. We could possibly extend the syntax of regression schedule files to have a way to say what's the expected exit status, but that seems like more work than it's worth. Would it be all right to just remove the test of on error stop mode? regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Would like to sponsor implementation of MATERIALIZED VIEWS
2008/5/15 [EMAIL PROTECTED]: A more sophisticated implementation would automatically retrieve from the summary table when the main table is referenced, if possible. If this means that e.g. a query would know by itself that it could get the data from the view instead of from the main table, then we don't need this feature at the moment. Otherwise: Could anyone explain? That is exactly what it means. Nicolas -- Nicolas Barbier http://www.gnu.org/philosophy/no-word-attachments.html -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] What in the world is happening on spoonbill?
I wrote: We could possibly extend the syntax of regression schedule files to have a way to say what's the expected exit status, but that seems like more work than it's worth. Would it be all right to just remove the test of on error stop mode? What I did for the moment is just make it annotate the report, rather than treating nonzero status as a failure in itself. That will at least help with diagnosing problems. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] WIP: Pg_upgrade - page layout converter (PLC) hook
Zdenek Kotala napsal(a): snip How it works: When PLC module is loaded, then for each page which does not have native page version conversion routine is called. Buffer is mark as a dirty and upgraded page is inserted into WAL. Unfortunately, this approach does not work between layout 3 and 4. It works only for heap on platfrom with maxallign=4. The main problem is that PageHeader has been extended to 24 bytes and it requires reindexing, TOAST chunk resizing, converted tuples does not fit on page on platform where maxallign=8. I'm now working on offline conversion method. Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] What in the world is happening on spoonbill?
Tom Lane wrote: We could possibly extend the syntax of regression schedule files to have a way to say what's the expected exit status, but that seems like more work than it's worth. Would it be all right to just remove the test of on error stop mode? Woulnd't it be enough to report the exist status if a test fails, instead of requiring a certain exit status for success? -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] WIP: Pg_upgrade - page layout converter (PLC) hook
Zdenek, Unfortunately, this approach does not work between layout 3 and 4. It works only for heap on platfrom with maxallign=4. The main problem is that PageHeader has been extended to 24 bytes and it requires reindexing, TOAST chunk resizing, converted tuples does not fit on page on platform where maxallign=8. I'm now working on offline conversion method. Hmmm. I thought we determined earlier that the use case for *offline* binary conversion was rather limited. It would have to be 10x faster than dump/reload to be worthwhile. Do you think this is possible? -- --Josh Josh Berkus PostgreSQL @ Sun San Francisco -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [rfc,patch] PL/Proxy in core
On 5/17/08, Steve Singer [EMAIL PROTECTED] wrote: Somewhat unrelated, I can see use-cases for replacing the call to random() with something that allows user defined polices for RUN ON ANY. Well, thats why the RUN ON userfunc(..); exists. Also notice the function can tag more that one partition for execution. Or did you mean something else than partition selection by user defined policy? -- marko -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Problems with CVS HEAD compile
Bruce Momjian wrote: Since ecpg localization was added today, I am unable to compile src/interfaces/ecpg. I get: $ gmake -w clean gmake: Entering directory `/usr/var/local/src/gen/pgsql/CURRENT/pgsql/src/interfaces/ecpg' rm -f usage: rm [-dfiPRrW] file ... Fixed. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] ecpg localization
Euler Taveira de Oliveira wrote: I mean that you need to put locale.h and setlocale(LC_MESSAGES, ) at .pgc so you get localized messages from ecpg program. That seems perfectly reasonable. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] What in the world is happening on spoonbill?
Peter Eisentraut [EMAIL PROTECTED] writes: Woulnd't it be enough to report the exist status if a test fails, instead of requiring a certain exit status for success? What I have it doing is reporting the exit status if not zero, but it's only an annotation on the short-form output; it doesn't control whether the test is considered to have succeeded or not. I'm not very happy with that because a crash after all the expected output has been produced would not result in a report of failure --- and we have seen problems with psql crashing at exit, so this isn't an academic point. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] WIP: Pg_upgrade - page layout converter (PLC) hook
Josh Berkus napsal(a): Zdenek, Unfortunately, this approach does not work between layout 3 and 4. It works only for heap on platfrom with maxallign=4. The main problem is that PageHeader has been extended to 24 bytes and it requires reindexing, TOAST chunk resizing, converted tuples does not fit on page on platform where maxallign=8. I'm now working on offline conversion method. Hmmm. I thought we determined earlier that the use case for *offline* binary conversion was rather limited. It would have to be 10x faster than dump/reload to be worthwhile. Do you think this is possible? It is difficult to say without prototype. However, binary conversion can be performed without (or minimal) WAL logging and without sync. I think async write is a big improvement. And there is not problem convert tables in parallel mode. And of course you don't need bin-text-bin conversion. Unfortunately reindex is necessary in both cases. Zdenek -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] use of pager on Windows psql
psql's print.c contains this piece of code:
/*
* PageOutput
*
* Tests if pager is needed and returns appropriate FILE pointer.
*/
FILE *
PageOutput(int lines, unsigned short int pager)
{
/* check whether we need / can / are supposed to use pager */
if (pager
#ifndef WIN32
isatty(fileno(stdin))
isatty(fileno(stdout))
#endif
)
{
Why are we not doing the isatty tests on Windows? We can and do use
isatty on Windows elsewhere, so I'm a bit mystified about this.
In fact, it looks to me like it would be much more sensible to #include
settings.h and then simply test pset.notty for all platforms.
cheers
andrew
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Link requirements creep
I was displeased to discover just now that in a standard RPM build of PG 8.3, psql and the other basic client programs pull in libxml2 and libxslt; this creates a package dependency that should not be there by any stretch of the imagination. The reason of course is that configure puts -lxml2 -lxslt into LIBS and psql's Makefile unquestioningly links with that list. Back in the days of static linking, this didn't hurt because unreferenced libraries didn't get pulled into the executable. But it seems that at least with Linux' linker, you get a reference to every shared library mentioned in the link command. One possible answer is to put these two libraries into a special make variable, comparable to the way that libossp-uuid is being handled, and use that variable only where wanted. However this seems to be a band-aid solution; we'll inevitably have the same kind of problem again in future. Another approach we could take is to allow configure to dump everything into LIBS, and institute a policy that no executable includes LIBS verbatim. Instead every link command must do something like libpq already does: $(filter -lcrypt -ldes -lcom_err -lcrypto -lk5crypto -lkrb5 -lgssapi_krb5 -lgss -lssl -lsocket -lnsl -lresolv -lintl, $(LIBS)) to explicitly list the libraries this executable might need. This seems a lot more fail-safe, but it'd probably take awhile to get the filter lists right; and this doesn't seem like a route to a readily back-patchable solution. Thoughts, better ideas? regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] use of pager on Windows psql
Andrew Dunstan wrote:
psql's print.c contains this piece of code:
/*
* PageOutput
*
* Tests if pager is needed and returns appropriate FILE pointer.
*/
FILE *
PageOutput(int lines, unsigned short int pager)
{
/* check whether we need / can / are supposed to use pager */
if (pager
#ifndef WIN32
isatty(fileno(stdin))
isatty(fileno(stdout))
#endif
)
{
Why are we not doing the isatty tests on Windows? We can and do use
isatty on Windows elsewhere, so I'm a bit mystified about this.
Not sure why ware are not. Should we enabled that code on Win32 and see
how it works? Can you test it? Was it some MinGW limitation? I do see
isatty() being used on lots of platforms.
This is kind of odd. Ah, I bet it came from libpq's PQprint(), which I
think we had working on Win32 long before we had psql working and
perhaps I copied it from there. I don't see the Win32 checks around
isatty() anywhere else.
In fact, it looks to me like it would be much more sensible to #include
settings.h and then simply test pset.notty for all platforms.
Yes, we could do that but does the isatty() value ever change while psql
is running? When you do '\g filename' does stdout then have isatty as
false?
--
Bruce Momjian [EMAIL PROTECTED]http://momjian.us
EnterpriseDB http://enterprisedb.com
+ If your life is a hard drive, Christ can be your backup. +
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Link requirements creep
On 5/18/08, Tom Lane [EMAIL PROTECTED] wrote: I was displeased to discover just now that in a standard RPM build of PG 8.3, psql and the other basic client programs pull in libxml2 and libxslt; this creates a package dependency that should not be there by any stretch of the imagination. Thoughts, better ideas? 1. Use -Wl,--as-needed as linker flag. Portability unknown... Can be autoconfed. 2. Lets have few top-level library dependencies as make variables: LIBPQ_LDFLAGS (LIBPQ_LIBS?) LIBPQ_CFLAGS Same for READLINE, LIBXML2, etc Maybe also BACKEND_* Then each component can pick groups as it needs. Basically the default autoconf style of putting all into LIBS work well only when you have single program to build. As in Postgres build-tree we have lots of modules that want different selection of libraries to link to, this style does not fit. -- marko -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] use of pager on Windows psql
Bruce Momjian wrote:
Andrew Dunstan wrote:
psql's print.c contains this piece of code:
/*
* PageOutput
*
* Tests if pager is needed and returns appropriate FILE pointer.
*/
FILE *
PageOutput(int lines, unsigned short int pager)
{
/* check whether we need / can / are supposed to use pager */
if (pager
#ifndef WIN32
isatty(fileno(stdin))
isatty(fileno(stdout))
#endif
)
{
Why are we not doing the isatty tests on Windows? We can and do use
isatty on Windows elsewhere, so I'm a bit mystified about this.
Not sure why ware are not. Should we enabled that code on Win32 and see
how it works? Can you test it? Was it some MinGW limitation? I do see
isatty() being used on lots of platforms.
This is kind of odd. Ah, I bet it came from libpq's PQprint(), which I
think we had working on Win32 long before we had psql working and
perhaps I copied it from there. I don't see the Win32 checks around
isatty() anywhere else.
In fact, it looks to me like it would be much more sensible to #include
settings.h and then simply test pset.notty for all platforms.
Yes, we could do that but does the isatty() value ever change while psql
is running? When you do '\g filename' does stdout then have isatty as
false?
Good point. I think the best thing would just be to remove the #ifndef
WIN32 / #endif lines
cheers
andrew
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] windows builds hanging on regression checks
Andrew Dunstan [EMAIL PROTECTED] writes: Something committed in about the last 24 hours or so has caused my Windows buildfarm members to hang completely while running the regression tests. It's happened for both MinGW and MSVC builds. baiji and dawn_bat are both green now, so apparently Stefan was correct to guess that the array indexing bug in psql/print.c was the problem. I don't understand, however, how that manifested as a hang and not a core dump. Does Windows not treat bogus pointer dereference the same as everyone else does? regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] windows builds hanging on regression checks
Tom Lane wrote: Andrew Dunstan [EMAIL PROTECTED] writes: Something committed in about the last 24 hours or so has caused my Windows buildfarm members to hang completely while running the regression tests. It's happened for both MinGW and MSVC builds. baiji and dawn_bat are both green now, so apparently Stefan was correct to guess that the array indexing bug in psql/print.c was the problem. I don't understand, however, how that manifested as a hang and not a core dump. Does Windows not treat bogus pointer dereference the same as everyone else does? Yes, it does. What hung apparently was the Dr Watson error reporting tool. I think I've disabled it for everything but the OS now. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Link requirements creep
Marko Kreen [EMAIL PROTECTED] writes: On 5/18/08, Tom Lane [EMAIL PROTECTED] wrote: I was displeased to discover just now that in a standard RPM build of PG 8.3, psql and the other basic client programs pull in libxml2 and libxslt; this creates a package dependency that should not be there by any stretch of the imagination. Thoughts, better ideas? 1. Use -Wl,--as-needed as linker flag. Portability unknown... Can be autoconfed. This might actually be the best solution. OS X has a similar disease but some trolling of the ld man page suggests that -dead_strip_dylibs might work there. Taking this path would amount to assuming that all linkers we care about either have an equivalent switch or don't link unrequired libraries in the first place. I'll do some experimenting... regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Link requirements creep
Tom Lane wrote: 1. Use -Wl,--as-needed as linker flag. Portability unknown... Can be autoconfed. This might actually be the best solution. OS X has a similar disease but some trolling of the ld man page suggests that -dead_strip_dylibs might work there. Taking this path would amount to assuming that all linkers we care about either have an equivalent switch or don't link unrequired libraries in the first place. Didn't we run into problems with this before? cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Link requirements creep
Andrew Dunstan [EMAIL PROTECTED] writes: 1. Use -Wl,--as-needed as linker flag. Portability unknown... Didn't we run into problems with this before? Hm, yeah, thanks for reminding me to check the archives. We tried to do exactly this three years ago, and it crashed and burned. See http://archives.postgresql.org/pgsql-hackers/2005-10/msg01364.php The followup discussion to that was along the lines of it's too late for 8.1, but we should try it for 8.2. Seems to have slipped through the cracks though. I've been testing a new patch and do not see the problem on Fedora 8, so at least that platform's readline seems to be fixed. I find the hack Martijn proposes in the above message to be pretty ugly, so what I'm inclined to do is leave that out for now and see what failures we get. The availability of the buildfarm makes experimenting with this kind of thing a lot safer ... regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [rfc,patch] PL/Proxy in core
On Sat, 17 May 2008, Marko Kreen wrote: On 5/17/08, Steve Singer [EMAIL PROTECTED] wrote: Somewhat unrelated, I can see use-cases for replacing the call to random() with something that allows user defined polices for RUN ON ANY. Well, thats why the RUN ON userfunc(..); exists. Also notice the function can tag more that one partition for execution. Or did you mean something else than partition selection by user defined policy? I see RUN ON userfunc() as being for partitioning where the correctness requires that the query be run on the result of userfunc. I see RUN ON ANY as being for load-balancing. You might want to RUN ON ANY with a round robin balancing, or maybe consider the load of servers for doing the balancing. In the case of RUN ON ANY it seems that the database the query gets sent to doesn't matter. It might make sense for plproxy to try the next database if it can't connect to the first one it picks. You wouldn't want this for partitioning queries. If plproxy knows if you mean 'the query has to be run on these partitions' versus 'run the query on any partition, using method x to choose' then that type of things would be possible. Steve -- marko -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] New DTrace probes proposal
(Resending since it didn't work the first time. Not sure if attaching a tar file was the culprit.) I'd like to propose adding the following probes (some of which came from Simon) to 8.4. I think these probe provide very useful data. Although some of the data can be collected now, the main advantages with probes, among others, are (1) they are always available and can be enabled only when needed especially in production (2) different combinations of probes can be used together to collect interesting data. They work on OS X Leopard Solaris now, and hopefully on FreeBSD soon. Preliminary patch attached along with sample DTrace scripts. -Robert --- * Probes to measure query time query-parse-start (int, char *) query-parse-done (int, char *) query-plan-start () query-plan-done () query-execute-start () query-execute-done () query-statement-start (int, char *) query-statement-done (int, char *) * Probes to measure dirty buffer writes by the backend because bgwriter is not effective dirty-buffer-write-start (int, int, int, int) dirty-buffer-write-done (int, int, int, int) * Probes to measure physical writes from the shared buffer buffer-write-start (int, int, int, int) buffer-write-done (int, int, int, int, int) * Probes to measure reads of a relation from a particular buffer block buffer-read-start (int, int, int, int, int) buffer-read-done (int, int, int, int, int, int) * Probes to measure the effectiveness of buffer caching buffer-hit () buffer-miss () * Probes to measure I/O time because wal_buffers is too small wal-buffer-write-start () wal-buffer-write-done () * Probes to measure checkpoint stats such as running time, buffers written, xlog files added, removed, recycled, etc checkpoint-start (int) checkpoint-done (int, int, int, int, int) * Probes to measure Idle in Transaction and client/network time idle-transaction-start (int, int) idle-transaction-done () * Probes to measure sort time sort-start (int, int, int, int, int) sort-done (int, long) * Probes to determine whether or not the deadlock detector has found a deadlock deadlock-found () deadlock-notfound (int) * Probes to measure reads/writes by block numbers and relations smgr-read-start (int, int, int, int) smgr-read-end (int, int, int, int, int, int) smgr-write-start (int, int, int, int) smgr-write-end (int, int, int, int, int, int) Index: backend/access/transam/xlog.c === RCS file: /projects/cvsroot/pgsql/src/backend/access/transam/xlog.c,v retrieving revision 1.295 diff -u -3 -p -r1.295 xlog.c --- backend/access/transam/xlog.c 25 Mar 2008 22:42:42 - 1.295 +++ backend/access/transam/xlog.c 16 May 2008 18:37:22 - @@ -50,6 +50,7 @@ #include utils/builtins.h #include utils/pg_locale.h #include utils/ps_status.h +#include pg_trace.h /* File path names (all relative to $PGDATA) */ @@ -1258,12 +1259,14 @@ AdvanceXLInsertBuffer(bool new_segment) * Have to write buffers while holding insert lock. This is * not good, so only write as much as we absolutely must. */ +TRACE_POSTGRESQL_WAL_BUFFER_WRITE_START(); WriteRqst.Write = OldPageRqstPtr; WriteRqst.Flush.xlogid = 0; WriteRqst.Flush.xrecoff = 0; XLogWrite(WriteRqst, false, false); LWLockRelease(WALWriteLock); Insert-LogwrtResult = LogwrtResult; +TRACE_POSTGRESQL_WAL_BUFFER_WRITE_DONE(); } } } @@ -5665,6 +5668,8 @@ CreateCheckPoint(int flags) MemSet(CheckpointStats, 0, sizeof(CheckpointStats)); CheckpointStats.ckpt_start_t = GetCurrentTimestamp(); + TRACE_POSTGRESQL_CHECKPOINT_START(flags); + /* * Use a critical section to force system panic if we have trouble. */ @@ -5726,6 +5731,7 @@ CreateCheckPoint(int flags) LWLockRelease(WALInsertLock); LWLockRelease(CheckpointLock); END_CRIT_SECTION(); + TRACE_POSTGRESQL_CHECKPOINT_DONE(UINT_MAX,0,0,0,UINT_MAX); return; } } @@ -5945,6 +5951,11 @@ CreateCheckPoint(int flags) if (log_checkpoints) LogCheckpointEnd(); + TRACE_POSTGRESQL_CHECKPOINT_DONE(CheckpointStats.ckpt_bufs_written, +NBuffers, CheckpointStats.ckpt_segs_added, +CheckpointStats.ckpt_segs_removed, +CheckpointStats.ckpt_segs_recycled); + LWLockRelease(CheckpointLock); } Index: backend/storage/buffer/bufmgr.c === RCS file: /projects/cvsroot/pgsql/src/backend/storage/buffer/bufmgr.c,v retrieving revision 1.228 diff -u -3 -p -r1.228 bufmgr.c --- backend/storage/buffer/bufmgr.c 1 Jan 2008 19:45:51 - 1.228 +++ backend/storage/buffer/bufmgr.c 16 May 2008 18:37:22 - @@ -42,6 +42,7 @@ #include storage/smgr.h #include utils/resowner.h #include pgstat.h +#include pg_trace.h /* Note: these two macros only work on shared buffers, not local ones! */ @@ -171,6 +172,7 @@ ReadBuffer_common(Relation reln, BlockNu if (isExtend) blockNum = smgrnblocks(reln-rd_smgr); +
Re: [HACKERS] Link requirements creep
On Sat, 17 May 2008, Tom Lane wrote: I was displeased to discover just now that in a standard RPM build of PG 8.3, psql and the other basic client programs pull in libxml2 and libxslt; this creates a package dependency that should not be there by any stretch of the imagination. When we noticed this recently, my digging suggested you'll be hard pressed to have a RedHat system now without those two installed. The libxslt RPM provides necessary components for KDE, GNOME, and Sun's Java RPM. libxml2 is far more intertwined even than that. These dependencies are unpleasant technically, but I don't think the introduce any real functional creep. It would be difficult to even strip a system down to the point where these packages weren't available. -- * Greg Smith [EMAIL PROTECTED] http://www.gregsmith.com Baltimore, MD -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] What in the world is happening on spoonbill?
Tom Lane [EMAIL PROTECTED] writes: Peter Eisentraut [EMAIL PROTECTED] writes: Woulnd't it be enough to report the exist status if a test fails, instead of requiring a certain exit status for success? What I have it doing is reporting the exit status if not zero, but it's only an annotation on the short-form output; it doesn't control whether the test is considered to have succeeded or not. I'm not very happy with that because a crash after all the expected output has been produced would not result in a report of failure --- and we have seen problems with psql crashing at exit, so this isn't an academic point. It might be a bit weird but pg_regress could stick a message in the output file before it does the comparison with the expected results. -- Gregory Stark EnterpriseDB http://www.enterprisedb.com Get trained by Bruce Momjian - ask me about EnterpriseDB's PostgreSQL training! -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Link requirements creep
Greg Smith [EMAIL PROTECTED] writes: On Sat, 17 May 2008, Tom Lane wrote: I was displeased to discover just now that in a standard RPM build of PG 8.3, psql and the other basic client programs pull in libxml2 and libxslt; this creates a package dependency that should not be there by any stretch of the imagination. When we noticed this recently, my digging suggested you'll be hard pressed to have a RedHat system now without those two installed. Indeed, I've not heard any squawks from the field yet. It's still wrong though ... regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [rfc,patch] PL/Proxy in core
On 5/18/08, Steve Singer [EMAIL PROTECTED] wrote: On Sat, 17 May 2008, Marko Kreen wrote: On 5/17/08, Steve Singer [EMAIL PROTECTED] wrote: Somewhat unrelated, I can see use-cases for replacing the call to random() with something that allows user defined polices for RUN ON ANY. Well, thats why the RUN ON userfunc(..); exists. Also notice the function can tag more that one partition for execution. Or did you mean something else than partition selection by user defined policy? I see RUN ON userfunc() as being for partitioning where the correctness requires that the query be run on the result of userfunc. I see RUN ON ANY as being for load-balancing. Here you see wrong. You should see RUN ON ANY simply as a shortcut for RUN ON random(); The actual random() would not work as it returns floats, but equivalent integer random(); So if you want smarter ANY, just implement your function. I don't see any need for tunable ANY. You might want to RUN ON ANY with a round robin balancing, or maybe consider the load of servers for doing the balancing. In the case of RUN ON ANY it seems that the database the query gets sent to doesn't matter. It might make sense for plproxy to try the next database if it can't connect to the first one it picks. You wouldn't want this for partitioning queries. If plproxy knows if you mean 'the query has to be run on these partitions' versus 'run the query on any partition, using method x to choose' then that type of things would be possible. Ok, here are 2 feature requests, that we have thought ourselves too: RUN ON LEAST LOADED; Sorry, this is unimplementable with current PL/Proxy design, as the per-backend PL-s do not coordinate their usage. And this is deliberate. If you want to implement this the design should look exactly like PL/Proxy 1 - each PL does special connection to special pooler that is responsible for partition selection and thus has information about partition usage. And the complexity went through the roof... You may achieve the same effect with smart tcp proxy or if not you can write load-balancing feature with load check for PgBouncer. RUN ON ANY PICK NEXT ON ERROR; This is implementable. But we have not found an actual need for it ourselves. So I have bothered to implement it as otherwise plproxy would have another implementable and maybe nice to have feature without actual reason like CONNECT, SELECT and get_cluster_config() turned out to be. OTOH, here we don't use read-only load balancing much. And such feature does not make sense when partitioning is used. But it indeed makes sense for load-balancing. So I'm not against adding it. -- marko -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
