Re: [HACKERS] build farm machine using make -j 8 mixed results
I wrote: And the answer is ... it's a gmake bug. Apparently introduced in 3.82. http://savannah.gnu.org/bugs/?30653 https://bugzilla.redhat.com/show_bug.cgi?id=835424 So I think .NOTPARALLEL is just masking the true problem, but nonetheless it's a problem. And given that the bug report on savannah has been ignored for two years, we should not hold our breath for a fix to appear upstream (much less propagate to everyone using 3.82). So no sooner do I complain about that, than the upstream maintainers wake up and commit it: http://lists.gnu.org/archive/html/bug-make/2012-09/msg00016.html No idea when a fixed release might appear, but at least somebody who knows the gmake code has signed off on the fix now. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] build farm machine using make -j 8 mixed results
On 09/10/2012 02:44 PM, Tom Lane wrote: I wrote: And the answer is ... it's a gmake bug. Apparently introduced in 3.82. http://savannah.gnu.org/bugs/?30653 https://bugzilla.redhat.com/show_bug.cgi?id=835424 So I think .NOTPARALLEL is just masking the true problem, but nonetheless it's a problem. And given that the bug report on savannah has been ignored for two years, we should not hold our breath for a fix to appear upstream (much less propagate to everyone using 3.82). So no sooner do I complain about that, than the upstream maintainers wake up and commit it: http://lists.gnu.org/archive/html/bug-make/2012-09/msg00016.html No idea when a fixed release might appear, but at least somebody who knows the gmake code has signed off on the fix now. When it does appear in a release I guess we could make the .NOTPARALLEL conditional on make version. If not, we'll have to wait a long time before removing it. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] build farm machine using make -j 8 mixed results
Peter Eisentraut pete...@gmx.net writes: On Sat, 2012-09-08 at 19:54 -0400, Tom Lane wrote: Anyway, what I notice is that I get different types of failures, but they are all under ecpg/. What I think we need to do is insert .NOTPARALLEL in ecpg/Makefile, I'd hate that, because the ecpg build is one of the slowest parts of the build, so de-parallelizing it would slow down everything quite a bit. There's only one bit of it that's slow, which is the bison build + preproc.c compile, which is necessarily serial anyway. So I think trying to avoid .NOTPARALLEL there is a complete waste of effort. But if you wanna fix it some other way, step right up. (This all does remind me of the Recursive Make Considered Harmful thread from awhile back. Is anyone prepared to go down that path now? I'm not terribly excited by that, as yet --- yeah, it would be better, but the work involved seems out of proportion to the benefit.) regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] build farm machine using make -j 8 mixed results
On 09/09/2012 03:29 AM, Tom Lane wrote: Peter Eisentraut pete...@gmx.net writes: On Sat, 2012-09-08 at 19:54 -0400, Tom Lane wrote: Anyway, what I notice is that I get different types of failures, but they are all under ecpg/. What I think we need to do is insert .NOTPARALLEL in ecpg/Makefile, I'd hate that, because the ecpg build is one of the slowest parts of the build, so de-parallelizing it would slow down everything quite a bit. There's only one bit of it that's slow, which is the bison build + preproc.c compile, which is necessarily serial anyway. So I think trying to avoid .NOTPARALLEL there is a complete waste of effort. But if you wanna fix it some other way, step right up. Yeah. I am going to add a config parameter to the buildfarm to allow parallelism for the make and make contrib stages, but I'm not going to release it until this is fixed. (I suppose this is also a lesson to me that I should not ignore things like this that annoy me persistently as I did with these failures until Robert Creager started this discussion. It just didn't seem important enough.) cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] build farm machine using make -j 8 mixed results
Andrew Dunstan and...@dunslane.net writes: On 09/09/2012 03:29 AM, Tom Lane wrote: Peter Eisentraut pete...@gmx.net writes: On Sat, 2012-09-08 at 19:54 -0400, Tom Lane wrote: Anyway, what I notice is that I get different types of failures, but they are all under ecpg/. What I think we need to do is insert .NOTPARALLEL in ecpg/Makefile, I'd hate that, because the ecpg build is one of the slowest parts of the build, so de-parallelizing it would slow down everything quite a bit. There's only one bit of it that's slow, which is the bison build + preproc.c compile, which is necessarily serial anyway. So I think trying to avoid .NOTPARALLEL there is a complete waste of effort. But if you wanna fix it some other way, step right up. Yeah. I am going to add a config parameter to the buildfarm to allow parallelism for the make and make contrib stages, but I'm not going to release it until this is fixed. Well, why don't we stick .NOTPARALLEL in there for the moment, and then if Peter thinks of a better solution, he can revert that change in favor of something cleaner. I assume we need this for all active branches, if the buildfarm is going to be stressing it? regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] build farm machine using make -j 8 mixed results
On 09/09/2012 11:31 AM, Tom Lane wrote: Yeah. I am going to add a config parameter to the buildfarm to allow parallelism for the make and make contrib stages, but I'm not going to release it until this is fixed. Well, why don't we stick .NOTPARALLEL in there for the moment, and then if Peter thinks of a better solution, he can revert that change in favor of something cleaner. Works for me I assume we need this for all active branches, if the buildfarm is going to be stressing it? I can restrict it to only modern branches. Didn't we supposedly improve support for this during the 9.1 cycle? That seems like a sensible place to go back to. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] build farm machine using make -j 8 mixed results
Andrew Dunstan and...@dunslane.net writes: On 09/09/2012 11:31 AM, Tom Lane wrote: I assume we need this for all active branches, if the buildfarm is going to be stressing it? I can restrict it to only modern branches. Didn't we supposedly improve support for this during the 9.1 cycle? That seems like a sensible place to go back to. Mmm, yeah, the commit history shows several different fixes for parallel make back in the 9.1 time frame. I doubt we want to try back-porting all that work. Let's just say 9.1 and up, then. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] build farm machine using make -j 8 mixed results
And the answer is ... it's a gmake bug. Apparently introduced in 3.82. http://savannah.gnu.org/bugs/?30653 https://bugzilla.redhat.com/show_bug.cgi?id=835424 So I think .NOTPARALLEL is just masking the true problem, but nonetheless it's a problem. And given that the bug report on savannah has been ignored for two years, we should not hold our breath for a fix to appear upstream (much less propagate to everyone using 3.82). regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] build farm machine using make -j 8 mixed results
On 09/09/2012 02:05 PM, Tom Lane wrote: And the answer is ... it's a gmake bug. Apparently introduced in 3.82. http://savannah.gnu.org/bugs/?30653 https://bugzilla.redhat.com/show_bug.cgi?id=835424 So I think .NOTPARALLEL is just masking the true problem, but nonetheless it's a problem. And given that the bug report on savannah has been ignored for two years, we should not hold our breath for a fix to appear upstream (much less propagate to everyone using 3.82). Thanks for pursuing this. Whether or not it masks the underlying problem, it's still something we should do, no? In fact, it seems to me like this makes it even less worth trying to do anything better. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] build farm machine using make -j 8 mixed results
Andrew Dunstan and...@dunslane.net writes: On 09/09/2012 02:05 PM, Tom Lane wrote: And the answer is ... it's a gmake bug. Thanks for pursuing this. Whether or not it masks the underlying problem, it's still something we should do, no? In fact, it seems to me like this makes it even less worth trying to do anything better. Yeah, exactly. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] build farm machine using make -j 8 mixed results
On Sun, 2012-09-09 at 14:05 -0400, Tom Lane wrote: And the answer is ... it's a gmake bug. Apparently introduced in 3.82. http://savannah.gnu.org/bugs/?30653 https://bugzilla.redhat.com/show_bug.cgi?id=835424 So I think .NOTPARALLEL is just masking the true problem, but nonetheless it's a problem. And given that the bug report on savannah has been ignored for two years, we should not hold our breath for a fix to appear upstream (much less propagate to everyone using 3.82). So does the issue go away when using GNU make 3.81? -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] build farm machine using make -j 8 mixed results
On Sun, 2012-09-09 at 14:57 -0400, Tom Lane wrote: Andrew Dunstan and...@dunslane.net writes: On 09/09/2012 02:05 PM, Tom Lane wrote: And the answer is ... it's a gmake bug. Thanks for pursuing this. Whether or not it masks the underlying problem, it's still something we should do, no? In fact, it seems to me like this makes it even less worth trying to do anything better. Yeah, exactly. But then the answer could be, if you want to use parallel make, use a version that's not broken. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] build farm machine using make -j 8 mixed results
Peter Eisentraut pete...@gmx.net writes: On Sun, 2012-09-09 at 14:05 -0400, Tom Lane wrote: So I think .NOTPARALLEL is just masking the true problem, but nonetheless it's a problem. And given that the bug report on savannah has been ignored for two years, we should not hold our breath for a fix to appear upstream (much less propagate to everyone using 3.82). So does the issue go away when using GNU make 3.81? I couldn't reproduce it in a few tries on my Mac laptop, which has 3.81. But that's not necessarily a comparable test, given the different hardware, OS, and compiler version. I'd put more faith in the multiple statements in the upstream bugs that people weren't seeing this with 3.81. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] build farm machine using make -j 8 mixed results
Peter Eisentraut pete...@gmx.net writes: But then the answer could be, if you want to use parallel make, use a version that's not broken. That's not a terribly practical answer for people who use the make supplied by their OS vendor, which is approximately 99.9% of people. It's even less practical for packagers, who don't have a choice about what tool set to use. Even if I wanted to use a locally-patched make, I'm not sure I'd trust a patch that doesn't seem to have been signed off on by any actual gmake developer or maintainer. That sort of cure is frequently worse than the disease. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] build farm machine using make -j 8 mixed results
On 09/09/2012 05:00 PM, Tom Lane wrote: Peter Eisentraut pete...@gmx.net writes: But then the answer could be, if you want to use parallel make, use a version that's not broken. That's not a terribly practical answer for people who use the make supplied by their OS vendor, which is approximately 99.9% of people. It's even less practical for packagers, who don't have a choice about what tool set to use. Even if I wanted to use a locally-patched make, I'm not sure I'd trust a patch that doesn't seem to have been signed off on by any actual gmake developer or maintainer. That sort of cure is frequently worse than the disease. Right. Meanwhile, I have buildfarm animal crake building with the experimental buildfarm code, and so far the results look good. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] build farm machine using make -j 8 mixed results
Andrew Dunstan and...@dunslane.net writes: This seems totally stupid, but it happens when the path to the current directory includes a cross-device symlink. If I cd following the link, then this effect doesn't happen. Weird. Huh. So maybe a gmake bug, or maybe there's something wrong with our make rules for the case that `pwd` doesn't match what gmake thinks the current path is. Could you specify the setup more fully? regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] build farm machine using make -j 8 mixed results
On 09/08/2012 11:06 AM, Tom Lane wrote: Andrew Dunstan and...@dunslane.net writes: This seems totally stupid, but it happens when the path to the current directory includes a cross-device symlink. If I cd following the link, then this effect doesn't happen. Weird. Huh. So maybe a gmake bug, or maybe there's something wrong with our make rules for the case that `pwd` doesn't match what gmake thinks the current path is. Could you specify the setup more fully? Scratch that theory, that was just a transient. If anything it looks like it is related to system load. When almost nothing was running on the machine it worked fine. When I started up a Browser and an MUA the problem occurred. This VM has 4 CPUs and 4Gb of memory and a load average around 0.4 right now. I'm a bit perplexed. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] build farm machine using make -j 8 mixed results
Andrew Dunstan and...@dunslane.net writes: Scratch that theory, that was just a transient. If anything it looks like it is related to system load. When almost nothing was running on the machine it worked fine. When I started up a Browser and an MUA the problem occurred. This VM has 4 CPUs and 4Gb of memory and a load average around 0.4 right now. I'm a bit perplexed. Hm ... you weren't using the -l (--max-load) option were you? That would make system load affect gmake's scheduling. Although it's clear when I test it that it is waiting for the bison run to finish before launching the dependent builds, so it still seems like it must be a bug if your copy isn't waiting for that. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] build farm machine using make -j 8 mixed results
Andrew Dunstan and...@dunslane.net writes: And it's the stock Fedora build of make. Which Fedora branch exactly? The package version I was trying to reproduce with here is make-3.82-8.fc16.x86_64. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] build farm machine using make -j 8 mixed results
On 09/08/2012 04:54 PM, Tom Lane wrote: Andrew Dunstan and...@dunslane.net writes: And it's the stock Fedora build of make. Which Fedora branch exactly? The package version I was trying to reproduce with here is make-3.82-8.fc16.x86_64. Same: $ rpm -q make make-3.82-8.fc16.x86_64 cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] build farm machine using make -j 8 mixed results
On 09/08/2012 04:46 PM, Tom Lane wrote: Andrew Dunstan and...@dunslane.net writes: Scratch that theory, that was just a transient. If anything it looks like it is related to system load. When almost nothing was running on the machine it worked fine. When I started up a Browser and an MUA the problem occurred. This VM has 4 CPUs and 4Gb of memory and a load average around 0.4 right now. I'm a bit perplexed. Hm ... you weren't using the -l (--max-load) option were you? That would make system load affect gmake's scheduling. Although it's clear when I test it that it is waiting for the bison run to finish before launching the dependent builds, so it still seems like it must be a bug if your copy isn't waiting for that. No. just make -j 4 And it's the stock Fedora build of make. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] build farm machine using make -j 8 mixed results
On 09/08/2012 04:52 PM, Andrew Dunstan wrote: On 09/08/2012 04:46 PM, Tom Lane wrote: Andrew Dunstan and...@dunslane.net writes: Scratch that theory, that was just a transient. If anything it looks like it is related to system load. When almost nothing was running on the machine it worked fine. When I started up a Browser and an MUA the problem occurred. This VM has 4 CPUs and 4Gb of memory and a load average around 0.4 right now. I'm a bit perplexed. Hm ... you weren't using the -l (--max-load) option were you? That would make system load affect gmake's scheduling. Although it's clear when I test it that it is waiting for the bison run to finish before launching the dependent builds, so it still seems like it must be a bug if your copy isn't waiting for that. No. just make -j 4 And it's the stock Fedora build of make. I have just repeated this on an absolutely fresh up to date F17 machine, with no symlink stuff in play. Steps to recreate: CC=ccache gcc ../postgres/configure --enable-depend --enable-debug --enable-cassert --with-perl --with-python --with-tcl --with-libxml --with-libxslt -with-openssl --with-gssapi --with-pam --with-ldap make -j 4 cheers andrew q -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] build farm machine using make -j 8 mixed results
Andrew Dunstan and...@dunslane.net writes: I have just repeated this on an absolutely fresh up to date F17 machine, with no symlink stuff in play. Steps to recreate: CC=ccache gcc ../postgres/configure --enable-depend --enable-debug --enable-cassert --with-perl --with-python --with-tcl --with-libxml --with-libxslt -with-openssl --with-gssapi --with-pam --with-ldap make -j 4 Huh ... that recipe works (er, fails) for me too, at least some of the time. I wonder what exactly is the key difference between the working and failing cases? Anyway, what I notice is that I get different types of failures, but they are all under ecpg/. What I think we need to do is insert .NOTPARALLEL in ecpg/Makefile, because there are several reasons not to run its sub-makes in parallel: * preproc/Makefile casually does this: ../ecpglib/typename.o: ../ecpglib/typename.c $(MAKE) -C $(dir $@) $(notdir $@) which is very likely to screw up any make proceeding in parallel in ecpglib. * compatlib and ecpglib will equally casually invoke make all in other subdirectories; ditto. And that's not even counting the bison-output problem you were seeing. I'm not entirely sure what's causing that, but I'm suspicious that the ultimate cause is the extra rules for the all...recurse targets in ecpg/Makefile, which look like they could result in additional instances of multiple make processes running in the same subdirectory. After adding the .NOTPARALLEL marker, I don't see these failures anymore. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] build farm machine using make -j 8 mixed results
On 09/08/2012 07:54 PM, Tom Lane wrote: Andrew Dunstan and...@dunslane.net writes: I have just repeated this on an absolutely fresh up to date F17 machine, with no symlink stuff in play. Steps to recreate: CC=ccache gcc ../postgres/configure --enable-depend --enable-debug --enable-cassert --with-perl --with-python --with-tcl --with-libxml --with-libxslt -with-openssl --with-gssapi --with-pam --with-ldap make -j 4 Huh ... that recipe works (er, fails) for me too, at least some of the time. I wonder what exactly is the key difference between the working and failing cases? Anyway, what I notice is that I get different types of failures, but they are all under ecpg/. What I think we need to do is insert .NOTPARALLEL in ecpg/Makefile, because there are several reasons not to run its sub-makes in parallel: * preproc/Makefile casually does this: ../ecpglib/typename.o: ../ecpglib/typename.c $(MAKE) -C $(dir $@) $(notdir $@) which is very likely to screw up any make proceeding in parallel in ecpglib. * compatlib and ecpglib will equally casually invoke make all in other subdirectories; ditto. And that's not even counting the bison-output problem you were seeing. I'm not entirely sure what's causing that, but I'm suspicious that the ultimate cause is the extra rules for the all...recurse targets in ecpg/Makefile, which look like they could result in additional instances of multiple make processes running in the same subdirectory. After adding the .NOTPARALLEL marker, I don't see these failures anymore. Well, I'm glad it's not just me. :-) This fix works for me too. I guess it should be applied back to 9.1, when it looks like we started using .NOTPARALLEL cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] build farm machine using make -j 8 mixed results
On Sat, 2012-09-08 at 19:54 -0400, Tom Lane wrote: Anyway, what I notice is that I get different types of failures, but they are all under ecpg/. What I think we need to do is insert .NOTPARALLEL in ecpg/Makefile, I'd hate that, because the ecpg build is one of the slowest parts of the build, so de-parallelizing it would slow down everything quite a bit. because there are several reasons not to run its sub-makes in parallel: * preproc/Makefile casually does this: ../ecpglib/typename.o: ../ecpglib/typename.c $(MAKE) -C $(dir $@) $(notdir $@) which is very likely to screw up any make proceeding in parallel in ecpglib. That should probably be fixed by symlinking the source file and building it in the preproc directory. And that's not even counting the bison-output problem you were seeing. I'm not entirely sure what's causing that, but I'm suspicious that the ultimate cause is the extra rules for the all...recurse targets in ecpg/Makefile, which look like they could result in additional instances of multiple make processes running in the same subdirectory. I think the point of these targets is exactly to prevent that. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] build farm machine using make -j 8 mixed results
On 09/04/2012 08:51 PM, Andrew Dunstan wrote: On 09/04/2012 08:37 PM, Tom Lane wrote: Andrew Dunstan and...@dunslane.net writes: Frankly, I have had enough failures of parallel make that I think doing this would generate a significant number of non-repeatable failures (I had one just the other day that took three invocations of make to get right). So I'm not sure doing this would advance us much, although I'm open to persuasion. Really? I routinely use -j4 for building, and it's been a long time since I've seen failures. I can believe that for instance make check in contrib would have a problem running in parallel, but the build process per se seems reliable enough from here. Both cases were vpath builds, which is what I usually use, if that's a useful data point. Maybe I run on lower level hardware than you do. I saw this again this afternoon after I posted the above. In both cases this was the machine that runs the buildfarm's crake. I'll try to get a handle on it. Well, it looks like it's always failing on ecpg, with preproc.h not being made in the right order. Here is the last bit of a make log starting from when it starts on ecpg. This is pretty repeatable. cheers andrew - make -C ecpg all make[3]: Entering directory `/home/pgl/npgl/vpath.testpar/src/interfaces/ecpg' make -C include all make[4]: Entering directory `/home/pgl/npgl/vpath.testpar/src/interfaces/ecpg/include' make[4]: Nothing to be done for `all'. make[4]: Leaving directory `/home/pgl/npgl/vpath.testpar/src/interfaces/ecpg/include' make -C pgtypeslib all make -C preproc all make[4]: Entering directory `/home/pgl/npgl/vpath.testpar/src/interfaces/ecpg/pgtypeslib' ccache gcc -O2 -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute -Wformat-security -fno-strict-aliasing -fwrapv -fexcess-precision=standard -g -pthread -D_REENTRANT -D_THREAD_SAFE -D_POSIX_PTHREAD_SEMANTICS -fpic -I../include -I/home/andrew/pgl/pg_head/src/interfaces/ecpg/include -I/home/andrew/pgl/pg_head/src/include/utils -I/home/andrew/pgl/pg_head/src/interfaces/libpq -I../../../../src/include -I/home/andrew/pgl/pg_head/src/include -D_GNU_SOURCE -I/usr/include/libxml2 -DSO_MAJOR_VERSION=3 -c -o numeric.o /home/andrew/pgl/pg_head/src/interfaces/ecpg/pgtypeslib/numeric.c -MMD -MP -MF .deps/numeric.Po ccache gcc -O2 -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute -Wformat-security -fno-strict-aliasing -fwrapv -fexcess-precision=standard -g -pthread -D_REENTRANT -D_THREAD_SAFE -D_POSIX_PTHREAD_SEMANTICS -fpic -I../include -I/home/andrew/pgl/pg_head/src/interfaces/ecpg/include -I/home/andrew/pgl/pg_head/src/include/utils -I/home/andrew/pgl/pg_head/src/interfaces/libpq -I../../../../src/include -I/home/andrew/pgl/pg_head/src/include -D_GNU_SOURCE -I/usr/include/libxml2 -DSO_MAJOR_VERSION=3 -c -o common.o /home/andrew/pgl/pg_head/src/interfaces/ecpg/pgtypeslib/common.c -MMD -MP -MF .deps/common.Po ccache gcc -O2 -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute -Wformat-security -fno-strict-aliasing -fwrapv -fexcess-precision=standard -g -pthread -D_REENTRANT -D_THREAD_SAFE -D_POSIX_PTHREAD_SEMANTICS -fpic -I../include -I/home/andrew/pgl/pg_head/src/interfaces/ecpg/include -I/home/andrew/pgl/pg_head/src/include/utils -I/home/andrew/pgl/pg_head/src/interfaces/libpq -I../../../../src/include -I/home/andrew/pgl/pg_head/src/include -D_GNU_SOURCE -I/usr/include/libxml2 -DSO_MAJOR_VERSION=3 -c -o datetime.o /home/andrew/pgl/pg_head/src/interfaces/ecpg/pgtypeslib/datetime.c -MMD -MP -MF .deps/datetime.Po make[4]: Entering directory `/home/pgl/npgl/vpath.testpar/src/interfaces/ecpg/preproc' make -C ../../../../src/port all make[5]: Entering directory `/home/pgl/npgl/vpath.testpar/src/port' make -C ../backend submake-errcodes make[6]: Entering directory `/home/pgl/npgl/vpath.testpar/src/backend' make[6]: Nothing to be done for `submake-errcodes'. make[6]: Leaving directory `/home/pgl/npgl/vpath.testpar/src/backend' make[5]: Leaving directory `/home/pgl/npgl/vpath.testpar/src/port' '/usr/bin/perl' /home/andrew/pgl/pg_head/src/interfaces/ecpg/preproc/parse.pl /home/andrew/pgl/pg_head/src/interfaces/ecpg/preproc /home/andrew/pgl/pg_head/src/interfaces/ecpg/preproc/../../../backend/parser/gram.y preproc.y ccache gcc -O2 -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute -Wformat-security -fno-strict-aliasing -fwrapv -fexcess-precision=standard -g -pthread -D_REENTRANT -D_THREAD_SAFE -D_POSIX_PTHREAD_SEMANTICS -fpic -I../include -I/home/andrew/pgl/pg_head/src/interfaces/ecpg/include -I/home/andrew/pgl/pg_head/src/include/utils -I/home/andrew/pgl/pg_head/src/interfaces/libpq -I../../../../src/include -I/home/andrew/pgl/pg_head/src/include
Re: [HACKERS] build farm machine using make -j 8 mixed results
Andrew Dunstan and...@dunslane.net writes: Well, it looks like it's always failing on ecpg, with preproc.h not being made in the right order. Here is the last bit of a make log starting from when it starts on ecpg. This is pretty repeatable. Hmph. I can't reproduce it at all on my Fedora 16 box. What version of make are you using? regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] build farm machine using make -j 8 mixed results
On 09/07/2012 08:43 PM, Tom Lane wrote: Andrew Dunstan and...@dunslane.net writes: Well, it looks like it's always failing on ecpg, with preproc.h not being made in the right order. Here is the last bit of a make log starting from when it starts on ecpg. This is pretty repeatable. Hmph. I can't reproduce it at all on my Fedora 16 box. What version of make are you using? $ make -v GNU Make 3.82 Built for x86_64-redhat-linux-gnu cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] build farm machine using make -j 8 mixed results
On 09/07/2012 09:55 PM, Andrew Dunstan wrote: On 09/07/2012 08:43 PM, Tom Lane wrote: Andrew Dunstan and...@dunslane.net writes: Well, it looks like it's always failing on ecpg, with preproc.h not being made in the right order. Here is the last bit of a make log starting from when it starts on ecpg. This is pretty repeatable. Hmph. I can't reproduce it at all on my Fedora 16 box. What version of make are you using? $ make -v GNU Make 3.82 Built for x86_64-redhat-linux-gnu OK, I just tried on a different F16 machine and it didn't happen. I wonder what's different. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] build farm machine using make -j 8 mixed results
On 09/07/2012 10:46 PM, Andrew Dunstan wrote: On 09/07/2012 09:55 PM, Andrew Dunstan wrote: On 09/07/2012 08:43 PM, Tom Lane wrote: Andrew Dunstan and...@dunslane.net writes: Well, it looks like it's always failing on ecpg, with preproc.h not being made in the right order. Here is the last bit of a make log starting from when it starts on ecpg. This is pretty repeatable. Hmph. I can't reproduce it at all on my Fedora 16 box. What version of make are you using? $ make -v GNU Make 3.82 Built for x86_64-redhat-linux-gnu OK, I just tried on a different F16 machine and it didn't happen. I wonder what's different. This seems totally stupid, but it happens when the path to the current directory includes a cross-device symlink. If I cd following the link, then this effect doesn't happen. Weird. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] build farm machine using make -j 8 mixed results
I change the build-farm.conf file to have the following make line: make = 'make -j 8', # or gmake if required. can include path if necessary. 2 pass, 4 fail. Is this a build configuration you want to pursue? I can either create a new machine, or change one of my existing machines. Makes no difference to me. roberts-imac:build-farm-4.7-j robert$ time ./run_branches.pl --run-all --test Sat Sep 1 09:26:49 2012: buildfarm run for CHANGEME:HEAD starting [09:26:49] checking out source ... [09:27:59] checking if build run needed ... [09:27:59] copying source to pgsql.54279 ... [09:28:03] running configure ... [09:28:25] running make ... [09:28:45] running make check ... Branch: HEAD Stage Check failed with status 2 Sat Sep 1 09:29:09 2012: buildfarm run for CHANGEME:REL9_2_STABLE starting [09:29:09] checking out source ... [09:29:57] checking if build run needed ... [09:29:57] copying source to pgsql.70926 ... [09:29:58] running configure ... [09:30:18] running make ... [09:30:45] running make check ... [09:31:09] running make contrib ... [09:31:12] running make install ... [09:31:15] running make contrib install ... [09:31:16] setting up db cluster (C)... [09:31:19] starting db (C)... [09:31:20] running make installcheck (C)... [09:31:39] restarting db (C)... [09:31:51] running make isolation check ... [09:32:00] restarting db (C)... [09:32:12] running make PL installcheck (C)... Branch: REL9_2_STABLE Stage PLCheck-C failed with status 2 Sat Sep 1 09:32:29 2012: buildfarm run for CHANGEME:REL9_1_STABLE starting [09:32:29] checking out source ... [09:33:08] checking if build run needed ... [09:33:08] copying source to pgsql.94883 ... [09:33:10] running configure ... [09:33:30] running make ... [09:33:54] running make check ... [09:34:18] running make contrib ... [09:34:21] running make install ... [09:34:23] running make contrib install ... [09:34:24] setting up db cluster (C)... [09:34:26] starting db (C)... [09:34:27] running make installcheck (C)... [09:34:47] restarting db (C)... [09:34:59] running make isolation check ... [09:35:16] restarting db (C)... [09:35:28] running make PL installcheck (C)... Branch: REL9_1_STABLE Stage PLCheck-C failed with status 2 Sat Sep 1 09:35:46 2012: buildfarm run for CHANGEME:REL9_0_STABLE starting [09:35:46] checking out source ... [09:36:08] checking if build run needed ... [09:36:08] copying source to pgsql.18851 ... [09:36:10] running configure ... [09:36:28] running make ... [09:37:00] running make check ... [09:37:23] running make contrib ... [09:37:27] running make install ... [09:37:30] running make contrib install ... [09:37:32] setting up db cluster (C)... [09:37:34] starting db (C)... [09:37:35] running make installcheck (C)... [09:37:52] restarting db (C)... [09:38:04] running make PL installcheck (C)... [09:38:06] restarting db (C)... [09:38:18] running make contrib installcheck (C)... [09:38:30] stopping db (C)... [09:38:31] running make ecpg check ... [09:38:43] OK Branch: REL9_0_STABLE All stages succeeded Sat Sep 1 09:38:43 2012: buildfarm run for CHANGEME:REL8_4_STABLE starting [09:38:43] checking out source ... [09:38:57] checking if build run needed ... [09:38:57] copying source to pgsql.45071 ... [09:38:59] running configure ... [09:39:19] running make ... [09:39:46] running make check ... [09:40:14] running make contrib ... [09:40:17] running make install ... [09:40:23] running make contrib install ... [09:40:25] setting up db cluster (C)... [09:40:30] starting db (C)... [09:40:31] running make installcheck (C)... [09:40:47] restarting db (C)... [09:40:59] running make PL installcheck (C)... [09:41:01] restarting db (C)... [09:41:13] running make contrib installcheck (C)... [09:41:25] stopping db (C)... [09:41:26] running make ecpg check ... [09:41:44] OK Branch: REL8_4_STABLE All stages succeeded Sat Sep 1 09:41:44 2012: buildfarm run for CHANGEME:REL8_3_STABLE starting [09:41:44] checking out source ... [09:42:39] checking if build run needed ... [09:42:39] copying source to pgsql.80222 ... [09:42:42] running configure ... [09:43:03] running make ... [09:43:38] running make check ... [09:44:04] running make contrib ... [09:44:09] running make install ... [09:44:14] running make contrib install ... [09:44:17] setting up db cluster (C)... [09:44:20] starting db (C)... [09:44:21] running make installcheck (C)... [09:44:39] restarting db (C)... [09:44:51] running make PL installcheck (C)... [09:44:52] restarting db (C)... [09:45:05] running make contrib installcheck (C)... Branch: REL8_3_STABLE Stage ContribCheck-C failed with status 2
Re: [HACKERS] build farm machine using make -j 8 mixed results
Excerpts from Robert Creager's message of sáb sep 01 12:12:51 -0400 2012: I change the build-farm.conf file to have the following make line: make = 'make -j 8', # or gmake if required. can include path if necessary. 2 pass, 4 fail. Is this a build configuration you want to pursue? Sure, why not? Parallel building is going to become more common, so it's a good idea to investigate the failures. I would have it build only HEAD though, because we're not likely to backpatch these fixes. I can either create a new machine, or change one of my existing machines. Makes no difference to me. If we want to have it run only HEAD I would say you should create a new animal. Perhaps you should wait until after 9.2 has been released, though, to avoid distracting people :-) -- Álvaro Herrerahttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] build farm machine using make -j 8 mixed results
On 9/1/12 12:12 PM, Robert Creager wrote: I change the build-farm.conf file to have the following make line: make = 'make -j 8', # or gmake if required. can include path if necessary. 2 pass, 4 fail. Is this a build configuration you want to pursue? Sure that would be useful, but it's pretty clear that the check stages don't work in parallel. It think it's because the ports conflict, but there could be any number of other problems. That said, it would be useful, in my mind, to support parallel checks. But unless someone is going to put in the work first, you should restrict your parallel runs to the build and install phases. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] build farm machine using make -j 8 mixed results
Excerpts from Peter Eisentraut's message of mar sep 04 18:49:46 -0300 2012: On 9/1/12 12:12 PM, Robert Creager wrote: I change the build-farm.conf file to have the following make line: make = 'make -j 8', # or gmake if required. can include path if necessary. 2 pass, 4 fail. Is this a build configuration you want to pursue? Sure that would be useful, but it's pretty clear that the check stages don't work in parallel. It think it's because the ports conflict, but there could be any number of other problems. Is that really the problem? As far as I know, buildfarm doesn't use anything like installcheck-world or similar targets; each check target is run separately, serially, by the BF script. -- Álvaro Herrerahttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] build farm machine using make -j 8 mixed results
On 09/04/2012 05:49 PM, Peter Eisentraut wrote: On 9/1/12 12:12 PM, Robert Creager wrote: I change the build-farm.conf file to have the following make line: make = 'make -j 8', # or gmake if required. can include path if necessary. 2 pass, 4 fail. Is this a build configuration you want to pursue? Sure that would be useful, but it's pretty clear that the check stages don't work in parallel. It think it's because the ports conflict, but there could be any number of other problems. That said, it would be useful, in my mind, to support parallel checks. But unless someone is going to put in the work first, you should restrict your parallel runs to the build and install phases. The buildfarm code doesn't contain a facility to use a different make incantation for each step. It's pretty much an all or nothing deal. Of course, you can hack run_build.pl to make it do that, but I highly discourage that. For one thing, it makes upgrading that much more difficult. All the tweaking is supposed to be done vie the config file. I guess I could add a setting that allowed for per step make flags. Frankly, I have had enough failures of parallel make that I think doing this would generate a significant number of non-repeatable failures (I had one just the other day that took three invocations of make to get right). So I'm not sure doing this would advance us much, although I'm open to persuasion. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] build farm machine using make -j 8 mixed results
On Sep 4, 2012 6:06 PM, Andrew Dunstan and...@dunslane.net wrote: Frankly, I have had enough failures of parallel make that I think doing this would generate a significant number of non-repeatable failures (I had one just the other day that took three invocations of make to get right). So I'm not sure doing this would advance us much, although I'm open to persuasion. Seeing as most PostgreSQL bugs appear with concurrency, I think we should leave our default config with 1 for max connections. ;-) Parallel make failures are bugs in the dependencies as described in our make files. For the build phase, I don't recall parallel problems and as a habit I usually use parallel makes. I would like that to be supported and I think I've seen fixes applied when we had issues before. Cutting build times to 1/2 to 1/4 is good. Checks and tests are harder because often they can't run in parallel. But then we shouldn't have them listed as independent prerequisites for targets. Ideally. But fixing it might not be worth the cost since an acceptable work around (rely upon make to serialize the test sequences in the particular order) is pretty painless (so far). Of course, having the ability to run the tests 8 at a time (or more) and reduce the time by 80% would be nice .;-) On Sep 4, 2012 6:06 PM, Andrew Dunstan and...@dunslane.net wrote: On 09/04/2012 05:49 PM, Peter Eisentraut wrote: On 9/1/12 12:12 PM, Robert Creager wrote: I change the build-farm.conf file to have the following make line: make = 'make -j 8', # or gmake if required. can include path if necessary. 2 pass, 4 fail. Is this a build configuration you want to pursue? Sure that would be useful, but it's pretty clear that the check stages don't work in parallel. It think it's because the ports conflict, but there could be any number of other problems. That said, it would be useful, in my mind, to support parallel checks. But unless someone is going to put in the work first, you should restrict your parallel runs to the build and install phases. The buildfarm code doesn't contain a facility to use a different make incantation for each step. It's pretty much an all or nothing deal. Of course, you can hack run_build.pl to make it do that, but I highly discourage that. For one thing, it makes upgrading that much more difficult. All the tweaking is supposed to be done vie the config file. I guess I could add a setting that allowed for per step make flags. Frankly, I have had enough failures of parallel make that I think doing this would generate a significant number of non-repeatable failures (I had one just the other day that took three invocations of make to get right). So I'm not sure doing this would advance us much, although I'm open to persuasion. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/**mailpref/pgsql-hackershttp://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] build farm machine using make -j 8 mixed results
Andrew Dunstan and...@dunslane.net writes: Frankly, I have had enough failures of parallel make that I think doing this would generate a significant number of non-repeatable failures (I had one just the other day that took three invocations of make to get right). So I'm not sure doing this would advance us much, although I'm open to persuasion. Really? I routinely use -j4 for building, and it's been a long time since I've seen failures. I can believe that for instance make check in contrib would have a problem running in parallel, but the build process per se seems reliable enough from here. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] build farm machine using make -j 8 mixed results
On 09/04/2012 08:37 PM, Tom Lane wrote: Andrew Dunstan and...@dunslane.net writes: Frankly, I have had enough failures of parallel make that I think doing this would generate a significant number of non-repeatable failures (I had one just the other day that took three invocations of make to get right). So I'm not sure doing this would advance us much, although I'm open to persuasion. Really? I routinely use -j4 for building, and it's been a long time since I've seen failures. I can believe that for instance make check in contrib would have a problem running in parallel, but the build process per se seems reliable enough from here. Both cases were vpath builds, which is what I usually use, if that's a useful data point. Maybe I run on lower level hardware than you do. I saw this again this afternoon after I posted the above. In both cases this was the machine that runs the buildfarm's crake. I'll try to get a handle on it. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Build farm coverage for isolation tests
On 04/19/2011 11:53 AM, Kevin Grittner wrote: Andrew Dunstanand...@dunslane.net wrote: I think the best thing might be to add a new step which runs the isolation check or installcheck. It would only need a small amount of perl code adde3d to the client to accomplish, and nothing on the server side. Since I'm unskilled at perl and have never looked at the buildfarm scripts, could someone with the appropriate skills and knowledge add this for me? In bash, when I'm in the normal base directory for a build from source, my usual incantation is: pushd src/test/isolation make check popd (or use installcheck instead of check if an appropriate cluster is already running) See http://www.pgbuildfarm.org/cgi-bin/show_stage_log.pl?nm=crakedt=2011-04-20%2010%3A17%3A02stg=isolation-check I'll include this in the upcoming release of the buildfarm client. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Build farm coverage for isolation tests
Andrew Dunstan and...@dunslane.net wrote: http://www.pgbuildfarm.org/cgi-bin/show_stage_log.pl?nm=crakedt=2011-04-20%2010%3A17%3A02stg=isolation-check On a cursory scan, that looks like a successful run of the right test suite. Was there anything in particular I should be checking there? I'll include this in the upcoming release of the buildfarm client. Thanks! This will only be attempted for 9.1 or higher, right? -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Build farm coverage for isolation tests
On 04/20/2011 09:54 AM, Kevin Grittner wrote: Andrew Dunstanand...@dunslane.net wrote: http://www.pgbuildfarm.org/cgi-bin/show_stage_log.pl?nm=crakedt=2011-04-20%2010%3A17%3A02stg=isolation-check On a cursory scan, that looks like a successful run of the right test suite. Was there anything in particular I should be checking there? Not that I know of. I just gave you the URL a) to show it's actually being done and b) to show you what the output would be like. I'll include this in the upcoming release of the buildfarm client. Thanks! This will only be attempted for 9.1 or higher, right? Yes, The code checks for the existence of the isolation directory and only runs the step if it exists. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Build farm coverage for isolation tests
I'm not sure what the right thing is to do here. Heikki added a new testing methodology under src/test/isolation which allows intermingling a series of statements on multiple connections in desired permutations. (By default each test defined for it runs all permutations of how the statement sequences can be interleaved, but you can specify one or more particular permutations if needed.) I filled it out with the SSI tests which had been run under dtester during initial development. To run it you change to the src/test/isolation directory and run `make check` or `make installcheck`. It takes too long to run this to think about including it in the main `make check`, but I'd feel better about things if this set of tests was normally run on the buildfarm. On my so-so development machine, the installcheck takes about 20 seconds. Setting up and tearing down a temp installation for the `make check` adds another 10 seconds. That seems unlikely to be a deal-breaker for the buildfarm, even if we were to add a little to the test set. Would it make sense to add this to `make check-world` and `make installcheck-world`? Would something else be a good idea to get buildfarm coverage? -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Build farm coverage for isolation tests
On 04/19/2011 11:16 AM, Kevin Grittner wrote: I'm not sure what the right thing is to do here. Heikki added a new testing methodology under src/test/isolation which allows intermingling a series of statements on multiple connections in desired permutations. (By default each test defined for it runs all permutations of how the statement sequences can be interleaved, but you can specify one or more particular permutations if needed.) I filled it out with the SSI tests which had been run under dtester during initial development. To run it you change to the src/test/isolation directory and run `make check` or `make installcheck`. It takes too long to run this to think about including it in the main `make check`, but I'd feel better about things if this set of tests was normally run on the buildfarm. On my so-so development machine, the installcheck takes about 20 seconds. Setting up and tearing down a temp installation for the `make check` adds another 10 seconds. That seems unlikely to be a deal-breaker for the buildfarm, even if we were to add a little to the test set. Would it make sense to add this to `make check-world` and `make installcheck-world`? Would something else be a good idea to get buildfarm coverage? The buildfarm doesn't run either of these. I think the best thing might be to add a new step which runs the isolation check or installcheck. It would only need a small amount of perl code adde3d to the client to accomplish, and nothing on the server side. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Build farm coverage for isolation tests
Andrew Dunstan and...@dunslane.net wrote: I think the best thing might be to add a new step which runs the isolation check or installcheck. It would only need a small amount of perl code adde3d to the client to accomplish, and nothing on the server side. Since I'm unskilled at perl and have never looked at the buildfarm scripts, could someone with the appropriate skills and knowledge add this for me? In bash, when I'm in the normal base directory for a build from source, my usual incantation is: pushd src/test/isolation make check popd (or use installcheck instead of check if an appropriate cluster is already running) Thanks in advance, -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Build farm coverage for isolation tests
Excerpts from Kevin Grittner's message of mar abr 19 12:53:07 -0300 2011: Since I'm unskilled at perl and have never looked at the buildfarm scripts, could someone with the appropriate skills and knowledge add this for me? In bash, when I'm in the normal base directory for a build from source, my usual incantation is: pushd src/test/isolation make check popd make -C src/test/isolation check -- Álvaro Herrera alvhe...@commandprompt.com The PostgreSQL Company - Command Prompt, Inc. PostgreSQL Replication, Consulting, Custom Development, 24x7 support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Build farm server database migration complete
It took a little longer than expected, due to a slightly clagged network between the old and new servers, but the database migration is complete and the server is back up and running. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Build farm tweaks
Greg Sabino Mullane wrote: -BEGIN PGP SIGNED MESSAGE- Hash: RIPEMD160 I mentioned earlier that buildfarm member jaguar (that's the one that builds with CLOBBER_CACHE_ALWAYS) was showing suspicious intermittent failures. Tom, this brings up another question: is there any flag, environment, forced resource limitation, etc. that you or others would like to see in the build farm? I'd be happy to apply anything to my animal, for example, as it's running on a fairly standard Linux distro and I'd like it to be more 'useful'. At the very least I think we would benefit from more cache-clobbering animals. (It'd be nice to display that flag in the buildfarm dashboard too ...) BTW I find it a bit surprising that we have more 8.3 animals than 8.4 ... I guess this is just because there's no auto-add of branches to animals when new major versions are released. -- Alvaro Herrerahttp://www.CommandPrompt.com/ The PostgreSQL Company - Command Prompt, Inc. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Build farm tweaks
-BEGIN PGP SIGNED MESSAGE- Hash: RIPEMD160 I mentioned earlier that buildfarm member jaguar (that's the one that builds with CLOBBER_CACHE_ALWAYS) was showing suspicious intermittent failures. Tom, this brings up another question: is there any flag, environment, forced resource limitation, etc. that you or others would like to see in the build farm? I'd be happy to apply anything to my animal, for example, as it's running on a fairly standard Linux distro and I'd like it to be more 'useful'. If such a list turns out to be more than a few items, perhaps doc it somewhere like the wiki? - -- Greg Sabino Mullane g...@turnstep.com PGP Key: 0x14964AC8 201001082147 http://biglumber.com/x/web?pk=2529DF6AB8F79407E94445B4BC9B906714964AC8 -BEGIN PGP SIGNATURE- iEYEAREDAAYFAktH7scACgkQvJuQZxSWSsjLPACgmof4H4ux/0qpOPbpHIfKrM5E gm4AoKN9BhihZqZeS6rDQo1j8l9B+uVL =cxEm -END PGP SIGNATURE- -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Build farm failure
dugong (icc on ia64) has been failing the contrib installcheck consistently since 6 days ago with errors like: ERROR: could not fsync segment 0 of relation 1663/40960/41403: No such file or directory I checked a cvs diff between the two timestamps and that's precisely when the self-adjusting bgwriter changes went in which knocked around bgwriter's checkpoint logic quite a bit so it seems likely this is a real bug. On the other hand it seems weird that it only occurs in contrib's check and not the normal tests. And it also seems weird it only happens on icc ia64 and not any other architecture under the sun. I installed icc on my machine (ia32) but didn't get the same problem (we don't seem to have a recent icc ia32 build farm member). The only things I know about icc are that it likes to pad structs unnecessarily and is picky about its -f arguments... LOG: database system was shut down at 2007-10-02 02:21:38 MSD LOG: autovacuum launcher started LOG: database system is ready to accept connections NOTICE: database contrib_regression does not exist, skipping NOTICE: type gbtreekey4 is not yet defined DETAIL: Creating a shell type definition. NOTICE: argument type gbtreekey4 is only a shell NOTICE: type gbtreekey8 is not yet defined DETAIL: Creating a shell type definition. NOTICE: argument type gbtreekey8 is only a shell NOTICE: type gbtreekey16 is not yet defined DETAIL: Creating a shell type definition. NOTICE: argument type gbtreekey16 is only a shell NOTICE: type gbtreekey32 is not yet defined DETAIL: Creating a shell type definition. NOTICE: argument type gbtreekey32 is only a shell NOTICE: type gbtreekey_var is not yet defined DETAIL: Creating a shell type definition. NOTICE: argument type gbtreekey_var is only a shell ERROR: could not fsync segment 0 of relation 1663/40960/41403: No such file or directory ERROR: checkpoint request failed HINT: Consult recent messages in the server log for details. STATEMENT: CREATE DATABASE contrib_regression TEMPLATE=template0 NOTICE: database contrib_regression does not exist, skipping ERROR: could not fsync segment 0 of relation 1663/40960/41403: No such file or directory ERROR: checkpoint request failed HINT: Consult recent messages in the server log for details. STATEMENT: CREATE DATABASE contrib_regression TEMPLATE=template0 NOTICE: database contrib_regression does not exist, skipping ERROR: could not fsync segment 0 of relation 1663/40960/41403: No such file or directory ERROR: checkpoint request failed HINT: Consult recent messages in the server log for details. STATEMENT: CREATE DATABASE contrib_regression TEMPLATE=template0 NOTICE: database contrib_regression does not exist, skipping ERROR: could not fsync segment 0 of relation 1663/40960/41403: No such file or directory ERROR: checkpoint request failed HINT: Consult recent messages in the server log for details. STATEMENT: CREATE DATABASE contrib_regression TEMPLATE=template0 NOTICE: database contrib_regression does not exist, skipping ERROR: could not fsync segment 0 of relation 1663/40960/41403: No such file or directory ERROR: checkpoint request failed HINT: Consult recent messages in the server log for details. STATEMENT: CREATE DATABASE contrib_regression TEMPLATE=template0 NOTICE: database contrib_regression does not exist, skipping ERROR: could not fsync segment 0 of relation 1663/40960/41403: No such file or directory ERROR: checkpoint request failed HINT: Consult recent messages in the server log for details. STATEMENT: CREATE DATABASE contrib_regression TEMPLATE=template0 NOTICE: database contrib_regression does not exist, skipping ERROR: could not fsync segment 0 of relation 1663/40960/41403: No such file or directory ERROR: checkpoint request failed HINT: Consult recent messages in the server log for details. STATEMENT: CREATE DATABASE contrib_regression TEMPLATE=template0 NOTICE: database contrib_regression does not exist, skipping ERROR: could not fsync segment 0 of relation 1663/40960/41403: No such file or directory ERROR: checkpoint request failed HINT: Consult recent messages in the server log for details. STATEMENT: CREATE DATABASE contrib_regression TEMPLATE=template0 NOTICE: database contrib_regression does not exist, skipping ERROR: could not fsync segment 0 of relation 1663/40960/41403: No such file or directory ERROR: checkpoint request failed HINT: Consult recent messages in the server log for details. STATEMENT: CREATE DATABASE contrib_regression TEMPLATE=template0 NOTICE: database contrib_regression does not exist, skipping ERROR: could not fsync segment 0 of relation 1663/40960/41403: No such file or directory ERROR: checkpoint request failed HINT: Consult recent messages in the server log for details. STATEMENT: CREATE DATABASE contrib_regression TEMPLATE=template0 -- Gregory Stark EnterpriseDB http://www.enterprisedb.com ---(end of
Re: [HACKERS] Build farm failure
Gregory Stark [EMAIL PROTECTED] writes: dugong (icc on ia64) has been failing the contrib installcheck consistently since 6 days ago with errors like: ERROR: could not fsync segment 0 of relation 1663/40960/41403: No such file or directory I checked a cvs diff between the two timestamps and that's precisely when the self-adjusting bgwriter changes went in which knocked around bgwriter's checkpoint logic quite a bit so it seems likely this is a real bug. Of course moments after I sent that it finally occurred to me to look further back in dugong's history and its previous failing periods were with exactly the same errors. So the bgwriter changes are off the hook. And given the consistency and the fact that the other icc machines didn't show the same problems it sounds like it's something about that machine, not a software problem. -- Gregory Stark EnterpriseDB http://www.enterprisedb.com ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] Build farm failure
Gregory Stark [EMAIL PROTECTED] writes: dugong (icc on ia64) has been failing the contrib installcheck consistently since 6 days ago with errors like: ERROR: could not fsync segment 0 of relation 1663/40960/41403: No such file or directory Yeah, I already asked Sergey about this but I guess he's not had time to poke at it yet: http://archives.postgresql.org/pgsql-hackers/2007-09/msg01061.php My theory is that putting an Assert right there is somehow breaking ForwardFsyncRequest --- maybe it becomes a complete no-op, maybe it forwards a corrupt request, who knows. The only way that there'd be any visible problem from that, if you weren't actually performing pull-the-power-plug tests, would be that lack of forwarding of revoke requests could lead to the bgwriter attempting to fsync files in already-dropped databases or tablespaces. Which matches the visible symptoms exactly. This looks like nothing so much as a compiler bug, particularly given that we're seeing it with only one compiler on only one platform. We should study it more carefully, both to look for workarounds and to file a suitable bug report, but I'll be pretty surprised if it's really our bug. regards, tom lane ---(end of broadcast)--- TIP 7: You can help support the PostgreSQL project by donating at http://www.postgresql.org/about/donate
Re: [HACKERS] Build farm failure
Gregory Stark [EMAIL PROTECTED] writes: And given the consistency and the fact that the other icc machines didn't show the same problems it sounds like it's something about that machine, not a software problem. Well, we haven't *got* any other icc-on-ia64 machines AFAICS, so it could easily be a software problem. Your remark about padding set off an alarm bell on re-reading --- what if RelFileNode is padded to 16 bytes on that architecture? Junk in the padding might break lookups in bgwriter's internal hashtable. But the Intel docs I can find do not suggest any such thing. Could someone confirm what sizeof(RelFileNode) is on ia64? regards, tom lane ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [HACKERS] Build farm failure
On Tue, 2 Oct 2007, Gregory Stark wrote: (we don't seem to have a recent icc ia32 build farm member). Sorry about that, my buildfarm member (mongoose) is down with hardware problems, and probably will be for the forseeable future. For some reason, it suddenly decided to stop recognizing its RAID card... -- In the beginning was the word. But by the time the second word was added to it, there was trouble. For with it came syntax ... -- John Simon ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
[HACKERS] build farm failures
Hi, we have two build farm members failing to make since I committed teh ecpg changes: echidna and herring. It looks like they are still using an old preproc.c although they checked out the new preproc.y. I have no idea how this is supposed to work so could someone please enlighten me? Michael -- Michael Meskes Email: Michael at Fam-Meskes dot De, Michael at Meskes dot (De|Com|Net|Org) ICQ: 179140304, AIM/Yahoo: michaelmeskes, Jabber: [EMAIL PROTECTED] Go SF 49ers! Go Rhein Fire! Use Debian GNU/Linux! Use PostgreSQL! ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [HACKERS] build farm failures
Michael Meskes wrote: Hi, we have two build farm members failing to make since I committed teh ecpg changes: echidna and herring. It looks like they are still using an old preproc.c although they checked out the new preproc.y. I have no idea how this is supposed to work so could someone please enlighten me? Yes it looks like that. But the buildfarm client doesn't actually build in the repo normally - it builds in a temp copy which is removed at the end of the run, precisely to avoid this kind of problem, so I'm a bit mystified how it can happen. In fact we go to some lengths to ensure that there are no extraneous files, but this one might not get caught by that because it is is in .cvsignore. This sort of thing is usually a symptom of somebody having run a build in the repo directly, a thing that buildfarm owners have been repeatedly advised not to do. Anyway, the simple solution is to ask Darcy to blow away the repo (these buildfarm clients share a single cvs checkout) so that the buildfarm client will get a fresh checkout next time it's run. cheers andrew ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] build farm failures
Darcy Buskermolen wrote: This sort of thing is usually a symptom of somebody having run a build in the repo directly, a thing that buildfarm owners have been repeatedly advised not to do. This is something I do not recall doing, however it's possible. though this does make me ask why are the build dependencies in the Makefile are not properly setup to tell that the .y needs to be rebuilt (which I would assume would make this problem also go away) Thje way cvs works is that it gives the file the date it has in the repository, so if your preproc.c is newer than the preproc.y, make will detect that and not rebuild it. If Michael's checkin occurs between the time the repo is updated and the time bison gets run on the original file this will happen. But if you never ever build in the repo then it won't, because buildfarm only ever builds in a copy (unless you're building with vpath, in which case it cleans up the generated files). cheers andrew ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [HACKERS] build farm failures
On Thursday 16 August 2007 04:29:41 Andrew Dunstan wrote: Michael Meskes wrote: Hi, we have two build farm members failing to make since I committed teh ecpg changes: echidna and herring. It looks like they are still using an old preproc.c although they checked out the new preproc.y. I have no idea how this is supposed to work so could someone please enlighten me? Yes it looks like that. But the buildfarm client doesn't actually build in the repo normally - it builds in a temp copy which is removed at the end of the run, precisely to avoid this kind of problem, so I'm a bit mystified how it can happen. In fact we go to some lengths to ensure that there are no extraneous files, but this one might not get caught by that because it is is in .cvsignore. This sort of thing is usually a symptom of somebody having run a build in the repo directly, a thing that buildfarm owners have been repeatedly advised not to do. This is something I do not recall doing, however it's possible. though this does make me ask why are the build dependencies in the Makefile are not properly setup to tell that the .y needs to be rebuilt (which I would assume would make this problem also go away) Anyway, the simple solution is to ask Darcy to blow away the repo (these buildfarm clients share a single cvs checkout) so that the buildfarm client will get a fresh checkout next time it's run. Let me go nuke the tree, and we'll try again cheers andrew ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] build farm failures
Andrew Dunstan wrote: Darcy Buskermolen wrote: This sort of thing is usually a symptom of somebody having run a build in the repo directly, a thing that buildfarm owners have been repeatedly advised not to do. This is something I do not recall doing, however it's possible. though this does make me ask why are the build dependencies in the Makefile are not properly setup to tell that the .y needs to be rebuilt (which I would assume would make this problem also go away) Thje way cvs works is that it gives the file the date it has in the repository, so if your preproc.c is newer than the preproc.y, make will detect that and not rebuild it. If Michael's checkin occurs between the time the repo is updated and the time bison gets run on the original file this will happen. But if you never ever build in the repo then it won't, because buildfarm only ever builds in a copy (unless you're building with vpath, in which case it cleans up the generated files). Hum, so why not clean up the files when not in vpath as well? find . -name .cvsignore | while read line do dir=$(dirname $line) cd $dir rm -fv `cat .cvsignore` cd $OLDPWD done -- Alvaro Herrerahttp://www.CommandPrompt.com/ PostgreSQL Replication, Consulting, Custom Development, 24x7 support ---(end of broadcast)--- TIP 7: You can help support the PostgreSQL project by donating at http://www.postgresql.org/about/donate
Re: [HACKERS] build farm failures
Alvaro Herrera wrote: Andrew Dunstan wrote: Darcy Buskermolen wrote: This sort of thing is usually a symptom of somebody having run a build in the repo directly, a thing that buildfarm owners have been repeatedly advised not to do. This is something I do not recall doing, however it's possible. though this does make me ask why are the build dependencies in the Makefile are not properly setup to tell that the .y needs to be rebuilt (which I would assume would make this problem also go away) Thje way cvs works is that it gives the file the date it has in the repository, so if your preproc.c is newer than the preproc.y, make will detect that and not rebuild it. If Michael's checkin occurs between the time the repo is updated and the time bison gets run on the original file this will happen. But if you never ever build in the repo then it won't, because buildfarm only ever builds in a copy (unless you're building with vpath, in which case it cleans up the generated files). Hum, so why not clean up the files when not in vpath as well? find . -name .cvsignore | while read line do dir=$(dirname $line) cd $dir rm -fv `cat .cvsignore` cd $OLDPWD done Because they are not supposed to be there in the first place! If the buildfarm owner builds in the repo that is pilot error. And, btw, buildfarm is not a shell script. We use File::Find to do this sort of thing. cheers andrew ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] build farm failures
On Thu, Aug 16, 2007 at 08:24:14AM -0700, Darcy Buskermolen wrote: This is something I do not recall doing, however it's possible. though this does make me ask why are the build dependencies in the Makefile are not properly setup to tell that the .y needs to be rebuilt (which I would assume would make this problem also go away) Frankly I have no idea. The dependencies are the same as with the backend's gram.y file: $(srcdir)/preproc.c: $(srcdir)/preproc.h ; $(srcdir)/preproc.h: preproc.y The backend has: $(srcdir)/gram.c: $(srcdir)/parse.h ; $(srcdir)/parse.h: gram.y So except for the different naming it's the same. However, we haven't had that problem with the backend so far, or did we? What do I fail to see? Michael -- Michael Meskes Email: Michael at Fam-Meskes dot De, Michael at Meskes dot (De|Com|Net|Org) ICQ: 179140304, AIM/Yahoo: michaelmeskes, Jabber: [EMAIL PROTECTED] Go SF 49ers! Go Rhein Fire! Use Debian GNU/Linux! Use PostgreSQL! ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] build farm failures
Michael Meskes wrote: The backend has: $(srcdir)/gram.c: $(srcdir)/parse.h ; $(srcdir)/parse.h: gram.y So except for the different naming it's the same. However, we haven't had that problem with the backend so far, or did we? What do I fail to see? We have had problems in the past. If the user builds at a point in time after the .y file is checked in then the generated file is newer and if it's not removed will never be regenerated, even if they do a subsequent cvs update. cheers andrew ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] build farm failures
Andrew Dunstan wrote: Michael Meskes wrote: The backend has: $(srcdir)/gram.c: $(srcdir)/parse.h ; $(srcdir)/parse.h: gram.y So except for the different naming it's the same. However, we haven't had that problem with the backend so far, or did we? What do I fail to see? We have had problems in the past. If the user builds at a point in time after the .y file is checked in then the generated file is newer and if it's not removed will never be regenerated, even if they do a subsequent cvs update. How do you create the copy of the repo to build? One idea would be to explicitely skip files that appear on .cvsignore (and maybe croak about them). -- Alvaro Herrerahttp://www.CommandPrompt.com/ The PostgreSQL Company - Command Prompt, Inc. ---(end of broadcast)--- TIP 7: You can help support the PostgreSQL project by donating at http://www.postgresql.org/about/donate
Re: [HACKERS] build farm failures
Alvaro Herrera wrote: How do you create the copy of the repo to build? One idea would be to explicitely skip files that appear on .cvsignore (and maybe croak about them). We are supposed to croak - see http://cvs.pgfoundry.org/cgi-bin/cvsweb.cgi/pgbuildfarm/client-code/run_build.pl.diff?r1=1.69r2=1.70 And these machines run the version after that. So either the code is buggy or my explanation is of what happened is :-) cheers andrew ---(end of broadcast)--- TIP 7: You can help support the PostgreSQL project by donating at http://www.postgresql.org/about/donate
[HACKERS] Build Farm: thrush
Hi all, I'm the administrator of that machine and PLCheck is failing. Is there anything I can do to fix it ? Regards Gaetano Mendola ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [HACKERS] Build Farm: thrush
Gaetano Mendola [EMAIL PROTECTED] writes: I'm the administrator of that machine and PLCheck is failing. Is there anything I can do to fix it ? What version of Python have you got on that thing? It seems to be emitting still another spelling of the encoding error message :-( regards, tom lane ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Build Farm: thrush
Tom Lane wrote: Gaetano Mendola [EMAIL PROTECTED] writes: I'm the administrator of that machine and PLCheck is failing. Is there anything I can do to fix it ? What version of Python have you got on that thing? It seems to be emitting still another spelling of the encoding error message :-( $ python -V Python 2.2.3 Regards Gaetano Mendola ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [HACKERS] Build Farm: thrush
On Mon, Oct 03, 2005 at 05:19:43PM +0200, Gaetano Mendola wrote:
Tom Lane wrote:
Gaetano Mendola [EMAIL PROTECTED] writes:
I'm the administrator of that machine and PLCheck is failing.
Is there anything I can do to fix it ?
What version of Python have you got on that thing? It seems to be
emitting still another spelling of the encoding error message :-(
$ python -V
Python 2.2.3
The attached variant file (plpython_error_2.out) should allow this
Python version's wording of the error message.
--
Michael Fuhr
-- test error handling, i forgot to restore Warn_restart in
-- the trigger handler once. the errors and subsequent core dump were
-- interesting.
SELECT invalid_type_uncaught('rick');
WARNING: plpython: in function invalid_type_uncaught:
DETAIL: plpy.SPIError: Unknown error in PLy_spi_prepare
ERROR: type test does not exist
SELECT invalid_type_caught('rick');
WARNING: plpython: in function invalid_type_caught:
DETAIL: plpy.SPIError: Unknown error in PLy_spi_prepare
ERROR: type test does not exist
SELECT invalid_type_reraised('rick');
WARNING: plpython: in function invalid_type_reraised:
DETAIL: plpy.SPIError: Unknown error in PLy_spi_prepare
ERROR: type test does not exist
SELECT valid_type('rick');
valid_type
(1 row)
--
-- Test Unicode error handling.
--
SELECT unicode_return_error();
ERROR: plpython: function unicode_return_error could not create return value
DETAIL: exceptions.UnicodeError: ASCII encoding error: ordinal not in
range(128)
INSERT INTO unicode_test (testvalue) VALUES ('test');
ERROR: plpython: function unicode_trigger_error could not modify tuple
DETAIL: exceptions.UnicodeError: ASCII encoding error: ordinal not in
range(128)
SELECT unicode_plan_error1();
WARNING: plpython: in function unicode_plan_error1:
DETAIL: plpy.Error: Unknown error in PLy_spi_execute_plan
ERROR: plpython: function unicode_plan_error1 could not execute plan
DETAIL: exceptions.UnicodeError: ASCII encoding error: ordinal not in
range(128)
SELECT unicode_plan_error2();
ERROR: plpython: function unicode_plan_error2 could not execute plan
DETAIL: exceptions.UnicodeError: ASCII encoding error: ordinal not in
range(128)
---(end of broadcast)---
TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] Build Farm: thrush
Michael Fuhr [EMAIL PROTECTED] writes: On Mon, Oct 03, 2005 at 05:19:43PM +0200, Gaetano Mendola wrote: What version of Python have you got on that thing? It seems to be emitting still another spelling of the encoding error message :-( $ python -V Python 2.2.3 The attached variant file (plpython_error_2.out) should allow this Python version's wording of the error message. Applied, thanks. regards, tom lane ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
[HACKERS] build farm failure tuna on OSX 10.4 for 8.0
It appears to be getting the wrong address for tsearch()
Program received signal EXC_BAD_ACCESS, Could not access memory.
Reason: KERN_PROTECTION_FAILURE at address: 0x011b
0x9009a7c8 in tsearch ()
However if I set a break point for tsearch, I get
br tsearch
Breakpoint 5 at 0xe08c64: file txtidx.c, line 533.
further
(gdb) display *finfo
5: *finfo = {
fn_addr = 0x9009a798 tsearch,
fn_oid = 377912,
fn_nargs = 0,
fn_strict = 0 '\0',
fn_retset = 0 '\0',
fn_extra = 0x0,
fn_mcxt = 0x20270f4,
fn_expr = 0x0
}
(gdb) list *0x9009a798
No source file for address 0x9009a798.
I admit to not being an expert in gdb here, but it certainly looks
like something isn't quite right
Dave
---(end of broadcast)---
TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] build farm failure tuna on OSX 10.4 for 8.0
Dave Cramer [EMAIL PROTECTED] writes: It appears to be getting the wrong address for tsearch() I applied a patch for that earlier today. It seems that in OS X 10.4 the compiler generates a function with the same name as the shared library, ie tsearch() for libtsearch ... and it doesn't tell you about the name conflict, it just lets the wrong function get called at runtime. Earlier OS X versions did not do this, and I have to say that it's one of the most brain-dead usurpations of application namespace I've seen in a long time. But there you have it. regards, tom lane ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] Build farm
Just a thought. You could also run the regression test automatically after a successful build? Andrew Dunstan [EMAIL PROTECTED] wrote in message news:[EMAIL PROTECTED] Jean-Michel POURE wrote: Le Vendredi 21 Novembre 2003 19:47, Tom Lane a écrit : I think the main value of a build farm is that we'd get nearly immediate feedback about the majority of simple porting problems. Your previous arguments that it wouldn't smoke everything out are certainly valid --- but we wouldn't abandon the regression tests just because they don't find everything. Immediate feedback is good because a patch can be fixed while it's still fresh in the author's mind. Dear friends, We have a small build farm for pgAdmin covering Win32, FreeBSD and most GNU/ Linux systems. See http://www.pgadmin.org/pgadmin3/download.php#snapshots The advantage are immediate feedback and correction of problems. Also, in a release cycle, developers and translators are quite motivated to see their work published fast. Of course, it is always hard to mesure the real impact of a build farm. My opinion it that it is quite positive, as it helps tighten the links between people, which is free software is mostly about. Right. But I think we have been talking about using the build farm to do test builds rather than to provide snapshots. I'd be very wary of providing arbitrary snapshots of postgres, whereas I'd be prepared to try a snapshot of pgadmin3 under certain circumstances. (Also, building your own snapshot of postgres is somewhat easier than building your own snapshot of pgadmin3). cheers andrew ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly ---(end of broadcast)--- TIP 8: explain analyze is your friend
Re: [HACKERS] Build farm
Peter Eisentraut wrote: Bruce Momjian writes: FYI, the HP testdrive farm, http://www.testdrive.hp.com, has shared directories for most of the machines, meaning you can CVS update once and telnet in to compile for each platform. Except that you can't open connections to the outside from these machines. Oh, yea. You can connect to the machines with ftp, so I guess you would have to CVS update on your local machine, then push the changes to the farm. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] Build farm
Bruce Momjian wrote: FYI, the HP testdrive farm, http://www.testdrive.hp.com, has shared directories for most of the machines, meaning you can CVS update once and telnet in to compile for each platform. As Peter pointed out, these machines are firewalled. But presumably one could upload a snapshot to them. What I had in mind was a more distributed system, though. Of course, these things are not mutually exclusive - using the HP testdrive farm looks like it might be nice. But it would be hard to automate, I suspect. cheers andrew ---(end of broadcast)--- TIP 8: explain analyze is your friend
Re: [HACKERS] Build farm
Andrew Dunstan wrote: Bruce Momjian wrote: FYI, the HP testdrive farm, http://www.testdrive.hp.com, has shared directories for most of the machines, meaning you can CVS update once and telnet in to compile for each platform. As Peter pointed out, these machines are firewalled. But presumably one could upload a snapshot to them. What I had in mind was a more distributed system, though. Of course, these things are not mutually exclusive - using the HP testdrive farm looks like it might be nice. But it would be hard to automate, I suspect. I figured you could just upload once and telnet and build on each machine. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] Build farm
Bruce Momjian wrote: Andrew Dunstan wrote: Bruce Momjian wrote: FYI, the HP testdrive farm, http://www.testdrive.hp.com, has shared directories for most of the machines, meaning you can CVS update once and telnet in to compile for each platform. As Peter pointed out, these machines are firewalled. But presumably one could upload a snapshot to them. What I had in mind was a more distributed system, though. Of course, these things are not mutually exclusive - using the HP testdrive farm looks like it might be nice. But it would be hard to automate, I suspect. I figured you could just upload once and telnet and build on each machine. What I'm working on (slowly - I'm quite busy right now, and about to be away from home for 5 days) is a system which would (or could) run from cron on every member of the farm, and upload its results to a central server where it could be displayed, in a somewhat similar way to the way the Samba build farm works - see http://build.samba.org/ - so we'd be able to see at a glance when something is broken and where and why. We could also incorporate email notification of breakage, as a refinement. I have a few pieces of this working but not a full suite yet - it will essentially be 3 perl scripts - one on the client (to run the update(s), build(s) and upload the results) and two on the central server (one for upload and one for display). When I get a demo page done I'll show it working with a couple of hosts. Of course, you can automate (almost) anything, including telnet, but right now I'm assuming the farm members will have internet connectivity. cheers andrew ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] Build farm
Le Vendredi 21 Novembre 2003 19:47, Tom Lane a crit : I think the main value of a build farm is that we'd get nearly immediate feedback about the majority of simple porting problems. Your previous arguments that it wouldn't smoke everything out are certainly valid --- but we wouldn't abandon the regression tests just because they don't find everything. Immediate feedback is good because a patch can be fixed while it's still fresh in the author's mind. Dear friends, We have a small build farm for pgAdmin covering Win32, FreeBSD and most GNU/ Linux systems. See http://www.pgadmin.org/pgadmin3/download.php#snapshots The advantage are immediate feedback and correction of problems. Also, in a release cycle, developers and translators are quite motivated to see their work published fast. Of course, it is always hard to mesure the real impact of a build farm. My opinion it that it is quite positive, as it helps tighten the links between people, which is free software is mostly about. Cheers, Jean-Michel Pour ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: [HACKERS] Build farm
Jean-Michel POURE wrote: Le Vendredi 21 Novembre 2003 19:47, Tom Lane a crit : I think the main value of a build farm is that we'd get nearly immediate feedback about the majority of simple porting problems. Your previous arguments that it wouldn't smoke everything out are certainly valid --- but we wouldn't abandon the regression tests just because they don't find everything. Immediate feedback is good because a patch can be fixed while it's still fresh in the author's mind. Dear friends, We have a small build farm for pgAdmin covering Win32, FreeBSD and most GNU/ Linux systems. See http://www.pgadmin.org/pgadmin3/download.php#snapshots The advantage are immediate feedback and correction of problems. Also, in a release cycle, developers and translators are quite motivated to see their work published fast. Of course, it is always hard to mesure the real impact of a build farm. My opinion it that it is quite positive, as it helps tighten the links between people, which is free software is mostly about. Right. But I think we have been talking about using the build farm to do test builds rather than to provide snapshots. I'd be very wary of providing arbitrary snapshots of postgres, whereas I'd be prepared to try a snapshot of pgadmin3 under certain circumstances. (Also, building your own snapshot of postgres is somewhat easier than building your own snapshot of pgadmin3). cheers andrew ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Build farm
Andrew Dunstan wrote: Jean-Michel POURE wrote: Le Vendredi 21 Novembre 2003 19:47, Tom Lane a crit : I think the main value of a build farm is that we'd get nearly immediate feedback about the majority of simple porting problems. Your previous arguments that it wouldn't smoke everything out are certainly valid --- but we wouldn't abandon the regression tests just because they don't find everything. Immediate feedback is good because a patch can be fixed while it's still fresh in the author's mind. Dear friends, We have a small build farm for pgAdmin covering Win32, FreeBSD and most GNU/ Linux systems. See http://www.pgadmin.org/pgadmin3/download.php#snapshots The advantage are immediate feedback and correction of problems. Also, in a release cycle, developers and translators are quite motivated to see their work published fast. Of course, it is always hard to mesure the real impact of a build farm. My opinion it that it is quite positive, as it helps tighten the links between people, which is free software is mostly about. Right. But I think we have been talking about using the build farm to do test builds rather than to provide snapshots. I'd be very wary of providing arbitrary snapshots of postgres, whereas I'd be prepared to try a snapshot of pgadmin3 under certain circumstances. (Also, building your own snapshot of postgres is somewhat easier than building your own snapshot of pgadmin3). Testing a build and creating a snapshot compilation is quite the same, just a different name and announcement. I agree that using a pgadmin snapshot is different from pgsql, somebody using a bleeding edge pgsql version should be prepared to compile it on his own machine. And a tiny correction: The farm member for win32 is my machine, and it's operated manually :-) Regards, Andreas ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Build farm
Le Lundi 24 Novembre 2003 16:38, Andreas Pflug a crit : And a tiny correction: The farm member for win32 is my machine, and it's operated manually :-) Some GNU/Linux farm animals are living in my garage running on very old 50 euros machines ... Ancient farming :-) By the way, we would love if someone could provide pgAdmin3 daily snapshots under other systems. The list of platforms can be viewed here, anyone is welcome to provide additional ones: http://www.pgadmin.org/pgadmin3/download.php#snapshots Cheers, Jean-Michel ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] Build farm
FYI, the HP testdrive farm, http://www.testdrive.hp.com, has shared directories for most of the machines, meaning you can CVS update once and telnet in to compile for each platform. --- Andrew Dunstan wrote: Tom Lane wrote: I think the main value of a build farm is that we'd get nearly immediate feedback about the majority of simple porting problems. Your previous arguments that it wouldn't smoke everything out are certainly valid --- but we wouldn't abandon the regression tests just because they don't find everything. Immediate feedback is good because a patch can be fixed while it's still fresh in the author's mind. Yes, I seem to recall seeing several instances of things like you mean foonix version 97 1/2 has a bad frobnitz.h? over the last 6 months. Having that caught early is exactly the advantage, I believe. I'm for it ... I'm working on it :-) Regarding make distcheck that Peter suggested I use, unless I'm mistaken it carefully does its own configure, thus ignoring the configure options set in the original directory. Perhaps we need either to have the distcheck target pick up all the --with/--without and --enable/--disable options, or to have a similar target that does that. Thoughts? cheers andrew ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] Build farm
On Wed, Nov 19, 2003 at 04:34:27PM +0100, Peter Eisentraut wrote: Hence the open-source community approach. Closed-source development teams can do all the above, with great effort. But by throwing out the code and have real people test them on real systems with real applications, you can do much better. Would it be reasonable to promote users testing daily snapshots with popular applications? I'm guessing there's not many applications that have automated test frameworks, but any that do would theoretically provide another good test of PGSQL changes. -- Jim C. Nasby, Database Consultant [EMAIL PROTECTED] Member: Triangle Fraternity, Sports Car Club of America Give your computer some brain candy! www.distributed.net Team #1828 Windows: Where do you want to go today? Linux: Where do you want to go tomorrow? FreeBSD: Are you guys coming, or what? ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] Build farm
Would it be reasonable to promote users testing daily snapshots with popular applications? I'm guessing there's not many applications that have automated test frameworks, but any that do would theoretically provide another good test of PGSQL changes. May I quote Joel on Software here? http://www.joelonsoftware.com/articles/fog43.html The Joel Test 1. Do you use source control? 2. Can you make a build in one step? 3. Do you make daily builds? 4. Do you have a bug database? 5. Do you fix bugs before writing new code? 6. Do you have an up-to-date schedule? 7. Do you have a spec? 8. Do programmers have quiet working conditions? 9. Do you use the best tools money can buy? 10. Do you have testers? 11. Do new candidates write code during their interview? 12. Do you do hallway usability testing? The neat thing about The Joel Test is that it's easy to get a quick yes or no to each question. You don't have to figure out lines-of-code-per-day or average-bugs-per-inflection-point. Give your team 1 point for each yes answer. The bummer about The Joel Test is that you really shouldn't use it to make sure that your nuclear power plant software is safe. A score of 12 is perfect, 11 is tolerable, but 10 or lower and you've got serious problems. The truth is that most software organizations are running with a score of 2 or 3, and they need serious help, because companies like Microsoft run at 12 full-time. Not everything there applies to us, of course. Chris ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Build farm
Bruce Momjian writes: FYI, the HP testdrive farm, http://www.testdrive.hp.com, has shared directories for most of the machines, meaning you can CVS update once and telnet in to compile for each platform. Except that you can't open connections to the outside from these machines. -- Peter Eisentraut [EMAIL PROTECTED] ---(end of broadcast)--- TIP 8: explain analyze is your friend
Re: [HACKERS] Build farm
Peter Eisentraut [EMAIL PROTECTED] writes: Andrew Dunstan writes: Maybe it wouldn't be of great value to PostgreSQL. And maybe it would. I have an open mind about it. I don't think incompleteness is an argument against it, though. If you want to do it, by all means go for it. I'm sure it would give everyone a fuzzy feeling to see the green lights everywhere. But realistically, don't expect any significant practical benefits, such cutting beta time by 10%. I think the main value of a build farm is that we'd get nearly immediate feedback about the majority of simple porting problems. Your previous arguments that it wouldn't smoke everything out are certainly valid --- but we wouldn't abandon the regression tests just because they don't find everything. Immediate feedback is good because a patch can be fixed while it's still fresh in the author's mind. I'm for it ... regards, tom lane ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] Build farm
Tom Lane wrote: I think the main value of a build farm is that we'd get nearly immediate feedback about the majority of simple porting problems. Your previous arguments that it wouldn't smoke everything out are certainly valid --- but we wouldn't abandon the regression tests just because they don't find everything. Immediate feedback is good because a patch can be fixed while it's still fresh in the author's mind. Yes, I seem to recall seeing several instances of things like you mean foonix version 97 1/2 has a bad frobnitz.h? over the last 6 months. Having that caught early is exactly the advantage, I believe. I'm for it ... I'm working on it :-) Regarding make distcheck that Peter suggested I use, unless I'm mistaken it carefully does its own configure, thus ignoring the configure options set in the original directory. Perhaps we need either to have the distcheck target pick up all the --with/--without and --enable/--disable options, or to have a similar target that does that. Thoughts? cheers andrew ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: [HACKERS] Build farm
Peter Eisentraut wrote: Andrew Dunstan writes: Essentially what I have is something like this pseudocode: cvs update Be sure check past branches as well. check if there really was an update and if not exit OK. configure; get config.log Ideally, you'd try all possible option combinations for configure. Or at least enable everything. I have had in mind from the start doing multiple configurations and multiple branches. Right now I'm working only with everything/head, but will make provision for multiple sets of both. How many branches back do you think should we go? Right now I'd be inclined only to do REL7_4_STABLE and HEAD as a default. Maybe we could set the default to be gettable from the web server so that as new releases come along build farm members using the default wouldn't need to make any changes. However, everything would also be settable locally on each build farm member in an options file. make 21 | make-filter makelog make check 21 | check-filter checklog You could also try out make distcheck. It tries out the complete build, installation, uninstallation, regression test, and distribution building. OK. (TBD) send config status, make status, check status, logfiles OK. make distclean When I played around with this, always copied the CVS tree to a new directory and deleted that one at the end. That way, bugs in the clean procedure (known to happen) don't trip up the whole process. OK. We've also seen odd problems with cvs update, I seem to recall, but I'd rather avoid having to fetch the entire tree for each run, to keep bandwidth use down. (I believe cvs update should be fairly reliable if there are no local changes, which would be true in this instance). The send piece will probably be a perl script using LWP and talking to a CGI script. That will be the difficult part to organize, if it's supposed to be distributed and autonomous. sending the results won't be a huge problem - storing and displaying them nicely will be a bit more fun :-) Upload of results would be over authenticated SSL to prevent spurious results being fed to us - all you would need to join the build farm would be a username/password from the buildfarm admin. Thanks for your input cheers andrew ---(end of broadcast)--- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] Build farm
Andrew Dunstan writes: If there's general interest I'll try to cook something up. (This kind of stuff is right up my alley). I'd prefer some automated display of results, though. A simple CGI script should be all that's required for that. The real problem will be to find enough machines so that the build farm becomes useful. IMO, that would mean *more* machines than are currently lines in the supported-platforms table. -- Peter Eisentraut [EMAIL PROTECTED] ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faqs/FAQ.html
Re: [HACKERS] Build farm
Peter Eisentraut wrote: Andrew Dunstan writes: If there's general interest I'll try to cook something up. (This kind of stuff is right up my alley). I'd prefer some automated display of results, though. A simple CGI script should be all that's required for that. The real problem will be to find enough machines so that the build farm becomes useful. IMO, that would mean *more* machines than are currently lines in the supported-platforms table. Useful is probably subjective. That list would at least be a good place to start, though. What combinations of variables do you think we would need? This would be a fairly painless way for users to be helpful to the project, btw - the way I am envisioning things this would be fairly much a set and forget process. I'll have an example page available in a few days. cheers andrew ---(end of broadcast)--- TIP 8: explain analyze is your friend
Re: [HACKERS] Build farm
Andrew Dunstan writes: Useful is probably subjective. That list would at least be a good place to start, though. What combinations of variables do you think we would need? First of all, I don't necessarily think that a large list of CPU/operation system combinations is going to help much. IIRC, this round of platform testing showed us two real problems, and both happened because the operating system version in question came out the previous day, so we could not have caught it. Much more problems arise when people use different versions of secondary packages, such as Tcl, Perl, Kerberos, Flex, Bison. So you would need to compile a large collection of these things. The problem again is that it is usually the brand-new or the odd intermediate version of such a tool that breaks things, so a set and forget build farm is not going to catch it. Another real source of problems are real systems. Weird combinations of packages, weird network setups, weird applications, custom kernels. These cannot be detected on out of the box setups. In fact, the regression tests might not detect them at all. Hence the open-source community approach. Closed-source development teams can do all the above, with great effort. But by throwing out the code and have real people test them on real systems with real applications, you can do much better. -- Peter Eisentraut [EMAIL PROTECTED] ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] Build farm
Peter Eisentraut wrote: Andrew Dunstan writes: Useful is probably subjective. That list would at least be a good place to start, though. What combinations of variables do you think we would need? First of all, I don't necessarily think that a large list of CPU/operation system combinations is going to help much. IIRC, this round of platform testing showed us two real problems, and both happened because the operating system version in question came out the previous day, so we could not have caught it. Much more problems arise when people use different versions of secondary packages, such as Tcl, Perl, Kerberos, Flex, Bison. So you would need to compile a large collection of these things. The problem again is that it is usually the brand-new or the odd intermediate version of such a tool that breaks things, so a set and forget build farm is not going to catch it. Another real source of problems are real systems. Weird combinations of packages, weird network setups, weird applications, custom kernels. These cannot be detected on out of the box setups. In fact, the regression tests might not detect them at all. Hence the open-source community approach. Closed-source development teams can do all the above, with great effort. But by throwing out the code and have real people test them on real systems with real applications, you can do much better. The fact that something doesn't find everything doesn't mean it is of no value. (Thinks of Scott Adams' nice example: Your theory of gravity doesn't prove why there are no unicorns, so it is wrong. ;-) ) I don't believe there is a single open source community approach - open source projects all have differing ways of handling problems. At least 2 very significant open source projects I know of run build farms, notwithstanding that your objections should apply equally to them. Mozilla's is fairly centralised and very complex and heavy, but gives fairly immediate feedback if anything gets broken. Samba's is much lighter, distributed, and they still apparently see good value in it. (Samba uses a torture test - perhaps we need one of those in addition to the regression tests.) Maybe it wouldn't be of great value to PostgreSQL. And maybe it would. I have an open mind about it. I don't think incompleteness is an argument against it, though. cheers andrew ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Build farm
Andrew Dunstan writes: Maybe it wouldn't be of great value to PostgreSQL. And maybe it would. I have an open mind about it. I don't think incompleteness is an argument against it, though. If you want to do it, by all means go for it. I'm sure it would give everyone a fuzzy feeling to see the green lights everywhere. But realistically, don't expect any significant practical benefits, such cutting beta time by 10%. The Samba build daemon suite is pretty good. We have a couple of those hosts in our office in fact. (I think they're building PostgreSQL regularly as well.) A tip: You might find that adopting the source code of the Samba suite to PostgreSQL is harder than writing a new one. -- Peter Eisentraut [EMAIL PROTECTED] ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] Build farm
Peter Eisentraut wrote: The Samba build daemon suite is pretty good. We have a couple of those hosts in our office in fact. (I think they're building PostgreSQL regularly as well.) A tip: You might find that adopting the source code of the Samba suite to PostgreSQL is harder than writing a new one. Yes, I agree. I have looked at it for ideas, but not for code. I'm not using rsync or anything like that, for instance. I'm going for something very simple to start with. Essentially what I have is something like this pseudocode: cvs update check if there really was an update and if not exit configure; get config.log make 21 | make-filter makelog make check 21 | check-filter checklog (TBD) send config status, make status, check status, logfiles make distclean The send piece will probably be a perl script using LWP and talking to a CGI script. cheers andrew ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] Build farm
Andrew Dunstan writes: Essentially what I have is something like this pseudocode: cvs update Be sure check past branches as well. check if there really was an update and if not exit OK. configure; get config.log Ideally, you'd try all possible option combinations for configure. Or at least enable everything. make 21 | make-filter makelog make check 21 | check-filter checklog You could also try out make distcheck. It tries out the complete build, installation, uninstallation, regression test, and distribution building. (TBD) send config status, make status, check status, logfiles OK. make distclean When I played around with this, always copied the CVS tree to a new directory and deleted that one at the end. That way, bugs in the clean procedure (known to happen) don't trip up the whole process. The send piece will probably be a perl script using LWP and talking to a CGI script. That will be the difficult part to organize, if it's supposed to be distributed and autonomous. -- Peter Eisentraut [EMAIL PROTECTED] ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
[HACKERS] Build farm
Marc G. Fournier wrote: On Tue, 18 Nov 2003, Andrew Dunstan wrote: Maybe some sort of automated distributed build farm would be a good idea. Check out http://build.samba.org/about.html to see how samba does it (much lighter than the Mozilla tinderbox approach). We wouldn't need to be as intensive as they appear to be - maybe a once or twice a day download and test run would do the trick, but it could pick up lots of breakage fairly quickly. That is not to say that more intensive testing isn't also needed on occasion. Check the archives on this, as its been hashed out already once at least ... I think the big issue/problem is that nobody seems able (or wants) to come up with a script that could be setup in cron on machines to do this ... something simple that would dump the output to a log file and, if regression tests failed, email'd the machine owner that it needs to be checked would do, I would think ... If there's general interest I'll try to cook something up. (This kind of stuff is right up my alley). I'd prefer some automated display of results, though. A simple CGI script should be all that's required for that. cheers andrew ---(end of broadcast)--- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] Build farm
On Tue, 2003-11-18 at 14:36, Andrew Dunstan wrote:
Marc G. Fournier wrote:
On Tue, 18 Nov 2003, Andrew Dunstan wrote:
Maybe some sort of automated distributed build farm would be a good
idea. Check out http://build.samba.org/about.html to see how samba does
it (much lighter than the Mozilla tinderbox approach).
We wouldn't need to be as intensive as they appear to be - maybe a once
or twice a day download and test run would do the trick, but it could
pick up lots of breakage fairly quickly.
That is not to say that more intensive testing isn't also needed on
occasion.
Check the archives on this, as its been hashed out already once at least
... I think the big issue/problem is that nobody seems able (or wants) to
come up with a script that could be setup in cron on machines to do this
... something simple that would dump the output to a log file and, if
regression tests failed, email'd the machine owner that it needs to be
checked would do, I would think ...
If there's general interest I'll try to cook something up. (This kind of stuff is
right up my alley). I'd prefer some automated display of results, though. A simple
CGI script should be all that's required for that.
look in the tools directory of cvs, i swear Bruce checked in a script he
uses for similar tasks..
Robert Treat
--
Build A Brighter Lamp :: Linux Apache {middleware} PostgreSQL
---(end of broadcast)---
TIP 5: Have you checked our extensive FAQ?
http://www.postgresql.org/docs/faqs/FAQ.html
Re: [HACKERS] Build farm
Robert Treat wrote: Check the archives on this, as its been hashed out already once at least ... I think the big issue/problem is that nobody seems able (or wants) to come up with a script that could be setup in cron on machines to do this ... something simple that would dump the output to a log file and, if regression tests failed, email'd the machine owner that it needs to be checked would do, I would think ... If there's general interest I'll try to cook something up. (This kind of stuff is right up my alley). I'd prefer some automated display of results, though. A simple CGI script should be all that's required for that. look in the tools directory of cvs, i swear Bruce checked in a script he uses for similar tasks.. /tools/pgtest -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
