Re: [HACKERS] [PATCH] big test separation POC

2013-07-11 Thread Samrat Revagade
Hi Fabien, While applying latest version of the patch (regress-big-v4.patch) on latest PostgreSQL version i encountered following errors: a) Using git: $git apply --index regress-big-v4.patch regress-big-v4.patch:10: trailing whitespace. $(srcdir)/parallel_schedule

Re: [HACKERS] [PATCH] big test separation POC

2013-07-11 Thread Fabien COELHO
While applying latest version of the patch (regress-big-v4.patch) on latest PostgreSQL version i encountered following errors: [...] Is that a problem ? Yes and no:-) My understanding is that there is a conflict because of commits between this patch and head: a file that this patch

Re: [HACKERS] [PATCH] big test separation POC

2013-07-11 Thread Alvaro Herrera
Fabien COELHO escribió: Note that this is really a POC. How to derive a file is under discussion: it has been suggested that the unix shell approach would not work on Windows. I've suggested perl or python (which version?) but I'm not sure that it is okay either. The other option, suggested

Re: [HACKERS] [PATCH] big test separation POC

2013-07-11 Thread Josh Berkus
On 07/11/2013 09:19 AM, Alvaro Herrera wrote: Fabien COELHO escribió: Note that this is really a POC. How to derive a file is under discussion: it has been suggested that the unix shell approach would not work on Windows. I've suggested perl or python (which version?) but I'm not sure that

Re: [HACKERS] [PATCH] big test separation POC

2013-07-11 Thread Fabien COELHO
The other option, suggested by Andres somewhere, is to have a new parameter to pg_regress, something like --run-serially. After looking at the source, ISTM that this option already exists under a different signature: --max-connections 1 So you would use the same parallel

Re: [HACKERS] [PATCH] big test separation POC

2013-07-04 Thread Fabien COELHO
+serial_%: parallel_% + echo # this file is generated automatically, do not edit! $@ + egrep '^(test|ignore):' $ | \ + while read op list ; do \ + for test in $$list ; do \ + echo $$op $$test ; \ + done ; \ + done $@ + This won't work on

Re: [HACKERS] [PATCH] big test separation POC

2013-07-03 Thread Fabien COELHO
Here is a v2 which is more likely to work under VPATH. Here is a POC v4 which relies on multiple --schedule instead of creating concatenated schedule files. -- Fabien.diff --git a/src/test/regress/GNUmakefile b/src/test/regress/GNUmakefile index d5935b6..8a39f7d 100644 ---

Re: [HACKERS] [PATCH] big test separation POC

2013-07-03 Thread Andres Freund
On 2013-07-03 21:07:03 +0200, Fabien COELHO wrote: Here is a v2 which is more likely to work under VPATH. Here is a POC v4 which relies on multiple --schedule instead of creating concatenated schedule files. -- Fabien. diff --git a/src/test/regress/GNUmakefile

Re: [HACKERS] [PATCH] big test separation POC

2013-07-01 Thread Samrat Revagade
Hi Fabien, On Mon, Jul 1, 2013 at 10:42 AM, Fabien COELHO coe...@cri.ensmp.fr wrote: - I do not understand why the makefile specifies $(srcdir) before local files in some places. For VPATH builds :-) Here is a v2 which is more likely to work under VPATH. I really appreciate your

Re: [HACKERS] [PATCH] big test separation POC

2013-07-01 Thread Fabien COELHO
While testing patch, I found that make installcheck breaks with your patch and gives following error: Indeed, I did not put the dependency for that target, I really tested check bigcheck. The attached patch adds the needed dependency for installcheck, and I could run it. I checked that no

Re: [HACKERS] [PATCH] big test separation POC

2013-06-30 Thread Fabien COELHO
Note about the POC patch limitations/questions: - is deriving a schedule with a piece of shell okay? or should perl/python/whatever scripting be better? - the big_schedule is assumed sequential, i.e. one test per line. maybe it could/should be parallel? - I'm not sure of the

Re: [HACKERS] [PATCH] big test separation POC

2013-06-30 Thread Andrew Dunstan
On 06/30/2013 02:54 PM, Fabien COELHO wrote: Note about the POC patch limitations/questions: - is deriving a schedule with a piece of shell okay? or should perl/python/whatever scripting be better? I would think all we need are the results, i.e. the schedule files, plus some Makefile

Re: [HACKERS] [PATCH] big test separation POC

2013-06-30 Thread Fabien COELHO
Note about the POC patch limitations/questions: - is deriving a schedule with a piece of shell okay? or should perl/python/whatever scripting be better? I would think all we need are the results, i.e. the schedule files, plus some Makefile entries for them. You can replicate data, but

Re: [HACKERS] [PATCH] big test separation POC

2013-06-30 Thread Fabien COELHO
- I do not understand why the makefile specifies $(srcdir) before local files in some places. For VPATH builds :-) Here is a v2 which is more likely to work under VPATH. -- Fabien.diff --git a/src/test/regress/GNUmakefile b/src/test/regress/GNUmakefile index 7309b00..5a6d0f9 100644 ---

[HACKERS] [PATCH] big test separation POC

2013-06-28 Thread Fabien COELHO
Dear hackers, Per various discussion about the potential impact of Robins non regression tests, here is a poc patch to separate big tests from others. paralle_schedule holds usual tests, big_schedule holds big tests. The makefile derives serial_schedule, parallel_big_schedule and