Re: [HACKERS] build farm machine using mixed results

[Tom Lane]

Peter Eisentraut  writes:
> On Sat, 2012-09-08 at 19:54 -0400, Tom Lane wrote:
>> Anyway, what I notice is that I get different types of failures, but
>> they are all under ecpg/.  What I think we need to do is insert
>> .NOTPARALLEL in ecpg/Makefile,

> I'd hate that, because the ecpg build is one of the slowest parts of the
> build, so de-parallelizing it would slow down everything quite a bit.

There's only one bit of it that's slow, which is the bison build +
preproc.c compile, which is necessarily serial anyway.  So I think
trying to avoid .NOTPARALLEL there is a complete waste of effort.
But if you wanna fix it some other way, step right up.

(This all does remind me of the "Recursive Make Considered Harmful"
thread from awhile back.  Is anyone prepared to go down that path
now?  I'm not terribly excited by that, as yet --- yeah, it would
be better, but the work involved seems out of proportion to the
benefit.)

regards, tom lane


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Supporting plpython 2+3 builds better

[Tom Lane]

Peter Eisentraut  writes:
> On Sat, 2012-09-08 at 19:18 -0400, Tom Lane wrote:
>> To give you an idea of what "unreasonably painful" means, attached is
>> the specfile diff needed to make this happen.  I will not comment on
>> the fragility of this beyond observing that the "touch -r" commands
>> are *necessary*, else you get incorrect results.

> I think an easier way is to configure two vpath builds and install the
> pieces you want from each one.

I thought about that, and didn't like it, because then you have no proof
that the plpython3 you built in the one tree will actually play with the
core code you built in the other tree.  Messy as my solution is, the
regression test step does actually test the intended live combination of
executables.

>> Another problem is that Makefile.shlib isn't designed to build more
>> than one shared library per directory,

> That's the main problem, but fixing it would be very useful in other
> places as well.  I had it on my radar to do something about that.

This would be a good thing.  Got any ideas how to do it?

regards, tom lane


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

[Amit kapila]

On Friday, September 07, 2012 11:19 PM Tom Lane wrote:
Heikki Linnakangas  writes:
>> Would socketpair(2) be simpler?



>I've not done anything yet about the potential security issues
>associated with untrusted libpq connection strings.  I think this
>is still at the proof-of-concept stage; in particular, it's probably
> time to see if we can make it work on Windows before we worry more
>about that.

I have started working on this patch to make it work on Windows. The 3 main 
things to make it work are:

1. Windows equivalent for socketpair - This as suggested previously in this 
thread earlier code of pgpipe can suffice the need. Infact I have checked on 
net as well, most implementations are similar to pgpipe implementation. So I 
prefered to use the existing code which was removed. 

2. Windows equivalent for fork-execv - This part can be done by 
CreateProcess,it can be similar to internal_forkexec except for path where it 
uses shared memory to pass parameters, I am trying to directly pass parameters 
to CreateProcess.

3. Windows equivalent for waitpid - Actually there can be 2 ways to accomplish 
this
  a. use 
waitforsingleobject with process handle, but in some places it is mentioned it 
might not work for all windows versions. Can someone pls confirm about. I shall 
try on my  
   PC to test the 
same.
  b. use existing 
infrastructure of waitpid, however it is not for single process and it might 
need some changes to make it work for single process or may be we can use it 
  directly. However 
currently it is in postmaster.c, so it need to be moved so that we can access 
it from fe-connect.c in libpq as well.
  c. suggest if you 
know of other ways to handle it or which from above 2 would be better?

Some other doubts:

1. does this follow the behavior that admin users will not be allowed to invoke 
postgres child process?
2. to find standalone_backend incase user didn't input, do we need mechanism 
similar to getInstallationPaths()?

Any other comments/suggestions?

With Regards,
Amit Kapila.




-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] build farm machine using mixed results

[Andrew Dunstan]

On 09/09/2012 03:29 AM, Tom Lane wrote:

Peter Eisentraut  writes:

On Sat, 2012-09-08 at 19:54 -0400, Tom Lane wrote:

Anyway, what I notice is that I get different types of failures, but
they are all under ecpg/.  What I think we need to do is insert
.NOTPARALLEL in ecpg/Makefile,

I'd hate that, because the ecpg build is one of the slowest parts of the
build, so de-parallelizing it would slow down everything quite a bit.

There's only one bit of it that's slow, which is the bison build +
preproc.c compile, which is necessarily serial anyway.  So I think
trying to avoid .NOTPARALLEL there is a complete waste of effort.
But if you wanna fix it some other way, step right up.



Yeah. I am going to add a config parameter to the buildfarm to allow 
parallelism for the "make" and "make contrib" stages, but I'm not going 
to release it until this is fixed.


(I suppose this is also a lesson to me that I should not ignore things 
like this that annoy me persistently as I did with these failures until 
Robert Creager started this discussion. It just didn't seem important 
enough.)


cheers

andrew




--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

[Tom Lane]

Amit kapila  writes:
> 1. does this follow the behavior that admin users will not be allowed to 
> invoke postgres child process?

That's an interesting question.  I'm not sure if we'd want to disable
the no-root check on the Unix side, but it might make sense to.  But
this has no bearing on what libpq does, does it?

> 2. to find standalone_backend incase user didn't input, do we need mechanism 
> similar to getInstallationPaths()?

No.  There is no reason at all for libpq to think it is part of a
postgres-supplied program, so it can't use any relative-path tricks,
even if it had the program's argv[0] which it does not.  Either the
compiled-in path works, or the user has to give one.

(But having said that, if Windows provides a way for a DLL to find
out its own filesystem location, maybe relative path from there would
work.)

regards, tom lane


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

[Andres Freund]

Hi,

On Wednesday, September 05, 2012 06:00:18 PM Tom Lane wrote:
> "anara...@anarazel.de"  writes:
> > I am not saying its bad that it is slower, that's absolutely OK. Just
> > that it will take a variable amount of time till you can run pgdump
> > again and its not easily detectable without looping and trying again.
> 
> Well, that's why the proposed libpq code is written to wait for the
> child postgres to exit when closing the connection.
> 
> Admittedly, if you forcibly kill pg_dump (or some other client) and then
> immediately try to start a new one, it's not clear how long you'll have
> to wait. 
Yep, thats the case I was talking about upthread ;)

On Monday, September 03, 2012 11:04:15 PM Andres Freund wrote:
> Don't we expect some more diligence in applications using this against
> letting such a child  continue to run after ctrl-c/SIGTERMing e.g. pg_dump
> in comparison to closing a normal database connection?
I would really expect any postgres supplied core tool to try to handle that 
case. Thats not exactly trivial to do right and requires cooperation of the 
application. Because that requires knowledge about the operation mode in those 
anyway I don't really see an extra libpq call as problematic...

> But so what?  Anything we might do in this space is going to
> have pluses and minuses.
True.

Greetings,

Andres

-- 
Andres Freund   http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] build farm machine using mixed results

[Tom Lane]

Andrew Dunstan  writes:
> On 09/09/2012 03:29 AM, Tom Lane wrote:
>> Peter Eisentraut  writes:
>>> On Sat, 2012-09-08 at 19:54 -0400, Tom Lane wrote:
 Anyway, what I notice is that I get different types of failures, but
 they are all under ecpg/.  What I think we need to do is insert
 .NOTPARALLEL in ecpg/Makefile,

>>> I'd hate that, because the ecpg build is one of the slowest parts of the
>>> build, so de-parallelizing it would slow down everything quite a bit.

>> There's only one bit of it that's slow, which is the bison build +
>> preproc.c compile, which is necessarily serial anyway.  So I think
>> trying to avoid .NOTPARALLEL there is a complete waste of effort.
>> But if you wanna fix it some other way, step right up.

> Yeah. I am going to add a config parameter to the buildfarm to allow 
> parallelism for the "make" and "make contrib" stages, but I'm not going 
> to release it until this is fixed.

Well, why don't we stick .NOTPARALLEL in there for the moment, and then
if Peter thinks of a better solution, he can revert that change in favor
of something cleaner.

I assume we need this for all active branches, if the buildfarm is
going to be stressing it?

regards, tom lane


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] embedded list v2

[Andres Freund]

Hi Alvaro,

Thanks for the review!

On Thursday, September 06, 2012 06:09:35 PM Alvaro Herrera wrote:
> Here's a prettified version of this stuff.  I found one bug in the macro
> ilist_s_head: the test was reversed.
Oh, good catch. I had only used the _unchecked version because my code checked 
that there are elements just some lines before that...

> Also, curiously, the macro had the same name as the struct, so I renamed the
> macro.  I take it you haven't used this macro, so maybe it shouldn't be 
there at all?  Or maybe I completely misread what the macro is supposed to do.
According to my patches here that got introduced by me whe renaming 
_front/back to _head/tail according to Roberts wishes. Sorry for that.

> I also renamed all the structs and functions by changing ilist_s_foo to
> Slist_foo.  Similarly for ilist_d_foo.  This is all mechanical so any
> subsequent patch should be trivial to refresh for this change.
Ok. I concur with robert that a lower case first letter might be better 
readable but again, I don't really care that much.

> I think README.ilist (which is what you had in the comment at the top of
> ilist.h) should be heavily expanded.  I don't find it at all clear.
Hm. I agree :(. Let me have a go when you have a state you find acceptable 
otherwise...

> There were other cosmetic changes, but the implementation is pretty much
> the same you submitted.
Good.

> I didn't look at the other patch you posted, replacing dllist.c usage;
> will do that next to verify that the list implementation works.
Thanks!

Greetings,

Andres
-- 
Andres Freund   http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] build farm machine using mixed results

[Andrew Dunstan]

On 09/09/2012 11:31 AM, Tom Lane wrote:

Yeah. I am going to add a config parameter to the buildfarm to allow
parallelism for the "make" and "make contrib" stages, but I'm not going
to release it until this is fixed.

Well, why don't we stick .NOTPARALLEL in there for the moment, and then
if Peter thinks of a better solution, he can revert that change in favor
of something cleaner.



Works for me



I assume we need this for all active branches, if the buildfarm is
going to be stressing it?




I can restrict it to only modern branches. Didn't we supposedly improve 
support for this during the 9.1 cycle? That seems like a sensible place 
to go back to.


cheers

andrew



--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] build farm machine using mixed results

[Tom Lane]

Andrew Dunstan  writes:
> On 09/09/2012 11:31 AM, Tom Lane wrote:
>> I assume we need this for all active branches, if the buildfarm is
>> going to be stressing it?

> I can restrict it to only modern branches. Didn't we supposedly improve 
> support for this during the 9.1 cycle? That seems like a sensible place 
> to go back to.

Mmm, yeah, the commit history shows several different fixes for parallel
make back in the 9.1 time frame.  I doubt we want to try back-porting
all that work.  Let's just say 9.1 and up, then.

regards, tom lane


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] [PATCH] XLogReader v2

[Andres Freund]

Hi Alvaro, hi all,

On Tuesday, September 04, 2012 09:33:54 PM Alvaro Herrera wrote:
> Excerpts from Andres Freund's message of jue jul 19 06:29:03 -0400 2012:
> > Hi,
> > 
> > Attached is v2 of the patch.
> 
> Hello,
> 
> I gave this code a quick read some days ago.  Here's the stuff I would
> change:
> 
> * There are way too many #ifdef VERBOSE_DEBUG stuff for my taste.  It
> might look better if you had macros such as elog_debug() that are defined
> to empty if VERBOSE_DEBUG is not defined.  (The problem with such an
> approach is that you have to get into the business of creating one macro
> for each different param count, so elog_debug1(), elog_debug2() and so
> on.  It also means you have to count the number of args in each call to
> ensure you're calling the right one.)
Hm. I am generally not very happy with the logging as is. I don't want to rely 
on elog() at all because that means the code suddently depends on just about 
the whole backend which sucks (see my god ulgy makefile hack for that...).

If we were to use that approach is there a platform that stops us from using 
vararg macros? I *think* it is C99...

I though about having a ->log(format, ...) callback, but that would mean loads 
of places add a unneccesary indirect function call :(

> * In the code beautification front, there are a number of cuddled braces
> and improperly indented function declarations.
I never seem to get those right. I really tried to make a pass over the whole 
file correcting them...

> * I noticed that you have the IDENTIFICATION tag wrong in both .c and .h
> files: evidently you renamed the files from readxlog.[ch] to xlogreader.
Yup. Readxlog was the name before someone (I think Simon) reminded me gently 
that it would be better placed in access/transam/ than replication/logical and 
that seemed to conform better to the local naming rules.

> * There are a few elog(PANIC) calls.  I am not sure that's a very good
> idea.  It seems to me that you should be using elog(FATAL) there instead
> ... or do you really want to make the whole server crash?  OTOH if we
> want to make it a true client program, all those elog() calls need to
> go.
The whole error handling needs to be changed. It really depends on the use-
case how failures should be handled. I am just not sure what the best way 
is...

Just a ->error(severity, message, format) callback?

> * XLogReaderRead() seems a bit too long to me.  I would split it with
> auxiliary functions -- say "read a header" and "read a record".  (I
> mentioned this to Andres on IM and he says he tried that but couldn't
> find any nice way to do it.  I may still try to do it.)
When I tried it the code got even more state-machinery with individual parts 
returning status codes and switch()es around that handling the control flow 
from that... Maybe I have stared at it too long to see the way forward.

> * xlogdump's Makefile trick to get all backend object files is ... ugly
> (an understatement).  Really we need the *_desc() routines split so that
> it can use only those functions, and have a client-side replacement for
> StringInfo (discussed elsewhere) and some auxilliary functions such as
> relpathbackend() so that it can compile like a normal client.
You seem to have a good grasp on that in the other thread...

> * why do we pass timeline_id to xlogdump?  I don't see that it's used
> anywhere, but maybe I'm missing something?
Its only unused because xlogdump as it submitted is just a POC hack... You 
need the timeline id to know which files to open. The only reason the parameter 
isn't parsed is that it is currently hardcoded in the callsites for 
XLogDumpXLogRead/write. At least there are FIXMEs arround it...

> This is not a full review.  After a new version with these fixes is
> published (either by Andres or myself) some more review might find more
> serious issues -- I didn't hunt for architectural problems in
> XLogReader.
Have you already started doing anything about it? I can redo a version but 
before we agree on the strategy for logging & error handling the only thing 
that would change is the cuddly braces and the IDENTIFCATION tags...

Thanks!

Andres
-- 
Andres Freund   http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

[HACKERS] a sentence in sepgsql.sgml says 180-degree opposite

[Kohei KaiGai]

I noticed a sentence in sepgsql says 180-degree opposite at:

  When DROP command is executed, drop will be
  checked on the object being removed for each object types.  Permissions
  "will not be" checked for objects dropped indirectly via CASCADE.

This should be "will also be", as our implementation doing.
I'm not sure why I described such a wrong stuff. Anyway, the attached
patch will fix it.

Thanks,
-- 
KaiGai Kohei 


pgsql-v9.2-sepgsql-document-fixup.patch
Description: Binary data

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] build farm machine using mixed results

[Tom Lane]

And the answer is ... it's a gmake bug.  Apparently introduced in 3.82.

http://savannah.gnu.org/bugs/?30653
https://bugzilla.redhat.com/show_bug.cgi?id=835424

So I think .NOTPARALLEL is just masking the true problem, but
nonetheless it's a problem.  And given that the bug report on savannah
has been ignored for two years, we should not hold our breath for a fix
to appear upstream (much less propagate to everyone using 3.82).

regards, tom lane


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] build farm machine using mixed results

[Andrew Dunstan]

On 09/09/2012 02:05 PM, Tom Lane wrote:

And the answer is ... it's a gmake bug.  Apparently introduced in 3.82.

http://savannah.gnu.org/bugs/?30653
https://bugzilla.redhat.com/show_bug.cgi?id=835424

So I think .NOTPARALLEL is just masking the true problem, but
nonetheless it's a problem.  And given that the bug report on savannah
has been ignored for two years, we should not hold our breath for a fix
to appear upstream (much less propagate to everyone using 3.82).




Thanks for pursuing this. Whether or not it masks the underlying 
problem, it's still something we should do, no? In fact, it seems to me 
like this makes it even less worth trying to do anything better.


cheers

andrew



--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] [PATCH] XLogReader v2

[Tom Lane]

Andres Freund  writes:
> On Tuesday, September 04, 2012 09:33:54 PM Alvaro Herrera wrote:
>> * There are way too many #ifdef VERBOSE_DEBUG stuff for my taste.  It
>> might look better if you had macros such as elog_debug() that are defined
>> to empty if VERBOSE_DEBUG is not defined.  (The problem with such an
>> approach is that you have to get into the business of creating one macro
>> for each different param count, so elog_debug1(), elog_debug2() and so
>> on.  It also means you have to count the number of args in each call to
>> ensure you're calling the right one.)

> Hm. I am generally not very happy with the logging as is. I don't want to 
> rely 
> on elog() at all because that means the code suddently depends on just about 
> the whole backend which sucks (see my god ulgy makefile hack for that...).

elog/ereport are already basically macros.  Can't they be redefined for
use in a standalone program, with just minimal backing code?

> If we were to use that approach is there a platform that stops us from using 
> vararg macros? I *think* it is C99...

C90 is still the project standard, and this is a pretty lame reason to
want to change it.

>> * In the code beautification front, there are a number of cuddled braces
>> and improperly indented function declarations.

> I never seem to get those right. I really tried to make a pass over the whole
> file correcting them...

Install pgindent?

regards, tom lane


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Draft release notes complete

[Stefan Kaltenbrunner]

On 09/06/2012 12:13 AM, Peter Eisentraut wrote:
> On 8/29/12 11:52 PM, Andrew Dunstan wrote:
 Why does this need to be tied into the build farm?  Someone can surely
>>> set up a script that just runs the docs build at every check-in, like it
>>> used to work.  What's being proposed now just sounds like a lot of
>>> complication for little or no actual gain -- net loss in fact.
>>
>> It doesn't just build the docs. It makes the dist snapshots too.
> 
> Thus making the turnaround time on a docs build even slower ... ?
> 
>> And the old script often broke badly, IIRC.
> 
> The script broke on occasion, but the main problem was that it wasn't
> monitored.  Which is something that could have been fixed.
> 
>> The current setup doesn't install
>> anything if the build fails, which is a distinct improvement.
> 
> You mean it doesn't build the docs if the code build fails?  Would that
> really be an improvement?

why would we want to publish docs for something that fails to build
and/or fails to pass regression testing - to me code and the docs for it
are a combined thing and there is no point in pushing docs for something
that fails even basic testing...


Stefan


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Draft release notes complete

[Stefan Kaltenbrunner]

On 09/06/2012 03:43 AM, Bruce Momjian wrote:
> On Wed, Sep  5, 2012 at 09:33:35PM -0400, Andrew Dunstan wrote:
>>
>> On 09/05/2012 09:25 PM, Bruce Momjian wrote:
>>> On Wed, Sep  5, 2012 at 09:56:32PM -0300, Alvaro Herrera wrote:
 Excerpts from Tom Lane's message of mié sep 05 20:24:08 -0300 2012:
> Andrew Dunstan  writes:
>> The only reason there is a significant delay is that the administrators
>> have chosen not to run the process more than once every 4 hours. That's
>> a choice not dictated by the process they are using, but by other
>> considerations concerning the machine it's being run on. Since I am not
>> one of the admins and don't really want to take responsibility for it I
>> am not going to second guess them. On the very rare occasions when I
>> absolutely have to have the totally up to date docs I build them myself
>> - it takes about 60 seconds on my modest hardware.
> I think the argument for having a quick docs build service is not about
> the time needed, but the need to have all the appropriate tools
> installed.  While I can understand that argument for J Random Hacker,
> I'm mystified why Bruce doesn't seem to have bothered to get a working
> SGML toolset installed.  It's not like editing the docs is a one-shot
> task for him.
 As far as I understand, Bruce's concern is not about seeing the docs
 built himself, but having an HTML copy published somewhere that he can
 point people to, after applying some patch.  To me, that's a perfectly
 legitimate reason to want to have them quickly.
>>> Correct.  I have always had a working SGML toolset.  If we are not going
>>> to have the developer site run more often, I will just go back to
>>> setting up my own public doc build, like I used to do.  I removed mine
>>> when the official one was more current/reliable --- if that has changed,
>>> I will return to my old setup, and publish my own URL for users to
>>> verify doc changes.
>>
>> How often do you want? After all,
>>  is
>> presumably going to keep pointing to where it now points.
> 
> Well, the old code checked every five minutes, and it rebuilt in 4
> minutes, so there was a max of 10 minutes delay.

the new code gives you a lot more though - it makes sure that the code
the docs refer to actually builds and passes testing, it uses the exact
same toolchain and setup/infrastructure that we build the official
snapshots/tarballs, the official PDFs and reuses an important piece of
our environment - the buildfarm-client.
I'm having a hard time understanding why getting a bit more frequency
for the odd "docs only"+"need to show somebody the html and not the
patch" requirement is really something we "need".


Stefan


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] build farm machine using mixed results

[Tom Lane]

Andrew Dunstan  writes:
> On 09/09/2012 02:05 PM, Tom Lane wrote:
>> And the answer is ... it's a gmake bug.

> Thanks for pursuing this. Whether or not it masks the underlying 
> problem, it's still something we should do, no? In fact, it seems to me 
> like this makes it even less worth trying to do anything better.

Yeah, exactly.

regards, tom lane


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Draft release notes complete

[Stefan Kaltenbrunner]

On 09/07/2012 06:50 PM, Andrew Dunstan wrote:
> 
> On 09/07/2012 09:57 AM, Magnus Hagander wrote:
>> On Thu, Sep 6, 2012 at 1:06 AM, Andrew Dunstan 
>> wrote:
>>>
>>> A complete run of this process takes less than 15 minutes. And as I have
>>> pointed out elsewhere that could be reduced substantially by skipping
>>> certain steps. It's as simple as changing the command line in the
>>> crontab
>>> entry.
>> Is it possible to run it only when the *docs* have changed, and not
>> when it's just a code-commit? meaning, is the detection smart enough
>> for that?
>>
>>
> 
> 
> There is a filter mechanism used in detecting is a run is needed, and in
> modern versions of the client (Release 4.7, one version later than
> guaibasaurus is currently using) it lets you have both include and
> exclude filters. For example, you could have this config setting:
> 
> trigger_include => qr(/doc/src/),
> 
> and it would then only match changed files in the docs tree.
> 
> It's a global mechanism, not per step. So it will run all the steps
> (other than those you have told it to skip) if it finds any files
> changed that match the filter conditions.
> 
> If you do that you would probably want to have two animals, one doing
> docs builds only and running frequently, one doing the dist stuff much
> less frequently.

hmm that might work, but it will only be a bandaid for what people
really seem to advocate for ie "commit triggered" docs builds?


Stefan


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] [PATCH] XLogReader v2

[Andres Freund]

On Sunday, September 09, 2012 08:40:38 PM Tom Lane wrote:
> Andres Freund  writes:
> > On Tuesday, September 04, 2012 09:33:54 PM Alvaro Herrera wrote:
> >> * There are way too many #ifdef VERBOSE_DEBUG stuff for my taste.  It
> >> might look better if you had macros such as elog_debug() that are
> >> defined to empty if VERBOSE_DEBUG is not defined.  (The problem with
> >> such an approach is that you have to get into the business of creating
> >> one macro for each different param count, so elog_debug1(),
> >> elog_debug2() and so on.  It also means you have to count the number of
> >> args in each call to ensure you're calling the right one.)
> > 
> > Hm. I am generally not very happy with the logging as is. I don't want to
> > rely on elog() at all because that means the code suddently depends on
> > just about the whole backend which sucks (see my god ulgy makefile hack
> > for that...).
> 
> elog/ereport are already basically macros.  Can't they be redefined for
> use in a standalone program, with just minimal backing code?
True, its not too hard. I had a *very minimal* version that just forwarded to 
vfprintf before ditching that because I needed to link to *_desc anyway.

Its a bit ugly though if you want to use the same object file for backend and 
standalone code. It means everybody using XLogReader's logging output is tied 
to elog internals.

> > If we were to use that approach is there a platform that stops us from
> > using vararg macros? I *think* it is C99...
> 
> C90 is still the project standard, and this is a pretty lame reason to
> want to change it.
Well, for the most part its a debugging utility, nothing enabled during normal 
builds... But I don't think its an important issue, if it comes to that we can 
do it just the same as elog.h does it. I.e. using a parameterless macro.

> >> * In the code beautification front, there are a number of cuddled braces
> >> and improperly indented function declarations.
> > 
> > I never seem to get those right. I really tried to make a pass over the
> > whole file correcting them...
> 
> Install pgindent?
I have, but it so often generates too much noise in unrelated parts that I 
stopped bothering. Which is a bad excuse in this case because its a new 
file...

Andres

-- 
 Andres Freund http://www.2ndQuadrant.com/
 PostgreSQL Development, 24x7 Support, Training & Services


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] build farm machine using mixed results

[Peter Eisentraut]

On Sun, 2012-09-09 at 14:05 -0400, Tom Lane wrote:
> And the answer is ... it's a gmake bug.  Apparently introduced in 3.82.
> 
> http://savannah.gnu.org/bugs/?30653
> https://bugzilla.redhat.com/show_bug.cgi?id=835424
> 
> So I think .NOTPARALLEL is just masking the true problem, but
> nonetheless it's a problem.  And given that the bug report on savannah
> has been ignored for two years, we should not hold our breath for a fix
> to appear upstream (much less propagate to everyone using 3.82).

So does the issue go away when using GNU make 3.81?




-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] build farm machine using mixed results

[Peter Eisentraut]

On Sun, 2012-09-09 at 14:57 -0400, Tom Lane wrote:
> Andrew Dunstan  writes:
> > On 09/09/2012 02:05 PM, Tom Lane wrote:
> >> And the answer is ... it's a gmake bug.
> 
> > Thanks for pursuing this. Whether or not it masks the underlying 
> > problem, it's still something we should do, no? In fact, it seems to me 
> > like this makes it even less worth trying to do anything better.
> 
> Yeah, exactly.

But then the answer could be, if you want to use parallel make, use a
version that's not broken.



-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] build farm machine using mixed results

[Tom Lane]

Peter Eisentraut  writes:
> On Sun, 2012-09-09 at 14:05 -0400, Tom Lane wrote:
>> So I think .NOTPARALLEL is just masking the true problem, but
>> nonetheless it's a problem.  And given that the bug report on savannah
>> has been ignored for two years, we should not hold our breath for a fix
>> to appear upstream (much less propagate to everyone using 3.82).

> So does the issue go away when using GNU make 3.81?

I couldn't reproduce it in a few tries on my Mac laptop, which has 3.81.
But that's not necessarily a comparable test, given the different
hardware, OS, and compiler version.  I'd put more faith in the multiple
statements in the upstream bugs that people weren't seeing this with
3.81.

regards, tom lane


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] build farm machine using mixed results

[Tom Lane]

Peter Eisentraut  writes:
> But then the answer could be, if you want to use parallel make, use a
> version that's not broken.

That's not a terribly practical answer for people who use the "make"
supplied by their OS vendor, which is approximately 99.9% of people.
It's even less practical for packagers, who don't have a choice about
what tool set to use.

Even if I wanted to use a locally-patched make, I'm not sure I'd trust a
patch that doesn't seem to have been signed off on by any actual gmake
developer or maintainer.  That sort of cure is frequently worse than the
disease.

regards, tom lane


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] build farm machine using mixed results

[Andrew Dunstan]

On 09/09/2012 05:00 PM, Tom Lane wrote:

Peter Eisentraut  writes:

But then the answer could be, if you want to use parallel make, use a
version that's not broken.

That's not a terribly practical answer for people who use the "make"
supplied by their OS vendor, which is approximately 99.9% of people.
It's even less practical for packagers, who don't have a choice about
what tool set to use.

Even if I wanted to use a locally-patched make, I'm not sure I'd trust a
patch that doesn't seem to have been signed off on by any actual gmake
developer or maintainer.  That sort of cure is frequently worse than the
disease.





Right.

Meanwhile, I have buildfarm animal crake building with the experimental 
buildfarm code, and so far the results look good.


cheers

andrew



--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Minor document updates

[Etsuro Fujita]

> From: Robert Haas [mailto:robertmh...@gmail.com]

>  wrote:
> > I noticed the syntax of the \copy command in the psql reference page is an
> old
> > style.  ISTM it's better to update the document.  Please find attached a
> patch.
> 
> Seems reasonable to me.  Committed.

Thank you.  Sorry for the delay.

Best regards,
Etsuro Fujita



-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

[Amit Kapila]

On Sunday, September 09, 2012 8:46 PM Tom Lane wrote:
Amit kapila  writes:
>> 1. does this follow the behavior that admin users will not be allowed to
invoke postgres child process?

> That's an interesting question.  I'm not sure if we'd want to disable
> the no-root check on the Unix side, but it might make sense to.  But
> this has no bearing on what libpq does, does it?

  No, it has no bearing with what libpq does. Its only related to postgres
invocation, because as soon as postgres get invoked, it first checks about
root. If we want the root check can be avoided easily by checking argv.

>> 2. to find standalone_backend incase user didn't input, do we need
mechanism similar to getInstallationPaths()?

> No.  There is no reason at all for libpq to think it is part of a
> postgres-supplied program, so it can't use any relative-path tricks,
> even if it had the program's argv[0] which it does not.  Either the
> compiled-in path works, or the user has to give one.

> (But having said that, if Windows provides a way for a DLL to find
> out its own filesystem location, maybe relative path from there would
> work.)

At first, I shall go ahead with the current way (Either the
compiled-in path works, or the user has to give one.).




-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers