Re: [HACKERS] Failed assertion during recovery of partial WAL file

2010-02-10 Thread Heikki Linnakangas
Fujii Masao wrote:
 This assertion failure derives from the recent refactoring of ReadRecord().
 Which changed the startup process so as to re-fetch the last applied WAL
 record at the end of recovery from the buffer instead of the WAL file if it's
 stored in the buffer. In this case, the last applied WAL file remains closed.
 So readFile (which ought to have been the file descriptor of that WAL file)
 might be -1 in exitArchiveRecovery().
 
 In the now, that assertion is obsolete. So I attached the patch that removes
 the assert() from exitArchiveRecovery().

Committed.

-- 
  Heikki Linnakangas
  EnterpriseDB   http://www.enterprisedb.com

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Patch: Remove gcc dependency in definition of inline functions

2010-02-10 Thread Kurt Harriman

On 1/18/2010 4:42 PM, Tom Lane wrote:

I think including MSVC in the set of compilers targeted by a configure
test is just a waste of code.  It's more likely to confuse people than
help them.


Ok, I have taken that out, and instead put the right stuff for MSVC into
pg_config.h.win32.

Regards, ... kurt

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Patch: Remove gcc dependency in definition of inline functions

2010-02-10 Thread Kurt Harriman

On 1/18/2010 11:48 PM, Peter Eisentraut wrote:

On mån, 2010-01-18 at 16:34 -0800, Kurt Harriman wrote:

MSVC does warn about unused static inline functions.  The warning
is prevented by using __forceinline instead of __inline.


Hmm, but forceinline is not the same as inline.  Are we confident that
forcing inlining is not going to lead to disadvantages?  Has this been
measured?

Is there not a setting to disable this particular warning.  I read that
MSVC has various ways to set that sort of thing.


On further investigation, plain __inline works alright, at least in
Visual Studio 2005.  The warning for unused static inline functions
only comes up when compiling C++, not C.  I've submitted a revised
patch, no longer using __forceinline.

Regards, ... kurt


--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: I: [HACKERS] About Our CLUSTER implementation is pessimal patch

2010-02-10 Thread Leonardo F
Hi all,


while testing the seq scan + sort CLUSTER implementation, I've found a
bug in write/readtup_rawheap.

The functions are needed by the sort part.
The code in 

http://archives.postgresql.org/pgsql-hackers/2008-08/msg01371.php

didn't copy the whole tuple, but only the HeapTuple header: the t_data
part wasn't copied (at least, that's my understanding), The original code
is at the bottom of the email. It didn't work (the table wasn't fully clustered
at the end of the CLUSTER command).

So I modified write/readtup_rawheap: they are supposed to store/retrieve
both the fixed part of HeapTupleData, plus the variable part t_data.
But a get a lot of:
WARNING:  problem in alloc set TupleSort: detected write past chunk end
  in block 0x96853f0, chunk 0x968723c
WARNING:  problem in alloc set TupleSort: detected write past chunk end 
  in block 0x96853f0, chunk 0x96872c8
warnings when calling tuplesort_end  and some of the data gets garbled
after the CLUSTER command.


What am I doing wrong? Using the debugger, data coming out of
readtup_rawheap seems fine...

I *really* need your help here...


static void
writetup_rawheap(Tuplesortstate *state, int tapenum, SortTuple *stup)
{
HeapTupletuple = (HeapTuple) stup-tuple;
tuple-t_len += sizeof(HeapTupleData); /* write out the header as well */

LogicalTapeWrite(state-tapeset, tapenum,
tuple, sizeof(HeapTupleData));
LogicalTapeWrite(state-tapeset, tapenum, tuple-t_data, 
 tuple-t_len-sizeof(HeapTupleData));
if (state-randomAccess)/* need trailing length word? */
  LogicalTapeWrite(state-tapeset, tapenum,
 tuple, sizeof(tuple-t_len));

FREEMEM(state, GetMemoryChunkSpace(tuple));
heap_freetuple(tuple);
}

static void
readtup_rawheap(Tuplesortstate *state, SortTuple *stup,
int tapenum, unsigned int tuplen)
{
HeapTupletuple = (HeapTuple) palloc(sizeof(HeapTupleData));
tuple-t_data = (HeapTupleHeader) palloc(tuplen-sizeof(HeapTupleData));

USEMEM(state, GetMemoryChunkSpace(tuple));

tuple-t_len = tuplen - sizeof(HeapTupleData);
if (LogicalTapeRead(state-tapeset, tapenum, tuple-t_self, 
sizeof(HeapTupleData)-sizeof(tuplen))
   != sizeof(HeapTupleData)-sizeof(tuplen))
  elog(ERROR, unexpected end of data);
if (LogicalTapeRead(state-tapeset, tapenum, tuple-t_data, tuple-t_len) != 
tuple-t_len)
  elog(ERROR, unexpected end of data);
if (state-randomAccess)/* need trailing length word? */
 if (LogicalTapeRead(state-tapeset, tapenum, tuplen,
   sizeof(tuplen)) != sizeof(tuplen))
  elog(ERROR, unexpected end of data);

stup-tuple = tuple;
/* set up first-column key value */
if (state-indexInfo-ii_Expressions == NULL)
{
/* no expression index, just get the key datum value */
stup-datum1 = heap_getattr((HeapTuple) stup-tuple,
state-indexInfo-ii_KeyAttrNumbers[0],
state-tupDesc,
stup-isnull1);
}
else
{
  [...] expression index part, removed for clarity 
}
}




Original code:

static void
writetup_rawheap(Tuplesortstate *state, int tapenum, SortTuple *stup)
{
HeapTuple   tuple = (HeapTuple) stup-tuple;
tuple-t_len += HEAPTUPLESIZE; /* write out the header as well */

LogicalTapeWrite(state-tapeset, tapenum,
tuple, tuple-t_len);

if (state-randomAccess)/* need trailing length word? */
 LogicalTapeWrite(state-tapeset, tapenum,
   tuple, sizeof(tuple-t_len));

FREEMEM(state, GetMemoryChunkSpace(tuple));
heap_freetuple(tuple);
}

static void
readtup_rawheap(Tuplesortstate *state, SortTuple *stup,
int tapenum, unsigned int tuplen)
{
HeapTuple   tuple = (HeapTuple) palloc(tuplen);

USEMEM(state, GetMemoryChunkSpace(tuple));

tuple-t_len = tuplen - HEAPTUPLESIZE;
if (LogicalTapeRead(state-tapeset, tapenum, tuple-t_self, 
  tuplen-sizeof(tuplen)) != tuplen-sizeof(tuplen))
  elog(ERROR, unexpected end of data);
if (state-randomAccess)/* need trailing length word? */
if (LogicalTapeRead(state-tapeset, tapenum, tuplen,
sizeof(tuplen)) != sizeof(tuplen))
elog(ERROR, unexpected end of data);

stup-tuple = tuple;
/* set up first-column key value */
stup-datum1 = heap_getattr(tuple,
state-scanKeys[0].sk_attno,
state-tupDesc,
stup-isnull1);
}





-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


[HACKERS] Re: [COMMITTERS] pgsql: Make standby server continuously retry restoring the next WAL

2010-02-10 Thread Fujii Masao
On Wed, Feb 10, 2010 at 4:32 PM, Heikki Linnakangas
heikki.linnakan...@enterprisedb.com wrote:
 Hmm, so after running restore_command, check the file size and if it's
 too short, treat it the same as if restore_command returned non-zero?

Yes, only in standby mode case. OTOH I think that normal archive recovery
should treat it as a FATAL error.

 And it will be retried on the next iteration. Works for me, though OTOH
 it will then fail to complain about a genuinely WAL file that's
 truncated for some reason. I guess there's no way around that, even if
 you have a script as restore_command that does the file size check, it
 will have the same problem.

Right. But the server in standby mode also needs to complain about that?
We might be able to read completely such a WAL file that looks truncated
from the primary via SR, or from the archive after a few seconds. So it's
odd for me to give up continuing the standby only by finding the WAL file
whose file size is short. I believe that the warm standby (+ pg_standby)
also is based on that thought.

Regards,

-- 
Fujii Masao
NIPPON TELEGRAPH AND TELEPHONE CORPORATION
NTT Open Source Software Center

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Patch: Remove gcc dependency in definition of inline functions

2010-02-10 Thread Kurt Harriman

On 1/19/2010 8:01 AM, Peter Eisentraut wrote:

On tis, 2010-01-19 at 01:29 -0800, Kurt Harriman wrote:

On 1/18/2010 11:48 PM, Peter Eisentraut wrote:
We have some existing inline functions in .c files.  These can be
more complicated, so it might be ok if the compiler decides to
leave them out-of-line.  And they are never unreferenced, so
suppression of unused-function warnings is not necessary and
perhaps not wanted.  To leave these functions undisturbed, my patch
doesn't redefine the inline keyword; instead it adds a new #define
PG_INLINE for use in header files where unused-function warnings
need to be suppressed.


One principle that I suppose should have been made more explicit is that
-- in my mind -- we should avoid littering our code with nonstandard
constructs in place of standard constructs.  Because the next generation
of developers won't know what PG_INLINE is and why we're not using plain
inline, even if we document it somewhere.


Unfortunately, to switch to an out-of-line function where inlining is
not supported, a separate preprocessor symbol is needed.  The already
existing inline define can't be used to test whether the compiler
supports inlining.  inline is defined as empty if configure doesn't
detect an acceptable variant of inline.  It is left undefined if
the compiler accepts the standard spelling inline.  But the C
preprocessor offers no way to test whether a symbol is defined as
empty.  #if can compare integers but not strings.

Everyone seems to hate the name PG_INLINE, so I've changed it to
inline_quietly, which is more descriptive.  Anyone who greps for
the definition of inline_quietly will find the comment in pg_config.h
telling what it is and how it should be used, as well as the examples
in pg_list.h and palloc.h.  Also it is explained, I hope clearly, in
the proposed CVS commit comment and in this email thread.


Then just replace in those two locations __GNUC__ by __GNUC__ ||
__MSVC__ (or whatever the symbol is).  Or if you want to make it extra
nice, create a symbol somewhere like in c.h that reads

#define USE_INLINE __GNUC__ || __MSVC__


That would just add to the compiler-specific preprocessor logic
to be duplicated in every header file in which inline functions
are defined.  I'm trying to factor out that compiler dependency
into a central location: pg_config.h.

Regards,
... kurt

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Patch: Remove gcc dependency in definition of inline functions

2010-02-10 Thread Kurt Harriman

On 12/16/2009 8:40 AM, Tom Lane wrote:

Alvaro Herreraalvhe...@commandprompt.com  writes:

IIRC Kurt was also on about getting rid of some ugly macros that could
instead be coded as inline functions (fastgetattr for example)


I'd just bounce that as useless activity.  If they are macros now,
and work, the only possible effects of changing them are negative.


fastgetattr has just been changed by Robert Haas on 10 Jan 2010:
Remove partial, broken support for NULL pointers when fetching attributes.

Changing fastgetattr to an inline function would make it

- easier to read, modify, and review for correctness

- debuggable: could set breakpoints, single-step, display the arguments

- profilable

and would make compiler warnings appear at the definition
rather than at every invocation.

Regards,
... kurt

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Writeable CTEs and empty relations

2010-02-10 Thread Marko Tiikkaja
On 2010-02-10 03:20 +0200, Tom Lane wrote:
 Marko Tiikkaja marko.tiikk...@cs.helsinki.fi writes:
 On 2010-02-10 02:19 +0200, Tom Lane wrote:
 You still haven't explained why it's a good idea to change the snapshot
 after the executor has started.  Right at the moment I'm prepared to
 reject the patch on that ground alone.
 
 The patch only touches the snapshot's curcid.  That's needed to allow
 the queries see the tuples of the DML WITHs ran before that particular
 query.
 
 ... and they will also see, for example, tuples inserted by triggers
 fired by those queries.  I thought the plan for this feature was to
 store and re-read the RETURNING data, not to go back and re-read the
 underlying tables.

I originally suggested this approach about four months ago.  During this
whole time you haven't expressed any concerns about this, but suddenly
it's a reason to reject the patch?

Anyway, if we want to avoid modifying the snapshot, we can't bump the
CID between queries.  I haven't thought this through, but it seems like
this could work.  However, none of the WITH queries can see the previous
statement's tuples, which means that someone may be suprised when they
try to modify the same tuples twice just to discover that only the first
modification took place.  I don't see this as a huge problem though.

This will also solve the problem I started this thread for and makes the
patch smaller, simpler and even a bit prettier.  Unless someone sees a
problem with this approach, I'm going to make the change.


Regards,
Marko Tiikkaja

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Patch: Remove gcc dependency in definition of inline functions

2010-02-10 Thread Kurt Harriman

On 12/16/2009 8:21 AM, Tom Lane wrote:

I would only suggest that the cleanest coding would be

#ifdef USE_INLINE

static inline foo(...) ...

#else

... non-inline definition of foo

#endif

ie, go ahead and rely on autoconf's definition (if any) of inline
and add a policy symbol USE_INLINE to determine whether to use it.


That would work for gcc and MSVC.  But it wouldn't allow for
configuring an alternative keyword (like __forceinline) or added
magic (like inserting an __attribute__ or __declspec) to silence
warnings for some compiler which we don't know about yet.


The proposed PG_INLINE coding conflates the symbol needed in the code
with the policy choice.


Everyone is familiar with this idiom: first test whether a pointer
is NULL, before dereferencing it.  We don't use a separate flag to
say whether the pointer is NULL.


Another possibility would be to call the policy symbol HAVE_INLINE,
but that (a) risks collision with a name defined by autoconf built-in
macros, and (b) looks like it merely indicates whether the compiler
*has* inline, not that we have made a choice about how to use it.


In the new 3rd edition of the patch, I've changed the name to
inline_quietly.  If not too many people hate this new name, I
can undertake a new career naming tablets for Apple. :)

Regards,
... kurt

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Some belated patch review for Buffers explain analyze patch

2010-02-10 Thread Greg Stark
On Wed, Feb 10, 2010 at 3:59 AM, Robert Haas robertmh...@gmail.com wrote:
 Yes.  We could add every bell and whistle imaginable to the text
 format and it still would not begin to approach the verbosity of the
 machine-readable formats.  Have you looked at them on a complex plan?
 They are really, really long, and in many cases quite unreadable by
 human beings.

Well I complained about that at the time. At least for XML that's
becuase you chose to make separate tags on separate lines for every
single value instead of just having one tag for estimates, one for
actual, and using attributes for the different values.

Incidentally my patch to use getrusage also reraises the other issue I
complained about. There's no organization to the tags so a tool to
view them can't make heads or tails of them without knowing in advance
what they all are. For example pgadmin can't make a tree with
expandable subtrees for each data source since it doesn't know which
attributes are related to which unless it hard codes knowledge about
them.

Tom pointedly ignored the getrusage thing -- presumably because it's
well past the time to consider new patches. But I didn't post the
example to discuss it, I posted it because I think it can inform these
earlier decisions. Ie, seeing that data might make it clearer which
things we want to be per-loop and which we want totals for. And
knowing that we'll likely have a place to hang the total wall clock
time if we want might make the decision easier.

-- 
greg

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


[HACKERS] Parameter name standby_mode

2010-02-10 Thread Joachim Wieland
We want to teach people that Hot Standby and Streaming Replication are
two different features. However, Streaming Replication calls its main
parameter standby_mode which reminds more of Hot Standby than of
Streaming Replication.

People could also run a warm standby without streaming replication,
which would result in a standby that has standby_mode = 'off'.

I found the parameter name confusing and I'd vote for changing its name.


Joachim

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Parameter name standby_mode

2010-02-10 Thread Heikki Linnakangas
Joachim Wieland wrote:
 We want to teach people that Hot Standby and Streaming Replication are
 two different features.

I'm not sure about that, actually. Now that they're both in the tree,
they work nicely together and many users will think of them as one.

 However, Streaming Replication calls its main
 parameter standby_mode which reminds more of Hot Standby than of
 Streaming Replication.
 
 People could also run a warm standby without streaming replication,
 which would result in a standby that has standby_mode = 'off'.

If they want to implement the warm standby using the (new) built-in
logic to keep retrying restore_command, they would set
standby_mode='on'. standby_mode='on' doesn't imply streaming replication.

If you want to use pg_standby or similar tools, then you would indeed
set standby_mode='off', but I think that makes sense because you're
implementing the standby functionality outside the server in that case.

-- 
  Heikki Linnakangas
  EnterpriseDB   http://www.enterprisedb.com

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] About psycopg2 (by its author)

2010-02-10 Thread Federico Di Gregorio
On 09/02/2010 23:37, Greg Smith wrote:
[snip]
 So the logical choice is plain LGPL3. I am open to motivated
 suggestions about other
 licenses but I'll ignore such crap as BSD is more open than LGPL.
   
 
 I agree with your general logic and while I can't speak for everyone, I
 would be happy enough with a LGPL3 licensed psycopg (obviously
 addressing the usual OpenSSL mess) to pull the license issue off the top
 of the list as a major problem preventing broader deployment of
 psycopg.  The main two points of contention seemed to be your unique
 customizations to the license, which make a lot of legal people nervous,
 and even worse that they were so clearly limiting many types of
 commercial use.  I hope you'd appreciate that while you have have
 legitimate reasons for your license choices, ones in that form are
 likely to remind this community of the split open/commercial licenses as
 seen in products like MySQL, and we've watch that combination lead
 toward a less open community than this one wants to be.

As I said before I agree that a license that grow so many exceptions
during its lifetime is bad and I am ready to change it. But note that it
never intended to be a split open/commercial license: the final phrase
is just an acknowledgment that some companies will always ask for a
customized proprietary license, no matter the actual license [ok, unless
the actual license is BSD ;)]

 As for arguments against the LGPL, the main one I care about is that
 you're more likely to have businesses who hire people adopt a product if
 it's BSD or MIT licensed.  I make a decent chunk of my living doing
 support and customization work on open-source projects.  Anything that
 has a GPL license attached is something I'm less likely to incorporate
 into custom project work I do, because it decreases the number of
 businesses who are then interested in it.  This is mainly because they
 have to incorporate all that background into their credits list for
 aggregate works, and that concern inevitably opens up more questions
 better avoided about the implications of the software being bundled.
 
 I'm more concerned about increasing the market I can provide such
 solutions to than I am about people stealing my work, crediting me, or
 not sharing their own customizations.  So my preference for BSD-ish
 licenses is a pragmatic one rooted in business goals.  If you wanted to
 improve your odds of companies adopting psycopg for projects that might
 then lead to them hiring you for support or improvements to the
 software, I'd suggest that using the GPL or even the LGPL is actually
 doing the exact opposite of that.  If your goals are more about
 releasing proper free software in the original Stallman inspired sense
 of the word, the LGPL3 might be exactly the right license for you.

I understand this. In fact my goals are more about releasing free
software than having companies hiring us for psycopg development. And
sincerely I don't care about people stealing my work but I do care
about customers (even not related to me) receiving free software and be
correctly informed of their rights when the product is based on free
software.

That's why we (as a company) release all our software as GPL or LGPL.
(Note that I don't have any problems with other licenses, for example
when sending patches for products we use. It is just that I better like
copyleft licenses for software I write myself.)

So, be it. Next version of psycopg2 will be released using LGPL3 (plus
ssl exceptions) and I hope this would solve all current licensing problems.

[snip]
 If the license issues get sorted out as you plan, that part I think we
 can end up helping out with using our infrastructure.  You might note
 Marko Kreen already created http://wiki.postgresql.org/wiki/Psycopg to
 start working on just that.  I think we'd all be fine with continuing to
 expand on that rather than worry about your revamping the initd.org site
 just to address the documentation goals we have.  And we would certainly
 want to work more closely with you and your other contributors on that,
 to make sure everything is accurate and complete.

initd.org will get a facelift first or later. But even if we could have
a psycopg web page ready tomorrow having a page dedicated to psycopg on
wiki.postgresql.org is great.

Also, piro is doing a great work on psycopg2 documentation:

http://piro.develer.com/psycopg2-doc/

make sure to check it out.

federico

-- 
Federico Di Gregorio   f...@initd.org
 I porcellini di terra sono davvero Crostacei! Non lo sapevo!
  Certo che sono crostacei, hanno la crosta!
  Allora la pizza è un crostaceo?!   -- discorso all'ESC2k07



signature.asc
Description: OpenPGP digital signature


Re: I: [HACKERS] About Our CLUSTER implementation is pessimal patch

2010-02-10 Thread Heikki Linnakangas
Leonardo F wrote:
 static void
 writetup_rawheap(Tuplesortstate *state, int tapenum, SortTuple *stup)
 {
 HeapTupletuple = (HeapTuple) stup-tuple;

I think you're confusing HeapTuple and HeapTupleHeader. SortTuple-tuple
field should point to a HeapTupleHeader, not a HeapTuple.

The right mental model is that HeapTupleHeader is a physical tuple, one
that you store on disk for example. A HeapTuple is an in-memory wrapper,
or pointer if you will, to a HeapTupleHeader, and holds some additional
information on the tuple that's useful when operating on it.

To add to the confusion, MinimalTuple is a shortened counterpart of
HeapTuple*Header*, not HeapTuple. And SortTuple is an in-memory wrapper
similar to HeapTuple, containing additional information on the tuple
that helps with sorting.

I didn't look at the rest of the code in detail, but I think that's
where your problems are stemming from.

-- 
  Heikki Linnakangas
  EnterpriseDB   http://www.enterprisedb.com

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: I: [HACKERS] About Our CLUSTER implementation is pessimal patch

2010-02-10 Thread Leonardo F
 I think you're confusing HeapTuple and HeapTupleHeader. SortTuple-tuple
 field should point to a HeapTupleHeader, not a HeapTuple.


Mmh, I don't get it: that would mean I might be using more info than required,
but I still don't understand why it's not working... at the end, CLUSTER is 
going
to need full HeapTuple(s) (if I'm not mistaken).
SortTuple-tuple can be any of MinimalTuple, IndexTuple, even a Datum.
So I added my own read/write_rawtuple (I'm not that good, they were in the
original patch and I copypasted them).

I can write and read those tuples (using some Asserts I think everything goes

as expected in write/readtup_rawheap). But when calling tuplesort_getrawtuple,
after some right tuples, the call to tuplesort_gettuple_common fails because
the lines:

/* pull next preread tuple from list, insert in heap */
newtup = state-memtuples[tupIndex];


give a wrong initialized newtup (in other words, newtup is random memory).
Since write/readtup_rawheap seems fine, I can't understand what's going on...

I write the HeapTuples with:

(in writetup_rawheap):
HeapTupletuple = (HeapTuple) stup-tuple;
tuple-t_len += HEAPTUPLESIZE; /* write out the header as well */

LogicalTapeWrite(state-tapeset, tapenum,
  tuple, HEAPTUPLESIZE);
LogicalTapeWrite(state-tapeset, tapenum, tuple-t_data, 
   tuple-t_len-HEAPTUPLESIZE);



then I read them with:
(in readtup_rawheap):
HeapTupletuple = (HeapTuple) palloc(tuplen);
USEMEM(state, GetMemoryChunkSpace(tuple));

tuple-t_len = tuplen - HEAPTUPLESIZE;
if (LogicalTapeRead(state-tapeset, tapenum, tuple-t_self, 
  HEAPTUPLESIZE-sizeof(tuplen)) != HEAPTUPLESIZE-sizeof(tuplen))
  elog(ERROR, unexpected end of data);
if (LogicalTapeRead(state-tapeset, tapenum, tuple-t_data, tuple-t_len) != 
tuple-t_len)
  elog(ERROR, unexpected end of data);




-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Some belated patch review for Buffers explain analyze patch

2010-02-10 Thread Robert Haas
On Wed, Feb 10, 2010 at 5:39 AM, Greg Stark st...@mit.edu wrote:
 On Wed, Feb 10, 2010 at 3:59 AM, Robert Haas robertmh...@gmail.com wrote:
 Yes.  We could add every bell and whistle imaginable to the text
 format and it still would not begin to approach the verbosity of the
 machine-readable formats.  Have you looked at them on a complex plan?
 They are really, really long, and in many cases quite unreadable by
 human beings.

 Well I complained about that at the time. At least for XML that's
 becuase you chose to make separate tags on separate lines for every
 single value instead of just having one tag for estimates, one for
 actual, and using attributes for the different values.

I had some reasons for that decision which I still believe are valid -
namely, not wanting to have completely custom code for every output
format.  You're free to disagree with that decision, of course.

That's also definitely not the only problem.  For example, EXPLAIN
(VERBOSE, FORMAT JSON) is often ridiculously wide because each output
list is printed on a single line.  The text format has that problem
too, but it's much worse in JSON because of the extra punctuation and
separators.  Now I could fix that by printing each output item on its
own line, but then the output would be even more ridiculously long
than it is already.  Probably the only readable way to do this is to
add some logic to keep printing items on a single line until we run
out of columns, and then jump down to the next line.  But I have a
feeling that a patch to add word-wrap logic to a machine-readable
output format would be DOA as far as Tom is concerned, and I can't say
I disagree.

 Incidentally my patch to use getrusage also reraises the other issue I
 complained about. There's no organization to the tags so a tool to
 view them can't make heads or tails of them without knowing in advance
 what they all are. For example pgadmin can't make a tree with
 expandable subtrees for each data source since it doesn't know which
 attributes are related to which unless it hard codes knowledge about
 them.

I would guess that it's possible to find a way to fix or improve on
this, but I'm not sure whether there's support for doing that at this
point in the cycle, or agreement on what it should look like.

I'm aware that there are a number of people who are not happy with the
way I implemented that patch for a number of different reasons.  Of
course, I like it best when everyone thinks that my work is the bees
knees, but the threads about this patch were long and very many people
expressed very many mutually contradictory opinions about how I ought
to change things.  I did the best I could to come up with something
that was useful to me and acceptable to the community.

Or as I said at the time... nobody liked anything about the patch
except that they didn't have to write it.

...Robert

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


[HACKERS] [PATCH] Output configuration status after ./configure run.

2010-02-10 Thread Priit Laes
Hi!

This patch enables showing configure status at the end of ./configure
run and thus makes ./configure process a bit easier to follow (in the
sense of what features are actually enabled).

Päikest,
Priit
From c6ab747e581c7ebeb954b2ccb4dbd932e1a61de7 Mon Sep 17 00:00:00 2001
From: Priit Laes pl...@plaes.org
Date: Wed, 10 Feb 2010 08:14:43 +0200
Subject: [PATCH] Output configuration status after ./configure run.

---
 configure.in |   30 ++
 1 files changed, 30 insertions(+), 0 deletions(-)

diff --git a/configure.in b/configure.in
index 5a27d3a..8bf8f1d 100644
--- a/configure.in
+++ b/configure.in
@@ -1867,3 +1867,33 @@ AC_CONFIG_HEADERS([src/interfaces/ecpg/include/ecpg_config.h],
   [echo src/interfaces/ecpg/include/stamp-h])
 
 AC_OUTPUT
+
+echo 
+PostgreSQL was configured with the following options:
+
+64-bit integer datetimes  : $enable_integer_datetimes
+Table block size  : ${blocksize}kB
+Table relation size   : ${segsize}GB
+WAL block size: ${wal_blocksize}kB
+
+Profiling support : $enable_profiling
+Code Coverage support : $enable_coverage
+DTrace support: $enable_dtrace
+
+Perl (PL/Perl): $with_perl
+Python (PL/Python): $with_python
+Tcl (PL/Tcl)  : $with_tcl
+
+GSSAPI support: $with_gssapi
+Kerberos 5 support: $with_krb5 (srvname: $with_krb_srvnam)
+PAM support   : $with_pam
+LDAP support  : $with_ldap
+Bonjour support   : $with_bonjour
+OpenSSL support   : $with_openssl
+Readline/Libedit support  : $with_readline/$with_libedit_preferred
+
+OSSP UUID library support : $with_ossp_uuid
+XML support (libXML2) : $with_libxml
+XSLT support  : $with_libxslt
+zlib support  : $with_zlib
+
-- 
1.6.6.1


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Re: [COMMITTERS] pgsql: Make standby server continuously retry restoring the next WAL

2010-02-10 Thread Aidan Van Dyk
* Heikki Linnakangas heikki.linnakan...@enterprisedb.com [100210 02:33]:
 
 Hmm, so after running restore_command, check the file size and if it's
 too short, treat it the same as if restore_command returned non-zero?
 And it will be retried on the next iteration. Works for me, though OTOH
 it will then fail to complain about a genuinely WAL file that's
 truncated for some reason. I guess there's no way around that, even if
 you have a script as restore_command that does the file size check, it
 will have the same problem.

But isn't this something every current PITR archive already works
around...  Everybody doing PITR archives already know the importance of
making the *appearance* of the WAL filename in the archive atomic.

Don't docs warn about plain cp not being atomic and allowing short
files to appear in the archive... 

I'm not sure why streaming recovery suddenly changes the requirements...

a.

-- 
Aidan Van Dyk Create like a god,
ai...@highrise.ca   command like a king,
http://www.highrise.ca/   work like a slave.


signature.asc
Description: Digital signature


Re: I: [HACKERS] About Our CLUSTER implementation is pessimal patch

2010-02-10 Thread Leonardo F
I think I've found the problem:

tuple-t_data wasn't at HEAPTUPLESIZE distance from tuple.
I guess the code makes that assumption somewhere, so I had
to do:

tuple-t_data = (HeapTupleHeader) ((char *) tuple + 
 HEAPTUPLESIZE);

Now that test works! Hope I don't find any more problems...



Leonardo





-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Some belated patch review for Buffers explain analyze patch

2010-02-10 Thread Greg Stark
On Wed, Feb 10, 2010 at 1:26 PM, Robert Haas robertmh...@gmail.com wrote:

 For example, EXPLAIN (VERBOSE, FORMAT JSON) is often ridiculously
 wide because each output list is printed on a single line.

Perhaps this is just a terminology difference but it seems
ridiculously *narrow* to me:

  QUERY PLAN
--
 [   +
   { +
 Plan: {   +
   Node Type: Seq Scan,  +
   Relation Name: i, +
   Schema: public,   +
   Alias: i, +
   Startup Cost: 0.00, +
   Total Cost: 63344.86,   +
   Plan Rows: 2399986, +
   Plan Width: 101,+
   Actual Startup Time: 0.115, +
   Actual Total Time: 3259.839,+
   Actual Rows: 240,   +
   Actual Loops: 1,+
   Output: [i]   +
 },  +
 Triggers: [   +
 ],  +
 Total Runtime: 5977.303   +
   } +
 ]
(1 row)


 Or as I said at the time... nobody liked anything about the patch
 except that they didn't have to write it.

I know I am often paralyzed by not being certain what the perfect
choice is and I think the project as a whole suffers from that
sometime. XML explain output was on the agenda for years but was
stalled because we weren't sure what XML schema would be useful. And
we couldn't find out what XML schema would be useful until we had some
tools trying to use the data

If pgadmin tries to use the xml data and comes back with some feedback
will we be able to rejigger the schema? Will pgadmin be happy with a
different xml schema for each version? I suppose this depends in part
with how powerful the xml schema parsers are these days, my
understanding is that they're complex but that doesn't necessarily
translate into being powerful.


-- 
greg

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Some belated patch review for Buffers explain analyze patch

2010-02-10 Thread Robert Haas
On Wed, Feb 10, 2010 at 9:09 AM, Greg Stark st...@mit.edu wrote:
 Perhaps this is just a terminology difference but it seems
 ridiculously *narrow* to me:

Try select * from pg_class.

 Or as I said at the time... nobody liked anything about the patch
 except that they didn't have to write it.

 I know I am often paralyzed by not being certain what the perfect
 choice is and I think the project as a whole suffers from that
 sometime. XML explain output was on the agenda for years but was
 stalled because we weren't sure what XML schema would be useful. And
 we couldn't find out what XML schema would be useful until we had some
 tools trying to use the data

 If pgadmin tries to use the xml data and comes back with some feedback
 will we be able to rejigger the schema? Will pgadmin be happy with a
 different xml schema for each version? I suppose this depends in part
 with how powerful the xml schema parsers are these days, my
 understanding is that they're complex but that doesn't necessarily
 translate into being powerful.

I sort of assumed we might get some feedback from pgadmin or other
tool writers between the time this was committed six months ago and
now, but I haven't seen a single message from anyone who has actually
tried to write a tool.  As you imply, I think it will be very hard to
change the format once this is released.  At this point I think we may
be stuck with using this format and hoping that it doesn't suck too
badly.

...Robert

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Some belated patch review for Buffers explain analyze patch

2010-02-10 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes:
 I sort of assumed we might get some feedback from pgadmin or other
 tool writers between the time this was committed six months ago and
 now, but I haven't seen a single message from anyone who has actually
 tried to write a tool.  As you imply, I think it will be very hard to
 change the format once this is released.  At this point I think we may
 be stuck with using this format and hoping that it doesn't suck too
 badly.

We can still hope that some feedback comes in during beta.  I think we
should be willing to adjust the output schema even late in beta, if
someone proposes a better idea.

But what we need to do is advertise for people to play around with
this...

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Patch: Remove gcc dependency in definition of inline functions

2010-02-10 Thread Tom Lane
Kurt Harriman harri...@acm.org writes:
 On 1/19/2010 8:01 AM, Peter Eisentraut wrote:
 One principle that I suppose should have been made more explicit is that
 -- in my mind -- we should avoid littering our code with nonstandard
 constructs in place of standard constructs.

 Everyone seems to hate the name PG_INLINE, so I've changed it to
 inline_quietly, which is more descriptive.

Kurt, you seem to be more or less impervious to advice :-(.

Please just make the thing be inline and have configure define
USE_INLINE, as per previous discussion.  Cluttering the code with
nonstandard constructs is not good for readability.  More, it is likely
to confuse syntax-aware editors and pgindent, neither of which will read
any of the discussion or commit messages.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] pg_restore --single-transaction and --clean

2010-02-10 Thread Tom Lane
Takahiro Itagaki itagaki.takah...@oss.ntt.co.jp writes:
 Is it a TODO item to replace DROP into DROP IF EXISTS
 for cleanup commands in pg_restore?

No.  We try to avoid using nonstandard SQL in dumps.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] buildfarm breakage

2010-02-10 Thread Magnus Hagander
2010/2/9 Magnus Hagander mag...@hagander.net:
 On Tue, Feb 9, 2010 at 17:11, Tom Lane t...@sss.pgh.pa.us wrote:
 Magnus Hagander mag...@hagander.net writes:
 Here's a patch that fixes this. I put it locally for the radius
 authentication for now, since we don't use this anywhere else. Should
 we put this in /port/ somewhere, or is this good for now?

 How about dropping it in src/backend/port/win32/mingwcompat.c ?

 Oh, meh. I had forgotten we had that file :-)

 Thanks for the reminder, will verify tonight that it still works after
 I do that.

If someone didn't notice, I have applied that fix and it appears to
have solved it.

-- 
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] buildfarm breakage

2010-02-10 Thread Tom Lane
Magnus Hagander mag...@hagander.net writes:
 If someone didn't notice, I have applied that fix and it appears to
 have solved it.

... and there was much rejoicing.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] patch to implement ECPG side tracing / tracking ...

2010-02-10 Thread Alvaro Herrera
Hans-Jürgen Schönig wrote:
 hi,
 
 this patch implements SQL side tracing / tracking of statements and
 statement execution times.
 it is primarily intended to allow programmers to gather information
 about the runtime behavior of a program and to figure out easily
 where the bottlenecks are.
 i used the ECPG prepared statement infrastructure to implement this.
 the goal of this code is allow people to port code from databases
 such as Informix to PostgreSQL more easily and to figure out as fast
 as possible which types of queries are fast and which ones are slow.

What happened to this patch?  Was it abandoned in favor of server-side
tracing?

-- 
Alvaro Herrerahttp://www.CommandPrompt.com/
The PostgreSQL Company - Command Prompt, Inc.

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Some belated patch review for Buffers explain analyze patch

2010-02-10 Thread Robert Haas
On Wed, Feb 10, 2010 at 9:46 AM, Tom Lane t...@sss.pgh.pa.us wrote:
 Robert Haas robertmh...@gmail.com writes:
 I sort of assumed we might get some feedback from pgadmin or other
 tool writers between the time this was committed six months ago and
 now, but I haven't seen a single message from anyone who has actually
 tried to write a tool.  As you imply, I think it will be very hard to
 change the format once this is released.  At this point I think we may
 be stuck with using this format and hoping that it doesn't suck too
 badly.

 We can still hope that some feedback comes in during beta.  I think we
 should be willing to adjust the output schema even late in beta, if
 someone proposes a better idea.

I'm not opposed to that in principal, but in practice the PGadmin
folks may not like us very much if we change things too drastically if
they've got it working the way we had it...  we'll just have to see
what reports we get, I suppose.

...Robert

 But what we need to do is advertise for people to play around with
 this...

                        regards, tom lane


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] pg_restore --single-transaction and --clean

2010-02-10 Thread Robert Haas
On Wed, Feb 10, 2010 at 10:16 AM, Tom Lane t...@sss.pgh.pa.us wrote:
 Takahiro Itagaki itagaki.takah...@oss.ntt.co.jp writes:
 Is it a TODO item to replace DROP into DROP IF EXISTS
 for cleanup commands in pg_restore?

 No.  We try to avoid using nonstandard SQL in dumps.

How often do we succeed?  It seems unlikely that our dumps would be
restorable into any other database.

...Robert

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] pg_restore --single-transaction and --clean

2010-02-10 Thread Kevin Grittner
Robert Haas robertmh...@gmail.com wrote:
 Tom Lane t...@sss.pgh.pa.us wrote:
 
 We try to avoid using nonstandard SQL in dumps.
 
 How often do we succeed?  It seems unlikely that our dumps would
 be restorable into any other database.
 
When we were running in a mixed environment we had several occasions
where it was useful to feed pg_dump --column-inserts output into
Sybase databases.  It was very nice to have that.  I think we did
sometimes have to filter it through sed to deal with BOOLEAN vs BIT
issues.
 
-Kevin

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] pg_restore --single-transaction and --clean

2010-02-10 Thread Alvaro Herrera
Kevin Grittner escribió:
 Robert Haas robertmh...@gmail.com wrote:
  Tom Lane t...@sss.pgh.pa.us wrote:
  
  We try to avoid using nonstandard SQL in dumps.
  
  How often do we succeed?  It seems unlikely that our dumps would
  be restorable into any other database.
  
 When we were running in a mixed environment we had several occasions
 where it was useful to feed pg_dump --column-inserts output into
 Sybase databases.  It was very nice to have that.  I think we did
 sometimes have to filter it through sed to deal with BOOLEAN vs BIT
 issues.

Maybe we should have a --compatible-mode or some such that enables these
things, instead of staying away from useful PG-only features.

The problem would then be how to test it ...

-- 
Alvaro Herrerahttp://www.CommandPrompt.com/
The PostgreSQL Company - Command Prompt, Inc.

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Some belated patch review for Buffers explain analyze patch

2010-02-10 Thread Dave Page
On Wed, Feb 10, 2010 at 4:15 PM, Robert Haas robertmh...@gmail.com wrote:
 On Wed, Feb 10, 2010 at 9:46 AM, Tom Lane t...@sss.pgh.pa.us wrote:
 Robert Haas robertmh...@gmail.com writes:
 I sort of assumed we might get some feedback from pgadmin or other
 tool writers between the time this was committed six months ago and
 now, but I haven't seen a single message from anyone who has actually
 tried to write a tool.  As you imply, I think it will be very hard to
 change the format once this is released.  At this point I think we may
 be stuck with using this format and hoping that it doesn't suck too
 badly.

 We can still hope that some feedback comes in during beta.  I think we
 should be willing to adjust the output schema even late in beta, if
 someone proposes a better idea.

 I'm not opposed to that in principal, but in practice the PGadmin
 folks may not like us very much if we change things too drastically if
 they've got it working the way we had it...  we'll just have to see
 what reports we get, I suppose.

We're not planning to reimplement our existing parser for this release
so it won't bother us if you want to bash about any of the new
formats.

-- 
Dave Page
EnterpriseDB UK: http://www.enterprisedb.com

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Some belated patch review for Buffers explain analyze patch

2010-02-10 Thread Alvaro Herrera
Robert Haas escribió:
 On Wed, Feb 10, 2010 at 9:46 AM, Tom Lane t...@sss.pgh.pa.us wrote:
  Robert Haas robertmh...@gmail.com writes:
  I sort of assumed we might get some feedback from pgadmin or other
  tool writers between the time this was committed six months ago and
  now, but I haven't seen a single message from anyone who has actually
  tried to write a tool.  As you imply, I think it will be very hard to
  change the format once this is released.  At this point I think we may
  be stuck with using this format and hoping that it doesn't suck too
  badly.
 
  We can still hope that some feedback comes in during beta.  I think we
  should be willing to adjust the output schema even late in beta, if
  someone proposes a better idea.
 
 I'm not opposed to that in principal, but in practice the PGadmin
 folks may not like us very much if we change things too drastically if
 they've got it working the way we had it...  we'll just have to see
 what reports we get, I suppose.

Just talked to Dave Page on IM.  They haven't written any code to deal
with the XML format yet, and doesn't sound like they are eager to start
right now, given that they already have the parser for the text format.

-- 
Alvaro Herrerahttp://www.CommandPrompt.com/
PostgreSQL Replication, Consulting, Custom Development, 24x7 support

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] pg_restore --single-transaction and --clean

2010-02-10 Thread Tom Lane
Alvaro Herrera alvhe...@commandprompt.com writes:
 Kevin Grittner escribió:
 Robert Haas robertmh...@gmail.com wrote:
 Tom Lane t...@sss.pgh.pa.us wrote:
 We try to avoid using nonstandard SQL in dumps.

 How often do we succeed?  It seems unlikely that our dumps would
 be restorable into any other database.

 When we were running in a mixed environment we had several occasions
 where it was useful to feed pg_dump --column-inserts output into
 Sybase databases.  It was very nice to have that.  I think we did
 sometimes have to filter it through sed to deal with BOOLEAN vs BIT
 issues.

 Maybe we should have a --compatible-mode or some such that enables these
 things, instead of staying away from useful PG-only features.

Well, the subtext of my comment was really that this case isn't useful
enough to justify introducing a nonstandard construct into dumps.
IMO the whole *point* of --single-transaction is to fail if the database
isn't in the state you thought it was.  If you want to restore into an
empty DB with --single-transaction, don't use --clean.  Problem solved.

--clean has got other issues anyway with a DB that isn't in exactly the
expected state.  If the inter-object dependencies aren't quite what they
were in the source, drops are likely to fail because dependent objects
still remain.  Should we therefore make all pg_dump's drop commands
CASCADE?  I don't think so; the side-effects could be nasty.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Some belated patch review for Buffers explain analyze patch

2010-02-10 Thread Tom Lane
Dave Page dp...@pgadmin.org writes:
 On Wed, Feb 10, 2010 at 4:15 PM, Robert Haas robertmh...@gmail.com wrote:
 On Wed, Feb 10, 2010 at 9:46 AM, Tom Lane t...@sss.pgh.pa.us wrote:
 We can still hope that some feedback comes in during beta.

 I'm not opposed to that in principal, but in practice the PGadmin
 folks may not like us very much if we change things too drastically if
 they've got it working the way we had it...  we'll just have to see
 what reports we get, I suppose.

 We're not planning to reimplement our existing parser for this release
 so it won't bother us if you want to bash about any of the new
 formats.

Well, you're going to have to do more than zero work even with that
plan, given the changes already made to the text format.  It would be
really nice if we could get some feedback on the non-text formats
*before* they're set in stone.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Re: [COMMITTERS] pgsql: Make standby server continuously retry restoring the next WAL

2010-02-10 Thread Heikki Linnakangas
Aidan Van Dyk wrote:
 * Heikki Linnakangas heikki.linnakan...@enterprisedb.com [100210 02:33]:
  
 Hmm, so after running restore_command, check the file size and if it's
 too short, treat it the same as if restore_command returned non-zero?
 And it will be retried on the next iteration. Works for me, though OTOH
 it will then fail to complain about a genuinely WAL file that's
 truncated for some reason. I guess there's no way around that, even if
 you have a script as restore_command that does the file size check, it
 will have the same problem.
 
 But isn't this something every current PITR archive already works
 around...  Everybody doing PITR archives already know the importance of
 making the *appearance* of the WAL filename in the archive atomic.

Well, pg_standby does defend against that, but you don't use pg_standby
with the built-in standby mode anymore. It would be reasonable to have
the same level of defenses built-in. It's essentially a one-line change,
and saves a lot of trouble and risk of subtle misconfiguration for admins.

 Don't docs warn about plain cp not being atomic and allowing short
 files to appear in the archive... 

Hmm, I don't see anything about that at quick glance. Besides, normal
PITR doesn't have a problem with that, because it will stop when it
reaches the end of archived WAL anyway.

-- 
  Heikki Linnakangas
  EnterpriseDB   http://www.enterprisedb.com

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] pg_restore --single-transaction and --clean

2010-02-10 Thread Alvaro Herrera
Tom Lane escribió:
 Alvaro Herrera alvhe...@commandprompt.com writes:
  Kevin Grittner escribi�:
  Robert Haas robertmh...@gmail.com wrote:
  Tom Lane t...@sss.pgh.pa.us wrote:
  We try to avoid using nonstandard SQL in dumps.
 
  How often do we succeed?  It seems unlikely that our dumps would
  be restorable into any other database.
 
  When we were running in a mixed environment we had several occasions
  where it was useful to feed pg_dump --column-inserts output into
  Sybase databases.  It was very nice to have that.  I think we did
  sometimes have to filter it through sed to deal with BOOLEAN vs BIT
  issues.
 
  Maybe we should have a --compatible-mode or some such that enables these
  things, instead of staying away from useful PG-only features.
 
 Well, the subtext of my comment was really that this case isn't useful
 enough to justify introducing a nonstandard construct into dumps.

That's true, but this is not the first time we've left out some feature
from dumps because they would make them standards-incompatible.  If we
have enough of these (and I have no idea that we do), maybe we could
start here.  This is of course just a future TODO item, not something to
consider for 9.0.

-- 
Alvaro Herrerahttp://www.CommandPrompt.com/
The PostgreSQL Company - Command Prompt, Inc.

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Some belated patch review for Buffers explain analyze patch

2010-02-10 Thread Dave Page
On Wed, Feb 10, 2010 at 4:50 PM, Tom Lane t...@sss.pgh.pa.us wrote:
 Dave Page dp...@pgadmin.org writes:
 On Wed, Feb 10, 2010 at 4:15 PM, Robert Haas robertmh...@gmail.com wrote:
 On Wed, Feb 10, 2010 at 9:46 AM, Tom Lane t...@sss.pgh.pa.us wrote:
 We can still hope that some feedback comes in during beta.

 I'm not opposed to that in principal, but in practice the PGadmin
 folks may not like us very much if we change things too drastically if
 they've got it working the way we had it...  we'll just have to see
 what reports we get, I suppose.

 We're not planning to reimplement our existing parser for this release
 so it won't bother us if you want to bash about any of the new
 formats.

 Well, you're going to have to do more than zero work even with that
 plan, given the changes already made to the text format.

The important bits didn't really change (at least in ways that would
hurt us). Guillaume already worked on adding the new info.

 It would be
 really nice if we could get some feedback on the non-text formats
 *before* they're set in stone.

I looked at them briefly when preparing for my talk at FOSDEM and
didn't see anything that I didn't like the look of. Honestly though,
pretty much any structured format would work for us - my main concern
is that we get the extra info that we couldn't previously get because
it would bloat the text output - for example, the schema name for
every relation.

-- 
Dave Page
EnterpriseDB UK: http://www.enterprisedb.com

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Some belated patch review for Buffers explain analyze patch

2010-02-10 Thread Alvaro Herrera
Tom Lane escribió:
 Dave Page dp...@pgadmin.org writes:
  On Wed, Feb 10, 2010 at 4:15 PM, Robert Haas robertmh...@gmail.com wrote:
  On Wed, Feb 10, 2010 at 9:46 AM, Tom Lane t...@sss.pgh.pa.us wrote:
  We can still hope that some feedback comes in during beta.
 
  I'm not opposed to that in principal, but in practice the PGadmin
  folks may not like us very much if we change things too drastically if
  they've got it working the way we had it... �we'll just have to see
  what reports we get, I suppose.
 
  We're not planning to reimplement our existing parser for this release
  so it won't bother us if you want to bash about any of the new
  formats.
 
 Well, you're going to have to do more than zero work even with that
 plan, given the changes already made to the text format.  It would be
 really nice if we could get some feedback on the non-text formats
 *before* they're set in stone.

Is Redhat's Visual Explain still alive?  And what about Tom Raney's stuff?

-- 
Alvaro Herrerahttp://www.CommandPrompt.com/
The PostgreSQL Company - Command Prompt, Inc.

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Some belated patch review for Buffers explain analyze patch

2010-02-10 Thread Robert Haas
On Wed, Feb 10, 2010 at 11:58 AM, Alvaro Herrera
alvhe...@commandprompt.com wrote:
 Tom Lane escribió:
 Dave Page dp...@pgadmin.org writes:
  On Wed, Feb 10, 2010 at 4:15 PM, Robert Haas robertmh...@gmail.com wrote:
  On Wed, Feb 10, 2010 at 9:46 AM, Tom Lane t...@sss.pgh.pa.us wrote:
  We can still hope that some feedback comes in during beta.

  I'm not opposed to that in principal, but in practice the PGadmin
  folks may not like us very much if we change things too drastically if
  they've got it working the way we had it...  we'll just have to see
  what reports we get, I suppose.

  We're not planning to reimplement our existing parser for this release
  so it won't bother us if you want to bash about any of the new
  formats.

 Well, you're going to have to do more than zero work even with that
 plan, given the changes already made to the text format.  It would be
 really nice if we could get some feedback on the non-text formats
 *before* they're set in stone.

 Is Redhat's Visual Explain still alive?  And what about Tom Raney's stuff?

The core of Tom Raney's work was not so much the EXPLAIN format per se
(which is really mooted by the changes already made in CVS) but the
ability to get information about plans the planner considered and
discarded.  I am not sure whether the latter is something we want to
accept into core (not for 9.0 at any rate, I would think).

...Robert

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Some belated patch review for Buffers explain analyze patch

2010-02-10 Thread Alvaro Herrera
Robert Haas escribió:
 On Wed, Feb 10, 2010 at 11:58 AM, Alvaro Herrera
 alvhe...@commandprompt.com wrote:

  Is Redhat's Visual Explain still alive?  And what about Tom Raney's stuff?
 
 The core of Tom Raney's work was not so much the EXPLAIN format per se
 (which is really mooted by the changes already made in CVS) but the
 ability to get information about plans the planner considered and
 discarded.  I am not sure whether the latter is something we want to
 accept into core (not for 9.0 at any rate, I would think).

I was thinking in his visual stuff ... didn't he also wrote a tool to
visualize plans?

-- 
Alvaro Herrerahttp://www.CommandPrompt.com/
The PostgreSQL Company - Command Prompt, Inc.

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] [PATCH] Output configuration status after ./configure run.

2010-02-10 Thread Priit Laes
Ühel kenal päeval, K, 2010-02-10 kell 10:39, kirjutas Tom Lane:
 Priit Laes pl...@plaes.org writes:
  This patch enables showing configure status at the end of ./configure
  run and thus makes ./configure process a bit easier to follow (in the
  sense of what features are actually enabled).
 
 I don't think anybody actually reads configure's output anyway, so I'm
 not sure about the point of this.  Usually you wish you knew this
 information long afterwards.  We do have tools (pg_config,
 pg_controldata) for extracting such information from an existing
 installation, which is the real use-case IMHO.

I do. And there are probably others. It provides a list of nicely
formatted options that configure enabled/disabled before you start the
build process. pg_config and pg_controldata are a bit too late.

 Also, it's quite unclear which items deserve a place in the list.
 If it's just to repeat what was in the configure command-line, what
 is the value of that?

It might avoid the 'UU, I forgot to enable python support.',
after you have waited a while for the build to finish...

Cheers,
Priit ;)

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] [PATCH] Output configuration status after ./configure run.

2010-02-10 Thread Ross J. Reedstrom
On Wed, Feb 10, 2010 at 07:01:19PM +0200, Priit Laes wrote:
 
 It might avoid the 'UU, I forgot to enable python support.',
 after you have waited a while for the build to finish...
 

+1 from me, for that very reason!

Ross
-- 
Ross Reedstrom, Ph.D. reeds...@rice.edu
Systems Engineer  Admin, Research Scientistphone: 713-348-6166
The Connexions Project  http://cnx.orgfax: 713-348-3665
Rice University MS-375, Houston, TX 77005
GPG Key fingerprint = F023 82C8 9B0E 2CC6 0D8E  F888 D3AE 810E 88F0 BEDE

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Some belated patch review for Buffers explain analyze patch

2010-02-10 Thread Tom Lane
Alvaro Herrera alvhe...@commandprompt.com writes:
 Tom Lane escribió:
 ... It would be
 really nice if we could get some feedback on the non-text formats
 *before* they're set in stone.

 Is Redhat's Visual Explain still alive?  And what about Tom Raney's stuff?

Visual Explain is dead as far as Red Hat is concerned :-( ... but it is
GPL and so anyone could pick it up.  I was under the impression that EDB
had done so.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] synchronized snapshots

2010-02-10 Thread Markus Wanner

Hi Joachim,

On Wed, 10 Feb 2010 11:36:41 +0100, Joachim Wieland j...@mcknight.de
wrote:

http://www.postgresql.org/docs/8.4/static/backup-dump.html already
states about pg_dump: In particular, it must have read access to all
tables that you want to back up, so in practice you almost always have
to run it as a database superuser. so I think there is not a big loss
here...


Hm.. I doubt somewhat that's common practice. After all, read access to 
all tables is still a *lot* less than superuser privileges. But yeah, 
the documentation currently states that.



They more or less get it by chance :-)  They acquire a snapshot when
they call pg_synchronize_snapshot_taken()


Oh, I see, calling the function by itself already acquires a snapshot.
Even in case of a fast path call, it seems. Then your approach is correct.

(I'd still feel more comfortable, it I had seen a 
GetTransactionSnapshot() or something akin in there).



and if all the backends do
it while the other backend holds the lock in shared mode, we know that
the snapshot won't change, so they all get the same snapshot.


Agreed, that works.

(Ab)using the ProcArrayLock for synchronization is probably acceptable 
for pg_dump, however, I'd rather take another approach for a more 
general implementation.



Also, you should probably ensure the calling transactions don't have a
snapshot already (let alone a transaction id).


True...


Hm.. realizing that a function call per-se acquires a snapshot, I fail 
to see how we could check if we really acquired a snapshot. Consider the 
following (admittedly stupid) example:


BEGIN;
SET TRANSACTION ISOLATION LEVEL SERIALIZABLE;
SELECT version();
 ... time goes by ...
SELECT pg_synchronize_snapshot_taken(..);

As it stands, your function would silently fail to synchronize the
snapshots, if other transactions committed in between the two function 
calls.



It seemed more robust and convenient to have an expiration in the
backend itself. What would happen if you called
pg_synchronize_snapshots() and if right after that your network
connection dropped? Without the server noticing, it would continue to
hold the lock and you could not log in anymore...


Hm.. that's a point. Given this approach uses the ProcArrayLock, it's
probably better to use an explicit timeout.


But you are right: The proposed feature is a pragmatic and quick
solution for pg_dump and similar but we might want to have a more
general snapshot cloning procedure instead. Not having a delay for
other activities at all and not requiring superuser privileges would
be a big advantage over what I have proposed.


Agreed.

Regards

Markus Wanner

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Parameter name standby_mode

2010-02-10 Thread Joachim Wieland
On Wed, Feb 10, 2010 at 12:16 PM, Heikki Linnakangas
heikki.linnakan...@enterprisedb.com wrote:
 If they want to implement the warm standby using the (new) built-in
 logic to keep retrying restore_command, they would set
 standby_mode='on'. standby_mode='on' doesn't imply streaming replication.

 If you want to use pg_standby or similar tools, then you would indeed
 set standby_mode='off', but I think that makes sense because you're
 implementing the standby functionality outside the server in that case.

Okay, got it now with your explanations.

For some reason it didn't work before with standby_mode = 'on' (it
does now) and the warning FATAL:  sorry, too many standbys already
gave me a first suspicion that SR is the only use case for this. Then
I checked the docs and there it said If this parameter is on, the
streaming replication is enabled. I understand now what it does and
that it is a prerequisite but that there is also a non-SR use case...
So the name is okay for me :-)


Thanks again,
Joachim

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] synchronized snapshots

2010-02-10 Thread Heikki Linnakangas
Markus Wanner wrote:
 On Wed, 10 Feb 2010 11:36:41 +0100, Joachim Wieland j...@mcknight.de
 wrote:
 http://www.postgresql.org/docs/8.4/static/backup-dump.html already
 states about pg_dump: In particular, it must have read access to all
 tables that you want to back up, so in practice you almost always have
 to run it as a database superuser. so I think there is not a big loss
 here...
 
 Hm.. I doubt somewhat that's common practice. After all, read access to
 all tables is still a *lot* less than superuser privileges. But yeah,
 the documentation currently states that.

I think running as database owner gets you pretty far as far as pg_dump
goes. It would be good to lift the limitation that you have to be superuser.

 But you are right: The proposed feature is a pragmatic and quick
 solution for pg_dump and similar but we might want to have a more
 general snapshot cloning procedure instead. Not having a delay for
 other activities at all and not requiring superuser privileges would
 be a big advantage over what I have proposed.
 
 Agreed.

Yeah, a big advantage of the proposed approach is that it's pretty
simple to implement as an external module, allowing you to write scripts
using it for older versions too.

-- 
  Heikki Linnakangas
  EnterpriseDB   http://www.enterprisedb.com

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: I: [HACKERS] About Our CLUSTER implementation is pessimal patch

2010-02-10 Thread Leonardo F
Perhaps you could supply a .sql file containing a testcase 
 illustrating the performance benefits you tested with your patch

Sure.


Attached the updated patch (should solve a bug) and a script.
The sql scripts generates a 2M rows table (orig); then the
table is copied and the copy clustered using seq + sort (since 
set enable_seqscan=false;).
Then the table orig is copied again, and the copy clustered
using regular index scan (set enable_indexscan=true; set 
enable_seqscan=false).
Then the same thing is done on a 5M rows table, and on a 10M
rows table.

On my system (Sol10 on a dual Opteron 2.8) single disc:


2M:  seq+sort 11secs; regular index scan: 33secs
5M:  seq+sort 39secs; regular index scan: 105secs
10M:seq+sort 83secs; regular index scan: 646secs


Maybe someone could suggest a better/different test?


Leonardo



  

sorted_cluster20100210.patch
Description: Binary data


cluster_tests.sql
Description: Binary data

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] [CFReview] Red-Black Tree

2010-02-10 Thread Teodor Sigaev

So suppose at this point that step is the largest integer that can be
represented...

!   step ++;

Boom.

!   step= 1;

step= 1;
step ++'

Unboom?



!
!   while(step  0) {
!   int i;

!   for (i = step-1; i  nentry; i += 2 * step)


And similarly here... if nentry is greater than maxint/2, then i += 2
* step will overflow, no?


Agree, so
for (i = step - 1; i  nentry  i = 0; i += step  1 /* *2 */)


Also, rb_free is removed per Tom's comment. Can I commit  the patch?
--
Teodor Sigaev   E-mail: teo...@sigaev.ru
   WWW: http://www.sigaev.ru/


rbtree-0.13.gz
Description: Unix tar archive

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] [CFReview] Red-Black Tree

2010-02-10 Thread Alvaro Herrera
Teodor Sigaev escribió:

 Also, rb_free is removed per Tom's comment. Can I commit  the patch?

Hey, but rb_freefunc is still there ...

One very minor quibble: please make the $PostgreSQL$ lines be just that
(i.e. remove everything between the : to the terminating $, keeping the
latter)

-- 
Alvaro Herrerahttp://www.CommandPrompt.com/
The PostgreSQL Company - Command Prompt, Inc.

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] [CFReview] Red-Black Tree

2010-02-10 Thread Robert Haas
2010/2/10 Teodor Sigaev teo...@sigaev.ru:
 So suppose at this point that step is the largest integer that can be
 represented...

 !       step ++;

 Boom.

 !       step= 1;

 step= 1;
 step ++'

 Unboom?

Yeah, that'll work.

 !
 !       while(step  0) {
 !               int i;

 !               for (i = step-1; i  nentry; i += 2 * step)

 And similarly here... if nentry is greater than maxint/2, then i += 2
 * step will overflow, no?

 Agree, so
 for (i = step - 1; i  nentry  i = 0; i += step  1 /* *2 */)

I don't think you should do it this way.  I can't immediately say
whether it's safe on all platforms, but it's certainly not clear.
Just put the test at the bottom of the loop the way I did it (after
fixing whatever I screwed up).

 Also, rb_free is removed per Tom's comment. Can I commit  the patch?

Pending the above, go for it.

...Robert

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Some belated patch review for Buffers explain analyze patch

2010-02-10 Thread Robert Haas
On Wed, Feb 10, 2010 at 12:08 PM, Alvaro Herrera
alvhe...@commandprompt.com wrote:
 Robert Haas escribió:
 On Wed, Feb 10, 2010 at 11:58 AM, Alvaro Herrera
 alvhe...@commandprompt.com wrote:

  Is Redhat's Visual Explain still alive?  And what about Tom Raney's stuff?

 The core of Tom Raney's work was not so much the EXPLAIN format per se
 (which is really mooted by the changes already made in CVS) but the
 ability to get information about plans the planner considered and
 discarded.  I am not sure whether the latter is something we want to
 accept into core (not for 9.0 at any rate, I would think).

 I was thinking in his visual stuff ... didn't he also wrote a tool to
 visualize plans?

Yeah, but again, that was to see all the other things that were
considered at any given node, not just to see the actual plan.

...Robert

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] [PATCH] Output configuration status after ./configure run.

2010-02-10 Thread Robert Haas
On Wed, Feb 10, 2010 at 12:01 PM, Priit Laes pl...@plaes.org wrote:
 Also, it's quite unclear which items deserve a place in the list.
 If it's just to repeat what was in the configure command-line, what
 is the value of that?

 It might avoid the 'UU, I forgot to enable python support.',
 after you have waited a while for the build to finish...

Hmm.  That implies that you didn't look at the command that you typed
but you did look at its output.  I'm not going to say no one does
that (who am I to judge?) but it seems kind of strange to me.

...Robert

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


[HACKERS] I still didn't get how tsquery is internally

2010-02-10 Thread Ivan Sergio Borgonovo
I've been working on this
http://pgsql.privatepaste.com/2a81432f4f

for 8.3 (there should be some comments to port it to 8.4).

tsvector_tsquery_size works as expected.

But whatever I pass to the function I get an empty tsquery. Yet no
memory allocation problem or hang etc... just an empty vector, but
that's not enough to say I'm not filling distance and left with the
wrong values etc... anyway.

CREATE OR REPLACE FUNCTION tsvector_to_tsquery(IN tsv tsvector, op
IN char(1), weights IN varchar(4), maxpos IN smallint
)
RETURNS tsquery
AS 'MODULE_PATHNAME'
LANGUAGE C STRICT;

The function should turn a tsvector into a tsquery.

op is the operator (| or )
weights is a filter. lexemes wil be picked up just if they don't
have a position/weight or they have one of the passed weights.
maxpos will filter out all lexemes that have a positionmaxpos.

So

tsvector_to_tsquery('java:1A,2B tano:3C,4D', '', 'ABC', 100) -

java:A  java:B  tano:C

tsvector_to_tsquery('java:1A,2B tano:3C,4D', '|', 'ABC', 100) -

java:AB | tano:C


There is also a small problem. The op is always char = 0x08 (even on
a much simpler function that just retrieve op and return it as a
masked int64.
It seems that
char op = PG_GETARG_CHAR(1);
is not really what I thought it to be.

-- 
Ivan Sergio Borgonovo
http://www.webthatworks.it


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] [PATCH] Output configuration status after ./configure run.

2010-02-10 Thread Alvaro Herrera
Robert Haas escribió:
 On Wed, Feb 10, 2010 at 12:01 PM, Priit Laes pl...@plaes.org wrote:
  Also, it's quite unclear which items deserve a place in the list.
  If it's just to repeat what was in the configure command-line, what
  is the value of that?
 
  It might avoid the 'UU, I forgot to enable python support.',
  after you have waited a while for the build to finish...
 
 Hmm.  That implies that you didn't look at the command that you typed
 but you did look at its output.  I'm not going to say no one does
 that (who am I to judge?) but it seems kind of strange to me.

Maybe you didn't type it, but it came from elsewhere?  Maybe you're
inheriting settings from some environment variable, or a file?  Maybe
you're eval'ing pg_config --configure?

The general idea seems sensible to me.  I can't comment on the
specifics.

-- 
Alvaro Herrerahttp://www.CommandPrompt.com/
The PostgreSQL Company - Command Prompt, Inc.

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] [PATCH] Output configuration status after ./configure run.

2010-02-10 Thread Robert Haas
On Wed, Feb 10, 2010 at 2:16 PM, Alvaro Herrera
alvhe...@commandprompt.com wrote:
 Maybe you didn't type it, but it came from elsewhere?  Maybe you're
 inheriting settings from some environment variable, or a file?  Maybe
 you're eval'ing pg_config --configure?

Yeah, could be.

 The general idea seems sensible to me.  I can't comment on the
 specifics.

I took a quick look at it.  It's basically just a block of output at
the end of configure that reflects the values for a subset of the
configuration parameters (for example, just off the top of my head,
--enable-debug, --enable-casserts, --enable-depend, and --enable-nls
aren't there).  It already won't fit in a 24x80 window, and if we
actually make it complete, it'll be considerably longer.  While not
denying its possible usefulness to the OP, I'm not sure that in
general more people would find it useful than annoying.  I might be
wrong, though.

...Robert

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] [CFReview] Red-Black Tree

2010-02-10 Thread Teodor Sigaev

Hey, but rb_freefunc is still there ...


It will be reintroduced when ts_stat will be rewrited to use rbtree


One very minor quibble: please make the $PostgreSQL$ lines be just that
(i.e. remove everything between the : to the terminating $, keeping the
latter)

ok

--
Teodor Sigaev   E-mail: teo...@sigaev.ru
   WWW: http://www.sigaev.ru/

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


[HACKERS] Postgres Triggers issue

2010-02-10 Thread u235sentinel
I have a strange problem we noticed the other day with triggers.  We're 
running 8.3.3 on Solaris 10 (intel) and have a feed that comes in 
regularly to populate a table we're working on.  The feed works just 
fine inserting rows however the following trigger stops the feed until 
we remove the trigger.  Any thoughts on what I'm doing wrong here?


Thanks!

---

CREATE OR REPLACE FUNCTION r.m_t()
RETURNS trigger AS
$BODY$
BEGIN
 INSERT INTO temp_m_t VALUES (NEW.*,1+1);
RETURN NULL;
END;
$BODY$
LANGUAGE 'plpgsql';


CREATE TRIGGER tafter
AFTER INSERT OR UPDATE
ON r.m_a
FOR EACH ROW
EXECUTE PROCEDURE r.m_t();


--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Writeable CTEs and empty relations

2010-02-10 Thread Robert Haas
On Wed, Feb 10, 2010 at 5:05 AM, Marko Tiikkaja
marko.tiikk...@cs.helsinki.fi wrote:
 On 2010-02-10 03:20 +0200, Tom Lane wrote:
 Marko Tiikkaja marko.tiikk...@cs.helsinki.fi writes:
 On 2010-02-10 02:19 +0200, Tom Lane wrote:
 You still haven't explained why it's a good idea to change the snapshot
 after the executor has started.  Right at the moment I'm prepared to
 reject the patch on that ground alone.

 The patch only touches the snapshot's curcid.  That's needed to allow
 the queries see the tuples of the DML WITHs ran before that particular
 query.

 ... and they will also see, for example, tuples inserted by triggers
 fired by those queries.  I thought the plan for this feature was to
 store and re-read the RETURNING data, not to go back and re-read the
 underlying tables.

 I originally suggested this approach about four months ago.  During this
 whole time you haven't expressed any concerns about this, but suddenly
 it's a reason to reject the patch?

 Anyway, if we want to avoid modifying the snapshot, we can't bump the
 CID between queries.  I haven't thought this through, but it seems like
 this could work.  However, none of the WITH queries can see the previous
 statement's tuples, which means that someone may be suprised when they
 try to modify the same tuples twice just to discover that only the first
 modification took place.  I don't see this as a huge problem though.

I think that we agreed previously that running all the DML queries to
completion before the main query, bumping the CID after each one, and
saving the outputs, was the only workable approach.

http://archives.postgresql.org/pgsql-hackers/2009-10/msg00566.php
http://archives.postgresql.org/pgsql-hackers/2009-10/msg00614.php

If the executor has buried in it the assumption that the snapshot
can't change after startup, then does that mean that we need to start
up and shut down the executor for each subquery?

http://archives.postgresql.org/pgsql-hackers/2009-11/msg01964.php

...Robert

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Writeable CTEs and empty relations

2010-02-10 Thread Marko Tiikkaja
On 2010-02-10 21:59 +0200, Robert Haas wrote:
 On Wed, Feb 10, 2010 at 5:05 AM, Marko Tiikkaja
 marko.tiikk...@cs.helsinki.fi wrote:
 On 2010-02-10 03:20 +0200, Tom Lane wrote:
 Marko Tiikkaja marko.tiikk...@cs.helsinki.fi writes:
 On 2010-02-10 02:19 +0200, Tom Lane wrote:
 You still haven't explained why it's a good idea to change the snapshot
 after the executor has started.  Right at the moment I'm prepared to
 reject the patch on that ground alone.

 The patch only touches the snapshot's curcid.  That's needed to allow
 the queries see the tuples of the DML WITHs ran before that particular
 query.

 ... and they will also see, for example, tuples inserted by triggers
 fired by those queries.  I thought the plan for this feature was to
 store and re-read the RETURNING data, not to go back and re-read the
 underlying tables.

 I originally suggested this approach about four months ago.  During this
 whole time you haven't expressed any concerns about this, but suddenly
 it's a reason to reject the patch?

 Anyway, if we want to avoid modifying the snapshot, we can't bump the
 CID between queries.  I haven't thought this through, but it seems like
 this could work.  However, none of the WITH queries can see the previous
 statement's tuples, which means that someone may be suprised when they
 try to modify the same tuples twice just to discover that only the first
 modification took place.  I don't see this as a huge problem though.
 
 I think that we agreed previously that running all the DML queries to
 completion before the main query, bumping the CID after each one, and
 saving the outputs, was the only workable approach.
 
 http://archives.postgresql.org/pgsql-hackers/2009-10/msg00566.php

Hmm.  Yeah.  Didn't think about that :-(

 If the executor has buried in it the assumption that the snapshot
 can't change after startup, then does that mean that we need to start
 up and shut down the executor for each subquery?

Then I guess that's pretty much the only option.


Regards,
Marko Tiikkaja

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] [PATCH] Output configuration status after ./configure run.

2010-02-10 Thread Euler Taveira de Oliveira
Alvaro Herrera escreveu:
 The general idea seems sensible to me.  I can't comment on the
 specifics.
 
+1. A lot of other programs have this summary at the end of configure
execution. The problem is that PostgreSQL has too many options. Do we want to
list all of them?


-- 
  Euler Taveira de Oliveira
  http://www.timbira.com/

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] [PATCH] Output configuration status after ./configure run.

2010-02-10 Thread Alvaro Herrera
Euler Taveira de Oliveira escribió:
 Alvaro Herrera escreveu:
  The general idea seems sensible to me.  I can't comment on the
  specifics.
  
 +1. A lot of other programs have this summary at the end of configure
 execution. The problem is that PostgreSQL has too many options. Do we want to
 list all of them?

Maybe not all, but my bike is colored PGPORT, and this shed doesn't seem
to have anything that combines with that.

If this doesn't fit in 24x80 maybe we need to find a more compact way to
display things.

BTW if this thing is reasonably complete, we could have buildfarm
display this output in a summary page for each animal.  Seems more
readable than configure options.

-- 
Alvaro Herrerahttp://www.CommandPrompt.com/
The PostgreSQL Company - Command Prompt, Inc.

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] TCP keepalive support for libpq

2010-02-10 Thread daveg
On Tue, Feb 09, 2010 at 09:34:10AM -0500, Andrew Chernow wrote:
 Tollef Fog Heen wrote:
 (please Cc me on replies, I am not subscribed)
 
 Hi,
 
 libpq currently does not use TCP keepalives.  This is a problem in our
 case where we have some clients waiting for notifies and then the
 connection is dropped on the server side.  The client never gets the FIN
 and thinks the connection is up.  The attached patch unconditionally
 adds keepalives.  I chose unconditionally as this is what the server
 does.  We didn't need the ability to tune the timeouts, but that could
 be added with reasonable ease.
 
 ISTM that the default behavior should be keep alives disabled, as it is 
 now, and those wanting it can just set it in their apps:
 
 setsockopt(PQsocket(conn), SOL_SOCKET, SO_KEEPALIVE, ...)

I disagree. I have clients who have problems with leftover client connections
due to server host failures. They do not write apps in C. For a non-default
change to be effective we would need to have all the client drivers, eg JDBC,
psycopg, DBD-DBI, and the apps like psql make changes to turn it on. Adding
this option as a non-default will not really help.

-dg
 

-- 
David Gould   da...@sonic.net  510 536 1443510 282 0869
If simplicity worked, the world would be overrun with insects.

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] [PATCH] Output configuration status after ./configure run.

2010-02-10 Thread Robert Haas
On Wed, Feb 10, 2010 at 3:30 PM, Alvaro Herrera
alvhe...@commandprompt.com wrote:
 Euler Taveira de Oliveira escribió:
 Alvaro Herrera escreveu:
  The general idea seems sensible to me.  I can't comment on the
  specifics.
 
 +1. A lot of other programs have this summary at the end of configure
 execution. The problem is that PostgreSQL has too many options. Do we want to
 list all of them?

 Maybe not all, but my bike is colored PGPORT, and this shed doesn't seem
 to have anything that combines with that.

Well said, sir.

 If this doesn't fit in 24x80 maybe we need to find a more compact way to
 display things.

+1.  I wouldn't mind a one-line summary, but a two page summary seems
like a lot.

...Robert

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Writeable CTEs and empty relations

2010-02-10 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes:
 If the executor has buried in it the assumption that the snapshot
 can't change after startup, then does that mean that we need to start
 up and shut down the executor for each subquery?

Yes, I think so.  That's the way it's always worked in the past;
see for example PortalRunMulti() and ProcessQuery().  I think trying
to change that is a high-risk, low-reward activity.

This probably means that the planner output for queries involving
writeable CTEs has to be a separate PlannedStmt per such CTE.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Writeable CTEs and empty relations

2010-02-10 Thread Marko Tiikkaja
On 2010-02-10 23:57 +0200, Tom Lane wrote:
 Robert Haas robertmh...@gmail.com writes:
 If the executor has buried in it the assumption that the snapshot
 can't change after startup, then does that mean that we need to start
 up and shut down the executor for each subquery?
 
 Yes, I think so.  That's the way it's always worked in the past;
 see for example PortalRunMulti() and ProcessQuery().  I think trying
 to change that is a high-risk, low-reward activity.
 
 This probably means that the planner output for queries involving
 writeable CTEs has to be a separate PlannedStmt per such CTE.

I'm looking at this, but I can't think of any good way of associating
the tuplestores from PortalRunMulti() with the correct CTEs.  Any ideas?


Regards,
Marko Tiikkaja

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Patch: Remove gcc dependency in definition of inline functions

2010-02-10 Thread Kurt Harriman

On 2/10/2010 7:12 AM, Tom Lane wrote:

Kurt, you seem to be more or less impervious to advice :-(.


Thank you for reviewing my patch.  It is a rare honor to
have my personal qualities reviewed here as well.

Since this forum is archived for posterity, I suppose I
must point out that I have in fact been responsive to all
the advice that has been offered here.  I have answered
each comment fully and politely.  I have acted upon most
of the suggestions, and have revised my small patch
accordingly and resubmitted it twice (now thrice,
incorporating this comment of yours).

Admittedly I have used my own judgment in how to adopt these
sometimes conflicting suggestions; and have explained the
rationale for my choices.  Everyone may judge whether my
choices and explanations are satisfactory and continue the
dialogue if they are not yet satisfied.  By sincerely
engaging in such a process, consensus may be reached.

All contributors to the pg_hackers list are well advised to
be impervious to brusque and sometimes rude dismissals, which
seem to be de rigueur here.  However, the evidence of this
thread shows that I have been far from impervious to advice.

By the way, suggestions which must be carried out without
question are orders, not advice.  When a statement,
meant to be imperative, is phrased softly as advice, it can
easily be mistaken as optional by newcomers who may not have
fully grasped the prevailing reality.  Thus, commands are
best stated in clear language.


Please just make the thing be inline and have configure define
USE_INLINE, as per previous discussion.


Just now I have resubmitted according to your instruction.


Cluttering the code with nonstandard constructs is not

 good for readability.

Agreed.  But any program consists of definitions of new
identifiers, data structures, macros, and conventions or
guidelines for their use.  What, or who, differentiates
ordinary programming practice, such as typedefs, from
nonstandard constructs?


More, it is likely to confuse syntax-aware editors and

 pgindent, neither of which will read any of the discussion
 or commit messages.

Good point.  This had not been mentioned before.  It works
alright with the syntax-aware editors that I use, and I
haven't had occasion to run pgindent, so I didn't think of
this earlier.

Does the same problem exist with the PGDLLIMPORT macro
defined in postgres.h?  It, too, is used in the same
syntactic niche where inline would be placed.

Regards,
... kurt

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Writeable CTEs and empty relations

2010-02-10 Thread Marko Tiikkaja
On 2010-02-11 00:50 +0200, Marko Tiikkaja wrote:
 On 2010-02-10 23:57 +0200, Tom Lane wrote:
 Robert Haas robertmh...@gmail.com writes:
 If the executor has buried in it the assumption that the snapshot
 can't change after startup, then does that mean that we need to start
 up and shut down the executor for each subquery?

 Yes, I think so.  That's the way it's always worked in the past;
 see for example PortalRunMulti() and ProcessQuery().  I think trying
 to change that is a high-risk, low-reward activity.

 This probably means that the planner output for queries involving
 writeable CTEs has to be a separate PlannedStmt per such CTE.
 
 I'm looking at this, but I can't think of any good way of associating
 the tuplestores from PortalRunMulti() with the correct CTEs.  Any ideas?

Ok, what about the following:
  - after planning the original query, standard_planner() goes through
the list of top-level CTEs and assigns a running number for each of
the DML WITHs, in the order they will be executed and returns a
list of PlannedStmts.  all necessary statements wi have a flag
signaling that the result should be stored in a tuplestore.
  - the portal keeps the list of tuplestores around and passes that
list to every query through PlannedStmt.  it keeps on executing
the statements until it finds a PlannedStmt that doesn't want its
results stored anywhere and then frees the list of tuplestores

Does this sound reasonable?  And more importantly, am I going to be
wasting my time implementing this?  The 15th isn't that far away..


Regards,
Marko Tiikkaja

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


[HACKERS] log_error_verbosity function display

2010-02-10 Thread Bruce Momjian
Right now, log_error_verbosity displays the source code error location
in this format:

LOCATION:  parserOpenTable, parse_relation.c:858

I think it would be clearer to add '()' next to the function name.  We
use '() to display function names in our docs and I think using '()'
would clarify the output, e.g.:

LOCATION:  parserOpenTable(), parse_relation.c:858

-- 
  Bruce Momjian  br...@momjian.ushttp://momjian.us
  EnterpriseDB http://enterprisedb.com

  + If your life is a hard drive, Christ can be your backup. +

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Writeable CTEs and empty relations

2010-02-10 Thread Robert Haas
On Wed, Feb 10, 2010 at 6:25 PM, Marko Tiikkaja
marko.tiikk...@cs.helsinki.fi wrote:
 On 2010-02-11 00:50 +0200, Marko Tiikkaja wrote:
 On 2010-02-10 23:57 +0200, Tom Lane wrote:
 Robert Haas robertmh...@gmail.com writes:
 If the executor has buried in it the assumption that the snapshot
 can't change after startup, then does that mean that we need to start
 up and shut down the executor for each subquery?

 Yes, I think so.  That's the way it's always worked in the past;
 see for example PortalRunMulti() and ProcessQuery().  I think trying
 to change that is a high-risk, low-reward activity.

 This probably means that the planner output for queries involving
 writeable CTEs has to be a separate PlannedStmt per such CTE.

 I'm looking at this, but I can't think of any good way of associating
 the tuplestores from PortalRunMulti() with the correct CTEs.  Any ideas?

 Ok, what about the following:
  - after planning the original query, standard_planner() goes through
    the list of top-level CTEs and assigns a running number for each of
    the DML WITHs, in the order they will be executed and returns a
    list of PlannedStmts.  all necessary statements wi have a flag
    signaling that the result should be stored in a tuplestore.
  - the portal keeps the list of tuplestores around and passes that
    list to every query through PlannedStmt.  it keeps on executing
    the statements until it finds a PlannedStmt that doesn't want its
    results stored anywhere and then frees the list of tuplestores

Wouldn't you'd want to stop when you ran out of PlannedStmts, not when
you hit one that didn't need its results stored?  What happens if
there's an unreferenced CTE in the middle of the list?

 Does this sound reasonable?  And more importantly, am I going to be
 wasting my time implementing this?  The 15th isn't that far away..

I have to admit I've been starting to have a feeling over the last
couple of days that this patch isn't going to make it for 9.0...  but
obviously I'm in no position to guarantee anything one way or the
other.  Please do keep us up to date on your plans, though.

Thanks,

...Robert

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Odd cruft in .psql_history in HEAD

2010-02-10 Thread Jim Nasby
On Jan 13, 2010, at 9:32 PM, Tom Lane wrote:
 Jim Nasby deci...@decibel.org writes:
 I noticed odd stuff showing up when I fired up an 8.3 psql after using psql 
 in HEAD. It shows up in .psql_history as well:
 
 Platform?  readline version?

This is on snow leopard. FWIW it's still doing it with today's HEAD.

deci...@platter.1[18:05]~/pgsql/HEAD:9%port installed readline|grep active
  readline @6.0.000_2+darwin (active)
deci...@platter.1[18:05]~/pgsql/HEAD:10%uname -a
Darwin platter 10.2.0 Darwin Kernel Version 10.2.0: Tue Nov  3 10:37:10 PST 
2009; root:xnu-1486.2.11~1/RELEASE_I386 i386
deci...@platter.1[18:05]~/pgsql/HEAD:11%

(Original thread is at 
http://archives.postgresql.org/pgsql-hackers/2010-01/msg01414.php; sorry it 
took so long to get back to this).
--
Jim C. Nasby, Database Architect   j...@nasby.net
512.569.9461 (cell) http://jim.nasby.net



-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Writeable CTEs and empty relations

2010-02-10 Thread Marko Tiikkaja
On 2010-02-11 01:58 +0200, Robert Haas wrote:
 On Wed, Feb 10, 2010 at 6:25 PM, Marko Tiikkaja
 marko.tiikk...@cs.helsinki.fi wrote:
 On 2010-02-11 00:50 +0200, Marko Tiikkaja wrote:
 Ok, what about the following:
  - after planning the original query, standard_planner() goes through
the list of top-level CTEs and assigns a running number for each of
the DML WITHs, in the order they will be executed and returns a
list of PlannedStmts.  all necessary statements wi have a flag
signaling that the result should be stored in a tuplestore.
  - the portal keeps the list of tuplestores around and passes that
list to every query through PlannedStmt.  it keeps on executing
the statements until it finds a PlannedStmt that doesn't want its
results stored anywhere and then frees the list of tuplestores
 
 Wouldn't you'd want to stop when you ran out of PlannedStmts, not when
 you hit one that didn't need its results stored?  What happens if
 there's an unreferenced CTE in the middle of the list?

Right, of course.

 Does this sound reasonable?  And more importantly, am I going to be
 wasting my time implementing this?  The 15th isn't that far away..
 
 I have to admit I've been starting to have a feeling over the last
 couple of days that this patch isn't going to make it for 9.0...  but
 obviously I'm in no position to guarantee anything one way or the
 other.  Please do keep us up to date on your plans, though.

Unfortunately, so have I.  I'll take a shot at implementing this, but if
I come across any problems, I have to give up.


Regards,
Marko Tiikkaja

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] psql tab completion for DO blocks

2010-02-10 Thread Bruce Momjian
Takahiro Itagaki wrote:
 
 Bruce Momjian br...@momjian.us wrote:
 
  Where are we on this patch?  We should at least implement the completion
  for 'LANGUAGE' in 'DO', and use the existing pg_language query for
  completion.  I am attaching a patch that does exactly this.
 
 I don't think we need the patch except adding DO to the top-level 
 sql_commands.
 
 Syntax of DO command is:
 DO code [ LANGUAGE lang_name ]
 We need 'code' before LANGUAGE, but the patch completes DO to DO LANGUAGE.
 It will be just a syntax error.
 
 Also, we've already had a completion after LANGUAGE (see words_after_create),
 so we don't need an additional Query_for_list_of_languages after LANGUAGE.

Ah, I see.  I had not checked the patch against the actual syntax; shame
on me.

 BTW, I'm working on psql tab-completion adjustment for new syntax in 9.0.
 I'll send it soon.

OK, thanks.

-- 
  Bruce Momjian  br...@momjian.ushttp://momjian.us
  EnterpriseDB http://enterprisedb.com

  + If your life is a hard drive, Christ can be your backup. +

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] log_error_verbosity function display

2010-02-10 Thread Tom Lane
Bruce Momjian br...@momjian.us writes:
 Right now, log_error_verbosity displays the source code error location
 in this format:

   LOCATION:  parserOpenTable, parse_relation.c:858

 I think it would be clearer to add '()' next to the function name.  We
 use '() to display function names in our docs and I think using '()'
 would clarify the output, e.g.:

   LOCATION:  parserOpenTable(), parse_relation.c:858

Seems like a waste of log space to me.  The convention about writing ()
to decorate function names is hardly universal, and anyway it's mainly
useful to mark things that the reader might not realize are function
names.  This can't be anything else.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


[HACKERS] knngist patch support

2010-02-10 Thread Ragi Y. Burhum
Hello,

I noticed this morning that the k nearest neighbor gist patch
https://commitfest.postgresql.org/action/patch_view?id=230 was still being
considered for inclusion in 9. Sadly, this feature appears to have been
dropped from 9.

It seems to me that the functionality this brings is one of the most common
use cases for mobile applications (or even web in some cases) that are
location enabled. How many times do we find ourselves Finding the 10
nearest restaurants in our smart phones? Well, this patch solves exactly
that in the most efficient manner.

Granted, one can currently solve this problem with PostgreSQL/PostGIS as it
stands, however the performance improvements that one can gain for those
types of (extremely common) queries leveraging the Gist enhancements are,
well, exciting.

300x time improvements? Sounds great to me.

I would love if you guys could consider applying this patch for this
upcoming release.

Best Regards,

- Ragi Burhum


Re: [HACKERS] Confusion over Python drivers {license}

2010-02-10 Thread Kevin Ar18

I hope people don't mind my asking about this on the list... as I hinted at 
before, I don't really follow the development of PostgreSQL, I was just 
interested in the Python driver project that I heard about.

Anyways, as I understand it, the current goal is to use psycopg and get it 
changed to LGPL (assuming all the contributors of psycopg agree and confirm 
they did not use GPL code from any other location).  Is this correct?


When I first heard about the endeavor, I thought the goal was to take one or 
several of the non-copyleft projects, which were rather unfocused, and work 
with those teams to produce a really good implementation for Python.  However, 
as I understand it (based on what Greg told me) the license is not really an 
issue as long as it is not GPL; instead, the PostgreSQL team would mostly 
prefer something that is nearly done, so as to have to do much more work.  Is 
this a correct assessment?


Based on that, I guess my question is what would it have taken to have picked 
one of BSD/MIT projects and working with those people instead?  In other words, 
what key things affected the decision for psycopg?  What areas is it so far 
ahead in or that would have just been too much work to fix in the other 
implementations?


Anyways, I hope this message doesn't come across as bad form.  It's unfortunate 
for me that there was not a good enough BSD/MIT project; but I can live without 
right? :)  Still, I just thought I might ask and find out a little more about 
what the team was looking for in a PostgreSQL implementation, and maybe do a 
little research myself (to see if anything was missed).
  
_
Hotmail: Free, trusted and rich email service.
http://clk.atdmt.com/GBL/go/201469228/direct/01/

Re: [HACKERS] Writeable CTEs and empty relations

2010-02-10 Thread Marko Tiikkaja
On 2010-02-11 02:13 +0200, I wrote:
 On 2010-02-11 01:58 +0200, Robert Haas wrote:
 I have to admit I've been starting to have a feeling over the last
 couple of days that this patch isn't going to make it for 9.0...  but
 obviously I'm in no position to guarantee anything one way or the
 other.  Please do keep us up to date on your plans, though.
 
 Unfortunately, so have I.  I'll take a shot at implementing this, but if
 I come across any problems, I have to give up.

I'm going to have to disappoint a bunch of people and give up. :-(

At this point, I've already used a huge amount of my time and energy to
work on this patch during this commit fest, and I simply can't continue
like this.  There's only 4 days left anyway, and I don't feel confident
enough to spend a lot of time during the next couple of days just to get
one more shot.  I don't even have any kind of certainty that there
would've been a shot, even if I finished the patch on time.

Thanks everyone for helping out and reviewing this, especially Robert
and Tom.


Regards,
Marko Tiikkaja

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Postgres Triggers issue

2010-02-10 Thread Tom Lane
u235sentinel u235senti...@gmail.com writes:
 I have a strange problem we noticed the other day with triggers.  We're 
 running 8.3.3 on Solaris 10 (intel) and have a feed that comes in 
 regularly to populate a table we're working on.  The feed works just 
 fine inserting rows however the following trigger stops the feed until 
 we remove the trigger.  Any thoughts on what I'm doing wrong here?

This is not a -hackers question, and even if it were, you didn't provide
nearly enough context for anybody to do more than guess.  I'm going to
guess foreign key conflict and leave it at that.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] [PATCH] Output configuration status after ./configure run.

2010-02-10 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes:
 On Wed, Feb 10, 2010 at 3:30 PM, Alvaro Herrera
 alvhe...@commandprompt.com wrote:
 If this doesn't fit in 24x80 maybe we need to find a more compact way to
 display things.

 +1.  I wouldn't mind a one-line summary, but a two page summary seems
 like a lot.

So it seems there's some consensus that:

1. This printout should display everything configurable from a configure
option, and nothing else (ie, not any of the platform-dependent
conclusions that configure draws).

2. The printout has to be made to fit in 24x80 or so.

I'm still quite dubious about the usefulness, but I could live with this
if someone explains to me how the printout is going to stay within 24x80
given the inevitable growth in number of configure options ...

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Re: Faster CREATE DATABASE by delaying fsync (was 8.4.1 ubuntu karmic slow createdb)

2010-02-10 Thread Andres Freund
On Monday 08 February 2010 05:53:23 Robert Haas wrote:
 On Sun, Feb 7, 2010 at 10:09 PM, Alvaro Herrera
 
 alvhe...@commandprompt.com wrote:
  Andres Freund escribió:
  I personally think the fsync on the directory should be added to the
  stable branches - other opinions?
  If wanted I can prepare patches for that.
  
  Yeah, it seems there are two patches here -- one is the addition of
  fsync_fname() and the other is the fsync_prepare stuff.
 
 Andres, you want to take a crack at splitting this up?
I hope I didnt duplicate Gregs work, but I didnt hear back from him, so...

Everything 8.1 is hopeless because cp is used there... I didnt see it worth 
to replace that. The patch applies cleanly for 8.1 to 8.4 and survives the 
regression tests

Given pg's heavy commit model I didnt see a point to split the patch for 9.0 
as well...

Andres
diff --git a/src/port/copydir.c b/src/port/copydir.c
index 72fbf36..b057ffa 100644
*** a/src/port/copydir.c
--- b/src/port/copydir.c
*** copydir(char *fromdir, char *todir, bool
*** 50,55 
--- 50,56 
  {
  	DIR		   *xldir;
  	struct dirent *xlde;
+ 	int dirfd;
  	char		fromfile[MAXPGPATH];
  	char		tofile[MAXPGPATH];
  
*** copydir(char *fromdir, char *todir, bool
*** 91,96 
--- 92,116 
  	}
  
  	FreeDir(xldir);
+ 
+ 	/*
+ 	 * fsync the directory to make sure data has reached the
+ 	 * disk. While needed by most filesystems, the window got bigger
+ 	 * with newer ones like ext4.
+ 	 */
+ 	dirfd = BasicOpenFile(todir,
+ 	  O_RDONLY | PG_BINARY,
+ 	  S_IRUSR | S_IWUSR);
+ 	if(dirfd == -1)
+ 		ereport(ERROR,
+ 		(errcode_for_file_access(),
+ 		 errmsg(could not open directory for fsync \%s\: %m, todir)));
+ 
+ 	if(pg_fsync(dirfd) == -1)
+ 		ereport(ERROR,
+ (errcode_for_file_access(),
+  errmsg(could not fsync directory \%s\: %m, todir)));
+ 	close(dirfd);
  }
  
  /*

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Odd cruft in .psql_history in HEAD

2010-02-10 Thread Tom Lane
Jim Nasby deci...@decibel.org writes:
 On Jan 13, 2010, at 9:32 PM, Tom Lane wrote:
 Jim Nasby deci...@decibel.org writes:
 I noticed odd stuff showing up when I fired up an 8.3 psql after using psql 
 in HEAD. It shows up in .psql_history as well:
 
 Platform?  readline version?

 This is on snow leopard. FWIW it's still doing it with today's HEAD.

Oh.  On OSX the regular readline (really libedit) library likes to put
strange stuff into history files --- it turns spaces, backslashes,
and I don't know what else into backslash-octal escape sequences.
It manages to reverse the transformation just fine on read though.

What it looks like to me is that you've started linking psql with
some other version of readline that doesn't follow that convention,
and accordingly shows you strange things from the history file.
When/if you go back to libedit, it likely won't like the history
entries the other library made.

Short answer: stick to one readline library.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] [PATCH] Output configuration status after ./configure run.

2010-02-10 Thread Robert Haas
On Wed, Feb 10, 2010 at 9:17 PM, Tom Lane t...@sss.pgh.pa.us wrote:
 Robert Haas robertmh...@gmail.com writes:
 On Wed, Feb 10, 2010 at 3:30 PM, Alvaro Herrera
 alvhe...@commandprompt.com wrote:
 If this doesn't fit in 24x80 maybe we need to find a more compact way to
 display things.

 +1.  I wouldn't mind a one-line summary, but a two page summary seems
 like a lot.

 So it seems there's some consensus that:

 1. This printout should display everything configurable from a configure
 option, and nothing else (ie, not any of the platform-dependent
 conclusions that configure draws).

 2. The printout has to be made to fit in 24x80 or so.

 I'm still quite dubious about the usefulness, but I could live with this
 if someone explains to me how the printout is going to stay within 24x80
 given the inevitable growth in number of configure options ...

I'm dubious too.  I'm +1 for shorter, but neutral on the idea in general.

...Robert

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Avoiding bad prepared-statement plans.

2010-02-10 Thread Bruce Momjian
Kris Jurka wrote:
 
 The JDBC driver has two methods of disabling permanently planned prepared 
 statements:
 
 1) Use the version two frontend/backend protocol via adding 
 protocolVersion=2 to your URL.  This interpolates all parameters into 
 the query on the client side.
 
 2) Execute PreparedStatements using the unnamed statement rather than a 
 named statement via adding prepareThreshold=0 to your URL.  A named 
 statement is expected to be re-used for later execution and ignores the 
 passed parameters for planning purposes.  An unnamed statement may be 
 re-used, but it doesn't expect to be.  The unnamed statement uses the 
 passed parameters for planning purposes, but still cannot make certain 
 optimatizations based on the parameter values because it may be 
 re-executed again later with different parameters.  For example a LIKE 
 query with a parameter value of 'ABC%' cannot be transformed into range 
 query because the next execution may use a different parameter value for 
 which the transform is not valid.  By default the driver switches to using 
 a named statement after the same PreparedStatement object is executed five 
 times.
 
 http://jdbc.postgresql.org/documentation/84/connect.html#connection-parameters
 http://jdbc.postgresql.org/documentation/84/server-prepare.html

Can someone explain to me why we only do delayed binding for unnamed
prepared queries?  Why do we not allow this option for named protocol
prepared queries and SQL prepared queries?

Here is what our documentation has in the protocols section:

The unnamed prepared statement is likewise planned during Parse processing
if the Parse message defines no parameters.  But if there are parameters,
query planning occurs during Bind processing instead.  This allows the
planner to make use of the actual values of the parameters provided in
the Bind message when planning the query.

and here is someone who is having problems with the generic plans we
create:

http://www.odecee.com.au/blogs/?p=134

Can we not document this better?

-- 
  Bruce Momjian  br...@momjian.ushttp://momjian.us
  EnterpriseDB http://enterprisedb.com

  + If your life is a hard drive, Christ can be your backup. +

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Avoiding bad prepared-statement plans.

2010-02-10 Thread Bruce Momjian
Tom Lane wrote:
 Robert Haas robertmh...@gmail.com writes:
  On Tue, Feb 9, 2010 at 7:08 AM, Jeroen Vermeulen j...@xs4all.nl wrote:
  Periodically re-plan prepared statements on EXECUTE. ?This is also a chance
  for queries that were being re-planned every time to go back to a generic
  plan.
 
  The most common problem here seems to be that (some?) MCVs need
  different treatment than non-MCVs, so I don't think periodically
  replanning is going to help very much.
 
 It won't help at all.  The only reason for replanning is if something
 about the schema or the statistics change, and we already have got
 automatic cached-plan invalidation in both those cases.  If you replan
 simply because some time has elapsed, you'll just get exactly the
 same plan.
 
 The only case that I think still has any merit is where you get a
 significantly better plan with known parameter values than without.
 The projected-cost threshold might be a reasonable approach for
 attacking that, ie, if estimated cost of generic plan exceeds X
 then take the time to build a custom plan instead.  I'm not sure that
 really will fix the problem, but it would be a very simple change to
 make to see how much it helps people.

Ideally we would do late binding (bind on the first supplied parameters,
like we do for unnamed protocol prepared queries now), and then replan
if the statistics for later parameters significantly differ from the
ones used for the the initial planning.

-- 
  Bruce Momjian  br...@momjian.ushttp://momjian.us
  EnterpriseDB http://enterprisedb.com

  + If your life is a hard drive, Christ can be your backup. +

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Confusion over Python drivers {license}

2010-02-10 Thread Tom Lane
Kevin Ar18 kevina...@hotmail.com writes:
 When I first heard about the endeavor, I thought the goal was to take
 one or several of the non-copyleft projects, which were rather
 unfocused, and work with those teams to produce a really good
 implementation for Python.  However, as I understand it (based on what
 Greg told me) the license is not really an issue as long as it is not
 GPL; instead, the PostgreSQL team would mostly prefer something that
 is nearly done, so as to have to do much more work.  Is this a correct
 assessment?

Well, all else being equal we'd certainly prefer a library that was
licensed more like the core Postgres database.  However, we don't have
infinite resources, and an LGPL license is not a showstopper (at least
not to the people who seem to be willing to work on this problem).
The attractiveness of the license has to be balanced against how much
work we'd have to put in and how long it will take to get results.

Not being a python user myself, I wasn't paying all that close attention
to the discussion, but that's my sense of how the decision went.

If you feel that a BSD/MIT license is a must-have for your purposes,
you're certainly free to push development of one of the other driver
projects instead, and to try to organize some other people to help.
I don't believe anyone is trying to funnel all development effort into
psycopg2.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] log_error_verbosity function display

2010-02-10 Thread Bruce Momjian
Tom Lane wrote:
 Bruce Momjian br...@momjian.us writes:
  Right now, log_error_verbosity displays the source code error location
  in this format:
 
  LOCATION:  parserOpenTable, parse_relation.c:858
 
  I think it would be clearer to add '()' next to the function name.  We
  use '() to display function names in our docs and I think using '()'
  would clarify the output, e.g.:
 
  LOCATION:  parserOpenTable(), parse_relation.c:858
 
 Seems like a waste of log space to me.  The convention about writing ()
 to decorate function names is hardly universal, and anyway it's mainly
 useful to mark things that the reader might not realize are function
 names.  This can't be anything else.

I suggested it because it wasn't obvious to me it was a function name,
so I figured others might not recognize it.  Remember, we deal with the
C code all the time so we have to consider how the general user would
see it.

-- 
  Bruce Momjian  br...@momjian.ushttp://momjian.us
  EnterpriseDB http://enterprisedb.com

  + If your life is a hard drive, Christ can be your backup. +

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] [PATCH] Output configuration status after ./configure run.

2010-02-10 Thread Euler Taveira de Oliveira
Tom Lane escreveu:
 I'm still quite dubious about the usefulness, but I could live with this
 if someone explains to me how the printout is going to stay within 24x80
 given the inevitable growth in number of configure options ...
 
AFAICS, we have  40 configure options. If we want this to fit in 24 rows (i)
we should choose popular options or (ii) print only features/packages that
have a non-default option/value. Both ideas aren't ideal for machine-readable
format (as someone mentioned pgbuildfarm) because the summary is partial i.e.
the software needs to know beforehand what are the default configure options.
Of course, parsing the configure output and greping the interested options is
a boring task but at least it is all there; but it's not a summary. :(


-- 
  Euler Taveira de Oliveira
  http://www.timbira.com/

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Confusion over Python drivers {license}

2010-02-10 Thread Kevin Ar18

 Well, all else being equal we'd certainly prefer a library that was
 licensed more like the core Postgres database.  However, we don't have
 infinite resources, and an LGPL license is not a showstopper (at least
 not to the people who seem to be willing to work on this problem).
 The attractiveness of the license has to be balanced against how much
 work we'd have to put in and how long it will take to get results.
 
 Not being a python user myself, I wasn't paying all that close attention
 to the discussion, but that's my sense of how the decision went.
 
 If you feel that a BSD/MIT license is a must-have for your purposes,
 you're certainly free to push development of one of the other driver
 projects instead, and to try to organize some other people to help.
 I don't believe anyone is trying to funnel all development effort into
 psycopg2.
Thanks for the reply.

I guess that's good advice; I suppose I should just do that and talk to some of 
the teams about it.  It would probably help a lot to focus on just one 
implementation instead of several, even if it's not the same one as what the 
PostgreSQL team works on. :)
  
_
Hotmail: Trusted email with Microsoft’s powerful SPAM protection.
http://clk.atdmt.com/GBL/go/201469226/direct/01/

Re: [HACKERS] Avoiding bad prepared-statement plans.

2010-02-10 Thread Tom Lane
Bruce Momjian br...@momjian.us writes:
 Can someone explain to me why we only do delayed binding for unnamed
 prepared queries?

It was a way of shoehorning in some driver control over the behavior
without the protocol bump that would be involved in adding an actual
option to Parse messages.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Postgres Triggers issue

2010-02-10 Thread u235sentinel

Tom Lane wrote:

u235sentinel u235senti...@gmail.com writes:
  
I have a strange problem we noticed the other day with triggers.  We're 
running 8.3.3 on Solaris 10 (intel) and have a feed that comes in 
regularly to populate a table we're working on.  The feed works just 
fine inserting rows however the following trigger stops the feed until 
we remove the trigger.  Any thoughts on what I'm doing wrong here?



This is not a -hackers question, and even if it were, you didn't provide
nearly enough context for anybody to do more than guess.  I'm going to
guess foreign key conflict and leave it at that.

regards, tom lane

  
I'm curious what context you were expecting and which group this should 
go to.  I've posted similar questions in the other groups and they 
seem... lost at times.


Regards'

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Confusion over Python drivers {license}

2010-02-10 Thread Greg Smith

Kevin Ar18 wrote:


Based on that, I guess my question is what would it have taken to have 
picked one of BSD/MIT projects and working with those people instead?  
In other words, what key things affected the decision for psycopg?  
What areas is it so far ahead in or that would have just been too much 
work to fix in the other implementations?


A lot of it as I see it is plain old fashioned popularity resulting from 
long sustained development.  psycopg has been around since 2001, the 
current version is already compatible with SQLAlchemy, Django, Zope, 
PyDO, and it's integral to the large Python/PostgreSQL toolchain from 
Skype including tools like Londiste.  When something is supported in 
that many places, and the main complaints are its license and 
documentation rather than its code, I know I'm not going to start by 
assuming I should just hack on somebody else's code to try and replace 
it just because their license is a little better.  And I'd consider 
BSD/MIT only a little better than the LGPL.


That sort of bad decision making is how we got to so many abandoned 
Python drivers in the first place.  If everybody who decided they were 
going to write their own from scratch had decided to work on carefully 
and painfully refactoring and improving PyGreSQL instead, in an 
incremental way that grew its existing community along the way, we might 
have a BSD driver with enough features and users to be a valid 
competitor to psycopg2.  But writing something shiny new from scratch is 
fun, while worrying about backward compatibility and implementing all 
the messy parts you need to really be complete on a project like this 
isn't, so instead we have a stack of not quite right drivers without any 
broad application support.


(Aside:  I have a bit of vocabulary I regularly use for this now.  
Open-source projects that just ignore working on an existing stack of 
working implementations, to instead start from scratch to build 
something of questionably improved fanciness instead regardless of its 
impact on backward compatibility and the existing userbase, have in my 
terminology PulseAudioed the older projects).


The gauntlet I would throw down for anyone who thinks there's a better 
approach here is to demonstrate a driver that has working support going 
back to Python 2.4 for any two of the apps on my list above.  Get even 
that much of a foothold, and maybe the fact that yours is more Pythonic, 
cleaner, or better licensed matters.  Until then, working apps have to 
be the primary motivation for what to work on here, unless there's a 
really terrible problem with the driver.  The existing psycopg license 
and the web site issues were in combination enough to reach that level 
of total issues for a number of people.  With the news from Federico 
that a license improvement is approaching and signs of major 
documentation improvements, that problem seems like it will soon be 
solved as far as I'm concerned.  When someone manages to make their 
alternative driver a prerequisite for an app I need, only at that point 
will that driver show back up on my radar.


--
Greg Smith2ndQuadrant   Baltimore, MD
PostgreSQL Training, Services and Support
g...@2ndquadrant.com  www.2ndQuadrant.com


--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Postgres Triggers issue

2010-02-10 Thread Euler Taveira de Oliveira
u235sentinel escreveu:
 I'm curious what context you were expecting and which group this should
 go to.  I've posted similar questions in the other groups and they
 seem... lost at times.
 
What group? AFAICS this question belongs to -general. What about starting to
show us the definition of m_a, temp_m_t, and related tables?


-- 
  Euler Taveira de Oliveira
  http://www.timbira.com/

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Confusion over Python drivers

2010-02-10 Thread Andrew McNamara
That's just a matter of prioritizing the issues.  Put the big ones at 
the top, the trivia at the bottom, [...]

I'd like to see a requirement for the use of PQexecParams() over PQexec() - 
even when using libpq's PQescapeStringConn(), PQexec() makes me uneasy.

-- 
Andrew McNamara, Senior Developer, Object Craft
http://www.object-craft.com.au/

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] knngist patch support

2010-02-10 Thread tomas
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On Wed, Feb 10, 2010 at 04:49:59PM -0800, Ragi Y. Burhum wrote:
 Hello,
 
 I noticed this morning that the k nearest neighbor gist patch
 https://commitfest.postgresql.org/action/patch_view?id=230 was still being
 considered for inclusion in 9. Sadly, this feature appears to have been
 dropped from 9.

This has been discussed recently on this list. Seems the patch would
need more review to be considered stable. So it's the hard choice of
letting the schedule for 9.0 slip or not letting this patch in.

But some prerequisites will go in, that's the good news.

(BTW: I tried to find this discussion in the Web archives, but had no
luck. It's in my mailbox, though --

e.g.

 message-ID 603c8f071002070527j1dada7cdseb42e7cbc71bf...@mail.gmail.com

part of the long thread Damage control mode, starting on Jan 8, 2010;
this one mail is from Feb 7 -- but that might be me)

Regards
- -- tomás
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.6 (GNU/Linux)

iD8DBQFLc5YoBcgs9XrR2kYRAoW3AJ94tYWPenLOjH4B4GHD9DCYSSWYOQCeOcoM
RYDhINv+k9YeD23xFHyj9yw=
=K1E0
-END PGP SIGNATURE-

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Confusion over Python drivers

2010-02-10 Thread Tom Lane
Andrew McNamara andr...@object-craft.com.au writes:
 That's just a matter of prioritizing the issues.  Put the big ones at 
 the top, the trivia at the bottom, [...]

 I'd like to see a requirement for the use of PQexecParams() over PQexec() - 
 even when using libpq's PQescapeStringConn(), PQexec() makes me uneasy.

Such a rule seems pretty entirely pointless, unless you have a way to
enforce that the query string passed to the function hasn't been
assembled from parts somewhere along the way.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Confusion over Python drivers {license}

2010-02-10 Thread Greg Smith

Tom Lane wrote:

If you feel that a BSD/MIT license is a must-have for your purposes,
you're certainly free to push development of one of the other driver
projects instead, and to try to organize some other people to help.
I don't believe anyone is trying to funnel all development effort into
psycopg2.
  


Certainly not, and I hope no one has gotten the impression that there's 
anything official being recognized about psycopg happening here 
because it hasn't.  Anointing a one true driver (a phrase that seems 
to keep popping up in external discussions of this topic) isn't the sort 
of thing the PostgreSQL core does.  And all that happens to people who 
ignore what I tell them to do is that they receive a steady stream of 
long, sarcastic e-mails for a while afterwards.


I just updated 
http://wiki.postgresql.org/wiki/Python_PostgreSQL_Driver_TODO to reflect 
a few corrections I noted while researching (everybody else is welcome 
to edit that page too you know), and to include some of the recent 
feedback showing up on this list.


I know I was just looking for a Python driver that is compatible with 
the apps I most often run into, documented well enough that I can write 
my own if I feel like it, fast enough, and free enough that I can deploy 
the result wherever I want.  That seemed similar to the priorities other 
people who had an opinion here suggested too.  Pragmatically, psycopg2 
just seemed to have the shortest path toward being something useful to 
the largest userbase in that sort of context, and we've unofficially 
rolled down that path a bit.


This rabble-rousing seems to have nudged both development communities 
toward being more closely aligned in the future in a couple of ways, 
which is great, but I wouldn't read much more into things than that.  
Other projects will continue to grow and shrink, and the pure Python 
ones in particular continue to be quite valuable because they fill a 
niche that psycopg2 doesn't target at all.  I'd sure like to see all 
three of those projects merge into one big one though.  My bet has to be 
on pg8000, since it has the perfect license, supports all the necessary 
Python versions, and it's been around long enough (almost two years) 
that support for it is already in the latest SQLAlchemy beta.


We seem to have revitalized discussion around modernizing PyGreSQL too, 
so I wouldn't discount that one completely yet either.  For those who 
feel a true BSD license is vital, I direct you toward 
http://mailman.vex.net/pipermail/pygresql/2010-February/002315.html to 
learn more about what direction they could use some help in going.  But 
whatever you do, don't start another project instead.


--
Greg Smith2ndQuadrant   Baltimore, MD
PostgreSQL Training, Services and Support
g...@2ndquadrant.com  www.2ndQuadrant.com


--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Confusion over Python drivers

2010-02-10 Thread Andrew McNamara
 I'd like to see a requirement for the use of PQexecParams() over PQexec() - 
 even when using libpq's PQescapeStringConn(), PQexec() makes me uneasy.

Such a rule seems pretty entirely pointless, unless you have a way to
enforce that the query string passed to the function hasn't been
assembled from parts somewhere along the way.

The point is that if the driver is doing the right thing, the user of
the driver at least has to choice to do things safely.

-- 
Andrew McNamara, Senior Developer, Object Craft
http://www.object-craft.com.au/

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] knngist patch support

2010-02-10 Thread Oleg Bartunov

This is very disgraceful from my point of view and reflects real problem
in scheduling of CF. The patch was submitted Nov 23 2009, discussed and 
reworked Nov 25. Long holidays in December-January, probably are reason why

there were no any movement on reviewing the patch. People with
inspiration spent time to discuss rbtree, while it was clear, that rbtree is 
a minor issue. Now we have no review and great feature is missing. I understand

that some  healthy resistance is useful to let developers more accurate and
discipline, but, hey,  not dropping great feature ! I'd understand if developer
is missing, or just not willing to contact, but I and Teodor are here and 
we readily answer any questions.


I failed to find any documents about commitfest to understand if we already
discussed all possible scenario of feature drop. If we say 'A', when started
to formalize our development process instead of old way discussvote 
in -hackers, then we should say 'B' - formalize procedure and possible

collision of interests.


Oleg
On Thu, 11 Feb 2010, to...@tuxteam.de wrote:


-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On Wed, Feb 10, 2010 at 04:49:59PM -0800, Ragi Y. Burhum wrote:

Hello,

I noticed this morning that the k nearest neighbor gist patch
https://commitfest.postgresql.org/action/patch_view?id=230 was still being
considered for inclusion in 9. Sadly, this feature appears to have been
dropped from 9.


This has been discussed recently on this list. Seems the patch would
need more review to be considered stable. So it's the hard choice of
letting the schedule for 9.0 slip or not letting this patch in.

But some prerequisites will go in, that's the good news.

(BTW: I tried to find this discussion in the Web archives, but had no
luck. It's in my mailbox, though --

e.g.

message-ID 603c8f071002070527j1dada7cdseb42e7cbc71bf...@mail.gmail.com

part of the long thread Damage control mode, starting on Jan 8, 2010;
this one mail is from Feb 7 -- but that might be me)

Regards
- -- tomЪЪs
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.6 (GNU/Linux)

iD8DBQFLc5YoBcgs9XrR2kYRAoW3AJ94tYWPenLOjH4B4GHD9DCYSSWYOQCeOcoM
RYDhINv+k9YeD23xFHyj9yw=
=K1E0
-END PGP SIGNATURE-




Regards,
Oleg
_
Oleg Bartunov, Research Scientist, Head of AstroNet (www.astronet.ru),
Sternberg Astronomical Institute, Moscow University, Russia
Internet: o...@sai.msu.su, http://www.sai.msu.su/~megera/
phone: +007(495)939-16-83, +007(495)939-23-83
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Avoiding bad prepared-statement plans.

2010-02-10 Thread Дмитрий Фефелов
 The only case that I think still has any merit is where you get a
 significantly better plan with known parameter values than without.
 The projected-cost threshold might be a reasonable approach for
 attacking that, ie, if estimated cost of generic plan exceeds X
 then take the time to build a custom plan instead.  I'm not sure that
 really will fix the problem, but it would be a very simple change to
 make to see how much it helps people.
 
   regards, tom lane
 

It will definitely help with partitioned tables. It's very common case when 
raw data taken from hardware stored in single table first, and later we start 
to make partitions for each month/week/day. Feature can improve performance 
transparently to client apps.

regards, 
Dmitry

 -- 
 Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
 To make changes to your subscription:
 http://www.postgresql.org/mailpref/pgsql-hackers
 

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] knngist patch support

2010-02-10 Thread Tom Lane
Oleg Bartunov o...@sai.msu.su writes:
 This is very disgraceful from my point of view and reflects real problem
 in scheduling of CF. The patch was submitted Nov 23 2009, discussed and 
 reworked Nov 25. Long holidays in December-January, probably are reason why
 there were no any movement on reviewing the patch.

There was a scheduling problem all right, which was that this patch *did
not make* the deadline for the November CF.  The fact that it got any
review at all in November was more than expected under the CF process.
And I remind you that we stated more than once that we didn't want major
feature patches to show up only at the last CF.  If it had come from
anyone other than you and Teodor, there would have not been even a
moment's consideration of letting it into 9.0.

My own feeling about it is that I much preferred the original proposal
of a contrib module with little or no change to core code.  I don't want
to be changing core code for this at this late hour.  If it were only
touching GIST I'd be willing to rely on your and Teodor's expertise in
that module, but it's not.  It whacks around the planner, it makes
questionable changes in the operator class structure, and the last
version I saw hadn't any documentation whatever.  It's not committable
on documentation grounds alone, even if everybody was satisfied about
the code.

How do you feel about going back to the original contrib module for now
and resubmitting the builtin version for 9.1?

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


  1   2   >