Re: [PATCHES] libpq object hooks

2008-05-16 Thread Andrew Chernow

Tom Lane wrote:

 ISTM the hook
ought to be able to request that libpq return an out-of-memory failure
for the query, just as would happen if the malloc failure had happened
directly to libpq.




I am working on this new patch and that is how I have been implementing it.  If 
the eventProc function returns FALSE for creation events (not destruction 
events), the libpq call that triggered it will fail.  For instance: for the 
creation of result objects "PGEVT_RESULTCREATE", I am clearing the result and 
returning an error value.


I think that is the expected behavior when one calls PQexec and an out-of-memory 
failure occurs.


--
Andrew Chernow
eSilo, LLC
every bit counts
http://www.esilo.com/

--
Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-patches


Re: [PATCHES] libpq object hooks

2008-05-16 Thread Tom Lane
"Merlin Moncure" <[EMAIL PROTECTED]> writes:
> On Fri, May 16, 2008 at 4:23 PM, Tom Lane <[EMAIL PROTECTED]> wrote:
>> So you're imagining that on creation of a PGconn or PGresult, the
>> hook function would be allowed to create some storage and libpq would
>> then be responsible for tracking that storage?  Okay, but that's a
>> separate feature from the sort of passthrough I had in mind.  Where
>> exactly does the hook hand off the storage pointer to libpq?  What
>> are you going to do if the hook fails to create the storage
>> (ie, out of memory during PGresult creation)?

> Right, this could happen for one of two likely reasons: OOM as you
> suggest or the callback fails for some reason only known to the
> hooking library.  In either case, the callback would set the return
> pointer as defined by the structure to null, return FALSE, and libpq
> would not add the state to the array of 'event state datas' it
> maintains.

So at that point we have an apparently-successful creation of a
PGresult, but some of the hook library's functionality is going to fail
to work with that PGresult.  That doesn't seem very nice.  ISTM the hook
ought to be able to request that libpq return an out-of-memory failure
for the query, just as would happen if the malloc failure had happened
directly to libpq.

regards, tom lane

-- 
Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-patches


Re: [PATCHES] [HACKERS] TRUNCATE TABLE with IDENTITY

2008-05-16 Thread Tom Lane
Neil Conway <[EMAIL PROTECTED]> writes:
> Ugh. The fact that the RESTART IDENTITY part of TRUNCATE is
> non-transactional is a pretty unsightly wort.

Actually, I agree.  Shall we just revert that feature?  The ALTER
SEQUENCE part of this patch is clean and useful, but I'm less than
enamored of the TRUNCATE part.

regards, tom lane

-- 
Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-patches


Re: [PATCHES] [HACKERS] TRUNCATE TABLE with IDENTITY

2008-05-16 Thread Neil Conway
On Fri, 2008-05-16 at 19:41 -0400, Tom Lane wrote:
> Applied with corrections.  Most notably, since ALTER SEQUENCE RESTART
> is nontransactional like most other ALTER SEQUENCE operations, I
> rearranged things to try to ensure that foreseeable failures like
> deadlock and lack of permissions would be detected before TRUNCATE
> starts to issue any RESTART commands.

Ugh. The fact that the RESTART IDENTITY part of TRUNCATE is
non-transactional is a pretty unsightly wort. I would also quarrel with
your addition to the docs that suggests this is only an issue "in
practice" if TRUNCATE RESTART IDENTITY is used in a transaction block:
unpredictable failures (such as OOM or query cancellation) can certainly
occur in practice, and would be very disruptive (e.g. if the sequence
values are stored into a column with a UNIQUE constraint, it would break
all inserting transactions until the DBA intervenes).

I wonder if it would be possible to make the sequence operations
performed by TRUNCATE transactional: while the TRUNCATE remains
uncommitted, it should be okay to block concurrent access to the
sequence.

-Neil



-- 
Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-patches


Re: [PATCHES] libpq object hooks

2008-05-16 Thread Andrew Chernow

Tom Lane wrote:

Where
exactly does the hook hand off the storage pointer to libpq?  What
are you going to do if the hook fails to create the storage
(ie, out of memory during PGresult creation)?


The current submitted patch has 3 of its function callbacks returning a 
void*, initHookData after the creation of a conn, resultCreate, and 
resultCopy.  We have recently changed this design so all hooks, now 
called events, go through a single callback ... PGEventProc.  The old 
function callbacks are just eventIds now.


/
// The libpq side (looping registered event procs)
PGEventResultCreate info;
info.stateData = NULL; /* our event data ptr */
info.conn = conn;
info.result = result;

if(!result->evtState[i].proc(PGEVT_RESULTCREATE,
  (void *)&info, result->evtState[i].passThrough)
{
  PQclear(result); // destroy result? ... not sure
  return error;// previously, we ignored it
}

// assign event data created by event proc.
result->evtState[i].data = info.stateData;


///
// example of what the event proc does
int my_evtproc(PGEventId evtId, void *evtInfo, void *passThrough)
{
  switch(eventId)
  {
case PGEVT_RESULTCREATE:
{
  void *data = makeresultdata()
  if(!data)
return FALSE;
  ((PGEventResultCreate *)evtInfo)->stateData = data;
  break;
}
  }

  return TRUE;
}


--
Andrew Chernow
eSilo, LLC
every bit counts
http://www.esilo.com/

--
Andrew Chernow
eSilo, LLC
every bit counts
http://www.esilo.com/

--
Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-patches


Re: [PATCHES] [HACKERS] TRUNCATE TABLE with IDENTITY

2008-05-16 Thread Tom Lane
Zoltan Boszormenyi <[EMAIL PROTECTED]> writes:
>> Attached patch implements the extension found in the current SQL200n draft,
>> implementing stored start value and supporting ALTER SEQUENCE seq RESTART;

> Updated patch implements TRUNCATE ... RESTART IDENTITY
> which restarts all owned sequences for the truncated table(s).

Applied with corrections.  Most notably, since ALTER SEQUENCE RESTART
is nontransactional like most other ALTER SEQUENCE operations, I
rearranged things to try to ensure that foreseeable failures like
deadlock and lack of permissions would be detected before TRUNCATE
starts to issue any RESTART commands.

One interesting point here is that the patch as submitted allowed
ALTER SEQUENCE MINVALUE/MAXVALUE to be used to set a sequence range
that the original START value was outside of.  This would result in
a failure at ALTER SEQUENCE RESTART.  Since, as stated above, we
really don't want that happening during TRUNCATE, I adjusted the
patch to make such an ALTER SEQUENCE fail.  This is at least potentially
an incompatible change: command sequences that used to be legal could
now fail.  I doubt it's very likely to bite anyone in practice, though.

regards, tom lane

-- 
Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-patches


Re: [PATCHES] libpq object hooks

2008-05-16 Thread Merlin Moncure
On Fri, May 16, 2008 at 4:23 PM, Tom Lane <[EMAIL PROTECTED]> wrote:
> "Merlin Moncure" <[EMAIL PROTECTED]> writes:
>> Right.  I actually overlooked the 'passthrough' in
>> PQregisterEventProc.  It turns out that we are still not quite on the
>> same page and this needs to be clarified before we move on.  The
>> passthrough cannot exist...
>
> Yes, it *will* exist.  You are assuming that the goals you have for
> these hooks are the only ones anyone will ever have.  There are other
> possible usage patterns and many of them will need some passthrough
> state data.  I'd venture to say that anytime I've ever used a callback
> design that did not include a passthrough, I've had reason to curse
> its designer sooner or later (qsort being a pretty typical example).

right.  It can exist, just not hold the event data (the extended
properties of PGresult/conn).  It has a meaning and scope that are not
defined for libpq purposes.  For the 'result' callbacks (see below),
we will just use the passthrough passed in for the connection.
libpqtypes will not use this, and we were getting nervous about people
understanding what the different pointers were used for (we would call
it a user pointer, not a pass through).

> However, it's clear that we are not on the same page, because what
> I thought you wanted the PQeventData function to return was the
> passthrough pointer.  Evidently that's not what you want it to do,
> so you'd better back up a few steps and say what you do want it to do.
> (I think that a function to get the passthrough value associated with
> a given hook function might be useful too, but it's clear that what

We will do this:
void *PQeventData(PGconn*, PGeventProc, void **passthrough);
void *PQresultEventData(PGresult*, PGeventProc, void **passthrough);

> you are after must be something else.)

The major challenge is that libpqtypes (and, presumably, other
libraries) must essentially store its own data in both conn and
result.  We allocate our data when these structures are created and
free it when they are destroyed.  The idea is that libpq fires
callbacks at appropriate times when conn/results are
constructed/destructed.  From these events we set up our private
'event' data and delete it.  Each connection and result maintains a
'list' of private event states, one for each registering callback
(libpqtypes only requires one).  So, registering an event proc is
analogous to adding one or fields to a conn or a result with a private
meaning.

So, libpq is defining 6 events:
*) when a connection is created (put into OK status)
*) when a connection is reset
*) when a connection is destroyed
*) when a result is created in non-error state
*) when a result is copied (we are adding this to libpq w/PQcopyResult)
*) when a result is destroyed

In these events we set up or tear down the private state for the libpq
objects.  We need to sometimes look up the data from outside the
library in public (non callback) code.  This is what PQeventData, etc.
accomplish...provide the private state for the object.   The
difference is that there can be more than one registered event proc on
a conn/result at a time;  thus the need for an additional "lookup"
value when getting event data (what was hookName and is now the event
proc address).

>> The purpose of the callbacks is to allow a hooking library to
>> establish private data that is associated with a PGconn or a PGresult.
>
> So you're imagining that on creation of a PGconn or PGresult, the
> hook function would be allowed to create some storage and libpq would
> then be responsible for tracking that storage?  Okay, but that's a
> separate feature from the sort of passthrough I had in mind.  Where
> exactly does the hook hand off the storage pointer to libpq?  What
> are you going to do if the hook fails to create the storage
> (ie, out of memory during PGresult creation)?

Right, this could happen for one of two likely reasons: OOM as you
suggest or the callback fails for some reason only known to the
hooking library.  In either case, the callback would set the return
pointer as defined by the structure to null, return FALSE, and libpq
would not add the state to the array of 'event state datas' it
maintains.

Here is the event proc with passthrough added:
int PGEventProc(PGEventId event, void *eventInfo, void *passThrough);
// FALSE = failure

>> Invoking PQregisterEventProc tells libpq 'I am interested in doing
>> this', for the supplied PGconn, future results created with that
>> connection, and (if PGconn is passed as null), all future connections.
>
> I am entirely serious about saying that I will not accept that last bit.
> To do that we have to buy into having permanent modifiable storage
> within libpq, protecting it with thread mutexes, etc etc.  And I don't
> like any of the possible usage scenarios for it.  I think hooking into a
> PGconn immediately after its creation is just as useful and a lot easier
> to manage.

Locking is not required so long as you follo

Re: [PATCHES] libpq object hooks

2008-05-16 Thread Tom Lane
"Merlin Moncure" <[EMAIL PROTECTED]> writes:
> Right.  I actually overlooked the 'passthrough' in
> PQregisterEventProc.  It turns out that we are still not quite on the
> same page and this needs to be clarified before we move on.  The
> passthrough cannot exist...

Yes, it *will* exist.  You are assuming that the goals you have for
these hooks are the only ones anyone will ever have.  There are other
possible usage patterns and many of them will need some passthrough
state data.  I'd venture to say that anytime I've ever used a callback
design that did not include a passthrough, I've had reason to curse
its designer sooner or later (qsort being a pretty typical example).

However, it's clear that we are not on the same page, because what
I thought you wanted the PQeventData function to return was the
passthrough pointer.  Evidently that's not what you want it to do,
so you'd better back up a few steps and say what you do want it to do.
(I think that a function to get the passthrough value associated with
a given hook function might be useful too, but it's clear that what
you are after must be something else.)

> The purpose of the callbacks is to allow a hooking library to
> establish private data that is associated with a PGconn or a PGresult.

So you're imagining that on creation of a PGconn or PGresult, the
hook function would be allowed to create some storage and libpq would
then be responsible for tracking that storage?  Okay, but that's a
separate feature from the sort of passthrough I had in mind.  Where
exactly does the hook hand off the storage pointer to libpq?  What
are you going to do if the hook fails to create the storage
(ie, out of memory during PGresult creation)?

> Invoking PQregisterEventProc tells libpq 'I am interested in doing
> this', for the supplied PGconn, future results created with that
> connection, and (if PGconn is passed as null), all future connections.

I am entirely serious about saying that I will not accept that last bit.
To do that we have to buy into having permanent modifiable storage
within libpq, protecting it with thread mutexes, etc etc.  And I don't
like any of the possible usage scenarios for it.  I think hooking into a
PGconn immediately after its creation is just as useful and a lot easier
to manage.

> Once that is established, libpq begins telling the hooking library
> when and what needs to be allocated or deleted.

Wait a minute --- you're trying to get between libpq and malloc?
Why?  That's getting a *whole* lot more invasive than I thought
this patch intended to be, and I don't see a good reason for it.

regards, tom lane

-- 
Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-patches


Re: [PATCHES] libpq object hooks

2008-05-16 Thread Merlin Moncure
On Fri, May 16, 2008 at 3:49 PM, Merlin Moncure <[EMAIL PROTECTED]> wrote:
> On Fri, May 16, 2008 at 2:34 PM, Andrew Chernow <[EMAIL PROTECTED]> wrote:
>> Tom Lane wrote:
>>>
>>> typedef void (*PGeventProc) (PGeventId eventId, const void *eventInfo,
>>> void *passthrough);
>>>
>>> int PQregisterEventProc(PGconn *conn, PGeventProc proc, void
>>> *passthrough);
>
>> The above prototypes will work and we will add our 'event instance pointer'
>> to the event info structures.  Should have a patch shortly.
>
>
> Right.  I actually overlooked the 'passthrough' in
> PQregisterEventProc.  It turns out that we are still not quite on the
> same page and this needs to be clarified before we move on.  The
> passthrough cannot exist...the correct prototypes (reasoning will
> follow) are:
>
> typedef void (*PGeventProc) (PGeventId eventId, const void *eventInfo);

small typo: eventInfo obviously can't be const

> int PQregisterEventProc(PGconn *conn, PGeventProc proc);
> PQhookData(const PGconn* conn, PGeventProc proc);

PQhookData is the old name...we are going with 'events' nowthe
proper names will come with the patch.

merlin

-- 
Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-patches


Re: [PATCHES] libpq object hooks

2008-05-16 Thread Merlin Moncure
On Fri, May 16, 2008 at 2:34 PM, Andrew Chernow <[EMAIL PROTECTED]> wrote:
> Tom Lane wrote:
>>
>> typedef void (*PGeventProc) (PGeventId eventId, const void *eventInfo,
>> void *passthrough);
>>
>> int PQregisterEventProc(PGconn *conn, PGeventProc proc, void
>> *passthrough);

> The above prototypes will work and we will add our 'event instance pointer'
> to the event info structures.  Should have a patch shortly.


Right.  I actually overlooked the 'passthrough' in
PQregisterEventProc.  It turns out that we are still not quite on the
same page and this needs to be clarified before we move on.  The
passthrough cannot exist...the correct prototypes (reasoning will
follow) are:

typedef void (*PGeventProc) (PGeventId eventId, const void *eventInfo);
int PQregisterEventProc(PGconn *conn, PGeventProc proc);
PQhookData(const PGconn* conn, PGeventProc proc);
...etc.

The purpose of the callbacks is to allow a hooking library to
establish private data that is associated with a PGconn or a PGresult.
 Invoking PQregisterEventProc tells libpq 'I am interested in doing
this', for the supplied PGconn, future results created with that
connection, and (if PGconn is passed as null), all future connections.

Once that is established, libpq begins telling the hooking library
when and what needs to be allocated or deleted.  This is done for both
connections and results at the appropriate times and the management
always occurs inside a callback function.  Thus, while the hooking
library controls what is in the state data, _it does not control when
it is created or destroyed_.  Passing a void* into PQregisterEventProc
suggests that is the wrapper's purvue to establish the private
information.  It isn't, and it can't be..it's as simple as that.

For example, consider that 'hooked' PGconn then produces a result.  If
a passthrough as Tom suggested were passed around, what would you do
to it when the connection was freed (which throws a connection closing
event)?  What exactly is passthrough pointing to in a result copy
event?  If passed around as a single poitner, they are not
defined.These are pointers to different structures who are created at
particular times that only libpq knows.

All of the callbacks (except for possibly conn reset) are basically
allocation and deletion events.  For example, result create is libpq
telling the hooking library: 'a result is coming, please initialize
your result extensions now'.  At that point, a very particular
structure is passed back to the hooking library:

{
  const PGconn* conn;   // the source conn the result
  const void *conn_state; // the conn's private state (it might be
interesting, and we don't want to look it up later)
  const PGresult* res; // the new result's poiter
  void *out_res_state; // we are going to allocate something and set
this in the callback
}

As you can see, there is more going on here than a single passthrough
pointer can handle.  We are in fact doing what a passthrough provides
in principle.  I can certainly understand why our original 'lookup by
name' concept threw up a smokescreen (it did obfuscate the passing
requirements), but hooking can't be serviced by a single pointer set
at callback registration.

With the above prototypes and carefully designed structures,
everything feels natural.  libpq is defining exactly what is supposed
to happen, and the structure parameters define what is coming in and
out.  Things are being 'passed through'...but in a more discrete way.

merlin

-- 
Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-patches


Re: [PATCHES] Partial match in GIN (next vesrion)

2008-05-16 Thread Oleg Bartunov

Wildspeed was designed as an example application of the GIN's partial
match and as a useful extension for *short* strings. It's also good 
standalone demonstration of GIN API. We tried to stay away from full text

search, parser, word delimiters and etc.
From that point of view it might be 

useful contrib, but I agree we have to think better to let it more
usable.

Oleg

On Fri, 16 May 2008, Tom Lane wrote:


Teodor Sigaev <[EMAIL PROTECTED]> writes:

http://www.sigaev.ru/misc/partial_match_gin-0.10.gz
http://www.sigaev.ru/misc/tsearch_prefix-0.9.gz
http://www.sigaev.ru/misc/wildspeed-0.12.tgz


I've applied the first two of these with minor editorialization (mostly
fixing documentation).  However, I'm having a hard time convincing myself
that anyone will find wildspeed useful in its current form.  I did a
simple experiment using a table of titles of database papers:

contrib_regression=# select count(*), avg(length(title)) from pub;
count  | avg
+-
236984 | 64.7647520507713601
(1 row)

This takes about 22MB on disk as a Postgres table.  I was expecting the
wildspeed index to be about 65 times as large, which is bad enough
already, but actually it weighed in at 2165MB or nearly 100X bigger.
Plus it took forever to build: 35 minutes on a fairly fast machine
with maintenance_work_mem set to 512MB.

In comparison, building a conventional full-text-search index (GIN
tsvector) took about 22 seconds including constructing the tsvector
column, and the tsvectors plus index take about 54MB.  The relative
search performance is about what you'd expect from the difference in
index sizes, ie, wildspeed loses.

So I'm thinking wildspeed really needs to be redesigned if it's to be
anything but a toy.  I can't see putting it into contrib in this form.

One idea that I had was to break the given string into words (splitting
at spaces or punctuation) and store the rotations of individual words
instead of the whole string.  (Actually, maybe you only need suffixes
not rotations, ie for 'abcd' store 'abcd', 'bcd', 'cd', 'd'.)  Then
similarly break the LIKE pattern apart at words to create word-fragment
search keys.  In this scheme the operator would always(?) require
rechecking since any part of the pattern involving punctuation wouldn't
be checkable by the index.  The advantage is that the index bloat factor
is governed by the average word length not the average whole-string
length.

There are probably other approaches that would help, too.

regards, tom lane



Regards,
Oleg
_
Oleg Bartunov, Research Scientist, Head of AstroNet (www.astronet.ru),
Sternberg Astronomical Institute, Moscow University, Russia
Internet: [EMAIL PROTECTED], http://www.sai.msu.su/~megera/
phone: +007(495)939-16-83, +007(495)939-23-83

--
Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-patches


Re: [PATCHES] libpq object hooks

2008-05-16 Thread Andrew Chernow

Tom Lane wrote:


typedef void (*PGeventProc) (PGeventId eventId, const void *eventInfo,
 void *passthrough);

int PQregisterEventProc(PGconn *conn, PGeventProc proc, void *passthrough);





The above prototypes will work and we will add our 'event instance 
pointer' to the event info structures.  Should have a patch shortly.


libpqtypes doesn't need a passthrough/user-pointer.  The object 
events/hooks allocate memory when the object is created "part of a 
conn/result object instance", it is not supplied by the API user 
registering the event/hook callback.


I think this is where some confusion has been occurring, there are two 
different pointers: user pointer and event instance pointer.


BTW, PQeventData and PQresultEventData return the event instance 
pointer, not the passthrough.  At least that is how we were using these 
functions, being how our previous patches do not include a 
passthrough/user-pointer feature because libpqtypes didn't need it.


--
Andrew Chernow
eSilo, LLC
every bit counts
http://www.esilo.com/

--
Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-patches


Re: [PATCHES] libpq thread-locking

2008-05-16 Thread Magnus Hagander
Bruce Momjian wrote:
> Magnus Hagander wrote:
> > Bruce Momjian wrote:
> > > Bruce Momjian wrote:
> > > > Magnus Hagander wrote:
> > > > > Attached patch adds some error checking to the thread locking
> > > > > stuff in libpq. Previously, if thread locking failed for some
> > > > > reason, we would just fall through and do things without
> > > > > locking. This patch makes us abort() instead. It's not the
> > > > > greatest thing probably, but our API doesn't let us pass back
> > > > > return values...
> > > > 
> > > > I have looked over the patch and it seems fine, though I am
> > > > concerned about the abort() case with no output.  I realize
> > > > stderr might be going nowhere, but in fe-print.c we do an
> > > > fprintf(stderr) for memory failures so for consistency I think
> > > > we should do the same here.  If there is concern about code
> > > > bloat, I suggest a macro at the top of the file for thread
> > > > failure exits:
> > > > 
> > > > #define THEAD_FAILURE(str) \
> > > > do { \
> > > > fprintf(stderr, libpq_gettext("Thread failure:
> > > > %s\n")); \ exit(1); \
> > > > } while(0)
> > > 
> > > Oh, this is Tom saying he doesn't like stderr and the added code
> > > lines for failure:
> > > 
> > >   http://archives.postgresql.org/pgsql-patches/2008-04/msg00254.php
> > > 
> > > I think the macro and consistency suggest doing as I outlined.
> > 
> > Does this one look like what you're suggesting?
> 
> Yep.

Ok, applied.

//Magnus

-- 
Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-patches


Re: [PATCHES] Partial match in GIN (next vesrion)

2008-05-16 Thread Tom Lane
Teodor Sigaev <[EMAIL PROTECTED]> writes:
> http://www.sigaev.ru/misc/partial_match_gin-0.10.gz
> http://www.sigaev.ru/misc/tsearch_prefix-0.9.gz
> http://www.sigaev.ru/misc/wildspeed-0.12.tgz

I've applied the first two of these with minor editorialization (mostly
fixing documentation).  However, I'm having a hard time convincing myself
that anyone will find wildspeed useful in its current form.  I did a
simple experiment using a table of titles of database papers:

contrib_regression=# select count(*), avg(length(title)) from pub;
 count  | avg 
+-
 236984 | 64.7647520507713601
(1 row)

This takes about 22MB on disk as a Postgres table.  I was expecting the
wildspeed index to be about 65 times as large, which is bad enough
already, but actually it weighed in at 2165MB or nearly 100X bigger.
Plus it took forever to build: 35 minutes on a fairly fast machine
with maintenance_work_mem set to 512MB.

In comparison, building a conventional full-text-search index (GIN
tsvector) took about 22 seconds including constructing the tsvector
column, and the tsvectors plus index take about 54MB.  The relative
search performance is about what you'd expect from the difference in
index sizes, ie, wildspeed loses.

So I'm thinking wildspeed really needs to be redesigned if it's to be
anything but a toy.  I can't see putting it into contrib in this form.

One idea that I had was to break the given string into words (splitting
at spaces or punctuation) and store the rotations of individual words
instead of the whole string.  (Actually, maybe you only need suffixes
not rotations, ie for 'abcd' store 'abcd', 'bcd', 'cd', 'd'.)  Then
similarly break the LIKE pattern apart at words to create word-fragment
search keys.  In this scheme the operator would always(?) require
rechecking since any part of the pattern involving punctuation wouldn't
be checkable by the index.  The advantage is that the index bloat factor
is governed by the average word length not the average whole-string
length.

There are probably other approaches that would help, too.

regards, tom lane

-- 
Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-patches


Re: [PATCHES] Patch to change psql default banner v6

2008-05-16 Thread Bruce Momjian
Bruce Momjian wrote:
> OK, here is the mega-print:
> 
>   $ psql test
>   psql (8.4devel, server 8.4devel)
>   WARNING: psql version 8.4, server version 8.4.
>Some psql features might not work.
>   WARNING: Console code page (44) differs from Windows code page (55)
>8-bit characters might not work correctly. See psql reference
>page "Notes for Windows users" for details.
>   SSL connection (cipher: 55, bits: 512)
>   Type "help" for help.
>   
>   test=>
> 

Updated patch applied, docs adjusted for new psql startup banner.

-- 
  Bruce Momjian  <[EMAIL PROTECTED]>http://momjian.us
  EnterpriseDB http://enterprisedb.com

  + If your life is a hard drive, Christ can be your backup. +
Index: doc/src/sgml/start.sgml
===
RCS file: /cvsroot/pgsql/doc/src/sgml/start.sgml,v
retrieving revision 1.46
diff -c -c -r1.46 start.sgml
*** doc/src/sgml/start.sgml	23 Jan 2008 02:04:47 -	1.46
--- doc/src/sgml/start.sgml	16 May 2008 17:06:38 -
***
*** 329,341 
  In psql, you will be greeted with the following
  message:
  
! Welcome to psql &version;, the PostgreSQL interactive terminal.
!  
! Type:  \copyright for distribution terms
!\h for help with SQL commands
!\? for help with psql commands
!\g or terminate with semicolon to execute query
!\q to quit
   
  mydb=>
  
--- 329,336 
  In psql, you will be greeted with the following
  message:
  
! psql (&version;)
! Type "help" for help.
   
  mydb=>
  
Index: doc/src/sgml/ref/psql-ref.sgml
===
RCS file: /cvsroot/pgsql/doc/src/sgml/ref/psql-ref.sgml,v
retrieving revision 1.205
diff -c -c -r1.205 psql-ref.sgml
*** doc/src/sgml/ref/psql-ref.sgml	16 May 2008 16:59:05 -	1.205
--- doc/src/sgml/ref/psql-ref.sgml	16 May 2008 17:06:38 -
***
*** 571,583 
  the string =>. For example:
  
  $ psql testdb
! Welcome to psql &version;, the PostgreSQL interactive terminal.
  
! Type:  \copyright for distribution terms
!\h for help with SQL commands
!\? for help with psql commands
!\g or terminate with semicolon to execute query
!\q to quit
  
  testdb=>
  
--- 571,580 
  the string =>. For example:
  
  $ psql testdb
! psql (&version;)
! Type "help" for help.
  
! test=>
  
  testdb=>
  
Index: src/bin/psql/help.c
===
RCS file: /cvsroot/pgsql/src/bin/psql/help.c,v
retrieving revision 1.127
diff -c -c -r1.127 help.c
*** src/bin/psql/help.c	14 May 2008 15:30:22 -	1.127
--- src/bin/psql/help.c	16 May 2008 17:06:39 -
***
*** 170,182 
  	 */
  	fprintf(output, _("General\n"));
  	fprintf(output, _("  \\copyright show PostgreSQL usage and distribution terms\n"));
  	fprintf(output, _("  \\h [NAME]  help on syntax of SQL commands, * for all commands\n"));
  	fprintf(output, _("  \\q quit psql\n"));
  	fprintf(output, "\n");
  
  	fprintf(output, _("Query Buffer\n"));
  	fprintf(output, _("  \\e [FILE]  edit the query buffer (or file) with external editor\n"));
- 	fprintf(output, _("  \\g [FILE]  send query buffer to server (and results to file or |pipe)\n"));
  	fprintf(output, _("  \\p show the contents of the query buffer\n"));
  	fprintf(output, _("  \\r reset (clear) the query buffer\n"));
  #ifdef USE_READLINE
--- 170,182 
  	 */
  	fprintf(output, _("General\n"));
  	fprintf(output, _("  \\copyright show PostgreSQL usage and distribution terms\n"));
+ 	fprintf(output, _("  \\g [FILE] or ; execute query (and send results to file or |pipe)\n"));
  	fprintf(output, _("  \\h [NAME]  help on syntax of SQL commands, * for all commands\n"));
  	fprintf(output, _("  \\q quit psql\n"));
  	fprintf(output, "\n");
  
  	fprintf(output, _("Query Buffer\n"));
  	fprintf(output, _("  \\e [FILE]  edit the query buffer (or file) with external editor\n"));
  	fprintf(output, _("  \\p show the contents of the query buffer\n"));
  	fprintf(output, _("  \\r reset (clear) the query buffer\n"));
  #ifdef USE_READLINE
Index: src/bin/psql/mainloop.c
===
RCS file: /cvsroot/pgsql/src/bin/psql/mainloop.c,v
retrieving revision 1.90
diff -c -c -r1.90 mainloop.c
*** src/bin/psql/mainloop.c	5 Apr 2008 03:40:15 -	1.90
--- src/bin/psql/mainloop.c	16 May 2008 17:06:39 -
***
*** 177,186 
  			(line[4] == '\0' || line[4] == ';' || isspace((unsigned char) line[4])))
  		{
  			free(line);
! 			puts(_("You are using psql, the command-line interface to PostgreSQL."));
! 			puts(_("Enter SQL commands, or type \\? for a list of backslash options."));
! 			puts(_("Use \\h for 

Re: [PATCHES] Fix for psql pager computations

2008-05-16 Thread Bruce Momjian
Bruce Momjian wrote:
> The attached patch causes psql to use the pager if newlines or
> 'format=wrapped' has caused a single row to span more than one line and
> the output is then too long for the screen.  It also uses the pager if
> psql thinks the data will wrap off the edge of the screen.  
> 
> The display width as defined by \pset columns, $COLUMNS, or the ioctl
> (as previously discussed) We don't have similar settings for the number
> of display rows.  I assume that is OK. 
> 
> I believe this makes the pager more reliable, and in fact I have removed
> the psql manual mention that pager computations are somewhat imperfect.

Update patch applied.

-- 
  Bruce Momjian  <[EMAIL PROTECTED]>http://momjian.us
  EnterpriseDB http://enterprisedb.com

  + If your life is a hard drive, Christ can be your backup. +
Index: doc/src/sgml/ref/psql-ref.sgml
===
RCS file: /cvsroot/pgsql/doc/src/sgml/ref/psql-ref.sgml,v
retrieving revision 1.204
diff -c -c -r1.204 psql-ref.sgml
*** doc/src/sgml/ref/psql-ref.sgml	14 May 2008 04:07:01 -	1.204
--- doc/src/sgml/ref/psql-ref.sgml	16 May 2008 16:49:19 -
***
*** 1555,1561 
columns


!   Controls the target width for the wrapped format.
Zero (the default) causes the wrapped format to
affect only screen output.

--- 1555,1562 
columns


!   Controls the target width for the wrapped format,
!   and width for determining if wide output requires the pager.
Zero (the default) causes the wrapped format to
affect only screen output.

***
*** 1717,1726 
When the pager is off, the pager is not used. When the pager
is on, the pager is used only when appropriate, i.e. the
output is to a terminal and will not fit on the screen.
!   (psql does not do a perfect job of estimating
!   when to use the pager.) \pset pager turns the
!   pager on and off. Pager can also be set to always,
!   which causes the pager to be always used.



--- 1718,1726 
When the pager is off, the pager is not used. When the pager
is on, the pager is used only when appropriate, i.e. the
output is to a terminal and will not fit on the screen.
!   \pset pager turns the pager on and off. Pager can
!   also be set to always, which causes the pager to be
!   always used.



***
*** 2734,2741 
  
  
   
!   Used for the wrapped output format if 
!   \pset columns is zero.
   
  
 
--- 2734,2742 
  
  
   
!   If \pset columns is zero, controls the
!   width for the wrapped format and width for determining
!   if wide output requires the pager.
   
  
 
Index: src/bin/psql/print.c
===
RCS file: /cvsroot/pgsql/src/bin/psql/print.c,v
retrieving revision 1.101
diff -c -c -r1.101 print.c
*** src/bin/psql/print.c	13 May 2008 00:14:11 -	1.101
--- src/bin/psql/print.c	16 May 2008 16:49:20 -
***
*** 45,50 
--- 45,52 
  
  /* Local functions */
  static int strlen_max_width(unsigned char *str, int *target_width, int encoding);
+ static void IsPagerNeeded(const printTableContent *cont, const int extra_lines,
+ 		  FILE **fout, bool *is_pager);
  
  
  static void *
***
*** 394,400 
   *	Print pretty boxes around cells.
   */
  static void
! print_aligned_text(const printTableContent *cont, bool is_pager, FILE *fout)
  {
  	bool		opt_tuples_only = cont->opt->tuples_only;
  	bool		opt_numeric_locale = cont->opt->numericLocale;
--- 396,402 
   *	Print pretty boxes around cells.
   */
  static void
! print_aligned_text(const printTableContent *cont, FILE *fout)
  {
  	bool		opt_tuples_only = cont->opt->tuples_only;
  	bool		opt_numeric_locale = cont->opt->numericLocale;
***
*** 416,421 
--- 418,425 
  	unsigned char **format_buf;
  	unsigned int width_total;
  	unsigned int total_header_width;
+ 	unsigned int extra_row_output_lines = 0;
+ 	unsigned int extra_output_lines = 0;
  
  	const char * const *ptr;
  
***
*** 424,429 
--- 428,434 
  	bool	   *header_done;	/* Have all header lines been output? */
  	int		   *bytes_output;	/* Bytes output for column value */
  	int			output_columns = 0;	/* Width of interactive console */
+ 	bool		is_pager = false;
  
  	if (cancel_pressed)
  		return;
***
*** 476,484 
--- 481,494 
  			max_nl_lines[i] = nl_lines;
  		if (bytes_required > max_bytes[i])
  			max_bytes[i] = bytes_required;
+ 		if (nl_lines > extra_row_output_lines)
+ 			extr

Re: [PATCHES] libpq object hooks

2008-05-16 Thread Andrew Dunstan



Tom Lane wrote:

Andrew Dunstan <[EMAIL PROTECTED]> writes:
  
All of this is getting quite a long way from what was in the commitfest 
queue. Do we still want to try to get this in this cycle, or should it 
be marked returned to author for more work?



So far I think it still falls within the category of allowing the author
to revise his work.  I don't want to hold commitfest open waiting on
revisions of this patch, but as long as there's still other stuff being
worked through I don't see why they can't keep trying.

Just for the record, I would really like to close this fest before
PGCon starts.  We still have a couple more days to get it done.


  


Apart from this one only two have not been claimed:

. Map Forks
. TRUNCATE TABLE with IDENTITY

cheers

andrew



--
Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-patches


Re: [PATCHES] libpq object hooks

2008-05-16 Thread Merlin Moncure
On Fri, May 16, 2008 at 11:21 AM, Andrew Dunstan <[EMAIL PROTECTED]> wrote:
> All of this is getting quite a long way from what was in the commitfest
> queue. Do we still want to try to get this in this cycle, or should it be
> marked returned to author for more work?

That's your call...we can have a patch up within 24 hours or so.  My
suggestion would be to let us put up an updated version in the May
queue, and if there are any further serious objections we can push it
to the next queue giving some more time for things to percolate.

merlin

-- 
Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-patches


Re: [PATCHES] libpq object hooks

2008-05-16 Thread Alvaro Herrera
Andrew Dunstan escribió:

> All of this is getting quite a long way from what was in the commitfest  
> queue. Do we still want to try to get this in this cycle, or should it  
> be marked returned to author for more work?

There are still patches open for which no discussion has even started,
so I think this is OK for this commitfest.

(Besides, it seems we're almost there on this patch.)

-- 
Alvaro Herrerahttp://www.CommandPrompt.com/
The PostgreSQL Company - Command Prompt, Inc.

-- 
Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-patches


Re: [PATCHES] libpq object hooks

2008-05-16 Thread Tom Lane
Andrew Dunstan <[EMAIL PROTECTED]> writes:
> All of this is getting quite a long way from what was in the commitfest 
> queue. Do we still want to try to get this in this cycle, or should it 
> be marked returned to author for more work?

So far I think it still falls within the category of allowing the author
to revise his work.  I don't want to hold commitfest open waiting on
revisions of this patch, but as long as there's still other stuff being
worked through I don't see why they can't keep trying.

Just for the record, I would really like to close this fest before
PGCon starts.  We still have a couple more days to get it done.

regards, tom lane

-- 
Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-patches


Re: [PATCHES] libpq object hooks

2008-05-16 Thread Tom Lane
"Merlin Moncure" <[EMAIL PROTECTED]> writes:
> Switch, plus struct (basically a union) will do the trick nicely.  Can
> it be a formal union, or is it better as a void*?

I don't think a union buys much notational convenience or safety here,
although admittedly it's a close question.  In one case you're trusting
to cast the pointer to the appropriate type, in the other you're
trusting to use the right union member.  One advantage of separate
structs is that there's no reason not to make the struct type names
long enough to be clear, whereas there's a very strong notational
temptation to make union member names short, because you'll be typing
them a lot.

regards, tom lane

-- 
Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-patches


Re: [PATCHES] libpq object hooks

2008-05-16 Thread Andrew Dunstan



Merlin Moncure wrote:
  

Also, even if varargs are safe they'd be notationally unpleasant
in the extreme.  varargs are just a PITA to work with --- you'd have
to do all the decoding in the first-level hook routine, even for
items you weren't going to use.  With something like the above
all you need is a switch() and some pointer casts.



Switch, plus struct (basically a union) will do the trick nicely.  Can
it be a formal union, or is it better as a void*?

The main issue was how what we called the 'hook data' was passed back
and forth.  We'll get a patch up.


  


All of this is getting quite a long way from what was in the commitfest 
queue. Do we still want to try to get this in this cycle, or should it 
be marked returned to author for more work?


cheers

andrew

--
Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-patches


Re: [PATCHES] libpq object hooks

2008-05-16 Thread Merlin Moncure
On Fri, May 16, 2008 at 10:59 AM, Tom Lane <[EMAIL PROTECTED]> wrote:
> "Merlin Moncure" <[EMAIL PROTECTED]> writes:
>>> typedef void *(*PGobjectEventProc)(PGobjectEventId evtId, ...);
>>> int PQregisterObjectEventProc(PGconn*, PGobjectEventProc);
>>> void *PQeventData(PGconn *, PGobjectEventProc);
>>> void *PQresultEventData(PGresult *, PGobjectEventProc);
>
>> This provides what we need...a key to lookup the hook data without
>> using a string. Also, it reduces the number of exports (it's a little
>> easier for us, while not essential, to not have to register each
>> callback individually).  Also, this AFAICT does not have any ABI
>> issues (no struct), and adds less exports which is nice.  We don't
>> have to 'look up' the data inside the callbacks..it's properly passed
>> through as an argument.  While vararg callbacks may be considered
>> unsafe in some scenarios, I think it's a good fit here.
>
> I don't think varargs callbacks are a good idea at all.  Please adjust
> this to something that doesn't do that.  Also, haven't you forgotten
> the passthrough void *?

We didn't...one of the functions (the init) doesn't need it so we
didn't expose it to the fixed arguments...we would just va_arg it off
in the other cases.  Regardless, your comments below make that moot:

> If you really want only one callback function, perhaps what would work
> is
>
> typedef void (*PGeventProc) (PGeventId eventId, const void *eventInfo,
> void *passthrough);
>
> int PQregisterEventProc(PGconn *conn, PGeventProc proc, void *passthrough);
>
> where eventInfo will point to one of several exported struct types
> depending on the value of eventId.  With this approach, you can add
> more fields to the end of any one such struct type without creating
> ABI issues.  I have little confidence in being able to make similar
> changes in a varargs environment.

this is fine.

> Also, even if varargs are safe they'd be notationally unpleasant
> in the extreme.  varargs are just a PITA to work with --- you'd have
> to do all the decoding in the first-level hook routine, even for
> items you weren't going to use.  With something like the above
> all you need is a switch() and some pointer casts.

Switch, plus struct (basically a union) will do the trick nicely.  Can
it be a formal union, or is it better as a void*?

The main issue was how what we called the 'hook data' was passed back
and forth.  We'll get a patch up.

merlin

-- 
Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-patches


Re: [PATCHES] libpq thread-locking

2008-05-16 Thread Bruce Momjian
Magnus Hagander wrote:
> Bruce Momjian wrote:
> > Bruce Momjian wrote:
> > > Magnus Hagander wrote:
> > > > Attached patch adds some error checking to the thread locking
> > > > stuff in libpq. Previously, if thread locking failed for some
> > > > reason, we would just fall through and do things without locking.
> > > > This patch makes us abort() instead. It's not the greatest thing
> > > > probably, but our API doesn't let us pass back return values...
> > > 
> > > I have looked over the patch and it seems fine, though I am
> > > concerned about the abort() case with no output.  I realize stderr
> > > might be going nowhere, but in fe-print.c we do an fprintf(stderr)
> > > for memory failures so for consistency I think we should do the
> > > same here.  If there is concern about code bloat, I suggest a macro
> > > at the top of the file for thread failure exits:
> > > 
> > >   #define THEAD_FAILURE(str) \
> > >   do { \
> > >   fprintf(stderr, libpq_gettext("Thread failure:
> > > %s\n")); \ exit(1); \
> > >   } while(0)
> > 
> > Oh, this is Tom saying he doesn't like stderr and the added code lines
> > for failure:
> > 
> > http://archives.postgresql.org/pgsql-patches/2008-04/msg00254.php
> > 
> > I think the macro and consistency suggest doing as I outlined.
> 
> Does this one look like what you're suggesting?

Yep.

-- 
  Bruce Momjian  <[EMAIL PROTECTED]>http://momjian.us
  EnterpriseDB http://enterprisedb.com

  + If your life is a hard drive, Christ can be your backup. +

-- 
Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-patches


Re: [PATCHES] libpq object hooks

2008-05-16 Thread Tom Lane
"Merlin Moncure" <[EMAIL PROTECTED]> writes:
>> typedef void *(*PGobjectEventProc)(PGobjectEventId evtId, ...);
>> int PQregisterObjectEventProc(PGconn*, PGobjectEventProc);
>> void *PQeventData(PGconn *, PGobjectEventProc);
>> void *PQresultEventData(PGresult *, PGobjectEventProc);

> This provides what we need...a key to lookup the hook data without
> using a string. Also, it reduces the number of exports (it's a little
> easier for us, while not essential, to not have to register each
> callback individually).  Also, this AFAICT does not have any ABI
> issues (no struct), and adds less exports which is nice.  We don't
> have to 'look up' the data inside the callbacks..it's properly passed
> through as an argument.  While vararg callbacks may be considered
> unsafe in some scenarios, I think it's a good fit here.

I don't think varargs callbacks are a good idea at all.  Please adjust
this to something that doesn't do that.  Also, haven't you forgotten
the passthrough void *?

If you really want only one callback function, perhaps what would work
is

typedef void (*PGeventProc) (PGeventId eventId, const void *eventInfo,
 void *passthrough);

int PQregisterEventProc(PGconn *conn, PGeventProc proc, void *passthrough);

where eventInfo will point to one of several exported struct types
depending on the value of eventId.  With this approach, you can add
more fields to the end of any one such struct type without creating
ABI issues.  I have little confidence in being able to make similar
changes in a varargs environment.

Also, even if varargs are safe they'd be notationally unpleasant
in the extreme.  varargs are just a PITA to work with --- you'd have
to do all the decoding in the first-level hook routine, even for
items you weren't going to use.  With something like the above
all you need is a switch() and some pointer casts.

regards, tom lane

-- 
Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-patches


Re: [PATCHES] libpq thread-locking

2008-05-16 Thread Magnus Hagander
Andrew Chernow wrote:
> ! int
>pthread_mutex_init(pthread_mutex_t *mp, void *attr)
>{
>   *mp = CreateMutex(0, 0, 0);
> + if (*mp == NULL)
> + return 1;
> + return 0;
>}
> 
> Maybe it would be better to emulate what pthreads does.  Instead of 
> returing 1 to indicate an error, return an errno.  In the above case, 
> ENOMEM seems like a good fit.
> 
> Also, maybe you should check the passed in mutex pointer.  If its
> NULL, you could return EINVAL.

Given that we only call this stuff internally, I don't think it's a big
issue. But either way - that part of the code will be replaced with the
critical_section code later anyway - I just want to get the "generic"
changes through first.

//Magnus

-- 
Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-patches


Re: [PATCHES] libpq thread-locking

2008-05-16 Thread Andrew Chernow

! int
  pthread_mutex_init(pthread_mutex_t *mp, void *attr)
  {
*mp = CreateMutex(0, 0, 0);
+   if (*mp == NULL)
+   return 1;
+   return 0;
  }

Maybe it would be better to emulate what pthreads does.  Instead of 
returing 1 to indicate an error, return an errno.  In the above case, 
ENOMEM seems like a good fit.


Also, maybe you should check the passed in mutex pointer.  If its NULL, 
you could return EINVAL.


--
Andrew Chernow
eSilo, LLC
every bit counts
http://www.esilo.com/

--
Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-patches


Re: [PATCHES] libpq thread-locking

2008-05-16 Thread Magnus Hagander
Bruce Momjian wrote:
> Bruce Momjian wrote:
> > Magnus Hagander wrote:
> > > Attached patch adds some error checking to the thread locking
> > > stuff in libpq. Previously, if thread locking failed for some
> > > reason, we would just fall through and do things without locking.
> > > This patch makes us abort() instead. It's not the greatest thing
> > > probably, but our API doesn't let us pass back return values...
> > 
> > I have looked over the patch and it seems fine, though I am
> > concerned about the abort() case with no output.  I realize stderr
> > might be going nowhere, but in fe-print.c we do an fprintf(stderr)
> > for memory failures so for consistency I think we should do the
> > same here.  If there is concern about code bloat, I suggest a macro
> > at the top of the file for thread failure exits:
> > 
> > #define THEAD_FAILURE(str) \
> > do { \
> > fprintf(stderr, libpq_gettext("Thread failure:
> > %s\n")); \ exit(1); \
> > } while(0)
> 
> Oh, this is Tom saying he doesn't like stderr and the added code lines
> for failure:
> 
>   http://archives.postgresql.org/pgsql-patches/2008-04/msg00254.php
> 
> I think the macro and consistency suggest doing as I outlined.

Does this one look like what you're suggesting?

//Magnus
Index: fe-connect.c
===
RCS file: /cvsroot/pgsql/src/interfaces/libpq/fe-connect.c,v
retrieving revision 1.357
diff -c -r1.357 fe-connect.c
*** fe-connect.c	31 Mar 2008 02:43:14 -	1.357
--- fe-connect.c	16 May 2008 14:03:10 -
***
*** 3835,3848 
  		while (InterlockedExchange(&mutex_initlock, 1) == 1)
  			 /* loop, another thread own the lock */ ;
  		if (singlethread_lock == NULL)
! 			pthread_mutex_init(&singlethread_lock, NULL);
  		InterlockedExchange(&mutex_initlock, 0);
  	}
  #endif
  	if (acquire)
! 		pthread_mutex_lock(&singlethread_lock);
  	else
! 		pthread_mutex_unlock(&singlethread_lock);
  #endif
  }
  
--- 3835,3857 
  		while (InterlockedExchange(&mutex_initlock, 1) == 1)
  			 /* loop, another thread own the lock */ ;
  		if (singlethread_lock == NULL)
! 		{
! 			if (pthread_mutex_init(&singlethread_lock, NULL))
! PGTHREAD_ERROR("failed to initialize mutex");
! 		}
  		InterlockedExchange(&mutex_initlock, 0);
  	}
  #endif
  	if (acquire)
! 	{
! 		if (pthread_mutex_lock(&singlethread_lock))
! 			PGTHREAD_ERROR("failed to lock mutex");
! 	}
  	else
! 	{
! 		if (pthread_mutex_unlock(&singlethread_lock))
! 			PGTHREAD_ERROR("failed to unlock mutex");
! 	}
  #endif
  }
  
Index: fe-secure.c
===
RCS file: /cvsroot/pgsql/src/interfaces/libpq/fe-secure.c,v
retrieving revision 1.104
diff -c -r1.104 fe-secure.c
*** fe-secure.c	31 Mar 2008 02:43:14 -	1.104
--- fe-secure.c	16 May 2008 14:03:11 -
***
*** 796,807 
  pq_lockingcallback(int mode, int n, const char *file, int line)
  {
  	if (mode & CRYPTO_LOCK)
! 		pthread_mutex_lock(&pq_lockarray[n]);
  	else
! 		pthread_mutex_unlock(&pq_lockarray[n]);
  }
  #endif   /* ENABLE_THREAD_SAFETY */
  
  static int
  init_ssl_system(PGconn *conn)
  {
--- 796,816 
  pq_lockingcallback(int mode, int n, const char *file, int line)
  {
  	if (mode & CRYPTO_LOCK)
! 	{
! 		if (pthread_mutex_lock(&pq_lockarray[n]))
! 			PGTHREAD_ERROR("failed to lock mutex");
! 	}
  	else
! 	{
! 		if (pthread_mutex_unlock(&pq_lockarray[n]))
! 			PGTHREAD_ERROR("failed to unlock mutex");
! 	}
  }
  #endif   /* ENABLE_THREAD_SAFETY */
  
+ /*
+  * Also see similar code in fe-connect.c, default_threadlock()
+  */
  static int
  init_ssl_system(PGconn *conn)
  {
***
*** 817,827 
  		while (InterlockedExchange(&mutex_initlock, 1) == 1)
  			 /* loop, another thread own the lock */ ;
  		if (init_mutex == NULL)
! 			pthread_mutex_init(&init_mutex, NULL);
  		InterlockedExchange(&mutex_initlock, 0);
  	}
  #endif
! 	pthread_mutex_lock(&init_mutex);
  
  	if (pq_initssllib && pq_lockarray == NULL)
  	{
--- 826,840 
  		while (InterlockedExchange(&mutex_initlock, 1) == 1)
  			 /* loop, another thread own the lock */ ;
  		if (init_mutex == NULL)
! 		{
! 			if (pthread_mutex_init(&init_mutex, NULL))
! return -1;
! 		}
  		InterlockedExchange(&mutex_initlock, 0);
  	}
  #endif
! 	if (pthread_mutex_lock(&init_mutex))
! 		return -1;
  
  	if (pq_initssllib && pq_lockarray == NULL)
  	{
***
*** 836,842 
  			return -1;
  		}
  		for (i = 0; i < CRYPTO_num_locks(); i++)
! 			pthread_mutex_init(&pq_lockarray[i], NULL);
  
  		CRYPTO_set_locking_callback(pq_lockingcallback);
  	}
--- 849,858 
  			return -1;
  		}
  		for (i = 0; i < CRYPTO_num_locks(); i++)
! 		{
! 			if (pthread_mutex_init(&pq_lockarray[i], NULL))
! return -1;
! 		}
  
  		CRYPTO_set_locking_callback(pq_lockingcallback);
  	}
Index: libpq-int.h
===
RCS file: /cvsroot/pgsql/src

Re: [PATCHES] libpq object hooks

2008-05-16 Thread Merlin Moncure
On Thu, May 15, 2008 at 8:38 PM, Andrew Chernow <[EMAIL PROTECTED]> wrote:
> We need to add members to a conn and result, that's pretty much it.  To do
> this, an api user can register callbacks to receive notifications about
> created/destroyed states of objects.  PQhookData is just like PQerrorMessage
> in that both are public accessor functions to private object data.  The
> difference is that there can be more than one hookData "dynamic struct
> member" on a conn/result at a time, unlike errorMessage;  thus the need for
> an additional "lookup" value when getting hook data (what was hookName).

> typedef void *(*PGobjectEventProc)(PGobjectEventId evtId, ...);
> int PQregisterObjectEventProc(PGconn*, PGobjectEventProc);
> void *PQeventData(PGconn *, PGobjectEventProc);
> void *PQresultEventData(PGresult *, PGobjectEventProc);

This provides what we need...a key to lookup the hook data without
using a string. Also, it reduces the number of exports (it's a little
easier for us, while not essential, to not have to register each
callback individually).  Also, this AFAICT does not have any ABI
issues (no struct), and adds less exports which is nice.  We don't
have to 'look up' the data inside the callbacks..it's properly passed
through as an argument.  While vararg callbacks may be considered
unsafe in some scenarios, I think it's a good fit here.

The most important part though is that it fits what we think is needed
to maintain the data we associate with the libpq with the proper
lifetime.  I'm not sure that everyone was on the same page in terms of
how this works...we may not have explained ourselves properly.  In our
defense, the interaction between libpq and a wrapping library like
libpqtypes is a bit involved and the 'best possible' way to link
things up did not necessarily suggest itself the first time out.

We would like to wrap this up into some form the community would
accept.  The event proc style of doing this is better than our initial
approach...faster and cleaner.  In fact, we are pleased with all the
changes that have come about due to community suggestions...there are
many positive results.

merlin

-- 
Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-patches


Re: [PATCHES] Patch to change psql default banner v6

2008-05-16 Thread David Fetter
On Thu, May 15, 2008 at 03:21:37PM -0400, Bruce Momjian wrote:
> Alvaro Herrera wrote:
> > 
> > I'm OK with thisG but please move the printSSLInfo() call just before
> > echoing the help line.
> 
> Oh, good catch, moved.  I also moved the Win32 code page message up too.
> Patch attached.
> 
> I hacked up an example that shows both SSL and Win32 code page messages:

I believe there's a bug in this patch, namely that the warnings when
there's a server-client mismatch only appear at startup time.  This is
a pretty clear POLA violation, IMHO.

On my laptop, I have two pg instances running: 8.3.0 on port 5432, CVS
TIP on 2225.

Here's what I get if I invoke psql from the command line:

$ psql -p 5432 postgres
Welcome to psql 8.4devel (server 8.3.0), the PostgreSQL interactive terminal.

Type:  \copyright for distribution terms
   \h for help with SQL commands
   \? for help with psql commands
   \g or terminate with semicolon to execute query
   \q to quit

WARNING:  You are connected to a server with major version 8.3,
but your psql client is major version 8.4.  Some backslash commands,
such as \d, might not work properly.

Here's what I get if I use \c, having connected to CVS TIP first:

[EMAIL PROTECTED] \c - - - 5432
You are now connected to database "postgres" at port "5432".

I think that the warning should be consistently there on connect
instead of just at program start.

Not coincidentally, moving all the checks into one spot, i.e. making
startup.c and command.c call and test the same things to connect to a
database, advances my Evil Plan™ to make more interesting things
happen when switching versions :)

Cheers,
David.
-- 
David Fetter <[EMAIL PROTECTED]> http://fetter.org/
Phone: +1 415 235 3778  AIM: dfetter666  Yahoo!: dfetter
Skype: davidfetter  XMPP: [EMAIL PROTECTED]

Remember to vote!
Consider donating to Postgres: http://www.postgresql.org/about/donate

-- 
Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-patches


Re: [PATCHES] Patch to change psql default banner v6

2008-05-16 Thread Tom Lane
David Fetter <[EMAIL PROTECTED]> writes:
> I believe there's a bug in this patch, namely that the warnings when
> there's a server-client mismatch only appear at startup time.

Please do not blame this patch for a problem that has been there all
along.

I don't say that the point doesn't need investigation, but blaming
the patch-at-hand for the issue is just misleading.

regards, tom lane

-- 
Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-patches


Re: [PATCHES] Patch to change psql default banner v6

2008-05-16 Thread David Fetter
On Fri, May 16, 2008 at 01:22:55AM -0400, Tom Lane wrote:
> David Fetter <[EMAIL PROTECTED]> writes:
> > I believe there's a bug in this patch, namely that the warnings when
> > there's a server-client mismatch only appear at startup time.
> 
> Please do not blame this patch for a problem that has been there all
> along.
> 
> I don't say that the point doesn't need investigation, but blaming
> the patch-at-hand for the issue is just misleading.

The patch at hand, as you point out, emphasizes a problem that's been
there all along, namely that \c doesn't do the same things that
command line connection does.

I'm volunteering to make them use the same methods :)

Cheers,
David.
-- 
David Fetter <[EMAIL PROTECTED]> http://fetter.org/
Phone: +1 415 235 3778  AIM: dfetter666  Yahoo!: dfetter
Skype: davidfetter  XMPP: [EMAIL PROTECTED]

Remember to vote!
Consider donating to Postgres: http://www.postgresql.org/about/donate

-- 
Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-patches


Re: [PATCHES] Partial match in GIN (next vesrion)

2008-05-16 Thread Teodor Sigaev

There seems to be something broken here: it's acting like prefix search
is on all the time, eg


I'm in sackcloth and ashes...

Fixed and extended regression tests.

http://www.sigaev.ru/misc/tsearch_prefix-0.9.gz


--
Teodor Sigaev   E-mail: [EMAIL PROTECTED]
   WWW: 
http://www.sigaev.ru/


--
Sent via pgsql-patches mailing list (pgsql-patches@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-patches