[PATCHES] tsearch2 makefile fixes
Hi, I needed to apply this patch in order for tsearch2 to build here. This is a VPATH build, so maybe it's a reason why it's not a common test scenario. The changes itself also seem sound to me, at least as far as I understand our makefile structure. Please consider applying this to 8.1 (or just let me know and I'll do it for you). -- Alvaro Herrerahttp://www.advogato.org/person/alvherre Granting software the freedom to evolve guarantees only different results, not better ones. (Zygo Blaxell) ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
[PATCHES] tsearch2 makefile fixes
[Resend, this time with the patch attached] Hi, I needed to apply this patch in order for tsearch2 to build here. This is a VPATH build, so maybe it's a reason why it's not a common test scenario. The changes itself also seem sound to me, at least as far as I understand our makefile structure. Please consider applying this to 8.1 (or just let me know and I'll do it for you). -- Alvaro Herrerahttp://www.advogato.org/person/alvherre Granting software the freedom to evolve guarantees only different results, not better ones. (Zygo Blaxell) Index: ispell/Makefile === RCS file: /home/alvherre/Code/cvs/pgsql/contrib/tsearch2/ispell/Makefile,v retrieving revision 1.8 diff -c -r1.8 Makefile *** ispell/Makefile 27 Sep 2005 17:13:11 - 1.8 --- ispell/Makefile 2 Nov 2005 09:56:58 - *** *** 8,14 PGXS := $(shell pg_config --pgxs) include $(PGXS) else ! subdir = contrib/tsearch2 top_builddir = ../../.. include $(top_builddir)/src/Makefile.global include $(top_srcdir)/contrib/contrib-global.mk --- 8,14 PGXS := $(shell pg_config --pgxs) include $(PGXS) else ! subdir = contrib/tsearch2/ispell top_builddir = ../../.. include $(top_builddir)/src/Makefile.global include $(top_srcdir)/contrib/contrib-global.mk Index: snowball/Makefile === RCS file: /home/alvherre/Code/cvs/pgsql/contrib/tsearch2/snowball/Makefile,v retrieving revision 1.7 diff -c -r1.7 Makefile *** snowball/Makefile 27 Sep 2005 17:13:12 - 1.7 --- snowball/Makefile 2 Nov 2005 09:53:41 - *** *** 9,15 PGXS := $(shell pg_config --pgxs) include $(PGXS) else ! subdir = contrib/tsearch2 top_builddir = ../../.. include $(top_builddir)/src/Makefile.global include $(top_srcdir)/contrib/contrib-global.mk --- 9,15 PGXS := $(shell pg_config --pgxs) include $(PGXS) else ! subdir = contrib/tsearch2/snowball top_builddir = ../../.. include $(top_builddir)/src/Makefile.global include $(top_srcdir)/contrib/contrib-global.mk Index: wordparser/Makefile === RCS file: /home/alvherre/Code/cvs/pgsql/contrib/tsearch2/wordparser/Makefile,v retrieving revision 1.7 diff -c -r1.7 Makefile *** wordparser/Makefile 27 Sep 2005 17:13:12 - 1.7 --- wordparser/Makefile 2 Nov 2005 09:54:30 - *** *** 9,15 PGXS := $(shell pg_config --pgxs) include $(PGXS) else ! subdir = contrib/tsearch2 top_builddir = ../../.. include $(top_builddir)/src/Makefile.global include $(top_srcdir)/contrib/contrib-global.mk --- 9,15 PGXS := $(shell pg_config --pgxs) include $(PGXS) else ! subdir = contrib/tsearch2/wordparser top_builddir = ../../.. include $(top_builddir)/src/Makefile.global include $(top_srcdir)/contrib/contrib-global.mk ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [PATCHES] tsearch2 makefile fixes
Hi! I don't see a patch :) Alvaro Herrera wrote: Hi, I needed to apply this patch in order for tsearch2 to build here. This is a VPATH build, so maybe it's a reason why it's not a common test scenario. The changes itself also seem sound to me, at least as far as I understand our makefile structure. Please consider applying this to 8.1 (or just let me know and I'll do it for you). -- Teodor Sigaev E-mail: [EMAIL PROTECTED] WWW: http://www.sigaev.ru/ ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [PATCHES] tsearch2 makefile fixes
As I can see, this patch already apllyed at 18 Oct: % grep subdir */Makefile ispell/Makefile:subdir = contrib/tsearch2/ispell snowball/Makefile:subdir = contrib/tsearch2/snowball wordparser/Makefile:subdir = contrib/tsearch2/wordparser CVS HEAD Alvaro Herrera wrote: [Resend, this time with the patch attached] Hi, I needed to apply this patch in order for tsearch2 to build here. This is a VPATH build, so maybe it's a reason why it's not a common test scenario. The changes itself also seem sound to me, at least as far as I understand our makefile structure. Please consider applying this to 8.1 (or just let me know and I'll do it for you). Index: ispell/Makefile === RCS file: /home/alvherre/Code/cvs/pgsql/contrib/tsearch2/ispell/Makefile,v retrieving revision 1.8 diff -c -r1.8 Makefile *** ispell/Makefile 27 Sep 2005 17:13:11 - 1.8 --- ispell/Makefile 2 Nov 2005 09:56:58 - *** *** 8,14 PGXS := $(shell pg_config --pgxs) include $(PGXS) else ! subdir = contrib/tsearch2 top_builddir = ../../.. include $(top_builddir)/src/Makefile.global include $(top_srcdir)/contrib/contrib-global.mk --- 8,14 PGXS := $(shell pg_config --pgxs) include $(PGXS) else ! subdir = contrib/tsearch2/ispell top_builddir = ../../.. include $(top_builddir)/src/Makefile.global include $(top_srcdir)/contrib/contrib-global.mk Index: snowball/Makefile === RCS file: /home/alvherre/Code/cvs/pgsql/contrib/tsearch2/snowball/Makefile,v retrieving revision 1.7 diff -c -r1.7 Makefile *** snowball/Makefile 27 Sep 2005 17:13:12 - 1.7 --- snowball/Makefile 2 Nov 2005 09:53:41 - *** *** 9,15 PGXS := $(shell pg_config --pgxs) include $(PGXS) else ! subdir = contrib/tsearch2 top_builddir = ../../.. include $(top_builddir)/src/Makefile.global include $(top_srcdir)/contrib/contrib-global.mk --- 9,15 PGXS := $(shell pg_config --pgxs) include $(PGXS) else ! subdir = contrib/tsearch2/snowball top_builddir = ../../.. include $(top_builddir)/src/Makefile.global include $(top_srcdir)/contrib/contrib-global.mk Index: wordparser/Makefile === RCS file: /home/alvherre/Code/cvs/pgsql/contrib/tsearch2/wordparser/Makefile,v retrieving revision 1.7 diff -c -r1.7 Makefile *** wordparser/Makefile 27 Sep 2005 17:13:12 - 1.7 --- wordparser/Makefile 2 Nov 2005 09:54:30 - *** *** 9,15 PGXS := $(shell pg_config --pgxs) include $(PGXS) else ! subdir = contrib/tsearch2 top_builddir = ../../.. include $(top_builddir)/src/Makefile.global include $(top_srcdir)/contrib/contrib-global.mk --- 9,15 PGXS := $(shell pg_config --pgxs) include $(PGXS) else ! subdir = contrib/tsearch2/wordparser top_builddir = ../../.. include $(top_builddir)/src/Makefile.global include $(top_srcdir)/contrib/contrib-global.mk -- Teodor Sigaev E-mail: [EMAIL PROTECTED] WWW: http://www.sigaev.ru/ ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
[PATCHES] Limit usage of tcop/dest.h
Hi, I just found out that tcop/dest.h is included in executor/spi.h, and it contains many things that aren't needed for compiling SPI programs/ libraries. By way of example I compiled the whole of contrib with the attached patch and it works fine. Notice that the only thing I'm doing is taking the forward declaration of Portal into a separate file, portalforw.h -- that's the only definition that's really needed by SPI programs. (It also allows PL/php to compile without having to patch PHP nor PostgreSQL sources). Note that since tcop/dest.h now includes portalforw.h, anybody who currently needs the Portal definition is still getting it. The only thing I'm doing is un-export the rest of tcop/dest.h from executor/spi.h. So instead of changing the names of the CommandDest enum, I'm hiding it from external view. Note that executor/spi.h does not follow the convention that #includes should be alphabetically ordered. I did not change that in this patch in order to show that this change is really minimal. Does anybody object to committing this patch to current CVS HEAD? (Comments about a better position/name for the new file are welcome.) -- Alvaro Herrera http://www.amazon.com/gp/registry/DXLWNGRJD34J The only difference is that Saddam would kill you on private, where the Americans will kill you in public (Mohammad Saleh, 39, a building contractor) Index: executor/spi.h === RCS file: /home/alvherre/Code/cvs/pgsql/src/include/executor/spi.h,v retrieving revision 1.53 diff -c -r1.53 spi.h *** executor/spi.h 15 Oct 2005 02:49:44 - 1.53 --- executor/spi.h 2 Nov 2005 10:32:33 - *** *** 28,34 #include tcop/pquery.h #include tcop/tcopprot.h #include tcop/utility.h ! #include tcop/dest.h #include nodes/params.h #include utils/builtins.h #include utils/datum.h --- 28,34 #include tcop/pquery.h #include tcop/tcopprot.h #include tcop/utility.h ! #include executor/portalforw.h #include nodes/params.h #include utils/builtins.h #include utils/datum.h Index: tcop/dest.h === RCS file: /home/alvherre/Code/cvs/pgsql/src/include/tcop/dest.h,v retrieving revision 1.47 diff -c -r1.47 dest.h *** tcop/dest.h 15 Oct 2005 02:49:46 - 1.47 --- tcop/dest.h 2 Nov 2005 10:18:40 - *** *** 62,67 --- 62,68 #define DEST_H #include executor/tuptable.h + #include executor/portalforw.h /* buffer size to use for command completion tags */ *** *** 117,126 extern DestReceiver *None_Receiver; /* permanent receiver for None */ - /* This is a forward reference to utils/portal.h */ - - typedef struct PortalData *Portal; - /* The primary destination management functions */ extern void BeginCommand(const char *commandTag, CommandDest dest); --- 118,123 /*- * * portalforw.h * * $PostgreSQL$ * *- */ #ifndef PORTALFORW_H #define PORTALFORW_H /* This is a forward reference to utils/portal.h */ typedef struct PortalData *Portal; #endif /* PORTALFORW_H */ ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [PATCHES] tsearch2 makefile fixes
Teodor Sigaev wrote: As I can see, this patch already apllyed at 18 Oct: % grep subdir */Makefile ispell/Makefile:subdir = contrib/tsearch2/ispell snowball/Makefile:subdir = contrib/tsearch2/snowball wordparser/Makefile:subdir = contrib/tsearch2/wordparser Dohh, you are right. Sorry for the noise. -- Alvaro Herrerahttp://www.advogato.org/person/alvherre La verdad no siempre es bonita, pero el hambre de ella sí ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [PATCHES] Limit usage of tcop/dest.h
Alvaro Herrera said: Hi, (It also allows PL/php to compile without having to patch PHP nor PostgreSQL sources). That will make some people I know happy ;-) cheers andrew ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [PATCHES] Limit usage of tcop/dest.h
Andrew Dunstan wrote: Alvaro Herrera said: Hi, (It also allows PL/php to compile without having to patch PHP nor PostgreSQL sources). That will make some people I know happy ;-) Yeah -- the current PL/php build system is a crock (not sure what that is, but it sounds nice and it appears on Pg sources) :-) so I'm currently modifying it to work using PGXS. They won't need Pg sources at all really (and conversely, only PHP headers and the shared lib, not the whole sources). -- Alvaro Herrera http://www.amazon.com/gp/registry/CTMLCN8V17R4 La realidad se compone de muchos sueños, todos ellos diferentes, pero en cierto aspecto, parecidos... (Yo, hablando de sueños eróticos) ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [PATCHES] Limit usage of tcop/dest.h
Alvaro Herrera [EMAIL PROTECTED] writes: So instead of changing the names of the CommandDest enum, I'm hiding it from external view. I thought renaming them was a better idea, actually. A whole separate include file to have one forward typedef seems pretty silly. Nor am I convinced that you won't break some people's code by removing the rest of dest.h from spi.h. Finally, for anyone who *does* need to include dest.h, this doesn't address the underlying problem of risk of conflict of names. regards, tom lane ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [PATCHES] tsearch2 makefile fixes
Alvaro Herrera [EMAIL PROTECTED] writes: I needed to apply this patch in order for tsearch2 to build here. I fixed that some weeks ago ... how old a version are you looking at? regards, tom lane ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [PATCHES] Limit usage of tcop/dest.h
Tom Lane wrote: Alvaro Herrera [EMAIL PROTECTED] writes: So instead of changing the names of the CommandDest enum, I'm hiding it from external view. I thought renaming them was a better idea, actually. A whole separate include file to have one forward typedef seems pretty silly. Nor am I convinced that you won't break some people's code by removing the rest of dest.h from spi.h. Finally, for anyone who *does* need to include dest.h, this doesn't address the underlying problem of risk of conflict of names. Does the change make building PL/PHP easier? -- Bruce Momjian| http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [PATCHES] Limit usage of tcop/dest.h
Bruce Momjian wrote: Tom Lane wrote: Alvaro Herrera [EMAIL PROTECTED] writes: So instead of changing the names of the CommandDest enum, I'm hiding it from external view. I thought renaming them was a better idea, actually. A whole separate include file to have one forward typedef seems pretty silly. Nor am I convinced that you won't break some people's code by removing the rest of dest.h from spi.h. Finally, for anyone who *does* need to include dest.h, this doesn't address the underlying problem of risk of conflict of names. Does the change make building PL/PHP easier? Yes, the point of these changes is to make PL/php much easier. Either one will do -- renaming the enum elements is what I'm doing now, so we don't have to change include file. (Mind you, I still believe that that particular declaration does not belong in that file, but that's a different discussion.) (We will still need some hack in order to build PL/php against 8.0, but that's another problem.) -- Alvaro Herrera http://www.amazon.com/gp/registry/DXLWNGRJD34J Nunca se desea ardientemente lo que solo se desea por razón (F. Alexandre) ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [PATCHES] Limit usage of tcop/dest.h
Tom Lane wrote:
I thought renaming them was a better idea, actually.
Here is a patch for that. I will apply this to HEAD later today.
--
Alvaro Herrera Valdivia, Chile ICBM: S 39º 49' 17.7, W 73º 14' 26.8
The eagle never lost so much time, as
when he submitted to learn of the crow. (William Blake)
Index: src/backend/access/common/printtup.c
===
RCS file: /home/alvherre/Code/cvs/pgsql/src/backend/access/common/printtup.c,v
retrieving revision 1.92
diff -c -r1.92 printtup.c
*** src/backend/access/common/printtup.c15 Oct 2005 02:49:08 -
1.92
--- src/backend/access/common/printtup.c2 Nov 2005 12:12:58 -
***
*** 94,101
self-portal = portal;
! /* Send T message automatically if Remote, but not if RemoteExecute */
! self-sendDescrip = (dest == Remote);
self-attrinfo = NULL;
self-nattrs = 0;
--- 94,104
self-portal = portal;
! /*
!* Send T message automatically if DestRemote, but not if
!* DestRemoteExecute
!*/
! self-sendDescrip = (dest == DestRemote);
self-attrinfo = NULL;
self-nattrs = 0;
Index: src/backend/commands/async.c
===
RCS file: /home/alvherre/Code/cvs/pgsql/src/backend/commands/async.c,v
retrieving revision 1.126
diff -c -r1.126 async.c
*** src/backend/commands/async.c15 Oct 2005 02:49:15 - 1.126
--- src/backend/commands/async.c2 Nov 2005 12:14:08 -
***
*** 994,1000
static void
NotifyMyFrontEnd(char *relname, int32 listenerPID)
{
! if (whereToSendOutput == Remote)
{
StringInfoData buf;
--- 994,1000
static void
NotifyMyFrontEnd(char *relname, int32 listenerPID)
{
! if (whereToSendOutput == DestRemote)
{
StringInfoData buf;
Index: src/backend/commands/copy.c
===
RCS file: /home/alvherre/Code/cvs/pgsql/src/backend/commands/copy.c,v
retrieving revision 1.253
diff -c -r1.253 copy.c
*** src/backend/commands/copy.c 15 Oct 2005 02:49:15 - 1.253
--- src/backend/commands/copy.c 2 Nov 2005 12:14:42 -
***
*** 966,972
}
if (pipe)
{
! if (whereToSendOutput == Remote)
ReceiveCopyBegin(cstate);
else
cstate-copy_file = stdin;
--- 966,972
}
if (pipe)
{
! if (whereToSendOutput == DestRemote)
ReceiveCopyBegin(cstate);
else
cstate-copy_file = stdin;
***
*** 1017,1023
}
if (pipe)
{
! if (whereToSendOutput == Remote)
cstate-fe_copy = true;
else
cstate-copy_file = stdout;
--- 1017,1023
}
if (pipe)
{
! if (whereToSendOutput == DestRemote)
cstate-fe_copy = true;
else
cstate-copy_file = stdout;
Index: src/backend/commands/portalcmds.c
===
RCS file: /home/alvherre/Code/cvs/pgsql/src/backend/commands/portalcmds.c,v
retrieving revision 1.43
diff -c -r1.43 portalcmds.c
*** src/backend/commands/portalcmds.c 15 Oct 2005 02:49:15 - 1.43
--- src/backend/commands/portalcmds.c 2 Nov 2005 12:49:43 -
***
*** 190,196
return; /* keep compiler happy
*/
}
! /* Adjust dest if needed. MOVE wants destination None */
if (stmt-ismove)
dest = None_Receiver;
--- 190,196
return; /* keep compiler happy
*/
}
! /* Adjust dest if needed. MOVE wants destination DestNone */
if (stmt-ismove)
dest = None_Receiver;
***
*** 369,375
ExecutorRewind(queryDesc);
/* Change the destination to output to the tuplestore */
! queryDesc-dest = CreateDestReceiver(Tuplestore, portal);
/* Fetch the result set into the tuplestore */
ExecutorRun(queryDesc, ForwardScanDirection, 0L);
--- 369,375
ExecutorRewind(queryDesc);
/* Change the destination to output to the tuplestore */
! queryDesc-dest = CreateDestReceiver(DestTuplestore, portal);
/* Fetch the
Re: [PATCHES] Limit usage of tcop/dest.h
Alvaro Herrera [EMAIL PROTECTED] writes: Tom Lane wrote: I thought renaming them was a better idea, actually. Here is a patch for that. I will apply this to HEAD later today. Looks ok in a quick eyeball pass. regards, tom lane ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [PATCHES] [HACKERS] Reducing the overhead of NUMERIC data
On Tue, 2005-11-01 at 17:55 -0500, Tom Lane wrote:
Jim C. Nasby [EMAIL PROTECTED] writes:
FWIW, most databases I've used limit NUMERIC to 38 digits, presumably to
fit length info into 1 or 2 bytes. So there's something to be said for a
small numeric type that has less overhead and a large numeric (what we
have today).
I don't think it'd be worth having 2 types. Remember that the weight is
measured in base-10k digits. Suppose for instance
sign1 bit
weight 7 bits (-64 .. +63)
dscale 8 bits (0..255)
This gives us a dynamic range of 1e-256 to 1e255 as well as the ability
to represent up to 255 displayable fraction digits. Does anyone know
any real database applications where that's not enough?
(I'm neglecting NaN here in supposing we need only 1 bit for sign,
but we could have a special encoding for NaN. Perhaps disallow the
weight = -64 case and use that to signal NaN.)
I've coded a short patch to do this, which is the result of two
alternate patches and some thinking, but maybe not enough yet.
The patch given here is different on two counts from above:
This sets...
#define NUMERIC_MAX_PRECISION 64
since
#define NUMERIC_MAX_RESULT_SCALE(NUMERIC_MAX_PRECISION * 2)
We don't seem to be able to use all of the bits actually available to us
in the format. Perhaps we need to decouple these now? Previously, we had
room for 14 bits, which gave a maximum of 16384. We were using
NUMERIC_MAX of 1000, so doubling it didn't give problems.
The above on-disk format showed sign weight together, whereas the
current code has sign and dscale together. Trying to put sign and weight
together is somewhat difficult, since weight is itself a signed value.
I coded it up that way around, which is reasonably straightforward but
harder than the patch enclosed here. But AFAICS - which isn't that far
normally I grant you, doing things that way around would require some
twos-complement work to get things correct when weight is negative. That
worries me.
IMHO we should accept the step down in maximum numeric precision (down
to only 64 digits) rather than put extra processing into every
manipulation of a NUMERIC datatype. With luck, I've misunderstood and we
can have both performance and precision.
If not, I commend 64 digits to you as sufficient for every imaginable
purpose - saving 2 bytes off every numeric column. (And still 28 decimal
places more accurate than Oracle).
Best Regards, Simon Riggs
Index: src/include/utils/numeric.h
===
RCS file: /projects/cvsroot/pgsql/src/include/utils/numeric.h,v
retrieving revision 1.20
diff -c -c -r1.20 numeric.h
*** src/include/utils/numeric.h 1 Jan 2005 05:43:09 - 1.20
--- src/include/utils/numeric.h 2 Nov 2005 18:06:03 -
***
*** 15,24
#define _PG_NUMERIC_H_
/*
! * Hardcoded precision limit - arbitrary, but must be small enough that
! * dscale values will fit in 14 bits.
*/
! #define NUMERIC_MAX_PRECISION 1000
/*
* Internal limits on the scales chosen for calculation results
--- 15,24
#define _PG_NUMERIC_H_
/*
! * Hardcoded precision limit - must be small enough that
! * dscale values will fit into the number of bits available in NumericData
*/
! #define NUMERIC_MAX_PRECISION 64
/*
* Internal limits on the scales chosen for calculation results
***
*** 39,49
/*
* Sign values and macros to deal with packing/unpacking n_sign_dscale
*/
! #define NUMERIC_SIGN_MASK 0xC000
! #define NUMERIC_POS 0x
! #define NUMERIC_NEG 0x4000
! #define NUMERIC_NAN 0xC000
! #define NUMERIC_DSCALE_MASK 0x3FFF
#define NUMERIC_SIGN(n) ((n)-n_sign_dscale NUMERIC_SIGN_MASK)
#define NUMERIC_DSCALE(n) ((n)-n_sign_dscale NUMERIC_DSCALE_MASK)
#define NUMERIC_IS_NAN(n) (NUMERIC_SIGN(n) != NUMERIC_POS \
--- 39,49
/*
* Sign values and macros to deal with packing/unpacking n_sign_dscale
*/
! #define NUMERIC_SIGN_MASK 0xC0
! #define NUMERIC_POS 0x00
! #define NUMERIC_NEG 0x40
! #define NUMERIC_NAN 0xC0
! #define NUMERIC_DSCALE_MASK 0x3F
#define NUMERIC_SIGN(n) ((n)-n_sign_dscale NUMERIC_SIGN_MASK)
#define NUMERIC_DSCALE(n) ((n)-n_sign_dscale NUMERIC_DSCALE_MASK)
#define NUMERIC_IS_NAN(n) (NUMERIC_SIGN(n) != NUMERIC_POS \
***
*** 61,74
typedef struct NumericData
{
int32 varlen; /* Variable size (std varlena header) */
! int16 n_weight; /* Weight of 1st digit */
! uint16 n_sign_dscale; /* Sign + display scale */
char n_data[1]; /* Digits (really array of NumericDigit) */
} NumericData;
typedef NumericData *Numeric;
! #define NUMERIC_HDRSZ (sizeof(int32) + sizeof(int16) + sizeof(uint16))
/*
--- 61,74
typedef struct NumericData
{
int32 varlen; /* Variable size (std varlena header) */
! int8 n_weight; /* Weight of 1st digit */
! uint8 n_sign_dscale; /* Sign +
Re: [PATCHES] [HACKERS] Reducing the overhead of NUMERIC data
Simon Riggs [EMAIL PROTECTED] writes: On Tue, 2005-11-01 at 17:55 -0500, Tom Lane wrote: I don't think it'd be worth having 2 types. Remember that the weight is measured in base-10k digits. Suppose for instance sign1 bit weight 7 bits (-64 .. +63) dscale 8 bits (0..255) I've coded a short patch to do this, which is the result of two alternate patches and some thinking, but maybe not enough yet. What your patch does is sign2 bits weight 8 bits (-128..127) dscale 6 bits (0..63) which is simply pretty lame: weight effectively has a factor of 8 more dynamic range than dscale in this representation. What's the point of being able to represent 1 * 1 ^ -128 (ie, 10^-512) if the dscale only lets you show 63 fractional digits? You've got to allocate the bits in a saner fashion. Yes, that takes a little more work. Also, since the internal (unpacked) calculation representation has a much wider dynamic range than this, it'd probably be appropriate to add some range checks to the code that forms a packed value from unpacked. regards, tom lane ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [PATCHES] [HACKERS] Reducing the overhead of NUMERIC data
On Wed, 2005-11-02 at 13:46 -0500, Tom Lane wrote: Simon Riggs [EMAIL PROTECTED] writes: On Tue, 2005-11-01 at 17:55 -0500, Tom Lane wrote: I don't think it'd be worth having 2 types. Remember that the weight is measured in base-10k digits. Suppose for instance sign1 bit weight 7 bits (-64 .. +63) dscale 8 bits (0..255) I've coded a short patch to do this, which is the result of two alternate patches and some thinking, but maybe not enough yet. What your patch does is Thanks for checking this so quickly. sign2 bits OK, thats just a mistake in my second patch. Thats easily corrected. Please ignore that for now. weight 8 bits (-128..127) dscale 6 bits (0..63) which is simply pretty lame: weight effectively has a factor of 8 more dynamic range than dscale in this representation. What's the point of being able to represent 1 * 1 ^ -128 (ie, 10^-512) if the dscale only lets you show 63 fractional digits? You've got to allocate the bits in a saner fashion. Yes, that takes a little more work. I wasn't trying to claim the bit assignment made sense. My point was that the work to mangle the two fields together to make it make sense looked like it would take more CPU (since the standard representation of signed integers is different for +ve and -ve values). It is the more CPU I'm worried about, not the wasted bits on the weight. Spending CPU cycles on *all* numerics just so we can have numbers with +/-64 decimal places doesn't seem a good trade. Hence I stuck the numeric sign back on the dscale, and so dscale and weight seem out of balance. So, AFAICS, the options are: 0 (current cvstip) Numeric range up to 1000, with additional 2 bytes per column value 1. Numeric range up to 128, but with overhead to extract last bit 2. Numeric range up to 64 I'm suggesting we choose (2) other views are welcome. (I'll code it whichever way we decide.) Also, since the internal (unpacked) calculation representation has a much wider dynamic range than this, it'd probably be appropriate to add some range checks to the code that forms a packed value from unpacked. Well, there already is one that does that, otherwise I would have added one as you suggest. (The unpacked code has int values, whereas the previous packed format used u/int16 values). Best Regards, Simon Riggs ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [PATCHES] Partitioning docs
On Tue, 2005-11-01 at 18:19 -0500, Neil Conway wrote: On Mon, 2005-31-10 at 22:41 +, Simon Riggs wrote: I believe this is now complete and ready for application. Comments: - INSERT, UPDATE, etc. should be marked with command/, unless xref/ would be more appropriate - The names of GUC variables should be marked up with varname/, unless xref/ would be more appropriate - xref tags that link to the reference page of an SQL command should be of the form: xref linkend=sql-... endterm=sql-...-title -- the endterm attribute should not be omitted. - PostgreSQL should be marked-up with productname/ - In text like You can use RULEs to ..., rules would be better. - The word following a colon should not be capitalized - mdash; is an em dash, -- and --- are not - indexes, not indices Thanks very much for a thorough review. - Why Constraint Exclusion (or worse, the Constraint Exclusion feature) rather than simply constraint exclusion? OK (I'm not even sure it's a good idea to mention this term in end-user documentation.) We now have a parameter called constraint_exclusion, so the term already exists and so requires explanation. I would have had no objection to modifications of that term, but it has been in use now for 4 months, so changing it doesn't seem practical. - I removed a few statements and paragraphs I thought were unnecessary (e.g. Postgres was the first DBMS to have inheritance, some vague and IMHO useless advice about query optimization differences with inherited tables, etc.). Feel free to resubmit them if you disagree (although perhaps not for 8.1.0). Trying to identify which bit of advice you refer to I put some comments in based upon feedback from the beta on specific queries that were not optimised the same as non-inherited tables. If thats what you're talking about, then I'd like to put that back. The manuals aren't written for you and me; why let others stumble when they could have it in black and white? + All constraints on all partitions of the master table are considered for + Constraint Exclusion, so large numbers of partitions are likely to + increase query parse time considerably. Wouldn't it primarily increase planning time, not parsing time? Yes. What generic term would you use for query compilation? query preparation? The distinction of parsing/planning/optimization etc is lost on most people. + para + CE only works when the query directly matches a constant. A + constant bound to a parameterised query will not work in the same way + since the plan is fixed and would need to vary with each execution. + Also, stable constants such as CURRENT_DATE may not be used, since + these are constant only for during the execution of a single query. + Joins conditions will not allow CE to work either. + /para I'm not sure what the last sentence is intended to mean. OK, I'll work on a longer explanation of that. Revised patch attached and applied. There are at least a few more things that need cleaning up -- if no one beats me to it I'll do that shortly. Best Regards, Simon Riggs ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [PATCHES] [HACKERS] Reducing the overhead of NUMERIC data
Simon Riggs [EMAIL PROTECTED] writes: I wasn't trying to claim the bit assignment made sense. My point was that the work to mangle the two fields together to make it make sense looked like it would take more CPU (since the standard representation of signed integers is different for +ve and -ve values). It is the more CPU I'm worried about, not the wasted bits on the weight. I think that's purely hypothetical. The existing representation, as well as the one you propose, both require shift-and-mask operations to pull the fields out of the packed format. The format I'm suggesting would require some different shift-and-mask operations. As a first approximation it'd be a wash, and any actual differences would be CPU-specific enough that we shouldn't put a whole lot of weight on the point. (C compilers tend to be fairly bright about optimizing field extraction operations.) Moreover, you've forgotten the basic gating factor here, which is whether such a proposal will get accepted at all. Reducing the available range from 1000 digits to 255 might pass without too much objection, but dropping it down another factor of 4 to 63 starts to bring it uncomfortably close to mattering to people. [ thinks for a moment... ] Actually, neither proposal is going to get off the ground, because the parser's handling of numeric constants is predicated on the assumption that type NUMERIC can handle any valid value of FLOAT8, and so we can get away with converting to NUMERIC on sight and then coercing to float later if parse analysis finds out the constant should be float. If the dynamic range of NUMERIC is less than 10^308 then this fails. So we have to find another bit somewhere, or the idea is dead in the water. regards, tom lane ---(end of broadcast)--- TIP 6: explain analyze is your friend
[PATCHES] AIX FAQ addition
We haven't seen any agreement emerge as to what is causing AIX 5.3 ML3
to fail to successfully build the release candidates.
However, a patch has emerged (thanks, Seneca!) that does allow it to
work, and which I'd expect to be portable (better still!).
We are still actively pursuing why it breaks, but supposing that still
remains outstanding, at least the following would allow AIX users to
better survive a build...
Index: FAQ_AIX
===
RCS file: /projects/cvsroot/pgsql/doc/FAQ_AIX,v
retrieving revision 1.13
diff -c -u -r1.13 FAQ_AIX
--- FAQ_AIX 24 Oct 2005 22:30:35 - 1.13
+++ FAQ_AIX 2 Nov 2005 20:33:01 -
@@ -99,7 +99,7 @@
Last modified date 2005-09-06
If you upgrade to maintenance level 5300-03, that will include this
-fix. Use the command oslevel -r to determine what maintenance level
+fix. Use the command oslevel -r to determine what maintenance level
you are at.
---
From: Christopher Browne [EMAIL PROTECTED]
@@ -113,3 +113,63 @@
http://www.faqs.org/faqs/aix-faq/part4/section-22.html
http://www.han.de/~jum/aix/ldd.c
+---
+From: Christopher Browne [EMAIL PROTECTED]
+Date: 2005-11-02
+
+On AIX 5.3 ML3 (e.g. maintenance level 5300-03), there is some problem
+with the handling of the pointer to memcpy. It is speculated that
+this relates to some linker bug that may have been introduced between
+5300-02 and 5300-03, but we have so far been unable to track down the
+cause.
+
+At any rate, the following patch, which unwraps the function
+reference, has been observed to allow PG 8.1 pre-releases to pass
+regression tests.
+
+The same behaviour (albeit with varying underlying functions to
+blame) has been observed when compiling with either GCC 4.0 or IBM
+XLC.
+
+ per Seneca Cunningham ---
+
+The following patch works on the AIX 5.3 ML3 box here and didn't cause
+any problems with postgres on the x86 desktop. It's just a cleaner
+version of what I tried earlier.
+
+*** dynahash.c.orig Tue Nov 1 19:41:42 2005
+--- dynahash.c Tue Nov 1 20:30:33 2005
+***
+*** 670,676
+
+
+/* copy key into record */
+currBucket-hashvalue = hashvalue;
+! hashp-keycopy(ELEMENTKEY(currBucket), keyPtr, keysize);
+
+
+/* caller is expected to fill the data field on return */
+
+
+--- 670,687
+
+
+/* copy key into record */
+currBucket-hashvalue = hashvalue;
+! if (hashp-keycopy == memcpy)
+! {
+! memcpy(ELEMENTKEY(currBucket), keyPtr, keysize);
+! }
+! else if (hashp-keycopy == strncpy)
+! {
+! strncpy(ELEMENTKEY(currBucket), keyPtr, keysize);
+! }
+! else
+! {
+! hashp-keycopy(ELEMENTKEY(currBucket), keyPtr, keysize);
+! }
+
+
+/* caller is expected to fill the data field on return */
+
+ per Seneca Cunningham ---
--
(reverse (concatenate 'string moc.enworbbc @ enworbbc))
http://cbbrowne.com/info/x.html
Never take life seriously. Nobody gets out alive anyway.
---(end of broadcast)---
TIP 6: explain analyze is your friend
Re: [PATCHES] [HACKERS] Reducing the overhead of NUMERIC data
On Wed, 2005-11-02 at 15:09 -0500, Tom Lane wrote: Simon Riggs [EMAIL PROTECTED] writes: I wasn't trying to claim the bit assignment made sense. My point was that the work to mangle the two fields together to make it make sense looked like it would take more CPU (since the standard representation of signed integers is different for +ve and -ve values). It is the more CPU I'm worried about, not the wasted bits on the weight. I think that's purely hypothetical. The existing representation, as well as the one you propose, both require shift-and-mask operations to pull the fields out of the packed format. The format I'm suggesting would require some different shift-and-mask operations. As a first approximation it'd be a wash, and any actual differences would be CPU-specific enough that we shouldn't put a whole lot of weight on the point. (C compilers tend to be fairly bright about optimizing field extraction operations.) OK Moreover, you've forgotten the basic gating factor here, which is whether such a proposal will get accepted at all. Reducing the available range from 1000 digits to 255 might pass without too much objection, but dropping it down another factor of 4 to 63 starts to bring it uncomfortably close to mattering to people. [ thinks for a moment... ] Actually, neither proposal is going to get off the ground, because the parser's handling of numeric constants is predicated on the assumption that type NUMERIC can handle any valid value of FLOAT8, and so we can get away with converting to NUMERIC on sight and then coercing to float later if parse analysis finds out the constant should be float. If the dynamic range of NUMERIC is less than 10^308 then this fails. So we have to find another bit somewhere, or the idea is dead in the water. Well, that certainly is obscure, I'll give you that. 308 huh? The middle ground between 64 and 308 is somewhere around 255, yes? :-) I'll get on it. Including Catch-308. Best Regards, Simon Riggs ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [PATCHES] [HACKERS] Reducing the overhead of NUMERIC data
On Wed, 2005-11-02 at 15:09 -0500, Tom Lane wrote: [ thinks for a moment... ] Actually, neither proposal is going to get off the ground, because the parser's handling of numeric constants is predicated on the assumption that type NUMERIC can handle any valid value of FLOAT8, and so we can get away with converting to NUMERIC on sight and then coercing to float later if parse analysis finds out the constant should be float. If the dynamic range of NUMERIC is less than 10^308 then this fails. So we have to find another bit somewhere, or the idea is dead in the water. We convert a Value node to a Const in backend/parser/parse_node.c:make_const It seems straightforward enough to put in an additional test, similar to the ones already there so that if its too big for a decimal we make it a float straight away - only a float can be that big in that case. After that I can't really see what the problem is? There is only a single call where numeric_float8() is called anywhere in the code, which is during selectivity calculations. In that case we actually call numeric_float8_no_overflow(). If its a FLOAT8OID, then we can simply avoid the conversion, since it already would be one. Can you explain further? Thanks, Best Regards, Simon Riggs ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [PATCHES] [HACKERS] Reducing the overhead of NUMERIC data
Simon Riggs [EMAIL PROTECTED] writes: It seems straightforward enough to put in an additional test, similar to the ones already there so that if its too big for a decimal we make it a float straight away - only a float can be that big in that case. After that I can't really see what the problem is? Wrong answer. You'll be introducing weird corner cases into the type resolution behavior. An approach that would actually have some credibility would be to not resolve constants to NUMERIC right away, but to invent an UNKNOWNNUMERIC pseudotype with coercion behavior comparable to the existing UNKNOWN type for string literals. This has been proposed before but hasn't really been needed so far. Of course, this converts the project from a minor localized hack on NUMERIC into a major piece of fiddling with the type resolution rules, with the potential for unforeseen side-effects on the behavior of other data types. It might be worth doing anyway --- I don't recall at the moment what problems UNKNOWNNUMERIC was intended to solve, but if they're still open issues then it's something we ought to get around to sometime. regards, tom lane ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [PATCHES] [HACKERS] Reducing the overhead of NUMERIC data
On Wed, Nov 02, 2005 at 06:12:37PM -0500, Tom Lane wrote: Simon Riggs [EMAIL PROTECTED] writes: It seems straightforward enough to put in an additional test, similar to the ones already there so that if its too big for a decimal we make it a float straight away - only a float can be that big in that case. After that I can't really see what the problem is? Wrong answer. You'll be introducing weird corner cases into the type resolution behavior. An approach that would actually have some credibility would be to not resolve constants to NUMERIC right away, but to invent an UNKNOWNNUMERIC pseudotype with coercion behavior comparable to the existing UNKNOWN type for string literals. This has been proposed before but hasn't really been needed so far. Of course, this converts the project from a minor localized hack on NUMERIC into a major piece of fiddling with the type resolution rules, with the potential for unforeseen side-effects on the behavior of other data types. It might be worth doing anyway --- I don't recall at the moment what problems UNKNOWNNUMERIC was intended to solve, but if they're still open issues then it's something we ought to get around to sometime. Thought I'd look to see if I could find anything about UNKNOWNNUMERIC, but no such luck (ISTM we really need a better way to find discussion on old ideas...) But while looking I did find this TODO, which might be relevant to the current discussion: # Change NUMERIC to enforce the maximum precision, and increase it Unfortunately I can't find any reference to that in the archives... -- Jim C. Nasby, Sr. Engineering Consultant [EMAIL PROTECTED] Pervasive Software http://pervasive.comwork: 512-231-6117 vcard: http://jim.nasby.net/pervasive.vcf cell: 512-569-9461 ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [PATCHES] [HACKERS] Reducing the overhead of NUMERIC data
[patches removed] Tom Lane wrote: Simon Riggs [EMAIL PROTECTED] writes: It seems straightforward enough to put in an additional test, similar to the ones already there so that if its too big for a decimal we make it a float straight away - only a float can be that big in that case. After that I can't really see what the problem is? Wrong answer. You'll be introducing weird corner cases into the type resolution behavior. An approach that would actually have some credibility would be to not resolve constants to NUMERIC right away, but to invent an UNKNOWNNUMERIC pseudotype with coercion behavior comparable to the existing UNKNOWN type for string literals. This has been proposed before but hasn't really been needed so far. Of course, this converts the project from a minor localized hack on NUMERIC into a major piece of fiddling with the type resolution rules, with the potential for unforeseen side-effects on the behavior of other data types. It might be worth doing anyway --- I don't recall at the moment what problems UNKNOWNNUMERIC was intended to solve, but if they're still open issues then it's something we ought to get around to sometime. Could someone please quantify how much bang we might get for what seems like quite a lot of bucks? I appreciate the need for speed, but the saving here strikes me as marginal at best, unless my instincts are all wrong (quite possible) cheers andrew ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [PATCHES] Partitioning docs
On Wed, 2005-02-11 at 19:55 +, Simon Riggs wrote: Trying to identify which bit of advice you refer to I put some comments in based upon feedback from the beta on specific queries that were not optimised the same as non-inherited tables. ISTM that query optimization *always* works differently for inherited versus non-inherited tables, so there are a wide variety of queries you could describe like that. The other problem is the documentation is sufficiently vague that it is of little use, IMHO. Simply saying query X is optimized differently without explaining what causes the difference, what the performance impact is likely to be, or how to workaround the problem isn't likely to be very helpful. -Neil ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [PATCHES] [HACKERS] Reducing the overhead of NUMERIC data
Andrew Dunstan [EMAIL PROTECTED] writes: Could someone please quantify how much bang we might get for what seems like quite a lot of bucks? I appreciate the need for speed, but the saving here strikes me as marginal at best, unless my instincts are all wrong (quite possible) Two bytes per numeric value is not a lot, agreed. If we were willing to invent the varlena2 datum format then we could save four bytes per numeric, plus reduce numeric's alignment requirement from int to short which would probably save another byte per value on average. I'm not sure that that's worth doing if numeric and inet are the only beneficiaries, but it might be. From a speed perspective the only thing favoring UNKNOWNNUMERIC is the possibility for saving some conversion operations when the grammar's initial guess about datatype is wrong. That's probably pretty marginal though. I was thinking of it more as a vehicle for helping us clean up the type-assignment behavior some more. The example I have in my notes is that float4col = 1.8 is certain to fail since 1.8 will be taken as float8, not float4, and then you have precision mismatch problems. You can make it work by quoting: float4col = '1.8' but that's surely pretty ugly. If the constant were initially UNKNOWNNUMERIC then it would end up as float4 without that hack. regards, tom lane ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [PATCHES] pgcrypto doc polish
Marko Kreen marko@l-t.ee writes: Few small things: [ snip ] Applied, thanks. I also fixed a few small grammatical problems in the text. regards, tom lane ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
