Re: [PATCHES] pg_ctl -D canonicalization
It seems pg_ctl calls canonicalize_path() only on the path as being used to access for example the pid file, and not the path that is sent along to the postmaster. Specifically, this causes failure on win32 when a path is passed with a trailing backslash, when it's inside quotes such as: pg_ctl start -D c:\program fiels\postgresql\data\ The quotes are of course necessary since there are spaces in the filename. In my specific case the trailing backslash is automatically added by windows installer. but other cases when the backslash is added can easily be seen... Attached patch makes pg_ctl call canonicalize_path() on the path as passed on the commandline as well. I have only tested this on win32, where it appears to work fine. I think it would be good on other platsforms as well which is why I didn't #ifdef it. If not, then please add #ifdefs. Uh, isn't the proper fix to fix the postmaster's handling of -D paths? Nope. The problem is that in this case, pg_ctl gets the path: c:\program files\postgresql\data\ (not unbalanced quotes). Yes, win32 commandline quoting is horrible. Anyway. then it properly quotes this for the commandline: c:\program files\postgresql\data\ and you can see where that breaks down... Another option would be simple check to simply strip the (and the trailing backslash too, probably, to make things safer at the next step). I just thought it'd be cleaner to use canonicalize_path(). But it's just the trailing quote stuff that causes the problem. (See the second step in canonicalize_path() where this has been noticed befoer, followed by the trim-trailing part which would also be nice). (Everything works if you register it as a service because then it uses pg_data and not pgdata_opt. This is just when you start things manually, which is why we missed it before...) //Magnus ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [PATCHES]Update 7 zh_TW message files for 8.0
Zhenbang Wei wrote: Update initdb-zh_TW.po, libpq-zh_TW.po, pg_config-zh_TW.po, pg_ctl-zh_TW.po, pg_dump-zh_TW.po, postgres-zh_TW.po, and psql-zh_TW.po. Translate new messages and update invalid entries. Installed. ---(end of broadcast)--- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [PATCHES] HP-UX PA-RISC/Itanium 64-bit Patch and HP-UX 11.23 Patch
On Tue, 26 Oct 2004 21:27:13 -0600, Ed L. [EMAIL PROTECTED] said:
Shinji Teragaito [EMAIL PROTECTED] writes:
I made a patch to let PostgreSQL work in the LP64 data model on
HP-UX PA-RISC and HP-UX Itanium platform.
I see Shinji's patch changed the library suffix from .sl to .so for ia64.
Is that is necessary? If so, why?
HP-UX Itanium native shared library suffix is .so, not .sl.
My patch conforms to such a convention.
Please take a look at the directory /usr/lib/hpux{32,64}. Note .sl
libraries under /usr/lib and /usr/lib/pa{1.1,2.0,20_32,20_64} are PA
binaries for Aries.
Cheers,
Shinji
---(end of broadcast)---
TIP 5: Have you checked our extensive FAQ?
http://www.postgresql.org/docs/faqs/FAQ.html
Re: [PATCHES] Updated Turkish translations
Devrim GUNDUZ wrote: http://www.gunduz.org/postgresql/translation/PostgreSQL-8.0/PgSQL-Dil Dosyalari/pg_config-tr.po is the updated pg_config translation. Also, the following are initial translations for libpq and pg_dump: http://www.gunduz.org/postgresql/translation/PostgreSQL-8.0/PgSQL-Dil Dosyalari/pg_dump-tr.po http://www.gunduz.org/postgresql/translation/PostgreSQL-8.0/PgSQL-Dil Dosyalari/libpq-tr.po Could you please apply them? Done. ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [PATCHES] Translation updates for 8.0: initdb-ru, libpq-ru, pg_config-ru, pg_ctl-ru, pg_dump-ru
Serguei Mokhov wrote: Please install the attached Russian translation updates. Done. ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: [PATCHES] Updated pg_ctl Turkish translation
Devrim GUNDUZ wrote: After the latest changes, I've updated pg_ctl Turkish translation: Installed. ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Re: [PATCHES] 8.0-NLS: czech
Karel Zak wrote: on URL: http://people.redhat.com/kzak/pg-nls-cs-21102004.patch.gz is new Czech translation of PostgreSQL messages. It's too big for e-mail attachment and mailing list limit. Please make available the individual PO files, not a patch. The patch you supplied does not apply cleanly. ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Re: [PATCHES] (Turkish) New translation: psql
Nicolai Tufar wrote: I've began translating psql into Turkish. The file is attached. Could you please apply it for 8.0? Done. ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
[PATCHES] sign parsing (was: Re: [HACKERS] to_char/to_number loses sign)
On Tue, 2004-10-26 at 13:27 -0400, Tom Lane wrote:
Karel Zak [EMAIL PROTECTED] writes:
Yes, you're right. It strange, but NUM_S missing there. The conversion
from string to number is less stable part of formatting.c...
The patch is in the attachment.
This patch causes the regression tests to fail. I think you need to
consider the to_char() side of it more carefully.
Sorry of this in beta version...
The problem was bigger than I expected. I hope it's fixed in actual
patch. All regression tests pass.
The patch changes two lines in regression test too. It's because old
version of to_number() allows to use wrong number input and this input
was in regression test. The call to_char(-0.01, ' 9 9 . 9 9 S'); never
produced ' . 0 1 -' but always ' . 0 1-' only.
-SELECT '' AS to_number_13, to_number(' . 0 1 -', ' 9 9 . 9 9 S');
+SELECT '' AS to_number_13, to_number(' . 0 1-', ' 9 9 . 9 9 S');
^^^
'S' = locale sign and it must be always anchored to last or first
number.
Again sorry,
Karel
--
Karel Zak
http://home.zf.jcu.cz/~zakkr
pgsql-formatting-10272004.patch.gz
Description: GNU Zip compressed data
---(end of broadcast)---
TIP 3: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to [EMAIL PROTECTED] so that your
message can get through to the mailing list cleanly
[PATCHES] contrib/tsearch2 missing LIBS
beta4 - cygwin: dllwrap -o tsearch2.dll --dllname tsearch2.dll --def tsearch2.def dict_ex.o dict.o snmap.o stopword.o common.o prs_dcfg.o dict_snowball.o dict_ispell.o dict_syn.o wparser.o wparser_def.o ts_cfg.o tsvector.o rewrite.o crc32.o query.o gistidx.o tsvector_op.o rank.o ts_stat.o snowball/SUBSYS.o ispell/SUBSYS.o wordparser/SUBSYS.o -L../../src/port -L/usr/local/lib -lm -L../../src/backend -lpostgres -lpgport ../../src/port/libpgport.a(path.o)(.text+0x6b7):path.c: undefined reference to `_libintl_bindtextdomain' ../../src/port/libpgport.a(path.o)(.text+0x6bf):path.c: undefined reference to `_libintl_textdomain' ../../src/port/libpgport.a(exec.o)(.text+0x5df):exec.c: undefined reference to `_libintl_gettext' ../../src/port/libpgport.a(exec.o)(.text+0x60c):exec.c: undefined reference to `_libintl_gettext' ../../src/port/libpgport.a(exec.o)(.text+0x625):exec.c: undefined reference to `_libintl_gettext' make -p: # Makefile (from `../../src/Makefile.shlib', line 237) SHLIB_LINK := -L../../src/port -L/usr/local/lib -lm -L../../src/backend -lpostgres -lpgport culprit: SHLIB_LINK := $(filter -L%, $(LDFLAGS)) $(SHLIB_LINK) SHLIB_LINK misses $(LIBS) my solution: --- postgresql-8.0.0beta4/src/Makefile.shlib.orig 2004-10-16 04:26:43.0 +0100 +++ postgresql-8.0.0beta4/src/Makefile.shlib 2004-10-27 13:21:19.184875000 +0100 @@ -219,7 +219,7 @@ ifeq ($(PORTNAME), cygwin) shlib = $(NAME)$(DLSUFFIX) # needed for /contrib modules, not sure why - SHLIB_LINK += -lpgport + SHLIB_LINK += $(LIBS) endif ifeq ($(PORTNAME), win32) -- Reini Urban http://xarch.tu-graz.ac.at/home/rurban/ ---(end of broadcast)--- TIP 8: explain analyze is your friend
Re: [PATCHES] 8.0-NLS: czech
On Wed, 2004-10-27 at 13:24 +0200, Peter Eisentraut wrote: Please make available the individual PO files, not a patch. The patch you supplied does not apply cleanly. OK. Please don't forget add cs to src/bin/pg_config/nls.mk http://people.redhat.com/kzak/pg-nls-cs-10272004.tar.gz Karel -- Karel Zak http://home.zf.jcu.cz/~zakkr ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
[PATCHES] Updated Turkish translations
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi, http://postgresql.gunduz.org/translation/PostgreSQL-8.0/PgSQL-DilDosyalari/libpq-tr.po http://postgresql.gunduz.org/translation/PostgreSQL-8.0/PgSQL-DilDosyalari/pg_ctl-tr.po http://postgresql.gunduz.org/translation/PostgreSQL-8.0/PgSQL-DilDosyalari/psql-tr.po These are %100 against current strings. Could you please apply them for 8.0? Regards, - -- Devrim GUNDUZ devrim~gunduz.orgdevrim.gunduz~linux.org.tr http://www.tdmsoft.com http://www.gunduz.org -BEGIN PGP SIGNATURE- Version: GnuPG v1.2.1 (GNU/Linux) iD8DBQFBf8VPtl86P3SPfQ4RAvkWAJ9eJtgDLNWSbze8I7OIYtJeY4zIUwCeJVGM SSPvEYbw+QlQJFjI/xmMs/M= =8OO/ -END PGP SIGNATURE- ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: [PATCHES] pgstat cleanup: use palloc and AllocateFile
Neil Conway [EMAIL PROTECTED] writes: I changed more_tabstat_space() (which is invoked at various times indirectly throughout the backend) to allocate memory in its own private memory context, rather than use malloc() -- we can't just use CurrentMemoryContext because that may not be sufficiently long-lived. I'd suggest just allocating the tabstat space in TopMemoryContext. The extra sub-context is useless overhead, since you don't actually use it for management purposes. regards, tom lane ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [PATCHES] pg_ctl -D canonicalization
OK, attached patch applied. Your original patch didn't canonicalize the
PGDATA environment variable. '-D' takes precidence but it should be
accurate.
I also updated the comments in canonicalize_path because it does a lot
more now that just win to unix path conversion.
---
Magnus Hagander wrote:
It seems pg_ctl calls canonicalize_path() only on the path as being
used to access for example the pid file, and not the path
that is sent
along to the postmaster.
Specifically, this causes failure on win32 when a path is
passed with
a trailing backslash, when it's inside quotes such as:
pg_ctl start -D c:\program fiels\postgresql\data\
The quotes are of course necessary since there are spaces in the
filename. In my specific case the trailing backslash is
automatically
added by windows installer. but other cases when the backslash is
added can easily be seen...
Attached patch makes pg_ctl call canonicalize_path() on the path as
passed on the commandline as well.
I have only tested this on win32, where it appears to work fine. I
think it would be good on other platsforms as well which is why I
didn't #ifdef it. If not, then please add #ifdefs.
Uh, isn't the proper fix to fix the postmaster's handling of -D paths?
Nope.
The problem is that in this case, pg_ctl gets the path:
c:\program files\postgresql\data\
(not unbalanced quotes). Yes, win32 commandline quoting is horrible.
Anyway. then it properly quotes this for the commandline:
c:\program files\postgresql\data\
and you can see where that breaks down...
Another option would be simple check to simply strip the (and the
trailing backslash too, probably, to make things safer at the next
step). I just thought it'd be cleaner to use canonicalize_path(). But
it's just the trailing quote stuff that causes the problem. (See the
second step in canonicalize_path() where this has been noticed befoer,
followed by the trim-trailing part which would also be nice).
(Everything works if you register it as a service because then it uses
pg_data and not pgdata_opt. This is just when you start things manually,
which is why we missed it before...)
//Magnus
---(end of broadcast)---
TIP 6: Have you searched our list archives?
http://archives.postgresql.org
--
Bruce Momjian| http://candle.pha.pa.us
[EMAIL PROTECTED] | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup.| Newtown Square, Pennsylvania 19073
Index: src/bin/pg_ctl/pg_ctl.c
===
RCS file: /cvsroot/pgsql/src/bin/pg_ctl/pg_ctl.c,v
retrieving revision 1.42
diff -c -c -r1.42 pg_ctl.c
*** src/bin/pg_ctl/pg_ctl.c 22 Oct 2004 00:24:18 - 1.42
--- src/bin/pg_ctl/pg_ctl.c 27 Oct 2004 17:10:58 -
***
*** 1279,1297
{
case 'D':
{
! int len =
strlen(optarg);
! char *env_var;
! env_var = xmalloc(len + 8);
! snprintf(env_var, len + 8,
PGDATA=%s, optarg);
putenv(env_var);
/*
!* Show -D for easier postmaster 'ps'
!* identification
*/
! pgdata_opt = xmalloc(len + 7);
! snprintf(pgdata_opt, len + 7, -D
\%s\ , optarg);
break;
}
case 'l':
--- 1279,1301
{
case 'D':
{
! char *pgdata_D =
xmalloc(strlen(optarg));
! char *env_var =
xmalloc(strlen(optarg) + 8);
! strcpy(pgdata_D, optarg);
! canonicalize_path(pgdata_D);
! snprintf(env_var, strlen(pgdata_D) +
8, PGDATA=%s,
!pgdata_D);
putenv(env_var);
/*
!
Re: [PATCHES] [HACKERS] ARC Memory Usage analysis
On Mon, Oct 25, 2004 at 11:34:25AM -0400, Jan Wieck wrote: On 10/22/2004 4:09 PM, Kenneth Marshall wrote: On Fri, Oct 22, 2004 at 03:35:49PM -0400, Jan Wieck wrote: On 10/22/2004 2:50 PM, Simon Riggs wrote: I've been using the ARC debug options to analyse memory usage on the PostgreSQL 8.0 server. This is a precursor to more complex performance analysis work on the OSDL test suite. I've simplified some of the ARC reporting into a single log line, which is enclosed here as a patch on freelist.c. This includes reporting of: - the total memory in use, which wasn't previously reported - the cache hit ratio, which was slightly incorrectly calculated - a useful-ish value for looking at the B lists in ARC (This is a patch against cvstip, but I'm not sure whether this has potential for inclusion in 8.0...) The total memory in use is useful because it allows you to tell whether shared_buffers is set too high. If it is set too high, then memory usage will continue to grow slowly up to the max, without any corresponding increase in cache hit ratio. If shared_buffers is too small, then memory usage will climb quickly and linearly to its maximum. The last one I've called turbulence in an attempt to ascribe some useful meaning to B1/B2 hits - I've tried a few other measures though without much success. Turbulence is the hit ratio of B1+B2 lists added together. By observation, this is zero when ARC gives smooth operation, and goes above zero otherwise. Typically, turbulence occurs when shared_buffers is too small for the working set of the database/workload combination and ARC repeatedly re-balances the lengths of T1/T2 as a result of near-misses on the B1/B2 lists. Turbulence doesn't usually cut in until the cache is fully utilized, so there is usually some delay after startup. We also recently discussed that I would add some further memory analysis features for 8.1, so I've been trying to figure out how. The idea that B1, B2 represent something really useful doesn't seem to have been borne out - though I'm open to persuasion there. I originally envisaged a shadow list operating in extension of the main ARC list. This will require some re-coding, since the variables and macros are all hard-coded to a single set of lists. No complaints, just it will take a little longer than we all thought (for me, that is...) My proposal is to alter the code to allow an array of memory linked lists. The actual list would be [0] - other additional lists would be created dynamically as required i.e. not using IFDEFs, since I want this to be controlled by a SIGHUP GUC to allow on-site tuning, not just lab work. This will then allow reporting against the additional lists, so that cache hit ratios can be seen with various other prototype shared_buffer settings. All the existing lists live in shared memory, so that dynamic approach suffers from the fact that the memory has to be allocated during ipc_init. What do you think about my other theory to make C actually 2x effective cache size and NOT to keep T1 in shared buffers but to assume T1 lives in the OS buffer cache? Jan Jan, From the articles that I have seen on the ARC algorithm, I do not think that using the effective cache size to set C would be a win. The design of the ARC process is to allow the cache to optimize its use in response to the actual workload. It may be the best use of the cache in some cases to have the entire cache allocated to T1 and similarly for T2. If fact, the ability to alter the behavior as needed is one of the key advantages. Only the working set of the database, that is the pages that are very frequently used, are worth holding in shared memory at all. The rest should be copied in and out of the OS disc buffers. The problem is, with a too small directory ARC cannot guesstimate what might be in the kernel buffers. Nor can it guesstimate what recently was in the kernel buffers and got pushed out from there. That results in a way too small B1 list, and therefore we don't get B1 hits when in fact the data was found in memory. B1 hits is what increases the T1target, and since we are missing them with a too small directory size, our implementation of ARC is propably using a T2 size larger than the working set. That is not optimal. If we would replace the dynamic T1 buffers with a max_backends*2 area of shared buffers, use a C value representing the effective cache size and limit the T1target on the lower bound to effective cache size - shared buffers, then we basically moved the T1 cache into the OS buffers. This all only holds water, if the OS is allowed to swap out shared memory. And that was my initial question, how likely is it to find this to be true these days? Jan I've asked our linux kernel guys some quick questions and they say you can lock mmapped memory
[PATCHES] dblink crash fix
This makes dblink pass its installcheck test on platforms where
snprintf(data, len, %s, NULL) crash.
The code was trying to find a connection by name when it already had an
unnamed connection and did not have a name to search with.
Kris JurkaIndex: contrib/dblink/dblink.c
===
RCS file: /projects/cvsroot/pgsql/contrib/dblink/dblink.c,v
retrieving revision 1.36
diff -c -r1.36 dblink.c
*** contrib/dblink/dblink.c 25 Oct 2004 00:46:39 - 1.36
--- contrib/dblink/dblink.c 28 Oct 2004 00:07:01 -
***
*** 310,319
conname = GET_STR(PG_GETARG_TEXT_P(0));
curname = GET_STR(PG_GETARG_TEXT_P(1));
sql = GET_STR(PG_GETARG_TEXT_P(2));
}
- rcon = getConnectionByName(conname);
- if (rcon)
- conn = rcon-con;
}
else if (PG_NARGS() == 4)
{
--- 310,319
conname = GET_STR(PG_GETARG_TEXT_P(0));
curname = GET_STR(PG_GETARG_TEXT_P(1));
sql = GET_STR(PG_GETARG_TEXT_P(2));
+ rcon = getConnectionByName(conname);
+ if (rcon)
+ conn = rcon-con;
}
}
else if (PG_NARGS() == 4)
{
---(end of broadcast)---
TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [PATCHES] [HACKERS] rmtree() failure on Windows
Tom Lane wrote: Try putting RequestCheckpoint(true) in dbcommands.c just before remove_dbtablespaces (about line 630). It looks like the bgwriter is not quite up-to-speed for this, either; you should rearrange things near line 350 of bgwriter.c so that smgrcloseall is performed before marking the checkpoint done in shmem. Else RequestCheckpoint could come back before the files are all closed. seems to do the trick. patch attached. I did wonder if there should be a call that instead of forcing a flush could tell bgwriter just to forget about the file(s) because we're discarding them. But that was just idle speculation - I haven't looked at bgwriter at all. Not necessary, as long as you put the checkpoint after the DropBuffers call in dbcommands.c. The bgwriter won't find anything to write. What about other databases? Or won't the forced checkpoint affect them? cheers andrew Index: src/backend/commands/dbcommands.c === RCS file: /home/cvsmirror/pgsql/src/backend/commands/dbcommands.c,v retrieving revision 1.145 diff -c -r1.145 dbcommands.c *** src/backend/commands/dbcommands.c 17 Oct 2004 20:47:20 - 1.145 --- src/backend/commands/dbcommands.c 27 Oct 2004 23:48:10 - *** *** 32,37 --- 32,38 #include commands/tablespace.h #include mb/pg_wchar.h #include miscadmin.h + #include postmaster/bgwriter.h #include storage/fd.h #include storage/freespace.h #include storage/sinval.h *** *** 625,630 --- 626,639 FreeSpaceMapForgetDatabase(db_id); /* + * On Windows, force a checkpoint so that the bgwriter doesn't hold any + * open files, which would cause rmdir() to fail. + */ + #ifdef WIN32 + RequestCheckpoint(true); + #endif + + /* * Remove all tablespace subdirs belonging to the database. */ remove_dbtablespaces(db_id); Index: src/backend/postmaster/bgwriter.c === RCS file: /home/cvsmirror/pgsql/src/backend/postmaster/bgwriter.c,v retrieving revision 1.9 diff -c -r1.9 bgwriter.c *** src/backend/postmaster/bgwriter.c 12 Oct 2004 21:54:40 - 1.9 --- src/backend/postmaster/bgwriter.c 27 Oct 2004 23:05:10 - *** *** 347,352 --- 347,361 CreateCheckPoint(false, force_checkpoint); /* + * After any checkpoint, close all smgr files. This is so we + * won't hang onto smgr references to deleted files + * indefinitely. (It is safe to do this because this process + * does not have a relcache, and so no dangling references + * could remain.) + */ + smgrcloseall(); + + /* * Indicate checkpoint completion to any waiting backends. */ BgWriterShmem-ckpt_done = BgWriterShmem-ckpt_started; *** *** 359,373 */ last_checkpoint_time = now; - /* - * After any checkpoint, close all smgr files. This is so we - * won't hang onto smgr references to deleted files - * indefinitely. (It is safe to do this because this process - * does not have a relcache, and so no dangling references - * could remain.) - */ - smgrcloseall(); - /* Nap for configured time before rechecking */ n = 1; } --- 368,373 ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [PATCHES] [HACKERS] rmtree() failure on Windows
Andrew Dunstan [EMAIL PROTECTED] writes: Tom Lane wrote: Try putting RequestCheckpoint(true) in dbcommands.c just before remove_dbtablespaces (about line 630). seems to do the trick. patch attached. Patch applied. Not necessary, as long as you put the checkpoint after the DropBuffers call in dbcommands.c. The bgwriter won't find anything to write. What about other databases? Or won't the forced checkpoint affect them? For them, it's just an ordinary checkpoint. It's kind of a brute-force solution, but I'm disinclined to worry about optimizing DROP DATABASE ... regards, tom lane ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Re: [PATCHES] pgstat cleanup: use palloc and AllocateFile
On Thu, 2004-10-28 at 02:20, Tom Lane wrote: I'd suggest just allocating the tabstat space in TopMemoryContext. The extra sub-context is useless overhead, since you don't actually use it for management purposes. I don't know why I didn't think of that :) Patch applied with your suggested change. -Neil ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
