Re: [PATCHES] Win32 signals sockets
Magnus Hagander [EMAIL PROTECTED] writes: This patch *replaces* the previous one. Contains the exact same changes, except it *also* contains the move of the backend parameter file to shared memory on win32. I think it's way too late in the beta cycle for significant changes in the fork mechanism ... especially if the gain is only 10% on something that isn't a performance issue for properly written applications anyway. Can I safely ignore this patch and use the prior one, or are there additional bug fixes in this? regards, tom lane ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [PATCHES] Win32 signals sockets
This patch *replaces* the previous one. Contains the exact same changes, except it *also* contains the move of the backend parameter file to shared memory on win32. I think it's way too late in the beta cycle for significant changes in the fork mechanism ... especially if the gain is only 10% on something that isn't a performance issue for properly written applications anyway. Can I safely ignore this patch and use the prior one, or are there additional bug fixes in this? You can, there are no additional bug fixes. But let me give it a shot to try to convince you to put it in there anyway :-) I realise it's late in the beta. But all the actually *complicated* code in this patch is in the first patch - the splitting up of the CreateProcess/ResumeThread steps and the WSADuplicateSocket code. The part that moves the param file - shared memory is a very small and simple part of the patch. It's basically two API calls int he postmaster (parent) and one in the backend. Considering that this is very little code, uses only very core features on the win32 API, and will be exercised *all the time* (every backend startup, ever), we should see errors in thise codepath pretty soon. If we do, we can always back it out before RC instead of trying to fix it. Just the speedup is not the only reason to apply it. Continually creating and deleting a file in the directory will also cause the filesystem journal and MFT entries to build up significantly. (Every file commit also requires a flush of the filesystem journal, which I beleive is where those 10% are found. The difference would probably be greater than 10% on a system that is I/O loaded, but I haven't tested that) (The reason the patch looks pretty much different from the first patch is that a couple of structures had to be moved higher up in postmaster.c causing the diff to be larger) But the summary still holds - this is not a *critical* bug fix. I'd argue it's a bug fix, but we *can* do without it. //Magnus (No, I'm not going to push for the next stage of forkexec cleanups to go in 8.0 :P) ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [PATCHES] Win32 signals sockets
Magnus Hagander [EMAIL PROTECTED] writes: I think it's way too late in the beta cycle for significant changes in the fork mechanism ... I realise it's late in the beta. But all the actually *complicated* code in this patch is in the first patch - the splitting up of the CreateProcess/ResumeThread steps and the WSADuplicateSocket code. The part that moves the param file - shared memory is a very small and simple part of the patch. Maybe so, but it also puts the final nail in the coffin of the illusion that testing EXEC_BACKEND behavior on Unix will give any confidence about not having broken the code on Windows. Also I think your thumb is on the scales a bit because the initial patch is doing more than it had to in this area (you didn't need to invent struct BackendParameters, did you?) It's the increase in variance between the Unix and Windows code paths that's really bothering me. We went into this project on the promise that there weren't going to be thousands of lines of #ifdef WIN32 stuff, and I'm not happy in the least with the way postmaster.c looks now, let alone after applying this patch. regards, tom lane ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [PATCHES] Win32 signals sockets
I realise it's late in the beta. But all the actually *complicated* code in this patch is in the first patch - the splitting up of the CreateProcess/ResumeThread steps and the WSADuplicateSocket code. The part that moves the param file - shared memory is a very small and simple part of the patch. Maybe so, but it also puts the final nail in the coffin of the illusion that testing EXEC_BACKEND behavior on Unix will give any confidence about not having broken the code on Windows. Not sure how it will be that much worse than what we have now. It will certainly not check if the actual exec backend code is broken, but if it's missing to pass a long a variable or so (which I beleive would be the most common things in the future) it should catch it just as well as today. Also I think your thumb is on the scales a bit because the initial patch is doing more than it had to in this area (you didn't need to invent struct BackendParameters, did you?) No, I didn't need to do that, but I figured it was the easiest way to be able to check the error values. Previously, the return code from fwrite() was never checked, which cannot possibly be said to be good. But sure, it could be changed to check the result code of every fwrite() and fread() instead. And sure, I may have been influenced by my plans (or the plans in general, since it's been on the TODO for ages) to eventually move it to shared memory when I chose that path - but I still think it's the easiest option regardless. It's the increase in variance between the Unix and Windows code paths that's really bothering me. We went into this project on the promise that there weren't going to be thousands of lines of #ifdef WIN32 stuff, and I'm not happy in the least with the way postmaster.c looks now, let alone after applying this patch. Right. This is what I was planning to address in the cleanup step, but that's a lot larger changes than these... The plan to get more #ifdefs out of there and into a port-specific file instead. It would still be different code, but it would be in more clearly defined parts. Similar to the rest of the win32 specific code that goes in backend/port. //Magnus ---(end of broadcast)--- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [PATCHES] Win32 signals sockets
Tom Lane wrote: It's the increase in variance between the Unix and Windows code paths that's really bothering me. We went into this project on the promise that there weren't going to be thousands of lines of #ifdef WIN32 stuff, and I'm not happy in the least with the way postmaster.c looks now, let alone after applying this patch. I've been following this thread for a bit and I have to admit I wouldn't mind seeing the shmmem part of Magnus's patch go in. Windows suffers vs unix generally on process creation times and any improvement here would be welcome. Merlin ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [PATCHES] Win32 signals sockets
Merlin Moncure [EMAIL PROTECTED] writes: I've been following this thread for a bit and I have to admit I wouldn't mind seeing the shmmem part of Magnus's patch go in. Windows suffers vs unix generally on process creation times and any improvement here would be welcome. [ grumble... ] OK, as long as Magnus is promising a code-beautification patch. postmaster.c is rapidly approaching a condition of unreadability == unmaintainability. regards, tom lane ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Re: [PATCHES] Win32 signals sockets
[ grumble... ] OK, as long as Magnus is promising a code-beautification patch. postmaster.c is rapidly approaching a condition of unreadability == unmaintainability == perl ;). regards, tom lane ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED]) -- Command Prompt, Inc., home of PostgreSQL Replication, and plPHP. Postgresql support, programming shared hosting and dedicated hosting. +1-503-667-4564 - [EMAIL PROTECTED] - http://www.commandprompt.com Mammoth PostgreSQL Replicator. Integrated Replication for PostgreSQL begin:vcard fn:Joshua D. Drake n:Drake;Joshua D. org:Command Prompt, Inc. adr:;;PO Box 215;Cascade Locks;Oregon;97014;USA email;internet:[EMAIL PROTECTED] title:Consultant tel;work:503-667-4564 tel;fax:503-210-0334 note:Command Prompt, Inc. is the largest and oldest US based commercial PostgreSQL support provider. We provide the only commercially viable integrated PostgreSQL replication solution, but also custom programming, and support. We authored the book Practical PostgreSQL, the procedural language plPHP, and adding trigger capability to plPerl. x-mozilla-html:FALSE url:http://www.commandprompt.com/ version:2.1 end:vcard ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: [PATCHES] [HACKERS] plperl Safe restrictions
Andrew Dunstan [EMAIL PROTECTED] writes: It has just been brought to my attention that we are being very restrictive about what we allow to be done in trusted plperl. ... OK, based on this and some further thought, I have prepared the attached patch which does the right thing, I think, both in terms of what we allow and what we don't. Applied, with changes to allow srand and disallow sprintf, as per subsequent discussion. regards, tom lane ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [PATCHES] Patch pg_start_backup() to ERROR if archiving is not
On Sun, 2004-11-14 at 23:05 +1100, Gavin Sherry wrote: Attached is a patch which checks early on in pg_start_back() that archive_command is defined and if not generates an error. This seemed like a reason foot gun prevention measure. I'll apply this before end of day today. -Neil ---(end of broadcast)--- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [PATCHES] Slovenian translations for 8.0
Aleksander Kmetec wrote: Translations for initdb, pg_config, pg_ctl and pg_dump are new, so please also add them to nls makefiles. Done. -- Peter Eisentraut http://developer.postgresql.org/~petere/ ---(end of broadcast)--- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [PATCHES] Translation updates for 8.0: initdb-ru, psql-ru
Serguei Mokhov wrote: Updated translations of initdb and psql. Please install. (German quotes replaced with English ones in the 2 source strings in initdb I pointed out earlier). Installed. -- Peter Eisentraut http://developer.postgresql.org/~petere/ ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: [PATCHES] [HACKERS] plperl Safe restrictions
Applied, with changes to allow srand and disallow sprintf, as per subsequent discussion. How about allowing: use utf8; use locale; ? Kind Regards, John ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Re: [PATCHES]Update traditional chinese translations for 8.0
Zhenbang Wei wrote: Eliminate fuzzy messages and fix characters destroyed by poEdit. Installed. -- Peter Eisentraut http://developer.postgresql.org/~petere/ ---(end of broadcast)--- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [PATCHES] Updated Turkish translations for 8.0
Devrim GUNDUZ wrote: We've updated Turkish translations for 8.0: Installed. -- Peter Eisentraut http://developer.postgresql.org/~petere/ ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faqs/FAQ.html
Re: [PATCHES] Translation updates for 7.4/8.0: postgres-ru
Serguei Mokhov wrote: Please apply this one to both, 7.4 and 8.0. Done. -- Peter Eisentraut http://developer.postgresql.org/~petere/ ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [PATCHES] Translation updates for 8.0: pg_dump-ru, pgscripts-ru
Serguei Mokhov wrote: Please install the attached translation updates. Done. -- Peter Eisentraut http://developer.postgresql.org/~petere/ ---(end of broadcast)--- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [PATCHES] Updated french translations for 8.0
Guillaume LELARGE wrote: Please apply these updated french .po files. Done. -- Peter Eisentraut http://developer.postgresql.org/~petere/ ---(end of broadcast)--- TIP 8: explain analyze is your friend
Re: [PATCHES] [HACKERS] plperl Safe restrictions
John Hansen wrote: Applied, with changes to allow srand and disallow sprintf, as per subsequent discussion. How about allowing: use utf8; use locale; ? I think it is *way* too late in the dev cycle to be proposing this. Maybe it should be a TODO item - I at least don't have time even to think about the implications os using these pragmas. The effect of the first is achievable via an environment setting, I believe. If you need these badly enough, use plperlu where there are no restrictions to overcome - the big problem is that 'use anything' requires that we enable the 'require' op, and that is certainly not going to happen without a great deal of thought. cheers andrew ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [PATCHES] [HACKERS] plperl Safe restrictions
I think it is *way* too late in the dev cycle to be proposing this. Maybe it should be a TODO item - I at least don't have time even to think about the implications os using these pragmas. The effect of the first is achievable via an environment setting, I believe. If you need these badly enough, use plperlu where there are no restrictions to overcome - the big problem is that 'use anything' requires that we enable the 'require' op, and that is certainly not going to happen without a great deal of thought. Fair enough, was just a suggestion as they seem obviously useful, even to the non-superuser plperl programmer. TODO item would suffice :) ... John ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
[PATCHES] minor restrpos / markpos optimization
This patch makes a minor optimization to the restrpos() and markpos()
implementations for btree and hash: to bump the reference count on a
buffer, it is more efficient to use IncrBufferRefCount() than to do a
ReadBuffer (for one thing we needn't acquire the BufMgrLock; we also
don't need to do a hash table lookup etc.)
Barring any objections I will apply to HEAD before end of day.
-Neil
#
# patch src/backend/access/hash/hash.c
# from [932df947ff5d32c4a90f7231fe30451a8a3ca21f]
#to [69c44f2b436d81702eccfd4010118ea9c14abbfd]
#
# patch src/backend/access/nbtree/nbtree.c
# from [a097527f8028f95b1e89eafa9fab335bc3e1e2d8]
#to [786fcf3dbe60d1fb0142aaee8b62b3422d5ee0a7]
#
--- src/backend/access/hash/hash.c
+++ src/backend/access/hash/hash.c
@@ -397,9 +397,8 @@
/* bump pin count on currentItemData and copy to currentMarkData */
if (ItemPointerIsValid((scan-currentItemData)))
{
- so-hashso_mrkbuf = _hash_getbuf(rel,
- BufferGetBlockNumber(so-hashso_curbuf),
- HASH_NOLOCK);
+ IncrBufferRefCount(so-hashso_curbuf);
+ so-hashso_mrkbuf = so-hashso_curbuf;
scan-currentMarkData = scan-currentItemData;
}
@@ -425,9 +424,8 @@
/* bump pin count on currentMarkData and copy to currentItemData */
if (ItemPointerIsValid((scan-currentMarkData)))
{
- so-hashso_curbuf = _hash_getbuf(rel,
- BufferGetBlockNumber(so-hashso_mrkbuf),
- HASH_NOLOCK);
+ IncrBufferRefCount(so-hashso_mrkbuf);
+ so-hashso_curbuf = so-hashso_mrkbuf;
scan-currentItemData = scan-currentMarkData;
}
--- src/backend/access/nbtree/nbtree.c
+++ src/backend/access/nbtree/nbtree.c
@@ -477,8 +477,8 @@
/* bump pin on current buffer for assignment to mark buffer */
if (ItemPointerIsValid((scan-currentItemData)))
{
- so-btso_mrkbuf = ReadBuffer(scan-indexRelation,
- BufferGetBlockNumber(so-btso_curbuf));
+ IncrBufferRefCount(so-btso_curbuf);
+ so-btso_mrkbuf = so-btso_curbuf;
scan-currentMarkData = scan-currentItemData;
so-mrkHeapIptr = so-curHeapIptr;
}
@@ -509,8 +509,8 @@
/* bump pin on marked buffer */
if (ItemPointerIsValid((scan-currentMarkData)))
{
- so-btso_curbuf = ReadBuffer(scan-indexRelation,
- BufferGetBlockNumber(so-btso_mrkbuf));
+ IncrBufferRefCount(so-btso_mrkbuf);
+ so-btso_curbuf = so-btso_mrkbuf;
scan-currentItemData = scan-currentMarkData;
so-curHeapIptr = so-mrkHeapIptr;
}
---(end of broadcast)---
TIP 9: the planner will ignore your desire to choose an index scan if your
joining column's datatypes do not match
[PATCHES] docs patch - CSV import limitation
That attached patch updates the docs for COPY CSV to include the recently discussed limitation on importing data with embedded line-end characters. Maybe we also need a TODO to revisit this during the 8.1 cycle, but I don't believe we should do more than this for now. cheers andrew Index: doc/src/sgml/ref/copy.sgml === RCS file: /home/cvsmirror/pgsql/doc/src/sgml/ref/copy.sgml,v retrieving revision 1.58 diff -c -r1.58 copy.sgml *** doc/src/sgml/ref/copy.sgml 15 Nov 2004 06:32:15 - 1.58 --- doc/src/sgml/ref/copy.sgml 17 Nov 2004 00:39:05 - *** *** 3,8 --- 3,9 PostgreSQL documentation -- + refentry id=SQL-COPY refmeta refentrytitle id=sql-copy-titleCOPY/refentrytitle *** *** 500,506 CSV mode will both recognize and produce CSV files with quoted values containing embedded carriage returns and line feeds. Thus the files are not strictly one line per table row like text-mode ! files. /para /note --- 501,511 CSV mode will both recognize and produce CSV files with quoted values containing embedded carriage returns and line feeds. Thus the files are not strictly one line per table row like text-mode ! files. emphasisPostgreSQL will encounter an error if it reads a ! line end character sequence embedded in a field that does not match ! those used in the CSV file itself. It is generally safer to import ! data containing embedded line end characters using TEXT or BINARY ! rather than CSV format./emphasis /para /note ---(end of broadcast)--- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [PATCHES] Patch pg_start_backup() to ERROR if archiving is not
On Sun, 2004-11-14 at 23:05 +1100, Gavin Sherry wrote: Attached is a patch which checks early on in pg_start_back() that archive_command is defined and if not generates an error. Applied, with some tweaks. Thanks. -Neil ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [PATCHES] docs patch - CSV import limitation
On Tue, 2004-11-16 at 19:46 -0500, Andrew Dunstan wrote: That attached patch updates the docs for COPY CSV to include the recently discussed limitation on importing data with embedded line-end characters. Applied, with changes: I thought emphasis was overkill, and IMHO the wording of the addition was needlessly obfuscated so I rewrote some of it. -Neil ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [PATCHES] minor restrpos / markpos optimization
On Wed, 2004-11-17 at 11:20 +1100, Neil Conway wrote: This patch makes a minor optimization to the restrpos() and markpos() implementations for btree and hash: to bump the reference count on a buffer, it is more efficient to use IncrBufferRefCount() than to do a ReadBuffer Applied. -Neil ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faqs/FAQ.html
