Re: [PATCHES] SRF memory leaks
On Mon, 2008-02-25 at 21:00 -0500, Tom Lane wrote: I find this part of the patch to be a seriously bad idea. nodeFunctionscan has no right to assume that the function has returned an expendable tupdesc; indeed, I would think that the other case is more nearly what's expected by the API for SRFs. AFAICS this is not true of any of the SRFs in the backend, which always return expendable tupdescs. A safer fix would be to try to make the tupdesc be in the new multi_call_ctx when it's being created by the funcapi.c code. funcapi.c doesn't create the tupdesc, though: it is created by user code, and merely assigned to a field of the RSI (note that the TupleDesc in question is the one in the ReturnSetInfo, not in FuncCallContext.) A minor detail is that the lifetime of the tupdesc also needs to exceed the lifetime of the multi_call_ctx, at least as currently implemented. From looking at the existing SRFs in the backend, the TupleDesc is always *explicitly* allocated in the estate's per-query context, so it will leak unless freed. Perhaps it would be sufficient to FreeTupleDesc iff tupdesc-refcount == -1? BTW, I'm thinking of changing the various SRFs that make allocations in the EState's per-query context to instead use the SRF's multi_call_ctx. This means the memory will be reclaimed sooner and avoids possible memory leaks. -Neil ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [PATCHES] SRF memory leaks
Neil Conway [EMAIL PROTECTED] writes: On Mon, 2008-02-25 at 21:00 -0500, Tom Lane wrote: I find this part of the patch to be a seriously bad idea. AFAICS this is not true of any of the SRFs in the backend, which always return expendable tupdescs. It's OK in the built-in SRFs is disastrously different from It's OK. It was never specified that SRFs had to return a free-able tupdesc, so I think it's a lead pipe cinch that there are some out there that don't. Nor would it be their fault if we change the specification. regards, tom lane ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [PATCHES] SRF memory leaks
On Tue, 2008-02-26 at 03:13 -0500, Tom Lane wrote: It's OK in the built-in SRFs is disastrously different from It's OK. Right, I never said that, I was just commenting on your view that the predominant use-case for SRFs is returning refcounted tupdescs. You didn't comment on my proposed solution (FreeTupleDesc() iff refcount == -1). ISTM that we *need* to free the TupleDesc in at least some cases, in order to defend against the practice of explicitly allocating the TupleDesc in the per-query context. -Neil ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [PATCHES] lc_time and localized dates
Euler Taveira de Oliveira napsal(a): Zdenek Kotala wrote: Please, Day names does not have capitalized first letter in Czech language. We have pondeli as a Monday. If locale does not do that it is probably intention :-). Hmmm... I don't know about that. I do it that way 'cause I'm concerned about some locales that don't capitalize (see above). In my head, (i) 'TMMonth' is February, Fevereiro, Febrero, Únor; (ii) 'TMmonth' is february, fevereiro, febrero, únor; (iii) 'TMMONTH' is FEBRUARY, FEVEREIRO, FEBRERO, ÚNOR. How would we handle that case? Is it wrong to write Únor? Yes it is. Only if it is a first word in a sentence or name you should use Ú. Also name of day is 'pondělí' (Monday) with small p If I read description of Month pattern, it says full mixed-case month name (blank-padded to 9 chars). By my opinion this patter should keep standard locale behavior - it means for English February, for Czech únor. Zdenek ---(end of broadcast)--- TIP 7: You can help support the PostgreSQL project by donating at http://www.postgresql.org/about/donate
Re: [PATCHES] Fix pgstatindex using for large indexes
Tom Lane napsal(a): Florian G. Pflug [EMAIL PROTECTED] writes: Maybe we should just bite the bullet, and implement int64 emulation for platforms that don't provide one? Why? Workarounds such as use double where needed have served us perfectly fine so far, with far less effort and notational ugliness than this would involve. Disadvantage of double usage is in CPUs. Current CPUs are perfectly optimized for int operation but there are limitation for floating point operatim. For example T1 (niagara) has only one FP unit per whole CPU (T2 has one FP per core) and AMD64 has three parallel execution unit for microcode but only one for FP and FP shares registry with MMX unit. And operation system has more work with saving FP register and so on... Not all these limitations are related to PostgreSQL but it is good to know. By my opinion avoid double is important in critical(bottleneck) places. How you mentioned double is OK for pgstattuple. There will come a time where either there's a really good reason to rely on int64, or we feel that it's moot because any platform without int64 is certainly dead anyway. I'm not sure how far off that time is, but it's probably some fairly small number of years. I don't think support more than five years older platform makes sense for new version of PostgreSQL. Very often users buy a new hardware for new database. IIRC all platform (x86, SPARC, MIPS, ALPHA, PARISC ...) do not have problem with 64bit for longer time. Zdenek ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [PATCHES] lc_time and localized dates
Magnus, Please look at this patch before you check my last patch about the locale. It seems that Euler's solution also fixes the windows locale bug that I was trying to fix. Replacing the lc_messages with lc_time when using to_char seems to be the correct direction. I have compiled and tested Euler's patch on MSVC and got the correct results. Euler, I think your patch is correct for the most part except you should not force the first letter of day/month names to uppercase hence this is not grammatically correct for some languages. After a quick glimpse of your cache mechanism, I think we can use this to solve some other TO_CHAR/TO_DATE items. Regards, Gevik. Resuts: Gevik=# set lc_time to 'Dutch, Netherlands'; SET Gevik=# select to_char(now() + '8 months'::interval, 'TMDay Day dy, DD TMMon Month TMmonth '); to_char - Zondag Sundaysun, 26 Okt October oktober 2008 (1 row) Gevik=# -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Euler Taveira de Oliveira Sent: Monday, February 25, 2008 5:53 AM To: PostgreSQL-patches Subject: [PATCHES] lc_time and localized dates Hi, Attached is a patch that replaces the lc_messages with lc_time when using to_char in translation mode (TM) [1]. It doesn't change the output behaviour. Per discussion [2], it's using some cache mechanism so we don't need to call setlocale() all the time. Some issues: (i) some locales don't capitalize the first letter. I'm using pg_toupper() to do the job but I'm afraid it's not appropriated. I'm using it 'cause i'm too lazy to move localized_str_toupper(). Any suggestions? (ii) it didn't address the problem spotted at [3][4]. IMHO, it's ok for a non-superuser to set lc_time. Opinions? euler=# show lc_time; lc_time - pt_BR (1 registro) euler=# select to_char(now(), 'TMDay Day dy, DD TMMon Month TMmonth '); to_char Segunda Mondaymon, 25 Fev February fevereiro 2008 (1 registro) euler=# set lc_time to 'es_ES'; SET euler=# select to_char(now(), 'TMDay Day dy, DD TMMon Month TMmonth '); to_char Lunes Mondaymon, 25 Feb February febrero 2008 (1 registro) euler=# set lc_time to 'de_DE'; SET euler=# select to_char(now(), 'TMDay Day dy, DD TMMon Month TMmonth '); to_char - Montag Mondaymon, 25 Feb February februar 2008 (1 registro) euler=# set lc_time to 'fr_FR'; SET euler=# select to_char(now(), 'TMDay Day dy, DD TMMon Month TMmonth '); to_char Lundi Mondaymon, 25 Fév February février 2008 (1 registro) euler=# set lc_time to 'C'; SET euler=# select to_char(now(), 'TMDay Day dy, DD TMMon Month TMmonth '); to_char -- Monday Mondaymon, 25 Feb February february 2008 (1 registro) [1] http://archives.postgresql.org/pgsql-hackers/2006-11/msg00539.php [2] http://archives.postgresql.org/pgsql-hackers/2006-11/msg00693.php [3] http://archives.postgresql.org/pgsql-hackers/2007-10/msg00214.php [4] http://archives.postgresql.org/pgsql-hackers/2006-11/msg00692.php -- Euler Taveira de Oliveira http://www.timbira.com/ l10ntochar.diff.gz Description: GNU Zip compressed data ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] [PATCHES] 2WRS [WIP]
For the joy of all of you: that's the correct WIP patch. At the moment it only tries to create runs uding two heaps. Hope you can help me with writing those runs on tapes. I'd be very pleased to give you more details. Thenks for your time. Regards, Manolo. -- From: Jaime Casanova [EMAIL PROTECTED] Sent: Friday, February 22, 2008 5:30 AM To: [EMAIL PROTECTED] Cc: Decibel! [EMAIL PROTECTED]; Manolo _ [EMAIL PROTECTED]; David Fetter [EMAIL PROTECTED]; pgsql-patches@postgresql.org; [EMAIL PROTECTED] Subject: Re: [HACKERS] [PATCHES] 2WRS [WIP] On Thu, Feb 21, 2008 at 6:44 AM, [EMAIL PROTECTED] wrote: Hi. That's the last release and refers to 8.3.0 and not to 8.2.5 as before. Hope you can tell me if I created it correctly please. no, it doesn't... ! /* GUC variables */ #ifdef TRACE_SORT bool trace_sort = false; #endif - #ifdef DEBUG_BOUNDED_SORT - bool optimize_bounded_sort = true; - #endif it's seems you're removing something added in 8.3 -- regards, Jaime Casanova Programming today is a race between software engineers striving to build bigger and better idiot-proof programs and the universe trying to produce bigger and better idiots. So far, the universe is winning. Richard Cook ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match tuplesort.patch Description: Binary data ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [PATCHES] DTrace not working after SUBSYS.o reorg
Magne Mæhre wrote: I noticed that the dtrace code wouldn't build after the recent SUBSYS.o reorganization. Attached is a small patch that expands the files.. --Magne Oooops... wrong patch file The correct one is attached Sorry --Magne Index: src/backend/Makefile === RCS file: /projects/cvsroot/pgsql/src/backend/Makefile,v retrieving revision 1.126 diff -u -r1.126 Makefile --- src/backend/Makefile 25 Feb 2008 17:55:42 - 1.126 +++ src/backend/Makefile 26 Feb 2008 13:42:45 - @@ -137,7 +137,8 @@ utils/probes.o: utils/probes.d $(SUBDIROBJS) - $(DTRACE) $(DTRACEFLAGS) -G -s $^ -o $@ + $(DTRACE) $(DTRACEFLAGS) -G -s $(srcdir)/utils/probes.d -o $@ $(call expand_subsys,$(SUBDIROBJS)) + ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
[PATCHES] Bulk Insert tuning
Following patch implements a simple mechanism to keep a buffer pinned
while we are bulk loading.
Performance gains measured as +20% gain for CREATE TABLE as SELECT, and
15-17% for COPY on very short rows. Measurable difference drops away and
is not measurable at all at 1000/bytes per row.
As a result, patch doesn't bother to implement buffer pinning for TOAST
operations.
Can I ask for some independent performance results please?
--
Simon Riggs
2ndQuadrant http://www.2ndQuadrant.com
Index: src/backend/access/heap/heapam.c
===
RCS file: /home/sriggs/pg/REPOSITORY/pgsql/src/backend/access/heap/heapam.c,v
retrieving revision 1.249
diff -c -r1.249 heapam.c
*** src/backend/access/heap/heapam.c 30 Jan 2008 18:35:55 - 1.249
--- src/backend/access/heap/heapam.c 26 Feb 2008 14:29:35 -
***
*** 1732,1737
--- 1732,1752
}
}
+ /*
+ * Begin/End Bulk Inserts
+ *
+ */
+ void
+ heap_begin_bulk_insert(void)
+ {
+ ReleaseBulkInsertBufferIfAny();
+ }
+
+ void
+ heap_end_bulk_insert(void)
+ {
+ ReleaseBulkInsertBufferIfAny();
+ }
/*
* heap_insert - insert tuple into a heap
***
*** 1759,1769
*/
Oid
heap_insert(Relation relation, HeapTuple tup, CommandId cid,
! bool use_wal, bool use_fsm)
{
TransactionId xid = GetCurrentTransactionId();
HeapTuple heaptup;
Buffer buffer;
if (relation-rd_rel-relhasoids)
{
--- 1774,1785
*/
Oid
heap_insert(Relation relation, HeapTuple tup, CommandId cid,
! bool use_wal, bool use_fsm, bool bulk_insert_request)
{
TransactionId xid = GetCurrentTransactionId();
HeapTuple heaptup;
Buffer buffer;
+ bool bulk_insert = bulk_insert_request !relation-rd_istemp;
if (relation-rd_rel-relhasoids)
{
***
*** 1816,1824
else
heaptup = tup;
! /* Find buffer to insert this tuple into */
! buffer = RelationGetBufferForTuple(relation, heaptup-t_len,
! InvalidBuffer, use_fsm);
/* NO EREPORT(ERROR) from here till changes are logged */
START_CRIT_SECTION();
--- 1832,1849
else
heaptup = tup;
! /*
! * Find buffer to insert this tuple into
! */
! if (bulk_insert)
! {
! buffer = RelationGetBufferForTuple(relation, heaptup-t_len,
! GetBulkInsertBuffer(), use_fsm, true);
! SetBulkInsertBuffer(buffer);
! }
! else
! buffer = RelationGetBufferForTuple(relation, heaptup-t_len,
! InvalidBuffer, use_fsm, false);
/* NO EREPORT(ERROR) from here till changes are logged */
START_CRIT_SECTION();
***
*** 1897,1903
END_CRIT_SECTION();
! UnlockReleaseBuffer(buffer);
/*
* If tuple is cachable, mark it for invalidation from the caches in case
--- 1922,1934
END_CRIT_SECTION();
! /*
! * Keep buffer pinned if we are in bulk insert mode
! */
! if (bulk_insert)
! LockBuffer(buffer, BUFFER_LOCK_UNLOCK);
! else
! UnlockReleaseBuffer(buffer);
/*
* If tuple is cachable, mark it for invalidation from the caches in case
***
*** 1934,1940
Oid
simple_heap_insert(Relation relation, HeapTuple tup)
{
! return heap_insert(relation, tup, GetCurrentCommandId(true), true, true);
}
/*
--- 1965,1971
Oid
simple_heap_insert(Relation relation, HeapTuple tup)
{
! return heap_insert(relation, tup, GetCurrentCommandId(true), true, true, false);
}
/*
***
*** 2557,2563
{
/* Assume there's no chance to put heaptup on same page. */
newbuf = RelationGetBufferForTuple(relation, heaptup-t_len,
! buffer, true);
}
else
{
--- 2588,2594
{
/* Assume there's no chance to put heaptup on same page. */
newbuf = RelationGetBufferForTuple(relation, heaptup-t_len,
! buffer, true, false);
}
else
{
***
*** 2574,2580
*/
LockBuffer(buffer, BUFFER_LOCK_UNLOCK);
newbuf = RelationGetBufferForTuple(relation, heaptup-t_len,
! buffer, true);
}
else
{
--- 2605,2611
*/
LockBuffer(buffer, BUFFER_LOCK_UNLOCK);
newbuf = RelationGetBufferForTuple(relation, heaptup-t_len,
! buffer, true, false);
}
else
{
Index: src/backend/access/heap/hio.c
===
RCS file: /home/sriggs/pg/REPOSITORY/pgsql/src/backend/access/heap/hio.c,v
retrieving revision 1.68
diff -c -r1.68 hio.c
*** src/backend/access/heap/hio.c 1 Jan 2008 19:45:46 - 1.68
--- src/backend/access/heap/hio.c 26 Feb 2008 14:29:35 -
***
*** 103,109
*/
Buffer
RelationGetBufferForTuple(Relation relation, Size len,
! Buffer otherBuffer, bool use_fsm)
{
Buffer buffer = InvalidBuffer;
Page pageHeader;
--- 103,109
*/
Buffer
RelationGetBufferForTuple(Relation relation, Size len,
!
Re: [PATCHES] DTrace not working after SUBSYS.o reorg
Am Dienstag, 26. Februar 2008 schrieb Magne Mæhre: I noticed that the dtrace code wouldn't build after the recent SUBSYS.o reorganization. Fixed. Is there no buildfarm coverage of dtrace? -- Peter Eisentraut http://developer.postgresql.org/~petere/ ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] [PATCHES] 2WRS [WIP]
For the joy of all of you: that's the correct WIP patch. At the moment it only tries to create runs uding two heaps. Hope you can help me with writing those runs on tapes. I'd be very pleased to give you more details. Thenks for your time. Regards, Manolo. -- From: Jaime Casanova [EMAIL PROTECTED] Sent: Friday, February 22, 2008 5:30 AM To: [EMAIL PROTECTED] Cc: Decibel! [EMAIL PROTECTED]; Manolo _ [EMAIL PROTECTED]; David Fetter [EMAIL PROTECTED]; pgsql-patches@postgresql.org; [EMAIL PROTECTED] Subject: Re: [HACKERS] [PATCHES] 2WRS [WIP] On Thu, Feb 21, 2008 at 6:44 AM, [EMAIL PROTECTED] wrote: Hi. That's the last release and refers to 8.3.0 and not to 8.2.5 as before. Hope you can tell me if I created it correctly please. no, it doesn't... ! /* GUC variables */ #ifdef TRACE_SORT bool trace_sort = false; #endif - #ifdef DEBUG_BOUNDED_SORT - bool optimize_bounded_sort = true; - #endif it's seems you're removing something added in 8.3 -- regards, Jaime Casanova Programming today is a race between software engineers striving to build bigger and better idiot-proof programs and the universe trying to produce bigger and better idiots. So far, the universe is winning. Richard Cook ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match tuplesort.patch Description: Binary data ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [PATCHES] lc_time and localized dates
Gevik Babakhani wrote: Have you tested this patch on MSVC and MinGW (Windows) builds? changing LC_MESSAGES/LC_TIME will most probably break the Windows behavior. No. [Looking at the code...] I think it only affects the LC_MESSAGES 'cause setlocale(LC_MESSAGES) don't work on Windows. AFAIK, some locales like the Dutch doesn't have capitalized first letters for month/day. Making the first letter capitalized for all values would be incorrect. Even if we have just the month word? Don't consider a sentence in this case. We're talking about a formatting function so you can choose if you want it capitalized or not. Is it ok to output TMDay as 'Dinsdag' ? -- Euler Taveira de Oliveira http://www.timbira.com/ ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [PATCHES] lc_time and localized dates
Zdenek Kotala wrote: Yes it is. Only if it is a first word in a sentence or name you should use Ú. Also name of day is 'pondělí' (Monday) with small p But we're not talking about *sentence*, we need to consider just the *word*. So I think TMMonth should be Xxxx, TMMONTH should be , and TMmonth should be . If you want to ensure lowercase in months, you could use TMmonth instead of TMMonth. -- Euler Taveira de Oliveira http://www.timbira.com/ ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [PATCHES] lc_time and localized dates
No. [Looking at the code...] I think it only affects the LC_MESSAGES 'cause setlocale(LC_MESSAGES) don't work on Windows. In order to make setlocale(LC_MESSAGES) affect on windows some additional steps must be taken. I don't go deep in that now. I have a fix with description etc. etc. Is it ok to output TMDay as 'Dinsdag' ? No. you don't write dinsdag with a D in Dutch. Looking into code for a couple of days now and the recent discussions, I must say that there is more broken in locale to_char/to_date (Windows) than we can see at first glance. I am going to report my finding a.s.a.p. Then we can discuss about a fix. Regards, Gevik ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [PATCHES] lc_time and localized dates
Euler Taveira de Oliveira napsal(a): Zdenek Kotala wrote: Yes it is. Only if it is a first word in a sentence or name you should use Ú. Also name of day is 'pondělí' (Monday) with small p But we're not talking about *sentence*, we need to consider just the *word*. So I think TMMonth should be Xxxx, TMMONTH should be , and TMmonth should be . If you want to ensure lowercase in months, you could use TMmonth instead of TMMonth. But how you handle situation when you have multi language application which needs correct output for all languages? You cannot use any of the pattern because it will be wrong for some group of languages. Zdenek ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [PATCHES] lc_time and localized dates
Am Dienstag, 26. Februar 2008 schrieb Zdenek Kotala: But how you handle situation when you have multi language application which needs correct output for all languages? You cannot use any of the pattern because it will be wrong for some group of languages. This is what you get when you copy a proprietary, poorly specified interface. The to_char functions are supposed to be Oracle-compatible, so we need to check what Oracle does. Whether that is useful in practice is a bit secondary. I'm beginning to think, if you want formatting functions that are more sane, stable, and standardized, export sprintf and strftime to PostgreSQL. -- Peter Eisentraut http://developer.postgresql.org/~petere/ ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [PATCHES] lc_time and localized dates
Zdenek Kotala wrote: But how you handle situation when you have multi language application which needs correct output for all languages? You cannot use any of the pattern because it will be wrong for some group of languages. FYI, strftime() [1] doesn't say anything about capitalization. I don't know if some translators are aware of it and even if they are, how would they know that a day or month is the first word of a sentence? IMHO we should provide X, x, and X so the programmer can use whatever he/she thinks is correct for him/her. [1] http://www.opengroup.org/onlinepubs/009695399/functions/strftime.html -- Euler Taveira de Oliveira http://www.timbira.com/ ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [PATCHES] lc_time and localized dates
This is what you get when you copy a proprietary, poorly specified interface. The to_char functions are supposed to be Oracle-compatible, so we need to check what Oracle does. Whether that is useful in practice is a bit secondary. I'm beginning to think, if you want formatting functions that are more sane, stable, and standardized, export sprintf and strftime to PostgreSQL. Perhaps this is good time to think about (I'm being careful to say redesign) a review both regarding date/time formatting and locale handling for these functions in general. Regards, Gevik ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [PATCHES] lc_time and localized dates
But as Peter remarks nearby, this discussion is wasted anyway, because there is only one correct answer: whatever Oracle does with these cases is what to_char() should do. ++1 ---(end of broadcast)--- TIP 7: You can help support the PostgreSQL project by donating at http://www.postgresql.org/about/donate
Re: [PATCHES] lc_time and localized dates
Euler Taveira de Oliveira [EMAIL PROTECTED] writes: FYI, strftime() [1] doesn't say anything about capitalization. I don't know if some translators are aware of it and even if they are, how would they know that a day or month is the first word of a sentence? IMHO we should provide X, x, and X so the programmer can use whatever he/she thinks is correct for him/her. I think you've missed the point, which is that the programmer can't possibly know in advance which capitalization is correct, if she's trying to build a system that works for different localizations. But as Peter remarks nearby, this discussion is wasted anyway, because there is only one correct answer: whatever Oracle does with these cases is what to_char() should do. regards, tom lane ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [PATCHES] Bulk Insert tuning
On Tue, 2008-02-26 at 14:43 +, Simon Riggs wrote: Following patch implements a simple mechanism to keep a buffer pinned while we are bulk loading. Performance gains measured as +20% gain for CREATE TABLE as SELECT, and 15-17% for COPY on very short rows. Measurable difference drops away and is not measurable at all at 1000/bytes per row. As a result, patch doesn't bother to implement buffer pinning for TOAST operations. Short guide for reviewers: backend/access/heap/heapam.c | 47 +! backend/access/heap/hio.c| 32 ++-!! backend/access/heap/tuptoaster.c | 2 ! backend/access/transam/xact.c| 1 backend/commands/copy.c | 9 +!! backend/executor/execMain.c | 9 !!! backend/storage/buffer/bufmgr.c | 32 ++ include/access/heapam.h | 4 +!! include/access/hio.h | 2 ! include/storage/bufmgr.h | 5 10 files changed, 66 insertions(+), 1 deletion(-), 76 modifications(!) New heap API calls in heapam.c, caled from copy.c and execMain.c Modified API to HeapGetBufferForTuple() in hio.c, causes changes elsewhere in that file and tuptoaster.c New buffer manager API to manage pinned buffer in bufmgr.c Transaction cleanup on abort in xact.c -- Simon Riggs 2ndQuadrant http://www.2ndQuadrant.com ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [PATCHES] SRF memory leaks
On Tue, 2008-02-26 at 00:17 -0800, Neil Conway wrote:
You didn't comment on my proposed solution (FreeTupleDesc() iff refcount
== -1).
Attached is a revised version of this patch. It makes the
FreeTupleDesc() change described above, and fixes a bug: in
SRF_RETURN_DONE(), we need to be sure to switch back to the caller's
context before deleting the multi_call_ctx, since some SRFs (e.g.
dblink) call SRF_RETURN_DONE() while still inside the multi_call_ctx.
I'd like to apply this change to back branches reasonably soon, so if
you have a better way to do the FreeTupleDesc() hack, let me know.
-Neil
Index: src/backend/executor/nodeFunctionscan.c
===
RCS file: /home/neilc/postgres/cvs_root/pgsql/src/backend/executor/nodeFunctionscan.c,v
retrieving revision 1.45
diff -p -c -r1.45 nodeFunctionscan.c
*** src/backend/executor/nodeFunctionscan.c 1 Jan 2008 19:45:49 - 1.45
--- src/backend/executor/nodeFunctionscan.c 26 Feb 2008 19:04:49 -
*** FunctionNext(FunctionScanState *node)
*** 77,83
--- 77,93
* do it always.
*/
if (funcTupdesc)
+ {
tupledesc_match(node-tupdesc, funcTupdesc);
+
+ /*
+ * If it is a dynamically-allocated TupleDesc, free it: it is
+ * typically allocated in the EState's per-query context, so we
+ * must avoid leaking it on rescan.
+ */
+ if (funcTupdesc-tdrefcount == -1)
+ FreeTupleDesc(funcTupdesc);
+ }
}
/*
Index: src/backend/utils/fmgr/funcapi.c
===
RCS file: /home/neilc/postgres/cvs_root/pgsql/src/backend/utils/fmgr/funcapi.c,v
retrieving revision 1.37
diff -p -c -r1.37 funcapi.c
*** src/backend/utils/fmgr/funcapi.c 1 Jan 2008 19:45:53 - 1.37
--- src/backend/utils/fmgr/funcapi.c 26 Feb 2008 19:32:47 -
***
*** 23,28
--- 23,29
#include utils/array.h
#include utils/builtins.h
#include utils/lsyscache.h
+ #include utils/memutils.h
#include utils/syscache.h
#include utils/typcache.h
*** init_MultiFuncCall(PG_FUNCTION_ARGS)
*** 63,75
/*
* First call
*/
! ReturnSetInfo *rsi = (ReturnSetInfo *) fcinfo-resultinfo;
/*
* Allocate suitably long-lived space and zero it
*/
retval = (FuncCallContext *)
! MemoryContextAllocZero(fcinfo-flinfo-fn_mcxt,
sizeof(FuncCallContext));
/*
--- 64,86
/*
* First call
*/
! ReturnSetInfo *rsi = (ReturnSetInfo *) fcinfo-resultinfo;
! MemoryContext multi_call_ctx;
!
! /*
! * Create a suitably long-lived context to hold cross-call data
! */
! multi_call_ctx = AllocSetContextCreate(fcinfo-flinfo-fn_mcxt,
! SRF multi-call context,
! ALLOCSET_SMALL_MINSIZE,
! ALLOCSET_SMALL_INITSIZE,
! ALLOCSET_SMALL_MAXSIZE);
/*
* Allocate suitably long-lived space and zero it
*/
retval = (FuncCallContext *)
! MemoryContextAllocZero(multi_call_ctx,
sizeof(FuncCallContext));
/*
*** init_MultiFuncCall(PG_FUNCTION_ARGS)
*** 81,87
retval-user_fctx = NULL;
retval-attinmeta = NULL;
retval-tuple_desc = NULL;
! retval-multi_call_memory_ctx = fcinfo-flinfo-fn_mcxt;
/*
* save the pointer for cross-call use
--- 92,98
retval-user_fctx = NULL;
retval-attinmeta = NULL;
retval-tuple_desc = NULL;
! retval-multi_call_memory_ctx = multi_call_ctx;
/*
* save the pointer for cross-call use
*** shutdown_MultiFuncCall(Datum arg)
*** 168,180
flinfo-fn_extra = NULL;
/*
! * Caller is responsible to free up memory for individual struct elements
! * other than att_in_funcinfo and elements.
*/
! if (funcctx-attinmeta != NULL)
! pfree(funcctx-attinmeta);
!
! pfree(funcctx);
}
--- 179,189
flinfo-fn_extra = NULL;
/*
! * Delete context that holds all multi-call data, including the
! * FuncCallContext itself
*/
! MemoryContextSwitchTo(flinfo-fn_mcxt);
! MemoryContextDelete(funcctx-multi_call_memory_ctx);
}
---(end of broadcast)---
TIP 1: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to [EMAIL PROTECTED] so that your
message can get through to the mailing list cleanly
Re: [PATCHES] Bulk Insert tuning
Simon Riggs [EMAIL PROTECTED] writes: Following patch implements a simple mechanism to keep a buffer pinned while we are bulk loading. This will fail to clean up nicely after a subtransaction abort, no? (For that matter I don't think it's right even for a top-level abort.) And I'm pretty sure it will trash your table entirely if someone inserts into another relation while a bulk insert is happening. (Not at all impossible, think of triggers for instance.) From a code structural point of view, we are already well past the number of distinct options that heap_insert ought to have. I was thinking the other day that bulk inserts ought to use a ring-buffer strategy to avoid having COPY IN trash the whole buffer arena, just as we've taught COPY OUT not to. So maybe a better idea is to generalize BufferAccessStrategy to be able to handle write as well as read concerns; or have two versions of it, one for writing and one for reading. In any case the point being to encapsulate all these random little options in a struct, which could also carry along state that needs to be saved across a series of inserts, such as the last pinned buffer. regards, tom lane ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [PATCHES] lc_time and localized dates
Tom Lane wrote: But as Peter remarks nearby, this discussion is wasted anyway, because there is only one correct answer: whatever Oracle does with these cases is what to_char() should do. My patch does exactly what Oracle does besides one thing: my code does not contain a real capitalization function. It sucks when we have composite names like the portuguese above. I'll post an updated version later. SQL SELECT TO_CHAR(SYSDATE, 'DY dy DAY Day day MONTH Month MON Mon mon', 'NLS_DATE_LANGUAGE = czech') FROM dual; TO_CHAR(SYSDATE,'DYDYDAYDAYDAYMONTHMONTHMONMONMON','NLS_DAT --- UT ut UTERY Utery utery UNOR Unor UNO Uno uno SQL SELECT TO_CHAR(SYSDATE, 'DY dy DAY Day day MONTH Month MON Mon mon', 'NLS_DATE_LANGUAGE = dutch') FROM dual; TO_CHAR(SYSDATE,'DYDYDAYDAYDAYMONTHMONTHMONMONMON','NLS_DATE_LANGUA --- DI di DINSDAG Dinsdag dinsdag FEBRUARI Februari FEB Feb feb SQL SELECT TO_CHAR(SYSDATE, 'DY dy DAY Day day MONTH Month MON Mon mon', 'NLS_DATE_LANGUAGE = portuguese') FROM dual; TO_CHAR(SYSDATE,'DYDYDAYDAYDAYMONTHMONTHMONMONMON','NLS_DATE_LANGUAGE=PORTUGUESE' - TER ter TERCA-FEIRA Terca-Feira terca-feira FEVEREIRO Fevereiro FEV Fev fev -- Euler Taveira de Oliveira http://www.timbira.com/ ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
