[HACKERS] Operator family docs
http://developer.postgresql.org/pgdocs/postgres/catalog-pg-opfamily.html states that Operator families are described at length in Section 33.14. which does not seem to be the case. Did it get missed in a commit? Regards, Dave. ---(end of broadcast)--- TIP 7: You can help support the PostgreSQL project by donating at http://www.postgresql.org/about/donate
Re: [HACKERS] Request for review: tsearch2 patch
Sorry for delay, I was on holidays :)
Did you test patch on Windows platform?
Tatsuo Ishii wrote:
I have tested with local-enabled environment and found a bug. Included
is the new version of patches.
Teodor, Oleg, what do you think about these patches?
If ok, shall I commit to CVS head?
--
Tatsuo Ishii
SRA OSS, Inc. Japan
Hi,
Here are patches against tsearch2 with CVS head. Currently tsearch2
does not work with multibyte encoding which uses C locale. These
patches are intended to solve the problem by using PostgreSQL in-house
multibyte function instead of mbstowcs which does not work with C
locale. Also iswalpha etc. will not be called in case of C locale
since they are not working with it. Tested with the EUC_JP encoding
(should be working with any multibye encodings). Existing single byte
encodings should not be broken by the patches, I did not test though.
--
Tatsuo Ishii
SRA OSS, Inc. Japan
Index: ts_locale.c
===
RCS file: /cvsroot/pgsql/contrib/tsearch2/ts_locale.c,v
retrieving revision 1.7
diff -c -r1.7 ts_locale.c
*** ts_locale.c 20 Nov 2006 14:03:30 - 1.7
--- ts_locale.c 4 Jan 2007 12:16:00 -
***
*** 63,68
--- 63,101
return mbstowcs(to, from, len);
}
+
+ #else /* WIN32 */
+
+ size_t
+ char2wchar(wchar_t *to, const char *from, size_t len)
+ {
+ wchar_t *result;
+ size_t n;
+
+ if (to == NULL)
+ return 0;
+
+ if (lc_ctype_is_c())
+ {
+ /* allocate neccesary memory for to including NULL terminate
*/
+ result = (wchar_t *)palloc((len+1)*sizeof(wchar_t));
+
+ /* do the conversion */
+ n = (size_t)pg_mb2wchar_with_len(from, (pg_wchar *)result, len);
+ if (n 0)
+ {
+ /* store the result */
+ if (n len)
+ n = len;
+ memcpy(to, result, n*sizeof(wchar_t));
+ pfree(result);
+ *(to + n) = '\0';
+ }
+ return n;
+ }
+ return mbstowcs(to, from, len);
+ }
+
#endif /* WIN32 */
int
***
*** 70,75
--- 103,113
{
wchar_t character;
+ if (lc_ctype_is_c())
+ {
+ return isalpha(TOUCHAR(ptr));
+ }
+
char2wchar(character, ptr, 1);
return iswalpha((wint_t) character);
***
*** 80,85
--- 118,128
{
wchar_t character;
+ if (lc_ctype_is_c())
+ {
+ return isprint(TOUCHAR(ptr));
+ }
+
char2wchar(character, ptr, 1);
return iswprint((wint_t) character);
***
*** 126,132
if ( wlen 0 )
ereport(ERROR,
(errcode(ERRCODE_CHARACTER_NOT_IN_REPERTOIRE),
!errmsg(transalation failed from server
encoding to wchar_t)));
Assert(wlen=len);
wstr[wlen] = 0;
--- 169,175
if ( wlen 0 )
ereport(ERROR,
(errcode(ERRCODE_CHARACTER_NOT_IN_REPERTOIRE),
!errmsg(translation failed from server
encoding to wchar_t)));
Assert(wlen=len);
wstr[wlen] = 0;
***
*** 152,158
if ( wlen 0 )
ereport(ERROR,
(errcode(ERRCODE_CHARACTER_NOT_IN_REPERTOIRE),
!errmsg(transalation failed from wchar_t to
server encoding %d, errno)));
Assert(wlen=len);
out[wlen]='\0';
}
--- 195,201
if ( wlen 0 )
ereport(ERROR,
(errcode(ERRCODE_CHARACTER_NOT_IN_REPERTOIRE),
!errmsg(translation failed from wchar_t to
server encoding %d, errno)));
Assert(wlen=len);
out[wlen]='\0';
}
Index: ts_locale.h
===
RCS file: /cvsroot/pgsql/contrib/tsearch2/ts_locale.h,v
retrieving revision 1.7
diff -c -r1.7 ts_locale.h
*** ts_locale.h 4 Oct 2006 00:29:47 - 1.7
--- ts_locale.h 4 Jan 2007 12:16:00 -
***
*** 38,45
#else /* WIN32 */
/* correct mbstowcs */
- #define char2wchar mbstowcs
#define wchar2char wcstombs
#endif /* WIN32 */
#define t_isdigit(x) ( pg_mblen(x)==1 isdigit( TOUCHAR(x) ) )
--- 38,46
#else /* WIN32 */
/* correct mbstowcs */
#define wchar2char wcstombs
+ size_t
Re: [HACKERS] Request for review: tsearch2 patch
Sorry for delay, I was on holidays :)
Did you test patch on Windows platform?
No. I myself does not use Windows platform.
Do you have any concern on Windows regarding my patches?
--
Tatsuo Ishii
SRA OSS, Inc. Japan
Tatsuo Ishii wrote:
I have tested with local-enabled environment and found a bug. Included
is the new version of patches.
Teodor, Oleg, what do you think about these patches?
If ok, shall I commit to CVS head?
--
Tatsuo Ishii
SRA OSS, Inc. Japan
Hi,
Here are patches against tsearch2 with CVS head. Currently tsearch2
does not work with multibyte encoding which uses C locale. These
patches are intended to solve the problem by using PostgreSQL in-house
multibyte function instead of mbstowcs which does not work with C
locale. Also iswalpha etc. will not be called in case of C locale
since they are not working with it. Tested with the EUC_JP encoding
(should be working with any multibye encodings). Existing single byte
encodings should not be broken by the patches, I did not test though.
--
Tatsuo Ishii
SRA OSS, Inc. Japan
Index: ts_locale.c
===
RCS file: /cvsroot/pgsql/contrib/tsearch2/ts_locale.c,v
retrieving revision 1.7
diff -c -r1.7 ts_locale.c
*** ts_locale.c20 Nov 2006 14:03:30 - 1.7
--- ts_locale.c4 Jan 2007 12:16:00 -
***
*** 63,68
--- 63,101
return mbstowcs(to, from, len);
}
+
+ #else/* WIN32 */
+
+ size_t
+ char2wchar(wchar_t *to, const char *from, size_t len)
+ {
+ wchar_t *result;
+ size_t n;
+
+ if (to == NULL)
+ return 0;
+
+ if (lc_ctype_is_c())
+ {
+ /* allocate neccesary memory for to including NULL terminate
*/
+ result = (wchar_t *)palloc((len+1)*sizeof(wchar_t));
+
+ /* do the conversion */
+ n = (size_t)pg_mb2wchar_with_len(from, (pg_wchar *)result, len);
+ if (n 0)
+ {
+ /* store the result */
+ if (n len)
+ n = len;
+ memcpy(to, result, n*sizeof(wchar_t));
+ pfree(result);
+ *(to + n) = '\0';
+ }
+ return n;
+ }
+ return mbstowcs(to, from, len);
+ }
+
#endif /* WIN32 */
int
***
*** 70,75
--- 103,113
{
wchar_t character;
+ if (lc_ctype_is_c())
+ {
+ return isalpha(TOUCHAR(ptr));
+ }
+
char2wchar(character, ptr, 1);
return iswalpha((wint_t) character);
***
*** 80,85
--- 118,128
{
wchar_t character;
+ if (lc_ctype_is_c())
+ {
+ return isprint(TOUCHAR(ptr));
+ }
+
char2wchar(character, ptr, 1);
return iswprint((wint_t) character);
***
*** 126,132
if ( wlen 0 )
ereport(ERROR,
(errcode(ERRCODE_CHARACTER_NOT_IN_REPERTOIRE),
! errmsg(transalation failed from
server encoding to wchar_t)));
Assert(wlen=len);
wstr[wlen] = 0;
--- 169,175
if ( wlen 0 )
ereport(ERROR,
(errcode(ERRCODE_CHARACTER_NOT_IN_REPERTOIRE),
! errmsg(translation failed from server
encoding to wchar_t)));
Assert(wlen=len);
wstr[wlen] = 0;
***
*** 152,158
if ( wlen 0 )
ereport(ERROR,
(errcode(ERRCODE_CHARACTER_NOT_IN_REPERTOIRE),
! errmsg(transalation failed from
wchar_t to server encoding %d, errno)));
Assert(wlen=len);
out[wlen]='\0';
}
--- 195,201
if ( wlen 0 )
ereport(ERROR,
(errcode(ERRCODE_CHARACTER_NOT_IN_REPERTOIRE),
! errmsg(translation failed from
wchar_t to server encoding %d, errno)));
Assert(wlen=len);
out[wlen]='\0';
}
Index: ts_locale.h
===
RCS file: /cvsroot/pgsql/contrib/tsearch2/ts_locale.h,v
retrieving revision 1.7
diff -c -r1.7 ts_locale.h
*** ts_locale.h4 Oct 2006 00:29:47 - 1.7
--- ts_locale.h4 Jan 2007 12:16:00 -
***
*** 38,45
#else/* WIN32 */
/* correct mbstowcs */
- #define char2wchar mbstowcs
#define wchar2char wcstombs
#endif /* WIN32 */
Re: [HACKERS] Mark/Restore and avoiding RandomAccess sorts
Simon Riggs [EMAIL PROTECTED] writes: Merge Joins require us to potentially Mark and Restore positions in the tuples arriving from executor sub-nodes. I came across an old note to myself suggesting that we handle this by interposing a Materialize node, and then teaching Material that if it's told EXEC_FLAG_MARK but not EXEC_FLAG_REWIND or EXEC_FLAG_BACKWARD, it need keep data only as far back as the Mark position. So the structural requirements are mostly in place already, it's just a matter of figuring out a nice way to implement the drop older parts of the tuplestore business. regards, tom lane ---(end of broadcast)--- TIP 7: You can help support the PostgreSQL project by donating at http://www.postgresql.org/about/donate
Re: [HACKERS] [PATCHES] COPY with no WAL, in certain circumstances
On Tue, 2007-01-09 at 16:31 -0500, Tom Lane wrote: Simon Riggs [EMAIL PROTECTED] writes: ...continuing this discussion about setting HEAP_XMIN_COMMITTED... BTW, a sufficient counterexample for that kluge is that neither SPI or SQL-function execution use a separate portal for invoked commands. What would the best/acceptable way be to test for this condition? My opinion is that the only reliable answer would be to find a way not to have to test. Tuples entered by your own transaction are normally considered good by tqual.c anyway, and thus I think we might be pretty close to having it Just Work, but you'd have to go through all the cases in tqual.c and see if any don't work. I agree we could get this to Just Work by altering HeapTupleSatisfies...() functions so that their first test is if (TransactionIdIsCurrentTransactionId(xvac)) rather then if (!(tuple-t_infomask HEAP_XMIN_COMMITTED)) I had ruled that out, unconsciously prefering the localised check in COPY, but I agree that the test was too complex. Taking this direct approach does have a lot of promise: Looks like HeapTupleSatisfiesSnapshot() currently does 4 if tests to check that an undeleted row is visible, and all other paths do much more work. Increasing the number of checks to 5 might not hurt that much. The branch prediction would work well for it, since when you are the CurrentTransactionId the test would pass 100% and when you're not the branch would fail 100% of the time, so the CPU would react to it positively I think. I'll run some tests and see if there's a noticeable difference. The other point is that to make such an optimization you have to consider the subtransaction history. For WAL you only have to know that the table will disappear if the top-level transaction fails, but to pre-set commit bits requires being sure that the table will disappear if the current subxact fails --- not the same thing at all. Right, you reminded me of that on the other part of the thread. It seems straightforward to put a test into COPY that the optimization will only work if you're in the Xid that made the relfilenode change. -- Simon Riggs EnterpriseDB http://www.enterprisedb.com ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [HACKERS] [PATCHES] COPY with no WAL, in certain circumstances
Simon Riggs [EMAIL PROTECTED] writes:
I agree we could get this to Just Work by altering
HeapTupleSatisfies...() functions so that their first test is
if (TransactionIdIsCurrentTransactionId(xvac))
rather then
if (!(tuple-t_infomask HEAP_XMIN_COMMITTED))
Huh? That doesn't make any sense at all. xvac wouldn't normally be
valid.
I don't want to put TransactionIdIsCurrentTransactionId into the main
path of the tqual routines if at all possible; it's not an especially
cheap test, particularly if deeply nested in subtransactions. My
thought was that for SatisfiesSnapshot, you'd fall through the first
big chunk if XMIN_COMMITTED is set and then fail the XidInSnapshot test.
Then a TransactionIdIsCurrentTransactionId test could be added in that
fairly-unusual failure path, where it wouldn't slow the main path of
control. Something like
if (XidInSnapshot(HeapTupleHeaderGetXmin(tuple), snapshot))
{
if (TransactionIdIsCurrentTransactionId(HeapTupleHeaderGetXmin(tuple)))
{
if (HeapTupleHeaderGetCmin(tuple) = snapshot-curcid)
return false;/* inserted after scan started */
}
else
return false;/* treat as still in progress */
}
This would require rejiggering snapshots to include our own xid, see
comment for XidInSnapshot. That would add some distributed cost to
executions of XidInSnapshot for recently-committed tuples, but it would
avoid adding any cycles to the path taken for tuples committed before
the xmin horizon, which is the normal case that has to be kept fast.
Haven't looked at whether an equivalent hack is possible for the other
tqual routines.
regards, tom lane
---(end of broadcast)---
TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] [PATCHES] COPY with no WAL, in certain circumstances
On Wed, 2007-01-10 at 10:37 -0500, Tom Lane wrote: Simon Riggs [EMAIL PROTECTED] writes: I agree we could get this to Just Work by altering HeapTupleSatisfies...() functions so that their first test is if (TransactionIdIsCurrentTransactionId(xvac)) Oh? Sorry, I meant xmin not xvac at that point. Cut N Paste thinko. rather then if (!(tuple-t_infomask HEAP_XMIN_COMMITTED)) Huh? That doesn't make any sense at all. xvac wouldn't normally be valid. I don't want to put TransactionIdIsCurrentTransactionId into the main path of the tqual routines if at all possible; it's not an especially cheap test, particularly if deeply nested in subtransactions. Phew, well I'm relieved. Such a mainline change did make me nervous. This would require rejiggering snapshots to include our own xid, see comment for XidInSnapshot. That would add some distributed cost to executions of XidInSnapshot for recently-committed tuples, but it would avoid adding any cycles to the path taken for tuples committed before the xmin horizon, which is the normal case that has to be kept fast. Haven't looked at whether an equivalent hack is possible for the other tqual routines. Will check, thanks. -- Simon Riggs EnterpriseDB http://www.enterprisedb.com ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] Mark/Restore and avoiding RandomAccess sorts
On Wed, 2007-01-10 at 10:46 -0500, Tom Lane wrote: Simon Riggs [EMAIL PROTECTED] writes: Presumably we'd need to teach the Materialize node to pass straight through when the node does not receive any of EXEC_FLAG_MARK, EXEC_FLAG_REWIND or EXEC_FLAG_BACKWARD. It does that already. The Materialize node would have to communicate with the Sort node so it could indicate when it had passed its max size limit, so the Sort could complete the final merge in-situ without wasting more space. Would it be ugly to have the Materialize poke into the SortState? I don't think this is workable; tuplesort is not designed to change from on-the-fly merge to not-on-the-fly on-the-fly. IIRC it's throwing away data as it goes in the first case, and you can't magically get it back. It would have required a full re-sort and then scroll through to the point it had got to. Which was kindof expensive, but possible. Changing this seems like a case of adding 90% more complexity to buy 10% more performance. It's already true that the planner avoids mergejoin when there are lots of duplicate inner tuples, so I do not think we need put lots of effort into performance improvements for the case of large distances back to the mark. Teaching Material how to handle a small mark distance cheaply should be sufficient. OK, I'm happier with that anyway. Sounds straightforward now. -- Simon Riggs EnterpriseDB http://www.enterprisedb.com ---(end of broadcast)--- TIP 7: You can help support the PostgreSQL project by donating at http://www.postgresql.org/about/donate
[HACKERS] Atomic Operations
Hi, what are the assumptions PostgreSQL normally does about atomic operations? I see sig_atomic_t is used in signal handlers. Additionally, there is a match for a cmpxchg instruction in some solaris ports code, but that's about what I found in the source. Am I safe assuming that pointer assignments are atomic (on all platforms PostgreSQL compiles on, that is)? (This is a 'practical advice' from the GNU Libc Manual) How about other integers smaller or equal in size to sizeof(sig_atomic_t)? I'm asking to make sure I rely on the same guarantees in my code. Regards Markus ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Atomic Operations
Markus Schiltknecht wrote: Hi Markus, what are the assumptions PostgreSQL normally does about atomic operations? I see sig_atomic_t is used in signal handlers. Additionally, there is a match for a cmpxchg instruction in some solaris ports code, but that's about what I found in the source. Am I safe assuming that pointer assignments are atomic (on all platforms PostgreSQL compiles on, that is)? (This is a 'practical advice' from the GNU Libc Manual) How about other integers smaller or equal in size to sizeof(sig_atomic_t)? I'm asking to make sure I rely on the same guarantees in my code. Currently we rely on TransactionId being atomic; see GetNewTransactionId. It's defined as uint32 somewhere, so I guess you could rely on that. -- Alvaro Herrerahttp://www.CommandPrompt.com/ PostgreSQL Replication, Consulting, Custom Development, 24x7 support ---(end of broadcast)--- TIP 7: You can help support the PostgreSQL project by donating at http://www.postgresql.org/about/donate
Re: [HACKERS] Atomic Operations
Markus Schiltknecht [EMAIL PROTECTED] writes: what are the assumptions PostgreSQL normally does about atomic operations? Rule of thumb: you want to touch shared memory, you use a lock. There are a few places that violate it, but in general you'd better have a pretty darn good reason to not use a lock. Offhand I recall that we assume TransactionId can be stored atomically in a couple of places where locking would be inconvenient. (This is one of the good reasons for not wanting to widen TransactionId to 64 bits ... the assumption would then fail on some platforms.) I do not believe we assume that pointers can be stored atomically. regards, tom lane ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Added the word TODO in comments
On Wed, 3 Jan 2007 10:16:41 -0500 Jim Nasby [EMAIL PROTECTED] wrote: Given that the TODO list is the official compilation of things that need to get done, ISTM that anything warranting a TODO or XXX in the code should probably be on the TODO list. There are a wide class of possible improvements / fixes that are too small to bother adding to the TODO list, but should still be recorded somewhere. Recording those improvements in the source code seems better than not recording them at all. Also, minor improvements to some part of the implementation are typically dependent on their context in the source code. Since TODO entries are often lacking in context as it is, I don't think trying to move everything to the TODO list would be wise. -Neil ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
[HACKERS] installcheck vs regression DLLs
Hi! When running make installcheck, the DLL files for the regression tests are loaded from the source tree ../../../contrib/ etc. While this certainly makes a bit sense, it poses a problem for binary distributions that want to run the regression tests. It also causes a small problem for the msvc build in that the DLL files are built into a $(top)/Debug/dllname/dllname.dll and thus needs to manually be copied there. Would it make sense to have a standard way to run the regression tests against DLL files on the *installed* system? Perhaps even have installcheck do so? Meaning it would load the DLLs or .so's from $libdir instead of the source tree? If not, other suggestions on how to solve it? //Magnus ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] installcheck vs regression DLLs
Magnus Hagander [EMAIL PROTECTED] writes: Would it make sense to have a standard way to run the regression tests against DLL files on the *installed* system? The RPMs do this, but their solution is pretty darn ugly: ship the test files along with a custom Makefile (and I think they have to patch the test files, too). I'm not entirely convinced that it's worth the trouble. regards, tom lane ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] installcheck vs regression DLLs
Would it make sense to have a standard way to run the regression tests against DLL files on the *installed* system? The RPMs do this, but their solution is pretty darn ugly: ship the test files along with a custom Makefile (and I think they have to patch the test files, too). I'm not entirely convinced that it's worth the trouble. It's just to avoid the ugliness i thought we might want to provide something like this in core. Otherwise there will be localized ugliness in the different packages because it has to be solved somehow. what really is the motivation for keeping some of the tested binaries in the sourcetree when doing installcheck? /Magnus ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] installcheck vs regression DLLs
Magnus Hagander [EMAIL PROTECTED] writes: what really is the motivation for keeping some of the tested binaries in the sourcetree when doing installcheck? As opposed to what? We're certainly not going to *install* regress.so, and I can't see requiring contrib to be there either. These are test files, not part of the installation-under-test. regards, tom lane ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [HACKERS] ideas for auto-processing patches
On Jan 9, 2007, at 20:41 , Jim C. Nasby wrote: On Mon, Jan 08, 2007 at 10:40:16PM -0600, Michael Glaesemann wrote: On Jan 8, 2007, at 19:25 , Jim C. Nasby wrote: Actually, I see point in both... I'd think you'd want to know if a patch worked against the CVS checkout it was written against. Regardless, it's unlikely that the patch was tested against all of the platforms available on the build farm. If it fails on some of the build|patch farm animals, or if it fails due to bitrot, the point is it fails: whatever version the patch was generated against is pretty much moot: the patch needs to be fixed. Wouldn't there be some value to knowing whether the patch failed due to bitrot vs it just didn't work on some platforms out of the gate? I'm having a hard time figuring out what that value would be. How would that knowledge affect what's needed to fix the patch? Michael Glaesemann grzm seespotcode net ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] Added the word TODO in comments
On Wed, Jan 10, 2007 at 02:45:24PM -0500, Neil Conway wrote: On Wed, 3 Jan 2007 10:16:41 -0500 Jim Nasby [EMAIL PROTECTED] wrote: Given that the TODO list is the official compilation of things that need to get done, ISTM that anything warranting a TODO or XXX in the code should probably be on the TODO list. There are a wide class of possible improvements / fixes that are too small to bother adding to the TODO list, but should still be recorded somewhere. Recording those improvements in the source code seems better than not recording them at all. Also, minor improvements to some part of the implementation are typically dependent on their context in the source code. Since TODO entries are often lacking in context as it is, I don't think trying to move everything to the TODO list would be wise. Yeah, 'anything' is too strong... but I've certainly run across some stuff that could go into the TODO. There's some things in there questioning algorithm choices that probably won't ever get done unless someone goes and researches them, for example. -- Jim Nasby[EMAIL PROTECTED] EnterpriseDB http://enterprisedb.com 512.569.9461 (cell) ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] ideas for auto-processing patches
On Wed, 10 Jan 2007, Jim C. Nasby wrote: On Thu, Jan 11, 2007 at 08:04:41AM +0900, Michael Glaesemann wrote: Wouldn't there be some value to knowing whether the patch failed due to bitrot vs it just didn't work on some platforms out of the gate? I'm having a hard time figuring out what that value would be. How would that knowledge affect what's needed to fix the patch? I was thinking that knowing it did work at one time would be useful, but maybe that's not the case... It might be useful to patch authors who submit code which remains unreviewed for some time. Then the submitter or reviewer will be able to know at what date the code drifted. This might be easier than looking through the commit history and trying to locate the problem, I think. Still, the more interesting thing to me would be to be able to provide in the patch a set of custom tests inside of the regression test frame work which aren't suitable as RTs in the long term but will be able to tell the patch author if their code works correctly on a variety of platforms. Thanks, Gavin ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [HACKERS] ideas for auto-processing patches
On Wed, 10 Jan 2007, Jim C. Nasby wrote: On Thu, Jan 11, 2007 at 08:04:41AM +0900, Michael Glaesemann wrote: Wouldn't there be some value to knowing whether the patch failed due to bitrot vs it just didn't work on some platforms out of the gate? I'm having a hard time figuring out what that value would be. How would that knowledge affect what's needed to fix the patch? I was thinking that knowing it did work at one time would be useful, but maybe that's not the case... Has it ever worked is the singularly most fundamental technical support question; yes, it has value. One question here - rhetorical, perhaps - is; What changed and when? Often when things changed can help get you to what changed. (This is what logs are for, and not just automated computer logs, but system management things like, I upgraded GCC today.) And that can help you focus in on what to do to fix the problem. (such as looking to the GCC release notes) A non-rhetorical question is; Shouldn't the build process mechanism/system know when _any_ aspect of a build has failed (including patches)? I'd think so, especially in a build-farm scenario. ...Just my two cents - and worth every penny! -smile- Richard -- Richard Troy, Chief Scientist Science Tools Corporation 510-924-1363 or 202-747-1263 [EMAIL PROTECTED], http://ScienceTools.com/ ---(end of broadcast)--- TIP 7: You can help support the PostgreSQL project by donating at http://www.postgresql.org/about/donate
Re: [HACKERS] Request for review: tsearch2 patch
From: Teodor Sigaev [EMAIL PROTECTED]
Subject: Re: [HACKERS] Request for review: tsearch2 patch
Date: Wed, 10 Jan 2007 18:50:44 +0300
Message-ID: [EMAIL PROTECTED]
I have tested with local-enabled environment and found a bug. Included
is the new version of patches.
Your patch causes crash on tsearch2's installcheck with 'initdb -E UTF8
--locale
C', simple way to reproduce:
# select to_tsquery('default', '''New York''');
server closed the connection unexpectedly
This probably means the server terminated abnormally
before or while processing the request.
The connection to the server was lost. Attempting reset: Failed.
It seems it's a bug with original tsearch2. Here is the patches.
--
*** wordparser/parser.c~2007-01-07 09:54:39.0 +0900
--- wordparser/parser.c 2007-01-11 10:33:41.0 +0900
***
*** 51,57
if (prs-charmaxlen 1)
{
prs-usewide = true;
! prs-wstr = (wchar_t *) palloc(sizeof(wchar_t) * prs-lenstr);
prs-lenwstr = char2wchar(prs-wstr, prs-str, prs-lenstr);
}
else
--- 51,57
if (prs-charmaxlen 1)
{
prs-usewide = true;
! prs-wstr = (wchar_t *) palloc(sizeof(wchar_t) *
(prs-lenstr+1));
prs-lenwstr = char2wchar(prs-wstr, prs-str, prs-lenstr);
}
else
--
! static int p_isalnum(TParser *prs) {
...
! if (lc_ctype_is_c())
! {
! if (c 0x7f)
! return 1;
I have some some doubts that any character greater than 0x7f is an alpha
symbol.
Is it simple assumption or workaround?
Yeah, it's a workaround. Since there's no concept other than
alpha/numeric/latin in tsearch2, Asian characters have to be fall in
one of them.
--
Tatsuo Ishii
SRA OSS, Inc. Japan
---(end of broadcast)---
TIP 3: Have you checked our extensive FAQ?
http://www.postgresql.org/docs/faq
Re: [HACKERS] Request for review: tsearch2 patch
I have tested with local-enabled environment and found a bug. Included
is the new version of patches.
Your patch causes crash on tsearch2's installcheck with 'initdb -E UTF8
--locale
C', simple way to reproduce:
# select to_tsquery('default', '''New York''');
server closed the connection unexpectedly
This probably means the server terminated abnormally
before or while processing the request.
The connection to the server was lost. Attempting reset: Failed.
It seems it's a bug with original tsearch2. Here is the patches.
--
*** wordparser/parser.c~2007-01-07 09:54:39.0 +0900
--- wordparser/parser.c 2007-01-11 10:33:41.0 +0900
***
*** 51,57
if (prs-charmaxlen 1)
{
prs-usewide = true;
! prs-wstr = (wchar_t *) palloc(sizeof(wchar_t) * prs-lenstr);
prs-lenwstr = char2wchar(prs-wstr, prs-str, prs-lenstr);
}
else
--- 51,57
if (prs-charmaxlen 1)
{
prs-usewide = true;
! prs-wstr = (wchar_t *) palloc(sizeof(wchar_t) *
(prs-lenstr+1));
prs-lenwstr = char2wchar(prs-wstr, prs-str, prs-lenstr);
}
else
--
! static int p_isalnum(TParser *prs) {
...
! if (lc_ctype_is_c())
! {
! if (c 0x7f)
! return 1;
I have some some doubts that any character greater than 0x7f is an alpha
symbol.
Is it simple assumption or workaround?
Yeah, it's a workaround. Since there's no concept other than
alpha/numeric/latin in tsearch2, Asian characters have to be fall in
one of them.
--
Tatsuo Ishii
SRA OSS, Inc. Japan
---(end of broadcast)---
TIP 6: explain analyze is your friend
Re: [HACKERS] ideas for auto-processing patches
On Jan 11, 2007, at 10:35 , Richard Troy wrote: On Wed, 10 Jan 2007, Jim C. Nasby wrote: On Thu, Jan 11, 2007 at 08:04:41AM +0900, Michael Glaesemann wrote: Wouldn't there be some value to knowing whether the patch failed due to bitrot vs it just didn't work on some platforms out of the gate? I'm having a hard time figuring out what that value would be. How would that knowledge affect what's needed to fix the patch? I was thinking that knowing it did work at one time would be useful, but maybe that's not the case... Has it ever worked is the singularly most fundamental technical support question; yes, it has value. You'd be able to see whether or not it ever worked by when the patch first hit the patch farm. One question here - rhetorical, perhaps - is; What changed and when? This is recorded in the current build farm. Michael Glaesemann grzm seespotcode net ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] TODO item: update source/timezone for 64-bit tz files
On Sep 17, 2006, at 2:34 , Tom Lane wrote: Back when we converted src/timezone to use int64 for pg_time_t, we wondered what to do about extending the compiled timezone data file format for int64, so that it would work for years beyound 2038. We shelved the problem waiting to see what the upstream zic folks would do. Well, it looks like they've done something about it. So I think we ought to plan on updating our code to match theirs, so that we fix the y2038 problem while keeping it possible to use a standard zic-database installation with Postgres. This is not urgent (I surely see no need to hold up 8.2 to fix it), but it ought to go on the TODO list. regards, tom lane Did this get fixed? I don't see it in the release notes for 8.2 or on the current TODO. Michael Glaesemann grzm seespotcode net ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [HACKERS] TODO item: update source/timezone for 64-bit tz files
On Jan 11, 2007, at 12:51 , Tom Lane wrote: Michael Glaesemann [EMAIL PROTECTED] writes: Did this get fixed? I don't see it in the release notes for 8.2 or on the current TODO. No, nothing's been done. It's going to be a minor PITA, likely, since our sources have diverged from upstream --- someone will have to go through the upstream changes by hand and apply them :-( Any volunteers? I just want to make sure it gets on the TODO if it hasn't been done. Thanks for confirming. Bruce, could this get added? Michael Glaesemann grzm seespotcode net ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] [COMMITTERS] pgsql: Stamp major release 8.3.0,
Bruce Momjian [EMAIL PROTECTED] writes: Joachim Wieland wrote: Attached is a patch that adds a --regression option to ecpg. I have added a checklist item to update the ecpg regression output for major release bumps. I think that is the easiest solution at this point. While I can't say whether Joachim's patch is the cleanest way, surely we will not condemn ourselves to fixing this manually in every future release cycle. While I'm whining ... we really need some support in the ecpg regression tests for platform-specific diffs, so that the consistent ECPG-check failures in buildfarm can go away. regards, tom lane ---(end of broadcast)--- TIP 7: You can help support the PostgreSQL project by donating at http://www.postgresql.org/about/donate
Re: [HACKERS] TODO item: update source/timezone for 64-bit tz
Michael Glaesemann wrote: On Jan 11, 2007, at 12:51 , Tom Lane wrote: Michael Glaesemann [EMAIL PROTECTED] writes: Did this get fixed? I don't see it in the release notes for 8.2 or on the current TODO. No, nothing's been done. It's going to be a minor PITA, likely, since our sources have diverged from upstream --- someone will have to go through the upstream changes by hand and apply them :-( Any volunteers? I just want to make sure it gets on the TODO if it hasn't been done. Thanks for confirming. Bruce, could this get added? Thanks. I was not aware of this item. Added: o Extend timezone code to allow 64-bit values so we can represent years beyond 2038 http://archives.postgresql.org/pgsql-hackers/2006-09/msg01363.php -- Bruce Momjian [EMAIL PROTECTED] EnterpriseDBhttp://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. + ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
[HACKERS] PANIC: block 463 unfound during REDO after out of disk space failure during VACUUM
Hi everyone Was running a VACUUM on a database on a partition which was running out of disk space. During VACUUM the server process died and failed to restart. Running PostgreSQL 8.1.4 I basically want to get the system back up and running ASAP with as little data loss as possible. All and any help is greatly appreciated. Here is output from error log: Jan 11 15:02:32 marshall postgres[71515]: [2-1] WARNING: terminating connection because of crash of another server process Jan 11 15:02:32 marshall postgres[71515]: [2-2] DETAIL: The postmaster has commanded this server process to roll back the current transaction and exit, because another server Jan 11 15:02:32 marshall postgres[71515]: [2-3] process exited abnormally and possibly corrupted shared memory. Jan 11 15:02:32 marshall postgres[71515]: [2-4] HINT: In a moment you should be able to reconnect to the database and repeat your command. Jan 11 15:02:32 marshall postgres[67977]: [4-1] LOG: all server processes terminated; reinitializing Jan 11 15:02:32 marshall postgres[73888]: [5-1] LOG: database system was interrupted at 2007-01-11 15:02:22 WST Jan 11 15:02:32 marshall postgres[73888]: [6-1] LOG: checkpoint record is at 4D/AA7B784 Jan 11 15:02:32 marshall postgres[73888]: [7-1] LOG: redo record is at 4D/AA7B784; undo record is at 0/0; shutdown FALSE Jan 11 15:02:32 marshall postgres[73888]: [8-1] LOG: next transaction ID: 376382676; next OID: 2891876 Jan 11 15:02:32 marshall postgres[73888]: [9-1] LOG: next MultiXactId: 44140; next MultiXactOffset: 91044 Jan 11 15:02:32 marshall postgres[73888]: [10-1] LOG: database system was not properly shut down; automatic recovery in progress Jan 11 15:02:32 marshall postgres[73888]: [11-1] LOG: redo starts at 4D/AA7B7C8 Jan 11 15:02:32 marshall postgres[73889]: [5-1] FATAL: the database system is starting up Jan 11 15:02:32 marshall postgres[73892]: [5-1] FATAL: the database system is starting up Jan 11 15:02:39 marshall postgres[73909]: [5-1] FATAL: the database system is starting up Jan 11 15:02:40 marshall postgres[73888]: [12-1] PANIC: block 463 unfound Jan 11 15:02:41 marshall postgres[67977]: [5-1] LOG: startup process (PID 73888) was terminated by signal 6 Jan 11 15:02:41 marshall postgres[67977]: [6-1] LOG: aborting startup due to startup process failure Thanks in advance -- Warren Guy System Administrator CalorieKing - Australia Tel: +618.9389.8777 Fax: +618.9389.8444 [EMAIL PROTECTED] www.calorieking.com ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [HACKERS] PANIC: block 463 unfound during REDO after out of
Warren Guy wrote: Hi everyone Was running a VACUUM on a database on a partition which was running out of disk space. During VACUUM the server process died and failed to restart. Running PostgreSQL 8.1.4 ... Jan 11 15:02:39 marshall postgres[73909]: [5-1] FATAL: the database system is starting up Jan 11 15:02:40 marshall postgres[73888]: [12-1] PANIC: block 463 unfound Jan 11 15:02:41 marshall postgres[67977]: [5-1] LOG: startup process (PID 73888) was terminated by signal 6 Jan 11 15:02:41 marshall postgres[67977]: [6-1] LOG: aborting startup due to startup process failure You say was running out of disk space - does that mean it did run out of disk space? I don't see the error that caused this, just the results. That would suggest to me that something unusual caused this (or you clipped the log fragment too far down :-) In any case, the first thing I'd try is to make your on-disk backups and set it up as though it's PITR recovery you're doing. That way you can stop the recovery before block 463 causes the failure. Oh, assuming you've got the space you need on your partition of course. HTH -- Richard Huxton Archonet Ltd ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] [COMMITTERS] pgsql: Stamp major release 8.3.0,
On Wed, Jan 10, 2007 at 11:31:41PM -0500, Tom Lane wrote: While I can't say whether Joachim's patch is the cleanest way, surely we will not condemn ourselves to fixing this manually in every future release cycle. I couldn't agree more. The reason why I haven't committed Joachim's patch yet is that I'd like to bring all regression specific changes to one place. Right now we have one change to the log file before diff'ing, one environment variable set and a command line option. The easiest way would be to just change the log files. This guarantees that regression ecpg behaves exactly as the productive one. However, this might get ugly quickly. I will have a closer look at this as soon as my time permits. While I'm whining ... we really need some support in the ecpg regression tests for platform-specific diffs, so that the consistent ECPG-check failures in buildfarm can go away. Hmm, I thought there was. Joachim, could you comment? Michael -- Michael Meskes Email: Michael at Fam-Meskes dot De, Michael at Meskes dot (De|Com|Net|Org) ICQ: 179140304, AIM/Yahoo: michaelmeskes, Jabber: [EMAIL PROTECTED] Go SF 49ers! Go Rhein Fire! Use Debian GNU/Linux! Use PostgreSQL! ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] [COMMITTERS] pgsql: Stamp major release 8.3.0,
On Wed, Jan 10, 2007 at 11:50:10PM -0500, Bruce Momjian wrote: I think we need to hear from Michael wither that version number in the C file is needed. It certainly is not for regression testing. However, I think it should be there for production use so people know how they created those .c files they have around. Michael -- Michael Meskes Email: Michael at Fam-Meskes dot De, Michael at Meskes dot (De|Com|Net|Org) ICQ: 179140304, AIM/Yahoo: michaelmeskes, Jabber: [EMAIL PROTECTED] Go SF 49ers! Go Rhein Fire! Use Debian GNU/Linux! Use PostgreSQL! ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] installcheck vs regression DLLs
what really is the motivation for keeping some of the tested binaries in the sourcetree when doing installcheck? As opposed to what? We're certainly not going to *install* regress.so, and I can't see requiring contrib to be there either. These are test files, not part of the installation-under-test. That would've been my suggestion since I'd say they're both. But if that's not happening and nobody has a better idea, then workaround it is. I'll try to make it as non-ugly as I can. /Magnus ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] [PATCHES] Last infomask bit
Heikki Linnakangas wrote: Bruce Momjian wrote: Tom Lane wrote: Bruce Momjian [EMAIL PROTECTED] writes: Patch applied. Thanks. I added a comment about the unused bits in the header file. Has anyone bothered to measure the overhead added by having to mask to fetch or store the natts value? This is not a zero-cost improvement. SHOW ALL has: log_temp_files | -1 | Log the use of temporary files larger than th and pg_settings has: log_temp_files| -1 | kB | Reporting and Logging / What to Log Looks OK to me. What? I'm completely lost here. What does log_temp_files have to do with the bits on the tuple header? Sorry, Tom wanted two things tested and I replied to the wrong email. -- Bruce Momjian [EMAIL PROTECTED] EnterpriseDBhttp://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. + ---(end of broadcast)--- TIP 7: You can help support the PostgreSQL project by donating at http://www.postgresql.org/about/donate
Re: [HACKERS] [PATCHES] Building libpq/psql with Borland BCC5
Tom Lane wrote: Gavin Sherry [EMAIL PROTECTED] writes: Can we be sure that a BCC build libpq is even safe to use given the problems seen when using psql? Well, I'd not trust it a lot, but surely we have to get it to build before anyone can debug it ... It does build, but the report is that psql crashes after a few commands. -- Bruce Momjian [EMAIL PROTECTED] EnterpriseDBhttp://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. + ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] [PATCHES] Building libpq/psql with Borland BCC5
Alvaro Herrera wrote: Bruce Momjian wrote: Tom Lane wrote: Gavin Sherry [EMAIL PROTECTED] writes: Can we be sure that a BCC build libpq is even safe to use given the problems seen when using psql? Well, I'd not trust it a lot, but surely we have to get it to build before anyone can debug it ... It does build, but the report is that psql crashes after a few commands. What about a Mingw or VC++ psql with a BCC libpq? Is it possible to link something like that? No idea. It would be nice to have the libpq at least able to pass the regression tests. True. -- Bruce Momjian [EMAIL PROTECTED] EnterpriseDBhttp://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. + ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] [PATCHES] COPY with no WAL, in certain circumstances
On Sat, Jan 06, 2007 at 09:20:53PM -0500, Tom Lane wrote: Bruce Momjian [EMAIL PROTECTED] writes: Simon Riggs wrote: Reason for no documentation was that CREATE INDEX and CREATE TABLE AS SELECT already use this optimisation, but to my knowledge neither was/is documented on those command pages. I wasn't aware those used the optimization. Seems they all should be documented somewhere. We don't document every single optimization in the system ... if we did, the docs would be as big as the source code and equally unreadable by non-programmers. I think it's a much better idea just to mention it one place and not try to enumerate exactly which commands have the optimization. I think it would be reasonable to refer to the 'tuning page' from the appropriate pages in the documentation... I'm thinking of something similar to the SEE ALSO section of man pages. The big complain that I have (and have heard) about the docs is that it's very hard to find something unless you know exactly what it is you're looking for. If you don't know that there are performance shortcuts associated with CREATE INDEX you're unlikely to find out about them. -- Jim Nasby[EMAIL PROTECTED] EnterpriseDB http://enterprisedb.com 512.569.9461 (cell) ---(end of broadcast)--- TIP 7: You can help support the PostgreSQL project by donating at http://www.postgresql.org/about/donate
Re: [HACKERS] [PATCHES] fix build on Solaris 10/x86_64 in 64bit mode with Sun Studio 11
Stefan Kaltenbrunner [EMAIL PROTECTED] writes: in src/include/port/solaris.h we define it to little endian only for __i386 - however in 64bit mode the compiler only defines __amd64 causing YTE_ORDER to be undefined. The other option would be to use __x86 which is defined on all intel architectures. Shouldn't this use the already-defined __x86_64__ symbol? I assume that's already defined because otherwise s_lock.h would fail... regards, tom lane ---(end of broadcast)--- TIP 7: You can help support the PostgreSQL project by donating at http://www.postgresql.org/about/donate
Re: [HACKERS] [PATCHES] fix build on Solaris 10/x86_64 in 64bit mode with Sun Studio 11
Stefan Kaltenbrunner [EMAIL PROTECTED] writes: on an Intel based Solaris 10U2 box using Sun Studio 11 with -xarch=generic64 we get a compile time failure in contrib/pgcrypto because BYTE_ORDER is not defined. After further thought I changed this to handle either __amd64 or __x86_64 (or both). Applied. regards, tom lane ---(end of broadcast)--- TIP 7: You can help support the PostgreSQL project by donating at http://www.postgresql.org/about/donate
Re: [HACKERS] [PATCHES] fix build on Solaris 10/x86_64 in 64bit mode with Sun
Tom Lane wrote: Stefan Kaltenbrunner [EMAIL PROTECTED] writes: on an Intel based Solaris 10U2 box using Sun Studio 11 with -xarch=generic64 we get a compile time failure in contrib/pgcrypto because BYTE_ORDER is not defined. After further thought I changed this to handle either __amd64 or __x86_64 (or both). Applied. I think it defines both at all times - if we want fewer rules in there we could switch to just testing __x86 für both 64bit and 32bit but I guess it's fine as it stands now. Stefan ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [pgsql-patches] [HACKERS] [PATCHES] Building libpq/psql with Borland BCC5
On 1/10/07, Alvaro Herrera [EMAIL PROTECTED] wrote: What about a Mingw or VC++ psql with a BCC libpq? Is it possible to link something like that? It would be nice to have the libpq at least able to pass the regression tests. you can use microsoft/mingw compiled DLL files but not library files. however, borland provides a command line tool (implib i thnk) to create an import library for it which works ok. (i think you have to pass a switch to fix underscore issue). libpq.lib is not directly usable (coff vs. omf) but digital mars makes a tool which can do this and I have confirmed it works. note: I've used borland compiled libpq (not psql) with borland C++ builder 3 5 with no problems. I had to hack pg_config.h however. merlin ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [HACKERS] [PATCHES] Allow the identifier length to be
Tom Lane wrote: Peter Eisentraut [EMAIL PROTECTED] writes: Tom Lane wrote: ... why is NAMEDATALEN exported at all?) I think because it used to be used in libpq's notification structure. Yeah, you're probably right. Maybe we should take it out of postgres_ext.h and move it to pg_config_manual.h. If no one complains after a release cycle or so, we could reconsider making it configurable more easily. Added to TODO: * Move NAMEDATALEN from postgres_ext.h to pg_config_manual.h and consider making it more configurable in future releases -- Bruce Momjian [EMAIL PROTECTED] EnterpriseDBhttp://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. + ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [HACKERS] [PATCHES] wal_checksum = on (default) | off
Simon Riggs wrote: On Fri, 2007-01-05 at 22:57 -0500, Tom Lane wrote: Jim Nasby [EMAIL PROTECTED] writes: On Jan 5, 2007, at 6:30 AM, Zeugswetter Andreas ADI SD wrote: Ok, so when you need CRC's on a replicate (but not on the master) you Which sounds to me like a good reason to allow the option in recovery.conf as well... Actually, I'm not seeing the use-case for a slave having a different setting from the master at all? My backup server is less reliable than the primary. My backup server is more reliable than the primary. Somehow, neither of these statements seem likely to be uttered by a sane DBA ... If I take a backup of a server and bring it up on a new system, the blocks in the backup will not have been CRC checked before they go to disk. If I take the same server and now stream log records across to it, why *must* that data be CRC checked, when the original data has not been? I'm proposing choice, with a safe default. That's all. I am assuming this item is dead because no performance results have been reported. -- Bruce Momjian [EMAIL PROTECTED] EnterpriseDBhttp://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. + ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
