Re: [HACKERS] Wierd performance issue with 8.1cvs
Permit me a digression for a pet peeve... "Weird" is spelled "weird". Not "wierd". Yes, I know the nursery rhyme as well as you do --- i before e except after c, etc etc. But weird is spelled weirdly. Appropriate, isn't it? It's also my pet peeve, but I long ago stopped bothering to correct people. One may also wish to consider "feisty" :) Chris ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] Wierd performance issue with 8.1cvs
On Sun, 24 Apr 2005, Christopher Kings-Lynne wrote: Permit me a digression for a pet peeve... "Weird" is spelled "weird". Not "wierd". Yes, I know the nursery rhyme as well as you do --- i before e except after c, etc etc. But weird is spelled weirdly. Appropriate, isn't it? It's also my pet peeve, but I long ago stopped bothering to correct people. I think it's quite useful to correct, because many of us use english only when reading mailing lists and documentation :) Regards, Oleg _ Oleg Bartunov, sci.researcher, hostmaster of AstroNet, Sternberg Astronomical Institute, Moscow University (Russia) Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/ phone: +007(095)939-16-83, +007(095)939-23-83 ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: [HACKERS] Constant WAL replay
what. Allowing that to be turned off would be interesting for a number of purposes, such as burning a database onto CD. FWIW, Oracle suggests a "transportable tablespace" for this feature. Which is a tablespace that is not written too and which can be read by any database. Would that solve the purposes you mean? Greetings, Klaus ---(end of broadcast)--- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] Wierd performance issue with 8.1cvs
Oleg Bartunov wrote: I think it's quite useful to correct, because many of us use english only when reading mailing lists and documentation :) I think that it's important to refrain from corrections on a public forum as long as the essence of the message is clear. Some people might get offended, others might just stop writing because they get intimidated by the seemingly high demands on correct spelling or sentence structure. I appreciate getting corrected by people I know in a limited forum. I would not expect it when I do a mistakes here. Can't say it ever has happend although there's often good grounds for it so I have nothing to complain about. :-) Regards, Thomas Hallgren ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] Wierd performance issue with 8.1cvs
> I appreciate getting corrected by people I know in a limited > forum. I would not expect it when I do a mistakes here. Can't > say it ever has happend although there's often good grounds > for it so I have nothing to complain about. :-) I think you meant to say 'I Can't say it ever has happened...' :) ... John ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send "unregister YourEmailAddressHere" to [EMAIL PROTECTED])
Re: [HACKERS] Wierd performance issue with 8.1cvs
On Sun, 24 Apr 2005, Thomas Hallgren wrote: Oleg Bartunov wrote: I think it's quite useful to correct, because many of us use english only when reading mailing lists and documentation :) I think that it's important to refrain from corrections on a public forum as long as the essence of the message is clear. Some people might get offended, others might just stop writing because they get intimidated by the seemingly high demands on correct spelling or sentence structure. I appreciate getting corrected by people I know in a limited forum. I would not expect it when I do a mistakes here. Can't say it ever has happend although there's often good grounds for it so I have nothing to complain about. :-) It's difficult to opposed you but "limited forum" is not we could afford. When I see a lot of mistakes, crude words in russian forums I always feel myself uncomfortable. Unfortunately, I can't imagine how it looks for english speaking people. Of course, there are simple misprints which could be painlessly ommited, but I see nothing offending against correcting gross mistakes. Usually, people just too busy to notice them. Regards, Thomas Hallgren ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster Regards, Oleg _ Oleg Bartunov, sci.researcher, hostmaster of AstroNet, Sternberg Astronomical Institute, Moscow University (Russia) Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/ phone: +007(095)939-16-83, +007(095)939-23-83 ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: [HACKERS] Wierd performance issue with 8.1cvs
Oleg Bartunov wrote: On Sun, 24 Apr 2005, Thomas Hallgren wrote: Oleg Bartunov wrote: I think it's quite useful to correct, because many of us use english only when reading mailing lists and documentation :) I think that it's important to refrain from corrections on a public forum as long as the essence of the message is clear. Some people might get offended, others might just stop writing because they get intimidated by the seemingly high demands on correct spelling or sentence structure. I appreciate getting corrected by people I know in a limited forum. I would not expect it when I do a mistakes here. Can't say it ever has happend although there's often good grounds for it so I have nothing to complain about. :-) It's difficult to opposed you but "limited forum" is not we could afford. When I see a lot of mistakes, crude words in russian forums I always feel myself uncomfortable. Unfortunately, I can't imagine how it looks for english speaking people. Of course, there are simple misprints which could be painlessly ommited, but I see nothing offending against correcting gross mistakes. Usually, people just too busy to notice them. This Sentience has tree errors. Regards, Thomas Hallgren ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] [pgsql-hackers-win32] UNICODE/UTF-8 on win32
Where are we on this? As far as I can tell, we never disabled UTF8 on Win32 in our code. The only thing we did do was to disable UTF8 in pginstaller. See this FAQ item: http://pginstaller.projects.postgresql.org/faq/FAQ_windows.html#2.6 Is the current setup OK? Should we allow UTF8 on Win32 for languages that can use C locale, like Asian languages? --- Tatsuo Ishii wrote: > I do understand the problem, but don't undertstand the decision you > guys made. The fact that UPPER/LOWER and some other functions does not > work in win32 is surely a problem for some languages, but not a > problem for otheres. For example, Japanese (and probably Chinese and > Korean) does not have a concept upper/lower. So the fact UPPER/LOWER > does not work with UTF-8/win32 is not problem for Japanese (and for > some other languages). Just using C locale with UTF-8 is enough in > this case. > > In summary, I think you guys are going to overkill the multibyte > support functionality on UTF-8/win32 because of the fact that some > langauges do not work. > > Same thing can be said to EUC-JP, EUC-CN and EUC-KR and so on as well. > > I strongly object the policy to try to unconditionaly disable UTF-8 > support on win32. > -- > Tatsuo Ishii > > From: "Magnus Hagander" <[EMAIL PROTECTED]> > Subject: RE: [pgsql-hackers-win32] UNICODE/UTF-8 on win32 > Date: Sat, 1 Jan 2005 14:48:04 +0100 > Message-ID: <[EMAIL PROTECTED]> > > > UNICODE/UTF-8 does not work on the win32 server. The reason is that > > strcoll() and friends don't work with it. To support it on win32, it > > needs to be converted to UTF16 and use the wide-character versions of > > the fucntion. Which we do not do. > > (See > > http://archives.postgresql.org/pgsql-hackers-win32/2004-11/msg00036.php > > and > > http://archives.postgresql.org/pgsql-hackers-win32/2004-12/msg00106.php) > > > > > > I don't *think* we need to disable ito n the client. AFAIK, the client > > interfaces don't use any of these functions, and I've seen reports of > > people using that long before we had a native win32 server. > > > > > > //Magnus > > > > > > >-Original Message- > > >From: Tatsuo Ishii [mailto:[EMAIL PROTECTED] > > >Sent: den 1 januari 2005 01:10 > > >To: [EMAIL PROTECTED] > > >Cc: Magnus Hagander; [EMAIL PROTECTED] > > >Subject: Re: [pgsql-hackers-win32] UNICODE/UTF-8 on win32 > > > > > > > > >Sorry, but I don't subscribe to pgsql-hackers-win32 list. What's the > > >problem here? > > >-- > > >Tatsuo Ishii > > > > > >> "Magnus Hagander" <[EMAIL PROTECTED]> writes: > > >> > We know it's broken and won't be fixed for 8.0. > > >> > > >> > If we just #ifndef WIN32 the definitions in > > >utils/mb/encnames.c it won't > > >> > be possible to select that encoding, right? Will that have > > >any other > > >> > unwanted effects (such as breaking client encodings)? If > > >not, I suggest > > >> > this is done. > > >> > > >> I believe the subscripts in those arrays have to match the encoding > > >> enum type, so you can't just ifdef out individual entries. > > >> > > >> > (Or perhaps something can be done in pg_valid_server_encoding?) > > >> > > >> Making the valid_server_encoding function reject it might work. > > >> Tatsuo-san would know for sure. > > >> > > >> Should we also reject it as a client encoding, or does that work OK? > > >> > > >> regards, tom lane > > >> > > > > > > > ---(end of broadcast)--- > TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED] > -- Bruce Momjian| http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
[HACKERS] I know I am behind
I know I am behind in applying patches. I plan to work all through May to apply recent patches and those held over from the 8.0 beta period. -- Bruce Momjian| http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] [pgsql-hackers-win32] UNICODE/UTF-8 on win32
John Hansen wrote: > Look at the upper/lower I sent to the list, they should be able to > replace upper/lower for the utf8 encoding (and works independent of > locale).. You mean ICU? Yes, it seems like a good approach for 8.1. --- > > ... John > > > -Original Message- > > From: [EMAIL PROTECTED] > > [mailto:[EMAIL PROTECTED] On Behalf Of Bruce Momjian > > Sent: Sunday, April 24, 2005 10:35 PM > > To: Tatsuo Ishii > > Cc: [EMAIL PROTECTED]; [EMAIL PROTECTED]; > > [EMAIL PROTECTED]; pgsql-hackers@postgresql.org > > Subject: Re: [HACKERS] [pgsql-hackers-win32] UNICODE/UTF-8 on win32 > > > > > > Where are we on this? As far as I can tell, we never disabled UTF8 on > > Win32 in our code. The only thing we did do was to disable > > UTF8 in pginstaller. See this FAQ item: > > > > > > http://pginstaller.projects.postgresql.org/faq/FAQ_windows.html#2.6 > > > > Is the current setup OK? Should we allow UTF8 on Win32 for > > languages that can use C locale, like Asian languages? > > > > -- > > - > > > > Tatsuo Ishii wrote: > > > I do understand the problem, but don't undertstand the decision you > > > guys made. The fact that UPPER/LOWER and some other > > functions does not > > > work in win32 is surely a problem for some languages, but not a > > > problem for otheres. For example, Japanese (and probably Chinese and > > > Korean) does not have a concept upper/lower. So the fact > > UPPER/LOWER > > > does not work with UTF-8/win32 is not problem for Japanese (and for > > > some other languages). Just using C locale with UTF-8 is enough in > > > this case. > > > > > > In summary, I think you guys are going to overkill the multibyte > > > support functionality on UTF-8/win32 because of the fact that some > > > langauges do not work. > > > > > > Same thing can be said to EUC-JP, EUC-CN and EUC-KR and so > > on as well. > > > > > > I strongly object the policy to try to unconditionaly disable UTF-8 > > > support on win32. > > > -- > > > Tatsuo Ishii > > > > > > From: "Magnus Hagander" <[EMAIL PROTECTED]> > > > Subject: RE: [pgsql-hackers-win32] UNICODE/UTF-8 on win32 > > > Date: Sat, 1 Jan 2005 14:48:04 +0100 > > > Message-ID: > > > <[EMAIL PROTECTED]> > > > > > > > UNICODE/UTF-8 does not work on the win32 server. The > > reason is that > > > > strcoll() and friends don't work with it. To support it > > on win32, it > > > > needs to be converted to UTF16 and use the wide-character > > versions > > > > of the fucntion. Which we do not do. > > > > (See > > > > > > http://archives.postgresql.org/pgsql-hackers-win32/2004-11/msg00036. > > > > php > > > > and > > > > > > http://archives.postgresql.org/pgsql-hackers-win32/2004-12/msg00106. > > > > php) > > > > > > > > > > > > I don't *think* we need to disable ito n the client. AFAIK, the > > > > client interfaces don't use any of these functions, and I've seen > > > > reports of people using that long before we had a native > > win32 server. > > > > > > > > > > > > //Magnus > > > > > > > > > > > > >-Original Message- > > > > >From: Tatsuo Ishii [mailto:[EMAIL PROTECTED] > > > > >Sent: den 1 januari 2005 01:10 > > > > >To: [EMAIL PROTECTED] > > > > >Cc: Magnus Hagander; [EMAIL PROTECTED] > > > > >Subject: Re: [pgsql-hackers-win32] UNICODE/UTF-8 on win32 > > > > > > > > > > > > > > >Sorry, but I don't subscribe to pgsql-hackers-win32 list. What's > > > > >the problem here? > > > > >-- > > > > >Tatsuo Ishii > > > > > > > > > >> "Magnus Hagander" <[EMAIL PROTECTED]> writes: > > > > >> > We know it's broken and won't be fixed for 8.0. > > > > >> > > > > >> > If we just #ifndef WIN32 the definitions in > > > > >utils/mb/encnames.c it won't > > > > >> > be possible to select that encoding, right? Will that have > > > > >any other > > > > >> > unwanted effects (such as breaking client encodings)? If > > > > >not, I suggest > > > > >> > this is done. > > > > >> > > > > >> I believe the subscripts in those arrays have to match the > > > > >> encoding enum type, so you can't just ifdef out > > individual entries. > > > > >> > > > > >> > (Or perhaps something can be done in > > pg_valid_server_encoding?) > > > > >> > > > > >> Making the valid_server_encoding function reject it might work. > > > > >> Tatsuo-san would know for sure. > > > > >> > > > > >> Should we also reject it as a client encoding, or does > > that work OK? > > > > >> > > > > >> regards, tom lane > > > > >> > > > > > > > > > > > > > > > ---(end of > > > broadcast)--- > > > TIP 1: subscribe and unsubscribe commands go to > > > [EMAIL PROTECTED] > > > > > > > -- > > Bruce Momjian| http://candle.pha.pa.us > > pgman@candle.pha.pa.us | (610) 359-1001 > > + If your life is a
Re: [HACKERS] [pgsql-hackers-win32] UNICODE/UTF-8 on win32
Look at the upper/lower I sent to the list, they should be able to replace upper/lower for the utf8 encoding (and works independent of locale).. ... John > -Original Message- > From: [EMAIL PROTECTED] > [mailto:[EMAIL PROTECTED] On Behalf Of Bruce Momjian > Sent: Sunday, April 24, 2005 10:35 PM > To: Tatsuo Ishii > Cc: [EMAIL PROTECTED]; [EMAIL PROTECTED]; > [EMAIL PROTECTED]; pgsql-hackers@postgresql.org > Subject: Re: [HACKERS] [pgsql-hackers-win32] UNICODE/UTF-8 on win32 > > > Where are we on this? As far as I can tell, we never disabled UTF8 on > Win32 in our code. The only thing we did do was to disable > UTF8 in pginstaller. See this FAQ item: > > > http://pginstaller.projects.postgresql.org/faq/FAQ_windows.html#2.6 > > Is the current setup OK? Should we allow UTF8 on Win32 for > languages that can use C locale, like Asian languages? > > -- > - > > Tatsuo Ishii wrote: > > I do understand the problem, but don't undertstand the decision you > > guys made. The fact that UPPER/LOWER and some other > functions does not > > work in win32 is surely a problem for some languages, but not a > > problem for otheres. For example, Japanese (and probably Chinese and > > Korean) does not have a concept upper/lower. So the fact > UPPER/LOWER > > does not work with UTF-8/win32 is not problem for Japanese (and for > > some other languages). Just using C locale with UTF-8 is enough in > > this case. > > > > In summary, I think you guys are going to overkill the multibyte > > support functionality on UTF-8/win32 because of the fact that some > > langauges do not work. > > > > Same thing can be said to EUC-JP, EUC-CN and EUC-KR and so > on as well. > > > > I strongly object the policy to try to unconditionaly disable UTF-8 > > support on win32. > > -- > > Tatsuo Ishii > > > > From: "Magnus Hagander" <[EMAIL PROTECTED]> > > Subject: RE: [pgsql-hackers-win32] UNICODE/UTF-8 on win32 > > Date: Sat, 1 Jan 2005 14:48:04 +0100 > > Message-ID: > > <[EMAIL PROTECTED]> > > > > > UNICODE/UTF-8 does not work on the win32 server. The > reason is that > > > strcoll() and friends don't work with it. To support it > on win32, it > > > needs to be converted to UTF16 and use the wide-character > versions > > > of the fucntion. Which we do not do. > > > (See > > > > http://archives.postgresql.org/pgsql-hackers-win32/2004-11/msg00036. > > > php > > > and > > > > http://archives.postgresql.org/pgsql-hackers-win32/2004-12/msg00106. > > > php) > > > > > > > > > I don't *think* we need to disable ito n the client. AFAIK, the > > > client interfaces don't use any of these functions, and I've seen > > > reports of people using that long before we had a native > win32 server. > > > > > > > > > //Magnus > > > > > > > > > >-Original Message- > > > >From: Tatsuo Ishii [mailto:[EMAIL PROTECTED] > > > >Sent: den 1 januari 2005 01:10 > > > >To: [EMAIL PROTECTED] > > > >Cc: Magnus Hagander; [EMAIL PROTECTED] > > > >Subject: Re: [pgsql-hackers-win32] UNICODE/UTF-8 on win32 > > > > > > > > > > > >Sorry, but I don't subscribe to pgsql-hackers-win32 list. What's > > > >the problem here? > > > >-- > > > >Tatsuo Ishii > > > > > > > >> "Magnus Hagander" <[EMAIL PROTECTED]> writes: > > > >> > We know it's broken and won't be fixed for 8.0. > > > >> > > > >> > If we just #ifndef WIN32 the definitions in > > > >utils/mb/encnames.c it won't > > > >> > be possible to select that encoding, right? Will that have > > > >any other > > > >> > unwanted effects (such as breaking client encodings)? If > > > >not, I suggest > > > >> > this is done. > > > >> > > > >> I believe the subscripts in those arrays have to match the > > > >> encoding enum type, so you can't just ifdef out > individual entries. > > > >> > > > >> > (Or perhaps something can be done in > pg_valid_server_encoding?) > > > >> > > > >> Making the valid_server_encoding function reject it might work. > > > >> Tatsuo-san would know for sure. > > > >> > > > >> Should we also reject it as a client encoding, or does > that work OK? > > > >> > > > >>regards, tom lane > > > >> > > > > > > > > > > > ---(end of > > broadcast)--- > > TIP 1: subscribe and unsubscribe commands go to > > [EMAIL PROTECTED] > > > > -- > Bruce Momjian| http://candle.pha.pa.us > pgman@candle.pha.pa.us | (610) 359-1001 > + If your life is a hard drive, | 13 Roberts Road > + Christ can be your backup.| Newtown Square, > Pennsylvania 19073 > > ---(end of > broadcast)--- > TIP 9: the planner will ignore your desire to choose an index > scan if your > joining column's datatypes do not match > > ---(end of broadcast)--- TIP 2: you can get off
Re: [HACKERS] Woo hoo ... a whole new set of compiler headaches!!
Alvaro Herrera wrote: We have plenty of very ugly macros anyway. See fastgetattr(), HeapKeyTest(), HeapTupleSatisfies(), HeapTupleHeaderSetXmax and friends, Assert() and friends. I don't think Assert() is too bad, but I agree some of the others are a bit ugly. In some places where we would like to use a macro but don't want to doubly-evaluate macro arguments, we define the function with "static inline" in a header file when using GCC, and include a normal function definition in a source file otherwise (see list_length() in pg_list.h / list.c for example). Needless to say, this is even uglier :) An alternative would be to define inline functions in headers using "static __inline". When using GCC (or other compilers that implement sane "static inline" semantics per C99, such as icc), __inline would expand to "inline"; otherwise it would expand to the empty string. Compilers that don't implement "static inline" would include multiple copies of the function definition, which would bloat the object code (if a compiler doesn't implement "static", it is a good bet that it also doesn't implement the unit-at-a-time analysis required to elide unused static function definitions). So I'm not really sure that this is a win overall. -Neil ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send "unregister YourEmailAddressHere" to [EMAIL PROTECTED])
Re: [HACKERS] [pgsql-hackers-win32] UNICODE/UTF-8 on win32
Ehmm,... No the upper/lower replacements I sent to -hackers ICU was not me Tho for win32 you're better off writing wrapper classes for the win32 native functions. > -Original Message- > From: Bruce Momjian [mailto:[EMAIL PROTECTED] > Sent: Sunday, April 24, 2005 10:50 PM > To: John Hansen > Cc: Tatsuo Ishii; [EMAIL PROTECTED]; [EMAIL PROTECTED]; > [EMAIL PROTECTED]; pgsql-hackers@postgresql.org > Subject: Re: [HACKERS] [pgsql-hackers-win32] UNICODE/UTF-8 on win32 > > John Hansen wrote: > > Look at the upper/lower I sent to the list, they should be able to > > replace upper/lower for the utf8 encoding (and works > independent > > of locale).. > > You mean ICU? Yes, it seems like a good approach for 8.1. > > -- > - > > > > > > ... John > > > > > -Original Message- > > > From: [EMAIL PROTECTED] > > > [mailto:[EMAIL PROTECTED] On Behalf Of Bruce > > > Momjian > > > Sent: Sunday, April 24, 2005 10:35 PM > > > To: Tatsuo Ishii > > > Cc: [EMAIL PROTECTED]; [EMAIL PROTECTED]; > > > [EMAIL PROTECTED]; pgsql-hackers@postgresql.org > > > Subject: Re: [HACKERS] [pgsql-hackers-win32] > UNICODE/UTF-8 on win32 > > > > > > > > > Where are we on this? As far as I can tell, we never > disabled UTF8 > > > on > > > Win32 in our code. The only thing we did do was to disable > > > UTF8 in pginstaller. See this FAQ item: > > > > > > > > > > http://pginstaller.projects.postgresql.org/faq/FAQ_windows.html#2.6 > > > > > > Is the current setup OK? Should we allow UTF8 on Win32 for > > > languages that can use C locale, like Asian languages? > > > > > > -- > > > - > > > > > > Tatsuo Ishii wrote: > > > > I do understand the problem, but don't undertstand the decision > > > > you guys made. The fact that UPPER/LOWER and some other > > > functions does not > > > > work in win32 is surely a problem for some languages, but not a > > > > problem for otheres. For example, Japanese (and > probably Chinese > > > > and > > > > Korean) does not have a concept upper/lower. So the fact > > > UPPER/LOWER > > > > does not work with UTF-8/win32 is not problem for Japanese (and > > > > for some other languages). Just using C locale with UTF-8 is > > > > enough in this case. > > > > > > > > In summary, I think you guys are going to overkill the > multibyte > > > > support functionality on UTF-8/win32 because of the > fact that some > > > > langauges do not work. > > > > > > > > Same thing can be said to EUC-JP, EUC-CN and EUC-KR and so > > > on as well. > > > > > > > > I strongly object the policy to try to unconditionaly disable > > > > UTF-8 support on win32. > > > > -- > > > > Tatsuo Ishii > > > > > > > > From: "Magnus Hagander" <[EMAIL PROTECTED]> > > > > Subject: RE: [pgsql-hackers-win32] UNICODE/UTF-8 on win32 > > > > Date: Sat, 1 Jan 2005 14:48:04 +0100 > > > > Message-ID: > > > > <[EMAIL PROTECTED]> > > > > > > > > > UNICODE/UTF-8 does not work on the win32 server. The > > > reason is that > > > > > strcoll() and friends don't work with it. To support it > > > on win32, it > > > > > needs to be converted to UTF16 and use the wide-character > > > versions > > > > > of the fucntion. Which we do not do. > > > > > (See > > > > > > > > > http://archives.postgresql.org/pgsql-hackers-win32/2004-11/msg00036. > > > > > php > > > > > and > > > > > > > > > http://archives.postgresql.org/pgsql-hackers-win32/2004-12/msg00106. > > > > > php) > > > > > > > > > > > > > > > I don't *think* we need to disable ito n the client. > AFAIK, the > > > > > client interfaces don't use any of these functions, and I've > > > > > seen reports of people using that long before we had a native > > > win32 server. > > > > > > > > > > > > > > > //Magnus > > > > > > > > > > > > > > > >-Original Message- > > > > > >From: Tatsuo Ishii [mailto:[EMAIL PROTECTED] > > > > > >Sent: den 1 januari 2005 01:10 > > > > > >To: [EMAIL PROTECTED] > > > > > >Cc: Magnus Hagander; [EMAIL PROTECTED] > > > > > >Subject: Re: [pgsql-hackers-win32] UNICODE/UTF-8 on win32 > > > > > > > > > > > > > > > > > >Sorry, but I don't subscribe to pgsql-hackers-win32 list. > > > > > >What's the problem here? > > > > > >-- > > > > > >Tatsuo Ishii > > > > > > > > > > > >> "Magnus Hagander" <[EMAIL PROTECTED]> writes: > > > > > >> > We know it's broken and won't be fixed for 8.0. > > > > > >> > > > > > >> > If we just #ifndef WIN32 the definitions in > > > > > >utils/mb/encnames.c it won't > > > > > >> > be possible to select that encoding, right? Will > that have > > > > > >any other > > > > > >> > unwanted effects (such as breaking client encodings)? If > > > > > >not, I suggest > > > > > >> > this is done. > > > > > >> > > > > > >> I believe the subscripts in those arrays have to match the > > > > > >> encoding enum type, so you can't just ifdef out > > > individual en
Re: [HACKERS] Constant WAL replay
Klaus Naumann <[EMAIL PROTECTED]> writes: >> what. Allowing that to be turned off would be interesting for a number >> of purposes, such as burning a database onto CD. > FWIW, Oracle suggests a "transportable tablespace" for this feature. > Which is a tablespace that is not written too and which can be read by > any database. > Would that solve the purposes you mean? It's a very long way from here to there. In particular, since different installations have different transaction histories, the XIDs in the table could not be transportable. You'd almost be forced to build something like the non-MVCC, XID-less table type that was being speculated about up-thread. regards, tom lane ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] [pgsql-hackers-win32] UNICODE/UTF-8 on win32
"John Hansen" <[EMAIL PROTECTED]> writes:
> Look at the upper/lower I sent to the list, they should be able to
> replace upper/lower for the utf8 encoding (and works independent of
> locale)..
I was under the impression we couldn't use these, precisely because they
weren't locale-aware. ("It works for most people" isn't good enough.)
In any case, don't we need a solution that covers sorting (strcoll) as
well as upper/lower?
regards, tom lane
---(end of broadcast)---
TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] Constant WAL replay
On Sun, Apr 24, 2005 at 11:41:17AM -0400, Tom Lane wrote: > Klaus Naumann <[EMAIL PROTECTED]> writes: > >> what. Allowing that to be turned off would be interesting for a number > >> of purposes, such as burning a database onto CD. > > > FWIW, Oracle suggests a "transportable tablespace" for this feature. > > Which is a tablespace that is not written too and which can be read by > > any database. > > Would that solve the purposes you mean? > > It's a very long way from here to there. In particular, since different > installations have different transaction histories, the XIDs in the > table could not be transportable. Unless the tables are frozen first. -- Alvaro Herrera (<[EMAIL PROTECTED]>) "En las profundidades de nuestro inconsciente hay una obsesiva necesidad de un universo lógico y coherente. Pero el universo real se halla siempre un paso más allá de la lógica" (Irulan) ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Constant WAL replay
On Sun, Apr 24, 2005 at 08:10:34AM +0200, Hans-Jürgen Schönig wrote: > The idea: We are looking for a way to implement a synchronous > single-master / multiple slaves systems. > Meanwhile we are able to serialize / deserialize WAL records and send > them to a group communication system which transports those records to > the slave database. > This is not hard to do. In fact, I believe Command Prompt's Mammoth Replicator does exactly this. -- Alvaro Herrera (<[EMAIL PROTECTED]>) "La victoria es para quien se atreve a estar solo" ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] [PERFORM] Bad n_distinct estimation; hacks suggested?
Tom Lane wrote: Josh Berkus writes: Overall, our formula is inherently conservative of n_distinct. That is, I believe that it is actually computing the *smallest* number of distinct values which would reasonably produce the given sample, rather than the *median* one. This is contrary to the notes in analyze.c, which seem to think that we're *overestimating* n_distinct. Well, the notes are there because the early tests I ran on that formula did show it overestimating n_distinct more often than not. Greg is correct that this is inherently a hard problem :-( I have nothing against adopting a different formula, if you can find something with a comparable amount of math behind it ... but I fear it'd only shift the failure cases around. The math in the paper does not seem to look at very low levels of q (= sample to pop ratio). The formula has a range of [d,N]. It appears intuitively (i.e. I have not done any analysis) that at very low levels of q, as f1 moves down from n, the formula moves down from N towards d very rapidly. I did a test based on the l_comments field in a TPC lineitems table. The test set has N = 6001215, D = 2921877. In my random sample of 1000 I got d = 976 and f1 = 961, for a DUJ1 figure of 24923, which is too low by 2 orders of magnitude. I wonder if this paper has anything that might help: http://www.stat.washington.edu/www/research/reports/1999/tr355.ps - if I were more of a statistician I might be able to answer :-) cheers andrew ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] Constant WAL replay
Alvaro Herrera wrote: On Sun, Apr 24, 2005 at 08:10:34AM +0200, Hans-Jürgen Schönig wrote: The idea: We are looking for a way to implement a synchronous single-master / multiple slaves systems. Meanwhile we are able to serialize / deserialize WAL records and send them to a group communication system which transports those records to the slave database. This is not hard to do. In fact, I believe Command Prompt's Mammoth Replicator does exactly this. Very close. We don't use the WAL (yet, slated for probably 8.1) but we do use a transaction log shipping method. So the implementation is almost the same. Sincerely, Joshua D. Drake Command Prompt, Inc. ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] [PERFORM] Bad n_distinct estimation; hacks suggested?
Andrew, > The math in the paper does not seem to look at very low levels of q (= > sample to pop ratio). Yes, I think that's the failing. Mind you, I did more testing and found out that for D/N ratios of 0.1 to 0.3, the formula only works within 5x accuracy (which I would consider acceptable) with a sample size of 25% or more (which is infeasable in any large table).The formula does work for populations where D/N is much lower, say 0.01. So overall it seems to only work for 1/4 of cases; those where n/N is large and D/N is low. And, annoyingly, that's probably the population where accurate estimation is least crucial, as it consists mostly of small tables. I've just developed (not original, probably, but original to *me*) a formula that works on populations where n/N is very small and D/N is moderate (i.e. 0.1 to 0.4): N * (d/n)^(sqrt(N/n)) However, I've tested it only on (n/N < 0.005 and D/N > 0.1 and D/N < 0.4) populations, and only 3 of them to boot. I'd appreciate other people trying it on their own data populations, particularly very different ones, like D/N > 0.7 or D/N < 0.01. Further, as Andrew points out we presumably do page sampling rather than purely random sampling so I should probably read the paper he referenced. Working on it now -- Josh Berkus Aglio Database Solutions San Francisco ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
[HACKERS] Old-style OR indexscan slated for destruction
I am about to rip out the code that supports multiple indexscans for OR conditions inside a single IndexScan plan node. As best I can tell, the new-style bitmap-OR code is as fast or faster than the old way even in fully cached test cases (ie, with no allowance for improved efficiency of disk access). So there's no percentage in maintaining support for the old way, and getting rid of it will allow simplification of code and data structures in the planner. regards, tom lane ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] W[i/e]rd performance issue with 8.1cvs
Guys, > >> "Weird" is spelled "weird". Not "wierd". OK, spelling errors taken into account. Now could we perhaps address the **postgresql** errors? I'm seeing this kind of "performance plunge" on 8.1cvs in one of every 3 runs. It's obviously a serious stability issue, whatever is causing it. -- Josh Berkus Aglio Database Solutions San Francisco ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: [HACKERS] W[i/e]rd performance issue with 8.1cvs
Josh Berkus writes: > I'm seeing this kind of "performance plunge" on 8.1cvs in one of every 3 > runs. > It's obviously a serious stability issue, whatever is causing it. I concur with the upthread suggestion that it may come from not doing checkpoints in a realistic fashion, thereby allowing too much queued I/O work to build up. What happens if you set the checkpoint interval to 5 or 10 minutes? regards, tom lane ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: [HACKERS] W[i/e]rd performance issue with 8.1cvs
Tom, > I concur with the upthread suggestion that it may come from not doing > checkpoints in a realistic fashion, thereby allowing too much queued > I/O work to build up. ÂWhat happens if you set the checkpoint interval > to 5 or 10 minutes? I'll test. Keep in mind that it takes me a couple of days to run a new test (I've got 11 in the queue right now) so it'll be a bit. -- Josh Berkus Aglio Database Solutions San Francisco ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: [HACKERS] [PERFORM] Bad n_distinct estimation; hacks suggested?
Folks, > I wonder if this paper has anything that might help: > http://www.stat.washington.edu/www/research/reports/1999/tr355.ps - if I > were more of a statistician I might be able to answer :-) Actually, that paper looks *really* promising. Does anyone here have enough math to solve for D(sub)Md on page 6? I'd like to test it on samples of < 0.01%. Tom, how does our heuristic sampling work? Is it pure random sampling, or page sampling? -- Josh Berkus Aglio Database Solutions San Francisco ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send "unregister YourEmailAddressHere" to [EMAIL PROTECTED])
Re: [HACKERS] [pgsql-hackers-win32] UNICODE/UTF-8 on win32
Right, they were meant as a starting point, but if you can point me to
how I can obtain the current locale, then I can fix them to cover the
remaining 15 special cases.
... John
> -Original Message-
> From: Tom Lane [mailto:[EMAIL PROTECTED]
> Sent: Monday, April 25, 2005 2:01 AM
> To: John Hansen
> Cc: Bruce Momjian; Tatsuo Ishii; [EMAIL PROTECTED];
> [EMAIL PROTECTED]; pgsql-hackers@postgresql.org
> Subject: Re: [HACKERS] [pgsql-hackers-win32] UNICODE/UTF-8 on win32
>
> "John Hansen" <[EMAIL PROTECTED]> writes:
> > Look at the upper/lower I sent to the list, they should be able to
> > replace upper/lower for the utf8 encoding (and works
> independent
> > of locale)..
>
> I was under the impression we couldn't use these, precisely
> because they weren't locale-aware. ("It works for most
> people" isn't good enough.)
>
> In any case, don't we need a solution that covers sorting
> (strcoll) as well as upper/lower?
>
> regards, tom lane
>
>
---(end of broadcast)---
TIP 6: Have you searched our list archives?
http://archives.postgresql.org
Re: [HACKERS] [pgsql-hackers-win32] UNICODE/UTF-8 on win32
That is pretty much where we are ;-) I think we're fine for 8.0.x with this, because if you actually need UTF-8 (and can live with sorting broken, no upper/lower etc), you can do it using a manual initdb. For 8.1, I think the ICU approach looks a lot more promising than trying to do "on the fly conversion to UTF-16 and back". Especially if there is profit in having ICU for other platforms as well, since we would do without win32 specific code for that (I seem to recall there being discussions about other platforms needing it as well - and the guy who did it didn't do it for win32, so there is at least some..) I was planning to test the ICU patch on win32 to see that it works at all, but I haven't had the time to do that just yet. //Magnus >Where are we on this? As far as I can tell, we never disabled UTF8 on >Win32 in our code. The only thing we did do was to disable UTF8 in >pginstaller. See this FAQ item: > > >http://pginstaller.projects.postgresql.org/faq/FAQ_windows.html#2.6 > >Is the current setup OK? Should we allow UTF8 on Win32 for languages >that can use C locale, like Asian languages? > >--- > > >Tatsuo Ishii wrote: >> I do understand the problem, but don't undertstand the decision you >> guys made. The fact that UPPER/LOWER and some other >functions does not >> work in win32 is surely a problem for some languages, but not a >> problem for otheres. For example, Japanese (and probably Chinese and >> Korean) does not have a concept upper/lower. So the fact UPPER/LOWER >> does not work with UTF-8/win32 is not problem for Japanese (and for >> some other languages). Just using C locale with UTF-8 is enough in >> this case. >> >> In summary, I think you guys are going to overkill the multibyte >> support functionality on UTF-8/win32 because of the fact that some >> langauges do not work. >> >> Same thing can be said to EUC-JP, EUC-CN and EUC-KR and so >on as well. >> >> I strongly object the policy to try to unconditionaly disable UTF-8 >> support on win32. >> -- >> Tatsuo Ishii >> >> From: "Magnus Hagander" <[EMAIL PROTECTED]> >> Subject: RE: [pgsql-hackers-win32] UNICODE/UTF-8 on win32 >> Date: Sat, 1 Jan 2005 14:48:04 +0100 >> Message-ID: ><[EMAIL PROTECTED]> >> >> > UNICODE/UTF-8 does not work on the win32 server. The reason is that >> > strcoll() and friends don't work with it. To support it on >win32, it >> > needs to be converted to UTF16 and use the wide-character >versions of >> > the fucntion. Which we do not do. >> > (See >> > >http://archives.postgresql.org/pgsql-hackers-win32/2004-11/msg00036.php >> > and >> > >http://archives.postgresql.org/pgsql-hackers-win32/2004-12/msg0 >0106.php) >> > >> > >> > I don't *think* we need to disable ito n the client. >AFAIK, the client >> > interfaces don't use any of these functions, and I've seen >reports of >> > people using that long before we had a native win32 server. >> > >> > >> > //Magnus >> > >> > >> > >-Original Message- >> > >From: Tatsuo Ishii [mailto:[EMAIL PROTECTED] >> > >Sent: den 1 januari 2005 01:10 >> > >To: [EMAIL PROTECTED] >> > >Cc: Magnus Hagander; [EMAIL PROTECTED] >> > >Subject: Re: [pgsql-hackers-win32] UNICODE/UTF-8 on win32 >> > > >> > > >> > >Sorry, but I don't subscribe to pgsql-hackers-win32 list. >What's the >> > >problem here? >> > >-- >> > >Tatsuo Ishii >> > > >> > >> "Magnus Hagander" <[EMAIL PROTECTED]> writes: >> > >> > We know it's broken and won't be fixed for 8.0. >> > >> >> > >> > If we just #ifndef WIN32 the definitions in >> > >utils/mb/encnames.c it won't >> > >> > be possible to select that encoding, right? Will that have >> > >any other >> > >> > unwanted effects (such as breaking client encodings)? If >> > >not, I suggest >> > >> > this is done. >> > >> >> > >> I believe the subscripts in those arrays have to match >the encoding >> > >> enum type, so you can't just ifdef out individual entries. >> > >> >> > >> > (Or perhaps something can be done in >pg_valid_server_encoding?) >> > >> >> > >> Making the valid_server_encoding function reject it might work. >> > >> Tatsuo-san would know for sure. >> > >> >> > >> Should we also reject it as a client encoding, or does >that work OK? >> > >> >> > >> regards, tom lane >> > >> >> > > >> > >> >> ---(end of >broadcast)--- >> TIP 1: subscribe and unsubscribe commands go to >[EMAIL PROTECTED] >> > >-- > Bruce Momjian| http://candle.pha.pa.us > pgman@candle.pha.pa.us | (610) 359-1001 > + If your life is a hard drive, | 13 Roberts Road > + Christ can be your backup.| Newtown Square, >Pennsylvania 19073 > ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] Postgres: pg_hba.conf, md5, pg_shadow, encrypted
On Sat, 2005-04-23 at 09:02 -0400, Stephen Frost wrote: > * Antoine Martin ([EMAIL PROTECTED]) wrote: > > Basically, multiple input data that have the same output hash, which is > > of no use when what you are trying to find is the input. > > Finding collisions quicker for a known input is one thing, but that is > > not going to reduce the search space, not even your storage space (it is > > unlikely that the colliding results would all be valid input). > > Erm, you aren't necessairly trying to find the input... It may be the > case that you're trying to find what you need to authenticate to this > server, or any other PostgreSQL server where the same userid & input are > used. In that case you just need something that hashes to the same > thing. Agreed, what I said was that it is highly unlikely you will find colliding inputs that are valid, so the "SHA weakness" does not really help you here as it does not reduce the search space: You are much better off pre-calculating hashes for possible usernames & passwords than working backwards and generating all possible hashes hoping that one would happen to be matching a real entry... Usernames are not exactly random, passwords are less predictable, the chance of a useful collision on the username+password is remote at best. > Using a random salt would mean that it's different per server so > breaking it on one doesn't help you against another server unless you > happened to find the actual original input. Absolutely. > > > Is adding the non-guessable salt that hard anyway? > > It is if you want to continue to support the 'md5' method in pg_hba.conf > because the wireline protocol will probably need to change. A less > intrusive alternative would be to add an 'with encrypted password 'xyz' > with random salt' or some such which would only be supported with the > 'password' method in pg_hba.conf. > > One problem with this is that you then can't just switch from password > to md5 or back again. Perhaps that's ok though? Comments? Just add another authentication method - call it 'md5-salt' (sharing most of the 'md5' code), you get backwards compatibility and you advise users to migrate to the new salt hash. Shouldn't be too hard... Might as well do a 'sha512-salt' too. Antoine ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: [HACKERS] [PERFORM] Bad n_distinct estimation; hacks suggested?
Here is my opinion. I hope this helps. Maybe there is no one good formula: On boolean type, there are at most 3 distinct values. There is an upper bound for fornames in one country. There is an upper bound for last names in one country. There is a fixed number of states and postal codes in one country. On the other hand, with timestamp, every value could be distinct. A primary key with only one column has only distinct values. If the integer column refers with a foreign key into another table's only primary key, we could take advantage of that knolege. A column with a unique index has only distinct values. First ones are for classifying and the second ones measure continuous or discrete time or something like the time. The upper bound for classifying might be 3 (boolean), or it might be one million. The properties of the distribution might be hard to guess. Here is one way: 1. Find out the number of distinct values for 500 rows. 2. Try to guess, how many distinct values are for 1000 rows. Find out the real number of distinct values for 1000 rows. 3. If the guess and the reality are 50% wrong, do the iteration for 2x1000 rows. Iterate using a power of two to increase the samples, until you trust the estimate enough. So, in the phase two, you could try to guess with two distinct formulas: One for the classifying target (boolean columns hit there). Another one for the timestamp and numerical values. If there are one million classifications on one column, how you can find it out, by other means than checking at least two million rows? This means, that the user should have a possibility to tell the lower bound for the number of rows for sampling. Regards, Marko Ristola Tom Lane wrote: Josh Berkus writes: Overall, our formula is inherently conservative of n_distinct. That is, I believe that it is actually computing the *smallest* number of distinct values which would reasonably produce the given sample, rather than the *median* one. This is contrary to the notes in analyze.c, which seem to think that we're *overestimating* n_distinct. Well, the notes are there because the early tests I ran on that formula did show it overestimating n_distinct more often than not. Greg is correct that this is inherently a hard problem :-( I have nothing against adopting a different formula, if you can find something with a comparable amount of math behind it ... but I fear it'd only shift the failure cases around. regards, tom lane ---(end of broadcast)--- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] Old-style OR indexscan slated for destruction
On Sun, 24 Apr 2005, Tom Lane wrote: I am about to rip out the code that supports multiple indexscans for OR conditions inside a single IndexScan plan node. As best I can tell, the new-style bitmap-OR code is as fast or faster than the old way even in fully cached test cases (ie, with no allowance for improved efficiency of disk access). So there's no percentage in maintaining support for the old way, and getting rid of it will allow simplification of code and data structures in the planner. very good, Tom ! regards, tom lane ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org Regards, Oleg _ Oleg Bartunov, sci.researcher, hostmaster of AstroNet, Sternberg Astronomical Institute, Moscow University (Russia) Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/ phone: +007(095)939-16-83, +007(095)939-23-83 ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
[HACKERS] CVS regression test problem
Just for information. I see many regression tests failed (26 of 96). regression.diffs shows many error messages like ! ERROR: unrecognized node type: 516 Regards, Oleg _ Oleg Bartunov, sci.researcher, hostmaster of AstroNet, Sternberg Astronomical Institute, Moscow University (Russia) Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/ phone: +007(095)939-16-83, +007(095)939-23-83 ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: [HACKERS] CVS regression test problem
On Mon, Apr 25, 2005 at 01:18:29AM +0400, Oleg Bartunov wrote: > Just for information. > I see many regression tests failed (26 of 96). regression.diffs shows many > error messages like ! ERROR: unrecognized node type: 516 Did you initdb? I see no such problem with today's tip and my shared row locking patch ... I think Tom committed something without increasing catversion, but I'm not sure (my patch changes PG_CONTROL_VERSION so I had to initdb anyway). I assume you make distclean'd ... -- Alvaro Herrera (<[EMAIL PROTECTED]>) "Nunca se desea ardientemente lo que solo se desea por razón" (F. Alexandre) ---(end of broadcast)--- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] CVS regression test problem
On Sun, 24 Apr 2005, Alvaro Herrera wrote: On Mon, Apr 25, 2005 at 01:18:29AM +0400, Oleg Bartunov wrote: Just for information. I see many regression tests failed (26 of 96). regression.diffs shows many error messages like ! ERROR: unrecognized node type: 516 Did you initdb? I see no such problem with today's tip and my shared yes, I did. row locking patch ... I think Tom committed something without increasing catversion, but I'm not sure (my patch changes PG_CONTROL_VERSION so I had to initdb anyway). I assume you make distclean'd ... aha, make distclean helps ! Sorry for bothering Regards, Oleg _ Oleg Bartunov, sci.researcher, hostmaster of AstroNet, Sternberg Astronomical Institute, Moscow University (Russia) Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/ phone: +007(095)939-16-83, +007(095)939-23-83 ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: [HACKERS] Constant WAL replay
Joshua D. Drake wrote: > Alvaro Herrera wrote: > > On Sun, Apr 24, 2005 at 08:10:34AM +0200, Hans-J?rgen Sch?nig wrote: > > > > > >>The idea: We are looking for a way to implement a synchronous > >>single-master / multiple slaves systems. > >>Meanwhile we are able to serialize / deserialize WAL records and send > >>them to a group communication system which transports those records to > >>the slave database. > >>This is not hard to do. > > > > > > In fact, I believe Command Prompt's Mammoth Replicator does exactly > > this. > > Very close. We don't use the WAL (yet, slated for probably 8.1) but we > do use a transaction log shipping method. So the implementation is > almost the same. Can you run queries on the slave? If so, how do you handle xid collisions? -- Bruce Momjian| http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] Constant WAL replay
Very close. We don't use the WAL (yet, slated for probably 8.1) but we do use a transaction log shipping method. So the implementation is almost the same. Can you run queries on the slave? If so, how do you handle xid collisions? You can run any query that does not modify data on a replicated table. You can run any non data modifying query on any of the tables. Sincerely, Joshua D. Drake ---(end of broadcast)--- TIP 8: explain analyze is your friend
Re: [HACKERS] Constant WAL replay
Joshua D. Drake wrote: > >> > >>Very close. We don't use the WAL (yet, slated for probably 8.1) but we > >>do use a transaction log shipping method. So the implementation is > >>almost the same. > > > > > > Can you run queries on the slave? If so, how do you handle xid collisions? > > You can run any query that does not modify data on a replicated table. > You can run any non data modifying query on any of the tables. So, do you modify the slave to prevent it from grabbing an xid? -- Bruce Momjian| http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] idea for concurrent seqscans
TODO description added: * Allow sequential scans to take advantage of other concurrent sequentiqal scans, also called "Synchronised Scanning" One possible implementation is to start sequential scans from the lowest numbered buffer in the shared cache, and when reaching the end wrap around to the beginning, rather than always starting sequential scans at the start of the table. --- Jeff Davis wrote: > I had an idea that might improve parallel seqscans on the same relation. > > If you have lots of concurrent seqscans going on a large relation, the > cache hit ratio is very low. But, if the seqscans are concurrent on the > same relation, there may be something to gain by starting a seqscan near > the page being accessed by an already-in-progress seqscan, and wrapping > back around to that start location. That would make some use of the > shared buffers, which would otherwise just be cache pollution. > > I made a proof-of-concept implementation, which is entirely in heapam.c, > except for one addition to the HeapScanDesc struct in relscan.h. It is > not at all up to production quality; there are things I know that need > to be addressed. Basically, I just modified heapam.c to be able to start > at any page in the relation. Then, every time it reads a new page, I > have it mark the relation's oid and the page number in a shared mem > segment. Everytime a new scan is started, it reads the shared mem > segment, and if the relation's oid matches, it starts the scan at the > page number it found in the shared memory. Otherwise, it starts the scan > at 0. > > There are a couple obvious issues, one is that my whole implementation > doesn't account for reverse scans at all (since initscan doesn't know > what direction the scan will move in), but that shouldn't be a major > problem since at worst it will be the current behavior (aside: can > someone tell me how to force reverse scans so I can test that better?). > Another is that there's a race condition with the shared mem, and that's > out of pure laziness on my part. > > This method is really only effective at all if there is a significant > amount of disk i/o. If it's pulling the data from O/S buffers the > various scans will diverge too much and not be using eachother's shared > buffers. > > I tested with shared_buffers=500 and all stats on. I used 60 threads > performing 30 seqscans each in my script ssf.rb (I refer to my > modification as "sequential scan follower" or ssf). > > Here are some results with my modifications: > $ time ./ssf.rb # my script > > real4m22.476s > user0m0.389s > sys 0m0.186s > > test=# select relpages from pg_class where relname='test_ssf'; > relpages > -- > 1667 > (1 row) > > test=# select count(*) from test_ssf; > count > > 20 > (1 row) > > test=# select pg_stat_get_blocks_hit(17232) as hit, > pg_stat_get_blocks_fetched(17232) as total; > hit | total > +- > 971503 | 3353963 > (1 row) > > Or, approx. 29% cache hit. > > Here are the results without my modifications: > > test=# select relpages from pg_class where relname='test_ssf'; > relpages > -- > 1667 > (1 row) > > test=# select count(*) from test_ssf; > count > > 20 > (1 row) > > test=# select pg_stat_get_blocks_hit(17231) as hit, > pg_stat_get_blocks_fetched(17231) as total; > hit | total > +- > 19 | 3353963 > (1 row) > > Or, approx. 6% cache hit. Note: the oid is different, because I have two > seperately initdb'd data directories, one for the modified version, one > for the unmodified 8.0.0. > > This is the first time I've really modified the PG source code to do > anything that looked promising, so this is more of a question than > anything else. Is it promising? Is this a potentially good approach? I'm > happy to post more test data and more documentation, and I'd also be > happy to bring the code to production quality. However, before I spend > too much more time on that, I'd like to get a general response from a > 3rd party to let me know if I'm off base. > > Regards, > Jeff Davis > [ Attachment, skipping... ] [ Attachment, skipping... ] > > ---(end of broadcast)--- > TIP 5: Have you checked our extensive FAQ? > >http://www.postgresql.org/docs/faq -- Bruce Momjian| http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] Old-style OR indexscan slated for destruction
I am about to rip out the code that supports multiple indexscans for OR conditions inside a single IndexScan plan node. As best I can tell, the new-style bitmap-OR code is as fast or faster than the old way even in fully cached test cases (ie, with no allowance for improved efficiency of disk access). So there's no percentage in maintaining support for the old way, and getting rid of it will allow simplification of code and data structures in the planner. For all index types? Even lossy ones? Chris ---(end of broadcast)--- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] Old-style OR indexscan slated for destruction
Christopher Kings-Lynne <[EMAIL PROTECTED]> writes: > I am about to rip out the code that supports multiple indexscans for OR > conditions inside a single IndexScan plan node. As best I can tell, > the new-style bitmap-OR code is as fast or faster than the old way > even in fully cached test cases (ie, with no allowance for improved > efficiency of disk access). So there's no percentage in maintaining > support for the old way, and getting rid of it will allow simplification > of code and data structures in the planner. > For all index types? Even lossy ones? Can't see that a lossy index would make any difference ... regards, tom lane ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] Old-style OR indexscan slated for destruction
For all index types? Even lossy ones? Can't see that a lossy index would make any difference ... OK, was just checking :) ---(end of broadcast)--- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] [PERFORM] Bad n_distinct estimation; hacks suggested?
Josh Berkus writes: > Tom, how does our heuristic sampling work? Is it pure random sampling, or > page sampling? Manfred probably remembers better than I do, but I think the idea is to approximate pure random sampling as best we can without actually examining every page of the table. regards, tom lane ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] CVS regression test problem
Oleg Bartunov writes: > On Sun, 24 Apr 2005, Alvaro Herrera wrote: >>> error messages like ! ERROR: unrecognized node type: 516 >> >> I assume you make distclean'd ... > aha, make distclean helps ! Sorry for bothering I added/removed some node types, so if you didn't do a full rebuild you'd probably have various files out of sync about which node type has which number. AFAICS there are two reasonable ways to work with updating from CVS: 1. make distclean (or at least make clean) each time you update; 2. configure with --enable-depend and trust gcc to get it right. If you don't do either then you *will* get burnt. I personally use method #1 --- I don't know what the reliability of method #2 is. regards, tom lane ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
[HACKERS] Passwords in PSQL
Hello All I am using \c [database] [user-name] to connect to any database. Any one can connect to any database. If any one knows user name, he/she connect to db. I want to provide some security that no one can connect without providing passwords. postgres uses a function do_coonect() for this perpose. I provide it passwords but it let me connect to db with any passwords provided. Can any one tell me how to set passwords for db user. and how can I implement password protection in psql. ---(end of broadcast)--- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
