Re: [HACKERS] client side syntax error localisation for psql (v1)
Dear Tatsuo, Thanks for your reply, as I noticed from the source code that your name often appears when dealing with multi-byte issues;-) On Fri, 12 Mar 2004, Tatsuo Ishii wrote: As far as I understand your code, it will be broken on many multi byte encodings. 1) a character is not always represented on a terminal propotional to the storage size. For example a kanji character in UTF-8 encoding has a storage size of 3 bytes while it occupies spaces only twice of ASCII characters on a terminal. Same thing can be said to LATIN 2,3 etc. in UTF-8 perhaps. I thought I dealt with that in the code by calling PQmblen for every char. Am I wrong ? 2) It assume all encodings are ASCII compatible. Apparently some client-side-only encodings do not satisfy this request. Examples include SJIS, Big5. What I mean by ASCII compatible is that spaces, new lines, carriage returns, tabs and NULL (C string terminaison) are one byte characters. This assumption seemed pretty safe to me. If this is not the case, I cannot understand how any error message could work in psql. If one printf( ), that would not be a space character? Or is the terminal doing some on the fly translation?? What if a file is read with such encoding??? Or is there a special compilation option to generate special strings, but in this case the executable would not be compatible with any other terminal Well, I just underline my lack of knowledge here:-( If not, how can I detect these special characters that I need to change ? Maybe I could translate the string to a pg_wchar[] if the function is available to psql? Also as I quick and dirty temporary fix, I can skip statement extraction for those encodings that do not meet my expectations. So I would need to know what encodings are at risk with the current scheme? -- Fabien Coelho - [EMAIL PROTECTED] ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] [pgsql-www] The Name Game: postgresql.net vs. pgfoundry.org
-Original Message- From: Josh Berkus [mailto:[EMAIL PROTECTED] Sent: 11 March 2004 23:14 To: [EMAIL PROTECTED] Cc: [EMAIL PROTECTED] Subject: [pgsql-www] The Name Game: postgresql.net vs. pgfoundry.org B) Favor www.pgfoundry.org I don't really mind too much either way, but if anything I would prefer B. /D ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: [HACKERS] client side syntax error localisation for psql (v1)
Fabien COELHO wrote: There is also a localisation issue here, as the translation of both lines must match so that the alignment is kept. I thought that if it is the very same word, the translation should be the same. You can just indent with as many spaces. This is done in other places as well. Yes, but then I need to know the length of the translation... I did that, so I hope sprintf did the translation job. Thanks, -- Fabien Coelho - [EMAIL PROTECTED] ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] The Name Game: postgresql.net vs. pgfoundry.org
Michael Glaesemann wrote: Just to speak up (as an avid lurker), I agree with Jeroen that this distinction is quite subtle and may cause confusion. Some may even expect the two to resolve to the same site, as a lot of popular sites own .com/.net/.org, all resolving to the same site. Speaking of .com vs .net vs .org, anyone remember the mysql.com vs mysql.org fiasco? Anyway, if I can vote, I'll vote for postgresql.net (for the lack of better choices). I agree with Tom that pgfoundry is kind of random. It's not apparent at all that it's a PostgreSQL entity. Besides, Tom Marc is already listed as the registrant of several domains including postgresql.com. Why not use them? Also, we're targetting the developers right? Please do not consider ourselves as being too stupid to differentiate between postgresql.org and postgresql.net... If people don't like to type long names, we can always do automatic redirection between projname.postgresql.net, postgresql.org/projects/projname, projname.projects.postgresql.org, etc. Or even perhaps use tinyurl :-) -- dave ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Default Stats Revisited
Josh Berkus But possible more error prone. If you crank up the default statistics to 50, but the index default is still 25... OTOH, you could always have the setting of used for index default be whichever is greater... hmmm. Well, I'm not 100% opposed to a multiplier. I'd like to take a poll of DBAs to find out which they would find more accessable. But since most people seem to be ignoring this thread, I'm not sure we'll get much response ... ...very interesting thoughts overall. I note that your idea has a very strong basis and is pretty much suggested to us, since it forms part of the allowable specification of TPC-H: - so let's do it! I enclose the following TPC-H text. Section 5.2.8, p.103, TPC Benchmark H (Decision Support), Standard Specification, revision 2.1.0: 5.2.8 The gathering of statistics is part of the database load (see Clause 4.3) but it also serves as an important configuration vehicle, particularly for the query optimizer. In order to satisfy the requirements of Clause 5.2.7, it is desirable to collect the same quality of statistics for every column of every table. However, in order to reduce processing requirements, it is permissible to segment columns into distinct classes and base the level of statistics collection for a particular column on class membership. Class definitions must rely solely on schema-related attributes of a column and must be applied consistently across all tables. For example: . membership in an index; . leading or other position in an index; . use in a constraint (including a primary or foreign key relationships). Statistics that operate in sets, such as distribution statistics, should employ a fixed set appropriate to the scale factor used. Knowledge of the cardinality, values or distribution of a non-key column as specified in Clause 4 cannot be used to tailor statistics gathering. Based upon that, might we add slightly to your index stats suggestion slightly and include constraint-member columns also? Overall, the problem you highlighted is: - if default stats is 10 and we don't think that is very useful, then clearly that should change, yet we are performance constrained I would note that DB2 uses default 20, as does Teradata. Oracle uses 75 histogram buckets as default. Clearly, 10 is not the accepted view...but I'm sure I'll be shot down for such thinking. You set me thinking about another possible solution: My understanding is that the default for stats on PostgreSQL is collect, whereas on other systems it is don't collect (until instructed). i.e. we collect stats on all columns by default. That's good, but the end result is that it is slower than the other default. [That might be important because dynamic sampling is not yet implemented, not sure] If we had the option not to collect stats at all on most columns, then that would speed things up, wouldn't it (just as TPC-H pretty much says). Perhaps it might be better to offer an option to alter that default? It seems that DEFAULT_STATISTICS_TARGET cannot be set to no thanks. If you could turn off the collection of what we seem to be agreeing is a relatively pointless collection of statistics, would that not improve stats collection performance? If set to 0, only collect number of nulls, number of distinct values, max and min. Turning off stats-by-default and yet ramping up the collection target where it is needed sounds good for TPC-H/DBT-3 environments - and will likely even improve perf numbers on the actual tests! ..this might also allow us to rename the two parameters, just as was recently done with work_mem etc.. statistics_target_default 0 -- i.e. don't collect by default statistics_target_index 100 I would also suggest a further class of columns for statistics collection: statistics_target_text which would include all fields with length 32 (pick a limit...), since these are very frequently all unique. Setting these to -1 would mean they use the statistics_target_default value, which would be their default setting. You might even provide a mechanism for defining statistics collection classes based upon their datatype...e.g. CREATE STATISTICS CLASS CLASSNAME STATISTICS COLLECTION TARGET 100 APPLIES TO (LEADING n) INDEX COLUMNS, CONSTRAINT COLUMNS; CREATE STATISTICS CLASS CLASSNAME STATISTICS COLLECTION TARGET 0 APPLIES TO COMMAS SEPARATED LIST OF DATATYPES; anything not specifically identified, would then use system parameter default That would be great, since we now have the ability to collects stats on user defined datatypes (don't we?). Back to the multiplier: Yes please, but only if it worked like this: Normal and index stats are settable differently. Index stats DEFAULT is a multiple of normal stats, unless specifically set. (You may wish to set it down as well as up, remember). That way, the default behaviour improves even when the index stats parameter is not actually set, yet is still controllable when you do. Best Regards, Simon
Re: [HACKERS] PITR Functional Design v2 for 7.5
Hannu Krosing Josh Berkus kirjutas T, 09.03.2004 kell 19:46: In my personal experience, the *primary* use of PITR is recovery from User Error. For example, with one SQL Server 7.0 installation for a law firm, I've made use of PITR 4 times over the last 4 years: once was because and HDD failed, the other three were all becuase of IT dept. staff running unconstrained UPDATE queries against the back end. For recovery with minimal loss of data, there are existing solutions, such as replication servers, in addition to PITR; for recovery from User Error, only PITR will suffice. Actually PostgreSQL used to have very good support for this until some time in Postgres95 development by supporting additional temporal qualifiers for queries. I.e. one could ask for 'SELECT SALARY FROM EMPLOYEES AS IT WAS AT YESTERDAY NOON' (the syntax was not exacly this :) Shame we can't run cc -as was 4 years ago. Or even better cc -as will be in 3 weeks; that would save me loads :) !! It was a very simple and logical result of PostgreSQL's MVCC storage and was supported by VACUUM allowing dropping only deleted tuples older than some specified time. Oracle has recently added something similar (using their WAL's) to ver. 9.0 or 10.x of their DBMS exactly for recovery from user errors. The support for this was dropped from postgreSQL citing performance reasons at that time, but I still hope that it can restored some time. Flashback query is a new feature in Oracle 9i. I believe it is regarded with some horror by the DBA community... I get your idea though, though I think it is a different thing. PITR is about the avoidance of risk, not really about fixing any particular classes of problem. If you have PITR you can recover from all problems, frequent or not, depending upon how carefully and for how long you protect your backups. Reading old MVCC copies won't take that away, though is a feature that would be useful within a particular time window. It's also hard to tell whether VACUUM has been run, and if so on which tables, since that will also change the answer you get from those MVCC-usage type queries. Oracle recognise this also. Flashback hasn't replaced backup/restore. Neither has it prevented them from enhancing log miner. Temporal support is a different issue anyway. It is up to you to come up with a database design that supports being able to ask that question, if that is a business requirement. Best regards, Simon Riggs ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faqs/FAQ.html
Re: [HACKERS] client side syntax error localisation for psql (v1)
Dear Tatsuo, Thanks for your reply, as I noticed from the source code that your name often appears when dealing with multi-byte issues;-) On Fri, 12 Mar 2004, Tatsuo Ishii wrote: As far as I understand your code, it will be broken on many multi byte encodings. 1) a character is not always represented on a terminal propotional to the storage size. For example a kanji character in UTF-8 encoding has a storage size of 3 bytes while it occupies spaces only twice of ASCII characters on a terminal. Same thing can be said to LATIN 2,3 etc. in UTF-8 perhaps. I thought I dealt with that in the code by calling PQmblen for every char. Am I wrong ? PQmblen returns the storage size, which is not necessarily same as the character width reprensented in a terminal. For example for a kanji character in UTF-8 PQmblen returns 3, but it ocuppies 2 x ASCII character space, not x 3. Isn't that a problem for you? 2) It assume all encodings are ASCII compatible. Apparently some client-side-only encodings do not satisfy this request. Examples include SJIS, Big5. What I mean by ASCII compatible is that spaces, new lines, carriage returns, tabs and NULL (C string terminaison) are one byte characters. This assumption seemed pretty safe to me. If this is not the case, I cannot understand how any error message could work in psql. If one printf( ), that would not be a space character? Or is the terminal doing some on the fly translation?? What if a file is read with such encoding??? Or is there a special compilation option to generate special strings, but in this case the executable would not be compatible with any other terminal Well, I just underline my lack of knowledge here:-( If not, how can I detect these special characters that I need to change ? Maybe I could translate the string to a pg_wchar[] if the function is available to psql? I think you can do it safely using PQmblen. 1) start from the begining of the target string 2) apply PQmblen 3) if it returns 1, you can do the spcecial character detection 4) otherwise it must not be an ASCII character and you can skip as many characters as PQmnlen returns 5) goto 1) if any characters remain Also as I quick and dirty temporary fix, I can skip statement extraction for those encodings that do not meet my expectations. So I would need to know what encodings are at risk with the current scheme? -- Fabien Coelho - [EMAIL PROTECTED] ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Timing of 'SELECT 1'
On Wed, Mar 10, 2004 at 11:43:48AM -0500, Bruce Momjian wrote: The problem with gprof is that I am going to see all the backend startup stuff too, no? Is there a way to get a dump just the run of the query? I was sort of lurking on this thread, waiting to see what became of it. Did nobody actually come to a conclusion on what that last msec was from? Alex -- [EMAIL PROTECTED] Alex J. Avriette, Unix Systems Gladiator ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faqs/FAQ.html
[HACKERS] index leaks ?
Hello, I'm watching a strange beahviour by postgres, I wonder if it's a memory leak: Creating an index and destroying it makes the postgres process grow in size. This is what I do: create table test (a int); create index test_idx on test (a); drop index test_idx; create index test_idx on test (a); drop index test_idx; create index test_idx on test (a); drop index test_idx; ... The size growth is reported by 'top' in the fields SIZE, RSS and SHARE. The database is a newly created one after first initdb as of CVS 20040311. Can it be a memory leak in postgres code ? TIA --strk; ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] The Name Game: postgresql.net vs. pgfoundry.org
However, some of the porting team felt that it would be confusing for people who typed in www.postgresql.net to be presented with the GForge interface, and suggested that we use the domain after what we'll be calling the new Tool, namely pgFoundry, thus putting stuff at www.pgFoundry.org and projectname.pgfoundry.org. Can't the frontpage for the GForge interface be changed slightly? If the frontpage was redesigned to state clearly that it's a community development site, and go here for the site about the main postgresql proejct, that should take care of the users that typed in postgresql.net instead of .org. Then the sites themselves would be whatever.postgresql.net as is usual with gforge. FWIW, I agree with those who feel that pgfoundry.org is not really intuitive to go when looking for postgresql stuff, if you can't tell that from above :-) //Magnus ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] client side syntax error localisation for psql (v1)
PQmblen returns the storage size, which is not necessarily same as the character width reprensented in a terminal. For example for a kanji character in UTF-8 PQmblen returns 3, but it ocuppies 2 x ASCII character space, not x 3. Isn't that a problem for you? 2) It assume all encodings are ASCII compatible. Apparently some client-side-only encodings do not satisfy this request. Examples include SJIS, Big5. What I mean by ASCII compatible is that spaces, new lines, carriage returns, tabs and NULL (C string terminaison) are one byte characters. This assumption seemed pretty safe to me. If this is not the case, I cannot understand how any error message could work in psql. If one printf( ), that would not be a space character? Or is the terminal doing some on the fly translation?? What if a file is read with such encoding??? Or is there a special compilation option to generate special strings, but in this case the executable would not be compatible with any other terminal Well, I just underline my lack of knowledge here:-( If not, how can I detect these special characters that I need to change ? Maybe I could translate the string to a pg_wchar[] if the function is available to psql? I think you can do it safely using PQmblen. 1) start from the begining of the target string 2) apply PQmblen 3) if it returns 1, you can do the spcecial character detection 4) otherwise it must not be an ASCII character and you can skip as many characters as PQmnlen returns 5) goto 1) if any characters remain ~~of course this should be 2) -- Tatsuo Ishii ---(end of broadcast)--- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] client side syntax error localisation for psql (v1)
Dear Tatsuo, 1) a character is not always represented on a terminal propotional to the storage size. For example a kanji character in UTF-8 encoding has a storage size of 3 bytes while it occupies spaces only twice of ASCII characters on a terminal. Same thing can be said to LATIN 2,3 etc. in UTF-8 perhaps. I thought I dealt with that in the code by calling PQmblen for every char. Am I wrong ? PQmblen returns the storage size, which is not necessarily same as the character width reprensented in a terminal. For example for a kanji character in UTF-8 PQmblen returns 3, but it ocuppies 2 x ASCII character space, not x 3. Isn't that a problem for you? If I read you correctly, you mean that 1 character may take 3 bytes of storage in the string, but it is not guaranteed to be 1 character from the terminal perspective... Argh, that's definitely an issue:-( I assumed that one character whatever the encoding would be 1 character on the display. If it is not the case, I think I can put/compute this information in the translation structures that is use by PQmblen, and implement a PQmbtermlen function... Maybe you could point me some source of information about display lengths of characters depending on the encoding? What I mean by ASCII compatible is that spaces, new lines, carriage returns, tabs and NULL (C string terminaison) are one byte characters. This assumption seemed pretty safe to me. I think you can do it safely using PQmblen. Ok, what you describe is basically what I've done with the qidx computation as suggested by Tom Lane and then later I check that the encoded length is one to find my special characters. Thanks for you reply, -- Fabien Coelho - [EMAIL PROTECTED] ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] client side syntax error localisation for psql (v1)
PQmblen returns the storage size, which is not necessarily same as the character width reprensented in a terminal. For example for a kanji character in UTF-8 PQmblen returns 3, but it ocuppies 2 x ASCII character space, not x 3. Isn't that a problem for you? If I read you correctly, you mean that 1 character may take 3 bytes of storage in the string, but it is not guaranteed to be 1 character from the terminal perspective... Argh, that's definitely an issue:-( I assumed that one character whatever the encoding would be 1 character on the display. That's not correct... One thing I have to note is that some Asian characters such as Japanese, Chinese require twice the space on a terminal for each character comparing with plain ASCII characters. This is hard to explain to those who are not familiar with kanji... Could you take a look at included screen shot? As you can see there are four ASCII characters in the first line. On the second line there are *two* kanji characters and they occupy same space as above four ASCII characters. Moreover the strage size for the first line is 4, but the strage size for the second line may vary depending on the encoding. If the encoding is EUC_JP or SJIS, it takes 4 bytes, however it takes 6 bytes if the encoding is UTF-8. Got it? If it is not the case, I think I can put/compute this information in the translation structures that is use by PQmblen, and implement a PQmbtermlen function... Maybe you could point me some source of information about display lengths of characters depending on the encoding? I could write PQmbtermlen function for every encoding supported by PostgreSQL except UTF-8. Such kind of info for UTF-8 might be quite complex. I believe there are some mapping tables or functions to get such kind of info somewhere on the Internet, but I don't remember. I think you can do it safely using PQmblen. Ok, what you describe is basically what I've done with the qidx computation as suggested by Tom Lane and then later I check that the encoded length is one to find my special characters. Oh, ok. Thanks for you reply, You are welcome! -- Tatsuo Ishii ---(end of broadcast)--- TIP 8: explain analyze is your friend
Re: [HACKERS] The Name Game: postgresql.net vs. pgfoundry.org
Josh Berkus wrote: Folks, As we discussed a couple weeks ago, Marc, Andrew, Tim Perdue, Chris Ryan and I are testing implementing GForge in place of GBorg for associated projects for PostgreSQL. One thing which was suggested initially was that this new project hosting site be at www.postgresql.net with projects being projectname.postgresql.net. However, some of the porting team felt that it would be confusing for people who typed in www.postgresql.net to be presented with the GForge interface, and suggested that we use the domain after what we'll be calling the new Tool, namely pgFoundry, thus putting stuff at www.pgFoundry.org and projectname.pgfoundry.org. So far, only 4 people, total, have expressed opinons on the matter. I'm throwing this on Hackers so that members of projects we will be hosting can indicate whether they: A) Favor www.postgresql.net B) Favor www.pgfoundry.org C) Don't care as long as the porting is relatively painless. Isn't gforge a pgsql related project itself? So I'd suggest: www.postgresql.org - main PostgreSQL site gforge.postgresql.org - gforge interface site projectname.postgresql.org - gforge hosted projects Regards, Andreas ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] Log rotation
Tom Lane wrote: Fernando Nasser [EMAIL PROTECTED] writes: Please remind me again why the postmaster cannot close and open the log file when it receives a SIGHUP (to re-read configuration)? (a) Because it never opened it in the first place --- the log file is whatever was passed as stderr. (b) Because it would not be sufficient to make the postmaster itself close and reopen the file; every child process would have to do so also. Doing this in any sort of synchronized fashion seems impossible. Now I remember. Thanks for reminding me of that. It's much cleaner to have stderr be a pipe to some separate collector program that can handle log rotation (ie, the Apache solution). We could also create a pipe and start a new process (logger) and give it the other end of the pipe and the name of the log file. We could send it a SIGHUP after we reread the configuration file. But just doing a pipe on the OS level is way simpler. I don't really care on how its done, but IMO an enterprise class database must be able to do log rotation. Logging to Syslog is not an option (specially with our verbosity) -- users must be able to use flat files for logging. I never seem some many customer complaints and bug reports about a single item like the ones we have received here about logging. I think this should be the number 1 item in te TODO list. Thanks again for the clarifications. Regards, Fernando ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] client side syntax error localisation for psql (v1)
Fabien COELHO [EMAIL PROTECTED] writes: As a compromise, I can suggest the following: LINE 4: WHERE id=123 AND name LIKE 'calvin' GROP BY name... ^ That works for me. I don't mind it saying LINE 1: in the one-line case. It's not going to add more than about one strlen() call to what you need to do. I cannot strlen an integer value;-) I have to convert to a string, or deal directly with the line number. Well, what I'm imagining is that you sprintf(LINE %d) and then strlen that in the process of constructing the first line, and later add that length to the offset you need in the second line. Tatsuo's complaints about characters that span more than one position are a much nastier problem, in any case :-(. Can we cope? regards, tom lane ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: [HACKERS] The Name Game: postgresql.net vs. pgfoundry.org
-Original Message- From: Andreas Pflug [mailto:[EMAIL PROTECTED] Sent: 12 March 2004 13:57 To: [EMAIL PROTECTED] Cc: [EMAIL PROTECTED] Subject: Re: [HACKERS] The Name Game: postgresql.net vs. pgfoundry.org Isn't gforge a pgsql related project itself? So I'd suggest: www.postgresql.org - main PostgreSQL site gforge.postgresql.org - gforge interface site projectname.postgresql.org - gforge hosted projects The problem with that approach is that our 'official' sites then get lost amongst the project sites. We need some distinction between the core project sites and other project sites - istm that a different domain is the only way to do that. Regards, Dave. ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] client side syntax error localisation for psql (v1)
Dear Tatsuo, One thing I have to note is that some Asian characters such as Japanese, Chinese require twice the space on a terminal for each character comparing with plain ASCII characters. This is hard to explain to those who are not familiar with kanji... I learnt a little bit of chinese a few years ago, but I never saw the computer version. Could you take a look at included screen shot? I haven't found it. However I've made a little bit of trying with my emacs 21 mule demonstration, and indeed there is two different character widths... As you can see there are four ASCII characters in the first line. On the second line there are *two* kanji characters and they occupy same space as above four ASCII characters. Moreover the strage size for the first line is 4, but the strage size for the second line may vary depending on the encoding. If the encoding is EUC_JP or SJIS, it takes 4 bytes, however it takes 6 bytes if the encoding is UTF-8. Got it? Yep. Maybe you could point me some source of information about display lengths of characters depending on the encoding? I could write PQmbtermlen function for every encoding supported by PostgreSQL That would be great ! ;-) except UTF-8. Such kind of info for UTF-8 might be quite complex. I believe there are some mapping tables or functions to get such kind of info somewhere on the Internet, but I don't remember. That would be even greater;-) I guess a return 1 version would be a false quick fix that could be improved later on... -- Fabien COELHO. ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faqs/FAQ.html
Re: [HACKERS] The Name Game: postgresql.net vs. pgfoundry.org
On Fri, Mar 12, 2004 at 02:42:47PM -, Dave Page wrote: We need some distinction between the core project sites and other project sites - istm that a different domain is the only way to do that. Okay, then how about postgres-extra.net, or forpostgres.net? Saying Postgres instead of PostgreSQL takes out a bit of that extra length and it's lots easier to pronounce. We've been through this whole what- shall-we-call-it thing months ago and IIRC the upshot was that the short version of the name is perfectly acceptable and much catchier. Here's a chance to use it! Even shorter and catchier would be pgprojects.net IMHO, but that again stretches the connection with PostgreSQL. Jeroen ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] index leaks ?
strk [EMAIL PROTECTED] writes: The size growth is reported by 'top' in the fields SIZE, RSS and SHARE. Can it be a memory leak in postgres code ? No, you are misinterpreting the 'top' output. You didn't say what platform you are on, but on some systems 'top' increases the reported size of a backend process each time that process touches a page of the shared memory area that it had not touched before. In this case you are simply watching the process use shared buffers it hadn't previously touched. regards, tom lane ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] The Name Game: postgresql.net vs. pgfoundry.org
Dave Page wrote: -Original Message- From: Andreas Pflug [mailto:[EMAIL PROTECTED]] Sent: 12 March 2004 13:57 To: [EMAIL PROTECTED] Cc: [EMAIL PROTECTED] Subject: Re: [HACKERS] The Name Game: postgresql.net vs. pgfoundry.org Isn't gforge a pgsql related project itself? So I'd suggest: www.postgresql.org - main PostgreSQL site gforge.postgresql.org - gforge interface site projectname.postgresql.org - gforge hosted projects The problem with that approach is that our 'official' sites then get lost amongst the project sites. We need some distinction between the core project sites and other project sites - istm that a different domain is the only way to do that. (breaking previous rule) I agree. Also, the gforge people would prefer us *not* to use a name that includes gforge, because of the risk of confusion. That's how we came up with "pgfoundry" in the first place. cheers andrew
Re: [pgsql-www] [HACKERS] The Name Game: postgresql.net vs. pgfoundry.org
[ I'm pushing Robert's comment over into the pghackers thread... ] Robert Treat [EMAIL PROTECTED] writes: I wasn't going to force the issue just for my own sake... but ISTM Tom, Peter, myself and possibly others were all confused somewhat by the switch. Anyway... the only real point that I have about the whole thing is that people used to complain that gborg was too nebulous a name (ie. whats a gborg?) and people didnt know to look at it, or were confused as to what its purpose was. the idea of projects.postgresql.(org|net) seem like a real easy way to make it crystal clear as to what exactly was going on at that site. By making it pgfoundry.org, i guess it is clear as to its purpose as far as project hosting, but it loses some of its ties to postgresql, to the point where I think folks will wonder if this is an independent site or if it has the backing of the greater postgresql community. I tend to think that would be a step back... I think that last is really the crux of the issue. Josh observed that whatever the site name is, it will be the task of the advocacy group to market it with the correct public perception. But choosing the right name will surely make it easier to control the perception. What we're really arguing about here, IMHO, is the perceived distance between the domain names for the core project and the other projects. If they're too different then it will be very hard to get people to see the projects as related to PostgreSQL, no matter what marketing efforts we try. OTOH if they're too similar that may confuse things in other ways. My feeling is that we want people to consider these projects as closely tied to the Postgres community and so postgresql.something is just right. I can see there are different opinions out there though... regards, tom lane ---(end of broadcast)--- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] client side syntax error localisation for psql (v1)
Could you take a look at included screen shot? I haven't found it. However I've made a little bit of trying with my Oops. I have included this time. Maybe you could point me some source of information about display lengths of characters depending on the encoding? I could write PQmbtermlen function for every encoding supported by PostgreSQL That would be great ! ;-) Ok, I will work on this. except UTF-8. Such kind of info for UTF-8 might be quite complex. I believe there are some mapping tables or functions to get such kind of info somewhere on the Internet, but I don't remember. That would be even greater;-) I guess a return 1 version would be a false quick fix that could be improved later on... Agreeed. inline: kanji.png ---(end of broadcast)--- TIP 8: explain analyze is your friend
Re: [HACKERS] client side syntax error localisation for psql (v1)
On Sat, 13 Mar 2004, Tatsuo Ishii wrote: Oops. I have included this time. How ! a japanese vi ! I could write PQmbtermlen function for every encoding supported by PostgreSQL That would be great ! ;-) Ok, I will work on this. Thanks. -- Fabien Coelho - [EMAIL PROTECTED] ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: [HACKERS] The Name Game: postgresql.net vs. pgfoundry.org
Jeroen T. Vermeulen [EMAIL PROTECTED] writes: On Fri, Mar 12, 2004 at 02:42:47PM -, Dave Page wrote: We need some distinction between the core project sites and other project sites - istm that a different domain is the only way to do that. Okay, then how about postgres-extra.net, or forpostgres.net? Saying Postgres instead of PostgreSQL takes out a bit of that extra length and it's lots easier to pronounce. We've been through this whole what- shall-we-call-it thing months ago and IIRC the upshot was that the short version of the name is perfectly acceptable and much catchier. Here's a chance to use it! Well, if you want to think along those lines, I believe that we (PGDG) currently hold these domain names: postgresql.org postgresql.com postgresql.net postgres.org postgres.com It looks like some domain squatter has his tentacles on postgres.net :-(. We are not doing much with any of these except redirecting to postgresql.org. You could make a case that postgres.org for the projects would be the perfect complement to postgresql.org for the core. After looking at this list I'm sort of inclined to the idea that we should *not* use postgresql.net for much of anything ... that will just help drive traffic to that squatter at postgres.net. This also brings up the thought that if we do want to use pgfoundry.org, we'd better register pgfoundry.net and pgfoundry.com before someone else does. regards, tom lane ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [pgsql-www] [HACKERS] The Name Game: postgresql.net vs.
On Fri, 2004-03-12 at 10:37, Tom Lane wrote:
Jeroen T. Vermeulen [EMAIL PROTECTED] writes:
On Fri, Mar 12, 2004 at 02:42:47PM -, Dave Page wrote:
We need some distinction between the core project sites and other
project sites - istm that a different domain is the only way to do that.
Okay, then how about postgres-extra.net, or forpostgres.net?
Saying Postgres instead of PostgreSQL takes out a bit of that extra length
and it's lots easier to pronounce. We've been through this whole what-
shall-we-call-it thing months ago and IIRC the upshot was that the short
version of the name is perfectly acceptable and much catchier. Here's a
chance to use it!
Well, if you want to think along those lines, I believe that we (PGDG)
currently hold these domain names:
postgresql.org
postgresql.com
postgresql.net
postgres.org
postgres.com
It looks like some domain squatter has his tentacles on postgres.net
:-(. We are not doing much with any of these except redirecting to
postgresql.org.
You could make a case that postgres.org for the projects would be the
perfect complement to postgresql.org for the core.
After looking at this list I'm sort of inclined to the idea that we
should *not* use postgresql.net for much of anything ... that will just
help drive traffic to that squatter at postgres.net.
This also brings up the thought that if we do want to use pgfoundry.org,
we'd better register pgfoundry.net and pgfoundry.com before someone
else does.
yug... if we go with postgres.org|net|com we are just asking for the
press to keep referring to the product as postgres instead of
postgresql, so i'd strongly be against that idea.
Robert Treat
--
Build A Brighter Lamp :: Linux Apache {middleware} PostgreSQL
---(end of broadcast)---
TIP 8: explain analyze is your friend
Re: [HACKERS] The Name Game: postgresql.net vs. pgfoundry.org
On Fri, Mar 12, 2004 at 10:37:58AM -0500, Tom Lane wrote: Well, if you want to think along those lines, I believe that we (PGDG) currently hold these domain names: [...] postgres.org This is the one I was silently rooting for, but figured was too good to be true. You could make a case that postgres.org for the projects would be the perfect complement to postgresql.org for the core. Still _slightly_ confusing, but I think the plain and simple idea of a prominent banner was mentioned. We can have them both ways to avoid all confusion. I say go for it! Jeroen ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Re: [HACKERS] The Name Game: postgresql.net vs. pgfoundry.org
On Fri, 2004-03-12 at 10:14, Andrew Dunstan wrote:
Dave Page wrote:
-Original Message-
From: Andreas Pflug [ mailto:[EMAIL PROTECTED]
mailto:[EMAIL PROTECTED] ]
Sent: 12 March 2004 13:57
To: [EMAIL PROTECTED] mailto:[EMAIL PROTECTED]
Cc: [EMAIL PROTECTED] mailto:[EMAIL PROTECTED]
Subject: Re: [HACKERS] The Name Game: postgresql.net vs. pgfoundry.org
Isn't gforge a pgsql related project itself?
So I'd suggest:
www.postgresql.org http://www.postgresql.org- main PostgreSQL
site
gforge.postgresql.org - gforge interface site
projectname.postgresql.org - gforge hosted projects
The problem with that approach is that our 'official' sites then get
lost amongst the project sites.
We need some distinction between the core project sites and other
project sites - istm that a different domain is the only way to do that.
(breaking previous rule) I agree.
Also, the gforge people would prefer us *not* to use a name that
includes gforge, because of the risk of confusion. That's how we came up
with pgfoundry in the first place.
maybe pgsqlfoundry is a better compromise?
Robert Treat
--
Build A Brighter Lamp :: Linux Apache {middleware} PostgreSQL
---(end of broadcast)---
TIP 4: Don't 'kill -9' the postmaster
[HACKERS] vacuum log are difficult to read ...
Hi, I know that we have more and more details to show, during vacuum analyze ... but since v7.4.x it's not really easy to read quicly a vacuum analyze to see important points ... like elapsed time or number of tupples deleted... I would like to know first if you could make an effort about this ... only for human use ... ;o) And also if it's possible to have a report of the abnormal situations ... may be something like the point I had once, with the number of pages for the table really cheaper than the pages of my index for the same table ... Is it possible in a vacuum also to get informations about the quality of the parameters of the database .. for example the size of some parameters that are not in adequation with the data (pages, size of index etc.) we have in reality ... I don't know if my request is possible, but if so it could be a good opportunity for some tuning points, isn't it ? I hope I'm clear with my demand ;o) Regards, -- Hervé Piedvache Elma Ingénierie Informatique 6 rue du Faubourg Saint-Honoré F-75008 - Paris - France Pho. 33-144949901 Fax. 33-144949902 ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Performance and WAL on big inserts/updates
[EMAIL PROTECTED] writes: - Re uni-directional logs Of course. I forgot about PG's non-in-place update mechanisms and the use of VACCUUM .. with versioning there are really no undo logging necessary. I guess that means that during VACCUUM you might have to significant work in indexes ? I'm assuming that you never merge index pages. Yes, VACUUM has to delete dead index entries as well as dead heap tuples, and there are some fine points about making sure that happens in a safe order. I believe the current state of index space recovery is * btree: recycles emptied index pages via a freelist; can return empty pages to the OS if they're at the end of the index file, but will not move pages around to make it possible to return more empty pages. (This is all new behavior as of 7.4, before we didn't do anything about reclaiming dead space in btrees.) Does not try to merge partially-full pages (this is a possible future improvement but there are concurrency problems that would need to be solved). * hash: recycles empty pages via a freelist, never returns them to OS short of a REINDEX. I think it does merge partially-empty pages within each bucket chain. No provision for reducing the number of buckets if the index population shrinks (again, short of REINDEX). * rtree, gist: no page recycling that I know of, but I've not looked carefully. regards, tom lane ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [pgsql-www] [HACKERS] The Name Game: postgresql.net vs.
quote who=Tom Lane My feeling is that we want people to consider these projects as closely tied to the Postgres community and so postgresql.something is just right. I can see there are different opinions out there though... foundry.postgresql.org? ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [pgsql-www] [HACKERS] The Name Game: postgresql.net vs.
On Fri, Mar 12, 2004 at 10:43:34AM -0600, Thomas Swan wrote: foundry.postgresql.org? Been through that one... Too long when you have to add project name as well. Jeroen ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] PITR Functional Design v2 for 7.5
On Tuesday 09 March 2004 17:38, Simon Riggs wrote:
Richard Huxton
On Monday 08 March 2004 23:28, Simon Riggs wrote:
PITR Functional Design v2 for 7.5
Review of current Crash Recovery
Is there any value in putting this section on techdocs or similar? We
do
get a
small but steady trickle of people asking for details on internals,
and I
think this covers things in a different way to the WAL section of the
manuals.
Certainly, though I would like to do all of that after it actually
works!
Just getting caught up on this thread and had similar thoughts as to Richards.
If there are no objections, I'd like to put the first part of this email up
on techdocs as an explination of our current crash recovery system.
Robert Treat
--
Build A Brighter Lamp :: Linux Apache {middleware} PostgreSQL
---(end of broadcast)---
TIP 8: explain analyze is your friend
Re: [pgsql-www] [HACKERS] The Name Game: postgresql.net vs.
On Thu, 11 Mar 2004, Joshua D. Drake wrote: IMHO, the domain name isn't the make/break of whether going to GForge will succeed ... the success will be a matter of marketing it, and making sure that its project are well known ... personally, focusing on the domain is like focusing on the name of a car when you buy it, not on its features and/or price ... Really? What about BMW, Volvo or Mercedes? What about them? My point is that as long as we market/advertise the *site*, the URL to get there isn't going to matter to anyone ... only that they can find it ... its a branding issue, not a 'how to get there' issue ... hell, in most cases, ppl are going to click on the link from www.postgresql.org without even looking at what the URL itself is ... Sorry, car analogy was a bad one in that case :) Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email: [EMAIL PROTECTED] Yahoo!: yscrappy ICQ: 7615664 ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] The Name Game: postgresql.net vs. pgfoundry.org
Robert, maybe pgsqlfoundry is a better compromise? No, too long.People'd end up calling it pgFoundry anyway. Besides, Gavin Roy already designed us a nice pgFoundry logo. ;-) -- Josh Berkus Aglio Database Solutions San Francisco ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [pgsql-www] [HACKERS] The Name Game: postgresql.net vs.
On Fri, 2004-03-12 at 11:52, Jeroen T. Vermeulen wrote: On Fri, Mar 12, 2004 at 10:43:34AM -0600, Thomas Swan wrote: foundry.postgresql.org? Been through that one... Too long when you have to add project name as well. I don't understand why. Presumably the postgresql.org website will have a search for it, or it'll be a link, or it'll be a bookmark. How many people actually type in the full url anymore? Heck, when I goto the postgresql website I do a search in google for postgres and slam the I'm feeling lucky button. Having all PostgreSQL related material under one domain is beneficial to the project. Our big issue isn't the domain is too long, it is difficult find the subproject in the first place. ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Re: [HACKERS] [DEFAULT] Daily digest v1.4327 (22 messages)
Fernando, I don't really care on how its done, but IMO an enterprise class database must be able to do log rotation. Logging to Syslog is not an option (specially with our verbosity) -- users must be able to use flat files for logging. Hmmm ... to differ: I have several (six, actually) customers with high-demand databases. *All* of them use syslog.This is becuase there are tools for parsing, monitoring, and forwarding the syslog, which do not exist for individual application logs. Heck, you can even have a syslog server that collects the syslogs for several machines -- a recommended setup in busy mulit-server data centers. Futher, log rotation can be easily accomplished by piping the log output to a perl script rather than a file, and letting the script handle the re-direction. Some users are using the approach to, for example, create seperate logs for each connection or each type of statement. I never seem some many customer complaints and bug reports about a single item like the ones we have received here about logging. I think this should be the number 1 item in te TODO list. I disagree. This is one of those features that would be nice to have but isn't particularly important given the truly substantial number of external OSS tools designed to handle the problem. You might consider educating your customers about those tools instead -- several of which ship on the redhat CDs, I believe. -- Josh Berkus Aglio Database Solutions San Francisco ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] The Name Game: postgresql.net vs. pgfoundry.org
Tom Lane wrote: Well, if you want to think along those lines, I believe that we (PGDG) currently hold these domain names: postgresql.org postgresql.com postgresql.net postgres.org postgres.com It looks like some domain squatter has his tentacles on postgres.net :-(. We are not doing much with any of these except redirecting to postgresql.org. Looks like he hasn't been squatting all that long: Domain Name: POSTGRES.NET Created on..: Wed, Aug 07, 2002 Expires on..: Sat, Aug 07, 2004 Record last updated on..: Fri, Oct 31, 2003 Also note the expiration date. Maybe we can convince him to let us have the domain. Is it worth asking? You could make a case that postgres.org for the projects would be the perfect complement to postgresql.org for the core. After looking at this list I'm sort of inclined to the idea that we should *not* use postgresql.net for much of anything ... that will just help drive traffic to that squatter at postgres.net. Hmmm, perhaps you're right. Too bad, I was going to vote for postgresql.net myself. If we could get control of postgres.net that option would definitely get my vote. This also brings up the thought that if we do want to use pgfoundry.org, we'd better register pgfoundry.net and pgfoundry.com before someone else does. I agree with the others who have said pgfoundry.org is not clearly enough linked. Joe ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faqs/FAQ.html
Re: [HACKERS] The Name Game: postgresql.net vs. pgfoundry.org
David Garamond [EMAIL PROTECTED] writes:
Also, we're targetting the developers right? Please do not consider
ourselves as being too stupid to differentiate between postgresql.org and
postgresql.net...
I can never remember whether the current site is postgresql.{com,org,net} even
now. Making postgresql.net one thing and postgresql.org another thing is a
recipe for confusion.
I would say follow the same model as modules.apache.org, pear.php.net, etc.
I don't understand the too long complaint at all. a) Nobody's forcing us to
use a subdomain for each project, and b) nobody's actually going to type
slony.gborg.postgres.org or gborg.postgres.org/slony anyways. They'll go
to gborg.postgres.org and type slony into the search box.
In any case, complaining about too long when each component means something
specific only means you want to leave off one of those meanings. Either the
name of the project, the distinction from the main site, or the association
with postgres.
Ie, Using postgres.org vs postgres.net or leaves the user with no clue
which site he's on or even that there is another site. Using something like
pgfoundry.com leaves the user with no idea the web site is related to the
postgres.org web site.
--
greg
---(end of broadcast)---
TIP 5: Have you checked our extensive FAQ?
http://www.postgresql.org/docs/faqs/FAQ.html
Re: [HACKERS] The Name Game: postgresql.net vs. pgfoundry.org
Joe, Looks like he hasn't been squatting all that long: Domain Name: POSTGRES.NET Created on..: Wed, Aug 07, 2002 Expires on..: Sat, Aug 07, 2004 Record last updated on..: Fri, Oct 31, 2003 Also note the expiration date. Maybe we can convince him to let us have the domain. Is it worth asking? Hmmm ... please let Core handle this. -- Josh Berkus Aglio Database Solutions San Francisco ---(end of broadcast)--- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] The Name Game: postgresql.net vs. pgfoundry.org
Josh Berkus wrote: Looks like he hasn't been squatting all that long: Domain Name: POSTGRES.NET Created on..: Wed, Aug 07, 2002 Expires on..: Sat, Aug 07, 2004 Record last updated on..: Fri, Oct 31, 2003 Also note the expiration date. Maybe we can convince him to let us have the domain. Is it worth asking? Hmmm ... please let Core handle this. Sure -- that's why I posted the idea instead of calling or emailing myself ;-) Joe ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Log rotation
On Fri, Mar 12, 2004 at 09:24:28 -0500, Fernando Nasser [EMAIL PROTECTED] wrote: I don't really care on how its done, but IMO an enterprise class database must be able to do log rotation. Logging to Syslog is not an option (specially with our verbosity) -- users must be able to use flat files for logging. I never seem some many customer complaints and bug reports about a single item like the ones we have received here about logging. I think this should be the number 1 item in te TODO list. Are you suggesting the that postgres project develop their own logger rather than people just using one that has already been developed by some other group? ---(end of broadcast)--- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] Log rotation
Bruno Wolff III [EMAIL PROTECTED] writes: Are you suggesting the that postgres project develop their own logger rather than people just using one that has already been developed by some other group? The problem from the point of view of Red Hat is to not introduce a dependency from the Postgres RPM to the Apache RPM ... this is no problem for people who don't mind hand-customizing their setup, but it is a problem if you want it to be part of the out-of-the-box setup. regards, tom lane ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Timing of 'SELECT 1'
The problem with gprof is that I am going to see all the backend startup stuff too, no? Is there a way to get a dump just the run of the query? I was sort of lurking on this thread, waiting to see what became of it. Did nobody actually come to a conclusion on what that last msec was from? I think the consensus was it was coming from the network layer somehow. If that's the case (it probably is), there isn't a whole lot that can be done about it except to bypass it using server side functions and such. Merlin ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] Log rotation
On Fri, Mar 12, 2004 at 13:17:50 -0500, Tom Lane [EMAIL PROTECTED] wrote: Bruno Wolff III [EMAIL PROTECTED] writes: Are you suggesting the that postgres project develop their own logger rather than people just using one that has already been developed by some other group? The problem from the point of view of Red Hat is to not introduce a dependency from the Postgres RPM to the Apache RPM ... this is no problem for people who don't mind hand-customizing their setup, but it is a problem if you want it to be part of the out-of-the-box setup. I can see their problem with making a dependency to all of apache or including multilog in their distribution. But they probably could include something that is only a logger either using some project that is only a logger or splitting out the logger that is bundled with apache. Then it wouldn't be unreasonable to make a dependency for postgres requiring that logging rpm. Other services could also make use of this logging package as well. ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [pgsql-www] [HACKERS] The Name Game: postgresql.net vs.
On Fri, 12 Mar 2004, Tom Lane wrote: This also brings up the thought that if we do want to use pgfoundry.org, we'd better register pgfoundry.net and pgfoundry.com before someone else does. I did all three simultaneously for exactly that reason Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email: [EMAIL PROTECTED] Yahoo!: yscrappy ICQ: 7615664 ---(end of broadcast)--- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [pgsql-www] [HACKERS] The Name Game: postgresql.net vs.
On Fri, 12 Mar 2004, Rod Taylor wrote: On Fri, 2004-03-12 at 13:30, Marc G. Fournier wrote: On Fri, 12 Mar 2004, Rod Taylor wrote: Having all PostgreSQL related material under one domain is beneficial to the project. Our big issue isn't the domain is too long, it is difficult find the subproject in the first place. the projects site will not be under postgresql.org ... postgresql.net is available for it, but not postgresql.org ... we are keeping that domain clean for any future stuff we want to do with the core project ... I hope there is heavy integration between the two, otherwise anyone who doesn't read this forum will be very confused. there was never any question about integration between the two ... only the URL that ppl will go to to get to the projects pages ... Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email: [EMAIL PROTECTED] Yahoo!: yscrappy ICQ: 7615664 ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: [pgsql-www] [HACKERS] The Name Game: postgresql.net vs.
On Fri, 2004-03-12 at 13:30, Marc G. Fournier wrote: On Fri, 12 Mar 2004, Rod Taylor wrote: Having all PostgreSQL related material under one domain is beneficial to the project. Our big issue isn't the domain is too long, it is difficult find the subproject in the first place. the projects site will not be under postgresql.org ... postgresql.net is available for it, but not postgresql.org ... we are keeping that domain clean for any future stuff we want to do with the core project ... I hope there is heavy integration between the two, otherwise anyone who doesn't read this forum will be very confused. ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [pgsql-www] [HACKERS] The Name Game: postgresql.net vs.
Marc G. Fournier [EMAIL PROTECTED] writes: the projects site will not be under postgresql.org ... postgresql.net is available for it, but not postgresql.org ... we are keeping that domain clean for any future stuff we want to do with the core project ... I agree we don't want project.postgresql.org, as that is likely to risk name conflicts. However, that objection doesn't apply to project.projects.postgresql.org, or variants of that. So far the only objection I've heard to that sort of setup is the domain name is too long, and as others have pointed out, it's a weak objection. Since we do already own pgfoundry.org, could we satisfy everybody by dual-naming the project sites? That is, have both project.pgfoundry.org project.pgfoundry.postgresql.org point to the same place? regards, tom lane ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faqs/FAQ.html
Re: [HACKERS] The Name Game: postgresql.net vs. pgfoundry.org
On Fri, 12 Mar 2004, Greg Stark wrote: I would say follow the same model as modules.apache.org, pear.php.net, etc. note that having projects.postgresql.org is cool ... its just the projects subpages that I'm objecting too ... the easiest is to have http://projects.postgresql.org point to the same as http://www.postgresql.net, and then have all the projects piggy back on *.postgresql.net ... I have no hassles with that ... Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email: [EMAIL PROTECTED] Yahoo!: yscrappy ICQ: 7615664 ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Re: [pgsql-www] [HACKERS] The Name Game: postgresql.net vs.
On Fri, 12 Mar 2004 13:36:47 -0500 Tom Lane [EMAIL PROTECTED] wrote: Marc G. Fournier [EMAIL PROTECTED] writes: the projects site will not be under postgresql.org ... postgresql.net is available for it, but not postgresql.org ... we are keeping that domainclean for any future stuff we want to do with the core project ... I agree we don't want project.postgresql.org, as that is likely to risk name conflicts. However, that objection doesn't apply to project.projects.postgresql.org, or variants of that. So far the only objection I've heard to that sort of setup is the domain name is too long, and as others have pointed out, it's a weak objection. Since we do already own pgfoundry.org, could we satisfy everybody by dual-naming the project sites? That is, have both project.pgfoundry.org project.pgfoundry.postgresql.org point to the same place? My first vote would have been for postgresql.net, but I think project.projects.postgresql.org makes the most sense. If I wasn't in the know I wouldn't associate something.pgfoundry.(pgfoundry|postgresql).org with a PostgreSQL related projects by looking at the URL only. As for the length of the URL, I think any developer or user of PostgreSQL is knowledgeable enough to take advantage of browser bookmarks. :) I'm definitely against using 'pgfoundry.org' as I believe sub-projects should all fall under the currently used postgresql.org domain. Another thing to think about is search engine placement. Most search engines give higher listings to keywords that are in the domain name. While people will search for 'postgres' and/or 'postgresql' no one is going to come up with 'pgfoundry' on their own. - Frank Wiles [EMAIL PROTECTED] http://frank.wiles.org - ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [pgsql-www] [HACKERS] The Name Game: postgresql.net vs.
On Fri, 12 Mar 2004, Tom Lane wrote: Since we do already own pgfoundry.org, could we satisfy everybody by dual-naming the project sites? That is, have both project.pgfoundry.org project.pgfoundry.postgresql.org point to the same place? no objection here ... my only object is/was the length issue Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email: [EMAIL PROTECTED] Yahoo!: yscrappy ICQ: 7615664 ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [pgsql-www] [HACKERS] The Name Game: postgresql.net vs.
Tom, Since we do already own pgfoundry.org, could we satisfy everybody by dual-naming the project sites? That is, have both project.pgfoundry.org project.pgfoundry.postgresql.org point to the same place? Sounds good to me if it's doable via DNS. -- -Josh Berkus Aglio Database Solutions San Francisco ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faqs/FAQ.html
Re: [pgsql-www] [HACKERS] The Name Game: postgresql.net vs.
On Fri, 12 Mar 2004, Josh Berkus wrote: Tom, Since we do already own pgfoundry.org, could we satisfy everybody by dual-naming the project sites? That is, have both project.pgfoundry.org project.pgfoundry.postgresql.org point to the same place? Sounds good to me if it's doable via DNS. DNS wise its easy ... if anything, we could extend the 'dns gen' script we are using for the mirrors to auto-gen the DNS for the projects too ... Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email: [EMAIL PROTECTED] Yahoo!: yscrappy ICQ: 7615664 ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] Log rotation
On Friday 12 March 2004 09:24 am, Fernando Nasser wrote: I don't really care on how its done, but IMO an enterprise class database must be able to do log rotation. Logging to Syslog is not an option (specially with our verbosity) -- users must be able to use flat files for logging. Uh, we have many many many different ways to use syslog. So my other message on the topic. I never seem some many customer complaints and bug reports about a single item like the ones we have received here about logging. I think this should be the number 1 item in te TODO list. Uh, upgrading? I'm sure we have more reports about upgrading than logging. But see my reply to bug 103767 for more. -- Lamar Owen Director of Information Technology Pisgah Astronomical Research Institute 1 PARI Drive Rosman, NC 28772 (828)862-5554 www.pari.edu ---(end of broadcast)--- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [pgsql-www] [HACKERS] The Name Game: postgresql.net vs.
On Fri, 2004-03-12 at 13:36, Tom Lane wrote:
Marc G. Fournier [EMAIL PROTECTED] writes:
the projects site will not be under postgresql.org ... postgresql.net is
available for it, but not postgresql.org ... we are keeping that domain
clean for any future stuff we want to do with the core project ...
I agree we don't want project.postgresql.org, as that is likely to
risk name conflicts. However, that objection doesn't apply to
project.projects.postgresql.org, or variants of that. So far the only
objection I've heard to that sort of setup is the domain name is too
long, and as others have pointed out, it's a weak objection.
Since we do already own pgfoundry.org, could we satisfy everybody by
dual-naming the project sites? That is, have both
project.pgfoundry.org
project.pgfoundry.postgresql.org
point to the same place?
I hate to be the fly in this ointment, but wouldn't
project.projects.postgresql.org be better? especially if you could
then point people to projects.postgresql.org as the main place to start
looking for projects related to postgresql.
Robert Treat
--
Build A Brighter Lamp :: Linux Apache {middleware} PostgreSQL
---(end of broadcast)---
TIP 3: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to [EMAIL PROTECTED] so that your
message can get through to the mailing list cleanly
Re: [HACKERS] Log rotation
Bruno Wolff III wrote: I can see their problem with making a dependency to all of apache or including multilog in their distribution. But they probably could include something that is only a logger either using some project that is only a logger or splitting out the logger that is bundled with apache. Then it wouldn't be unreasonable to make a dependency for postgres requiring that logging rpm. Other services could also make use of this logging package as well. Yes that would be nice. I have no idea how difficult it would be to extricate the logrotate program from Apache. I also don't know if there would be any license restrictions, would we be able to redistribute it as an independently package? I don't know the answer. Regards, Fernando ---(end of broadcast)--- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] libpq thread safety
Bruce Momjian wrote:
What killed the idea of doing ssl or kerberos locking inside libpq was
that there was no way to be sure that outside code didn't also access
those routines.
A callback based implementation can handle that: libpq has a default
implementation for apps that do not use openssl or kerberos themself. If
the app wants to use the libraries, too, then it must replace the hooks
with their own locks.
I've attached a simple proposal, just for kerberos 4. If you agree on
the general approach, I'll add it to all functions that are not thread safe.
I have documented that SSL and Kerberos are not
thread-safe in the libpq docs. Let's wait and see If we need additional
work in this area.
It means that multithreading is not usable: As Tom explained, the
connect string is often set directly by the end user. Setting sslmode
would result is races - impossible to support. In the very least,
sslmode and Kerberos would have to fail if the app is multithreaded.
--
Manfred
Index: src/interfaces/libpq/fe-auth.c
===
RCS file: /projects/cvsroot/pgsql-server/src/interfaces/libpq/fe-auth.c,v
retrieving revision 1.89
diff -u -r1.89 fe-auth.c
--- src/interfaces/libpq/fe-auth.c 7 Jan 2004 18:56:29 - 1.89
+++ src/interfaces/libpq/fe-auth.c 12 Mar 2004 20:07:02 -
@@ -590,6 +590,7 @@
case AUTH_REQ_KRB4:
#ifdef KRB4
+ pglock_thread();
if (pg_krb4_sendauth(PQerrormsg, conn-sock,
(struct sockaddr_in *)
conn-laddr.addr,
(struct sockaddr_in *)
conn-raddr.addr,
@@ -597,8 +598,10 @@
{
snprintf(PQerrormsg, PQERRORMSG_LENGTH,
libpq_gettext(Kerberos 4 authentication
failed\n));
+ pgunlock_thread();
return STATUS_ERROR;
}
+ pgunlock_thread();
break;
#else
snprintf(PQerrormsg, PQERRORMSG_LENGTH,
@@ -722,6 +725,7 @@
if (authsvc == 0)
return NULL;/* leave original error message in
place */
+ pglock_thread();
#ifdef KRB4
if (authsvc == STARTUP_KRB4_MSG)
name = pg_krb4_authname(PQerrormsg);
@@ -759,5 +763,6 @@
if (name (authn = (char *) malloc(strlen(name) + 1)))
strcpy(authn, name);
+ pgunlock_thread();
return authn;
}
Index: src/interfaces/libpq/fe-connect.c
===
RCS file: /projects/cvsroot/pgsql-server/src/interfaces/libpq/fe-connect.c,v
retrieving revision 1.268
diff -u -r1.268 fe-connect.c
--- src/interfaces/libpq/fe-connect.c 10 Mar 2004 21:12:47 - 1.268
+++ src/interfaces/libpq/fe-connect.c 12 Mar 2004 20:07:03 -
@@ -3163,4 +3163,34 @@
#undef LINELEN
}
+/*
+ * To keep the API consistent, the locking stubs are always provided, even
+ * if they are not required.
+ */
+pgthreadlock_t *g_threadlock;
+static pgthreadlock_t default_threadlock;
+static void
+default_threadlock(bool acquire)
+{
+#if defined(ENABLE_THREAD_SAFETY)
+ static pthread_mutex_t singlethread_lock = PTHREAD_MUTEX_INITIALIZER;
+ if (acquire)
+ pthread_mutex_lock(singlethread_lock);
+ else
+ pthread_mutex_unlock(singlethread_lock);
+#endif
+}
+
+pgthreadlock_t *
+PQregisterThreadLock(pgthreadlock_t *newhandler)
+{
+ pgthreadlock_t *prev;
+
+ prev = g_threadlock;
+ if (newhandler)
+ g_threadlock = newhandler;
+ else
+ g_threadlock = default_threadlock;
+ return prev;
+}
Index: src/interfaces/libpq/libpq-fe.h
===
RCS file: /projects/cvsroot/pgsql-server/src/interfaces/libpq/libpq-fe.h,v
retrieving revision 1.102
diff -u -r1.102 libpq-fe.h
--- src/interfaces/libpq/libpq-fe.h 9 Jan 2004 02:02:43 - 1.102
+++ src/interfaces/libpq/libpq-fe.h 12 Mar 2004 20:07:03 -
@@ -274,6 +274,22 @@
PQnoticeProcessor proc,
void *arg);
+typedef void (pgsigpipehandler_t)(bool enable, void **state);
+
+extern pgsigpipehandler_t *
+PQregisterSigpipeCallback(pgsigpipehandler_t *newhandler);
+
+/*
+ * Used to set callback that prevents concurrent access to
+ * non-thread safe functions that libpq needs.
+ * The default implementation uses a libpq internal mutex.
+ * Only required for multithreaded apps that use kerberos
+ * both within their app and for postgresql connections.
+ */
+typedef void (pgthreadlock_t)(bool acquire);
+
+extern pgthreadlock_t *
Re: [HACKERS] Log rotation
Hi Lamar, Lamar Owen wrote: On Friday 12 March 2004 09:24 am, Fernando Nasser wrote: I don't really care on how its done, but IMO an enterprise class database must be able to do log rotation. Logging to Syslog is not an option (specially with our verbosity) -- users must be able to use flat files for logging. Uh, we have many many many different ways to use syslog. So my other message on the topic. Which other message? Anyway, Syslog is not an option for us. We need flat files. I never seem some many customer complaints and bug reports about a single item like the ones we have received here about logging. I think this should be the number 1 item in te TODO list. Uh, upgrading? I'm sure we have more reports about upgrading than logging. Yeah, but that only comes with a full version upgrade :-) The logging one keeps popping up as people try and use the server for production. But see my reply to bug 103767 for more. See my reply to your reply ;-) Best regards, Fernando ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Log rotation
On Fri, Mar 12, 2004 at 15:19:29 -0500, Fernando Nasser [EMAIL PROTECTED] wrote: Bruno Wolff III wrote: I can see their problem with making a dependency to all of apache or including multilog in their distribution. But they probably could include something that is only a logger either using some project that is only a logger or splitting out the logger that is bundled with apache. Then it wouldn't be unreasonable to make a dependency for postgres requiring that logging rpm. Other services could also make use of this logging package as well. Yes that would be nice. I have no idea how difficult it would be to extricate the logrotate program from Apache. I also don't know if there would be any license restrictions, would we be able to redistribute it as an independently package? I don't know the answer. I was suggesting this as something a distro maintainer (such as Redhat) could do. I think that the postgres developers shouldn't be spending time doing this. They should be just suggesting some possibilities in the admin part of the documentation. ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
[HACKERS] Stability of planner estimates given multiple redundant clauses
I've been looking into Paolo Tavalazzi's recent report of discrepancies
in the planner's estimates and resulting plan choices when the order of
FROM-clause entries is changed. Here is a boiled-down example:
paolo=# explain select * from seat, spettacoli, tran
paolo-# where tran.id = 42 and spettacoli.system = tran.system and
paolo-# tran.system = seat.system;
QUERY PLAN
---
Hash Join (cost=2220.72..231242.86 rows=9735500 width=408)
Hash Cond: (outer.system = inner.system)
- Seq Scan on seat (cost=0.00..11028.46 rows=362446 width=117)
- Hash (cost=2149.46..2149.46 rows=1703 width=291)
- Nested Loop (cost=0.00..2149.46 rows=1703 width=291)
- Index Scan using tran_id2_idx on tran (cost=0.00..10.00 rows=2
width=183)
Index Cond: (id = 42)
- Index Scan using spet_system_idx on spettacoli (cost=0.00..1065.98
rows=300 width=108)
Index Cond: (spettacoli.system = outer.system)
(9 rows)
paolo=# explain select * from seat, tran, spettacoli
paolo-# where tran.id = 42 and spettacoli.system = tran.system and
paolo-# tran.system = seat.system;
QUERY PLAN
Merge Join (cost=25710.35..255604.83 rows=14794210 width=408)
Merge Cond: (outer.system = inner.system)
- Sort (cost=16883.32..16926.76 rows=17375 width=300)
Sort Key: tran.system
- Hash Join (cost=10.00..13930.56 rows=17375 width=300)
Hash Cond: (outer.system = inner.system)
- Seq Scan on seat (cost=0.00..11028.46 rows=362446 width=117)
- Hash (cost=10.00..10.00 rows=2 width=183)
- Index Scan using tran_id2_idx on tran (cost=0.00..10.00
rows=2 width=183)
Index Cond: (id = 42)
- Sort (cost=8827.02..8967.22 rows=56079 width=108)
Sort Key: spettacoli.system
- Seq Scan on spettacoli (cost=0.00..1734.79 rows=56079 width=108)
(13 rows)
The reason for the difference in row-count estimates turns out to be the
redundant clause spettacoli.system = seat.system that is generated by
implied equality deduction. We generate this clause so that we can
investigate the alternative of joining spettacoli to seat first.
However, when we come to estimate the size of the three-way join
relation, we will have two redundant clauses associated with the join.
For example if the join path being considered is {{seat, tran}, spettacoli}
then both spettacoli.system = tran.system and spettacoli.system =
seat.system will be available join clauses. The planner understands
that the two clauses are redundant and shouldn't both be counted in
estimating the selectivity of the join. However, its choice of which
one it *should* count is arbitrary. It turns out to depend on
processing order and thereby on the order of the FROM items. In Paolo's
example, the estimates derived from the two clauses are different and so
the row counts and plan come out different.
It's not clear that this is a bug, exactly; it could be seen as the
inevitable consequence of planning uncertainties. But it's annoying.
As of CVS tip there is already some code in
remove_redundant_join_clauses() that chooses which of two redundant
clauses to eliminate on the basis of which one is cheaper to evaluate.
This doesn't affect most simple cases because all simple comparisons are
costed alike (one cpu_operator_cost). I am toying with the idea of
adding a further test to prefer the clause with smaller estimated
selectivity, if they are different. This might seem pretty ad hoc
but it's not completely out of the blue. I can prove mathematically
that the smaller selectivity is an upper bound for the correct value in
some restricted cases (eg, when keys are evenly distributed in each
table). I don't have a general proof, but hand-waving: the join of the
first two tables could only remove rows from their Cartesian product,
and estimating the selectivity of the 3-way join on the basis of either
individual join clause is like assuming that the first join is
Cartesian, so it's an upper bound for the correct value.
Comments? Anyone see any fatal problems, or have a better idea what to
do?
regards, tom lane
---(end of broadcast)---
TIP 7: don't forget to increase your free space map settings
[HACKERS] Should planner fold stable functions for estimation purposes?
I've been toying with the notion of allowing the planner to compute the current values of stable functions when it's trying to estimate selectivities. For instance, in a query like select ... where timestampcol = now() - interval '1 day'; we currently throw up our hands and treat the righthand side as an unknown quantity for estimation purposes, which leads to selection of a very conservative default selectivity estimate. That often discourages the planner from selecting an indexscan, and can lead to unreasonably slow join choices at upper levels of the plan. It would not be correct to reduce the righthand side to a constant in advance of execution, of course, but is it reasonable to compute its current value solely for purposes of comparison to column statistics? The risk we take if we do so is that the estimate we thereby derive could be stale by the time the generated plan is used, and in the worst case the plan could be really inappropriate. On the other hand, in most of the practical examples that I've seen, the current planner behavior is producing a pretty inappropriate plan. A possibly useful compromise is to do this reduction only in scalarineqsel, where not having any comparison value is really a serious blow, and not risk it in eqsel, where we can often generate a not-too-awful estimate without any specific comparison value. Comments? regards, tom lane ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Should planner fold stable functions for estimation purposes?
Rod Taylor [EMAIL PROTECTED] writes: It would not be correct to reduce the righthand side to a constant in advance of execution, of course, but is it reasonable to compute its current value solely for purposes of comparison to column statistics? So this means it would be double evaluated? A flag will be required to prevent this for functions that do more than just return a value or have a high cost in execution. Functions with side-effects had better be marked volatile anyway, so I'm not worried about that case. As for the expense argument, keep in mind that the one extra evaluation in the planner is likely to save you an awful lot of evaluations at runtime, if it convinces the planner to use an indexscan and not a seqscan. We are after all talking about functions appearing in WHERE, and I wouldn't think that people can reasonably expect those to get evaluated just once. regards, tom lane ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] Log rotation
Andrew Dunstan [EMAIL PROTECTED] writes: Did anything ever come from this thread? http://archives.postgresql.org/pgsql-hackers/2003-05/msg00603.php (Heading: Plan B for log rotation support: borrow Apache code) Only an entry on my depressingly long personal to-do list :-( I did take a look at the Apache rotator program, and found that it was probably more trouble to adopt than it's worth. It seemed to depend on a lot of configuration and library-routine infrastructure that we don't share. (No big surprise; I suppose someone trying to pull out a random bit of our backend code would be at least as unhappy.) I suspect it would be less trouble, as well as legalistically cleaner, to write our own from scratch. Andrew Sullivan offered Afilias' rotator script awhile back also. I think that works fine if you like a Perl script. regards, tom lane ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faqs/FAQ.html
Re: [pgsql-www] [HACKERS] The Name Game: postgresql.net vs.
On Fri, Mar 12, 2004 at 01:02:00PM -0600, Frank Wiles wrote: As for the length of the URL, I think any developer or user of PostgreSQL is knowledgeable enough to take advantage of browser bookmarks. :) I've heard this said a several times now, but that doesn't make me feel any better. I frequently find myself in situations where I *must* get to my project site from a memorized URL, and clicking through to it is a luxury I can ill afford. I travel. Sometimes I'm dependent on slow lines and/or other people's machines. For instance, whether I will be able to respond promptly to new support requests and bug reports over the entire month of May this year will depend partly on that ability. Apart from that, we could do with some public attention and that's where catchiness matters. Jeroen ---(end of broadcast)--- TIP 8: explain analyze is your friend
