Re: [HACKERS] pgFoundry
Tom Lane wrote: We did do that (not very rigorously) during the 7.4 release cycle. I'm not sure why we fell out of the habit again for 8.0. It seems like a reasonable idea to me. In the past I have suggested incrementally maintaining release.sgml (or some plaintext version of it), rather than having Bruce generate the release notes from a single sweep through the CVS logs prior to a release. The current process can easily lose information: Bruce needs to make a snap decision about which changes are relevant, and it's not always easy to make that decision correctly. It also means the person who implemented a feature (and therefore knows the problem well) is not writing the release notes for it. And it means the release notes are always at least a little bit behind the times. IIRC, the previous discussion petered out when Bruce said he prefers the current process. -Neil ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] SQL99 hierarchical queries stalled
On T, 2005-05-17 at 00:42 -0400, Tom Lane wrote: David Fetter [EMAIL PROTECTED] writes: What's the next step? I suppose the first thing would be to look over the patches I mentioned and the SQL:2003 specification, then put together a preliminary patch and send it to -hackers. ... I seem to recall some discussion of how to do this in the past; have you trolled the pghackers archives? I think that Jasons inspiration for doing it came from the the fact that there are already now abandoned patches for doing it. So studying/understanding the current patch, and describing and getting feedback from pgsql-hackers should be quite a good way of gaining insight in trickier parts of postgres. So it will not be jumping at new problem and writing a patch, but rather trying to get an existing patch into good shape for being accepted in the backend. -- Hannu Krosing [EMAIL PROTECTED] ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] Returning the name of a primary key
Tom, Juan, Wouldn't this simple SQL do the trick? CREATE OR REPLACE FUNCTION pk_column(text) RETURNS SETOF text AS ' SELECT attname::text FROM pg_class, pg_constraint, pg_attribute WHERE pg_class.oid = conrelid AND contype=''p'' AND attrelid = pg_class.oid AND attnum = ANY (conkey) AND relname=$1; ' LANGUAGE sql VOLATILE STRICT; -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Tom Lane Sent: Tuesday, May 17, 2005 4:49 AM To: Juan Pablo Espino Cc: pgsql-hackers@postgresql.org Subject: Re: [HACKERS] Returning the name of a primary key Juan Pablo Espino [EMAIL PROTECTED] writes: I need to write a function that retrieve the name of at least one table primary key, if it exists. The only argument passed to the function is the table name. I have thought something like this: You need to be searching the list of indexes, not the attributes per se. ATExecDropNotNull() might be a useful example. regards, tom lane ---(end of broadcast)--- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: Learning curves and such (was Re: [HACKERS] pgFoundry)
On T, 2005-05-17 at 01:32 -0400, Tom Lane wrote: As against that I notice some new arrivals proposing to add deductive reasoning to Postgres: http://archives.postgresql.org/pgsql-hackers/2005-05/msg01045.php or implement SQL99 hierarchical queries: http://archives.postgresql.org/pgsql-hackers/2005-05/msg01089.php I guess you have missed most of the latest discussion around hierarchical queries ;) From what I understand, what is proposed is a code beautification project, (although likely not minor :) , because the pathes have been around (and allegedly in production use) for a few years already, originally supporting Oracle-style HQs and then, for about a year also subset of SQL99 (it misses SEARCH and CYCLE clauses, see the railroad diagram at http://gppl.moonbone.ru/with_clause.gif ) The patch is available at http://gppl.moonbone.ru/index.shtml I might be wrong, but I'll bet lunch that neither of those projects will come to anything. You can't run before you learn to crawl. Maybe you could take a look at the existing patch and comment what are the points that are most wrong with it. The last one was for 8.0.1. I'm sure someone more at home with pg internals would get the patch to acceptable level faster, but my feeling is that somehow these patches have been just not interesting to core developers. Maybe what we need is some documentation about how to get started as a Postgres hacker --- what to read, what sort of things to tackle for your first hack, etc. I think the people who have been successful around here are the ones who have managed to figure out the syllabus by themselves ... but surely we could try to teach those who come after. A code documentation or beautification project is usually a good way to get newcomers up to speed. And though the getting the the HQ patch into acceptable shape is probably quite big work, just starting with understanding and documenting what it does now and getting further help from pgsql-hackers on what it should do may be something that is possible without existing thorough understanding. -- Hannu Krosing [EMAIL PROTECTED] ---(end of broadcast)--- TIP 8: explain analyze is your friend
Re: Learning curves and such (was Re: [HACKERS] pgFoundry)
I think it might also be valuable to have a list of things that are good 'starter projects'. Maybe some indication of TODO items that are simpler. Possibly a list of bugs, too. As someone who has looked at hacking the pg code, I agree it is difficult to know what to look at to get started. I love fixing bugs, but I'm used to the bug tracker type situation and being able to find something that looks relatively easy. That is how I've started on a number of other projects. With no formal bug tracker, I'm not sure what bugs I could look at. I have talked to some people on IRC about tackling the infinite date issue, but was warned off that, as the date/time code is a scary place. Then I looked into the possibility of working on the autovacuum stuff, and again got myself in a little too deep. Not low enough fruit for me. The curve to get the meaning of some things is sometimes hard PG_TRY and things like that just need reading code lots. Not meaning to attack Tom, but for new people proposing new patches he can seem intimidating. I have been around for a little while now and am adjusting to the reality. Maybe people shouldn't be hacking the code before being here a year. It would probably also be useful to point out ways people can help that don't involve hacking C code (or at least not much). Things like docs, the website, advocacy, performance testing, etc. It would be useful to outline positions that are actually available for people to take. It's easy to give a general list. I've asked and seen may like it. For me, what does helping with advocacy mean? What should be performance tested (I assume new code, like the bitmap scan). But at the same time, how do I not get into something that is being duplicated by somebody else? I'm just trying to give a bit of a perspective on the way I see things as some who has been around pg for about 12 months and attempted to hack the code a couple of times. Regards Russell Smith ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: Learning curves and such (was Re: [HACKERS] pgFoundry)
On Tue, May 17, 2005 at 09:23:42PM +1000, Russell Smith wrote: I think it might also be valuable to have a list of things that are good 'starter projects'. Maybe some indication of TODO items that are simpler. Possibly a list of bugs, too. As someone who has looked at hacking the pg code, I agree it is difficult to know what to look at to get started. I love fixing bugs, but I'm used to the bug tracker type situation and being able to find something that looks relatively easy. That is how I've started on a number of other projects. With no formal bug tracker, I'm not sure what bugs I could look at. I have talked to some people on IRC about tackling the infinite date issue, but was warned off that, as the date/time code is a scary place. I'd say the datetime code is a good place to start, the most important characteristic being that it's self contained. It would be useful to outline positions that are actually available for people to take. It's easy to give a general list. I've asked and seen may like it. For me, what does helping with advocacy mean? What should be performance tested (I assume new code, like the bitmap scan). Yes, performance testing may also show some implementation bugs that are important to find too. Stress-testing is important too. Or find corner cases, push implementation limits, etc. Or, find some area that people mentions as needing testing, the current example being SIGTERM handling in busy backends. But at the same time, how do I not get into something that is being duplicated by somebody else? Tell -hackers. But if you are going to do testing, it doesn't matter there is multiple people doing it. -- Alvaro Herrera (alvherre[a]surnet.cl) The first of April is the day we remember what we are the other 364 days of the year (Mark Twain) ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] Cost of XLogInsert CRC calculations
(BI tested crctest in two machines and two versions of gcc. (B (BUltraSPARC III, gcc 2.95.3: (Bgcc -O1 crctest.c 1.321517 s (Bgcc -O2 crctest.c 1.099186 s (Bgcc -O3 crctest.c 1.099330 s (Bgcc -O1 crctest64.c 1.651599 s (Bgcc -O2 crctest64.c 1.429089 s (Bgcc -O3 crctest64.c 1.434296 s (B (BUltraSPARC III, gcc 3.4.3: (Bgcc -O1 crctest.c 1.209168 s (Bgcc -O2 crctest.c 1.206253 s (Bgcc -O3 crctest.c 1.209762 s (Bgcc -O1 crctest64.c 1.545899 s (Bgcc -O2 crctest64.c 1.545290 s (Bgcc -O3 crctest64.c 1.540993 s (B (BPentium III, gcc 2.95.3: (Bgcc -O1 crctest.c 1.548432 s (Bgcc -O2 crctest.c 1.226873 s (Bgcc -O3 crctest.c 1.227699 s (Bgcc -O1 crctest64.c 1.362152 s (Bgcc -O2 crctest64.c 1.259324 s (Bgcc -O3 crctest64.c 1.259608 s (B (BPentium III, gcc 3.4.3: (Bgcc -O1 crctest.c 1.084822 s (Bgcc -O2 crctest.c 0.921594 s (Bgcc -O3 crctest.c 0.921910 s (Bgcc -O1 crctest64.c 1.188287 s (Bgcc -O2 crctest64.c 1.242013 s (Bgcc -O3 crctest64.c 1.638812 s (B (BI think that it can improve the performance by loop unrolling. (BI measured the performance when the loop unrolled by -funroll-loops (Boption or hand-tune. (hand-tune version is attached.) (B (BUltraSPARC III, gcc 2.95.3: (Bgcc -O2 crctest.c 1.098880 s (Bgcc -O2 -funroll-loops crctest.c0.874165 s (Bgcc -O2 crctest_unroll.c0.808208 s (B (BUltraSPARC III, gcc 3.4.3: (Bgcc -O2 crctest.c 1.209168 s (Bgcc -O2 -funroll-loops crctest.c1.127973 s (Bgcc -O2 crctest_unroll.c1.017485 s (B (BPentium III, gcc 2.95.3: (Bgcc -O2 crctest.c 1.226873 s (Bgcc -O2 -funroll-loops crctest.c1.077475 s (Bgcc -O2 crctest_unroll.c1.051375 s (B (BPentium III, gcc 3.4.3: (Bgcc -O2 crctest.c 0.921594 s (Bgcc -O2 -funroll-loops crctest.c0.873614 s (Bgcc -O2 crctest_unroll.c0.839384 s (B (Bregards, (B (B--- (B (BAtsushi Ogawa crctest.tar.gz Description: Binary data ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
[HACKERS] SQL99 hierarchical queries stalled
David-- My boss has given me approval to put up to 8 hours per week of SourceLabs' time in on the SQL99 hierarchical query implementation. (I'm free, of course, to supplement this with whatever of my own time I can spare.) I'm willing to take on the work. What's the next step? --Jason ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: [HACKERS] Cost of XLogInsert CRC calculations
-Original Message- From: Tom Lane [mailto:[EMAIL PROTECTED] Sent: 16 May 2005 17:36 To: Mark Cave-Ayland (External) Cc: pgsql-hackers@postgresql.org Subject: Re: [HACKERS] Cost of XLogInsert CRC calculations (cut) I did some experimentation and concluded that gcc is screwing up big-time on optimizing the CRC64 code for 32-bit Intel. It does much better on every other architecture though. Hi Tom, Thanks very much for showing that the unint64 slowdown was caused by the optimisation done by gcc - I've had a go at filing a bug with the gcc people at http://gcc.gnu.org/bugzilla/show_bug.cgi?id=21617 so it would be interesting see if they can solve this. Perhaps like you suggest, the short term solution is to use the uint32 CRC64 code everywhere at the moment. I hope to find some time later this week to write and test some CRC32 routines, and will post the results back to the list. Many thanks, Mark. WebBased Ltd 17 Research Way Plymouth PL6 8BT T: +44 (0)1752 797131 F: +44 (0)1752 791023 W: http://www.webbased.co.uk ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: Learning curves and such (was Re: [HACKERS] pgFoundry)
It would be useful to outline positions that are actually available for people to take. It's easy to give a general list. I've asked and seen may like it. For me, what does helping with advocacy mean? What should be performance tested (I assume new code, like the bitmap scan). But at the same time, how do I not get into something that is being duplicated by somebody else? I reckon a good newbie task at the moment would be to add ALTER object SET SCHEMA blah; on all objects... Chris ---(end of broadcast)--- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: Learning curves and such (was Re: [HACKERS] pgFoundry)
Russell Smith wrote: Maybe people shouldn't be hacking the code before being here a year. If you want to code, jump in. It is a practical craft that you only learn by doing. You might learn something of the people but you'll probably learn precious little of the code by just watching. But don't jump in at the deep end - massive reorganisation of the planner should not be your first project ;-). Ask lots of questions. Tell people what you're doing. I like the idea of a relatively simple low hanging fruit list that we could point newcomers at. Here are some nominations: . add some more tests for the PLs now we have a standard testing framework. . clean up and document some of the contrib modules so they can go into the core (e.g. pgcrypto, dbsize, cube, seg) . rewrite contrib/reindexdb in C so it can be used on Windows . this TODO item: Remove Money type, add money formatting for decimal type I'm sure other people will have additions to such a list. cheers andrew ---(end of broadcast)--- TIP 8: explain analyze is your friend
Re: Learning curves and such (was Re: [HACKERS] pgFoundry)
Tom Lane [EMAIL PROTECTED] Date: Tue, 17 May 2005 01:32:03 -0400 As against that I notice some new arrivals proposing to add deductive reasoning to Postgres: http://archives.postgresql.org/pgsql-hackers/2005-05/msg01045.php or implement SQL99 hierarchical queries: http://archives.postgresql.org/pgsql-hackers/2005-05/msg01089.php I might be wrong, but I'll bet lunch that neither of those projects will come to anything. You can't run before you learn to crawl. The main advantage of deductive reasoning implementation I propose is in fundamental extension of database query language, I don't propose to extent SQL with some cumbersome constructions, for example, like WITH to express recursive queries (inefficiency of such constructions can be easily seen if you try to express a bit more complex recursive query then finding ancestors, e.g. query which finds minimal paths). Also it should be mentioned that original query language (SQL) de facto remains without great changes, the logic program which defines predicates (intensional relations) is located in the system table (there can be put the name and both the text and inner code of logic program). When we want to get intensional relation we simply write in SQL query the name of the logic program (deductive database) and the name of the predicate with the list of arguments. This syntax is identical to the syntax of table function calling with the schema name. Regards, Dmitriy ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] SQL99 hierarchical queries stalled
Hannu Krosing [EMAIL PROTECTED] writes: On T, 2005-05-17 at 00:42 -0400, Tom Lane wrote: I seem to recall some discussion of how to do this in the past; have you trolled the pghackers archives? I think that Jasons inspiration for doing it came from the the fact that there are already now abandoned patches for doing it. Having looked over the latest patch, my advice would be to ignore it :-( It's almost completely devoid of documentation, except for comments that he copied-and-pasted from elsewhere without modification. Wrong comments are even worse than none. What I'd like to see before anyone writes a line of code is a text document explaining how this is going to work: what's the plan tree structure, what happens at execution time, how much of the SQL99 spec is going to get implemented. If you don't have that understanding first, you're going to get buried in irrelevant details. regards, tom lane ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] SQL99 hierarchical queries stalled
Tom Lane wrote: Hannu Krosing [EMAIL PROTECTED] writes: On T, 2005-05-17 at 00:42 -0400, Tom Lane wrote: I seem to recall some discussion of how to do this in the past; have you trolled the pghackers archives? I think that Jasons inspiration for doing it came from the the fact that there are already now abandoned patches for doing it. Having looked over the latest patch, my advice would be to ignore it :-( It's almost completely devoid of documentation, except for comments that he copied-and-pasted from elsewhere without modification. Wrong comments are even worse than none. What I'd like to see before anyone writes a line of code is a text document explaining how this is going to work: what's the plan tree structure, what happens at execution time, how much of the SQL99 spec is going to get implemented. If you don't have that understanding first, you're going to get buried in irrelevant details. I have updated the developer's FAQ to cover these suggestions on how to start a patch: 1.4) What do I do after choosing an item to work on? Send an email to pgsql-hackers with a proposal for what you want to do (assuming your contribution is not trivial). Working in isolation is not advisable because others might be working on the same TODO item, or you might have misunderstood the TODO item. In the email, discuss both the internal implementation method you plan to use, and any user-visible changes (new syntax, etc). For complex patches, it is important to get community feeback on your proposal before starting work. Failure to do so might mean your patch is rejected. -- Bruce Momjian| http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
[HACKERS] ARRAY[] with \'s is broken?
Can anyone explain what's going on with the slashes?
ssdb=# select version();
version
---
PostgreSQL 8.0.1 on sparc-sun-solaris2.9, compiled by GCC gcc (GCC)
3.4.2
(1 row)
ssdb=# select ARRAY['\\a'];
array
-
{\\a}
(1 row)
ssdb=# select ARRAY['\a'];
array
---
{a}
(1 row)
ssdb=# select ARRAY['\\\a'];
array
-
{\\a}
(1 row)
--
---(end of broadcast)---
TIP 3: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to [EMAIL PROTECTED] so that your
message can get through to the mailing list cleanly
Re: [HACKERS] alternate regression dbs?
Andrew Dunstan wrote: Anyway, see http://archives.postgresql.org/pgsql-patches/2005-05/msg00179.php Sorry for the delay -- I'm on the final stretch of a major project at work. No objections from me. Joe ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: Learning curves and such (was Re: [HACKERS] pgFoundry)
Russell, What should be performance tested (I assume new code, like the bitmap scan). I've been meaning to put a task list for performance testing up on the TestPerf project. Yet another personal TODO ... -- Josh Berkus Aglio Database Solutions San Francisco ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] ARRAY[] with \'s is broken?
I don't see anything broken. V 7.4.8 shows the same:
andrew=# select ARRAY['\\a'] as f1, ARRAY['\a']as f2 , ARRAY['\\\a'] as f2;
f1| f2 | f2
+-+-
{\\a} | {a} | {\\a}
It might be mildly confusing because '\a' == 'a'
cheers
andrew
Rod Taylor wrote:
Can anyone explain what's going on with the slashes?
ssdb=# select version();
version
---
PostgreSQL 8.0.1 on sparc-sun-solaris2.9, compiled by GCC gcc (GCC)
3.4.2
(1 row)
ssdb=# select ARRAY['\\a'];
array
-
{\\a}
(1 row)
ssdb=# select ARRAY['\a'];
array
---
{a}
(1 row)
ssdb=# select ARRAY['\\\a'];
array
-
{\\a}
(1 row)
---(end of broadcast)---
TIP 8: explain analyze is your friend
Re: [HACKERS] Cost of XLogInsert CRC calculations
On Mon, 16 May 2005, Tom Lane wrote: I did some experimentation and concluded that gcc is screwing up big-time on optimizing the CRC64 code for 32-bit Intel. It does much better on every other architecture though. Anyone want to try it with non-gcc compilers? Solaris 9 x86 - Sun Workshop 6 update 2 C 5.3, gcc 3.2.3 gcc -O1 crctest.c .251422 gcc -O3 crctest.c .240223 gcc -O1 crctest64.c.281369 gcc -O3 crctest64.c.631290 cc -O crctest.c.268905 cc -fast crctest.c .242429 cc -O crctest64.c .283278 cc -fast crctest64.c .255560 Kris Jurka ---(end of broadcast)--- TIP 8: explain analyze is your friend
Re: [HACKERS] ARRAY[] with \'s is broken?
On Tue, 2005-05-17 at 13:18 -0400, Andrew Dunstan wrote:
I don't see anything broken. V 7.4.8 shows the same:
How the heck do you store a single backslash in an text array?
andrew=# select ARRAY['\\a'] as f1, ARRAY['\a']as f2 , ARRAY['\\\a'] as f2;
f1| f2 | f2
+-+-
{\\a} | {a} | {\\a}
It might be mildly confusing because '\a' == 'a'
cheers
andrew
Rod Taylor wrote:
Can anyone explain what's going on with the slashes?
ssdb=# select version();
version
---
PostgreSQL 8.0.1 on sparc-sun-solaris2.9, compiled by GCC gcc (GCC)
3.4.2
(1 row)
ssdb=# select ARRAY['\\a'];
array
-
{\\a}
(1 row)
ssdb=# select ARRAY['\a'];
array
---
{a}
(1 row)
ssdb=# select ARRAY['\\\a'];
array
-
{\\a}
(1 row)
--
---(end of broadcast)---
TIP 2: you can get off all lists at once with the unregister command
(send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Re: Learning curves and such (was Re: [HACKERS] pgFoundry)
As someone who's been lurking on the postgres lists for quite some time, and submitted one (minor) patch, I think the notion of an entry-level task list for we beginners is fantastic. And, I'm sure this has been asked and answered a billion times already, but why *don't* we have a real bug tracking system? BJ ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: [HACKERS] ARRAY[] with \'s is broken?
Rod Taylor [EMAIL PROTECTED] writes:
On Tue, 2005-05-17 at 13:18 -0400, Andrew Dunstan wrote:
I don't see anything broken. V 7.4.8 shows the same:
How the heck do you store a single backslash in an text array?
You just did. array_out doubles the backslashes again, though.
regression=# select ARRAY['\\a'];
array
-
{\\a}
(1 row)
regression=# select (ARRAY['\\a'])[1];
array
---
\a
(1 row)
regards, tom lane
---(end of broadcast)---
TIP 2: you can get off all lists at once with the unregister command
(send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Re: [HACKERS] ARRAY[] with \'s is broken?
How the heck do you store a single backslash in an text array?
... array_out doubles the backslashes again, though.
That explains how I got confused.
regression=# select ARRAY['\\a'];
array
-
{\\a}
(1 row)
regression=# select (ARRAY['\\a'])[1];
array
---
\a
(1 row)
regards, tom lane
--
---(end of broadcast)---
TIP 5: Have you checked our extensive FAQ?
http://www.postgresql.org/docs/faq
Re: Learning curves and such (was Re: [HACKERS] pgFoundry)
On Wed, 18 May 2005, Brendan Jurd wrote: As someone who's been lurking on the postgres lists for quite some time, and submitted one (minor) patch, I think the notion of an entry-level task list for we beginners is fantastic. And, I'm sure this has been asked and answered a billion times already, but why *don't* we have a real bug tracking system? Because none of the core developers will use it, so bugs would be added, but never removed ... Also, how many 'bugs' have we seen go through the lists that someone hasn't jump'd on and fixed in a couple of days? We have a long list of 'TODO' items, but could anyone generate a list of known bugs? Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email: [EMAIL PROTECTED] Yahoo!: yscrappy ICQ: 7615664 ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] ARRAY[] with \'s is broken?
Rod Taylor wrote: On Tue, 2005-05-17 at 13:18 -0400, Andrew Dunstan wrote: I don't see anything broken. V 7.4.8 shows the same: How the heck do you store a single backslash in an text array? Well, in v 8.0 I use dollar quoting :-) select ARRAY[$\$]; But as Tom says the text output function will double it. cheers andrew ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Re: Learning curves and such (was Re: [HACKERS] pgFoundry)
Marc G. Fournier wrote: And, I'm sure this has been asked and answered a billion times already, but why *don't* we have a real bug tracking system? Because none of the core developers will use it, so bugs would be added, but never removed ... Last time it came up I thought the problem was that there was not a consensus on *which* bugtracker to use. Incidentally, I'm not advocating we use bugzilla (if anything I think I'd lean towards using RT), but this seems like a good opportunity to note that as of a week or two ago bugzilla's HEAD branch supports using PostgreSQL as its backing store, and this will be maintained. Also, how many 'bugs' have we seen go through the lists that someone hasn't jump'd on and fixed in a couple of days? We have a long list of 'TODO' items, but could anyone generate a list of known bugs? Bug tracking systems are used to track more than just bugs ... they are often used to track enhancements, support requests, and other tasks. GForge (and hence pgfoundry) provides each project by default with several trackers, one for each of these classes. But then, as a pgfoundry admin you know that, right? :-) cheers andrew ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Re: Learning curves and such (was Re: [HACKERS] pgFoundry)
On Tue, 17 May 2005, Andrew Dunstan wrote: Last time it came up I thought the problem was that there was not a consensus on *which* bugtracker to use. The key requirement that has always come up is that the core developers wouldn't use anything web based, so the tracker would have to somehow tie into the mailing lists themselves ... Bug tracking systems are used to track more than just bugs ... they are often used to track enhancements, support requests, and other tasks. GForge (and hence pgfoundry) provides each project by default with several trackers, one for each of these classes. But then, as a pgfoundry admin you know that, right? :-) Again, it comes down to who will maintain it? I believe that Bruce has already stated that he hasn't got the time/desire to do much more then his current TODO lists, Tom has stated a lack of desire to use a web-based tracking tool ... so we'd need to find someone with the time to act as intermediary to update things accordingly ... ... and I think *that* is probably one of hte major show stoppers ... Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email: [EMAIL PROTECTED] Yahoo!: yscrappy ICQ: 7615664 ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] alternate regression dbs?
Andrew Dunstan [EMAIL PROTECTED] writes: Try attached ... season to taste. The bulk of it is changes for dblink which has the dbname hardcoded. There is probably more to be done with the regression stuff, but this and the earlier change give us the low hanging fruit at least, I think. Applied with light editorialization on the makefile variables ... regards, tom lane ---(end of broadcast)--- TIP 8: explain analyze is your friend
Re: Learning curves and such (was Re: [HACKERS] pgFoundry)
Andrew, Last time it came up I thought the problem was that there was not a consensus on *which* bugtracker to use. Or whether to use one.Roughly 1/3 bugzilla, 1/3 something else, and 1/3 don't want a bugtracker. And, among the people who didn't want bugzilla, some were vehemently opposed to it. Bugtrackers discussed included GForge, bugzilla, RT, Roundup, Jura (they offered a free license) and a few I don't remember. Incidentally, I'm not advocating we use bugzilla (if anything I think I'd lean towards using RT), but this seems like a good opportunity to note that as of a week or two ago bugzilla's HEAD branch supports using PostgreSQL as its backing store, and this will be maintained. One of the things which came out of the bugtracker discussion is that anything we use must have the ability for developers to interact 100% by e-mail, as some critical developers will not use a web interface. -- --Josh Josh Berkus Aglio Database Solutions San Francisco ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: Learning curves and such (was Re: [HACKERS] pgFoundry)
Incidentally, I'm not advocating we use bugzilla (if anything I think I'd lean towards using RT), but this seems like a good opportunity to note that as of a week or two ago bugzilla's HEAD branch supports using PostgreSQL as its backing store, and this will be maintained. One of the things which came out of the bugtracker discussion is that anything we use must have the ability for developers to interact 100% by e-mail, as some critical developers will not use a web interface. Request Tracker (RT) can do that. We use it for all of our support ticket stuff. Sincerely, Joshua D. Drake -- Your PostgreSQL solutions company - Command Prompt, Inc. 1.800.492.2240 PostgreSQL Replication, Consulting, Custom Programming, 24x7 support Managed Services, Shared and Dedication Hosting Co-Authors: plPHP, plPerlNG - http://www.commandprompt.com/ ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: Learning curves and such (was Re: [HACKERS] pgFoundry)
On 5/18/05, Marc G. Fournier [EMAIL PROTECTED] wrote: The key requirement that has always come up is that the core developers wouldn't use anything web based, so the tracker would have to somehow tie into the mailing lists themselves ... What's the basis of this objection to a web-based dev management system? Seems like web-based makes plenty of sense for a physically disparate development community like this one. It also seems that, once you get it up and running, any worthwhile dev management system is going to actually take less time / effort to maintain than, say, maintaining manually concocted todo lists and coordinating development via a mailing list. Call me a normaliser, but even if the maintenance cost is higher, I think it's worth it to have a centralised, authoratitive, organised repository for dev task data. ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: Learning curves and such (was Re: [HACKERS] pgFoundry)
What's the basis of this objection to a web-based dev management system? Seems like web-based makes plenty of sense for a physically disparate development community like this one. I can't speak for the people who don't like web based but my guess is that the web is not their primary medium of communication. Email is. It also seems that, once you get it up and running, any worthwhile dev management system is going to actually take less time / effort to maintain than, say, maintaining manually concocted todo lists and coordinating development via a mailing list. This is true or at least, this is my experience but you are not going to convince many people of that. Call me a normaliser, but even if the maintenance cost is higher, I think it's worth it to have a centralised, authoratitive, organised repository for dev task data. I agree. Sincerely, Joshua D. Drake -- Your PostgreSQL solutions company - Command Prompt, Inc. 1.800.492.2240 PostgreSQL Replication, Consulting, Custom Programming, 24x7 support Managed Services, Shared and Dedication Hosting Co-Authors: plPHP, plPerlNG - http://www.commandprompt.com/ ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: Learning curves and such (was Re: [HACKERS] pgFoundry)
Joshua D. Drake [EMAIL PROTECTED] writes: It also seems that, once you get it up and running, any worthwhile dev management system is going to actually take less time / effort to maintain than, say, maintaining manually concocted todo lists and coordinating development via a mailing list. This is true or at least, this is my experience but you are not going to convince many people of that. The Postgres project has been exceedingly successful while using email lists as the primary means of communication/organization. I for one am disinclined to tinker with such a fundamental aspect of the way that the community operates. If we try to substitute a bug tracker for the mailing lists, I think we'll be making a very basic change in the community's communication structure, and not one for the better. Call me a normaliser, but even if the maintenance cost is higher, I think it's worth it to have a centralised, authoratitive, organised repository for dev task data. I agree. Since the development community is neither centralised nor organized, why would you expect such a repository to have anything to do with what actually happens? regards, tom lane ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: Learning curves and such (was Re: [HACKERS] pgFoundry)
Brendan Jurd wrote: What's the basis of this objection to a web-based dev management system? Seems like web-based makes plenty of sense for a physically disparate development community like this one. Brendan, please review the past versions of this thread. For example, see here: http://groups-beta.google.com/group/comp.databases.postgresql.hackers/browse_thread/thread/1cca714cc34865a5/f802a2a78c94faa3?q=bugzillarnum=1hl=en#f802a2a78c94faa3 cheers andrew ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: [HACKERS] Cost of XLogInsert CRC calculations
On Mon, 16 May 2005 12:35:35 -0400, Tom Lane [EMAIL PROTECTED] wrote: Anyone want to try it with non-gcc compilers? MS VC++ 6.0 with various predefined optimizer settings 2x3264 Default (without any /O) 0.828125 0.906250 MinSize (contains /O1) 0.468750 0.593750 MaxSpeed (contains /O2) 0.312500 0.640625 Not that it really matters -- but at least this looks like another hint that letting the compiler emulate 64 bit operations on non 64 bit hardware is suboptimal. Servus Manfred ---(end of broadcast)--- TIP 8: explain analyze is your friend
Re: [HACKERS] BTW, if anyone wants to work on it...
On Tue, 03 May 2005 02:45:09 -0400, Tom Lane [EMAIL PROTECTED] wrote:
I'm starting to think
it'd be worth setting up a mechanism to handle such changes
automatically.
I've been using this skeleton for quite some time now. Magnus'
psql ... | while read D
might be more robust than my
for db in `enumdatabases`
Servus
Manfred
#!/bin/sh
#---
# pg_plug -- painless upgrade
# PG 7.4 standard - PG 7.4 with various enhancements
#
# 2003-10-04 mk new
# 2003-10-14 mk addpatchinfo
# 2003-11-01 mk VIEW pg_patchinfo
# 2004-11-04 mk renamed VIEW to pg_patches;
# features selectable via command line (-F)
# 2004-12-14 mk freeze
#---
ALLFEATURES=22 27
#---
abortscript(){
echo 12
echo $1 12
exit 1
}
setconnectable(){
db=$1
allow=$2
$CMD template1 /dev/null EOF
UPDATE pg_database SET datallowconn = $allow WHERE datname = '$db';
EOF
}
freeze(){
db=$1
$CMD $db /dev/null EOF
VACUUM FREEZE;
EOF
}
# This does not work for database names with spaces or other funny
# characters.
enumdatabases(){
echo SELECT datname FROM pg_database; \
| $CMD template1 \
| sed -e '/1: datname = /!d' -e 's/.*1: datname = //' -e 's/.*//'
}
add22indexstat(){
db=$1
echo adding \pg_indexstat\, ignore error if it already exists
$CMD $db /dev/null EOF
CREATE TABLE pg_catalog.pg_indexstat( \
istindex oid NOT NULL, \
istcorrel float4 NOT NULL \
) WITHOUT OIDS;
CREATE UNIQUE INDEX pg_indexstat_index_index \
ON pg_indexstat(istindex);
EOF
}
add27patchinfo(){
db=$1
echo adding \pg_patchinfo()\, ignore error if it already exists
$CMD $db /dev/null EOF
COPY pg_proc FROM stdin WITH OIDS;
2981pg_patchinfo11 1 12 f f t t
i 0 2249show_patchinfo - \N
\.
-- copy acl from version()
UPDATE pg_proc SET proacl=(SELECT proacl FROM pg_proc WHERE oid=89) WHERE
oid=2981;
CREATE VIEW pg_patches AS SELECT * FROM pg_patchinfo() AS p(name text, version
text, base text, descr text);
EOF
}
#---
CMDNAME=`basename $0`
BINDIR=bin
FEATURES=
while [ $# -gt 0 ]
do
case $1 in
--username|-U)
POSTGRES_SUPERUSERNAME=$2
shift;;
--username=*)
POSTGRES_SUPERUSERNAME=`echo $1 | sed
's/^--username=//'`
;;
-U*)
POSTGRES_SUPERUSERNAME=`echo $1 | sed 's/^-U//'`
;;
--feature|-F)
FEATURES=$FEATURES $2
shift;;
--feature=*)
F=`echo $1 | sed 's/^--feature=//'`
FEATURES=$FEATURES $F
;;
-F*)
F=`echo $1 | sed 's/^-F//'`
FEATURES=$FEATURES $F
;;
# Data directory. No default, unless the environment
# variable PGDATA is set.
--pgdata|-D)
PGDATA=$2
shift;;
--pgdata=*)
PGDATA=`echo $1 | sed 's/^--pgdata=//'`
;;
-D*)
PGDATA=`echo $1 | sed 's/^-D//'`
;;
-*)
echo $CMDNAME: invalid option: $1
exit 1
;;
# Non-option argument specifies data directory
*)
PGDATA=$1
;;
esac
shift
done
#---
if [ -z $PGDATA ]
then
(
echo $CMDNAME: no data directory specified
echo You must identify the directory where the data for this database
system
echo resides. Do this with either the invocation option -D or the
echo environment variable PGDATA.
) 12
exit 1
fi
#---
PG_OPT=-O
PG_OPT=$PG_OPT -c search_path=pg_catalog
PG_OPT=$PG_OPT -c exit_on_error=true
#PG_OPT=$PG_OPT -c enable_indexstat=false
PG_OPT=$PG_OPT -D $PGDATA
CMD=$BINDIR/postgres $PG_OPT
: ${FEATURES:=$ALLFEATURES}
# Enable connections to template0
setconnectable template0 true || abortscript cannot enable connections to
template0
for db in `enumdatabases`
do
echo converting database $db ...
for f in $FEATURES
do
case $f in
22)
add22indexstat $db # || abortscript cannot convert
database $db
Re: [HACKERS] pgFoundry
On Mon, 16 May 2005 20:54:15 -0400 (EDT), Bruce Momjian pgman@candle.pha.pa.us wrote: I have modifed the TODO HTML so the completed items are in italics. Isn't it a bit misleading to have those items on the TODO list at all? Shouldn't there be a separate list: DONE for the next release? Servus Manfred ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
[HACKERS] Compiling tsearch2 on AIX
I am trying to compile PostgreSQL 8.0.3/tsearch2 contrib module on AIX 5.2 ML 3 gcc version 3.3.2 GNU Make 3.80 bash-2.05b# gmake gcc -O2 -Wall -Wmissing-prototypes -Wpointer-arith -Wendif-labels -fno-strict-aliasing -I./snowball -I./ispell -I./wordparser -I. -I../../src/include -c -o dict_ex.o dict_ex.c In file included from ../../src/include/c.h:64, from ../../src/include/postgres.h:48, from dict_ex.c:9: /opt/freeware/lib/gcc-lib/powerpc-ibm-aix5.2.0.0/3.3.2/include/stdio.h:484: error: conflicting types for `fgetpos64' /opt/freeware/lib/gcc-lib/powerpc-ibm-aix5.2.0.0/3.3.2/include/stdio.h:310: error: previous declaration of `fgetpos64' /opt/freeware/lib/gcc-lib/powerpc-ibm-aix5.2.0.0/3.3.2/include/stdio.h:487: error: conflicting types for `fseeko64' /opt/freeware/lib/gcc-lib/powerpc-ibm-aix5.2.0.0/3.3.2/include/stdio.h:430: error: previous declaration of `fseeko64' /opt/freeware/lib/gcc-lib/powerpc-ibm-aix5.2.0.0/3.3.2/include/stdio.h:488: error: conflicting types for `fsetpos64' /opt/freeware/lib/gcc-lib/powerpc-ibm-aix5.2.0.0/3.3.2/include/stdio.h:312: error: previous declaration of `fsetpos64' /opt/freeware/lib/gcc-lib/powerpc-ibm-aix5.2.0.0/3.3.2/include/stdio.h:489: error: conflicting types for `ftello64' /opt/freeware/lib/gcc-lib/powerpc-ibm-aix5.2.0.0/3.3.2/include/stdio.h:431: error: previous declaration of `ftello64' gmake: *** [dict_ex.o] Error 1 Can someone please let me know why this is happening? ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Re: Learning curves and such (was Re: [HACKERS] pgFoundry)
On 5/18/05, Tom Lane [EMAIL PROTECTED] wrote: The Postgres project has been exceedingly successful while using email lists as the primary means of communication/organization. I for one am disinclined to tinker with such a fundamental aspect of the way that the community operates. If we try to substitute a bug tracker for the mailing lists, I think we'll be making a very basic change in the community's communication structure, and not one for the better. I agree that it's a major change, and the significance of changing the communication structure should not be underestimated. But a) I believe it would be a change for the better, and b) BZ uses a very flexible and verbose email notification system, so the departure from the existing email list structure would not be so drastic. I read through the discussion link that Andrew provided (thanks Andrew), and during that discussion you appeared to be in favour of bugzilla, for the same sorts of reasons I am promoting it now. What changed? Call me a normaliser, but even if the maintenance cost is higher, I think it's worth it to have a centralised, authoratitive, organised repository for dev task data. I agree. Since the development community is neither centralised nor organized, why would you expect such a repository to have anything to do with what actually happens? I think the decentralised nature of the community is one of the things that is responsible for this steep learning curve, and could stand to be improved. And deploying a more centralised system for development management would be a crucial first step in said improvement. In the interests of putting my money where my mouth is, I would be willing to enlist in the housekeeping effort for this hypothetical new system. ---(end of broadcast)--- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: Learning curves and such (was Re: [HACKERS] pgFoundry)
On Tue, 17 May 2005 14:45:00 -0300 (ADT), Marc G. Fournier [EMAIL PROTECTED] wrote: Also, how many 'bugs' have we seen go through the lists that someone hasn't jump'd on and fixed in a couple of days? Just imagine our marketing crew being able to say: According to our great bug tracking system NN serious bugs have been reported last year, 98% of which have been fixed within three days. Servus Manfred ---(end of broadcast)--- TIP 8: explain analyze is your friend
Re: Learning curves and such (was Re: [HACKERS] pgFoundry)
Manfred, Just imagine our marketing crew being able to say: According to our great bug tracking system NN serious bugs have been reported last year, 98% of which have been fixed within three days. grin You're not going to win over many people on *this* list with marketing arguments. -- --Josh Josh Berkus Aglio Database Solutions San Francisco ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Re: Learning curves and such (was Re: [HACKERS] pgFoundry)
Brendan Jurd wrote: In the interests of putting my money where my mouth is, I would be willing to enlist in the housekeeping effort for this hypothetical new system. It's a nice offer, but honestly, my experience in the commercial world as well as in FOSS tells me that a bug tracking system would need loving care from somebody with years of experience in the postgres development community. When I was managing this stuff in a commercial setting it took a large part of my day - I had a 30 to 60 minute meeting every morning and spent a further similar period updating, assigning, etc. The people who could do it are just too damned busy. Given the amount that Tom gets done now I'm amazed that he finds time to eat drink and sleep. The things I tried to suggest earlier in this thread would be much less burdensome. As for central management ... the phrase herding cats springs to mind. cheers andrew ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Learning curves and such
On Tue, 17 May 2005 14:29:49 -0700, Josh Berkus josh@agliodbs.com wrote: grin You're not going to win over many people on *this* list with marketing arguments. Yeah, that's the problem with *my* learning curve ... Servus Manfred ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: Learning curves and such (was Re: [HACKERS] pgFoundry)
* Brendan Jurd ([EMAIL PROTECTED]) wrote: In the interests of putting my money where my mouth is, I would be willing to enlist in the housekeeping effort for this hypothetical new system. If you're willing to create it, host it, update it and keep it current, and feel it'd be so worthwhile to people that you'd be willing to continue to maintain it... Then go for it. You don't need anyone's approval or even agreement about it. *That* would be putting your money where your mouth is. Thanks, Stephen signature.asc Description: Digital signature
Re: Learning curves and such (was Re: [HACKERS] pgFoundry)
* Joshua D. Drake ([EMAIL PROTECTED]) wrote: One of the things which came out of the bugtracker discussion is that anything we use must have the ability for developers to interact 100% by e-mail, as some critical developers will not use a web interface. Request Tracker (RT) can do that. We use it for all of our support ticket stuff. debbugs can do it too, though I don't know of anyone who's actually got it working outside of the Debian stuff. :) Personally, I like Debian's bug tracking system, but I suppose that's just me... I believe OTRS can do it too. Havn't played with the email interface of it really though. Stephen signature.asc Description: Digital signature
Re: Learning curves and such (was Re: [HACKERS] pgFoundry)
On 5/18/05, Stephen Frost [EMAIL PROTECTED] wrote: * Brendan Jurd ([EMAIL PROTECTED]) wrote: In the interests of putting my money where my mouth is, I would be willing to enlist in the housekeeping effort for this hypothetical new system. If you're willing to create it, host it, update it and keep it current, and feel it'd be so worthwhile to people that you'd be willing to continue to maintain it... Then go for it. You don't need anyone's approval or even agreement about it. *That* would be putting your money where your mouth is. I'm detecting sarcasm here, but just in case you're being serious ... For such a tool to serve its intended purpose, the postgres community needs to be, to a certain extent, agreed on and aware of its use as the primary dev management system. There's no point creating, hosting, updating and maintaining anything if the community isn't using it. ---(end of broadcast)--- TIP 8: explain analyze is your friend
Re: Learning curves and such (was Re: [HACKERS] pgFoundry)
On Wed, May 18, 2005 at 08:04:51AM +1000, Brendan Jurd wrote: On 5/18/05, Stephen Frost [EMAIL PROTECTED] wrote: * Brendan Jurd ([EMAIL PROTECTED]) wrote: In the interests of putting my money where my mouth is, I would be willing to enlist in the housekeeping effort for this hypothetical new system. If you're willing to create it, host it, update it and keep it current, and feel it'd be so worthwhile to people that you'd be willing to continue to maintain it... Then go for it. You don't need anyone's approval or even agreement about it. *That* would be putting your money where your mouth is. I'm detecting sarcasm here, but just in case you're being serious ... I don't think Stephen was being sarcastic. Such a system would need an enormous bootstrap effort. Once it's in place, and having shown its usefulness, maybe the community would start using it. (Of course no one can make promises on that other than for himself.) -- Alvaro Herrera (alvherre[a]surnet.cl) Oh, oh, las chicas galacianas, lo harán por las perlas, ¡Y las de Arrakis por el agua! Pero si buscas damas Que se consuman como llamas, ¡Prueba una hija de Caladan! (Gurney Halleck) ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: Learning curves and such (was Re: [HACKERS] pgFoundry)
I don't think Stephen was being sarcastic. Such a system would need an enormous bootstrap effort. Once it's in place, and having shown its usefulness, maybe the community would start using it. (Of course no one can make promises on that other than for himself.) Well sorry but that is ridiculous. Either the community (or more specifically core) chooses to use it upfront or not. I think it is a complete waste of time and energy to expect someone to put in all that effort just to have the community give them the finger. This isn't something that is going to serve the person who loose all the sleep to configure and maintain it. It is something that is going to help the PostgreSQL community has a whole, to grow in a reasonably organized and hopefully less painful way. Sincerely, Joshua D. Drake -- Your PostgreSQL solutions company - Command Prompt, Inc. 1.800.492.2240 PostgreSQL Replication, Consulting, Custom Programming, 24x7 support Managed Services, Shared and Dedication Hosting Co-Authors: plPHP, plPerlNG - http://www.commandprompt.com/ ---(end of broadcast)--- TIP 8: explain analyze is your friend
Re: Learning curves and such (was Re: [HACKERS] pgFoundry)
* Brendan Jurd ([EMAIL PROTECTED]) wrote: I'm detecting sarcasm here, but just in case you're being serious ... Yeah, for the most part I *was* being quite serious. For such a tool to serve its intended purpose, the postgres community needs to be, to a certain extent, agreed on and aware of its use as the primary dev management system. There's no point creating, hosting, updating and maintaining anything if the community isn't using it. Nope, that's not the way the world works. If you build it, they will come. You'll want to make the *community* aware of it, sure, but that's just to encourage people to use it. You don't need anything to be agreed upon, either people will use it, or they won't. If enough people use it that it becomes apparent that it's clearly better *then* you'll likely get a more buy-in and acceptance from developers. Until the developers are on-board you'll need to act as an intermediary (unless you can automate it) between the people using your system and the developers. That's more of your time, but if you're willing to spend it on this to prove it's a better way to work, then go for it. You're never going to get everyone to whole-sale jump over to a new system. It's just not going to happen. You need to build the basics and then get people to start using it. Eventually if it manages to get to a critical mass of some sort you'll get enough people using it that some of them may be willing to help maintain it- perhaps not even developers but other people who are willing to help with the interaction with the developers. You could always start by just doing the 'todo' list that Bruce has and maintaining it as a set of 'enhancements' in the system you build. That shouldn't even be very hard to keep up to date w/ Bruce's todo list provided you watch for his commits to it on the CVS mailing list. Then, if people decide to use your system to open up new enhancement requests you can forward them on to the appropriate list/people and if it makes it onto Bruce's 'todo' list then some how mark it as 'approved' or something to distinguish it from stuff that's been suggested/asked for that *doesn't* make it on the list (and thus is unlikely to be done or worked on). Having the list of stuff that didn't make it would actually be useful and is something Bruce's list misses and thus would be a valuable addition (imv) you would be providing. Now, generally the way this kind of stuff works is that someone gets bitten by a bug and just decides this would be useful and just *does* it w/o asking permission or getting approval from anyone. When people just ask permission or nebulously volunteer their time towards it, generally it *doesn't* happen. Just my 2c. Stephen signature.asc Description: Digital signature
Re: [HACKERS] Cost of XLogInsert CRC calculations
Manfred Koizar wrote: On Mon, 16 May 2005 12:35:35 -0400, Tom Lane [EMAIL PROTECTED] wrote: Anyone want to try it with non-gcc compilers? MS VC++ 6.0 with various predefined optimizer settings 2x3264 Default (without any /O) 0.828125 0.906250 MinSize (contains /O1) 0.468750 0.593750 MaxSpeed (contains /O2) 0.312500 0.640625 Not that it really matters -- but at least this looks like another hint that letting the compiler emulate 64 bit operations on non 64 bit hardware is suboptimal. I don't understand why we are testing 64-bit CRC when I thought we agreed that 32-bit was good enough for our purposes. -- Bruce Momjian| http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 8: explain analyze is your friend
Re: Learning curves and such (was Re: [HACKERS] pgFoundry)
On Tue, May 17, 2005 at 06:25:16PM -0400, Alvaro Herrera wrote: On Wed, May 18, 2005 at 08:04:51AM +1000, Brendan Jurd wrote: On 5/18/05, Stephen Frost [EMAIL PROTECTED] wrote: * Brendan Jurd ([EMAIL PROTECTED]) wrote: In the interests of putting my money where my mouth is, I would be willing to enlist in the housekeeping effort for this hypothetical new system. If you're willing to create it, host it, update it and keep it current, and feel it'd be so worthwhile to people that you'd be willing to continue to maintain it... Then go for it. You don't need anyone's approval or even agreement about it. *That* would be putting your money where your mouth is. I'm detecting sarcasm here, but just in case you're being serious ... I don't think Stephen was being sarcastic. Such a system would need an enormous bootstrap effort. Once it's in place, and having shown its usefulness, maybe the community would start using it. (Of course no one can make promises on that other than for himself.) The useful bug tracking systems I've used have also included QA. Any bug submitted doesn't get accepted without a standalone test case. That test case is used both to confirm that the bug has been fixed once it's fixed, and is added into a set of regression tests. If someone were to create test cases for (some or all of the) submitted bugs (or handle the management of their creation) and track which test cases passed (via a tinderbox, or even the existing build-farm) that'd be a useful thing in itself. I suspect it'd be a useful thing for someone who has energy to expend on bug-tracking to do, and probably more immediately useful than anything that requires all the primary developers to change how they're currently handling bugs and to-do lists. Just a thought. Cheers, Steve ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: Learning curves and such (was Re: [HACKERS] pgFoundry)
On Tue, May 17, 2005 at 03:30:28PM -0700, Joshua D. Drake wrote: I don't think Stephen was being sarcastic. Such a system would need an enormous bootstrap effort. Once it's in place, and having shown its usefulness, maybe the community would start using it. (Of course no one can make promises on that other than for himself.) Well sorry but that is ridiculous. Either the community (or more specifically core) chooses to use it upfront or not. I think it is a complete waste of time and energy to expect someone to put in all that effort just to have the community give them the finger. This isn't something that is going to serve the person who loose all the sleep to configure and maintain it. It is something that is going to help the PostgreSQL community has a whole, to grow in a reasonably organized and hopefully less painful way. Maybe I didn't express myself properly. I didn't mean that Brendan should do all the work by himself -- but expecting the whole community, or the usual developers, to do the initial effort, is not likely to make this plane fly. Because the current developers are already comfortable with the current state of affairs, why should they worry about changing the process? This is something the rest of the people can do, for their own sake. It can, of course, serve the community eventually. Would you expect Bruce to enter each and every TODO item in a bug tracking system? I wouldn't. Would I register my own pet peeves in such a system? Probably I would. -- Alvaro Herrera (alvherre[a]surnet.cl) Los románticos son seres que mueren de deseos de vida ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] pgFoundry
Manfred Koizar wrote: On Mon, 16 May 2005 20:54:15 -0400 (EDT), Bruce Momjian pgman@candle.pha.pa.us wrote: I have modifed the TODO HTML so the completed items are in italics. Isn't it a bit misleading to have those items on the TODO list at all? Shouldn't there be a separate list: DONE for the next release? The idea is that people who are running 8.0 need to see those items because they aren't _done_ in their release. -- Bruce Momjian| http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: Learning curves and such (was Re: [HACKERS] pgFoundry)
This isn't something that is going to serve the person who loose all the sleep to configure and maintain it. It is something that is going to help the PostgreSQL community has a whole, to grow in a reasonably organized and hopefully less painful way. Maybe I didn't express myself properly. I didn't mean that Brendan should do all the work by himself -- but expecting the whole community, or the usual developers, to do the initial effort, is not likely to make this plane fly. Because the current developers are already comfortable with the current state of affairs, why should they worry about changing the process? This is something the rest of the people can do, for their own sake. It can, of course, serve the community eventually. O.k. then I misunderstood and apologize. I think the above is reasonable. I wouldn't think that the main developers would stop developing to create this system, nor would I want them to take time away from development to implement it. I would however think that the main developer buy off would be essential. If the primary memebers of the development community said o.k. we are going to implement foo... who wants to spear head it? That would be a good thing. Would you expect Bruce to enter each and every TODO item in a bug tracking system? Well that depends... if it replaced the current TODO list as the definitive source then yes I would. I wouldn't. Would I register my own pet peeves in such a system? Probably I would. Sincerely, Joshua D. Drake -- Your PostgreSQL solutions company - Command Prompt, Inc. 1.800.492.2240 PostgreSQL Replication, Consulting, Custom Programming, 24x7 support Managed Services, Shared and Dedication Hosting Co-Authors: plPHP, plPerlNG - http://www.commandprompt.com/ ---(end of broadcast)--- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: Learning curves and such (was Re: [HACKERS] pgFoundry)
On Wed, 18 May 2005 04:31 am, Josh Berkus wrote: Andrew, Last time it came up I thought the problem was that there was not a consensus on *which* bugtracker to use. Or whether to use one.Roughly 1/3 bugzilla, 1/3 something else, and 1/3 don't want a bugtracker. And, among the people who didn't want bugzilla, some were vehemently opposed to it. Bugtrackers discussed included GForge, bugzilla, RT, Roundup, Jura (they offered a free license) and a few I don't remember. Incidentally, I'm not advocating we use bugzilla (if anything I think I'd lean towards using RT), but this seems like a good opportunity to note that as of a week or two ago bugzilla's HEAD branch supports using PostgreSQL as its backing store, and this will be maintained. One of the things which came out of the bugtracker discussion is that anything we use must have the ability for developers to interact 100% by e-mail, as some critical developers will not use a web interface. Doesn't pgfoundry offer this? If not in 3.3, I'm sure it's in Gforge 4.0, or 4.1 which will be released soon. Regards Russell ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: Learning curves and such (was Re: [HACKERS] pgFoundry)
On 5/18/05, Joshua D. Drake [EMAIL PROTECTED] wrote: O.k. then I misunderstood and apologize. I think the above is reasonable. I wouldn't think that the main developers would stop developing to create this system, nor would I want them to take time away from development to implement it. I think we can all agree on that. I would however think that the main developer buy off would be essential. Yep ... I get the feeling that in this community we have a handful of people doing the overwhelming majority of the development. Getting at least an in principle nod of the head from these people is a prerequisite for any major effort IMO. BJ ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: Learning curves and such (was Re: [HACKERS] pgFoundry)
Russell Smith wrote: One of the things which came out of the bugtracker discussion is that anything we use must have the ability for developers to interact 100% by e-mail, as some critical developers will not use a web interface. Doesn't pgfoundry offer this? If not in 3.3, I'm sure it's in Gforge 4.0, or 4.1 which will be released soon. 3.3 does not - have not looked at 4.x. Unless it's gone through a radical upgrade I don't think it's good enough. A mail interface is only part of the story. cheers andrew ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] patches for items from TODO list
Dnia 13-05-2005, pi o godzinie 16:01 -0700, Sergey Ten napisa(a): ?xml version='1.0'? table row col name='col1' null='n'Jackson, Sam/col col name='col2' null='n'\h/col /row row col name='col1' null='n'It is quot;perfectquot;./col col name='col2' null='n'#09;/col /row row col name='col1' null='n'/col col name='col2' null='y'/col /row /table Why didn't you do something to the effect of ?xml version='1.0'? table cols col name='col1'/ col name='col2'/ /cols row col null='n'Jackson, Sam/col col null='n'\h/col /row row col null='n'It is quot;perfectquot;./col col null='n'#09;/col /row row col null='n'/col col null='y'/col /row /table This avoids repeating the column names in every row, which don't change over the rows anyway. By reducing redundant information it also makes structurally invalid XML less likely (whether that is relevant depends on what people do with the XML data). Also you could encode the XML output as UTF-8, which would make the files more readable for humans if the text data is not ASCII. Markus -- Markus Bertheau [EMAIL PROTECTED] signature.asc Description: This is a digitally signed message part
Re: Learning curves and such (was Re: [HACKERS] pgFoundry)
Brendan Jurd wrote: What's the basis of this objection to a web-based dev management system? Beyond the core developers want to stick to email, I think there is a good reason that we should stick primarily to email for project management: Bugzilla and similar systems are point to point, whereas a mailing list is multicast[1]. When someone submits a patch or a bug report to a mailing list, any of the developers can see the report, discuss it, and contribute to resolving it. More often than not, a web-based interface like Bugzilla leads to a single bug master, who does most of this work by themselves. Besides the fact we don't have such a person, it would also mean that knowledge of bugs/patches and the discussion about resolving issues is distributed among a smaller pool of people. There is definitely room for improvement; submitted patches do occasionally fall through the cracks, for example. I would personally be interested in a bug-tracking system that is closer to a shared email archive. Individuals would send mail to a mailing list and other people would reply and eventually resolve the thread, as happens now. The process would be slightly more formalized: there would be a way to specify a few commands via email to close/open/resolve/etc. reports, and some kind of interface (perhaps web-based) for viewing unresolved issues, searching through issues, etc. But the point is that the current system works well; this would just be a slight formalization of existing procedures (we don't *want* a revolutionary change, nor do we need one). I think the administrative overhead wouldn't be too high, either. I'm not sure which existing systems fit this model (suggestions are welcome) -- email needs to be the primary interface, not an afterthought (as is often the case). Perhaps RT would work, I'm not sure. -Neil [1] Hat-tip to Andrew Morton's keynote at LCA, which made this point effectively. ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: Learning curves and such (was Re: [HACKERS] pgFoundry)
discuss it, and contribute to resolving it. More often than not, a web-based interface like Bugzilla leads to a single bug master, who does most of this work by themselves. Besides the fact we don't have such a person, it would also mean that knowledge of bugs/patches and the discussion about resolving issues is distributed among a smaller pool of people. I can only speak for RT but with RT you can easily avoid this. For example you can set it up so that anything that would go to [EMAIL PROTECTED] would automatically create a ticket an alert all people within the patches group. Multiple people can be assigned to a ticket as a maintainer or just a watcher. You can even respond to specific messages within the thread instead of just a top down (one email after the other). There is definitely room for improvement; submitted patches do occasionally fall through the cracks, for example. I would personally be interested in a bug-tracking system that is closer to a shared email archive. That would be another nice part of RT. RT automatically deals with attachments and although I wouldn't use it for this you could even use it as a semi patch repository until the patch is actually approved for submission. issues, searching through issues, etc. But the point is that the current system works well; Well does it though? I am not saying it is bad, well yes I am ;). There is no central place for me to point one of my developers and say -- Hey, look at this patch... weren't we working on something like this? Let's help them out. I have to have the dig through the mail archives which is fairly counter productive. It would be much better to be able to say, hey... look at patch #42345. What do you think? I'm not sure which existing systems fit this model (suggestions are welcome) -- email needs to be the primary interface, not an afterthought (as is often the case). Perhaps RT would work, I'm not sure. RT supports complete email integration. Most of the interaction I do with it is actually through email not through the web interface. It also has the ability to have a knowledge base dropped right on top of it. Sincerely, Joshua D. Drake -Neil [1] Hat-tip to Andrew Morton's keynote at LCA, which made this point effectively. ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: Learning curves and such (was Re: [HACKERS] pgFoundry)
Joshua D. Drake wrote: You can even respond to specific messages within the thread instead of just a top down (one email after the other). Well, that seems pretty fundamental... But the point is that the current system works well; Well does it though? I am not saying it is bad, well yes I am ;). There is no central place for me to point one of my developers and say -- Hey, look at this patch... weren't we working on something like this? Let's help them out. I'm not saying there is not room for improvement -- my point is that the fundamental workflow (email submission of patches, discussion and resolution via mailing list discussion) works well, and I don't see any need to change it. If there are tools that help us improve this process (e.g. archiving the resolution of old issues, as you suggest), they are worth considering. In other words, I'd like a tool that fits the way we work now, not one that forces us to change our processes to adapt to its requirements. Anyway, RT sounds like perhaps it is worth investigating. -Neil ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: Learning curves and such (was Re: [HACKERS] pgFoundry)
It also seems that, once you get it up and running, any worthwhile dev management system is going to actually take less time / effort to maintain than, say, maintaining manually concocted todo lists and coordinating development via a mailing list. Call me a normaliser, but even if the maintenance cost is higher, I think it's worth it to have a centralised, authoratitive, organised repository for dev task data. I 100% agree... There's also this: http://www.issue-tracker.com/ Chris ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: [HACKERS] patches for items from TODO list
Sergey Ten wrote: After a careful consideration we decided to - put XML implementation in the backend What advantage does putting the XML output mode in the backend provide? -Neil ---(end of broadcast)--- TIP 8: explain analyze is your friend
Re: [HACKERS] Cost of XLogInsert CRC calculations
Bruce Momjian pgman@candle.pha.pa.us writes: I don't understand why we are testing 64-bit CRC when I thought we agreed that 32-bit was good enough for our purposes. Well, we need to understand exactly what is going on here. I'd not like to think that we dropped back from 64 to 32 bit because of one possibly-minor optimization bug in one compiler on one platform. Even if that compiler+platform is 90% of the market. regards, tom lane ---(end of broadcast)--- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] Compiling tsearch2 on AIX
Mag Gam [EMAIL PROTECTED] writes: I am trying to compile PostgreSQL 8.0.3/tsearch2 contrib module on AIX 5.2 ML 3 /opt/freeware/lib/gcc-lib/powerpc-ibm-aix5.2.0.0/3.3.2/include/stdio.h:484: error: conflicting types for `fgetpos64' /opt/freeware/lib/gcc-lib/powerpc-ibm-aix5.2.0.0/3.3.2/include/stdio.h:310: error: previous declaration of `fgetpos64' I'm just guessing, but this smells to me like a problem with largefile- vs-not-largefile support. We discovered long ago that when pg_config.h defines _FILE_OFFSET_BITS or _LARGEFILE_SOURCE, it's critical that the compiler see that *before* it includes stdio.h. Unfortunately it seems this hard-won knowledge didn't get propagated into tsearch2 until just recently :-(. Try applying the tsearch2 changes linked here: http://archives.postgresql.org/pgsql-committers/2005-05/msg00068.php and let us know if that helps. I have not back-patched these changes into existing release branches, but if it turns out to address a real problem-seen-in-the-field then that should get done ... regards, tom lane ---(end of broadcast)--- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: Learning curves and such (was Re: [HACKERS] pgFoundry)
Neil Conway wrote: Joshua D. Drake wrote: You can even respond to specific messages within the thread instead of just a top down (one email after the other). Well, that seems pretty fundamental... But the point is that the current system works well; Well does it though? I am not saying it is bad, well yes I am ;). There is no central place for me to point one of my developers and say -- Hey, look at this patch... weren't we working on something like this? Let's help them out. I'm not saying there is not room for improvement -- my point is that the fundamental workflow (email submission of patches, discussion and resolution via mailing list discussion) works well, and I don't see any need to change it. If there are tools that help us improve this process (e.g. archiving the resolution of old issues, as you suggest), they are worth considering. In other words, I'd like a tool that fits the way we work now, not one that forces us to change our processes to adapt to its requirements. Agreed. Basically, there is the ideal world, where everything is organized around bug numbers, and there is a tracking system of who has reported the bugs and who fixed them, with a roadmap, and there is the real world, where such organization takes time, and takes time away from doing actual productive work. Now, you can argue that the time to do the maintenance of such a system will pay off, but it is far from clear that is true. No one has shown me a project that has such a system that works better than what we have now, nor a roadmap that is better than ours. If this new setup is so great, please point me to a real project that uses it effectively. The only two projects I have worked with in this area are Mozilla (bugzilla, terribly complex bug tracking and roadmap that drove them off the road), and gaim, which seems to work (sourceforge, everything gets put into a box of feature/bug, etc, and someone comes along and manages that). I don't think either are more effective than what we have now. What we have now is _bad_ for new people trying to jump in and figure things out --- that is the real problem with our current setup. You can't just point someone at bug number XXX. How much is that worth to us in terms of time? I am unsure. I imagine the Mozilla guys spending all day closing bugs and putting things in little boxes (oh, that bug is a duplicate), but the whole time the project is not user-focused and they get superceeded by Firefox because the new project actually gives users what they want. Do we really want people to fill out a bug form, and have us get an email and get an email and reply to the request and close the bug, rather than just email the guy to give them the fix? What does the big bug database actually do for us except give us a big list of thousands of items. I just don't see the huge value in tracking stuff for little payback. Sure, it sounds great, but in practice it seems pretty useless, and there is maintenance cost. -- Bruce Momjian| http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 8: explain analyze is your friend
Re: [HACKERS] Cost of XLogInsert CRC calculations
Tom Lane wrote: Bruce Momjian pgman@candle.pha.pa.us writes: I don't understand why we are testing 64-bit CRC when I thought we agreed that 32-bit was good enough for our purposes. Well, we need to understand exactly what is going on here. I'd not like to think that we dropped back from 64 to 32 bit because of one possibly-minor optimization bug in one compiler on one platform. Even if that compiler+platform is 90% of the market. But isn't it obvious that almost any problem that CRC64 is going to catch, CRC32 is going to catch, and we know CRC32 has to be faster than CRC64? -- Bruce Momjian| http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: Learning curves and such (was Re: [HACKERS] pgFoundry)
Neil Conway [EMAIL PROTECTED] writes: Beyond the core developers want to stick to email, I think there is a good reason that we should stick primarily to email for project management: Bugzilla and similar systems are point to point, whereas a mailing list is multicast[1]. That seems to me to be a great summary of the issue. I've been dealing with Bugzilla for a few years now in my employment with Red Hat, and I think the bottom line for that kind of system is that it's designed to limit the visibility of issues, rather than expose them. Now that is just exactly what you want for a corporate-sized bug tracking system --- at Red Hat, I do not want to hear about bugs in the kernel, or X, or a thousand other components that I have no expertise in --- but I cannot see that restricting the flow of information is what we need for Postgres. I think most of the real advantages of bug trackers that have been mentioned in this thread have to do with history and searchability. We have the raw info for that, in the pgsql-bugs and pgsql-commmitters mail archives, and so it seems to me that this reduces to the perennial gripe that we don't have good enough search tools for the archives. regards, tom lane ---(end of broadcast)--- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: Learning curves and such (was Re: [HACKERS] pgFoundry)
Steve Atkins [EMAIL PROTECTED] writes: The useful bug tracking systems I've used have also included QA. Any bug submitted doesn't get accepted without a standalone test case. Side note: while test cases are certainly Good Things that make life easier for developers, so we should encourage people to provide 'em, I can't say that I like the idea of a tracking system designed around the concept that a bug for which you don't have a test case isn't real. It's not all that easy to make a test case for bugs involving concurrent behavior. I'd go so far as to say that most of the seriously interesting bugs that I've dealt with in this project were ones that the original reporter didn't have a reproducible test case for. regards, tom lane ---(end of broadcast)--- TIP 8: explain analyze is your friend
Re: Learning curves and such (was Re: [HACKERS] pgFoundry)
Tom Lane wrote: Neil Conway [EMAIL PROTECTED] writes: Beyond the core developers want to stick to email, I think there is a good reason that we should stick primarily to email for project management: Bugzilla and similar systems are point to point, whereas a mailing list is multicast[1]. That seems to me to be a great summary of the issue. I've been dealing with Bugzilla for a few years now in my employment with Red Hat, and I think the bottom line for that kind of system is that it's designed to limit the visibility of issues, rather than expose them. Now that is just exactly what you want for a corporate-sized bug tracking system --- at Red Hat, I do not want to hear about bugs in the kernel, or X, or a thousand other components that I have no expertise in --- but I cannot see that restricting the flow of information is what we need for Postgres. I think most of the real advantages of bug trackers that have been mentioned in this thread have to do with history and searchability. We have the raw info for that, in the pgsql-bugs and pgsql-commmitters mail archives, and so it seems to me that this reduces to the perennial gripe that we don't have good enough search tools for the archives. Good point, and I think the big criticism of our current system is that it is hard for someone to come in and see only a limited view of our issues. For example, if you only want to see ecpg issues, we don't make it very easy. This is the same issue Joshua Drake was mentioning, that there is no easy way to point someone at bug number XXX and assume they will have a full summary of the issue. -- Bruce Momjian| http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] Cost of XLogInsert CRC calculations
Bruce Momjian pgman@candle.pha.pa.us writes: Tom Lane wrote: Well, we need to understand exactly what is going on here. I'd not like to think that we dropped back from 64 to 32 bit because of one possibly-minor optimization bug in one compiler on one platform. Even if that compiler+platform is 90% of the market. But isn't it obvious that almost any problem that CRC64 is going to catch, CRC32 is going to catch, and we know CRC32 has to be faster than CRC64? Do we know that? The results I showed put at least one fundamentally 32bit platform (the PowerBook I'm typing this on) at dead par for 32bit and 64bit CRCs. We have also established that 64bit CRC can be done much faster on 32bit Intel than it's currently done by the default PG-on-gcc build (hint: don't use -O2 or above). So while Mark's report that 64bit CRC is an issue on Intel is certainly true, it doesn't immediately follow that the only sane response is to give up 64bit CRC. We need to study it and see what alternatives we have. I do personally feel that 32bit is the way to go, but that doesn't mean I think it's a done deal. We owe it to ourselves to understand what we are buying and what we are paying for it. regards, tom lane ---(end of broadcast)--- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: Learning curves and such (was Re: [HACKERS] pgFoundry)
On Wed, May 18, 2005 at 12:07:14AM -0400, Tom Lane wrote: Steve Atkins [EMAIL PROTECTED] writes: The useful bug tracking systems I've used have also included QA. Any bug submitted doesn't get accepted without a standalone test case. Side note: while test cases are certainly Good Things that make life easier for developers, so we should encourage people to provide 'em, I can't say that I like the idea of a tracking system designed around the concept that a bug for which you don't have a test case isn't real. It's not all that easy to make a test case for bugs involving concurrent behavior. I'd go so far as to say that most of the seriously interesting bugs that I've dealt with in this project were ones that the original reporter didn't have a reproducible test case for. Depends on the context. For a code base like PGs (with, as you say, many possibilities for weird concurrency issues or data driven bugs), or for a development style like PGs (i.e. mostly volunteer), _requiring_ a test case before a bug is worked on makes no sense. Some environments I've worked in, though, have had a stage between bug submitted and bug passed to developer where someone in QA makes an attempt to create a test case where one was not submitted with the bug. That was more the idea I was suggesting as a possibility - if someone has a QA itch to scratch then trolling postgresql-bugs for bugs without test cases and creating recreatable test cases to attach to those bugs might be a useful thing to do. Cheers, Steve ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Re: [HACKERS] Cost of XLogInsert CRC calculations
I probably shouldn't jump in, because I do not know the nature of the usage of the CRC values. But if the birthday paradox can come into play, with a 32 bit CRC, you will get one false mismatch every 78,643 items or so. http://mathworld.wolfram.com/BirthdayProblem.html Probably you already knew that, and probably the birthday paradox does not apply. I generally use 64 bit CRCs (UMAC) for just about anything that needs a CRC. http://www.cs.ucdavis.edu/~rogaway/umac/ A plausible work-around is to compute two distinct 32-bit hash values for platforms with awful 64 bit math/emulation (e.g. [SDBM hash and FNV hash] or [Bob Jenkins hash and D. J. Bernstein hash]) to create two distinct 32 bit hash values -- both of which must match. -Original Message- From: [EMAIL PROTECTED] [mailto:pgsql-hackers- [EMAIL PROTECTED] On Behalf Of Tom Lane Sent: Tuesday, May 17, 2005 9:26 PM To: Bruce Momjian Cc: Manfred Koizar; Mark Cave-Ayland; pgsql-hackers@postgresql.org Subject: Re: [HACKERS] Cost of XLogInsert CRC calculations Bruce Momjian pgman@candle.pha.pa.us writes: Tom Lane wrote: Well, we need to understand exactly what is going on here. I'd not like to think that we dropped back from 64 to 32 bit because of one possibly-minor optimization bug in one compiler on one platform. Even if that compiler+platform is 90% of the market. But isn't it obvious that almost any problem that CRC64 is going to catch, CRC32 is going to catch, and we know CRC32 has to be faster than CRC64? Do we know that? The results I showed put at least one fundamentally 32bit platform (the PowerBook I'm typing this on) at dead par for 32bit and 64bit CRCs. We have also established that 64bit CRC can be done much faster on 32bit Intel than it's currently done by the default PG-on-gcc build (hint: don't use -O2 or above). So while Mark's report that 64bit CRC is an issue on Intel is certainly true, it doesn't immediately follow that the only sane response is to give up 64bit CRC. We need to study it and see what alternatives we have. I do personally feel that 32bit is the way to go, but that doesn't mean I think it's a done deal. We owe it to ourselves to understand what we are buying and what we are paying for it. regards, tom lane ---(end of broadcast)--- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match ---(end of broadcast)--- TIP 8: explain analyze is your friend
Re: Learning curves and such (was Re: [HACKERS] pgFoundry)
Steve Atkins [EMAIL PROTECTED] writes: On Wed, May 18, 2005 at 12:07:14AM -0400, Tom Lane wrote: It's not all that easy to make a test case for bugs involving concurrent behavior. I'd go so far as to say that most of the seriously interesting bugs that I've dealt with in this project were ones that the original reporter didn't have a reproducible test case for. ... Some environments I've worked in, though, have had a stage between bug submitted and bug passed to developer where someone in QA makes an attempt to create a test case where one was not submitted with the bug. That was more the idea I was suggesting as a possibility - if someone has a QA itch to scratch then trolling postgresql-bugs for bugs without test cases and creating recreatable test cases to attach to those bugs might be a useful thing to do. It's absolutely useful --- in fact, creating a case that fails at least once-in-a-while is normally the first thing that a developer will try to do when faced with this sort of report. But that effort doesn't always require intimacy with the code, so farming it out is not a bad idea for spreading the work around. This certainly ties in with the recent discussions about how can you contribute when you haven't already learned all of the code base ... but to bring it back to the immediate topic, would a bugzilla or RT system really help compared to our existing mailing-list system? I've noticed Michael Fuhr and some other folk doing the confirm-bug- and-provide-test-cases spadework recently, so it seems like this is already something we can handle. What it comes down to is that a mailing list encourages many-eyes-on- one-bug synergy, whereas Bugzilla is designed to send a bug report to just one pair of eyes, or at most a small number of eyes. I haven't used RT but I doubt it's fundamentally different. regards, tom lane ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] Cost of XLogInsert CRC calculations
Tom Lane [EMAIL PROTECTED] writes: Do we know that? The results I showed put at least one fundamentally 32bit platform (the PowerBook I'm typing this on) at dead par for 32bit and 64bit CRCs. Wait, par for 32-bit CRCs? Or for 64-bit CRCs calculated using 32-bit ints? -- greg ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] Cost of XLogInsert CRC calculations
Greg Stark [EMAIL PROTECTED] writes: Tom Lane [EMAIL PROTECTED] writes: Do we know that? The results I showed put at least one fundamentally 32bit platform (the PowerBook I'm typing this on) at dead par for 32bit and 64bit CRCs. Wait, par for 32-bit CRCs? Or for 64-bit CRCs calculated using 32-bit ints? Right, the latter. We haven't actually tried to measure the cost of plain 32bit CRCs... although I seem to recall that when we originally decided to use 64bit, someone put up some benchmarks purporting to show that there wasn't much difference. regards, tom lane ---(end of broadcast)--- TIP 8: explain analyze is your friend
[HACKERS] Bitmap scan cost model (was Re: bitmap scans, btree scans, and tid order)
On Mon, 2005-05-16 at 14:35 -0400, Tom Lane wrote: Jeffrey W. Baker [EMAIL PROTECTED] writes: On Mon, 2005-05-16 at 09:53 -0400, Tom Lane wrote: This is a fallacy, and I think your concern is largely mistaken. Have you experimented with the cases you are worried about? Perhaps I have not stated the problem clearly. Believe me, I have experimented extensively with this problem. Sorry, perhaps I wasn't clear: have you experimented *with CVS tip*? The current code is certainly capable of choosing either seqscan, bitmap scan, or traditional index scan depending on the given query. For example, [...] I think the work that's most needed at the moment is to test the bitmap-scan cost model to see if it has much to do with reality... The cost model seems to work pretty darn well, as a matter of fact. This table is in-order by id, in-order by date, and randomly ordered by random. scratch=# \d test Table public.test Column | Type | Modifiers +-+--- id | integer | date | date| random | integer | Indexes: test_pk UNIQUE, btree (id) test_date_idx btree (date) test_rand_idx btree (random) scratch=# select count(1) from test; count -- 1000 The size of the tables and indexes is about 1G, or double physical memory. I tested by selecting on the random column. For 100 random values, I selected the row that matches exactly, then in ranges of 1000, 1, 10, and 100. These touch roughly 5, 50, 500, and 5000 tuples, respectively. The test is repeated for index scan, bitmap scan, and sequential scan. Example query: select count(1) from test where random = 1429076987 and random 1429076987 + 1; QUERY PLAN -- Aggregate (cost=171.31..171.31 rows=1 width=0) (actual time=0.996..1.000 rows=1 loops=1) - Bitmap Heap Scan on test (cost=2.26..171.20 rows=43 width=0) (actual time=0.145..0.829 rows=42 loops=1) Recheck Cond: ((random = 1429076987) AND (random 1429086987)) - Bitmap Index Scan on test_rand_idx (cost=0.00..2.26 rows=43 width=0) (actual time=0.063..0.063 rows=42 loops=1) Index Cond: ((random = 1429076987) AND (random 1429086987)) Total runtime: 1.114 ms 100 queries returning | 1 | 5 | 50 | 500 | 5000 | --+-+-+--+---++ bitmap scan | 1s | 2s | 13s | 1m41s | 11m16s | index scan| 1s | 1s | 12s | 2m6s | 24m19s | --+-+-+--+---++ sequential scan | 28m20s| The planner uses index scan for the first two columns, and chooses bitmap scan for the rest. This would seem to be a reasonable decision, given the results. The only real problem with the cost model in further testing is that it doesn't switch to seq scan quickly enough. If bitmap scan is disabled, the planner will pick index scan even in cases when sequential scan is 10x faster: scratch=# set enable_bitmapscan to off; SET scratch=# explain analyze select count(1) from test where random = 1429076987 and random 1429076987 + 1000; QUERY PLAN Aggregate (cost=170142.03..170142.03 rows=1 width=0) (actual time=177419.182..177419.185 rows=1 loops=1) - Index Scan using test_rand_idx on test (cost=0.00..170034.11 rows=43167 width=0) (actual time=0.035..177255.696 rows=46764 loops=1) Index Cond: ((random = 1429076987) AND (random 1439076987)) Total runtime: 177419.302 ms (4 rows) scratch=# set enable_indexscan to off; SET scratch=# explain analyze select count(1) from test where random = 1429076987 and random 1429076987 + 1000; QUERY PLAN -- Aggregate (cost=204165.55..204165.55 rows=1 width=0) (actual time=12334.042..12334.045 rows=1 loops=1) - Seq Scan on test (cost=0.00..204057.62 rows=43167 width=0) (actual time=17.436..12174.150 rows=46764 loops=1) Filter: ((random = 1429076987) AND (random 1439076987)) Total runtime: 12334.156 ms (4 rows) Obviously in this case sequential scan was (would have been) a huge win. Incrementing random_page_cost from 4 (the default) to 5 causes the planner to make a better decision. -jwb
Re: [HACKERS] Cost of XLogInsert CRC calculations
Tom Lane wrote: Greg Stark [EMAIL PROTECTED] writes: Tom Lane [EMAIL PROTECTED] writes: Do we know that? The results I showed put at least one fundamentally 32bit platform (the PowerBook I'm typing this on) at dead par for 32bit and 64bit CRCs. Wait, par for 32-bit CRCs? Or for 64-bit CRCs calculated using 32-bit ints? Right, the latter. We haven't actually tried to measure the cost of plain 32bit CRCs... although I seem to recall that when we originally decided to use 64bit, someone put up some benchmarks purporting to show that there wasn't much difference. OK, thanks. I didn't know that. -- Bruce Momjian| http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Re: [HACKERS] Cost of XLogInsert CRC calculations
Dann Corbit [EMAIL PROTECTED] writes: Probably you already knew that, and probably the birthday paradox does not apply. I generally use 64 bit CRCs (UMAC) for just about anything that needs a CRC. http://www.cs.ucdavis.edu/~rogaway/umac/ The birthday paradox doesn't come up here. The CRC has to match the actual data for that specific xlog, not just any CRC match with any xlog from a large list. So if an xlog is corrupted or truncated then the chances that a 64-bit CRC would match and the xlog be mishandled is one in 16 quadrillion or so. A 32-bit CRC will match the invalid data is about one in 4 billion. The chances that a 16-bit CRC would match would be one in 64 thousand. I mention 16-bit CRC because you use a system every day that uses 16-bit CRCs and you trust thousands of data blocks each day to this protection (actually probably thousands each second). I refer to TCP/IP. Every TCP/IP segment is protected by just a 16-bit CRC. Have you ever seen a corrupted TCP stream caused by the use of such a short CRC? Actually I have. A router with a bad memory card caused about 1% packet loss due to corrupted segments. Low enough not to be noticed but in a large FTP transfer it meant about one corrupted packet got through every 2.4GB of data or so. Now consider the data being protected by a the xlog CRC. If 1% of every disk write were being corrupted would one incorrect xlog being read in and mishandled about once every few gigabytes of logs really be the worst of your worries? More realistically, if you were losing power frequently and having truncated xlog writes frequently, say about once every 5 minutes (if you could get it to boot that fast). Would one incorrectly handled truncated log every 56 days be considered unacceptable? That would be the consequence of 16-bit checksums. If you ran the same experiment with 32-bit checksums it would mean the database wouldn't correctly replay once every two thousand five hundred and fifty three years. -- greg ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
