Re: [HACKERS] monolithic distro
Peter Eisentraut wrote: Lukas Kahwe Smith wrote: whereas PostgreSQL is continuously complaing that MySQL is inferior yet way more popular. Maybe MySQL's popularity is not even PostgreSQL's goal, but I am sure a bit more would be welcome. Does MySQL have a monolithic distribution? Well obviously MySQL is missing alot of functionality that you will not get in any version of MySQL though. However it comes with replication, fulltext indexes out of the box. They currently only have a single stored procedure language (partial SQL:2003 implementation). regards, Lukas ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
[HACKERS] ShmemAlloc() alignment patch
In ShmemAlloc() we have: newStart = BUFFERALIGN(newStart); newSpace = (void *) (ShmemBase + newStart); return newSpace; Notice that though newStart is ALIGNOF_BUFFER, ShmemBase is not. Thus the newSpace is not aligned as we disired. Attached please find the patch. Regards, Qingqing Index: src/backend/storage/ipc/shmem.c === RCS file: /projects/cvsroot/pgsql/src/backend/storage/ipc/shmem.c,v retrieving revision 1.92 diff -c -r1.92 shmem.c *** src/backend/storage/ipc/shmem.c 13 Jul 2006 16:49:16 - 1.92 --- src/backend/storage/ipc/shmem.c 14 Jul 2006 06:47:43 - *** *** 169,178 SpinLockAcquire(ShmemLock); newStart = shmemseghdr-freeoffset; /* extra alignment for large requests, since they are probably buffers */ if (size = BLCKSZ) ! newStart = BUFFERALIGN(newStart); newFree = newStart + size; if (newFree = shmemseghdr-totalsize) --- 169,179 SpinLockAcquire(ShmemLock); newStart = shmemseghdr-freeoffset; + newSpace = (void *) MAKE_PTR(newStart); /* extra alignment for large requests, since they are probably buffers */ if (size = BLCKSZ) ! newStart = MAKE_OFFSET(BUFFERALIGN(newSpace)); newFree = newStart + size; if (newFree = shmemseghdr-totalsize) ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] monolithic distro
Lukas Smith wrote: Peter Eisentraut wrote: Lukas Kahwe Smith wrote: whereas PostgreSQL is continuously complaing that MySQL is inferior yet way more popular. Maybe MySQL's popularity is not even PostgreSQL's goal, but I am sure a bit more would be welcome. Does MySQL have a monolithic distribution? Well obviously MySQL is missing alot of functionality that you will not get in any version of MySQL though. However it comes with replication, fulltext indexes out of the box. They currently only have a single stored procedure language (partial SQL:2003 implementation). Oh and they also ship a federated (AFAIK their dblink answer) along with several other storage engines for various specific tasks. regards, Lukas ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] Fwd: Three weeks left until feature freeze
Joshua D. Drake wrote: For example there is NOT an PostgreSQL 8.1 for Ubuntu Breezy. http://packages.ubuntu.com/breezy-backports/misc/ -- Peter Eisentraut http://developer.postgresql.org/~petere/ ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] ShmemAlloc() alignment patch
On Fri, Jul 14, 2006 at 02:50:31PM +0800, Qingqing Zhou wrote: In ShmemAlloc() we have: newStart = BUFFERALIGN(newStart); newSpace = (void *) (ShmemBase + newStart); return newSpace; Notice that though newStart is ALIGNOF_BUFFER, ShmemBase is not. Thus the newSpace is not aligned as we disired. How can ShmemBase not be aligned? Surely it's page-aligned? Have a nice day, -- Martijn van Oosterhout kleptog@svana.org http://svana.org/kleptog/ From each according to his ability. To each according to his ability to litigate. signature.asc Description: Digital signature
Re: [HACKERS] pgsql-patches considered harmful
Ühel kenal päeval, K, 2006-07-12 kell 23:04, kirjutas Marc G. Fournier: On Wed, 12 Jul 2006, Magnus Hagander wrote: There are list servers out there capable of simply ripping any attachments to a message (possibly over a certain size) and stick it on a website, replacing it with a link in the email. Is majordomo one of them? Majordomo2 has a 'hook' for it, but, like most OSS software, nobody has had the requirement to actually code it ... any perl experts here interested in doing it? Does it have to be perl ? I can do it in python in an hour or two. Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email . [EMAIL PROTECTED] MSN . [EMAIL PROTECTED] Yahoo . yscrappy Skype: hub.orgICQ . 7615664 ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match -- Hannu Krosing Database Architect Skype Technologies OÜ Akadeemia tee 21 F, Tallinn, 12618, Estonia Skype me: callto:hkrosing Get Skype for free: http://www.skype.com ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] Three weeks left until feature freeze
Ühel kenal päeval, K, 2006-07-12 kell 17:48, kirjutas Thomas Hallgren: Andrew Dunstan wrote: There is in effect no API at all, other than what is available to all backend modules. If someone wants to create an API which will be both sufficiently stable and sufficiently complete to meet the needs of the various PLs (especially, as Hannu rightly observes, any new PLs that come along) then we can revisit this question. Until then I suggest that it is at best premature. I am not even sure such a thing is actually possible. I concur with this. The needs for a module like PL/Java is very different then the needs of PL/Perl so let's get some more PL's in before we do a refactoring effort to create common API's. Personally, I'm not sure what would be included. The call handler API's together with the SPI API's are in essence what you need. The rest is fairly specialized anyway. http://pgfoundry.org/projects/python seems to do something similar to what you describe for PL/Java. For example it is using postgreSQLs native types and operators for calculations instead of converting types to pl's native types. and it also has lots of code lines ;) -- Hannu Krosing Database Architect Skype Technologies OÜ Akadeemia tee 21 F, Tallinn, 12618, Estonia Skype me: callto:hkrosing Get Skype for free: http://www.skype.com ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] monolithic distro
Lukas Smith wrote: Lukas Smith wrote: Peter Eisentraut wrote: Lukas Kahwe Smith wrote: whereas PostgreSQL is continuously complaing that MySQL is inferior yet way more popular. Maybe MySQL's popularity is not even PostgreSQL's goal, but I am sure a bit more would be welcome. Does MySQL have a monolithic distribution? Well obviously MySQL is missing alot of functionality that you will not get in any version of MySQL though. However it comes with replication, fulltext indexes out of the box. They currently only have a single stored procedure language (partial SQL:2003 implementation). Oh and they also ship a federated (AFAIK their dblink answer) along with several other storage engines for various specific tasks. Since I appreantly like monologs .. MySQL also has other features that are not available via pgfoundery like being able to determine the default charset on the database, table and column level, as well as COLLATE support to determine the sort order at runtime. Anyways what I want to make clear is simply that there are plenty of features that come with the default distro of other RDBMS that are only available via the pgfoundery. There are also plenty of features available in pgfoundry not available in any other RDBMS. However newbies that evaluate which RDBMS to use will probably never know. regards, Lukas ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] monolithic distro
Lukas Smith wrote: Since I appreantly like monologs .. MySQL also has other features that are not available via pgfoundery like being able to determine the default charset on the database, table and column level, as well as COLLATE support to determine the sort order at runtime. Anyways what I want to make clear is simply that there are plenty of features that come with the default distro of other RDBMS that are only available via the pgfoundery. There are also plenty of features available in pgfoundry not available in any other RDBMS. However newbies that evaluate which RDBMS to use will probably never know. None of these is really connected in any way with any sort of modularisation. When we get table and column level charset and collation support it will surely be in the core, and not in an addon module. The topic here is NOT what features are missing from postgres. Oh, and we *do* have per database charsets. cheers andrew ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [HACKERS] monolithic distro
Andrew Dunstan wrote: The topic here is NOT what features are missing from postgres. Of course it is ;-) Regards, Thomas Hallgren ---(end of broadcast)--- TIP 6: explain analyze is your friend
[HACKERS] xlogdump enhancements
I'm working on Tom's xlogdump tool to add some functionality.IMHO some useful improvements would be and an idea of implementation:- have an options to output only the transactions with their status and some aggregate data (transaction size). When the user pass a -t parameter, instead of printing echa record info the program uses a linked list to acumulate the total length and status info about each xid. The status begin with NOT COMMITED, it can change to COMMITED or ABORTED. If the user pass all the files in the pg_xlog directory as parameters he knows the current status of each transaction. - Find loser transactions (not commited to the end of the log)I though of doing this for the only transactions option (see above). - Have a filter to get a specifid rmidThere's always grep, but for windows users it might be interesting to filter records by rmid. There can be a flag like -r (from rmname), or -o (operation) where the user can pass a parameter like: xact, heap, etc... And then filter the output records.- Option to translate OIDs to names given a database connectionI receive parameters to open a connection (the usual -h, -p and -U). In the startup I try to open a connection. If the user passes the parameters of the connection the application automaticaly enter in the translate oids mode. Then each data returning function tries to get from the connected database the objects name. - Extract the exact SQL statement in cases of xlog generated by insert/update/delete. This is the one where I need help from you the most. I'm not certain of how to implement. But as I can see, I'll need to get object's structure from the system catalog to decode the statement. So this option will be available only if you have a connection to the database. Is this right? I'd like to have opinions on the subject. If somebody could give me some advice on how to start the last feature (decode the statemets) I'd apreciate very much.-- Diogo Biazus - [EMAIL PROTECTED]Móvel Consultoria http://www.movelinfo.com.br http://www.postgresql.org.br
Re: [HACKERS] pgsql-patches considered harmful
On Fri, 14 Jul 2006, Hannu Krosing wrote: Ühel kenal päeval, K, 2006-07-12 kell 23:04, kirjutas Marc G. Fournier: On Wed, 12 Jul 2006, Magnus Hagander wrote: There are list servers out there capable of simply ripping any attachments to a message (possibly over a certain size) and stick it on a website, replacing it with a link in the email. Is majordomo one of them? Majordomo2 has a 'hook' for it, but, like most OSS software, nobody has had the requirement to actually code it ... any perl experts here interested in doing it? Does it have to be perl ? To tie into the list manager, it has to be perl, yes ... Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email . [EMAIL PROTECTED] MSN . [EMAIL PROTECTED] Yahoo . yscrappy Skype: hub.orgICQ . 7615664 ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [HACKERS] xlogdump enhancements
On Fri, Jul 14, 2006 at 10:43:15AM -0300, Diogo Biazus wrote: I'm working on Tom's xlogdump tool to add some functionality. IMHO some useful improvements would be and an idea of implementation: snip Neato. Looks like good stuff there. - Extract the exact SQL statement in cases of xlog generated by insert/update/delete. This is the one where I need help from you the most. I'm not certain of how to implement. But as I can see, I'll need to get object's structure from the system catalog to decode the statement. So this option will be available only if you have a connection to the database. Is this right? Well, obvously you can only create dummy statements that acheives the same effect, you can't get exactly the statements executed. That's still useful though. Decoding tuples is tricky, if done externally. If you're inside the backend you could use the functions there. What you need to do is build a table of all the columns and their types. For each type you need to get the info needed to decode it. Finally, you extract the data and convert it to a readable form. If you really want to tackle this the hard way, find some other program that does it. Here one written in Perl that can decode most tuples, but not all. It fails because it doesn't recognise all the types. http://svana.org/kleptog/pgsql/pgfsck.html Have a ncie day, -- Martijn van Oosterhout kleptog@svana.org http://svana.org/kleptog/ From each according to his ability. To each according to his ability to litigate. signature.asc Description: Digital signature
Re: [HACKERS] xlogdump enhancements
On 7/14/06, Martijn van Oosterhout kleptog@svana.org wrote: If you really want to tackle this the hard way, find some other program that does it. Here one written in Perl that can decode most tuples, but not all. It fails because it doesn't recognise all the types. Yep Diogo, Martijn is correct. You have to reassemble the tuple and then generate the appropriate SQL statement for it. For this, you'll definitely need the catalog information to resolve relations, type handling, etc. -- Jonah H. Harris, Software Architect | phone: 732.331.1300 EnterpriseDB Corporation| fax: 732.331.1301 33 Wood Ave S, 2nd Floor| [EMAIL PROTECTED] Iselin, New Jersey 08830| http://www.enterprisedb.com/ ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] 8.2 features?
--On Freitag, Juli 14, 2006 01:23:11 +0200 Peter Eisentraut [EMAIL PROTECTED] wrote: . multiple values clauses for INSERT Susanne Ebrecht [EMAIL PROTECTED] was last heard to work on it. Updates, Susanne? I've talked to Susanne today and she's just back from hospital and not available online until next week. She was working on the SET (col1, col2) = (val1, val2) syntax for UPDATE commands. Don't know what the status is on this, though. -- Thanks Bernd ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [HACKERS] [PATCHES] include compile problems
Tom Lane wrote: Bruce Momjian [EMAIL PROTECTED] writes: Patch applied. Thanks. I suspect the point was that limits.h is needed *instead of* math.h, not *in addition to*. How many of those headers had math.h before? The issue was that an include file included another include file that had math.h, but that include wasn't needed, so when it was removed, a C file including the first file didn't have math.h anymore. (I didn't actually check system include file usage because that is platform-specific). Anyway, thanks for the patch you made, and glad you looked at the system includes, which I did not. -- Bruce Momjian [EMAIL PROTECTED] EnterpriseDBhttp://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. + ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] Fwd: Three weeks left until feature freeze
Peter Eisentraut wrote: Joshua D. Drake wrote: For example there is NOT an PostgreSQL 8.1 for Ubuntu Breezy. http://packages.ubuntu.com/breezy-backports/misc/ Thanks Peter :), I knew about backports but didn't know what was in there. But what about when 8.2 comes out? Doubtful that they will release for breezy because a new version of Ubuntu will be out by then. Example, Hoary doesn't have 8.1 available :) Sincerely, Joshua D. Drake -- === The PostgreSQL Company: Command Prompt, Inc. === Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240 Providing the most comprehensive PostgreSQL solutions since 1997 http://www.commandprompt.com/ ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] 8.2 features?
Bernd Helmle wrote: --On Freitag, Juli 14, 2006 01:23:11 +0200 Peter Eisentraut [EMAIL PROTECTED] wrote: . multiple values clauses for INSERT Susanne Ebrecht [EMAIL PROTECTED] was last heard to work on it. Updates, Susanne? I've talked to Susanne today and she's just back from hospital and not available online until next week. She was working on the SET (col1, col2) = (val1, val2) syntax for UPDATE commands. Don't know what the status is on this, though. Not the same thing, surely. So maybe we should gratefully accept Joe Conway's offer to work on it. cheers andrew ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
[HACKERS] contrib promotion?
There has been action to clean up and remove some contrib modules, and this is good. I would like to suggest that we should try to move one or two the other way, namely right into the core proper, on the ground that they have widespread applicability and should have maximum visibility. I'm talking particularly about pgcrypto and tsearch2, and the main thing lacking that I see with these is documentation. Thoughts? cheers andrew ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [HACKERS] xlogdump enhancements
On 7/14/06, Jonah H. Harris [EMAIL PROTECTED] wrote: On 7/14/06, Martijn van Oosterhout kleptog@svana.org wrote: If you really want to tackle this the hard way, find some other program that does it. Here one written in Perl that can decode most tuples, but not all. It fails because it doesn't recognise all the types.Yep Diogo, Martijn is correct.You have to reassemble the tuple andthen generate the appropriate SQL statement for it.For this, you'll definitely need the catalog information to resolve relations, typehandling, etc.Thanks for the info, I'm taking a look in that program. Is good to have a starting point :-)-- Diogo Biazus - [EMAIL PROTECTED]Móvel Consultoriahttp://www.movelinfo.com.brhttp://www.postgresql.org.br
Re: [HACKERS] monolithic distro
Since I appreantly like monologs .. MySQL also has other features that are not available via pgfoundery like being able to determine the default charset on the database, table and column level, as well as COLLATE support to determine the sort order at runtime. SHOW ALL; ? Anyways what I want to make clear is simply that there are plenty of features that come with the default distro of other RDBMS that are only available via the pgfoundery. There are also plenty of features available in pgfoundry not available in any other RDBMS. However newbies that evaluate which RDBMS to use will probably never know. regards, Lukas ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq -- === The PostgreSQL Company: Command Prompt, Inc. === Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240 Providing the most comprehensive PostgreSQL solutions since 1997 http://www.commandprompt.com/ ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [HACKERS] [PATCHES] putting CHECK_FOR_INTERRUPTS in qsort_comparetup()
Charles Duffy [EMAIL PROTECTED] writes: On 7/12/06, Qingqing Zhou [EMAIL PROTECTED] wrote: the problem here is that 29247 doesn't look like a big number so I can't see why your patch solved the problem, unless the qsort_comparetup() function of the data type eats too many circles or the cpu is too slow. Well...when exhibiting the problem, execution stays inside qsort() for the entire 'delay period' - I don't think its off in some other recess which is also lacking in interrupt flag checking. As for the CPU - the machine is a 4 way Opteron, and otherwise performs well. It does seem odd that the in-memory sort takes as long as it does - the plan may suggest a reason. Well, there's something awfully fishy here. Compare the two lower-level sorts in your EXPLAIN output: - Sort (cost=2909.44..2944.94 rows=14201 width=320) (actual time=78196.698..78239.799 rows=29247 loops=1) Sort Key: record, commr1, envr1, docin, creat, flati, flato, doc, docst, vlord, vl0, vl1, vl2, vl3, vl4, vl5, vl6, vl7, vl8, vl9 - Append (cost=0.00..1930.02 rows=14201 width=320) (actual time=0.052..396.577 rows=29247 loops=1) - Sort (cost=403.42..403.59 rows=71 width=320) (actual time=318.727..339.305 rows=10932 loops=1) Sort Key: record, commr1, envr1, docin, creat, flati, flato, doc, docst, vlord, vl0, vl1, vl2, vl3, vl4, vl5, vl6, vl7, vl8, vl9 - Append (cost=78.88..401.23 rows=71 width=320) (actual time=5.197..192.868 rows=10932 loops=1) The first one took 77843.222 msec to sort 29247 rows, the second took 146.437 msec to sort 10932 rows with what appears to be the same sort key. One of these things is not like the other ... What are the data types of the sort columns? Is there anything much different in the statistics of the two subqueries --- for example, maybe one produces a lot of output that only differs in the rightmost sort columns, where the other has keys that always differ in a leading column? I forget if you are running 8.1 or not, but if you are, turning on trace_sort might produce useful information. regards, tom lane ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] ShmemAlloc() alignment patch
Martijn van Oosterhout kleptog@svana.org writes: On Fri, Jul 14, 2006 at 02:50:31PM +0800, Qingqing Zhou wrote: Notice that though newStart is ALIGNOF_BUFFER, ShmemBase is not. Thus the newSpace is not aligned as we disired. How can ShmemBase not be aligned? Surely it's page-aligned? That's certainly what the code expects. I'm disinclined to apply this patch unless you can identify a real system where ShmemBase might not point to a page boundary. (Note: in a standalone backend, the shared memory segment is just a huge malloc chunk, and so depending on your platform it might not be page-aligned. I don't feel a need to add cycles to ShmemAlloc to optimize this case, though. We only care about performance in the normal shared-memory case.) regards, tom lane ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] ShmemAlloc() alignment patch
Tom Lane wrote: Martijn van Oosterhout kleptog@svana.org writes: On Fri, Jul 14, 2006 at 02:50:31PM +0800, Qingqing Zhou wrote: Notice that though newStart is ALIGNOF_BUFFER, ShmemBase is not. Thus the newSpace is not aligned as we disired. How can ShmemBase not be aligned? Surely it's page-aligned? That's certainly what the code expects. I'm disinclined to apply this patch unless you can identify a real system where ShmemBase might not point to a page boundary. (Note: in a standalone backend, the shared memory segment is just a huge malloc chunk, and so depending on your platform it might not be page-aligned. I don't feel a need to add cycles to ShmemAlloc to optimize this case, though. We only care about performance in the normal shared-memory case.) Should we add an assert? -- Bruce Momjian [EMAIL PROTECTED] EnterpriseDBhttp://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. + ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [HACKERS] ShmemAlloc() alignment patch
Bruce Momjian [EMAIL PROTECTED] writes: How can ShmemBase not be aligned? Surely it's page-aligned? Should we add an assert? No, because even if it's not page-aligned, there's no correctness issue here. (Besides, how would you know what the page size is on any given platform?) regards, tom lane ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
[HACKERS] Forcing wal rotation
Hi For my warm-standby-cluster I'm now saving the currently used wal using rsync, to avoid loosing data from a few hours (or days) ago, when there is little traffic, and thus the wal isn't rotated. For online backups, the problem is even worse, because a backup might me unuseable even hours after I called pg_stop_backup(), because the wal segment needed to bring the backup to a consistent state might not have been archived at that time. I've now thought about how to fix that without doing that rather crude rsync-pg_xlog-hack. I've read through the code, and learned that wal-segments are expected to have a specific size - thus rotating them early is not that easy. But now I figured that another short-term solution could probably be implemented easily. I'd like to write a function that just fills the wal with nop records, until the current used wal is full. Since I assume that there are already different kind of wal records, adding a NOP-record that just takes up space, and does nothing else, should be quite easy. And even if wals contain only one kind of record (Write this page to that datafile), I could just repeat the last records over and over again, until the wal is filled, couldn't I? Do you think that this is feasable? If so, I'd like to try it. greetings, Florian Pflug ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [HACKERS] Forcing wal rotation
A.M. wrote: On Fri, July 14, 2006 11:20 am, Florian G. Pflug wrote: Hi For my warm-standby-cluster I'm now saving the currently used wal using rsync, to avoid loosing data from a few hours (or days) ago, when there is little traffic, and thus the wal isn't rotated. For online backups, the problem is even worse, because a backup might me unuseable even hours after I called pg_stop_backup(), because the wal segment needed to bring the backup to a consistent state might not have been archived at that time. How about an SQL-level function that calls the wal scripts? This would also allow important transactions to push data to the standy server regardless of the wal size. That was the idea - providing pg_rotate_wal(), which would guarantee that the wal is rotatted at least once if called. Thinking further about this, for a first prove of concept, I'd be enough to write a C function pg_current_walsegment(). pg_rotate_wal() could then be a plpgsql function, that e.g. creates a temporary table, and fills it with data, until the return value of pg_current_walsegment() changes. ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] Forcing wal rotation
On Fri, Jul 14, 2006 at 05:36:58PM +0200, Florian G. Pflug wrote: That was the idea - providing pg_rotate_wal(), which would guarantee that the wal is rotatted at least once if called. Thinking further about this, for a first prove of concept, I'd be enough to write a C function pg_current_walsegment(). pg_rotate_wal() could then be a plpgsql function, that e.g. creates a temporary table, and fills it with data, until the return value of pg_current_walsegment() changes. Temporary tables don't get xlogged. It would probably be easier to hook into the xlog machinery and create NOP records, like you originally suggested. Have a nice day, -- Martijn van Oosterhout kleptog@svana.org http://svana.org/kleptog/ From each according to his ability. To each according to his ability to litigate. signature.asc Description: Digital signature
Re: [HACKERS] Forcing wal rotation
Martijn van Oosterhout wrote: On Fri, Jul 14, 2006 at 05:36:58PM +0200, Florian G. Pflug wrote: That was the idea - providing pg_rotate_wal(), which would guarantee that the wal is rotatted at least once if called. Thinking further about this, for a first prove of concept, I'd be enough to write a C function pg_current_walsegment(). pg_rotate_wal() could then be a plpgsql function, that e.g. creates a temporary table, and fills it with data, until the return value of pg_current_walsegment() changes. Temporary tables don't get xlogged. It would probably be easier to hook into the xlog machinery and create NOP records, like you originally suggested. From further sourcecode reading, I got the following implementation plan .) Add new entry to RmgrTable (Should I add it at the end, or use one of the reserved entries?) .) Create nop_redo and nop_desc - for nop_redo an empty function should be sufficient .) Create pg_rotate_wal, which calls XLogInsert(RM_NOP_ID, XLOG_NO_TRAN, rdata) with rdata.data = pointer to WAL_SEGMENT_SIZE zero bytes rdata.len = WAL_SEGMENT_SIZE rdata.buffer = InvalidBuffer rdata.next = NULL Since I insert WAL_SIZE bytes, I shouldn't even have to loop, because that records has no chance to fit into the current wal segment, right? Am I overlooking something? greetings, Florian Pflug ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [HACKERS] contrib promotion?
Andrew Dunstan [EMAIL PROTECTED] writes: There has been action to clean up and remove some contrib modules, and this is good. I would like to suggest that we should try to move one or two the other way, namely right into the core proper, on the ground that they have widespread applicability and should have maximum visibility. I'm talking particularly about pgcrypto and tsearch2, and the main thing lacking that I see with these is documentation. I don't see a strong need for moving pgcrypto into core, and there's at least one argument against it: if someone needs a crypto-free version of postgres for use someplace with benighted laws, they would be screwed. tsearch2 is functionality that definitely should be in core eventually, but even Oleg still says it's not done. Aside from the documentation issue, it's not clear that we've got a stable API for it. regards, tom lane ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] [COMMITTERS] pgsql: Have find_static skip main() functions.
[EMAIL PROTECTED] (Bruce Momjian) writes: Have find_static skip main() functions. Uh-oh, don't tell me you are cranking up to run *that* thing again. This time around, please do not remove API functions just because you can't find a reference to them in the core code. I would like to see a posted, discussed patch first. regards, tom lane ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Forcing wal rotation
Florian G. Pflug [EMAIL PROTECTED] writes: I've now thought about how to fix that without doing that rather crude rsync-pg_xlog-hack. I've read through the code, and learned that wal-segments are expected to have a specific size - thus rotating them early is not that easy. Simon was working on a patch for this at the code sprint; I think it's submitted to -patches already. Explicitly filling the segment as you propose would be really bad for performance. regards, tom lane ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] Forcing wal rotation
-Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Florian G. Pflug Sent: 14 July 2006 16:37 To: Postgresql-General Cc: A.M. Subject: Re: [HACKERS] Forcing wal rotation How about an SQL-level function that calls the wal scripts? This would also allow important transactions to push data to the standy server regardless of the wal size. That was the idea - providing pg_rotate_wal(), which would guarantee that the wal is rotatted at least once if called. Thinking further about this, for a first prove of concept, I'd be enough to write a C function pg_current_walsegment(). pg_rotate_wal() could then be a plpgsql function, that e.g. creates a temporary table, and fills it with data, until the return value of pg_current_walsegment() changes. You might wanna speak to Simon Riggs before starting anything - he was planning to work on this (log rotation) at the Code Sprint. Don't know if he actually got to it though. Regards, Dave. ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] contrib promotion?
Tom Lane wrote: Andrew Dunstan [EMAIL PROTECTED] writes: There has been action to clean up and remove some contrib modules, and this is good. I would like to suggest that we should try to move one or two the other way, namely right into the core proper, on the ground that they have widespread applicability and should have maximum visibility. I'm talking particularly about pgcrypto and tsearch2, and the main thing lacking that I see with these is documentation. I don't see a strong need for moving pgcrypto into core, and there's at least one argument against it: if someone needs a crypto-free version of postgres for use someplace with benighted laws, they would be screwed. Could that be handled with a configure option? tsearch2 is functionality that definitely should be in core eventually, but even Oleg still says it's not done. Aside from the documentation issue, it's not clear that we've got a stable API for it. Well, that's a pity. This will be the 4th release with it in contrib, IIRC. I know it's advanced stuff, but surely it has to settle down sometime. Quite apart from anything else, it's important that we do get better docco on these modules. cheers andrew ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] [PATCHES] [patch 0/9] annual pgcrypto update
On 7/13/06, Neil Conway [EMAIL PROTECTED] wrote: Marko, can you take a look at what is causing this regression test failure? The failing machine is kudu: Seems you have skipped the CAST5 patch. Could you recheck? -- marko ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] contrib promotion?
Tom Lane wrote: Andrew Dunstan [EMAIL PROTECTED] writes: There has been action to clean up and remove some contrib modules, and this is good. I would like to suggest that we should try to move one or two the other way, namely right into the core proper, on the ground that they have widespread applicability and should have maximum visibility. I'm talking particularly about pgcrypto and tsearch2, and the main thing lacking that I see with these is documentation. I don't see a strong need for moving pgcrypto into core, and there's at least one argument against it: if someone needs a crypto-free version of postgres for use someplace with benighted laws, they would be screwed. Doesn't our inclusion of md5() pretty much blow that argument away? (Just asking). Sincerely, Joshua D. Drake -- === The PostgreSQL Company: Command Prompt, Inc. === Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240 Providing the most comprehensive PostgreSQL solutions since 1997 http://www.commandprompt.com/ ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] contrib promotion?
Joshua D. Drake [EMAIL PROTECTED] writes: Tom Lane wrote: I don't see a strong need for moving pgcrypto into core, and there's at least one argument against it: if someone needs a crypto-free version of postgres for use someplace with benighted laws, they would be screwed. Doesn't our inclusion of md5() pretty much blow that argument away? No: md5 is hashing, not encryption. The difference is that you can't retrieve the original plaintext from a hash. That is a very large difference in the eyes of most munitions laws --- encryption is useful for spies, hashing not so much. regards, tom lane ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] Forcing wal rotation
On Fri, 2006-07-14 at 12:09 -0400, Tom Lane wrote: Florian G. Pflug [EMAIL PROTECTED] writes: I've now thought about how to fix that without doing that rather crude rsync-pg_xlog-hack. I've read through the code, and learned that wal-segments are expected to have a specific size - thus rotating them early is not that easy. Simon was working on a patch for this at the code sprint; I think it's submitted to -patches already. Slightly different patch. I'm working on this one still. Explicitly filling the segment as you propose would be really bad for performance. Yes, current approach I am taking is better than that. -- Simon Riggs EnterpriseDB http://www.enterprisedb.com ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [HACKERS] contrib promotion?
On Jul 14, 2006, at 12:32 PM, Joshua D. Drake wrote: Doesn't our inclusion of md5() pretty much blow that argument away? (Just asking). I don't think so because md5 is just a one way hash function. There is no method to decrypt anything :). John DeSoi, Ph.D. http://pgedit.com/ Power Tools for PostgreSQL ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] contrib promotion?
Andrew Dunstan wrote: I don't see a strong need for moving pgcrypto into core, and there's at least one argument against it: if someone needs a crypto-free version of postgres for use someplace with benighted laws, they would be screwed. Could that be handled with a configure option? In such a jurisdiction you would possibly have to distribute rebuilt tarballs with all the source code removed. The contrib option makes that much easier. Quite apart from anything else, it's important that we do get better docco on these modules. I'm willing to help with DocBook options. What do you have in mind? -- Peter Eisentraut http://developer.postgresql.org/~petere/ ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] [PATCHES] build with different options than Bruce
Kris Jurka [EMAIL PROTECTED] writes: When building with --enable-cassert, without --enable-thread-safety, or when the OS supports USE_WIDE_UPPER_LOWER we need some more include files. Done, thanks. regards, tom lane ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] [PATCHES] c.h is the problem of msvc.
Hiroshi Saito [EMAIL PROTECTED] writes: --- src/include/c.h.orig Sat Jul 15 01:38:59 2006 +++ src/include/c.h Sat Jul 15 01:40:04 2006 @@ -60,7 +60,9 @@ #if defined(_MSC_VER) || defined(__BORLANDC__) #define WIN32_ONLY_COMPILER #define errcode __vc_errcode +#if (_MSC_VER 1400) #include crtdefs.h +#endif #undef errcode #endif This patch certainly looks like it will break the __BORLANDC__ build ... regards, tom lane ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] [PATCHES] [patch 0/9] annual pgcrypto update
I wrote: Applied, thanks. What I now see is that pgp-pubkey-decrypt passes on a 32-bit machine but dumps core on a 64-bit machine, with SIGSEGV here: Addendum: seems it only fails without openssl. regards, tom lane ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] contrib promotion?
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Doesn't our inclusion of md5() pretty much blow that argument away? (Just asking). I don't think so because md5 is just a one way hash function. There is no method to decrypt anything :). Actually, I've had to install pgcrypto on more than one occasion for clients who needed to have sha1 instead of md5. I've had to install pgcrypto for other functions as well, so +1 for me on coring it, but at the least please consider adding in sha1. - -- Greg Sabino Mullane [EMAIL PROTECTED] PGP Key: 0x14964AC8 200607141355 http://biglumber.com/x/web?pk=2529DF6AB8F79407E94445B4BC9B906714964AC8 -BEGIN PGP SIGNATURE- iD8DBQFEt9sfvJuQZxSWSsgRAhrCAJ9VZneUN6+3pvq+vJf+Y6lhjamj/QCfUDYc +rYM+ITWvhw2dvo2B1hta6g= =h4aD -END PGP SIGNATURE- ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [HACKERS] contrib promotion?
Greg Sabino Mullane wrote: Doesn't our inclusion of md5() pretty much blow that argument away? (Just asking). I don't think so because md5 is just a one way hash function. There is no method to decrypt anything :). Actually, I've had to install pgcrypto on more than one occasion for clients who needed to have sha1 instead of md5. I've had to install pgcrypto for other functions as well, so +1 for me on coring it, but at the least please consider adding in sha1. I don't have a very strong opinion on that but sha1() is something I need on a regular base too from pgcrypto. Stefan ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
[HACKERS] src/tools/pginclude considered harmful (was Re: [PATCHES] toast index entries again)
Kris Jurka [EMAIL PROTECTED] writes: The inclusion of access/tuptoaster.h in access/common/indextuple.c brought in the define of TOAST_INDEX_HACK which compresses large index entries. When this was removed the entries were no longer compressed which caused btree_gist to fail. This is seriously scary, as it seems quite possible that there are other similar cases in which we have simply silently lost functionality or performance, and none of the regression tests will expose it. In combination with the amount of time wasted over the past two days, it is now perfectly clear that the existing pginclude tools are not NEARLY good enough to detect what they are breaking. I would like to propose that we revert all the include-related changes of the past two days, and that src/tools/pginclude be removed from the CVS tree, until such time as it is rewritten to be much smarter about what it is doing. regards, tom lane ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [HACKERS] [PATCHES] putting CHECK_FOR_INTERRUPTS in qsort_comparetup()
Charles Duffy [EMAIL PROTECTED] writes: [ CHECK_FOR_INTERRUPTS inside qsort comparison routine ] It occurs to me that there's a nonzero risk of problems here, because if the interrupt occurs qsort() will lose control. I'm wondering whether there are any implementations of qsort() that allocate memory and then release it before returning. If so, interrupting the sort would lead to memory leaked permanently (till backend exit anyway). We might have to just tolerate this, but if it occurs on a lot of platforms I'd have second thoughts about applying the patch. Anyone familiar with the internals of glibc's qsort, in particular? regards, tom lane ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] contrib promotion?
Peter Eisentraut wrote: Quite apart from anything else, it's important that we do get better docco on these modules. I'm willing to help with DocBook options. What do you have in mind? Well, if we could make provision for sucking in a chapter per contrib module if it exists that would be a good start. (Speaking of DocBook, did we ever make progress on switching to XML?) cheers andrew ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] [PATCHES] putting CHECK_FOR_INTERRUPTS in qsort_comparetup()
Tom Lane wrote: We might have to just tolerate this, but if it occurs on a lot of platforms I'd have second thoughts about applying the patch. Anyone familiar with the internals of glibc's qsort, in particular? Doesn't look like it's allocating any nonlocal memory: http://sourceware.org/cgi-bin/cvsweb.cgi/libc/stdlib/qsort.c?rev=1.12content-type=text/x-cvsweb-markupcvsroot=glibc -- Peter Eisentraut http://developer.postgresql.org/~petere/ ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] [PATCHES] putting CHECK_FOR_INTERRUPTS in qsort_comparetup()
Peter Eisentraut [EMAIL PROTECTED] writes: Tom Lane wrote: We might have to just tolerate this, but if it occurs on a lot of platforms I'd have second thoughts about applying the patch. Anyone familiar with the internals of glibc's qsort, in particular? Doesn't look like it's allocating any nonlocal memory: http://sourceware.org/cgi-bin/cvsweb.cgi/libc/stdlib/qsort.c?rev=1.12content-type=text/x-cvsweb-markupcvsroot=glibc But this file defines _quicksort() not qsort(). I was under the impression that the latter is actually a mergesort in glibc ... regards, tom lane ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] src/tools/pginclude considered harmful (was Re: [PATCHES]
Tom Lane wrote: In combination with the amount of time wasted over the past two days, it is now perfectly clear that the existing pginclude tools are not NEARLY good enough to detect what they are breaking. I would like to propose that we revert all the include-related changes of the past two days, and that src/tools/pginclude be removed from the CVS tree, until such time as it is rewritten to be much smarter about what it is doing. I agree with reverting. The tool looks pretty broken anyway, with hardcoded paths and all sorts of stuff quite apart from logic problems. cheers andrew ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] [PATCHES] putting CHECK_FOR_INTERRUPTS in qsort_comparetup()
Tom Lane wrote: Doesn't look like it's allocating any nonlocal memory: http://sourceware.org/cgi-bin/cvsweb.cgi/libc/stdlib/qsort.c?rev=1. 12content-type=text/x-cvsweb-markupcvsroot=glibc But this file defines _quicksort() not qsort(). I was under the impression that the latter is actually a mergesort in glibc ... The merge sort is here: http://sourceware.org/cgi-bin/cvsweb.cgi/libc/stdlib/msort.c?rev=1.21content-type=text/x-cvsweb-markupcvsroot=glibc It uses alloca, so we're good here. -- Peter Eisentraut http://developer.postgresql.org/~petere/ ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] src/tools/pginclude considered harmful (was Re: [PATCHES]
Andrew Dunstan [EMAIL PROTECTED] writes: I agree with reverting. The tool looks pretty broken anyway, with hardcoded paths and all sorts of stuff quite apart from logic problems. Well, it's only intended to work on Bruce's system, so until someone else takes over the position of chief gruntwork-doer I don't think the portability issues are much of a factor. What I'm worried about at the moment is the correctness issue. After some reflection it seems that there is only one case where removal of a needed include file would not lead to a compiler error or warning, assuming gcc with ordinary -W settings (notably -Wmissing-prototypes). That case is exactly what Kris found: removal of a #define that is tested via #ifdef or #if defined(). (Can anyone think of other cases?) It's not hard to imagine getting burnt by this through ordinary manual rearrangement or cleanup of inclusions, so I'm thinking that blaming pgrminclude may be too hasty. It'd be worth our time to make a tool to check for it. I'm going to work on a Perl script that sucks up all the #defines in all our .h files and looks for #ifdef foo or defined foo where foo is defined in a file not included by the referencing file (gcc -H looks like the right tool for checking that). Another thing such a script could easily do is check for inclusion of system headers before c.h. regards, tom lane ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] Three weeks left until feature freeze
Hi, Bruce Momjian írta: There are roughly three weeks left until the feature freeze on August 1. If people are working on items, they should be announced before August 1, and the patches submitted by August 1. If the patch is large, it should be discussed now and an intermediate patch posted to the lists soon. FYI, we don't have many major features ready for 8.2. -- Bruce Momjian [EMAIL PROTECTED] EnterpriseDBhttp://www.enterprisedb.com I am working on adding a new column contraint, namely the GENERATED [ALWAYS | BY DEFAULT ] AS [ IDENTITY ( sequence_options ) | ( expression )] I surely won't be ready by August 1 as I am going on a holiday this weekend for two weeks. Anyway, I submit what I have in my hands now and describe what it does and what lacks. Maybe someone can fill the missing gaps, I didn't have too much time to dig deeper in the PostgreSQL code. Lacks: - proper sgml documentation - pg_dump support - GENERATED ALWAYS doesn't work (yet[*]) - ALTER TABLE support What works: - GENERATED ... AS IDENTITY creates the sequence with the given parameters - any data types that has an implicit cast from int8 can be defined as GENERATED ... AS IDENTITY DEFAULT and GENERATED are mutually exclusive and I tweaked gram.y to make it so and GENERATED ... is now parsed as a column constraint. Unfortunately I introduced one shift/reduce conflict and I don't see a way to solve it. SQL2003 says that only one IDENTITY column may exist in a table, I didn't make a check for that, it may be useful to allow more than one. Also, the parsing is very simplistic, it allows GENERATED BY DEFAULT AS ( expr ) and I made it to behave exactly like DEFAULT. It can be eliminated in the parser, though. Tom Lane was unhappy with my previous attempt as it introduced SERIAL, SERIAL4, SERIAL8 and BIGSERIAL as keywords. This time I kept it the way PostgreSQL always had. [*] I introduced a new column attribute (attforceddef) and it could be used in rewriteHandler.c to override given data with the default for GENERATED ALWAYS column. I must have missed some places where I have to assign ColumnDef-force_default to FormData_pg_attribute-attforceddef or vice-versa but I am certain that not too many left. Needless to say it passes make check. Please, review and point out my mistakes again. :-) Best regards, Zoltán Böszörményi psql-serial-8.diff.gz Description: Unix tar archive ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] src/tools/pginclude considered harmful (was Re: [PATCHES]
Tom Lane wrote: Andrew Dunstan [EMAIL PROTECTED] writes: I agree with reverting. The tool looks pretty broken anyway, with hardcoded paths and all sorts of stuff quite apart from logic problems. Well, it's only intended to work on Bruce's system, so until someone else takes over the position of chief gruntwork-doer I don't think the portability issues are much of a factor. Shouldn't the README reflect that, then? cheers andrew ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] [PATCHES] putting CHECK_FOR_INTERRUPTS in qsort_comparetup()
Peter Eisentraut [EMAIL PROTECTED] writes: The merge sort is here: http://sourceware.org/cgi-bin/cvsweb.cgi/libc/stdlib/msort.c?rev=1.21content-type=text/x-cvsweb-markupcvsroot=glibc It uses alloca, so we're good here. Uh ... but it also uses malloc, and potentially a honkin' big malloc at that (up to a quarter of physical RAM). So I'm worried again. Anyway, Qingqing's question still needs to be answered: how can a sort of under 30k items take so long? regards, tom lane ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] src/tools/pginclude considered harmful (was Re:
Tom Lane wrote: Martijn van Oosterhout kleptog@svana.org writes: On Fri, Jul 14, 2006 at 04:24:59PM -0400, Tom Lane wrote: After some reflection it seems that there is only one case where removal of a needed include file would not lead to a compiler error or warning, assuming gcc with ordinary -W settings (notably -Wmissing-prototypes). That case is exactly what Kris found: removal of a #define that is tested via #ifdef or #if defined(). (Can anyone think of other cases?) My off-the-top-of-my-head solution would be a script that would pass each file through gcc -E (the preprocessor), and compare before and after rearrangement. You'd have to ignore the effects of included header files, but it would pick up the cases where a block of code that was previously included no longer is. Or if a macro is expanded differently... You'd still have to try to compile the code though; AFAICS the above doesn't catch whether you've removed a typedef or function declaration that's referenced in the file. BTW, one of the remaining holes in pgrminclude is that it compiles with -fsyntax-only, which apparently causes it to fail to detect some errors of significance --- I assume that's how it managed to foul up lmgr.c, inet_net_ntop.c, etc. But I do run a full compile and regression before commit, so that should have caught it even if pgrminclude didn't. -- Bruce Momjian [EMAIL PROTECTED] EnterpriseDBhttp://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. + ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] src/tools/pginclude considered harmful (was Re:
Andrew Dunstan wrote: Tom Lane wrote: Andrew Dunstan [EMAIL PROTECTED] writes: I agree with reverting. The tool looks pretty broken anyway, with hardcoded paths and all sorts of stuff quite apart from logic problems. Well, it's only intended to work on Bruce's system, so until someone else takes over the position of chief gruntwork-doer I don't think the portability issues are much of a factor. Shouldn't the README reflect that, then? The script should work on any computer. I have just never tried it. Feel free to send in a patch to make it work on yours. -- Bruce Momjian [EMAIL PROTECTED] EnterpriseDBhttp://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. + ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] src/tools/pginclude considered harmful (was Re:
Tom Lane wrote: Bruce Momjian [EMAIL PROTECTED] writes: Tom Lane wrote: BTW, one of the remaining holes in pgrminclude is that it compiles with -fsyntax-only, which apparently causes it to fail to detect some errors of significance --- I assume that's how it managed to foul up lmgr.c, inet_net_ntop.c, etc. But I do run a full compile and regression before commit, so that should have caught it even if pgrminclude didn't. Doesn't catch warnings (unless you add -Werror during the build, which will have problems with the remaining expected warnings...) /tools/pgtest always greps out the warnings and displays them at the end of the regression tests. -- Bruce Momjian [EMAIL PROTECTED] EnterpriseDBhttp://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. + ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [HACKERS] src/tools/pginclude considered harmful (was Re: [PATCHES]
On Fri, Jul 14, 2006 at 04:24:59PM -0400, Tom Lane wrote: After some reflection it seems that there is only one case where removal of a needed include file would not lead to a compiler error or warning, assuming gcc with ordinary -W settings (notably -Wmissing-prototypes). That case is exactly what Kris found: removal of a #define that is tested via #ifdef or #if defined(). (Can anyone think of other cases?) My off-the-top-of-my-head solution would be a script that would pass each file through gcc -E (the preprocessor), and compare before and after rearrangement. You'd have to ignore the effects of included header files, but it would pick up the cases where a block of code that was previously included no longer is. Or if a macro is expanded differently... Have a nice day, -- Martijn van Oosterhout kleptog@svana.org http://svana.org/kleptog/ From each according to his ability. To each according to his ability to litigate. signature.asc Description: Digital signature
Re: [HACKERS] Three weeks left until feature freeze
Zoltan Boszormenyi [EMAIL PROTECTED] writes: I am working on adding a new column contraint, namely the GENERATED [ALWAYS | BY DEFAULT ] AS [ IDENTITY ( sequence_options ) | ( expression )] Doesn't this still have the issue that we're taking over spec-defined syntax to represent behavior that does not quite match the spec-defined semantics? It's not clear to me how closely tied this syntax is to NEXT VALUE FOR, but if it's got any of the latter's action-at-a-distance subtleties then I think we ought to leave well enough alone till we have a solution for that. regards, tom lane ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] src/tools/pginclude considered harmful (was Re:
On Fri, 2006-07-14 at 14:20 -0400, Tom Lane wrote: I would like to propose that we revert all the include-related changes of the past two days, and that src/tools/pginclude be removed from the CVS tree, until such time as it is rewritten to be much smarter about what it is doing. Rather than reverting the changes in CVS and then redoing them correctly, perhaps we could make the necessary improvements to the tools, apply the improved tools to the pre-cleaned-up version of the tree, get a diff against HEAD, and then apply any fixes the improved tools have made as a patch. That would avoid cluttering CVS with two redundant changes to almost every single source file in the tree. In any case, I agree that the cleanup was not very well done. Widespread cleanup of this sort should be done with a lot more care. One useful improvement would be to make the changes on a local repository first, and then post the suggested changes as a patch. Making the entire set of required changes as a single CVS commit would also be good: it makes it easier for folks to back out the patch locally if it causes problems, as well as making the CVS history more comprehensible. -Neil ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] [PATCHES] c.h is the problem of msvc.
Ooops, I am uncertain at the reason for not knowing __BORLANDC__... It will be sure if __BORLANDC__ has the definition. Thanks. Regards, Hiroshi Saito From: Tom Lane Hiroshi Saito [EMAIL PROTECTED] writes: --- src/include/c.h.orig Sat Jul 15 01:38:59 2006 +++ src/include/c.h Sat Jul 15 01:40:04 2006 @@ -60,7 +60,9 @@ #if defined(_MSC_VER) || defined(__BORLANDC__) #define WIN32_ONLY_COMPILER #define errcode __vc_errcode +#if (_MSC_VER 1400) #include crtdefs.h +#endif #undef errcode #endif This patch certainly looks like it will break the __BORLANDC__ build ... regards, tom lane msvc_c_patch Description: Binary data ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [HACKERS] src/tools/pginclude considered harmful (was Re: [PATCHES]
Bruce Momjian [EMAIL PROTECTED] writes: Tom Lane wrote: BTW, one of the remaining holes in pgrminclude is that it compiles with -fsyntax-only, which apparently causes it to fail to detect some errors of significance --- I assume that's how it managed to foul up lmgr.c, inet_net_ntop.c, etc. But I do run a full compile and regression before commit, so that should have caught it even if pgrminclude didn't. Doesn't catch warnings (unless you add -Werror during the build, which will have problems with the remaining expected warnings...) regards, tom lane ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [HACKERS] src/tools/pginclude considered harmful (was Re: [PATCHES]
Martijn van Oosterhout kleptog@svana.org writes: On Fri, Jul 14, 2006 at 04:24:59PM -0400, Tom Lane wrote: After some reflection it seems that there is only one case where removal of a needed include file would not lead to a compiler error or warning, assuming gcc with ordinary -W settings (notably -Wmissing-prototypes). That case is exactly what Kris found: removal of a #define that is tested via #ifdef or #if defined(). (Can anyone think of other cases?) My off-the-top-of-my-head solution would be a script that would pass each file through gcc -E (the preprocessor), and compare before and after rearrangement. You'd have to ignore the effects of included header files, but it would pick up the cases where a block of code that was previously included no longer is. Or if a macro is expanded differently... You'd still have to try to compile the code though; AFAICS the above doesn't catch whether you've removed a typedef or function declaration that's referenced in the file. BTW, one of the remaining holes in pgrminclude is that it compiles with -fsyntax-only, which apparently causes it to fail to detect some errors of significance --- I assume that's how it managed to foul up lmgr.c, inet_net_ntop.c, etc. regards, tom lane ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] contrib promotion?
tsearch2 is functionality that definitely should be in core eventually, but even Oleg still says it's not done. Aside from the documentation issue, it's not clear that we've got a stable API for it. Issues/TODO to move tsearch2 into core (by fast look) * memory management. Dictionaries and tsearch2 itself cache a lot of data in memory allocated by malloc/palloc(TopContext) and not all can be correctly freed by reset_tsearch() function. * tsearch2 doesn't automatically reinit dictionary/configuration after configuration changes * Also, dictionary doesn't automatically recompile its files if they were changed * It will be good to store shared information (such as dictionary's structure) in shared memory. For example, ispell dict takes for compile its files about 1-5 seconds, sharing should help * I suppose, at least snowball stemmers should be compiled in postgresql or compiled them into separate *.so libs. Sources of all snowball's files take ~1Mb on disk. The goal is easier configuration for end-user. Ispell dict's files take 1-3 Mb per language. * Now database can be created with another encoding then cluster was initialized, but locale can't be changed. Tsearch2 depends on locale because of lower/upper casing and isalpha(and somr another is*). So, it's possible to get unworking tsearch2 configuration. I'm going to vacation tomorrow for two week, so I havn't time to resolve that issues before feature freeze, sorry. ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] Forcing wal rotation
Simon Riggs wrote: On Fri, 2006-07-14 at 12:09 -0400, Tom Lane wrote: Florian G. Pflug [EMAIL PROTECTED] writes: I've now thought about how to fix that without doing that rather crude rsync-pg_xlog-hack. I've read through the code, and learned that wal-segments are expected to have a specific size - thus rotating them early is not that easy. Simon was working on a patch for this at the code sprint; I think it's submitted to -patches already. Slightly different patch. I'm working on this one still. Cool - what are the chances of this being included in 8.2? Explicitly filling the segment as you propose would be really bad for performance. Yes, current approach I am taking is better than that. Well, my proposal wasn't really a long-term solution - I was thinking about I quick fix that I could implement for 8.1, basically to let my warm-standby-setup feel less like as house of cards as someone put it ;-) I didn't care too much about the performance hit - I don't expect the database I indent to use it for to have much load, otherwise the wal segments are rotated quite often anyway. But I agree that for a general solution, my approach is not really ideal ;-) Since we just ported the application in question to 8.1, I'm not sure that we will switch to 8.2 when it is released - so I'm still interested in finding a solution for 8.1 Do you think I could backport your patch to 8.1 - or does it depend on some other new features of 8.2? greetings, Florian Pflug ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] src/tools/pginclude considered harmful (was Re: [PATCHES] toast index entries again)
Neil Conway [EMAIL PROTECTED] writes: On Fri, 2006-07-14 at 14:20 -0400, Tom Lane wrote: I would like to propose that we revert all the include-related changes of the past two days, and that src/tools/pginclude be removed from the CVS tree, until such time as it is rewritten to be much smarter about what it is doing. Rather than reverting the changes in CVS and then redoing them correctly, perhaps we could make the necessary improvements to the tools, apply the improved tools to the pre-cleaned-up version of the tree, get a diff against HEAD, and then apply any fixes the improved tools have made as a patch. That would avoid cluttering CVS with two redundant changes to almost every single source file in the tree. I've calmed down a little and am no longer wanting to insist on reversion. My little Perl script is turning over and has found a number of issues besides the original TOAST_INDEX_HACK one; some of them may have been there before, so I'm thinking I'm going to add it to src/tools rather than just treat it as a one-shot. We still have the issue of how Bruce managed to miss undeclared-function warnings in some files. I'm concerned that that problem may remain for some platforms or option combinations. Is there a way to get the buildfarm to highlight compiler warnings? regards, tom lane ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] Patch to mark items as static or not used
Bruce Momjian [EMAIL PROTECTED] writes: Tom Lane wrote: This time around, please do not remove API functions just because you can't find a reference to them in the core code. I would like to see a posted, discussed patch first. OK, here is my match to mark items as static or not used: ftp://momjian.us/pub/postgresql/mypatches/static By and large, this just demonstrates the silliness of using an automated tool for this purpose :-(. The hits in gist and gin might be valid --- Teodor would need to comment on that --- but almost every one of the others is a no, don't do that. As an example, you've successfully reverted this recent patch in toto: 2006-04-26 20:46 tgl * src/: backend/utils/adt/selfuncs.c, include/utils/selfuncs.h: If we're going to expose VariableStatData for contrib modules to use, then we should export a reasonable set of the supporting routines too. The fundamental problem with find_static is that it hasn't got a clue about likely future changes, nor about what we think external add-ons might want ... regards, tom lane ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [HACKERS] Patch to mark items as static or not used
Tom Lane wrote: Bruce Momjian [EMAIL PROTECTED] writes: Tom Lane wrote: This time around, please do not remove API functions just because you can't find a reference to them in the core code. I would like to see a posted, discussed patch first. OK, here is my match to mark items as static or not used: ftp://momjian.us/pub/postgresql/mypatches/static By and large, this just demonstrates the silliness of using an automated tool for this purpose :-(. The hits in gist and gin might be valid --- Teodor would need to comment on that --- but almost every one of the others is a no, don't do that. As an example, you've successfully reverted this recent patch in toto: 2006-04-26 20:46 tgl * src/: backend/utils/adt/selfuncs.c, include/utils/selfuncs.h: If we're going to expose VariableStatData for contrib modules to use, then we should export a reasonable set of the supporting routines too. The fundamental problem with find_static is that it hasn't got a clue about likely future changes, nor about what we think external add-ons might want ... OK, I don't really have a clue either. Is any of it valid? -- Bruce Momjian [EMAIL PROTECTED] EnterpriseDBhttp://www.enterprisedb.com + If your life is a hard drive, Christ can be your backup. + ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [HACKERS] Patch to mark items as static or not used
Bruce Momjian [EMAIL PROTECTED] writes: Tom Lane wrote: The fundamental problem with find_static is that it hasn't got a clue about likely future changes, nor about what we think external add-ons might want ... OK, I don't really have a clue either. Is any of it valid? I don't object to static-izing AlterOpClassOwner_oid or RenameRewriteRule, and I defer to Teodor about the gist and gin functions. The others range somewhere between no and hell no. regards, tom lane ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] Patch to mark items as static or not used
On Sat, 2006-07-15 at 00:05 -0400, Tom Lane wrote: The fundamental problem with find_static is that it hasn't got a clue about likely future changes, nor about what we think external add-ons might want We could annotate the source to indicate that some functions are deliberately intended to be externally visible, but not referenced within the source tree, and then teach find_static to grok that annotation. -Neil ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
