Re: [HACKERS] Spread checkpoint sync

2010-12-01 Thread Heikki Linnakangas
On 01.12.2010 06:25, Greg Smith wrote: Jeff Janes wrote: I ask because I don't have a mental model of how the pause can help. Given that this dirty data has been hanging around for many minutes already, what is a 3 second pause going to heal? The difference is that once an fsync call is made,

Re: [HACKERS] GiST insert algorithm rewrite

2010-12-01 Thread Heikki Linnakangas
On 01.12.2010 04:10, Robert Haas wrote: On Tue, Nov 30, 2010 at 10:26 AM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: Does the current code cope with the corruption? It's not corruption, but intended degradation. Yes, the current code copes with it, that's how GiST survives

[HACKERS] Proposal: First step towards Intelligent,integrated database

2010-12-01 Thread ghatpande
Hello, Here is the proposal: My 1st step towards Intelligent, Integrated database. I am explaining the proposal with the use of example. Example: We will have a master table say CustMast and a transaction table say salesOrder table. View of CustMast: CustCodeNumber(5), CustName

Re: [HACKERS] KNNGIST next step: adjusting indexAM API

2010-12-01 Thread Dimitri Fontaine
Tom Lane t...@sss.pgh.pa.us writes: Lastly, I'm pretty un-thrilled with the way that the KNNGIST patch implements the interface to the opclass-specific hook functions. Seems like it would be cleaner to leave the Consistent function alone and invent a new, separate hook function for processing

Re: [HACKERS] Proposal: First step towards Intelligent,integrated database

2010-12-01 Thread Dann Corbit
I am probably just being thick, but how is your idea different from create domain: http://www.postgresql.org/docs/current/static/sql-createdomain.html From: pgsql-hackers-ow...@postgresql.org [mailto:pgsql-hackers-ow...@postgresql.org] On Behalf Of ghatpa...@vsnl.net Sent: Wednesday, December

Re: [HACKERS] We really ought to do something about O_DIRECT and data=journalled on ext4

2010-12-01 Thread Dimitri Fontaine
Tom Lane t...@sss.pgh.pa.us writes: As things stand, though, I think the only thing that's really open for discussion is how wide to make the scope of the default-change: should we just do it across the board, or try to limit it to some subset of the platforms where open_datasync is currently

Re: [HACKERS] Proposal: First step towards Intelligent, integrateddatabase

2010-12-01 Thread ghatpande
Create domain is only useful for abstracting common constraints on fields into single location for maintenance. It may not be useful to link tables. - Original Message - From: Dann Corbit dcor...@connx.com Date: Wednesday, December 1, 2010 4:04 pm Subject: RE: [HACKERS] Proposal: First

Re: [HACKERS] DELETE with LIMIT (or my first hack)

2010-12-01 Thread Daniel Loureiro
its pretty clear to me that's 2 different needs here, both linked to DELETE/UPDATE behavior. A) an feature MySQL-like which will DELETE/UPDATE just K tuples B) an feature to protect the database in case the DBA forget the WHERE statement I think that the first feature its pretty reasonable for

Re: [HACKERS] Improved JDBC driver part 2

2010-12-01 Thread Valentine Gogichashvili
Hi, I cannot get the file: wget http://www.rsmogura.net/pgsql/pgjdbc_exp_20101130_C.tar.gz --2010-12-01 12:05:28-- http://www.rsmogura.net/pgsql/pgjdbc_exp_20101130_C.tar.gz Resolving www.rsmogura.net... 64.120.14.83 Connecting to www.rsmogura.net|64.120.14.83|:80... connected. HTTP request

Re: [JDBC] [HACKERS] Improved JDBC driver part 2

2010-12-01 Thread Radosław Smogura
I've just started small clean up - now it's there. On Wed, 1 Dec 2010 12:06:19 +0100, Valentine Gogichashvili val...@gmail.com wrote: Hi, I cannot get the file: wget http://www.rsmogura.net/pgsql/pgjdbc_exp_20101130_C.tar.gz --2010-12-01 12:05:28--

[HACKERS] Hi- How frequently Postgres Poll for trigger file

2010-12-01 Thread aaliya zarrin
Hi All, I am new to postgres. I want to know how frequently postgres search for trigger file to switch over. Can this switch over time be reduced? Plz let me know where postgres poll for trigger file. I could find it out in backend/access/trans/xlog.c ? am i right? -- Thanks Regards, Aaliya

Re: [HACKERS] pg_execute_from_file review

2010-12-01 Thread Itagaki Takahiro
On Tue, Nov 30, 2010 at 18:47, Dimitri Fontaine dimi...@2ndquadrant.fr wrote: Itagaki Takahiro itagaki.takah...@gmail.com writes: There are no discussion yet for 1, but I think we need some restrictions Well, as a first level of restrictions, the function is superuser only. I understand and

Re: [HACKERS] DELETE with LIMIT (or my first hack)

2010-12-01 Thread Valentine Gogichashvili
Hi, actually introducing LIMIT and OFFSET with ORDER BY to DELETE/UPDATE would make it much easier to push data from one (say queue) table to another. And to fetch chunks of queue entries updating their status in one statement. Now I have to do SELECT...ORDER BY...LIMIT and then do some magic

Re: [HACKERS] Improved JDBC driver part 2

2010-12-01 Thread Magnus Hagander
On Tue, Nov 30, 2010 at 19:49, Radosław Smogura m...@smogura.eu wrote: Hello, Maybe you are interested about this what I done with JDBC snip Driver is here http://www.rsmogura.net/pgsql/pgjdbc_exp_20101130_C.tar.gz is currently JDK 6 compatible (will be not), compressed patch takes about

Re: [HACKERS] We really ought to do something about O_DIRECT and data=journalled on ext4

2010-12-01 Thread Marti Raudsepp
On Wed, Dec 1, 2010 at 12:35, Dimitri Fontaine dimi...@2ndquadrant.fr wrote: PANIC:  could not open file pg_xlog/00010001 (log file 0, segment 1): Invalid argument +1 I got the same error when trying to get PostgreSQL working on tmpfs and gave up. Now I understand that you

Re: [HACKERS] Hi- How frequently Postgres Poll for trigger file

2010-12-01 Thread Heikki Linnakangas
On 01.12.2010 13:27, aaliya zarrin wrote: I want to know how frequently postgres search for trigger file to switch over. In 9.0, every 100ms while streaming replication is active and connected. 5 seconds otherwise. In current git master branch, it's always 5 s. Can this switch over time be

Re: [HACKERS] We need index-only scans

2010-12-01 Thread Kristian Nielsen
Greg Stark gsst...@mit.edu writes: Just so everyone is on the same page Even once we have index-only scans they won't be anywhere near as useful with Postgres as they are with Oracle and other databases. At least not unless we find a solution for a different problem -- our inability to

Re: [HACKERS] DELETE with LIMIT (or my first hack)

2010-12-01 Thread Marko Tiikkaja
On 2010-12-01 1:46 PM, Valentine Gogichashvili wrote: And speaking about pushing data from one table to another, what I really would like to be able to do would be also something like: INSERT INTO ... DELETE FROM ... WHERE... ORDER BY.. [LIMIT...] RETURNING...; this would be also quite

Re: [HACKERS] crash-safe visibility map, take three

2010-12-01 Thread Bruce Momjian
Heikki Linnakangas wrote: On 01.12.2010 03:35, Bruce Momjian wrote: Heikki Linnakangas wrote: Let's recap what happens when a VM bit is set: You set the PD_ALL_VISIBLE flag on the heap page (assuming it's not set already, it usually isn't), and then set the bit in the VM while keeping the

Re: [HACKERS] Proposal: First step towards Intelligent,integrated database

2010-12-01 Thread Pavel Stehule
Hello there was a very similar design in ANSI SQL 99. I have documentation only in Czech, but probably you can find a sources about OOP part in ANSI/SQL. CREATE TABLE children( id int primary key, parent ref(parents) name .. ... and you can write queries like SELECT name, parent-name

Re: [HACKERS] KNNGIST next step: adjusting indexAM API

2010-12-01 Thread Robert Haas
On Wed, Dec 1, 2010 at 5:22 AM, Dimitri Fontaine dimi...@2ndquadrant.fr wrote: Tom Lane t...@sss.pgh.pa.us writes: Lastly, I'm pretty un-thrilled with the way that the KNNGIST patch implements the interface to the opclass-specific hook functions. Seems like it would be cleaner to leave the

Re: [HACKERS] We really ought to do something about O_DIRECT and data=journalled on ext4

2010-12-01 Thread Robert Haas
On Wed, Dec 1, 2010 at 12:31 AM, Tom Lane t...@sss.pgh.pa.us wrote: Josh Berkus j...@agliodbs.com writes: On 11/30/10 7:09 PM, Tom Lane wrote: Josh Berkus j...@agliodbs.com writes: Apparently, testing for O_DIRECT at compile time isn't adequate.  Ideas? We should wait for the outcome of the

Re: [HACKERS] crash-safe visibility map, take three

2010-12-01 Thread Heikki Linnakangas
On 01.12.2010 15:39, Bruce Momjian wrote: Heikki Linnakangas wrote: On 01.12.2010 03:35, Bruce Momjian wrote: Heikki Linnakangas wrote: Let's recap what happens when a VM bit is set: You set the PD_ALL_VISIBLE flag on the heap page (assuming it's not set already, it usually isn't), and then

Re: [JDBC] [HACKERS] Improved JDBC driver part 2

2010-12-01 Thread Radosław Smogura
On Wed, 1 Dec 2010 12:47:13 +0100, Magnus Hagander mag...@hagander.net wrote: On Tue, Nov 30, 2010 at 19:49, Radosław Smogura m...@smogura.eu wrote: Hello, Maybe you are interested about this what I done with JDBC snip Driver is here

Re: [HACKERS] We really ought to do something about O_DIRECT and data=journalled on ext4

2010-12-01 Thread Andrew Dunstan
On 11/30/2010 11:17 PM, Tom Lane wrote: Andrew Dunstanand...@dunslane.net writes: On 11/30/2010 10:09 PM, Tom Lane wrote: We should wait for the outcome of the discussion about whether to change the default wal_sync_method before worrying about this. we've just had a significant PGX

[HACKERS] FK's to refer to rows in inheritance child

2010-12-01 Thread Yeb Havinga
Hello list, FK's cannot refer to rows in inheritance childs. With some changes in LockRows, together with removing the ONLY keyword in ri_trigger.c, it was possible to refer to the rows in child relations. (WIP patch attached) Though it passes simple tests, it is far from complete. To our

Re: [HACKERS] directory archive format for pg_dump

2010-12-01 Thread Heikki Linnakangas
On 29.11.2010 22:21, Heikki Linnakangas wrote: On 29.11.2010 07:11, Joachim Wieland wrote: On Mon, Nov 22, 2010 at 3:44 PM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: * wrap long lines * use extern in function prototypes in header files * inline some functions like

Re: [HACKERS] directory archive format for pg_dump

2010-12-01 Thread Heikki Linnakangas
On 01.12.2010 16:03, Heikki Linnakangas wrote: On 29.11.2010 22:21, Heikki Linnakangas wrote: I combined those, and the Free/Flush steps, and did a bunch of other editorializations and cleanups. Here's an updated patch, also available in my git repository at

Re: [HACKERS] profiling connection overhead

2010-12-01 Thread Robert Haas
On Tue, Nov 30, 2010 at 11:32 PM, Jeff Janes jeff.ja...@gmail.com wrote: On 11/28/10, Robert Haas robertmh...@gmail.com wrote: In a close race, I don't think we should get bogged down in micro-optimization here, both because micro-optimizations may not gain much and because what works well on

Re: [HACKERS] profiling connection overhead

2010-12-01 Thread Andres Freund
On Wednesday 01 December 2010 15:20:32 Robert Haas wrote: On Tue, Nov 30, 2010 at 11:32 PM, Jeff Janes jeff.ja...@gmail.com wrote: On 11/28/10, Robert Haas robertmh...@gmail.com wrote: To some degree we're a victim of our own flexible and extensible architecture here, but I find it pretty

Re: [HACKERS] FK's to refer to rows in inheritance child

2010-12-01 Thread Tom Lane
Yeb Havinga yebhavi...@gmail.com writes: FK's cannot refer to rows in inheritance childs. With some changes in LockRows, together with removing the ONLY keyword in ri_trigger.c, it was possible to refer to the rows in child relations. (WIP patch attached) Though it passes simple tests, it

Re: [HACKERS] GiST insert algorithm rewrite

2010-12-01 Thread Robert Haas
On Wed, Dec 1, 2010 at 4:00 AM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: On 01.12.2010 04:10, Robert Haas wrote: On Tue, Nov 30, 2010 at 10:26 AM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com  wrote: Does the current code cope with the corruption? It's not

Re: [HACKERS] DELETE with LIMIT (or my first hack)

2010-12-01 Thread Rob Wultsch
On Wed, Dec 1, 2010 at 4:01 AM, Daniel Loureiro loureir...@gmail.com wrote: A) an feature MySQL-like which will DELETE/UPDATE just K tuples B) an feature to protect the database in case the DBA forget the WHERE statement MySQL has B as well. To quote the manual: For beginners, a useful

Re: [HACKERS] Proposal: First step towards Intelligent, integrateddatabase

2010-12-01 Thread Tom Lane
ghatpa...@vsnl.net writes: Create domain is only useful for abstracting common constraints on fields into single location for maintenance. It may not be useful to link tables. It's still unclear what this does that you don't get from inheritance, typed tables, use of a table's rowtype as a

Re: [HACKERS] KNNGIST next step: adjusting indexAM API

2010-12-01 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Wed, Dec 1, 2010 at 5:22 AM, Dimitri Fontaine dimi...@2ndquadrant.fr wrote: IIRC, the goal here was to be able to benefit from KNN GiST from existing GiST indexes as soon as you restart the server with the new code compiled in. I'm not sure it's

Re: [HACKERS] crash-safe visibility map, take three

2010-12-01 Thread Kevin Grittner
Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: it would be annoying to have to checkpoint after a data load Heck, in my world it's currently pretty much a necessity to run VACUUM FREEZE ANALYZE on a table after a data load before it's reasonable to expose the table to

Re: [HACKERS] DELETE with LIMIT (or my first hack)

2010-12-01 Thread Mario Weilguni
Am 01.12.2010 15:37, schrieb Rob Wultsch: For beginners, a useful startup option is --safe-updates (or --i-am-a-dummy, which has the same effect). This option was introduced in MySQL 3.23.11. It is helpful for cases when you might have issued a DELETE FROM tbl_name statement but forgotten the

Re: [HACKERS] Proposal: First step towards Intelligent, integrateddatabase

2010-12-01 Thread Andrew Dunstan
On 12/01/2010 09:41 AM, Tom Lane wrote: ghatpa...@vsnl.net writes: Create domain is only useful for abstracting common constraints on fields into single location for maintenance. It may not be useful to link tables. It's still unclear what this does that you don't get from inheritance,

Re: [HACKERS] FK's to refer to rows in inheritance child

2010-12-01 Thread Yeb Havinga
On 2010-12-01 15:27, Tom Lane wrote: Yeb Havingayebhavi...@gmail.com writes: FK's cannot refer to rows in inheritance childs. With some changes in LockRows, together with removing the ONLY keyword in ri_trigger.c, it was possible to refer to the rows in child relations. (WIP patch attached)

Re: [HACKERS] FK's to refer to rows in inheritance child

2010-12-01 Thread Tom Lane
Yeb Havinga yebhavi...@gmail.com writes: On 2010-12-01 15:27, Tom Lane wrote: Indeed. This isn't even worth the time to review, unless you have a proposal for fixing the unique-index-across-multiple-tables problem. That was in the part that you chose to not quote. Perhaps I should have said

Re: [HACKERS] Proposal: First step towards Intelligent,integrated database

2010-12-01 Thread David Fetter
On Wed, Dec 01, 2010 at 03:19:32PM +0500, ghatpa...@vsnl.net wrote: Hello, Here is the proposal: My 1st step towards Intelligent, Integrated database. You're implying that databases are stupid and incoherent. This is *not* a great way to start. Cheers, David. -- David Fetter

Re: [HACKERS] DELETE with LIMIT (or my first hack)

2010-12-01 Thread Kevin Grittner
Mario Weilguni roadrunn...@gmx.at wrote: Is it really up to the database to decide what queries are ok? It's the task of the developers to test their applikations. We're talking about ad hoc queries here, entered directly through psql or similar. -Kevin -- Sent via pgsql-hackers mailing

Re: [HACKERS] Hi- How frequently Postgres Poll for trigger file

2010-12-01 Thread Euler Taveira de Oliveira
Heikki Linnakangas escreveu: On 01.12.2010 13:27, aaliya zarrin wrote: I want to know how frequently postgres search for trigger file to switch over. In 9.0, every 100ms while streaming replication is active and connected. 5 seconds otherwise. In current git master branch, it's always 5 s.

Re: [HACKERS] Proposal: First step towards Intelligent, integrateddatabase

2010-12-01 Thread ghatpande
Be positive ... Negative thoughts are not good... - Original Message - From: David Fetter da...@fetter.org Date: Wednesday, December 1, 2010 8:42 pm Subject: Re: [HACKERS] Proposal: First step towards Intelligent,integrateddatabase To: ghatpa...@vsnl.net Cc: pgsql hackers

Re: [HACKERS] pg_execute_from_file review

2010-12-01 Thread Dimitri Fontaine
Itagaki Takahiro itagaki.takah...@gmail.com writes: My suggestion is to introduce pg_read_binary_file() function that can read any files in the server, and make CREATE EXTENSION to use the function. Of course, pg_execute_[sql|from]_file() can simplify queries It seems like all you're missing

Re: [HACKERS] KNNGIST next step: adjusting indexAM API

2010-12-01 Thread Dimitri Fontaine
Tom Lane t...@sss.pgh.pa.us writes: Right, AFAIK there is nothing in KNNGIST that would involve an on-disk data change. Nice, that matches my Royal Oak memories. But any external module relying on GiST will have to provide for the new function you're thinking about, right? Updating was already

Re: [HACKERS] profiling connection overhead

2010-12-01 Thread Kevin Grittner
Robert Haas robertmh...@gmail.com wrote: Jeff Janes jeff.ja...@gmail.com wrote: Oracle's backend start up time seems to be way higher than PG's. Interesting. How about MySQL and SQL Server? My recollection of Sybase ASE is that establishing a connection doesn't start a backend or even a

Re: [HACKERS] KNNGIST next step: adjusting indexAM API

2010-12-01 Thread Tom Lane
Dimitri Fontaine dimi...@2ndquadrant.fr writes: Tom Lane t...@sss.pgh.pa.us writes: Right, AFAIK there is nothing in KNNGIST that would involve an on-disk data change. But any external module relying on GiST will have to provide for the new function you're thinking about, right? Updating was

Re: [HACKERS] crash-safe visibility map, take three

2010-12-01 Thread Bruce Momjian
Heikki Linnakangas wrote: On 01.12.2010 15:39, Bruce Momjian wrote: Heikki Linnakangas wrote: On 01.12.2010 03:35, Bruce Momjian wrote: Heikki Linnakangas wrote: Let's recap what happens when a VM bit is set: You set the PD_ALL_VISIBLE flag on the heap page (assuming it's not set

Re: [HACKERS] crash-safe visibility map, take three

2010-12-01 Thread Robert Haas
On Wed, Dec 1, 2010 at 9:57 AM, Kevin Grittner kevin.gritt...@wicourts.gov wrote: Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: it would be annoying to have to checkpoint after a data load Heck, in my world it's currently pretty much a necessity to run VACUUM FREEZE ANALYZE

Re: [HACKERS] FK's to refer to rows in inheritance child

2010-12-01 Thread Florian Pflug
On Dec1, 2010, at 15:27 , Tom Lane wrote: Yeb Havinga yebhavi...@gmail.com writes: FK's cannot refer to rows in inheritance childs. With some changes in LockRows, together with removing the ONLY keyword in ri_trigger.c, it was possible to refer to the rows in child relations. (WIP patch

Re: [HACKERS] Proposal: First step towards Intelligent, integrateddatabase

2010-12-01 Thread David Fetter
My point exactly. You started off with high negativity, and you should not expect good results from same. Cheers, David. On Wed, Dec 01, 2010 at 08:15:25PM +0500, ghatpa...@vsnl.net wrote: Be positive ... Negative thoughts are not good... - Original Message - From: David Fetter

Re: [HACKERS] Improved JDBC driver part 2

2010-12-01 Thread David Fetter
On Wed, Dec 01, 2010 at 12:47:13PM +0100, Magnus Hagander wrote: On Tue, Nov 30, 2010 at 19:49, Radosław Smogura m...@smogura.eu wrote: Hello, Maybe you are interested about this what I done with JDBC snip Driver is here http://www.rsmogura.net/pgsql/pgjdbc_exp_20101130_C.tar.gz is

Re: [HACKERS] improving foreign key locks

2010-12-01 Thread Florian Pflug
On Nov29, 2010, at 22:33 , Tom Lane wrote: Alvaro Herrera alvhe...@commandprompt.com writes: Excerpts from Alvaro Herrera's message of lun nov 29 18:00:55 -0300 2010: Additionally, we'd have to expend some more cycles at the parse analysis phase (of the FOR SHARE OF x.col1, x.col2 query) to

Re: [HACKERS] FK's to refer to rows in inheritance child

2010-12-01 Thread Kevin Grittner
Florian Pflug f...@phlo.org wrote: BTW, my serializable_lock_consisteny patch would allow you to do this purely within pl/pgsql in a race-condition free way. So if that patch should get applied you might want to consider this as a workaround. Whether it will get applied is yet to be seen,

Re: [HACKERS] Improved JDBC driver part 2

2010-12-01 Thread Kevin Grittner
David Fetter da...@fetter.org wrote: Would the people now developing the JDBC driver object to switching to git? If we move to git, don't forget that there is not one repository which has the entire history for PostgreSQL JDBC -- the current repository is missing some work, including

Re: [HACKERS] FK's to refer to rows in inheritance child

2010-12-01 Thread Simon Riggs
On Wed, 2010-12-01 at 15:07 +0100, Yeb Havinga wrote: FK's cannot refer to rows in inheritance childs. With some changes in LockRows, together with removing the ONLY keyword in ri_trigger.c, it was possible to refer to the rows in child relations. (WIP patch attached) This has a userspace

Re: [HACKERS] improving foreign key locks

2010-12-01 Thread Tom Lane
Florian Pflug f...@phlo.org writes: The validity wouldn't change, only the kind of lock taken. If all columns to be locked are part of some unique index, we'd record that fact in the locked tuple's infomask, and thus know that only a certain subset of columns are to be prevented from being

Re: [HACKERS] crash-safe visibility map, take three

2010-12-01 Thread Robert Haas
On Wed, Dec 1, 2010 at 10:36 AM, Bruce Momjian br...@momjian.us wrote: Oh, we don't update the LSN when we set the PD_ALL_VISIBLE flag?  OK, please let me think some more.  Thanks. As far as I can tell, there are basically two viable solutions on the table here. 1. Every time we observe a page

Re: [HACKERS] FK's to refer to rows in inheritance child

2010-12-01 Thread Florian Pflug
On Dec1, 2010, at 17:11 , Kevin Grittner wrote: Florian Pflug f...@phlo.org wrote: BTW, my serializable_lock_consisteny patch would allow you to do this purely within pl/pgsql in a race-condition free way. So if that patch should get applied you might want to consider this as a workaround.

Re: [HACKERS] crash-safe visibility map, take three

2010-12-01 Thread Heikki Linnakangas
On 01.12.2010 18:25, Robert Haas wrote: I think we can improve this a bit further by also introducing a HEAP_XMIN_FROZEN bit that we set in lieu of overwriting XMIN with FrozenXID. This allows us to freeze tuples aggressively - if we want - without losing any forensic information. We can then

Re: [HACKERS] crash-safe visibility map, take three

2010-12-01 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: As far as I can tell, there are basically two viable solutions on the table here. 1. Every time we observe a page as all-visible, (a) set the PD_ALL_VISIBLE bit on the page, without bumping the LSN; (b) set the bit in the visibility map page, bumping

Re: [HACKERS] crash-safe visibility map, take three

2010-12-01 Thread Tom Lane
Heikki Linnakangas heikki.linnakan...@enterprisedb.com writes: Hmm, actually, if we're willing to believe PD_ALL_VISIBLE in the page header over the xmin/xmax on the tuples, we could simply not bother doing anti-wraparound vacuums for pages that have the flag set. I'm not sure what changes

Re: [HACKERS] crash-safe visibility map, take three

2010-12-01 Thread Heikki Linnakangas
On 01.12.2010 18:40, Tom Lane wrote: Robert Haasrobertmh...@gmail.com writes: As far as I can tell, there are basically two viable solutions on the table here. 1. Every time we observe a page as all-visible, (a) set the PD_ALL_VISIBLE bit on the page, without bumping the LSN; (b) set the

Re: [HACKERS] crash-safe visibility map, take three

2010-12-01 Thread Tom Lane
Heikki Linnakangas heikki.linnakan...@enterprisedb.com writes: On 01.12.2010 18:40, Tom Lane wrote: Um, no it isn't. Suppose the heap page gets to disk but we crash before the WAL record does. Now we have a persistent state where the heap page is marked PD_ALL_VISIBLE but the corresponding

Re: [HACKERS] improving foreign key locks

2010-12-01 Thread Florian Pflug
On Dec1, 2010, at 17:17 , Tom Lane wrote: Florian Pflug f...@phlo.org writes: The validity wouldn't change, only the kind of lock taken. If all columns to be locked are part of some unique index, we'd record that fact in the locked tuple's infomask, and thus know that only a certain subset

Re: [HACKERS] crash-safe visibility map, take three

2010-12-01 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: I think we can improve this a bit further by also introducing a HEAP_XMIN_FROZEN bit that we set in lieu of overwriting XMIN with FrozenXID. This allows us to freeze tuples aggressively - if we want - without losing any forensic information. So far

Re: [HACKERS] Hi- How frequently Postgres Poll for trigger file

2010-12-01 Thread aaliya zarrin
Thanks for quick response.. Can I change this 5 second time? I have seen the postgres code as well. What is the functionality of WaitLatch() function. I could not understand completely. Plz help.. On Wed, Dec 1, 2010 at 5:53 PM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: On

Re: [HACKERS] crash-safe visibility map, take three

2010-12-01 Thread Robert Haas
On Wed, Dec 1, 2010 at 11:40 AM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: On 01.12.2010 18:25, Robert Haas wrote: I think we can improve this a bit further by also introducing a HEAP_XMIN_FROZEN bit that we set in lieu of overwriting XMIN with FrozenXID.  This allows us

Re: [HACKERS] crash-safe visibility map, take three

2010-12-01 Thread Robert Haas
On Wed, Dec 1, 2010 at 12:22 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: I think we can improve this a bit further by also introducing a HEAP_XMIN_FROZEN bit that we set in lieu of overwriting XMIN with FrozenXID.  This allows us to freeze tuples

Re: [HACKERS] improving foreign key locks

2010-12-01 Thread Tom Lane
Florian Pflug f...@phlo.org writes: On Dec1, 2010, at 17:17 , Tom Lane wrote: There's not enough space in the infomask to record which columns (or which unique index) are involved. And if you're talking about data that could remain on disk long after the unique index is gone, that's not

[HACKERS] Hypothetical Indexes - PostgreSQL extension - PGCON 2010

2010-12-01 Thread Ana Carolina Brito de Almeida
Hackers, We would like to inform you all that our extension to PostgreSQL, that includes hypothetical indexes (and soon index self-tuning), is available through a sourgeforge project. This was suggested at PgCon 2010 and we hope some of you may find it useful, contribute and give us your

Re: [HACKERS] We really ought to do something about O_DIRECT and data=journalled on ext4

2010-12-01 Thread Josh Berkus
Tom, Well, no, actually it's the same (only) argument. We'd never consider back-patching such a change if our hand weren't being forced by kernel changes :-( I think we have to back-patch the change. The way it is now, a DBA who thinks they are doing normal sensible configuration can cause

Re: [HACKERS] Hypothetical Indexes - PostgreSQL extension - PGCON 2010

2010-12-01 Thread Josh Berkus
Ana, We would like to inform you all that our extension to PostgreSQL, that includes hypothetical indexes (and soon index self-tuning), is available through a sourgeforge project. This was suggested at PgCon 2010 and we hope some of you may find it useful, contribute and give us your

Re: [HACKERS] Improved JDBC driver part 2

2010-12-01 Thread David Fetter
On Wed, Dec 01, 2010 at 10:15:38AM -0600, Kevin Grittner wrote: David Fetter da...@fetter.org wrote: Would the people now developing the JDBC driver object to switching to git? If we move to git, don't forget that there is not one repository which has the entire history for PostgreSQL

Re: [HACKERS] DELETE with LIMIT (or my first hack)

2010-12-01 Thread Josh Berkus
We need a convincing use case for it. So far the only one that's seemed at all convincing to me is the one about deleting in batches. But that might be enough. Queueing. If logless tables are in 9.1, then using PostgreSQL as the backend for a queue becomes a sensible thing to do. And what

Re: [HACKERS] DELETE with LIMIT (or my first hack)

2010-12-01 Thread Dmitriy Igrishin
Hey, I don't clearly understand why anybody should perform DELETE directly from a psql terminal on a production system. WHY ? I can't understand what problem with DELETE without WHERE clause for application developers and why DBMS should protect them from DELETE FROM table. PS. Anybody can

Re: [HACKERS] We really ought to do something about O_DIRECT and data=journalled on ext4

2010-12-01 Thread Tom Lane
Josh Berkus j...@agliodbs.com writes: It's a bug and it's our bug. No, it's a filesystem bug that this particular filesystem doesn't support a perfectly reasonable combination of options, and doesn't even fail gracefully as it could easily do. But assigning blame doesn't help much. Back when

Re: [HACKERS] We really ought to do something about O_DIRECT and data=journalled on ext4

2010-12-01 Thread Josh Berkus
I think the best answer is to get out of the business of using O_DIRECT by default, especially seeing that available evidence suggests it might not be a performance win anyway. Well, we don't have any performance evidence ... there's an issue with the fsync-test script which causes it not to

Re: [HACKERS] We really ought to do something about O_DIRECT and data=journalled on ext4

2010-12-01 Thread Andres Freund
On Wednesday 01 December 2010 19:09:05 Tom Lane wrote: Josh Berkus j...@agliodbs.com writes: It's a bug and it's our bug. No, it's a filesystem bug that this particular filesystem doesn't support a perfectly reasonable combination of options, and doesn't even fail gracefully as it could

Re: [HACKERS] Hot Standby: too many KnownAssignedXids

2010-12-01 Thread Heikki Linnakangas
On 24.11.2010 12:48, Heikki Linnakangas wrote: When recovery starts, we fetch the oldestActiveXid from the checkpoint record. Let's say that it's 100. We then start replaying WAL records from the Redo pointer, and the first record (heap insert in your case) contains an Xid that's much larger

Re: [HACKERS] We really ought to do something about O_DIRECT and data=journalled on ext4

2010-12-01 Thread Tom Lane
Josh Berkus j...@agliodbs.com writes: It might be nice to add new sync_method options, osync_odirect and odatasync_odirect for DBAs who think they know enough to tune with non-defaults. That would have the benefit that we'd not have to argue with people who liked the current behavior (assuming

Re: [HACKERS] Improved JDBC driver part 2

2010-12-01 Thread Tom Lane
David Fetter da...@fetter.org writes: On Wed, Dec 01, 2010 at 10:15:38AM -0600, Kevin Grittner wrote: If we move to git, don't forget that there is not one repository which has the entire history for PostgreSQL JDBC -- the current repository is missing some work, including releases, from one

Re: [HACKERS] We really ought to do something about O_DIRECT and data=journalled on ext4

2010-12-01 Thread Josh Berkus
However, this doesn't really address the question of what a sensible choice of default is. If there's little evidence about whether the current flavor of open_datasync is really the fastest way, there's none whatsoever that establishes open_datasync_without_o_direct being a sane choice of

Re: [HACKERS] Hi- How frequently Postgres Poll for trigger file

2010-12-01 Thread Heikki Linnakangas
On 01.12.2010 19:23, aaliya zarrin wrote: Thanks for quick response.. Can I change this 5 second time? I have seen the postgres code as well. You can, if you don't mind changing the sources. What is the functionality of WaitLatch() function. I could not understand completely. The

Re: [HACKERS] We really ought to do something about O_DIRECT and data=journalled on ext4

2010-12-01 Thread Andrew Dunstan
On 12/01/2010 01:41 PM, Andres Freund wrote: On Wednesday 01 December 2010 19:09:05 Tom Lane wrote: Josh Berkusj...@agliodbs.com writes: It's a bug and it's our bug. No, it's a filesystem bug that this particular filesystem doesn't support a perfectly reasonable combination of options, and

Re: [HACKERS] DELETE with LIMIT (or my first hack)

2010-12-01 Thread Peter Eisentraut
On tis, 2010-11-30 at 14:20 -0500, Andrew Dunstan wrote: I agree, that argument is completely misconceived. If the DBA is paying enough attention to use LIMIT, s/he should be paying enough attention not to do damage in the first place. If that were the only argument in its favor I'd be

Re: [HACKERS] Another proposal for table synonyms

2010-12-01 Thread Alexey Klyukin
On Nov 30, 2010, at 10:05 PM, Josh Berkus wrote: Alexey, Here is the proposal to add synonyms to PostgreSQL. Initial goal is to add synonyms for relations (tables, views, sequences) and an infrastructure to allow synonyms for other database objects in the future. Can you explain,

Re: [HACKERS] Another proposal for table synonyms

2010-12-01 Thread Josh Berkus
Well, porting applications from other database systems that support synonyms (i.e. Oracle, DB2, SQL Server). SQL Server supports synonyms? If it's not Oracle-only, it's a more powerful argument to have the feature. (IMHO, the main reason why Oracle has synonyms is that their implementation

Re: [HACKERS] Idle git question: how come so many objects?

2010-12-01 Thread Robert Haas
On Wed, Dec 1, 2010 at 2:08 AM, Martijn van Oosterhout klep...@svana.org wrote: On Wed, Dec 01, 2010 at 01:03:26AM -0500, Tom Lane wrote: So I just made a commit that touched four files in all six active branches, and I see: $ git push Counting objects: 172, done. Compressing objects: 100%

Re: [HACKERS] crash-safe visibility map, take three

2010-12-01 Thread Jeff Davis
On Wed, 2010-12-01 at 11:25 -0500, Robert Haas wrote: 1. Every time we observe a page as all-visible, (a) set the PD_ALL_VISIBLE bit on the page, without bumping the LSN; ... 2. Every time we observe a page as all-visible, (a) set the PD_ALL_VISIBLE bit on the page, without bumping the LSN,

Re: [HACKERS] Another proposal for table synonyms

2010-12-01 Thread Joshua D. Drake
On Wed, 2010-12-01 at 12:13 -0800, Josh Berkus wrote: Well, porting applications from other database systems that support synonyms (i.e. Oracle, DB2, SQL Server). SQL Server supports synonyms? If it's not Oracle-only, it's a more powerful argument to have the feature. Oracle, DB2 and

Re: [HACKERS] Another proposal for table synonyms

2010-12-01 Thread Josh Berkus
I'd love to hear from someone at EDB: how are you dealing with synonym name collisions right now? I think the way we deal with that is the way PostgreSQL deals with it. Unique names per search path. Have you had an employment change I didn't know about, JD? ;-) --

Re: [HACKERS] Another proposal for table synonyms

2010-12-01 Thread Dave Page
On Wed, Dec 1, 2010 at 8:46 PM, Josh Berkus j...@agliodbs.com wrote: I'd love to hear from someone at EDB: how are you dealing with synonym name collisions right now? I think the way we deal with that is the way PostgreSQL deals with it. Unique names per search path. Have you had an

Re: [HACKERS] Another proposal for table synonyms

2010-12-01 Thread Joshua D. Drake
On Wed, 2010-12-01 at 12:46 -0800, Josh Berkus wrote: I'd love to hear from someone at EDB: how are you dealing with synonym name collisions right now? I think the way we deal with that is the way PostgreSQL deals with it. Unique names per search path. Have you had an employment

Re: [HACKERS] Another proposal for table synonyms

2010-12-01 Thread Alvaro Herrera
Excerpts from Josh Berkus's message of mié dic 01 17:13:35 -0300 2010: Well, porting applications from other database systems that support synonyms (i.e. Oracle, DB2, SQL Server). SQL Server supports synonyms? If it's not Oracle-only, it's a more powerful argument to have the feature.

Re: [HACKERS] crash-safe visibility map, take three

2010-12-01 Thread Robert Haas
On Wed, Dec 1, 2010 at 3:31 PM, Jeff Davis pg...@j-davis.com wrote: On Wed, 2010-12-01 at 11:25 -0500, Robert Haas wrote: 1. Every time we observe a page as all-visible, (a) set the PD_ALL_VISIBLE bit on the page, without bumping the LSN; ... 2. Every time we observe a page as all-visible,

Re: [HACKERS] crash-safe visibility map, take three

2010-12-01 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: If we switched from per-tuple MVCC based on XIDs to per-page MVCC based on LSNs and a rollback segment, all of this stuff would go out the window. Hint bits, gone. Anti-wraparound VACUUM, gone. CRCs, feasible. Visibility map... we might still need

Re: [HACKERS] Spread checkpoint sync

2010-12-01 Thread Greg Smith
Heikki Linnakangas wrote: Do you have any idea how to autotune the delay between fsyncs? I'm thinking to start by counting the number of relations that need them at the beginning of the checkpoint. Then use the same basic math that drives the spread writes, where you assess whether you're

Re: [HACKERS] improving foreign key locks

2010-12-01 Thread Florian Pflug
On Dec1, 2010, at 18:44 , Tom Lane wrote: Florian Pflug f...@phlo.org writes: On Dec1, 2010, at 17:17 , Tom Lane wrote: There's not enough space in the infomask to record which columns (or which unique index) are involved. And if you're talking about data that could remain on disk long after

  1   2   >