Re: [HACKERS] Tab completion for view triggers in psql

2010-11-30 Thread Itagaki Takahiro
On Wed, Nov 24, 2010 at 12:21, David Fetter da...@fetter.org wrote: Please find attached a patch changing both this and updateable to updatable, also per the very handy git grep I just learned about :) I think the patch has two issues to be fixed. It expands all tables (and views) in

Re: [HACKERS] Tab completion for view triggers in psql

2010-11-30 Thread David Fetter
On Tue, Nov 30, 2010 at 05:48:04PM +0900, Itagaki Takahiro wrote: On Wed, Nov 24, 2010 at 12:21, David Fetter da...@fetter.org wrote: Please find attached a patch changing both this and updateable to updatable, also per the very handy git grep I just learned about :) I think the patch has

Re: [HACKERS] pg_execute_from_file review

2010-11-30 Thread Dimitri Fontaine
Itagaki Takahiro itagaki.takah...@gmail.com writes: I think there are two topics here: 1. Do we need to restrict locations in which sql files are executable? 2. Which is better, pg_execute_sql_file() or EXECUTE pg_read_file() ? There are no discussion yet for 1, but I think we need some

Re: [HACKERS] pg_execute_from_file review

2010-11-30 Thread Dimitri Fontaine
Itagaki Takahiro itagaki.takah...@gmail.com writes: client_encoding won't work at all because read_sql_queries_from_file() uses pg_verifymbstr(), that is verify the input with *server_encoding*. Even if we replace it with pg_verify_mbstr(client_encoding, ...) and

Re: [HACKERS] GiST insert algorithm rewrite

2010-11-30 Thread Heikki Linnakangas
On 27.11.2010 21:31, Bruce Momjian wrote: Heikki Linnakangas wrote: There's no on-disk format changes, except for the additional flag in the page headers, so this does not affect pg_upgrade. However, if there's any invalid keys in the old index because of an incomplete insertion, the new code

Re: [HACKERS] GiST insert algorithm rewrite

2010-11-30 Thread Heikki Linnakangas
On 30.11.2010 11:55, Heikki Linnakangas wrote: On 27.11.2010 21:31, Bruce Momjian wrote: Heikki Linnakangas wrote: There's no on-disk format changes, except for the additional flag in the page headers, so this does not affect pg_upgrade. However, if there's any invalid keys in the old index

Re: [HACKERS] Tab completion for view triggers in psql

2010-11-30 Thread Itagaki Takahiro
On Tue, Nov 30, 2010 at 18:18, David Fetter da...@fetter.org wrote: It expands all tables (and views) in tab-completion after INSERT, UPDATE, and DELETE FROM.  Is it an intended change? I found it was a simple bug; we need ( ) around selcondition. In addition, I modified your patch a bit: * I

Re: [HACKERS] DELETE with LIMIT (or my first hack)

2010-11-30 Thread Csaba Nagy
Hi all, The workaround recommended some time ago by Tom is: DELETE FROM residents_of_athens WHERE ctid = any(array(SELECT ctid FROM residents_of_athens ORDER BY ostracism_votes DESC LIMIT 1)); It is about as efficient as the requested feature would be, just uglier to write down. I use it all

Re: [HACKERS] crash-safe visibility map, take three

2010-11-30 Thread Rob Wultsch
On Mon, Nov 29, 2010 at 9:57 PM, Robert Haas robertmh...@gmail.com wrote: 1. Pin each visibility map page.  If any VM_BECOMING_ALL_VISIBLE bits are set, take the exclusive content lock for long enough to clear them. I wonder what the performance hit will be to workloads with contention and if

Re: [HACKERS] DELETE with LIMIT (or my first hack)

2010-11-30 Thread Rob Wultsch
On Mon, Nov 29, 2010 at 10:50 PM, Marti Raudsepp ma...@juffo.org wrote: On Tue, Nov 30, 2010 at 05:09, Jaime Casanova ja...@2ndquadrant.com wrote: at least IMHO the only sensible way that LIMIT is usefull is with an ORDER BY clause with make the results very well defined... DELETE with LIMIT

Re: [HACKERS] [GENERAL] column-level update privs + lock table

2010-11-30 Thread Josh Kupershmidt
On Mon, Nov 29, 2010 at 10:06 PM, Robert Haas robertmh...@gmail.com wrote: On Mon, Nov 29, 2010 at 9:37 PM, Josh Kupershmidt schmi...@gmail.com wrote: I actually hadn't thought of that, for some reason. We used to similarly recommend that people handle TRUNCATE privileges with a security

Re: [HACKERS] Tab completion for view triggers in psql

2010-11-30 Thread David Fetter
On Tue, Nov 30, 2010 at 08:13:37PM +0900, Itagaki Takahiro wrote: On Tue, Nov 30, 2010 at 18:18, David Fetter da...@fetter.org wrote: It expands all tables (and views) in tab-completion after INSERT, UPDATE, and DELETE FROM.  Is it an intended change? I found it was a simple bug; we need (

Re: [HACKERS] DELETE with LIMIT (or my first hack)

2010-11-30 Thread Robert Haas
On Tue, Nov 30, 2010 at 4:25 AM, Csaba Nagy ncsli...@googlemail.com wrote: The workaround recommended some time ago by Tom is: DELETE FROM residents_of_athens WHERE ctid = any(array(SELECT ctid FROM residents_of_athens ORDER BY ostracism_votes DESC LIMIT 1)); It is about as efficient as the

Re: [HACKERS] GiST insert algorithm rewrite

2010-11-30 Thread Robert Haas
On Tue, Nov 30, 2010 at 5:02 AM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: On 30.11.2010 11:55, Heikki Linnakangas wrote: On 27.11.2010 21:31, Bruce Momjian wrote: Heikki Linnakangas wrote: There's no on-disk format changes, except for the additional flag in the page

Re: [HACKERS] DELETE with LIMIT (or my first hack)

2010-11-30 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Mon, Nov 29, 2010 at 11:25 PM, Andrew Dunstan and...@dunslane.net wrote: I can't say I'd be excited by this feature. In quite a few years of writing SQL I don't recall ever wanting such a gadget. It's something I've wanted periodically, though not

Re: [HACKERS] DELETE with LIMIT (or my first hack)

2010-11-30 Thread Andrew Dunstan
On 11/30/2010 09:57 AM, Csaba Nagy wrote: So it is really an ideological thing and not lack of demand or implementation attempts... I for myself can't write working C code anyway, so I got my peace with the workaround - I wish you good luck arguing Tom :-) We need a convincing use case

Re: [HACKERS] GiST insert algorithm rewrite

2010-11-30 Thread Heikki Linnakangas
On 30.11.2010 16:23, Robert Haas wrote: On Tue, Nov 30, 2010 at 5:02 AM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: On 30.11.2010 11:55, Heikki Linnakangas wrote: On 27.11.2010 21:31, Bruce Momjian wrote: Heikki Linnakangas wrote: There's no on-disk format changes,

Re: [HACKERS] crash-safe visibility map, take three

2010-11-30 Thread Robert Haas
On Tue, Nov 30, 2010 at 2:34 AM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: Some care is needed with checkpoints. Setting visibility map bits in step 2 is safe because crash recovery will replay the intent XLOG record and clear any incorrectly set bits. But if a checkpoint

Re: [HACKERS] crash-safe visibility map, take three

2010-11-30 Thread Tom Lane
Heikki Linnakangas heikki.linnakan...@enterprisedb.com writes: On 30.11.2010 06:57, Robert Haas wrote: I can't say I'm totally in love with any of these designs. Anyone else have any ideas, or any opinions about which one is best? Well, the design I've been pondering goes like this:

Re: [HACKERS] DELETE with LIMIT (or my first hack)

2010-11-30 Thread Robert Haas
On Tue, Nov 30, 2010 at 10:04 AM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: On Mon, Nov 29, 2010 at 11:25 PM, Andrew Dunstan and...@dunslane.net wrote: I can't say I'd be excited by this feature. In quite a few years of writing SQL I don't recall ever wanting

Re: [HACKERS] crash-safe visibility map, take three

2010-11-30 Thread Heikki Linnakangas
On 30.11.2010 17:32, Robert Haas wrote: On Tue, Nov 30, 2010 at 2:34 AM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: Some care is needed with checkpoints. Setting visibility map bits in step 2 is safe because crash recovery will replay the intent XLOG record and clear any

Re: [HACKERS] crash-safe visibility map, take three

2010-11-30 Thread Robert Haas
On Tue, Nov 30, 2010 at 10:38 AM, Tom Lane t...@sss.pgh.pa.us wrote: Heikki Linnakangas heikki.linnakan...@enterprisedb.com writes: On 30.11.2010 06:57, Robert Haas wrote: I can't say I'm totally in love with any of these designs.  Anyone else have any ideas, or any opinions about which one is

Re: [HACKERS] crash-safe visibility map, take three

2010-11-30 Thread Heikki Linnakangas
On 30.11.2010 17:38, Tom Lane wrote: Heikki Linnakangasheikki.linnakan...@enterprisedb.com writes: On 30.11.2010 06:57, Robert Haas wrote: I can't say I'm totally in love with any of these designs. Anyone else have any ideas, or any opinions about which one is best? Well, the design I've

Re: [HACKERS] crash-safe visibility map, take three

2010-11-30 Thread Robert Haas
On Tue, Nov 30, 2010 at 10:43 AM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: It seems like you'll need to hold some kind of lock between the time you examine RedoRecPtr and the time you actually examine the bit. WALInsertLock in shared mode, maybe? It's enough to hold an

Re: [HACKERS] crash-safe visibility map, take three

2010-11-30 Thread Heikki Linnakangas
Here's one more idea: The trivial solution to this is to WAL-log setting the visibility map bit, like we WAL-log any other operation. Lock the heap page, lock the visibility map page, write WAL-record, and release locks. That works, but the problem is that it creates quite a lot of new WAL

Re: [HACKERS] DELETE with LIMIT (or my first hack)

2010-11-30 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Tue, Nov 30, 2010 at 10:04 AM, Tom Lane t...@sss.pgh.pa.us wrote: Unfortunately, the UPDATE case would be an order of magnitude harder (think inheritance trees where the children aren't all alike). I don't understand why there's anything more to

Re: [HACKERS] DELETE with LIMIT (or my first hack)

2010-11-30 Thread Csaba Nagy
Hi Robert, On Tue, 2010-11-30 at 09:19 -0500, Robert Haas wrote: That's a very elegant hack, but not exactly obvious to a novice user or, say, me. So I think it'd be nicer to have the obvious syntax work. I fully agree - but you first have to convince core hackers that this is not just a

[HACKERS] Another proposal for table synonyms

2010-11-30 Thread Alexey Klyukin
Hello, Here is the proposal to add synonyms to PostgreSQL. Initial goal is to add synonyms for relations (tables, views, sequences) and an infrastructure to allow synonyms for other database objects in the future. A thread with discussion of an old proposal by Jonah Harris is here:

Re: [HACKERS] crash-safe visibility map, take three

2010-11-30 Thread Tom Lane
Heikki Linnakangas heikki.linnakan...@enterprisedb.com writes: On 30.11.2010 17:38, Tom Lane wrote: Wouldn't it be easier and more robust to just consider VM bit changes to be part of the WAL-logged actions? That would include updating LSNs on VM pages and flushing VM pages to disk during

Re: [HACKERS] DELETE with LIMIT (or my first hack)

2010-11-30 Thread Robert Haas
On Tue, Nov 30, 2010 at 11:04 AM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: On Tue, Nov 30, 2010 at 10:04 AM, Tom Lane t...@sss.pgh.pa.us wrote: Unfortunately, the UPDATE case would be an order of magnitude harder (think inheritance trees where the children

Re: [HACKERS] crash-safe visibility map, take three

2010-11-30 Thread Tom Lane
Heikki Linnakangas heikki.linnakan...@enterprisedb.com writes: The trivial solution to this is to WAL-log setting the visibility map bit, like we WAL-log any other operation. Lock the heap page, lock the visibility map page, write WAL-record, and release locks. That works, but the problem

Re: [HACKERS] crash-safe visibility map, take three

2010-11-30 Thread Robert Haas
On Tue, Nov 30, 2010 at 11:16 AM, Tom Lane t...@sss.pgh.pa.us wrote: How much is quite a lot?  Do we have any real reason to think that this solution is unacceptable performance-wise? Well, let's imagine a 1GB insert-only table. It has 128K pages. If you XLOG setting the bit on each page,

Re: [HACKERS] [GENERAL] column-level update privs + lock table

2010-11-30 Thread Robert Haas
On Tue, Nov 30, 2010 at 9:07 AM, Josh Kupershmidt schmi...@gmail.com wrote: On Mon, Nov 29, 2010 at 10:06 PM, Robert Haas robertmh...@gmail.com wrote: On Mon, Nov 29, 2010 at 9:37 PM, Josh Kupershmidt schmi...@gmail.com wrote: I actually hadn't thought of that, for some reason. We used to

Re: [HACKERS] crash-safe visibility map, take three

2010-11-30 Thread Robert Haas
On Tue, Nov 30, 2010 at 11:22 AM, Robert Haas robertmh...@gmail.com wrote: On Tue, Nov 30, 2010 at 11:16 AM, Tom Lane t...@sss.pgh.pa.us wrote: How much is quite a lot?  Do we have any real reason to think that this solution is unacceptable performance-wise? Well, let's imagine a 1GB

Re: [HACKERS] crash-safe visibility map, take three

2010-11-30 Thread Heikki Linnakangas
On 30.11.2010 18:22, Robert Haas wrote: On Tue, Nov 30, 2010 at 11:16 AM, Tom Lanet...@sss.pgh.pa.us wrote: How much is quite a lot? Do we have any real reason to think that this solution is unacceptable performance-wise? Well, let's imagine a 1GB insert-only table. It has 128K pages. If

Re: [HACKERS] Another proposal for table synonyms

2010-11-30 Thread Tom Lane
Alexey Klyukin al...@commandprompt.com writes: To support addition of new database objects types that can be referenced by synonyms a new system catalog, pg_synonym, is to be added, with an oid to support comments on synonym, and the following schema: This is not going to work, at least not

Re: [HACKERS] crash-safe visibility map, take three

2010-11-30 Thread Heikki Linnakangas
On 30.11.2010 18:10, Tom Lane wrote: Heikki Linnakangasheikki.linnakan...@enterprisedb.com writes: Yeah, I'm not terribly excited about any of these schemes. The intent record seems like the simplest one, but even that is quite different from the traditional WAL-logging we do that it makes me

Re: [HACKERS] crash-safe visibility map, take three

2010-11-30 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: Oh, but it's worse than that. When you XLOG a WAL record for each of those pages, you're going to trigger full-page writes for all of them. So now you've turned 1GB of data to write into 2+ GB of data to write. No, because only the first mod of each

Re: [HACKERS] crash-safe visibility map, take three

2010-11-30 Thread Tom Lane
Heikki Linnakangas heikki.linnakan...@enterprisedb.com writes: On 30.11.2010 18:10, Tom Lane wrote: I'm not convinced it works at all. Consider write intent record, checkpoint, set bit, crash before completing vacuum. There will be no second intent record at which you could clean up if

Re: [HACKERS] Instrument checkpoint sync calls

2010-11-30 Thread Greg Smith
Jeff Janes wrote: For the individual file sync times emitted under debug1, it would be very handy if the file being synced was identified, for example relation base/16384/16523. Rather than being numbered sequentially within a given checkpoint. I was numbering them sequentially so that it's

Re: [HACKERS] crash-safe visibility map, take three

2010-11-30 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: That's definitely sucky, but in some ways it would be more complicated if they did, because I don't think all-visible on the master implies all-visible on the standby. Ouch. That seems like it could shoot down all these proposals. There definitely

Re: [HACKERS] crash-safe visibility map, take three

2010-11-30 Thread Robert Haas
On Tue, Nov 30, 2010 at 11:33 AM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: Oh, but it's worse than that.  When you XLOG a WAL record for each of those pages, you're going to trigger full-page writes for all of them.  So now you've turned 1GB of data to write

Re: [HACKERS] crash-safe visibility map, take three

2010-11-30 Thread Heikki Linnakangas
On 30.11.2010 18:40, Tom Lane wrote: Robert Haasrobertmh...@gmail.com writes: That's definitely sucky, but in some ways it would be more complicated if they did, because I don't think all-visible on the master implies all-visible on the standby. Ouch. That seems like it could shoot down all

Re: [HACKERS] crash-safe visibility map, take three

2010-11-30 Thread Heikki Linnakangas
On 30.11.2010 18:33, Tom Lane wrote: Robert Haasrobertmh...@gmail.com writes: Oh, but it's worse than that. When you XLOG a WAL record for each of those pages, you're going to trigger full-page writes for all of them. So now you've turned 1GB of data to write into 2+ GB of data to write.

Re: [HACKERS] crash-safe visibility map, take three

2010-11-30 Thread Robert Haas
On Tue, Nov 30, 2010 at 11:40 AM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: That's definitely sucky, but in some ways it would be more complicated if they did, because I don't think all-visible on the master implies all-visible on the standby. Ouch.  That

Re: [HACKERS] crash-safe visibility map, take three

2010-11-30 Thread Tom Lane
Heikki Linnakangas heikki.linnakan...@enterprisedb.com writes: Can we get away with not setting the LSN on the heap page, even though we set the PD_ALL_VISIBLE flag? If we don't set the LSN, the heap page can be flushed to disk before the WAL record, but I think that's fine because it's OK

Re: [HACKERS] Another proposal for table synonyms

2010-11-30 Thread Alexey Klyukin
On Nov 30, 2010, at 6:28 PM, Tom Lane wrote: Alexey Klyukin al...@commandprompt.com writes: To support addition of new database objects types that can be referenced by synonyms a new system catalog, pg_synonym, is to be added, with an oid to support comments on synonym, and the following

Re: [HACKERS] crash-safe visibility map, take three

2010-11-30 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Tue, Nov 30, 2010 at 11:40 AM, Tom Lane t...@sss.pgh.pa.us wrote: Ouch.  That seems like it could shoot down all these proposals.  There definitely isn't any way to make VM crash-safe if there is no WAL-driven mechanism for setting the bits.

Re: [HACKERS] crash-safe visibility map, take three

2010-11-30 Thread Robert Haas
On Tue, Nov 30, 2010 at 11:49 AM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: On 30.11.2010 18:33, Tom Lane wrote: Robert Haasrobertmh...@gmail.com  writes: Oh, but it's worse than that.  When you XLOG a WAL record for each of those pages, you're going to trigger full-page

Re: [HACKERS] crash-safe visibility map, take three

2010-11-30 Thread Robert Haas
On Tue, Nov 30, 2010 at 11:55 AM, Tom Lane t...@sss.pgh.pa.us wrote: Heikki Linnakangas heikki.linnakan...@enterprisedb.com writes: Can we get away with not setting the LSN on the heap page, even though we set the PD_ALL_VISIBLE flag? If we don't set the LSN, the heap page can be flushed to

Re: [HACKERS] crash-safe visibility map, take three

2010-11-30 Thread Robert Haas
On Tue, Nov 30, 2010 at 11:59 AM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: On Tue, Nov 30, 2010 at 11:40 AM, Tom Lane t...@sss.pgh.pa.us wrote: Ouch.  That seems like it could shoot down all these proposals.  There definitely isn't any way to make VM

Re: [HACKERS] Another proposal for table synonyms

2010-11-30 Thread Tom Lane
Alexey Klyukin al...@commandprompt.com writes: On Nov 30, 2010, at 6:28 PM, Tom Lane wrote: This is not going to work, at least not without making every type of lookup consult pg_synonym too, which I think can be considered DOA because of its performance impact on people who aren't even using

Re: [HACKERS] crash-safe visibility map, take three

2010-11-30 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: We're not going to double the cost of VACUUM to get index-only scans. And that's exactly what will happen if you do full-page writes of every heap page to set a single bit. It's ridiculous to claim that that doubles the cost of VACUUM. In the worst

Re: [HACKERS] crash-safe visibility map, take three

2010-11-30 Thread Robert Haas
On Tue, Nov 30, 2010 at 12:10 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: We're not going to double the cost of VACUUM to get index-only scans. And that's exactly what will happen if you do full-page writes of every heap page to set a single bit. It's

Re: [HACKERS] crash-safe visibility map, take three

2010-11-30 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Tue, Nov 30, 2010 at 12:10 PM, Tom Lane t...@sss.pgh.pa.us wrote: It's ridiculous to claim that that doubles the cost of VACUUM.  In the worst case, it will add 25% to the cost of setting an all-visible bit on a page where there is no other work to

Re: [HACKERS] crash-safe visibility map, take three

2010-11-30 Thread Heikki Linnakangas
On 30.11.2010 19:22, Tom Lane wrote: But having said that, I wonder whether we need a full-page image for a WAL-logged action that is known to involve only setting a single bit and updating LSN. Would omitting the FPI be any more risky than what happens now (ie, the page does get written back

Re: [HACKERS] crash-safe visibility map, take three

2010-11-30 Thread Robert Haas
On Tue, Nov 30, 2010 at 12:22 PM, Tom Lane t...@sss.pgh.pa.us wrote: But having said that, I wonder whether we need a full-page image for a WAL-logged action that is known to involve only setting a single bit and updating LSN.  Would omitting the FPI be any more risky than what happens now

Re: [HACKERS] crash-safe visibility map, take three

2010-11-30 Thread Robert Haas
On Tue, Nov 30, 2010 at 12:25 PM, Robert Haas robertmh...@gmail.com wrote: On Tue, Nov 30, 2010 at 12:22 PM, Tom Lane t...@sss.pgh.pa.us wrote: But having said that, I wonder whether we need a full-page image for a WAL-logged action that is known to involve only setting a single bit and

Re: [HACKERS] crash-safe visibility map, take three

2010-11-30 Thread Tom Lane
Heikki Linnakangas heikki.linnakan...@enterprisedb.com writes: On 30.11.2010 19:22, Tom Lane wrote: But having said that, I wonder whether we need a full-page image for a WAL-logged action that is known to involve only setting a single bit and updating LSN. You have to write a full-page

Re: [HACKERS] [GENERAL] column-level update privs + lock table

2010-11-30 Thread Simon Riggs
On Mon, 2010-11-29 at 21:37 -0500, Josh Kupershmidt wrote: I still see little reason to make LOCK TABLE permissions different for column-level vs. table-level UPDATE privileges Agreed. This is the crux of the debate. Why should this inconsistency be allowed to continue? Are there covert

[HACKERS]

2010-11-30 Thread rickytato rickytato

Re: [HACKERS] DELETE with LIMIT (or my first hack)

2010-11-30 Thread Daniel Loureiro
3. This doesn't work tremendously well for inheritance trees, where ModifyTable acts as sort of an implicit Append node. You can't just funnel all the tuples through one Sort or Limit node because they aren't all the same rowtype. (Limit might perhaps not care, but Sort will.) But you can't

Re: [HACKERS] [GENERAL] column-level update privs + lock table

2010-11-30 Thread Robert Haas
On Tue, Nov 30, 2010 at 7:26 AM, Simon Riggs si...@2ndquadrant.com wrote: On Mon, 2010-11-29 at 21:37 -0500, Josh Kupershmidt wrote: I still see little reason to make LOCK TABLE permissions different for column-level vs. table-level UPDATE privileges Agreed. This is the crux of the debate.

Re: [HACKERS] DELETE with LIMIT (or my first hack)

2010-11-30 Thread Daniel Loureiro
to me the key its security - its a anti-DBA-with-lack-of-attention feature. If i forget the WHERE statement, I will delete some valid tuples and messed up the bd, but its less-than-worst that exclude all the table. A DBA who never forgot an WHERE in an DELETE is not an DBA. Just kidding, but this

Re: [HACKERS] Instrument checkpoint sync calls

2010-11-30 Thread Cédric Villemain
2010/11/30 Greg Smith g...@2ndquadrant.com: Jeff Janes wrote: For the individual file sync times emitted under debug1, it would be very handy if the file being synced was identified, for example relation base/16384/16523. Rather than being numbered sequentially within a given checkpoint.

Re: [HACKERS] [GENERAL] column-level update privs + lock table

2010-11-30 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: Well, a user with full-table UPDATE privileges can trash the whole thing, so, from a security perspective, letting them lock is only allowing them to deny access to data they could have just as easily trashed. A user with only single-column UPDATE

Re: [HACKERS] DELETE with LIMIT (or my first hack)

2010-11-30 Thread Kevin Grittner
Daniel Loureiro dan...@termasa.com.br wrote: to me the key its security - its a anti-DBA-with-lack-of-attention feature. Well, it seems pretty weak to me for that purpose. You still trash data, and you don't have any immediate clue as to what. If you wanted protection from that you'd want

Re: [HACKERS] DELETE with LIMIT (or my first hack)

2010-11-30 Thread Andrew Dunstan
On 11/30/2010 02:12 PM, Kevin Grittner wrote: Daniel Loureirodan...@termasa.com.br wrote: to me the key its security - its a anti-DBA-with-lack-of-attention feature. Well, it seems pretty weak to me for that purpose. You still trash data, and you don't have any immediate clue as to what.

Re: [HACKERS] DELETE with LIMIT (or my first hack)

2010-11-30 Thread Marko Tiikkaja
On 11/30/2010 02:12 PM, Kevin Grittner wrote: Daniel Loureirodan...@termasa.com.br wrote: to me the key its security - its a anti-DBA-with-lack-of-attention feature. Well, it seems pretty weak to me for that purpose. You still trash data, and you don't have any immediate clue as to what.

Re: [HACKERS] profiling connection overhead

2010-11-30 Thread Peter Eisentraut
On mån, 2010-11-29 at 13:10 -0500, Tom Lane wrote: Rolling in calloc in place of malloc/memset made no particular difference either, which says that Fedora 13's glibc does not have any optimization for that case as I'd hoped. glibc's calloc is either mmap of /dev/zero or malloc followed by

Re: [HACKERS] DELETE with LIMIT (or my first hack)

2010-11-30 Thread Jeff Davis
On Tue, 2010-11-30 at 11:12 -0500, Robert Haas wrote: 3. This doesn't work tremendously well for inheritance trees, where ModifyTable acts as sort of an implicit Append node. You can't just funnel all the tuples through one Sort or Limit node because they aren't all the same rowtype.

Re: [HACKERS] DELETE with LIMIT (or my first hack)

2010-11-30 Thread Marko Tiikkaja
While reading this thread, I thought of two things I think we could do if this feature was implemented: 1. Sort large UPDATE/DELETEs so it is done in heap order This is actually a TODO item. I imagine it would be possible to do something like: DELETE FROM foo USING (...) ORDER BY ctid;

[HACKERS] KNNGIST next step: adjusting indexAM API

2010-11-30 Thread Tom Lane
In the current KNNGIST patch, the indexable ORDER BY clauses are transmitted to the executor by cramming them in with the index qual conditions (the IndexScan plan node's indexqual list), from whence they become part of the ScanKey array passed to the index AM. Robert complained that this was an

Re: [HACKERS] Fix for seg picksplit function

2010-11-30 Thread Yeb Havinga
On 2010-11-16 09:57, Alexander Korotkov wrote: On Tue, Nov 16, 2010 at 3:07 AM, Robert Haas robertmh...@gmail.com mailto:robertmh...@gmail.com wrote: The loop that begins here: for (i = 0; i maxoff; i++) { /* First half of segs goes to the left datum. */

Re: [HACKERS] Another proposal for table synonyms

2010-11-30 Thread Josh Berkus
Alexey, Here is the proposal to add synonyms to PostgreSQL. Initial goal is to add synonyms for relations (tables, views, sequences) and an infrastructure to allow synonyms for other database objects in the future. Can you explain, for our benefit, the use case for this? Specifically,

[HACKERS] Where are we on Standby Promotion?

2010-11-30 Thread Josh Berkus
Fujii, Simon, Greg, etc.: Has anyone submitted or committed a patch to make Standby Promotion* easier, at this point? We discussed it earlier in the dev cycle, but I can't find anything which has actually been submitted. -- -- Josh Berkus

Re: [HACKERS] Instrument checkpoint sync calls

2010-11-30 Thread Jeff Janes
On Tue, Nov 30, 2010 at 8:38 AM, Greg Smith g...@2ndquadrant.com wrote: Hi Greg, Thanks for the update. This might be ready for some proper review now.  I know there's at least one blatant bug still in here I haven't found yet, related to how the averages are computed. Yes, the blatant

Re: [HACKERS] DELETE with LIMIT (or my first hack)

2010-11-30 Thread Andres Freund
On Tuesday 30 November 2010 20:24:52 Marko Tiikkaja wrote: On 11/30/2010 02:12 PM, Kevin Grittner wrote: Daniel Loureirodan...@termasa.com.br wrote: to me the key its security - its a anti-DBA-with-lack-of-attention feature. Well, it seems pretty weak to me for that purpose. You

Re: [HACKERS] Spread checkpoint sync

2010-11-30 Thread Greg Smith
Ron Mayer wrote: Might smoother checkpoints be better solved by talking to the OS vendors virtual-memory-tunning-knob-authors to work with them on exposing the ideal knobs; rather than saying that our only tool is a hammer(fsync) so the problem must be handled as a nail. Maybe, but it's

Re: [HACKERS] DELETE with LIMIT (or my first hack)

2010-11-30 Thread Alastair Turner
On Tue, Nov 30, 2010 at 9:24 PM, Marko Tiikkaja marko.tiikk...@cs.helsinki.fi wrote: On 11/30/2010 02:12 PM, Kevin Grittner wrote: Daniel Loureirodan...@termasa.com.br   wrote: to me the key its security - its a anti-DBA-with-lack-of-attention feature. Well, it seems pretty weak to me for

Re: [HACKERS] DELETE with LIMIT (or my first hack)

2010-11-30 Thread Andrew Dunstan
On 11/30/2010 03:16 PM, Andres Freund wrote: On Tuesday 30 November 2010 20:24:52 Marko Tiikkaja wrote: On 11/30/2010 02:12 PM, Kevin Grittner wrote: Daniel Loureirodan...@termasa.com.brwrote: to me the key its security - its a anti-DBA-with-lack-of-attention feature. Well, it seems

Re: [HACKERS] profiling connection overhead

2010-11-30 Thread Tom Lane
Peter Eisentraut pete...@gmx.net writes: On mån, 2010-11-29 at 13:10 -0500, Tom Lane wrote: Rolling in calloc in place of malloc/memset made no particular difference either, which says that Fedora 13's glibc does not have any optimization for that case as I'd hoped. glibc's calloc is

Re: [HACKERS] DELETE with LIMIT (or my first hack)

2010-11-30 Thread Robert Haas
On Tue, Nov 30, 2010 at 2:45 PM, Jeff Davis pg...@j-davis.com wrote: On Tue, 2010-11-30 at 11:12 -0500, Robert Haas wrote: 3. This doesn't work tremendously well for inheritance trees, where ModifyTable acts as sort of an implicit Append node.  You can't just funnel all the tuples through

Re: [HACKERS] Spread checkpoint sync

2010-11-30 Thread Josh Berkus
Maybe, but it's hard to argue that the current implementation--just doing all of the sync calls as fast as possible, one after the other--is going to produce worst-case behavior in a lot of situations. Given that it's not a huge amount of code to do better, I'd rather do some work in that

Re: [HACKERS] KNNGIST next step: adjusting indexAM API

2010-11-30 Thread Robert Haas
On Tue, Nov 30, 2010 at 2:50 PM, Tom Lane t...@sss.pgh.pa.us wrote: In the current KNNGIST patch, the indexable ORDER BY clauses are transmitted to the executor by cramming them in with the index qual conditions (the IndexScan plan node's indexqual list), from whence they become part of the

Re: [HACKERS] DELETE with LIMIT (or my first hack)

2010-11-30 Thread Tom Lane
Marko Tiikkaja marko.tiikk...@cs.helsinki.fi writes: While reading this thread, I thought of two things I think we could do if this feature was implemented: 1. Sort large UPDATE/DELETEs so it is done in heap order This is actually a TODO item. I imagine it would be possible to do

Re: [HACKERS] DELETE with LIMIT (or my first hack)

2010-11-30 Thread Dimitri Fontaine
Andres Freund and...@anarazel.de writes: On Tuesday 30 November 2010 20:24:52 Marko Tiikkaja wrote: I don't buy the argument either; why would you put a LIMIT there and delete one row by accident when you could put a BEGIN; in front and not do any damage at all? Because the delete of the

Re: [HACKERS] DELETE with LIMIT (or my first hack)

2010-11-30 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: You seem to be imagining the MergeAppend node on top, but I had it in the other order in my mind. The ModifyTable node would be the outermost plan node, pulling from the Limit, which would deliver the first n table rows from the MergeAppend, which

Re: [HACKERS] DELETE with LIMIT (or my first hack)

2010-11-30 Thread Alvaro Herrera
Excerpts from Daniel Loureiro's message of mar nov 30 15:04:17 -0300 2010: So I guess that I have choose the wrong hack to start. So it seems :-D -- Álvaro Herrera alvhe...@commandprompt.com The PostgreSQL Company - Command Prompt, Inc. PostgreSQL Replication, Consulting, Custom Development,

Re: [HACKERS] DELETE with LIMIT (or my first hack)

2010-11-30 Thread Jeff Davis
On Tue, 2010-11-30 at 15:52 -0500, Robert Haas wrote: On Tue, Nov 30, 2010 at 2:45 PM, Jeff Davis pg...@j-davis.com wrote: On Tue, 2010-11-30 at 11:12 -0500, Robert Haas wrote: Could you possibly have ModifyTable - Limit - MergeAppend? Before MergeAppend knows which tuple to produce, it

Re: [HACKERS] Spread checkpoint sync

2010-11-30 Thread Jeff Janes
On Sun, Nov 14, 2010 at 3:48 PM, Greg Smith g...@2ndquadrant.com wrote: ... One change that turned out be necessary rather than optional--to get good performance from the system under tuning--was to make regular background writer activity, including fsync absorb checks, happen during these

Re: [HACKERS] crash-safe visibility map, take three

2010-11-30 Thread Bruce Momjian
Heikki Linnakangas wrote: On 30.11.2010 18:33, Tom Lane wrote: Robert Haasrobertmh...@gmail.com writes: Oh, but it's worse than that. When you XLOG a WAL record for each of those pages, you're going to trigger full-page writes for all of them. So now you've turned 1GB of data to write

Re: [HACKERS] DELETE with LIMIT (or my first hack)

2010-11-30 Thread Bruce Momjian
Daniel Loureiro wrote: 3. This doesn't work tremendously well for inheritance trees, where ModifyTable acts as sort of an implicit Append node. You can't just funnel all the tuples through one Sort or Limit node because they aren't all the same rowtype. (Limit might perhaps not care, but

Re: [HACKERS] Where are we on Standby Promotion?

2010-11-30 Thread Bruce Momjian
Josh Berkus wrote: Fujii, Simon, Greg, etc.: Has anyone submitted or committed a patch to make Standby Promotion* easier, at this point? We discussed it earlier in the dev cycle, but I can't find anything which has actually been submitted. * that is, promotion to be a new master of other

Re: [HACKERS] GiST insert algorithm rewrite

2010-11-30 Thread Bruce Momjian
Heikki Linnakangas wrote: Does the current code cope with the corruption? It's not corruption, but intended degradation. Yes, the current code copes with it, that's how GiST survives a crash. However, even with the current code, VACUUM will nag if it finds any invalid tuples with this

Re: [HACKERS] GiST insert algorithm rewrite

2010-11-30 Thread Robert Haas
On Tue, Nov 30, 2010 at 10:26 AM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: Does the current code cope with the corruption? It's not corruption, but intended degradation. Yes, the current code copes with it, that's how GiST survives a crash. However, even with the current

[HACKERS] We really ought to do something about O_DIRECT and data=journalled on ext4

2010-11-30 Thread Josh Berkus
Hackers, Some of you might already be aware that this combination produces a fatal startup crash in PostgreSQL: 1. Create an Ext3 or Ext4 partition and mount it with data=journal on a server with linux kernel 2.6.30 or later. 2. Initdb a PGDATA on that partition 3. Start PostgreSQL with the

Re: [HACKERS] Where are we on Standby Promotion?

2010-11-30 Thread Robert Haas
On Tue, Nov 30, 2010 at 9:03 PM, Bruce Momjian br...@momjian.us wrote: Josh Berkus wrote: Fujii, Simon, Greg, etc.: Has anyone submitted or committed a patch to make Standby Promotion* easier, at this point?  We discussed it earlier in the dev cycle, but I can't find anything which has

Re: [HACKERS] We really ought to do something about O_DIRECT and data=journalled on ext4

2010-11-30 Thread Tom Lane
Josh Berkus j...@agliodbs.com writes: Apparently, testing for O_DIRECT at compile time isn't adequate. Ideas? We should wait for the outcome of the discussion about whether to change the default wal_sync_method before worrying about this. regards, tom lane -- Sent via

Re: [HACKERS] We really ought to do something about O_DIRECT and data=journalled on ext4

2010-11-30 Thread Josh Berkus
On 11/30/10 7:09 PM, Tom Lane wrote: Josh Berkus j...@agliodbs.com writes: Apparently, testing for O_DIRECT at compile time isn't adequate. Ideas? We should wait for the outcome of the discussion about whether to change the default wal_sync_method before worrying about this. Are we

  1   2   >