Re: [HACKERS] Hypothetical Indexes - PostgreSQL extension - PGCON 2010

2010-12-03 Thread Jeroen Vermeulen
On 2010-12-02 00:48, Ana Carolina Brito de Almeida wrote: We would like to inform you all that our extension to PostgreSQL, that includes hypothetical indexes (and soon index self-tuning), is available through a sourgeforge project. This was suggested at PgCon 2010 and we hope some of you may

Re: [HACKERS] Hi- How frequently Postgres Poll for trigger file

2010-12-03 Thread Heikki Linnakangas
On 03.12.2010 03:55, Fujii Masao wrote: On Thu, Dec 2, 2010 at 12:16 AM, Euler Taveira de Oliveira eu...@timbira.com wrote: As you said, there are platforms that a signal doesn't wake up a process, so I suggest (ii) but I'm fine to include (i) at docs too. Can we use pg_ctl kill to send

Re: [HACKERS] Author names in source files

2010-12-03 Thread Dimitri Fontaine
Itagaki Takahiro itagaki.takah...@gmail.com writes: Dimitri, your name cannot be added in source files, but will be recorded in commit logs ;-) Oh, I've only been adding it to src/backend/utils/adt/genfile.c because I saw another name in there. So of course a stricter following of the project's

Re: [HACKERS] pg_execute_from_file review

2010-12-03 Thread Dimitri Fontaine
Itagaki Takahiro itagaki.takah...@gmail.com writes: I fixed and cleanup some of codes in it; v9 patch attached. Please check my modifications, and set the status to Ready to Committer if you find no problems. I think documentation and code comments might need to be checked by native English

Re: [HACKERS] directory archive format for pg_dump

2010-12-03 Thread Heikki Linnakangas
On 02.12.2010 23:12, Alvaro Herrera wrote: Excerpts from Heikki Linnakangas's message of jue dic 02 16:52:27 -0300 2010: Ok, committed, with some small cleanup since the last patch I posted. I think the comments on _ReadBuf and friends need to be updated, since they are not just for headers

Re: [HACKERS] Hypothetical Indexes - PostgreSQL extension - PGCON 2010

2010-12-03 Thread Richard Huxton
On 03/12/10 08:14, Jeroen Vermeulen wrote: On 2010-12-02 00:48, Ana Carolina Brito de Almeida wrote: We would like to inform you all that our extension to PostgreSQL, that includes hypothetical indexes (and soon index self-tuning), is available through a sourgeforge project. Looking at the

Re: [HACKERS] WIP patch for parallel pg_dump

2010-12-03 Thread Robert Haas
On Thu, Dec 2, 2010 at 9:33 PM, Tom Lane t...@sss.pgh.pa.us wrote: Andrew Dunstan and...@dunslane.net writes: Umm, nobody has attributed ridiculousness to anyone. Please don't put words in my mouth. But I think this is a perfectly reasonable discussion to have. Nobody gets to come along and

Re: [HACKERS] unlogged tables

2010-12-03 Thread Robert Haas
On Thu, Dec 2, 2010 at 10:53 PM, Alvaro Herrera alvhe...@commandprompt.com wrote: Excerpts from Andy Colson's message of vie dic 03 00:37:17 -0300 2010: Ok, forget the time thing.  Has nothing to do with it.  (Which everyone already assumed I imagine). Its truncate. Create unloged table,

Re: [HACKERS] Another proposal for table synonyms

2010-12-03 Thread Alexey Klyukin
On Dec 3, 2010, at 2:17 AM, Alvaro Herrera wrote: Excerpts from Robert Haas's message of jue dic 02 21:10:48 -0300 2010: On Thu, Dec 2, 2010 at 3:43 PM, Alvaro Herrera alvhe...@commandprompt.com wrote: Excerpts from Robert Haas's message of jue dic 02 17:27:01 -0300 2010: Yeah, the Oracle

Re: [HACKERS] [PATCH] Custom code int(32|64) = text conversions out of performance reasons

2010-12-03 Thread Florian Weimer
* Tom Lane: Yeah. You certainly don't want to do the division sequence twice, and a log() call wouldn't be cheap either, and there don't seem to be many other alternatives. What about a sequence of comparisons, and unrolling the loop? That could avoid the final division, too. It might also

Re: [HACKERS] WIP patch for parallel pg_dump

2010-12-03 Thread Andrew Dunstan
On 12/02/2010 11:44 PM, Joachim Wieland wrote: On Thu, Dec 2, 2010 at 9:33 PM, Tom Lanet...@sss.pgh.pa.us wrote: In particular, this issue *has* been discussed before, and there was a consensus that preserving dump consistency was a requirement. I don't think that Joachim gets to bypass

Re: [HACKERS] Hypothetical Indexes - PostgreSQL extension - PGCON 2010

2010-12-03 Thread Jeroen Vermeulen
On 2010-12-03 19:44, Sergio Lifschitz wrote: Indeed, hypothetical indexes are good to check potentially good configurations without harming the whole system with actual index creation. Please observer that we've added an explain hypothetical command, that will include plans considering

Re: [HACKERS] Extensions, this time with a patch

2010-12-03 Thread Robert Haas
On Thu, Nov 25, 2010 at 4:06 PM, Dimitri Fontaine dimi...@2ndquadrant.fr wrote: Itagaki Takahiro itagaki.takah...@gmail.com writes: Thanks. I'll move the patch to Ready for Committer. Thanks! I have committed the cfparser patch to which the above comments refer. One patch per thread makes

Re: [HACKERS] Extensions, this time with a patch

2010-12-03 Thread Dimitri Fontaine
Robert Haas robertmh...@gmail.com writes: I have committed the cfparser patch to which the above comments refer. Excellent, thank you! On to merging that into the extension's main branch, will still wait until after pg_execute_sql_file() is in to produce extension.v15.patch, unless advised

Re: [HACKERS] Hypothetical Indexes - PostgreSQL extension - PGCON 2010

2010-12-03 Thread Ana Carolina Brito de Almeida
Jeroen, We add a simple case study (sourceforge page): http://sourceforge.net/projects/hypotheticalind/files/TUTORIAL_8_4.pdf/download Although this tutorial is for version 8.4, it also applies to other versions. Att, Ana Carolina 2010/12/3 Jeroen Vermeulen j...@xs4all.nl On 2010-12-03

Re: [HACKERS] crash-safe visibility map, take three

2010-12-03 Thread Florian Weimer
* Robert Haas: Those hint bit tests are a single machine instruction. It's tough to beat that. It's tough to get within two orders of magnitude. I'd like to, but I don't see how. For some scans, it might be possible to hoist the checks out of inner loops. (At least in principle, I'm not

Re: [HACKERS] [GENERAL] column-level update privs + lock table

2010-12-03 Thread Robert Haas
On Tue, Nov 30, 2010 at 1:48 PM, Tom Lane t...@sss.pgh.pa.us wrote: On the whole I agree with Robert --- let's just adjust the documentation, and not enlarge privileges in a way we might regret later. Done. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL

Re: [HACKERS] should we set hint bits without dirtying the page?

2010-12-03 Thread Robert Haas
On Thu, Dec 2, 2010 at 7:00 PM, Robert Haas robertmh...@gmail.com wrote: But maybe we could ameliorate that problem by freezing more aggressively. I realized as I was falling asleep last night any sort of more aggressive freezing is going to be a huge bummer for Hot Standby users, for which

Re: [HACKERS] WIP patch for parallel pg_dump

2010-12-03 Thread Robert Haas
On Fri, Dec 3, 2010 at 8:02 AM, Andrew Dunstan and...@dunslane.net wrote: I think Josh Berkus' comments in the thread you mentioned are correct: Actually, I'd say that there's a broad set of cases of people who want to do a parallel pg_dump while their system is active.  Parallel pg_dump on a

Re: [HACKERS] improving foreign key locks

2010-12-03 Thread Alvaro Herrera
Of course, there is a much more serious problem with the whole idea. I have worked through most of the necessary changes and I'm now down to changing heap_udate to comply with the new locking protocol. The problem I'm now facing is that we need to know the set of updated columns pretty early --

Re: [HACKERS] WIP patch for parallel pg_dump

2010-12-03 Thread Andrew Dunstan
On 12/03/2010 11:23 AM, Robert Haas wrote: On Fri, Dec 3, 2010 at 8:02 AM, Andrew Dunstanand...@dunslane.net wrote: I think Josh Berkus' comments in the thread you mentioned are correct: Actually, I'd say that there's a broad set of cases of people who want to do a parallel pg_dump while

Re: [HACKERS] WIP patch for parallel pg_dump

2010-12-03 Thread Robert Haas
On Fri, Dec 3, 2010 at 11:40 AM, Andrew Dunstan and...@dunslane.net wrote: On 12/03/2010 11:23 AM, Robert Haas wrote: On Fri, Dec 3, 2010 at 8:02 AM, Andrew Dunstanand...@dunslane.net  wrote: I think Josh Berkus' comments in the thread you mentioned are correct: Actually, I'd say that

Re: [HACKERS] WIP patch for parallel pg_dump

2010-12-03 Thread Alvaro Herrera
Excerpts from Robert Haas's message of vie dic 03 13:56:32 -0300 2010: I know the use cases are limited, but I think it's still useful on its own. I don't understand what's so difficult about starting with the snapshot cloning patch. AFAIR it's already been written anyway, no? -- Álvaro

Re: [HACKERS] Hypothetical Indexes - PostgreSQL extension - PGCON 2010

2010-12-03 Thread Sergio Lifschitz
Indeed, hypothetical indexes are good to check potentially good configurations without harming the whole system with actual index creation. Please observer that we've added an explain hypothetical command, that will include plans considering hypothetical indexes! We'll try to add a simple case

[HACKERS] disk caching for writing log

2010-12-03 Thread flyusa2010 fly
When writing log, dbms should synchronously flush log to disk. I'm wondering, if it is possible that the logs are in disk cache, while the control is returned to dbms again, so dbms thinks logs are persistent on disk. In this case, if the disk fails, then there's incorrectness for dbms log

Re: [HACKERS] Idle git question: how come so many objects?

2010-12-03 Thread Florian Weimer
* Tom Lane: $ git push Counting objects: 172, done. Compressing objects: 100% (89/89), done. Writing objects: 100% (89/89), 17.07 KiB, done. Total 89 (delta 80), reused 0 (delta 0) To ssh://g...@gitmaster.postgresql.org/postgresql.git 35a3def..8a6eb2e REL8_1_STABLE - REL8_1_STABLE

Re: [HACKERS] WIP patch for parallel pg_dump

2010-12-03 Thread Andrew Dunstan
On 12/03/2010 12:17 PM, Alvaro Herrera wrote: Excerpts from Robert Haas's message of vie dic 03 13:56:32 -0300 2010: I know the use cases are limited, but I think it's still useful on its own. I don't understand what's so difficult about starting with the snapshot cloning patch. AFAIR it's

Re: [HACKERS] disk caching for writing log

2010-12-03 Thread Heikki Linnakangas
On 03.12.2010 13:49, flyusa2010 fly wrote: When writing log, dbms should synchronously flush log to disk. I'm wondering, if it is possible that the logs are in disk cache, while the control is returned to dbms again, so dbms thinks logs are persistent on disk. In this case, if the disk fails,

Re: [HACKERS] disk caching for writing log

2010-12-03 Thread Stefan Kaltenbrunner
On 12/03/2010 06:43 PM, Heikki Linnakangas wrote: On 03.12.2010 13:49, flyusa2010 fly wrote: When writing log, dbms should synchronously flush log to disk. I'm wondering, if it is possible that the logs are in disk cache, while the control is returned to dbms again, so dbms thinks logs are

Re: [HACKERS] Extensions

2010-12-03 Thread David E. Wheeler
On Dec 3, 2010, at 8:38 AM, Dimitri Fontaine wrote: David, and anyone feeling like reviewing or trying the patch, if you're not already crawling into the v14 patch, you could as well begin with this cleaner version — no behavior changes, some cleaner code, make check passes, no bitrot against

Re: [HACKERS] should we set hint bits without dirtying the page?

2010-12-03 Thread Jeff Davis
On Thu, 2010-12-02 at 19:00 -0500, Robert Haas wrote: Untidy buffers would be treated as dirty by the background writer cleaning scan, but as clean by checkpoints and by backends doing emergency buffer cleaning to feed new allocations. Differentiating between a backend write and a bgwriter

Re: [HACKERS] Patch to add a primary key using an existing index

2010-12-03 Thread Peter Eisentraut
On sön, 2010-11-28 at 20:40 -0500, Robert Haas wrote: On Sun, Nov 28, 2010 at 8:06 PM, Itagaki Takahiro itagaki.takah...@gmail.com wrote: On Fri, Nov 26, 2010 at 05:58, Steve Singer ssin...@ca.afilias.info wrote: The attached version of the patch gets your regression tests to pass. I'm

Re: [HACKERS] Patch to add a primary key using an existing index

2010-12-03 Thread Robert Haas
On Fri, Dec 3, 2010 at 2:23 PM, Peter Eisentraut pete...@gmx.net wrote: On sön, 2010-11-28 at 20:40 -0500, Robert Haas wrote: On Sun, Nov 28, 2010 at 8:06 PM, Itagaki Takahiro itagaki.takah...@gmail.com wrote: On Fri, Nov 26, 2010 at 05:58, Steve Singer ssin...@ca.afilias.info wrote: The

Re: [HACKERS] On-the-fly index tuple deletion vs. hot_standby

2010-12-03 Thread Heikki Linnakangas
On 29.11.2010 08:10, Noah Misch wrote: I have a hot_standby system and use it to bear the load of various reporting queries that take 15-60 minutes each. In an effort to avoid long pauses in recovery, I set a vacuum_defer_cleanup_age constituting roughly three hours of the master's

Re: [HACKERS] Patch to add a primary key using an existing index

2010-12-03 Thread Heikki Linnakangas
On 03.12.2010 21:43, Robert Haas wrote: On Fri, Dec 3, 2010 at 2:23 PM, Peter Eisentrautpete...@gmx.net wrote: On sön, 2010-11-28 at 20:40 -0500, Robert Haas wrote: On Sun, Nov 28, 2010 at 8:06 PM, Itagaki Takahiro itagaki.takah...@gmail.com wrote: On Fri, Nov 26, 2010 at 05:58, Steve

Re: [HACKERS] We really ought to do something about O_DIRECT and data=journalled on ext4

2010-12-03 Thread Josh Berkus
All, So, I've been doing some reading about this issue, and I think regardless of what other changes we make we should never enable O_DIRECT automatically on Linux, and it was a mistake for us to do so in the first place. First, in the Linux docs for open(): = In summary, O_DIRECT is a

Re: [HACKERS] Patch to add a primary key using an existing index

2010-12-03 Thread Alvaro Herrera
Excerpts from Heikki Linnakangas's message of vie dic 03 16:45:59 -0300 2010: ALTER TABLE table_name SET PRIMARY KEY USING INDEX index_name. Quite verbose, but imho USING makes it much more clear that it's an existing index. I was going to post the same thing (well except I was still

Re: [HACKERS] We really ought to do something about O_DIRECT and data=journalled on ext4

2010-12-03 Thread Heikki Linnakangas
On 03.12.2010 21:55, Josh Berkus wrote: All, So, I've been doing some reading about this issue, and I think regardless of what other changes we make we should never enable O_DIRECT automatically on Linux, and it was a mistake for us to do so in the first place. First, in the Linux docs for

Re: [HACKERS] Patch to add a primary key using an existing index

2010-12-03 Thread r t
On Fri, Dec 3, 2010 at 2:43 PM, Robert Haas robertmh...@gmail.com wrote: On Fri, Dec 3, 2010 at 2:23 PM, Peter Eisentraut pete...@gmx.net wrote: On sön, 2010-11-28 at 20:40 -0500, Robert Haas wrote: On Sun, Nov 28, 2010 at 8:06 PM, Itagaki Takahiro itagaki.takah...@gmail.com wrote: On

Re: [HACKERS] Patch to add a primary key using an existing index

2010-12-03 Thread Heikki Linnakangas
On 03.12.2010 21:58, Alvaro Herrera wrote: Excerpts from Heikki Linnakangas's message of vie dic 03 16:45:59 -0300 2010: ALTER TABLE table_name SET PRIMARY KEY USING INDEX index_name. Quite verbose, but imho USING makes it much more clear that it's an existing index. I was going to post the

Re: [HACKERS] [PATCH] Revert default wal_sync_method to fdatasync on Linux 2.6.33+

2010-12-03 Thread Josh Berkus
All, So, this week I've had my hands on a medium-high-end test system where I could test various wal_sync_methods. This is a 24-core Intel Xeon machine with 72GB of ram, and 8 internal 10K SAS disks attached to a raid controller with 512MB BBU write cache. 2 of the disks are in a RAID1, which

Re: [HACKERS] Patch to add a primary key using an existing index

2010-12-03 Thread Robert Haas
On Fri, Dec 3, 2010 at 2:56 PM, r t pg...@xzilla.net wrote: What exactly was the objection to the following -- ALTER TABLE table_name ADD PRIMARY KEY (column_list) USING index_name; Is the objection that you might have been trying to specify a constraint named using ? I'm willing to make that

Re: [HACKERS] Patch to add a primary key using an existing index

2010-12-03 Thread Josh Berkus
On 12/3/10 12:27 PM, Robert Haas wrote: On Fri, Dec 3, 2010 at 2:56 PM, r t pg...@xzilla.net wrote: What exactly was the objection to the following -- ALTER TABLE table_name ADD PRIMARY KEY (column_list) USING index_name; Is the objection that you might have been trying to specify a constraint

[HACKERS] ERROR: could not identify an equality operator for type box

2010-12-03 Thread Oleg Bartunov
Hi there, create table qq (b box); CREATE TABLE select count(*), b from qq group by b; ERROR: could not identify an equality operator for type box LINE 1: select count(*), b from qq group by b; but following select works fine select ' (43.6038,48.8664536),(1.3620777,1.44327)'::box = '

Re: [HACKERS] ERROR: could not identify an equality operator for type box

2010-12-03 Thread Oleg Bartunov
Ok, casting to ::text solves the problem, but still I think we need to fix it in the right way Oleg On Sat, 4 Dec 2010, Oleg Bartunov wrote: Hi there, create table qq (b box); CREATE TABLE select count(*), b from qq group by b; ERROR: could not identify an equality operator for type box

Re: [HACKERS] ERROR: could not identify an equality operator for type box

2010-12-03 Thread Heikki Linnakangas
On 03.12.2010 23:53, Oleg Bartunov wrote: create table qq (b box); CREATE TABLE select count(*), b from qq group by b; ERROR: could not identify an equality operator for type box LINE 1: select count(*), b from qq group by b; but following select works fine select '

Re: [HACKERS] Patch to add a primary key using an existing index

2010-12-03 Thread Robert Treat
On Fri, Dec 3, 2010 at 4:41 PM, Josh Berkus j...@agliodbs.com wrote: On 12/3/10 12:27 PM, Robert Haas wrote: On Fri, Dec 3, 2010 at 2:56 PM, r t pg...@xzilla.net wrote: What exactly was the objection to the following -- ALTER TABLE table_name ADD PRIMARY KEY (column_list) USING index_name;

Re: [HACKERS] Patch to add a primary key using an existing index

2010-12-03 Thread Ross J. Reedstrom
On Fri, Dec 03, 2010 at 05:16:04PM -0500, Robert Treat wrote: On Fri, Dec 3, 2010 at 4:41 PM, Josh Berkus j...@agliodbs.com wrote: However, I don't see why we need (column_list). Surely the index has a column list already? ALTER TABLE table_name ADD CONSTRAINT pk_name PRIMARY KEY USING

Re: [HACKERS] Patch to add a primary key using an existing index

2010-12-03 Thread Josh Berkus
On 12/3/10 2:16 PM, Robert Treat wrote: Uh, the syntax I posted was based on this currently valid syntax: ALTER TABLE table_name ADD PRIMARY KEY (column_list); The constraint bit is optional, which is why I left it out, but I presume it would be optional with the new syntax as well...

Re: [HACKERS] ERROR: could not identify an equality operator for type box

2010-12-03 Thread Tom Lane
Heikki Linnakangas heikki.linnakan...@enterprisedb.com writes: On 03.12.2010 23:53, Oleg Bartunov wrote: create table qq (b box); CREATE TABLE select count(*), b from qq group by b; ERROR: could not identify an equality operator for type box What does it means ? GROUP BY requires b-tree

Re: [HACKERS] pg_execute_from_file review

2010-12-03 Thread Itagaki Takahiro
On Fri, Dec 3, 2010 at 18:02, Dimitri Fontaine dimi...@2ndquadrant.fr wrote:   Schema   |  Name   | Result data type | Argument data types |  Type +-+--+-+  pg_catalog | replace | text             | text, VARIADIC text | normal  

[HACKERS] Review: Extensions Patch

2010-12-03 Thread David E. Wheeler
Extensions Patch v15 Review === Submission review - * Is the patch in context diff format? Yes. * Does it apply cleanly to the current git master? Yes. * Does it include reasonable tests, necessary doc patches, etc? `make check` passes. `make

[HACKERS] Streaming replication document

2010-12-03 Thread Tatsuo Ishii
Hi, There is a description about streaming replication in the doc: -- 25.2.5. Streaming Replication : : If you set up a WAL archive that's accessible from the standby, wal_keep_segments is not required as the standby can always use the

Re: [HACKERS] We need index-only scans

2010-12-03 Thread MARK CALLAGHAN
On Wed, Dec 1, 2010 at 3:53 AM, Kristian Nielsen kniel...@knielsen-hq.org wrote: Greg Stark gsst...@mit.edu writes: Just so everyone is on the same page Even once we have index-only scans they won't be anywhere near as useful with Postgres as they are with Oracle and other databases. At

[HACKERS] Requirement for the Buffer manager to load multiple pages at once

2010-12-03 Thread Vaibhav Kaushal
Hi all, I have gone through the code a bit more this time and I think that what I originally thought of as a task of executor in the beginning (as I read the documentation) seems to be split at multiple points. So i am asking a question based on what I have understood so far. Keeping it a bit

Re: [HACKERS] GiST insert algorithm rewrite

2010-12-03 Thread Robert Haas
On Dec 3, 2010, at 4:54 PM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: Here's an updated patch. How carefully have you perf-tested this? On closer look, supporting the invalid tuples in scans was trivial, so I kept that after all. So you can still query an index with

[HACKERS] Note about KNNGIST limitation: no mark/restore

2010-12-03 Thread Tom Lane
While working on the KNNGIST documentation I noticed that it's possible to create queries that will try to use a KNN scan as the inside of a mergejoin, leading to a failure because GIST hasn't got mark/restore support. For example, in the current HEAD regression database: regression=# set

Re: [HACKERS] knngist - 0.8

2010-12-03 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: 2010/9/13 Teodor Sigaev teo...@sigaev.ru: [updated patch] I realize I'm repeating myself, but... this patch needs documentation. It's not optional. I've applied all of this, and written documentation for all of it, except for the contrib/btree_gist

Re: [HACKERS] Patch to add a primary key using an existing index

2010-12-03 Thread Tom Lane
Ross J. Reedstrom reeds...@rice.edu writes: If you consider that an index basically is, in some sense, a pre-canned column list, then: ALTER TABLE table_name ADD PRIMARY KEY (column_list); ALTER TABLE table_name ADD PRIMARY KEY USING index_name; are parallel constructions. And it avoids the

[HACKERS] Crash safe visibility map vs hint bits

2010-12-03 Thread jes...@krogh.cc
There has been a lot discussion about index-only scans and how to make the visibillity map crash safe. Then followed by a good discussion about hint bits. What seems to be the main concern is the added wal volume and it makes me wonder if there is a way in-between that looks more like hint

Re: [HACKERS] Crash safe visibility map vs hint bits

2010-12-03 Thread Heikki Linnakangas
On 04.12.2010 09:14, jes...@krogh.cc wrote: There has been a lot discussion about index-only scans and how to make the visibillity map crash safe. Then followed by a good discussion about hint bits. What seems to be the main concern is the added wal volume and it makes me wonder if there is a

Re: [HACKERS] Streaming replication document

2010-12-03 Thread Heikki Linnakangas
On 04.12.2010 04:18, Tatsuo Ishii wrote: There is a description about streaming replication in the doc: -- 25.2.5. Streaming Replication : : If you set up a WAL archive that's accessible from the standby, wal_keep_segments is not required