Re: [HACKERS] Allowing multiple concurrent base backups

2011-03-21 Thread Heikki Linnakangas
On 18.03.2011 13:56, Heikki Linnakangas wrote: On 18.03.2011 10:48, Heikki Linnakangas wrote: On 17.03.2011 21:39, Robert Haas wrote: On Mon, Jan 31, 2011 at 10:45 PM, Fujii Masaomasao.fu...@gmail.com wrote: On Tue, Feb 1, 2011 at 1:31 AM, Heikki Linnakangas

Re: [HACKERS] 2nd Level Buffer Cache

2011-03-21 Thread Greg Stark
On Fri, Mar 18, 2011 at 11:55 PM, Josh Berkus j...@agliodbs.com wrote: To take the opposite approach... has anyone looked at having the OS just manage all caching for us? Something like MMAPed shared buffers? Even if we find the issue with large shared buffers, we still can't dedicate serious

Re: [HACKERS] Rectifying wrong Date outputs

2011-03-21 Thread Heikki Linnakangas
On 21.03.2011 07:40, Piyush Newe wrote: On Thu, Mar 17, 2011 at 7:56 PM, Tom Lanet...@sss.pgh.pa.us wrote: Robert Haasrobertmh...@gmail.com writes: On Thu, Mar 17, 2011 at 9:46 AM, Alvaro Herrera alvhe...@commandprompt.com wrote: Keep in mind that the datetime stuff was abandoned by the

[HACKERS] Planner regression in 9.1: min(x) cannot use partial index with NOT NULL

2011-03-21 Thread Marti Raudsepp
Hi list, When I have fields with lots of null values, I often create indexes like this: CREATE INDEX foo_i ON foo(i) WHERE i IS NOT NULL; This saves me some space, as most indexed queries exclude NULLs anyway. In PostgreSQL 9.0.3, min(i) can successfully use this index: --- marti=#

[HACKERS] How to look at the Expression Trees

2011-03-21 Thread Vaibhav Kaushal
Hi all, I have been trying to work on the expression evaluator (trying to alter it just for the seqscan case). I have understood a few things. I wish someone could tell me if I am wrong at some point. As far as I have gone through the code, I think: 1. Quals are formulated by planner 2. Quals

Re: [HACKERS] How to look at the Expression Trees

2011-03-21 Thread Heikki Linnakangas
On 21.03.2011 13:44, Vaibhav Kaushal wrote: Hi all, I have been trying to work on the expression evaluator (trying to alter it just for the seqscan case). I have understood a few things. I wish someone could tell me if I am wrong at some point. As far as I have gone through the code, I think:

Re: [HACKERS] How to look at the Expression Trees

2011-03-21 Thread Vaibhav Kaushal
On Mon, Mar 21, 2011 at 5:47 PM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: Yes. There's actually two trees involved. The planner produces a tree of Expr nodes, and ExecInitExpr prepares a tree of ExprState nodes that mirrors the first tree. The ExprStates contain run-time

Re: [HACKERS] How to look at the Expression Trees

2011-03-21 Thread Heikki Linnakangas
On 21.03.2011 14:37, Vaibhav Kaushal wrote: I am already using the postgresql server with -d 4 option and it shows a lot of things. But I am not able to see the Expression State trees. To be precise, debug_print_plan=on prints the expression tree that comes from planner, not the execution

Re: [HACKERS] Planner regression in 9.1: min(x) cannot use partial index with NOT NULL

2011-03-21 Thread Robert Haas
On Mon, Mar 21, 2011 at 7:17 AM, Marti Raudsepp ma...@juffo.org wrote: Hi list, When I have fields with lots of null values, I often create indexes like this: CREATE INDEX foo_i ON foo(i) WHERE i IS NOT NULL; This saves me some space, as most indexed queries exclude NULLs anyway. In

Re: [HACKERS] Rectifying wrong Date outputs

2011-03-21 Thread Robert Haas
On Mon, Mar 21, 2011 at 6:24 AM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: Having said that, it's not entirely clear to me what sane behavior is here.  Personally I would expect that an n-Ys format spec would consume at most n digits from the input.  Otherwise how are you

Re: [HACKERS] Rectifying wrong Date outputs

2011-03-21 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Mon, Mar 21, 2011 at 6:24 AM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: Having said that, it's not entirely clear to me what sane behavior is here.  Personally I would expect that an n-Ys format spec would consume at most n

Re: [HACKERS] Rectifying wrong Date outputs

2011-03-21 Thread Robert Haas
On Mon, Mar 21, 2011 at 9:57 AM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: On Mon, Mar 21, 2011 at 6:24 AM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: Having said that, it's not entirely clear to me what sane behavior is here.  Personally

Re: [HACKERS] Rectifying wrong Date outputs

2011-03-21 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Mon, Mar 21, 2011 at 9:57 AM, Tom Lane t...@sss.pgh.pa.us wrote: What I was thinking was that would take either 2 or 4 digits. Whatever you do here, the year will have to be delimited by a non-digit for such cases to be parseable. I was

Re: [HACKERS] Planner regression in 9.1: min(x) cannot use partial index with NOT NULL

2011-03-21 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Mon, Mar 21, 2011 at 7:17 AM, Marti Raudsepp ma...@juffo.org wrote: I know that the Merge Append patch required some changes in the min/max optimization, which is probably the cause. Yeah, I think this is a direct result of commit

[HACKERS] When and where do PG invoke PLs module?

2011-03-21 Thread _石头
Hello, Thank you for reading my question! Lately,I‘am reading PostgreSQL’s source code! As I understand,the PLs module(such as pl/pgSQL)allow user-defined functions to be writter in other languages. AS postgresql-9.0-document.pdf says :For a function written in a

Re: [HACKERS] 2nd Level Buffer Cache

2011-03-21 Thread rsmogura
On Mon, 21 Mar 2011 10:24:22 +, Greg Stark wrote: On Fri, Mar 18, 2011 at 11:55 PM, Josh Berkus j...@agliodbs.com wrote: To take the opposite approach... has anyone looked at having the OS just manage all caching for us? Something like MMAPed shared buffers? Even if we find the issue with

Re: [HACKERS] Planner regression in 9.1: min(x) cannot use partial index with NOT NULL

2011-03-21 Thread Robert Haas
On Mon, Mar 21, 2011 at 10:25 AM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: On Mon, Mar 21, 2011 at 7:17 AM, Marti Raudsepp ma...@juffo.org wrote: I know that the Merge Append patch required some changes in the min/max optimization, which is probably the

Re: [HACKERS] Rectifying wrong Date outputs

2011-03-21 Thread Robert Haas
On Mon, Mar 21, 2011 at 10:18 AM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: On Mon, Mar 21, 2011 at 9:57 AM, Tom Lane t...@sss.pgh.pa.us wrote: What I was thinking was that would take either 2 or 4 digits. Whatever you do here, the year will have to be

Re: [HACKERS] 2nd Level Buffer Cache

2011-03-21 Thread Merlin Moncure
On Mon, Mar 21, 2011 at 5:24 AM, Greg Stark gsst...@mit.edu wrote: On Fri, Mar 18, 2011 at 11:55 PM, Josh Berkus j...@agliodbs.com wrote: To take the opposite approach... has anyone looked at having the OS just manage all caching for us? Something like MMAPed shared buffers? Even if we find

Re: [HACKERS] 2nd Level Buffer Cache

2011-03-21 Thread Heikki Linnakangas
On 21.03.2011 17:54, Merlin Moncure wrote: Can't you make just one large mapping and lock it in 8k regions? I thought the problem with mmap was not being able to detect other processes (http://www.mail-archive.com/pgsql-general@postgresql.org/msg122301.html) compatibility issues (possibly

Re: [HACKERS] Planner regression in 9.1: min(x) cannot use partial index with NOT NULL

2011-03-21 Thread Kevin Grittner
Robert Haas wrote: Tom Lane wrote: I don't think that suppressing nulls from an index this way is really very useful. Using a partial index probably eats more planner cycles than you'll save, overall. If only 1% of the table has non-NULL values in that column, maybe not. We

Re: Sync Rep and shutdown Re: [HACKERS] Sync Rep v19

2011-03-21 Thread Yeb Havinga
On 2011-03-21 02:05, Robert Haas wrote: On Sun, Mar 20, 2011 at 11:03 AM, Yeb Havingayebhavi...@gmail.com wrote: On 2011-03-20 05:44, Robert Haas wrote: Hmm, I'm not going to be able to reproduce this here, and my test setup didn't show a clear regression. I can try beating on it some more,

Re: [HACKERS] Rectifying wrong Date outputs

2011-03-21 Thread Tom Lane
of digits, not more. Yeah, I thought about that, but it seems that use case is already hopelessly broken in both PostgreSQL and Oracle, so I'm disinclined to worry about it. How so? regression=# select to_date('20110321', 'MMDD'); to_date 2011-03-21 (1 row) regression

Re: [HACKERS] 2nd Level Buffer Cache

2011-03-21 Thread Josh Berkus
On 3/21/11 3:24 AM, Greg Stark wrote: 2-level caches work well for a variety of applications. I think 2-level caches with simple heuristics like pin all the indexes is unlikely to be helpful. At least it won't optimize the average case and I think that's been proven. It might be helpful for

Re: [HACKERS] Rectifying wrong Date outputs

2011-03-21 Thread Robert Haas
-digit delimiters and so you have to take exactly the specified number of digits, not more. Yeah, I thought about that, but it seems that use case is already hopelessly broken in both PostgreSQL and Oracle, so I'm disinclined to worry about it. How so? regression=# select to_date('20110321

[HACKERS] Missing semicolon in parser's gram.y

2011-03-21 Thread Gurjeet Singh
Attached is a single-line patch to add a missing semicolon to gram.y; caught using yyextract. Regards, -- Gurjeet Singh EnterpriseDB Corporation The Enterprise PostgreSQL Company diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index d32e480..44c4fd6 100644 ---

Re: [HACKERS] Planner regression in 9.1: min(x) cannot use partial index with NOT NULL

2011-03-21 Thread Tom Lane
Kevin Grittner kevin.gritt...@wicourts.gov writes: Robert Haas wrote: Tom Lane wrote: I don't think that suppressing nulls from an index this way is really very useful. Using a partial index probably eats more planner cycles than you'll save, overall. If only 1% of the table has non-NULL

Re: Sync Rep and shutdown Re: [HACKERS] Sync Rep v19

2011-03-21 Thread Robert Haas
On Mon, Mar 21, 2011 at 12:29 PM, Yeb Havinga yebhavi...@gmail.com wrote: pgbench -i -s 50 test Two runs of pgbench -c 10 -M prepared -T 600 test with 1 sync standby - server configs etc were mailed upthread. - performance as of commit e148443ddd95cd29edf4cc1de6188eb9cee029c5 1158 and 1306

[HACKERS] Re: Planner regression in 9.1: min(x) cannot use partial index with NOT NULL

2011-03-21 Thread Greg Stark
So it's a clever hack that we used to allow the partial indexes to be used. It relied on the implicit assumption that min(x) and max(x) where the only values of x where NULL were both NULL. It would be nice if we were clever enough to support *any* strict aggregate using partial indexes on WHERE

Re: [HACKERS] Planner regression in 9.1: min(x) cannot use partial index with NOT NULL

2011-03-21 Thread Tom Lane
Greg Stark gsst...@mit.edu writes: So it's a clever hack that we used to allow the partial indexes to be used. It relied on the implicit assumption that min(x) and max(x) where the only values of x where NULL were both NULL. It would be nice if we were clever enough to support *any* strict

[HACKERS] psql \dt and table size

2011-03-21 Thread Bernd Helmle
It stroke me today again, that \dt+ isn't displaying the acurate table size for tables, since it uses pg_relation_size() till now. With having pg_table_size() since PostgreSQL 9.0 available, i believe it would be more useful to have the total acquired storage displayed, including implicit

Re: [HACKERS] Missing semicolon in parser's gram.y

2011-03-21 Thread Tom Lane
Gurjeet Singh singh.gurj...@gmail.com writes: Attached is a single-line patch to add a missing semicolon to gram.y; caught using yyextract. Done some time ago ... http://git.postgresql.org/gitweb?p=postgresql.git;a=commitdiff;h=a051ef699c3ed1f89088dd6bbc2574f13d0b20eb#patch16

Re: Sync Rep and shutdown Re: [HACKERS] Sync Rep v19

2011-03-21 Thread Yeb Havinga
On 2011-03-21 18:04, Robert Haas wrote: On Mon, Mar 21, 2011 at 12:29 PM, Yeb Havingayebhavi...@gmail.com wrote: pgbench -i -s 50 test Two runs of pgbench -c 10 -M prepared -T 600 test with 1 sync standby - server configs etc were mailed upthread. - performance as of commit

Re: [HACKERS] 2nd Level Buffer Cache

2011-03-21 Thread Greg Stark
On Mon, Mar 21, 2011 at 3:54 PM, Merlin Moncure mmonc...@gmail.com wrote: Can't you make just one large mapping and lock it in 8k regions? I thought the problem with mmap was not being able to detect other processes (http://www.mail-archive.com/pgsql-general@postgresql.org/msg122301.html)

Re: [HACKERS] 2nd Level Buffer Cache

2011-03-21 Thread Greg Stark
On Mon, Mar 21, 2011 at 4:47 PM, Josh Berkus j...@agliodbs.com wrote: You're missing my point ... Postgres already *has* a 2-level cache: shared_buffers and the FS cache.  Anything we add to that will be adding levels. I don't think those two levels are interesting -- they don't interact

[HACKERS] Re: Planner regression in 9.1: min(x) cannot use partial index with NOT NULL

2011-03-21 Thread Greg Stark
On Mon, Mar 21, 2011 at 5:40 PM, Tom Lane t...@sss.pgh.pa.us wrote: Greg Stark gsst...@mit.edu writes: So it's a clever hack that we used to allow the partial indexes to be used. It relied on the implicit assumption that min(x) and max(x) where the only values of x where NULL were both NULL.

Re: [HACKERS] tolower() identifier downcasing versus multibyte encodings

2011-03-21 Thread Francisco Figueiredo Jr.
I just received a feedback from our bug report about this problem and it seems the problem also occurred on a windows machine. http://pgfoundry.org/tracker/index.php?func=detailaid=1010988group_id=1000140atid=590 On Sat, Mar 19, 2011 at 14:13, Marko Kreen mark...@gmail.com wrote: On Sat, Mar

[HACKERS] Chinese initdb on Windows

2011-03-21 Thread Heikki Linnakangas
On windows, if you have OS locale set to Chinese (Simplified, PRC), initdb fails: X:\C:\pgsql-install\bin\initdb.exe -D data2 The files belonging to this database system will be owned by user Heikki. This user must also own the server process. The database cluster will be initialized with

Re: [HACKERS] 2nd Level Buffer Cache

2011-03-21 Thread Alvaro Herrera
Excerpts from Josh Berkus's message of lun mar 21 13:47:21 -0300 2011: We already did that, actually, when we implemented ARC: effectively gave PostgreSQL a 3-level cache. The results were not very good, although the algorithm could be at fault there. Was it really all that bad? IIRC we

Re: [HACKERS] Planner regression in 9.1: min(x) cannot use partial index with NOT NULL

2011-03-21 Thread Tom Lane
Greg Stark gsst...@mit.edu writes: On Mon, Mar 21, 2011 at 5:40 PM, Tom Lane t...@sss.pgh.pa.us wrote: For general aggregates, you have to scan the table anyway.  If an index is useful for that, it'll get picked up in the normal planning process. if I do SELECT count(col) from tab with no

Re: [HACKERS] 2nd Level Buffer Cache

2011-03-21 Thread Josh Berkus
Was it really all that bad? IIRC we replaced ARC with the current clock sweep due to patent concerns. (Maybe there were performance concerns as well, I don't remember). Yeah, that was why the patent was frustrating. Performance was poor and we were planning on replacing ARC in 8.2 anyway.

Re: [HACKERS] 2nd Level Buffer Cache

2011-03-21 Thread Merlin Moncure
On Mon, Mar 21, 2011 at 2:08 PM, Greg Stark gsst...@mit.edu wrote: On Mon, Mar 21, 2011 at 3:54 PM, Merlin Moncure mmonc...@gmail.com wrote: Can't you make just one large mapping and lock it in 8k regions? I thought the problem with mmap was not being able to detect other processes

Re: [HACKERS] Chinese initdb on Windows

2011-03-21 Thread Dave Page
On Mon, Mar 21, 2011 at 7:29 PM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: On windows, if you have OS locale set to Chinese (Simplified, PRC), initdb fails: X:\C:\pgsql-install\bin\initdb.exe -D data2 The files belonging to this database system will be owned by user

Re: [HACKERS] really lazy vacuums?

2011-03-21 Thread Jim Nasby
On Mar 16, 2011, at 7:44 PM, Robert Haas wrote: It would be really nice (for this and for other things) if we had some way of measuring the I/O saturation of the system, so that we could automatically adjust the aggressiveness of background processes accordingly. Has anyone looked at the

Re: [HACKERS] 2nd Level Buffer Cache

2011-03-21 Thread Radosław Smogura
Merlin Moncure mmonc...@gmail.com Monday 21 March 2011 20:58:16 On Mon, Mar 21, 2011 at 2:08 PM, Greg Stark gsst...@mit.edu wrote: On Mon, Mar 21, 2011 at 3:54 PM, Merlin Moncure mmonc...@gmail.com wrote: Can't you make just one large mapping and lock it in 8k regions? I thought the

Re: Sync Rep and shutdown Re: [HACKERS] Sync Rep v19

2011-03-21 Thread Yeb Havinga
On Mon, Mar 21, 2011 at 7:51 PM, Yeb Havinga yebhavi...@gmail.com wrote: On 2011-03-21 18:04, Robert Haas wrote: On Mon, Mar 21, 2011 at 12:29 PM, Yeb Havingayebhavi...@gmail.com wrote: pgbench -i -s 50 test Two runs of pgbench -c 10 -M prepared -T 600 test with 1 sync standby - server

Re: [HACKERS] Planner regression in 9.1: min(x) cannot use partial index with NOT NULL

2011-03-21 Thread Tom Lane
I wrote: Hmm. We could possibly fix this by having planagg.c do a completely separate planner run for each aggregate, wherein it actually does build the equivalent query SELECT col FROM tab WHERE existing-quals AND col IS NOT NULL ORDER BY col ASC/DESC LIMIT 1 and plan that.

Re: [HACKERS] Chinese initdb on Windows

2011-03-21 Thread Tom Lane
Heikki Linnakangas heikki.linnakan...@enterprisedb.com writes: Any objections to the 2nd attached patch, which adds the mapping of those locale names on Windows? I think the added initdb message isn't following our style guidelines --- it certainly doesn't match the adjacent existing message.

Re: [HACKERS] psql \dt and table size

2011-03-21 Thread David Fetter
+1 for fixing this behavior in 9.1. -1 for changing in 9.0, as the change in behavior mid-release will cause more confusion than the incomplete accounting does. Cheers, David. On Mon, Mar 21, 2011 at 06:44:51PM +0100, Bernd Helmle wrote: It stroke me today again, that \dt+ isn't displaying the

Re: [HACKERS] really lazy vacuums?

2011-03-21 Thread Greg Stark
On Mon, Mar 21, 2011 at 6:08 PM, Jim Nasby j...@nasby.net wrote: Has anyone looked at the overhead of measuring how long IO requests to the kernel take? If we did that not only could we get an idea of what our IO workload looked like, we could also figure out whether a block came out of

Re: [HACKERS] GSoC 2011 - Mentors? Projects?

2011-03-21 Thread Devrim GÜNDÜZ
On Mon, 2011-03-14 at 16:08 +, Dave Page wrote: Please count me in as a mentor for a GUI project. It is an open source dashboard for PostgreSQL DBAs, called ADJ Dashboard. I can't find any info on that via Google. URL please? Erdinc said that he will setup a web page for this, and

Re: [HACKERS] How to look at the Expression Trees

2011-03-21 Thread Vaibhav Kaushal
Hi, You said: ExecMakeFunctionResultNoSets is used to evaluate function calls. What are the 'functions' there? Are they the user supplied pl/PGSQL style user functions, the functions handled by fmgr or are they just another C function which make the Expression Evaluator? Regards, Vaibhav

[HACKERS] Re: Planner regression in 9.1: min(x) cannot use partial index with NOT NULL

2011-03-21 Thread Greg Stark
On Tue, Mar 22, 2011 at 4:35 AM, Tom Lane t...@sss.pgh.pa.us wrote: Reimplement planner's handling of MIN/MAX aggregate optimization (again). I'm just curious, Why is this no longer an interesting special case? --- this is an interesting special case as of 9.1 -explain (costs off) - select

Re: [HACKERS] Planner regression in 9.1: min(x) cannot use partial index with NOT NULL

2011-03-21 Thread Tom Lane
Greg Stark gsst...@mit.edu writes: On Tue, Mar 22, 2011 at 4:35 AM, Tom Lane t...@sss.pgh.pa.us wrote: Reimplement planner's handling of MIN/MAX aggregate optimization (again). I'm just curious, Why is this no longer an interesting special case? --- this is an interesting special case as of