[GENERAL] File system level copy

2012-11-14 Thread Wang, Hao
Hi I installed PostgresSQL-8.3 on my linux machine. The cluster directory is /usr/local/data and I created three databases named db1, db2, and db3. db1 is in the default tablespace 'pg_default'. db2 is in '/home/tablespace/space1/' and db3 is in '/home/tablespace/space2/'. I want to copy the cl

[GENERAL] How do query optimizers affect window functions

2012-11-14 Thread Tianyin Xu
Hi, Postgresql, I want to understand how the query optimizers affect the output of the window functions. For example, set "cpu_tuple_cost = 50" in postgresql.conf and start the server, I apply the regress test (make installcheck). The test of window function fails. Checking the diff and I found

Re: [GENERAL] Using window functions to get the unpaginated count for paginated queries

2012-11-14 Thread Albe Laurenz
Clemens Park wrote: > Recently, during a performance improvement sweep for an application at my company, one of the hotspots > that was discovered was pagination. > > In order to display the correct pagination links on the page, the pagination library we used (most > pagination libraries for that

Re: [GENERAL] Understanding streaming replication

2012-11-14 Thread Albe Laurenz
Philippe Amelant wrote: >>> So i was thinking it was just a reconnect to the sender (and I can see >>> the standby trying to reconnect in the log) >> Hmmm. I think I was too quick when I said no. >> >> If you ship the WAL archives including the "history" file to the >> standby, then the standby s

[GENERAL] File system level copy

2012-11-14 Thread Wang, Hao
Hi, I installed PostgresSQL-8.3 on my linux machine. The cluster directory is /usr/local/data and I created three databases named db1, db2, and db3. db1 is in the default tablespace 'pg_default'. db2 is in '/home/tablespace/space1/' and db3 is in '/home/tablespace/space2/'. I want to copy the c

Re: [GENERAL] How do query optimizers affect window functions

2012-11-14 Thread Igor Romanchenko
On Wed, Nov 14, 2012 at 10:12 AM, Tianyin Xu wrote: > Hi, Postgresql, > > I want to understand how the query optimizers affect the output of the > window functions. > > For example, set "cpu_tuple_cost = 50" in postgresql.conf and start the > server, I apply the regress test (make installcheck).

Re: [GENERAL] Running out of memory while making a join

2012-11-14 Thread Carlos Henrique Reimer
Hi Tom, Thank you for the analyzes! No problem, there is no problem to use "select wm_nfsp.*" but as my concern is to prevent this in the future I think I should apply the fix or is there a config parameter to abend the backend if it reaches some kind of storage limit? Thank you! Reimer On Tu

Re: [GENERAL] File system level copy

2012-11-14 Thread Albe Laurenz
Hao Wang wrote: > I installed PostgresSQL-8.3 on my linux machine. > > The cluster directory is /usr/local/data and I created three databases named db1, db2, and db3. db1 is > in the default tablespace 'pg_default'. db2 is in '/home/tablespace/space1/' and db3 is in > '/home/tablespace/space2/'.

[GENERAL] pgBadger 2.2 released : Improvements and benchmarking

2012-11-14 Thread damien clochard
Paris, France - November 14th, 2012 DALIBO is proud to announce the release of version 2.2 of pgBadger, the new PostgreSQL log analyzer. pgBadger is built for speed with fully detailed reports from your PostgreSQL log files. It's a single and small Perl script that aims to replace and to outperfor

Re: [GENERAL] Understanding streaming replication

2012-11-14 Thread Shaun Thomas
On 11/13/2012 02:40 AM, Albe Laurenz wrote: The only thing I have seen is RedHat's Cluster Suite, which is commercial. I would recommend to have at least three nodes though, because the two node cluster we had was subject to spurious failovers on short quorum disk hiccups. There's also the Pa

Re: [GENERAL] Understanding streaming replication

2012-11-14 Thread Devrim GÜNDÜZ
Hi, On Tue, 2012-11-13 at 09:40 +0100, Albe Laurenz wrote: > > The only thing I have seen is RedHat's Cluster Suite, which > is commercial. Depends. It is open source, and all components are also available in CentOS and Scientific Linux, and there are companies out there who support clusters

Re: [GENERAL] general fear question about move PGDATA from one Disc to another

2012-11-14 Thread Aleksandar Lazic
Dear Craig, Am 14-11-2012 00:44, schrieb Craig Ringer: On 11/13/2012 11:26 PM, Aleksandar Lazic wrote: Dear listmembers, I need to move /var/lib/postgresql/8.4/main from the / partion to another disc. If so, you're probably using `pg_wrapper` for cluster management. Confirm that with `pg_l

Re: [GENERAL] SSDs - SandForce or not?

2012-11-14 Thread Shaun Thomas
On 11/14/2012 01:11 AM, Toby Corkindale wrote: I'm wondering which type of SSDs would be better for use with PostgreSQL. A few things: 1. While the controller may or may not have an impact, the presence of an on-board super-capacitor will have more. SSDs should be considered malignant devic

Re: [GENERAL] Running out of memory while making a join

2012-11-14 Thread Tom Lane
Carlos Henrique Reimer writes: > No problem, there is no problem to use "select wm_nfsp.*" but as my concern > is to prevent this in the future I think I should apply the fix or is there > a config parameter to abend the backend if it reaches some kind of storage > limit? You could start the post

[GENERAL] Access disk from plpython

2012-11-14 Thread Rhys A.D. Stewart
Greetings all, having a permission issue with writing a file using plpython to a local folder, changed permissions to everyone read and write and even changed the owner to postgres. but no joy, any suggestions? Regards, Rhys

Re: [GENERAL] Access disk from plpython

2012-11-14 Thread Adrian Klaver
On 11/14/2012 08:44 AM, Rhys A.D. Stewart wrote: Greetings all, having a permission issue with writing a file using plpython to a local folder, changed permissions to everyone read and write and even changed the owner to postgres. but no joy, any suggestions? What is the actual error message?

Re: [GENERAL] Access disk from plpython

2012-11-14 Thread Adrian Klaver
On 11/14/2012 08:48 AM, Rhys A.D. Stewart wrote: This is it: ERROR: IOError: [Errno 13] Permission denied: '/root/p1/me.txt' CONTEXT: Traceback (most recent call last): PL/Python anonymous code block, line 3, in t = open('/root/p1/me.txt','w') PL/Python anonymous code block CCi

Re: [GENERAL] Access disk from plpython

2012-11-14 Thread Jeff Janes
On Wed, Nov 14, 2012 at 8:44 AM, Rhys A.D. Stewart wrote: > Greetings all, > > having a permission issue with writing a file using plpython to a local > folder, changed permissions to everyone read and write and even changed the > owner to postgres. but no joy, any suggestions? plpython is a "tru

Re: [GENERAL] Access disk from plpython

2012-11-14 Thread Adrian Klaver
On 11/14/2012 09:03 AM, Jeff Janes wrote: On Wed, Nov 14, 2012 at 8:44 AM, Rhys A.D. Stewart wrote: Greetings all, having a permission issue with writing a file using plpython to a local folder, changed permissions to everyone read and write and even changed the owner to postgres. but no joy,

[GENERAL] Using Postgresql 9.2 on windows 7 and windows vista

2012-11-14 Thread D T
Hi, I am going to use PostgreSQL 9.2 with my application which runs on Windows 7/WIndows Visa 64 bit OS. Since these platforms are not officially supported by PostgreSQL, can i go ahead and use PostgreSQL on these platform? Regards D T

Re: [GENERAL] Access disk from plpython

2012-11-14 Thread Adrian Klaver
On 11/14/2012 08:56 AM, Rhys A.D. Stewart wrote: No it doesn't, I was hoping to create the file. Some testing here confirms it is saving file with postgres user permissions. I could get it to save by creating a directory owned by the postgres user in my home directory and saving to there. My

[GENERAL] word/phrase extraction & ranking

2012-11-14 Thread Marius Andreiana
Hello, From selected rows in a table, how can one extract and rank words/phrases based on how often they occur? Here's an example: http://developer.yahoo.com/search/content/V1/termExtraction.html INPUT: CREATE TABLE phrases ( idBIGSERIAL, phrase VARCHAR(1)); INSERT INTO phrases (phrase)

Re: [GENERAL] Using Postgresql 9.2 on windows 7 and windows vista

2012-11-14 Thread Raymond O'Donnell
On 14/11/2012 17:19, D T wrote: > Hi, > > I am going to use PostgreSQL 9.2 with my application which runs on > Windows 7/WIndows Visa 64 bit OS. Since these platforms are not > officially supported by PostgreSQL, can i go ahead and use PostgreSQL on > these platform? Are they not? I didn't know t

Re: [GENERAL] Using Postgresql 9.2 on windows 7 and windows vista

2012-11-14 Thread Steve Crawford
On 11/14/2012 11:13 AM, Raymond O'Donnell wrote: On 14/11/2012 17:19, D T wrote: Hi, I am going to use PostgreSQL 9.2 with my application which runs on Windows 7/WIndows Visa 64 bit OS. Since these platforms are not officially supported by PostgreSQL, can i go ahead and use PostgreSQL on these

Re: [GENERAL] How do query optimizers affect window functions

2012-11-14 Thread Jeff Janes
On Wed, Nov 14, 2012 at 12:12 AM, Tianyin Xu wrote: > Hi, Postgresql, > > I want to understand how the query optimizers affect the output of the > window functions. Use "EXPLAIN". One is an index scan, one is a bitmap scan. They return rows in a different order. .. > I don't understand why th

[GENERAL] Failed Login Attempts parameter

2012-11-14 Thread Frank Cavaliero
Hi, I've been searching the web and reviewing documentation, but I cannot find any reference to whether or not a parameter, for example, failed_login_attempts, exists in PostgreSQL that determines the number of attempts a user can make before being locked. In addition, if such a parameter or

[GENERAL] High SYS CPU - need advise

2012-11-14 Thread Vlad
Hello everyone, I'm seeking help in diagnosing / figuring out the issue that we have with our DB server: Under some (relatively non-heavy) load: 300...400 TPS, every 10-30 seconds server drops into high cpu system usage (90%+ SYSTEM across all CPUs - it's pure SYS cpu, i.e. it's not io wait, not

Re: [GENERAL] Failed Login Attempts parameter

2012-11-14 Thread Tom Lane
Frank Cavaliero writes: > I've been searching the web and reviewing documentation, but I cannot find > any reference to whether or not a parameter, for example, > failed_login_attempts, exists in PostgreSQL that determines the number of > attempts a user can make before being locked. There is

Re: [GENERAL] High SYS CPU - need advise

2012-11-14 Thread John R Pierce
On 11/14/12 1:13 PM, Vlad wrote: Postgresql 9.1.6. Postgres usually has 400-500 connected clients, most of them are idle. Database is over 1000 tables (across 5 namespaces), taking ~150Gb on disk. thats a really high client connection count for a 8 core system. I'd consider implementing a conn

Re: [GENERAL] High SYS CPU - need advise

2012-11-14 Thread Vlad
John, thanks for your feedback. While implementing connection pooling would make resources utilization more efficient, I don't think it's the root of my problem. Most of the connected clients are at IDLE. When I do select * from pg_stat_activity where current_query not like '%IDLE%'; I only see

Re: [GENERAL] Using window functions to get the unpaginated count for paginated queries

2012-11-14 Thread Clemens Park
Thanks for the reply everyone. In my case, it looks like there is no real drawback then, since what used to happen is: SELECT a,b,c FROM table WHERE clauses OFFSET x LIMIT y; followed by: SELECT COUNT(*) FROM ( SELECT a,b,c FROM table WHERE clauses ); (notice the lack of OFFSET and LIMIT)

[GENERAL] FATAL: index contains unexpected zero page at block

2012-11-14 Thread Dmitriy Tyugaev
Hi list, After planned rebooting the server dropped the database server PostgreSQL 8.4 When it start the server writes to the log: Nov 14 18:24:01 uno postgres84[24207]: [1-1] user=,db= LOG: could not bind IPv6 socket: Cannot assign requested address Nov 14 18:24:01 uno postgres84[24207]: [1-2]

Re: [GENERAL] High SYS CPU - need advise

2012-11-14 Thread John R Pierce
On 11/14/12 1:34 PM, Vlad wrote: thanks for your feedback. While implementing connection pooling would make resources utilization more efficient, I don't think it's the root of my problem. Most of the connected clients are at IDLE. When I do select * from pg_stat_activity where current_query n

Re: [GENERAL] How do query optimizers affect window functions

2012-11-14 Thread Tianyin Xu
Thanks a lot, Jeff! On Wed, Nov 14, 2012 at 11:59 AM, Jeff Janes wrote: > On Wed, Nov 14, 2012 at 12:12 AM, Tianyin Xu wrote: > > Hi, Postgresql, > > > > I want to understand how the query optimizers affect the output of the > > window functions. > > Use "EXPLAIN". > > One is an index scan, on

Re: [GENERAL] FATAL: index contains unexpected zero page at block

2012-11-14 Thread VB N
> user=,db= FATAL: index "316879235" contains unexpected zero page at block > 264 > user=,db= HINT: Please REINDEX it. > > Please tell me what can I do to recover? > Did you try re-building the index ? Re-Indexing or re-creating an new index should resolve this. Regards, VBN

Re: [GENERAL] How do query optimizers affect window functions

2012-11-14 Thread Jeff Janes
On Wed, Nov 14, 2012 at 2:16 PM, Tianyin Xu wrote: > Thanks a lot, Jeff! > > > On Wed, Nov 14, 2012 at 11:59 AM, Jeff Janes wrote: >> >> On Wed, Nov 14, 2012 at 12:12 AM, Tianyin Xu wrote: >> > Hi, Postgresql, >> > >> > I want to understand how the query optimizers affect the output of the >> >

Re: [GENERAL] FATAL: index contains unexpected zero page at block

2012-11-14 Thread Tom Lane
Dmitriy Tyugaev writes: > Nov 14 18:24:04 uno postgres84[24208]: [6-1] user=,db= LOG: redo done at > 237B/90A1DF98 > Nov 14 18:24:04 uno postgres84[24208]: [7-1] user=,db= LOG: last completed > transaction was at log time 2012-11-10 10:26:28.484922+04 > Nov 14 18:24:04 uno postgres84[24208]: [8-

Re: [GENERAL] How do query optimizers affect window functions

2012-11-14 Thread Tom Lane
Jeff Janes writes: > On Wed, Nov 14, 2012 at 2:16 PM, Tianyin Xu wrote: >> What do you mean by "refused to run"? > I mean that it could throw an error. Kind of like the way this > currently throws an error: > select b, sum(b) from foo; > ERROR: column "foo.b" must appear in the GROUP BY claus

Re: [GENERAL] SSDs - SandForce or not?

2012-11-14 Thread Toby Corkindale
On 15/11/12 01:42, Shaun Thomas wrote: On 11/14/2012 01:11 AM, Toby Corkindale wrote: I'm wondering which type of SSDs would be better for use with PostgreSQL. Hi Shaun, thanks for your info. I should probably have made it clear that I was curious to know how the compression stuff affected t

Re: [GENERAL] Access disk from plpython

2012-11-14 Thread Craig Ringer
On 11/15/2012 01:08 AM, Adrian Klaver wrote: > On 11/14/2012 09:03 AM, Jeff Janes wrote: >> On Wed, Nov 14, 2012 at 8:44 AM, Rhys A.D. Stewart >> wrote: >>> Greetings all, >>> >>> having a permission issue with writing a file using plpython to a local >>> folder, changed permissions to everyone re

Re: [GENERAL] Using Postgresql 9.2 on windows 7 and windows vista

2012-11-14 Thread Craig Ringer
On 11/15/2012 01:19 AM, D T wrote: > Hi, > > I am going to use PostgreSQL 9.2 with my application which runs on > Windows 7/WIndows Visa 64 bit OS. Since these platforms are not > officially supported by PostgreSQL, can i go ahead and use PostgreSQL > on these platform? Not officially supported ac

Re: [GENERAL] File system level copy

2012-11-14 Thread Wang, Hao
This is PITR, right? I don't want to use this way because I'm not allowed to change the configuration parameter of database server. I just want to use some whole DB copy to restore db3 in another machine. And I don't want to use pg_dump because I think db3 is so large that pg_dump will probably