Re: [GENERAL] Overlapping ranges

2014-06-19 Thread Alban Hertroys
On 19 Jun 2014, at 1:47, Jason Long mailing.li...@octgsoftware.com wrote: I have a large table of access logs to an application. I want is to find all rows that overlap startdate and enddate with any other rows. The query below seems to work, but does not finish unless I specify a

[GENERAL] Insert query hangs what could be the reason

2014-06-19 Thread M Tarkeshwar Rao
Hi, Insert query hangs what could be the reason. Is there any way to find out? Any timeout feature there with query which can be set at client or server end? Regards Tarkeshwar -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes to your subscription:

Re: [GENERAL] Best backup strategy for production systems

2014-06-19 Thread Oliver
Hi, thank you very much for your reply. Ok, I've read again the official documentation about backup, slowly now ;-) Is it correct if I use same location for archiving wal files and base backups, isn't it? It will be in a different filesystem of $PGDATA. OmniPITR allows be configured without having

Re: [GENERAL] Insert query hangs what could be the reason

2014-06-19 Thread dinesh kumar
On Thu, Jun 19, 2014 at 1:59 PM, M Tarkeshwar Rao m.tarkeshwar@ericsson.com wrote: Hi, Insert query hangs what could be the reason. Is there any way to find out? Any timeout feature there with query which can be set at client or server end? It might be due to concurrent primary

Re: [GENERAL] pg_restore: custom archiver unexpected end of file , ERROR: missing data for column

2014-06-19 Thread Khangelani Gama
Adrian you might be right I have just tried to pg_restore on the same server(8.4.2), I get the same errors related to xml. Can someone advise if the following XML file as an example is in the correct state or not: Hi all I have a question, why is that pg_dump does not fail when dumping the

Re: [GENERAL] pg_restore: custom archiver unexpected end of file , ERROR: missing data for column

2014-06-19 Thread Alban Hertroys
On 19 June 2014 13:11, Khangelani Gama kg...@argility.com wrote: Adrian you might be right I have just tried to pg_restore on the same server(8.4.2), I get the same errors related to xml. Can someone advise if the following XML file as an example is in the correct state or not: Hi all I

Re: [GENERAL] pg_restore: custom archiver unexpected end of file , ERROR: missing data for column

2014-06-19 Thread Khangelani Gama
-Original Message- From: Alban Hertroys [mailto:haram...@gmail.com] Sent: Thursday, June 19, 2014 1:54 PM To: Khangelani Gama Cc: pgsql-general@postgresql.org Subject: Re: [GENERAL] pg_restore: custom archiver unexpected end of file , ERROR: missing data for column On 19 June 2014 13:11,

Re: [GENERAL] pg_restore: custom archiver unexpected end of file , ERROR: missing data for column

2014-06-19 Thread Adrian Klaver
On 06/19/2014 05:03 AM, Khangelani Gama wrote: -Original Message- From: Alban Hertroys [mailto:haram...@gmail.com] Sent: Thursday, June 19, 2014 1:54 PM To: Khangelani Gama Cc: pgsql-general@postgresql.org Subject: Re: [GENERAL] pg_restore: custom archiver unexpected end of file , ERROR:

[GENERAL] pros/cons of using synchronous commit=off - AWS in particular

2014-06-19 Thread Larry J Prikockis
so from the much-loved https://wiki.postgresql.org/wiki/Tuning_Your_PostgreSQL_Server page, we have this: PostgreSQL can only safely use a write cache if it has a battery backup. See WAL reliability for an essential introduction to this topic. No, really; go read that right now, it's vital

Re : [GENERAL] Global value/global variable?

2014-06-19 Thread Edson Richter
Yes. It's similar, but consider I'm using a connection pool, so I''ve no control on when the connections are established/destroyed.I think I need something with scope broader than session...Atenciosamente,Edson Richter-- Mensagem original --De:John

Re: [GENERAL] How to store fixed size images?

2014-06-19 Thread sunpeng
Thank you, Jeff! peng On Wed, Jun 18, 2014 at 12:15 AM, Jeff Janes jeff.ja...@gmail.com wrote: On Mon, Jun 16, 2014 at 6:10 PM, sunpeng blueva...@gmail.com wrote: We have many small size(most fixed size) images, how to store them? There are two options: 1. Store images in folders,

[GENERAL] python modul pre-import to avoid importing each time

2014-06-19 Thread Rémi Cura
Hey List, I use plpython with postgis and 2 python modules (numpy and shapely). Sadly importing such module in the plpython function is very slow (several hundreds of milliseconds). I also don't know if this overhead is applied each time the function is called in the same session. Is there a

Re: [GENERAL] Overlapping ranges

2014-06-19 Thread Steve Crawford
On 06/18/2014 04:47 PM, Jason Long wrote: I have a large table of access logs to an application. I want is to find all rows that overlap startdate and enddate with any other rows. The query below seems to work, but does not finish unless I specify a single id. select distinct a1.id from

Re: [GENERAL] How to store fixed size images?

2014-06-19 Thread Andy Colson
On 06/16/2014 08:10 PM, sunpeng wrote: We have many small size(most fixed size) images, how to store them? There are two options: 1. Store images in folders, managed by os file system, only store path in postgresql 2. Store image as bytea in postgresql How do you usually store images? Thanks!

Re: [GENERAL] pg_restore: custom archiver unexpected end of file , ERROR: missing data for column

2014-06-19 Thread Khangelani Gama
-Original Message- From: Adrian Klaver [mailto:adrian.kla...@aklaver.com] Sent: Thursday, June 19, 2014 3:34 PM To: Khangelani Gama; Alban Hertroys Cc: pgsql-general@postgresql.org Subject: Re: [GENERAL] pg_restore: custom archiver unexpected end of file , ERROR: missing data for column

Re: [GENERAL] How to store fixed size images?

2014-06-19 Thread Alan Hodgson
On Thursday, June 19, 2014 10:21:56 AM Andy Colson wrote: On 06/16/2014 08:10 PM, sunpeng wrote: We have many small size(most fixed size) images, how to store them? There are two options: 1. Store images in folders, managed by os file system, only store path in postgresql 2. Store image as

Re: [GENERAL] Best backup strategy for production systems

2014-06-19 Thread John R Pierce
On 6/19/2014 3:14 AM, Oliver wrote: About wal files and archiving of them, I must delete both manually, isn't it? There isn't any option for automatically delete wal files with a given age in the postgresql.conf, isn't it? (Away of archive_command). Do you use Linux? Could you pass me your

[GENERAL] \COPY from CSV ERROR: unterminated CSV quoted field

2014-06-19 Thread ogromm
Hi, I get the error unterminated CSV quoted field when I try to copy text with new line \. new line For example: CREATE TABLE test (text TEXT); \COPY test FROM 'test.csv' WITH DELIMITER ',' CSV HEADER; test.csv: Text some text \. more text Can any one please give me some clue how to get rid

Re: [GENERAL] max_connections reached in postgres 9.3.3

2014-06-19 Thread Vasudevan, Ramya
Merlin, Thank you for the response. On the waiting queries - When we reached 1500 connections, we had 759 connections that were in active state (116 COMMIT, 238 INSERT, UPDATE 176, 57 AUTHENTICATION, 133 BIND. These active INSERTS and UPDATES also includes the 80-90 waiting sessions (We

Re: Re : [GENERAL] Global value/global variable?

2014-06-19 Thread Edson Richter
I've ended creating a table to store database parameters, and using a subquery, I can achieve the global variable effect. Thanks for your insight, Edson. From: edsonrich...@hotmail.com To: pgsql-general@postgresql.org Subject: Re : [GENERAL] Global value/global variable? Date: Thu, 19 Jun 2014

Re: [GENERAL] max_connections reached in postgres 9.3.3

2014-06-19 Thread Kevin Grittner
Vasudevan, Ramya ramya.vasude...@classmates.com wrote: On the waiting queries - When  we reached 1500 connections, we had 759 connections that were in active state (116 COMMIT, 238 INSERT, UPDATE 176, 57 AUTHENTICATION, 133 BIND. These active INSERTS and UPDATES also includes the 80-90

Re: [GENERAL] max_connections reached in postgres 9.3.3

2014-06-19 Thread Merlin Moncure
On Thu, Jun 19, 2014 at 2:35 PM, Kevin Grittner kgri...@ymail.com wrote: Vasudevan, Ramya ramya.vasude...@classmates.com wrote: On the waiting queries - When we reached 1500 connections, we had 759 connections that were in active state (116 COMMIT, 238 INSERT, UPDATE 176, 57 AUTHENTICATION,

Re: [GENERAL] \COPY from CSV ERROR: unterminated CSV quoted field

2014-06-19 Thread Tom Lane
ogromm alex.schiller1...@web.de writes: I get the error unterminated CSV quoted field when I try to copy text with new line \. new line For example: CREATE TABLE test (text TEXT); \COPY test FROM 'test.csv' WITH DELIMITER ',' CSV HEADER; test.csv: Text some text \. more text Yeah,

Re: [GENERAL] max_connections reached in postgres 9.3.3

2014-06-19 Thread Borislav Ivanov
Hi Ramya, We experience exactly the same problem here at Bitbucket. From what I can tell the major difference between your setup and ours is that you run 9.3.3 and we run 9.2.8. Our post for the issues is at

Re: [GENERAL] Best backup strategy for production systems

2014-06-19 Thread Borislav Ivanov
If your database is relatively small, I would recommend http://www.pgbarman.org/. It does binary backup and will take care of your WAL files. The laster version of pgbarman can also take backups from a slave using pgespresso extension. Note that pgbarman runs over streaming replication protocol.

Re: [GENERAL] max_connections reached in postgres 9.3.3

2014-06-19 Thread Merlin Moncure
On Thu, Jun 19, 2014 at 5:12 PM, Borislav Ivanov biva...@atlassian.com wrote: However, most people on our team think that the number of connections is purely a symptom of the actual problem. We would love to be wrong about this. But for now we feel the high number of connections contributes for

Re: [GENERAL] max_connections reached in postgres 9.3.3

2014-06-19 Thread Vasudevan, Ramya
Hi Borislav – Thank You for the update and all the information. It does look like we are on the same boat. And I feel the same too - maxing out on max_connections is just a symptom. pgbouncer may help alleviate the problem (though in your case it didn’t) and is definitely good to have either

Re: [GENERAL] max_connections reached in postgres 9.3.3

2014-06-19 Thread Borislav Ivanov
We do record perf data. For each incident we've had the data looks about the same. Unfortunately, I can't read much into it. Besides it getting stuck on a spinlock. But why and with what? ### from perf report 53.28% postmaster postgres [.] s_lock 6.22% postmaster

Re: [GENERAL] max_connections reached in postgres 9.3.3

2014-06-19 Thread Erik van Zijst
On Thu, Jun 19, 2014 at 3:57 PM, Merlin Moncure mmonc...@gmail.com wrote: In your case user% is dominating system load. Along with the high cs this is really suggesting spinlock contention. A 'perf top' is essential for identifying the culprit. It's very possible that 9.4 will fix your