Re: [PERFORM] How to reduce writing on disk ? (90 gb on pgsql_tmp)

2015-06-11 Thread ben.play
Of course ! I'm an idiot ... Thank you a lot ! A question : is it possible with Postgres to change the temp_tablespace only for a session (or page) ? I have a cron which takes a lot of memory. I would like to say to PostGreSql to use this temp_tablespace only on this command and not affect my

Re: [PERFORM] How to reduce writing on disk ? (90 gb on pgsql_tmp)

2015-06-11 Thread Claudio Freire
On Thu, Jun 11, 2015 at 5:56 AM, ben.play benjamin.co...@playrion.com wrote: A question : is it possible with Postgres to change the temp_tablespace only for a session (or page) ? I have a cron which takes a lot of memory. I would like to say to PostGreSql to use this temp_tablespace only on

Re: [PERFORM] How to reduce writing on disk ? (90 gb on pgsql_tmp)

2015-06-09 Thread ben.play
Hi, Thank you a lot for your answer. I've done that (create a tablespace in another HD with POSTGRES role + put it as the main temp_tablespace in the conf). But ... my command ~# df show me that all queries use the default tablespace ... This was my commands (the directory is owned by

Re: [PERFORM] How to reduce writing on disk ? (90 gb on pgsql_tmp)

2015-06-09 Thread Claudio Freire
On Tue, Jun 9, 2015 at 12:58 PM, ben.play benjamin.co...@playrion.com wrote: Hi, Thank you a lot for your answer. I've done that (create a tablespace in another HD with POSTGRES role + put it as the main temp_tablespace in the conf). But ... my command ~# df show me that all queries use the

Re: [PERFORM] How to reduce writing on disk ? (90 gb on pgsql_tmp)

2015-06-04 Thread Tomas Vondra
On 06/04/15 02:58, Scott Marlowe wrote: On Wed, Jun 3, 2015 at 6:53 PM, Scott Marlowe scott.marl...@gmail.com wrote: On Wed, Jun 3, 2015 at 4:29 PM, Joshua D. Drake j...@commandprompt.com wrote: On 06/03/2015 03:16 PM, Tomas Vondra wrote: Cache is not free memory - it's there for a

Re: [PERFORM] How to reduce writing on disk ? (90 gb on pgsql_tmp)

2015-06-03 Thread Tomas Vondra
On 06/03/15 17:09, Scott Marlowe wrote: On Wed, Jun 3, 2015 at 8:56 AM, Tomas Vondra I don't see why you think you have less than 3GB used. The output you posted clearly shows there's only ~300MB memory free - there's 15GB shared buffers and ~45GB of page cache (file system cache).

Re: [PERFORM] How to reduce writing on disk ? (90 gb on pgsql_tmp)

2015-06-03 Thread Scott Marlowe
On Wed, Jun 3, 2015 at 1:24 PM, Tomas Vondra tomas.von...@2ndquadrant.com wrote: On 06/03/15 17:09, Scott Marlowe wrote: On Wed, Jun 3, 2015 at 8:56 AM, Tomas Vondra I don't see why you think you have less than 3GB used. The output you posted clearly shows there's only ~300MB memory

Re: [PERFORM] How to reduce writing on disk ? (90 gb on pgsql_tmp)

2015-06-03 Thread Joshua D. Drake
On 06/03/2015 03:16 PM, Tomas Vondra wrote: What is more important, though, is the amount of memory. OP reported the query writes ~95GB of temp files (and dies because of full disk, so there may be more). The on-disk format is usually more compact than the in-memory representation - for

Re: [PERFORM] How to reduce writing on disk ? (90 gb on pgsql_tmp)

2015-06-03 Thread Claudio Freire
On Wed, Jun 3, 2015 at 6:18 PM, Scott Marlowe scott.marl...@gmail.com wrote: On Wed, Jun 3, 2015 at 1:24 PM, Tomas Vondra tomas.von...@2ndquadrant.com wrote: On 06/03/15 17:09, Scott Marlowe wrote: On Wed, Jun 3, 2015 at 8:56 AM, Tomas Vondra I don't see why you think you have less than

Re: [PERFORM] How to reduce writing on disk ? (90 gb on pgsql_tmp)

2015-06-03 Thread Tomas Vondra
On 06/03/15 23:18, Scott Marlowe wrote: On Wed, Jun 3, 2015 at 1:24 PM, Tomas Vondra tomas.von...@2ndquadrant.com wrote: On 06/03/15 17:09, Scott Marlowe wrote: On Wed, Jun 3, 2015 at 8:56 AM, Tomas Vondra Well, except that 15GB of that is shared_buffers, and I wouldn't call that 'free'.

Re: [PERFORM] How to reduce writing on disk ? (90 gb on pgsql_tmp)

2015-06-03 Thread Tomas Vondra
On 06/04/15 01:54, Yves Dorfsman wrote: On 2015-06-03 16:29, Joshua D. Drake wrote: On 06/03/2015 03:16 PM, Tomas Vondra wrote: What is more important, though, is the amount of memory. OP reported the query writes ~95GB of temp files (and dies because of full disk, so there may be more).

Re: [PERFORM] How to reduce writing on disk ? (90 gb on pgsql_tmp)

2015-06-03 Thread Scott Marlowe
On Wed, Jun 3, 2015 at 6:53 PM, Scott Marlowe scott.marl...@gmail.com wrote: On Wed, Jun 3, 2015 at 4:29 PM, Joshua D. Drake j...@commandprompt.com wrote: On 06/03/2015 03:16 PM, Tomas Vondra wrote: What is more important, though, is the amount of memory. OP reported the query writes ~95GB

Re: [PERFORM] How to reduce writing on disk ? (90 gb on pgsql_tmp)

2015-06-03 Thread Scott Marlowe
On Wed, Jun 3, 2015 at 4:29 PM, Joshua D. Drake j...@commandprompt.com wrote: On 06/03/2015 03:16 PM, Tomas Vondra wrote: What is more important, though, is the amount of memory. OP reported the query writes ~95GB of temp files (and dies because of full disk, so there may be more). The

Re: [PERFORM] How to reduce writing on disk ? (90 gb on pgsql_tmp)

2015-06-03 Thread Yves Dorfsman
On 2015-06-03 16:29, Joshua D. Drake wrote: On 06/03/2015 03:16 PM, Tomas Vondra wrote: What is more important, though, is the amount of memory. OP reported the query writes ~95GB of temp files (and dies because of full disk, so there may be more). The on-disk format is usually more

Re: [PERFORM] How to reduce writing on disk ? (90 gb on pgsql_tmp)

2015-06-03 Thread ben.play
The query is (unfortunately) generated by Doctrine 2 (Symfony 2). We can’t change the query easily. This is my config : max_connections = 80 shared_buffers = 15GB work_mem = 384MB maintenance_work_mem = 1GB #temp_buffers = 8MB #temp_file_limit = -1 effective_cache_size = 44GB If I put a

Re: [PERFORM] How to reduce writing on disk ? (90 gb on pgsql_tmp)

2015-06-03 Thread Tomas Vondra
On 06/03/15 15:27, chiru r wrote: Hi Benjamin, It looks you are facing disk space issue for queries. In order to avid the disk space issue you can do the following. 1) Increase the work_mem parameter session level before executing the queries. 2) If you observe diskspace issue particular user

Re: [PERFORM] How to reduce writing on disk ? (90 gb on pgsql_tmp)

2015-06-03 Thread Tomas Vondra
On 06/03/15 16:06, ben.play wrote: The query is (unfortunately) generated by Doctrine 2 (Symfony 2). We can’t change the query easily. Well, then you'll probably have to buy more RAM, apparently. This is my config : max_connections = 80 shared_buffers = 15GB work_mem = 384MB

Re: [PERFORM] How to reduce writing on disk ? (90 gb on pgsql_tmp)

2015-06-03 Thread David G. Johnston
You should repost this directly and not through Nabble. It has wrapped your code in raw tags which the PostgreSQL mailing list software strips. On Wednesday, June 3, 2015, ben.play benjamin.co...@playrion.com wrote: Hi all, We have a big database (more than 300 Gb) and we run a lot of

Re: [PERFORM] How to reduce writing on disk ? (90 gb on pgsql_tmp)

2015-06-03 Thread chiru r
Hi Benjamin, It looks you are facing disk space issue for queries. In order to avid the disk space issue you can do the following. 1) Increase the work_mem parameter session level before executing the queries. 2) If you observe diskspace issue particular user queries,increase the work_mem

Re: [PERFORM] How to reduce writing on disk ? (90 gb on pgsql_tmp)

2015-06-03 Thread amulsul
SQLSTATE[53100]: Disk full: 7 ERROR: could not write block 1099247 of temporary file Its looks like there is no room to write temporary file, try with limiting temporary file size by setting temp_file_limit GUC. -- View this message in context:

Re: [PERFORM] How to reduce writing on disk ? (90 gb on pgsql_tmp)

2015-06-03 Thread Scott Marlowe
On Wed, Jun 3, 2015 at 8:56 AM, Tomas Vondra tomas.von...@2ndquadrant.com wrote: On 06/03/15 16:06, ben.play wrote: The query is (unfortunately) generated by Doctrine 2 (Symfony 2). We can’t change the query easily. Well, then you'll probably have to buy more RAM, apparently. This is my

Re: [PERFORM] How to reduce writing on disk ? (90 gb on pgsql_tmp)

2015-06-03 Thread Claudio Freire
On Wed, Jun 3, 2015 at 11:56 AM, Tomas Vondra tomas.von...@2ndquadrant.com wrote: On 06/03/15 16:06, ben.play wrote: The query is (unfortunately) generated by Doctrine 2 (Symfony 2). We can’t change the query easily. Well, then you'll probably have to buy more RAM, apparently. There's an

Re: [PERFORM] How to reduce writing on disk ? (90 gb on pgsql_tmp)

2015-06-03 Thread Mark Kirkwood
On 04/06/15 12:58, Scott Marlowe wrote: On Wed, Jun 3, 2015 at 6:53 PM, Scott Marlowe scott.marl...@gmail.com wrote: On Wed, Jun 3, 2015 at 4:29 PM, Joshua D. Drake j...@commandprompt.com wrote: On 06/03/2015 03:16 PM, Tomas Vondra wrote: What is more important, though, is the amount of