Re: [GENERAL] Which replication is the best for our case ?

2015-07-01 Thread John R Pierce
On 7/1/2015 3:08 AM, ben.play wrote: In fact, the cron job will : - select about 10 000 lines from a big table (100 Gb of data). 1 user has about 10 lines. - each line will be examinate by an algorithm - at the end of each line, the cron job updates a few parameters for the user (add some points

Re: [GENERAL] Which replication is the best for our case ?

2015-07-01 Thread ben.play
In fact, the cron job will : - select about 10 000 lines from a big table (100 Gb of data). 1 user has about 10 lines. - each line will be examinate by an algorithm - at the end of each line, the cron job updates a few parameters for the user (add some points for example) - Then, it inserts a

Re: [GENERAL] Which replication is the best for our case ?

2015-07-01 Thread Arthur Silva
On Wed, Jul 1, 2015 at 7:08 AM, ben.play benjamin.co...@playrion.com wrote: In fact, the cron job will : - select about 10 000 lines from a big table (100 Gb of data). 1 user has about 10 lines. - each line will be examinate by an algorithm - at the end of each line, the cron job updates a

Re: [GENERAL] Which replication is the best for our case ?

2015-06-30 Thread Arthur Silva
On Tue, Jun 30, 2015 at 1:57 PM, ben.play benjamin.co...@playrion.com wrote: Hi guys, Thank you a lot for your answers. In fact, I tried to write the easiest explanation of my problem in order to be understood... My project is developed with Symfony and Doctrine (BERK, i know ...). The

Re: [GENERAL] Which replication is the best for our case ?

2015-06-30 Thread ben.play
Hi guys, Thank you a lot for your answers. In fact, I tried to write the easiest explanation of my problem in order to be understood... My project is developed with Symfony and Doctrine (BERK, i know ...). The project has more than 2 years and Doctrine makes some bad decisions and lock all

Re: [GENERAL] Which replication is the best for our case ?

2015-06-30 Thread John R Pierce
On 6/30/2015 9:57 AM, ben.play wrote: To be more precise : We have a database with more than 400 Gb and ONE table with more than 100 Gb of data. This is huge for doctrine. When the cron runs, it writes a lot on the disks in temporary file (although we have 128 GB of Ram...). Of course, each

[GENERAL] Which replication is the best for our case ?

2015-06-29 Thread ben.play
Hi guys, We have a PG database with more than 400 GB of data. At this moment, a cron runs each ten minutes and updates about 10 000 lines with complex algorithms in PHP. Each time the cron runs, the website is almost down because some queries have to make an update on the FULL table...

Re: [GENERAL] Which replication is the best for our case ?

2015-06-29 Thread Adrian Klaver
On 06/29/2015 06:02 AM, ben.play wrote: Hi guys, We have a PG database with more than 400 GB of data. At this moment, a cron runs each ten minutes and updates about 10 000 lines with complex algorithms in PHP. Each time the cron runs, the website is almost down because some queries have to

Re: [GENERAL] Which replication is the best for our case ?

2015-06-29 Thread Adrian Klaver
On 06/29/2015 06:02 AM, ben.play wrote: Hi guys, We have a PG database with more than 400 GB of data. At this moment, a cron runs each ten minutes and updates about 10 000 lines with complex algorithms in PHP. Each time the cron runs, the website is almost down because some queries have to

Re: [GENERAL] Which replication is the best for our case ?

2015-06-29 Thread Holger.Friedrich-Fa-Trivadis
Arthur Silva wrote on Monday, June 29, 2015 5:23 PM: Therefore, I'm asking if it's possible to duplicate my main database on a slave server in order to run these cron on this second server... then, replicate these changes on the main database (master).

Re: [GENERAL] Which replication is the best for our case ?

2015-06-29 Thread Arthur Silva
On Mon, Jun 29, 2015 at 10:02 AM, ben.play benjamin.co...@playrion.com wrote: Hi guys, We have a PG database with more than 400 GB of data. At this moment, a cron runs each ten minutes and updates about 10 000 lines with complex algorithms in PHP. Each time the cron runs, the website is

Re: [GENERAL] Which replication is the best for our case ?

2015-06-29 Thread Adrian Klaver
On 06/29/2015 08:23 AM, Arthur Silva wrote: On Mon, Jun 29, 2015 at 10:02 AM, ben.play benjamin.co...@playrion.com mailto:benjamin.co...@playrion.com wrote: Hi guys, We have a PG database with more than 400 GB of data. At this moment, a cron runs each ten minutes and updates about

Re: [GENERAL] Which replication is the best for our case ?

2015-06-29 Thread Jeff Janes
On Mon, Jun 29, 2015 at 6:02 AM, ben.play benjamin.co...@playrion.com wrote: Hi guys, We have a PG database with more than 400 GB of data. At this moment, a cron runs each ten minutes and updates about 10 000 lines with complex algorithms in PHP. Each time the cron runs, the website is

Re: [GENERAL] Which replication is the best for our case ?

2015-06-29 Thread John R Pierce
On 6/29/2015 10:41 AM, Melvin Davidson wrote: I think it would help immensely if you provided details such as table_structure, indexes the actual UPDATE query and the reason all rows of the table must be updated. indeed, the whole model of massaging the entire database every 10 minutes is

Re: [GENERAL] Which replication is the best for our case ?

2015-06-29 Thread Melvin Davidson
I think it would help immensely if you provided details such as table_structure, indexes the actual UPDATE query and the reason all rows of the table must be updated. On Mon, Jun 29, 2015 at 1:15 PM, Jeff Janes jeff.ja...@gmail.com wrote: On Mon, Jun 29, 2015 at 6:02 AM, ben.play