[PERFORM] Blocking every 20 sec while mass copying.

2014-07-18 Thread Benjamin Dugast
Hello, I'm working on Postgres 9.3.4 for a project. We are using Scala, Akka and JDBC to insert data in the database, we have around 25M insert to do which are basically lines from 5000 files. We issue a DELETE according to the file (mandatory) and then a COPY each 1000 lines of that file.

Re: [PERFORM] Blocking every 20 sec while mass copying.

2014-07-18 Thread Albe Laurenz
Benjamin Dugast wrote: I'm working on Postgres 9.3.4 for a project. We are using Scala, Akka and JDBC to insert data in the database, we have around 25M insert to do which are basically lines from 5000 files. We issue a DELETE according to the file (mandatory) and then a COPY each 1000

Re: [PERFORM] Blocking every 20 sec while mass copying.

2014-07-18 Thread Guillaume Cottenceau
Benjamin Dugast bdugast 'at' excilys.com writes: • fsync to off (that helped but we can't do this) not exactly your question, but maybe synchronous_commit=off is a nice enough intermediary solution for you (it may give better performances at other places too for only an affordable cost) --

Re: [PERFORM] Blocking every 20 sec while mass copying.

2014-07-18 Thread Jeff Janes
On Fri, Jul 18, 2014 at 3:52 AM, Benjamin Dugast bdug...@excilys.com wrote: Hello, I'm working on Postgres 9.3.4 for a project. We are using Scala, Akka and JDBC to insert data in the database, we have around 25M insert to do which are basically lines from 5000 files. We issue a DELETE