Re: [PERFORM] Having some problems with concurrent COPY commands

2015-10-12 Thread Shaun Thomas
On Mon, Oct 12, 2015 at 1:28 PM, Andres Freund wrote: > Any chance > you could provide profiles of such a run? This is as simple as I could make it reliably. With one copy running, the thread finishes in about 1 second. With 2, it's 1.5s each, and with all 4, it's a little over 3s for each accor

Re: [PERFORM] Having some problems with concurrent COPY commands

2015-10-12 Thread Jeff Janes
On Mon, Oct 12, 2015 at 11:17 AM, Shaun Thomas wrote: > Hi guys, > > I've been doing some design investigation and ran into an interesting snag > I didn't expect to find on 9.4 (and earlier). I wrote a quick python script > to fork multiple simultaneous COPY commands to several separate tables an

Re: [PERFORM] Having some problems with concurrent COPY commands

2015-10-12 Thread Andres Freund
Hi, On 2015-10-12 13:17:53 -0500, Shaun Thomas wrote: > It would appear I'm running into whatever issue the xloginsert_slots patch > tried to address, but not much discussion exists afterwards. That patch is merged, it's just that the number of slots is hardcoded. You can recompile postgres with

[PERFORM] Having some problems with concurrent COPY commands

2015-10-12 Thread Shaun Thomas
Hi guys, I've been doing some design investigation and ran into an interesting snag I didn't expect to find on 9.4 (and earlier). I wrote a quick python script to fork multiple simultaneous COPY commands to several separate tables and found that performance apparently degrades based on how many CO