Re: [HACKERS] Parallel COPY FROM execution

2017-10-02 Thread Daniel Gustafsson
> On 11 Aug 2017, at 20:07, Robert Haas wrote: > > On Fri, Aug 11, 2017 at 9:55 AM, Alex K wrote: >> - I have used both Latch and ConditionalVariable for the same >> purpose–wait until some signal >> occurs–and for me as an end user they

Re: [HACKERS] Parallel COPY FROM execution

2017-08-11 Thread Robert Haas
On Fri, Aug 11, 2017 at 9:55 AM, Alex K wrote: > - I have used both Latch and ConditionalVariable for the same > purpose–wait until some signal > occurs–and for me as an end user they perform quite similar. I > looked into the condition_variable.c > code and it

Re: [HACKERS] Parallel COPY FROM execution

2017-08-11 Thread Alex K
Greetings pgsql-hackers, I have completed the general infrastructure for parallel COPY FROM execution, consisting of Main (master) process and multiple BGWorkers connected with master using a personal message query (shm_mq). Master process does: - Dynamic shared memory allocation with parallel

Re: [HACKERS] Parallel COPY FROM execution

2017-06-30 Thread Pavel Stehule
2017-06-30 15:45 GMT+02:00 Pavel Stehule : > > > 2017-06-30 15:42 GMT+02:00 Alex K : > >> On Fri, Jun 30, 2017 at 3:35 PM, Pavel Stehule >> wrote: >> > >> > >> > 2017-06-30 14:23 GMT+02:00 Alex K

Re: [HACKERS] Parallel COPY FROM execution

2017-06-30 Thread Pavel Stehule
2017-06-30 15:42 GMT+02:00 Alex K : > On Fri, Jun 30, 2017 at 3:35 PM, Pavel Stehule > wrote: > > > > > > 2017-06-30 14:23 GMT+02:00 Alex K : > >> > >> Thus, it results in a ~60% performance boost per each x2

Re: [HACKERS] Parallel COPY FROM execution

2017-06-30 Thread Alex K
On Fri, Jun 30, 2017 at 3:35 PM, Pavel Stehule wrote: > > > 2017-06-30 14:23 GMT+02:00 Alex K : >> >> Thus, it results in a ~60% performance boost per each x2 multiplication of >> parallel processes, which is consistent with the initial

Re: [HACKERS] Parallel COPY FROM execution

2017-06-30 Thread Pavel Stehule
2017-06-30 14:23 GMT+02:00 Alex K : > Greetings pgsql-hackers, > > I am a GSOC student this year, my initial proposal has been discussed > in the following thread > https://www.postgresql.org/message-id/flat/7179F2FD-49CE- > 4093-AE14-1B26C5DFB0DA%40gmail.com > >

[HACKERS] Parallel COPY FROM execution

2017-06-30 Thread Alex K
Greetings pgsql-hackers, I am a GSOC student this year, my initial proposal has been discussed in the following thread https://www.postgresql.org/message-id/flat/7179F2FD-49CE-4093-AE14-1B26C5DFB0DA%40gmail.com Patch with COPY FROM errors handling seems to be quite finished, so I have started