Re: [HACKERS] GSOC'17 project introduction: Parallel COPY execution with errors handling

2017-04-10 Thread Alex K
Hi Alexander! I've missed your reply, since proposal submission deadline have passed last Monday and I didn't check hackers mailing list too frequently. (1) It seems that starting new subtransaction at step 4 is not necessary. We can just gather all error lines in one pass and at the end of

Re: [HACKERS] Parallel COPY FROM execution

2017-08-11 Thread Alex K
Greetings pgsql-hackers, I have completed the general infrastructure for parallel COPY FROM execution, consisting of Main (master) process and multiple BGWorkers connected with master using a personal message query (shm_mq). Master process does: - Dynamic shared memory allocation with parallel

[HACKERS] Parallel COPY FROM execution

2017-06-30 Thread Alex K
Greetings pgsql-hackers, I am a GSOC student this year, my initial proposal has been discussed in the following thread https://www.postgresql.org/message-id/flat/7179F2FD-49CE-4093-AE14-1B26C5DFB0DA%40gmail.com Patch with COPY FROM errors handling seems to be quite finished, so I have started

Re: [HACKERS] Parallel COPY FROM execution

2017-06-30 Thread Alex K
On Fri, Jun 30, 2017 at 3:35 PM, Pavel Stehule <pavel.steh...@gmail.com> wrote: > > > 2017-06-30 14:23 GMT+02:00 Alex K <kondratov.alek...@gmail.com>: >> >> Thus, it results in a ~60% performance boost per each x2 multiplication of >> parallel processe

Re: [HACKERS] GSOC'17 project introduction: Parallel COPY execution with errors handling

2017-06-07 Thread Alex K
Hi pgsql-hackers, Thank you again for all these replies. I have started working under this project and learnt a lot of new stuff last month, so here are some new thoughts about ERRORS handling in COPY. I decided to stick to the same thread, since it has a neutral subject. (1) One of my