2017-06-30 14:23 GMT+02:00 Alex K <kondratov.alek...@gmail.com>:

> Greetings pgsql-hackers,
> I am a GSOC student this year, my initial proposal has been discussed
> in the following thread
> https://www.postgresql.org/message-id/flat/7179F2FD-49CE-
> 4093-AE14-1B26C5DFB0DA%40gmail.com
> Patch with COPY FROM errors handling seems to be quite finished, so
> I have started thinking about parallelism in COPY FROM, which is the next
> point in my proposal.
> In order to understand are there any expensive calls in COPY, which
> can be executed in parallel, I did a small research. First, please, find
> flame graph of the most expensive copy.c calls during the 'COPY FROM file'
> attached (copy_from.svg). It reveals, that inevitably serial operations
> like
> CopyReadLine (<15%), heap_multi_insert (~15%) take less than 50% of
> time in summary, while remaining operations like heap_form_tuple and
> multiple checks inside NextCopyFrom probably can be executed well in
> parallel.
> Second, I have compared an execution time of 'COPY FROM a single large
> file (~300 MB, 50000000 lines)' vs. 'COPY FROM four equal parts of the
> original file executed in the four parallel processes'. Though it is a
> very rough test, it helps to obtain an overall estimation:
> Serial:
> real 0m56.571s
> user 0m0.005s
> sys 0m0.006s
> Parallel (x4):
> real 0m22.542s
> user 0m0.015s
> sys 0m0.018s
> Thus, it results in a ~60% performance boost per each x2 multiplication of
> parallel processes, which is consistent with the initial estimation.
the important use case is big table with lot of indexes. Did you test
similar case?



Reply via email to