Re: How to improve the performance of COPY commands?

2019-07-12 Thread liyuj
Thank you for your reply. I'll try it. 在 2019/7/12 下午6:14, Ilya Kasnacheev 写道: Hello! The recommendation here is to disable WAL before ingesting data in a table. You can do that by issuing ALTER TABLE tbl NOLOGGING; After data is loaded, you should turn it back on by ALTER TABLE tbl

Re: How to improve the performance of COPY commands?

2019-07-12 Thread Ilya Kasnacheev
Hello! The recommendation here is to disable WAL before ingesting data in a table. You can do that by issuing ALTER TABLE tbl NOLOGGING; After data is loaded, you should turn it back on by ALTER TABLE tbl LOGGING. Regards, -- Ilya Kasnacheev пт, 12 июл. 2019 г. в 11:33, liyuj

Re: How to improve the performance of COPY commands?

2019-07-12 Thread liyuj
Hi, The CSV file is about 250 GB, with about 1 billion rows of data. Persistence is on and there is enough memory. It has been successfully imported, but it takes a long time. The problem at present is that the data of this large table is imported successfully, and then 50 million tables are

Re: How to improve the performance of COPY commands?

2019-07-12 Thread Majid Salimi
Unsubscribe On Fri, Jul 12, 2019 at 10:17 AM Павлухин Иван wrote: > Hi, > > Currently COPY is a mechanism designed for the fastest data load. Yes, > you can try separate your data in chunks and execute COPY in parallel. > By the way, where is your input located and what is it size in bytes >

Re: How to improve the performance of COPY commands?

2019-07-11 Thread Павлухин Иван
Hi, Currently COPY is a mechanism designed for the fastest data load. Yes, you can try separate your data in chunks and execute COPY in parallel. By the way, where is your input located and what is it size in bytes (Gb)? Is persistence enabled? Does a DataRegion have enough memory to keep all

How to improve the performance of COPY commands?

2019-07-09 Thread 18624049226
If the COPY command is used to import a large amount of data, the execution time is a little long. In the current test environment, the performance is about more than 10,000/s, so if it is 100 million data, it will take several hours. Is there a faster way to import, or is COPY working in