Re: [PERFORM] Performance die when COPYing to table with bigint PK

2011-08-09 Thread Robert Ayrapetyan
Yes, you are right. Performance become even more awful. Can some techniques from pg_bulkload be implemented in postgres core? Current performance is not suitable for any enterprise-wide production system. 2011/8/5 Віталій Тимчишин : > > In my tests it greatly depends on if index writes are random

Re: [PERFORM] Performance die when COPYing to table with bigint PK

2011-08-05 Thread Vitalii Tymchyshyn
05.08.11 11:44, Robert Ayrapetyan написав(ла): Yes, you are right. Performance become even more awful. Can some techniques from pg_bulkload be implemented in postgres core? Current performance is not suitable for any enterprise-wide production system. BTW: I was thinking this morning about index

Re: [PERFORM] Performance die when COPYing to table with bigint PK

2011-08-05 Thread Віталій Тимчишин
In my tests it greatly depends on if index writes are random or sequential. My test time goes down from few hours to seconds if I add to the end of index. As for me, best comparision would be to make two equal int4 columns with same data as in int8, two indexes, then perform the test. My bet it wil

Re: [PERFORM] Performance die when COPYing to table with bigint PK

2011-08-04 Thread Robert Ayrapetyan
All you are saying disproves following: in experiment I replaces bigint index: CREATE INDEX ix_t_big ON test.t USING btree (id_big) TABLESPACE tblsp_ix; with 4 (!) other indexes: CREATE INDEX ix_t2 ON test.t USING btree (ip) TABLESPACE tblsp_ix; CREATE INDEX ix_t3 ON test.t USING btree (id_smal

Re: [PERFORM] Performance die when COPYing to table with bigint PK

2011-08-04 Thread Robert Ayrapetyan
If you look at the rest of my mail - you would notice 50 times difference in performance. What you would say? On Thu, Aug 4, 2011 at 7:11 PM, Vitalii Tymchyshyn wrote: > 04.08.11 18:59, Kevin Grittner написав(ла): >> >> Robert Ayrapetyan  wrote: >>> >>> Kevin Grittner  wrote: >> >>> [regarding te

Re: [PERFORM] Performance die when COPYing to table with bigint PK

2011-08-04 Thread Robert Ayrapetyan
Hi. Timings for your test: foo=# create table bi (big bigint not null, medium int not null); CREATE TABLE foo=# insert into bi with x(n) as (select generate_series(1, 100)) foo-# select n + 50, n from x; INSERT 0 100 foo=# \timing on Timing is on. foo=# truncate table bi; insert i

Re: [PERFORM] Performance die when COPYing to table with bigint PK

2011-08-04 Thread Kevin Grittner
Robert Ayrapetyan wrote: > If you look at the rest of my mail - you would notice 50 times > difference in performance. > What you would say? That accessing a page from RAM is more than 50 times as fast as a random access of that page from disk. -Kevin -- Sent via pgsql-performance mailing

Re: [PERFORM] Performance die when COPYing to table with bigint PK

2011-08-04 Thread Vitalii Tymchyshyn
04.08.11 18:59, Kevin Grittner написав(ла): Robert Ayrapetyan wrote: Kevin Grittner wrote: [regarding tests which do show the problem] tried same with 2 columns (bigint and int) - it didn't produced such effect probably because data volume has critical effect. Based on what you're showing

Re: [PERFORM] Performance die when COPYing to table with bigint PK

2011-08-04 Thread Kevin Grittner
Robert Ayrapetyan wrote: > Kevin Grittner wrote: >> What timings do you get for the insert statements if you run the >> following in your environment? >> Here's what I get: >> >> Time: 1629.141 ms >> Time: 1638.060 ms >> Time: 1711.833 ms >> >> Time: 4151.953 ms >> Time: 4602.679 ms >> Time:

Re: [PERFORM] Performance die when COPYing to table with bigint PK

2011-08-02 Thread Kevin Grittner
Robert Ayrapetyan wrote: > So I'm still convinced - this bug relates to FreeBSD 64-bit + UFS > + bigint column index > (some of these may be superfluous, but I have no resources to > check on different platforms with different filesystems). Linux 64 bit XFS bigint column index only shows a sli

Re: [PERFORM] Performance die when COPYing to table with bigint PK

2011-08-02 Thread Robert Ayrapetyan
Seems this assumption is not right. Just created simple index on bigint column - situation with huge performance degradation repeated. Dropping this index solved COPY issues on the fly. So I'm still convinced - this bug relates to FreeBSD 64-bit + UFS + bigint column index (some of these may be sup

Re: [PERFORM] Performance die when COPYing to table with bigint PK

2011-08-02 Thread Robert Ayrapetyan
Quite possible. But anyway - I don't think performance degradation must be so huge in case of using UNIQUE indexes. On Mon, Aug 1, 2011 at 12:06 PM, Vitalii Tymchyshyn wrote: > 31.07.11 16:51, Robert Ayrapetyan написав(ла): >> >> Hello. >> >> I've found strange behavior of my pg installation (tes

Re: [PERFORM] Performance die when COPYing to table with bigint PK

2011-08-02 Thread Vitalii Tymchyshyn
02.08.11 11:26, Robert Ayrapetyan написав(ла): Seems this assumption is not right. Just created simple index on bigint column - situation with huge performance degradation repeated. Dropping this index solved COPY issues on the fly. So I'm still convinced - this bug relates to FreeBSD 64-bit + UF

Re: [PERFORM] Performance die when COPYing to table with bigint PK

2011-08-01 Thread Vitalii Tymchyshyn
31.07.11 16:51, Robert Ayrapetyan написав(ла): Hello. I've found strange behavior of my pg installation (tested both 8.4 and 9.0 - they behave same) on FreeBSD platform. In short - when some table have PK on bigint field - COPY to that table from file becomes slower and slower as table grows. Wh

Re: [PERFORM] Performance die when COPYing to table with bigint PK

2011-08-01 Thread Simon Riggs
On Sun, Jul 31, 2011 at 2:51 PM, Robert Ayrapetyan wrote: > I've found strange behavior of my pg installation (tested both 8.4 and > 9.0 - they behave same) on FreeBSD platform. > In short - when some table have PK on bigint field - COPY to that > table from file becomes slower and slower as tabl

[PERFORM] Performance die when COPYing to table with bigint PK

2011-07-31 Thread Robert Ayrapetyan
Hello. I've found strange behavior of my pg installation (tested both 8.4 and 9.0 - they behave same) on FreeBSD platform. In short - when some table have PK on bigint field - COPY to that table from file becomes slower and slower as table grows. When table reaches ~5GB - COPY of 100k records may