Re: [PERFORM] Postgres INSERT performance and scalability

2011-09-19 Thread Igor Chudov
On Mon, Sep 19, 2011 at 6:32 PM, Scott Marlowe wrote: > On Mon, Sep 19, 2011 at 4:11 PM, Igor Chudov wrote: > > Let's say that I want to do INSERT SELECT of 1,000 items into a table. > The > > table has some ints, varchars, TEXT and BLOB fields. > > Would the time

[PERFORM] Postgres INSERT performance and scalability

2011-09-19 Thread Igor Chudov
Let's say that I want to do INSERT SELECT of 1,000 items into a table. The table has some ints, varchars, TEXT and BLOB fields. Would the time that it takes, differ a great deal, depending on whether the table has only 100,000 or 5,000,000 records? Thanks i

Re: [PERFORM] Postgres for a "data warehouse", 5-10 TB

2011-09-13 Thread Igor Chudov
I do not need to do insert updates from many threads. I want to do it from one thread. My current MySQL architecture is that I have a table with same layout as the main one, to hold new and updated objects. When there is enough objects, I begin a big INSERT SELECT ... ON DUPLICATE KEY UPDATE and

Re: [PERFORM] Postgres for a "data warehouse", 5-10 TB

2011-09-11 Thread Igor Chudov
On Sun, Sep 11, 2011 at 6:01 PM, Stephen Frost wrote: > * Igor Chudov (ichu...@gmail.com) wrote: > > Right now I have a personal (one user) project to create a 5-10 > > Terabyte data warehouse. The largest table will consume the most space > > and will take, perhaps, 200,00

Re: [PERFORM] Postgres for a "data warehouse", 5-10 TB

2011-09-11 Thread Igor Chudov
On Sun, Sep 11, 2011 at 9:16 AM, Claudio Freire wrote: > On Sun, Sep 11, 2011 at 3:59 PM, Igor Chudov wrote: > > Well, right now, my server has twelve 7,200 RPM 2TB hard drives in a > RAID-6 > > configuration. > > They are managed by a 3WARE 9750 RAID CARD. > > >

Re: [PERFORM] Postgres for a "data warehouse", 5-10 TB

2011-09-11 Thread Igor Chudov
2011/9/11 pasman pasmaƄski > For 10 TB table and 3hours, disks should have a transfer about 1GB/s > (seqscan). > > I have 6 Gb/s disk drives, so it should be not too far, maybe 5 hours for a seqscan. i > 2011/9/11, Scott Marlowe : > > On Sun, Sep 11, 2011 at 6:35 A

Re: [PERFORM] Postgres for a "data warehouse", 5-10 TB

2011-09-11 Thread Igor Chudov
On Sun, Sep 11, 2011 at 7:52 AM, Scott Marlowe wrote: > On Sun, Sep 11, 2011 at 6:35 AM, Igor Chudov wrote: > > I have a server with about 18 TB of storage and 48 GB of RAM, and 12 > > CPU cores. > > 1 or 2 fast cores is plenty for what you're doing. I need those co

[PERFORM] Postgres for a "data warehouse", 5-10 TB

2011-09-11 Thread Igor Chudov
I have been a MySQL user for years, including owning a few multi-gigabyte databases for my websites, and using it to host algebra.com (about 12 GB database). I have had my ups and downs with MySQL. The ups were ease of use and decent performance for small databases such as algebra.com. The downs w