Re: [GENERAL] questions on (parallel) COPY and when to REINDEX

2009-08-02 Thread Janet Jacobsen
Hi, Andy. Thanks for the responses and information. Just to let you know... what we are storing in the db are the image attributes - about 40 of them - not the images. So the COPY is reading an ascii file of the image attributes. It turns out to be useful to have the image attributes handy - m

Re: [GENERAL] questions on (parallel) COPY and when to REINDEX

2009-08-02 Thread Andy Colson
On 1 Aug 2009, at 23:24, Janet Jacobsen wrote: My questions are: (2) Should I REINDEX these two tables daily after the pipeline completes? Is this what other people do in practice? No need, but as people are querying as soon as data is arriving, an analyse of the table you just copied to shou

Re: [GENERAL] questions on (parallel) COPY and when to REINDEX

2009-08-02 Thread andy
Janet Jacobsen wrote: Thanks for your reply. Responses below to answer your questions, and one follow-up question on REINDEX. Janet Both tables have one or two foreign key constraints. (2) Should I REINDEX these two tables daily after the pipeline completes? Is this what other people do in

Re: [GENERAL] questions on (parallel) COPY and when to REINDEX

2009-08-02 Thread Janet Jacobsen
Thanks for your reply. Responses below, and one follow-up question about when/how often to use analyze. Janet On 02/08/2009 05:09 a.m., Alban Hertroys wrote: On 1 Aug 2009, at 23:24, Janet Jacobsen wrote: My questions are: (2) Should I REINDEX these two tables daily after the pipeline comple

Re: [GENERAL] questions on (parallel) COPY and when to REINDEX

2009-08-02 Thread Alban Hertroys
On 1 Aug 2009, at 23:24, Janet Jacobsen wrote: Both of the big tables (now up to > 15 M rows each) have indexes on several of the columns. The indexes were created using CREATE INDEX CONCURRENTLY... Both tables have one or two foreign key constraints. Others have answered some of your questio

Re: [GENERAL] questions on (parallel) COPY and when to REINDEX

2009-08-01 Thread Tom Lane
Andy Colson writes: > I have never tried two processes copy'ing into the same table at the same > time, but I'd bet its possible. It's no different from concurrent INSERTs ... regards, tom lane -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To ma

Re: [GENERAL] questions on (parallel) COPY and when to REINDEX

2009-08-01 Thread Andy Colson
Janet Jacobsen wrote: Hi. We are running a data processing/analysis pipeline that writes about 100K records to two tables on a daily basis. The pipeline runs from about 6:00 a.m. to 10:00 a.m. Our user base is small - about five people. Each accesses the database in a different way (generally

[GENERAL] questions on (parallel) COPY and when to REINDEX

2009-08-01 Thread Janet Jacobsen
Hi. We are running a data processing/analysis pipeline that writes about 100K records to two tables on a daily basis. The pipeline runs from about 6:00 a.m. to 10:00 a.m. Our user base is small - about five people. Each accesses the database in a different way (generally using some script - eit