On Sat, 05 Jan 2008 14:07:05 -0500 Adam Prime <[EMAIL PROTECTED]> wrote:
AP> Ted Zlatanov wrote: >> I realize much of this question belongs to database-specific domains, >> but here goes: I want RDBO insert operations to be faster. I need to >> load thousands of records per second into a database. I'm using >> PostgreSQL, and everything works correctly. The extra DB operations per >> insert are killing performance, however. I would rather optimize this >> through RDBO than hand-code it in SQL, hence the questions below. AP> I don't know the internals, but the first thing i'd ask here is are you AP> actually using the objects associated with these inserts, are you doing AP> a thousand inserts, then never actually using the objects? It seems to AP> me that if you don't need the objects, it might make more sense to batch AP> insert them into the DB outside of rose using whatever method is AP> quickest on your DB. Yes, I'm considering COPY FROM in PostgreSQL. It's not an easy problem because so many of my tables have foreign keys, so the agent that writes the batch files has to be smart. AP> I assume you're using $obj->insert, not $obj->save right? Because I have to use load_or_insert from RDBO::Helpers (10% of the data fills in extra fields on existing rows) I can't avoid the SELECT+UPDATE cycle. Referential integrity is very expensive too, and using RDBO::Cached as John suggested helped a bit, but not enough, I'm definitely using COPY FROM. Thanks Ted ------------------------------------------------------------------------- Check out the new SourceForge.net Marketplace. It's the best place to buy or sell services for just about anything Open Source. http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace _______________________________________________ Rose-db-object mailing list Rose-db-object@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/rose-db-object
