Re: has anyone used threads to parallelize bulk loads?

[Tim Gorman]

Title: Re: has anyone used threads to parallelize bulk loads?

Or forget about 3GL programming altogether and just query 9i external tables in parallel and pass the results into an INSERT /*+ APPEND PARALLEL */?  Combine it all with PL/SQL pipelined table functions and you have parallel ETL --  “extraction” (using 9i external tables), “transformation” (using PL/SQL pipelined table functions), and “loading” (using INSERT APPEND) -- in a single SQL statement.  Also, don’t forget about 9i multi-table INSERT statements, conditional INSERT statements, and of course the MERGE (a.k.a “up-sert”) statements.  There is a tremendous amount of flexibility and new features around ETL built into Oracle these days...

I’ve been programming “C” for 20 years and there are very few reasons to write another line of “C” code these days.  I still find reasons, but only rarely.  And never related to ETL...



on 8/30/03 1:54 PM, Ryan at [EMAIL PROTECTED] wrote:

I find this useful when I have large loads with alot of small to medium sized DML statements. Instead of running them serially I use DBMS_JOB and break them up into pieces.

DBMS_JOB is limiting in that it can only do 36 at once per instance. If I want to do more, Id have to do it outside the database. I have read about unix 'pthreads'. These appear to have the same wait,notify,synchronize interface that java has. Im also on Solaris.

Has anyone done anything like this? Any features on Solaris that I can use? I know solaris has some built-in low level C functions to handle locking(such as a mutex script). Any idea of some sources I can use to look this up?

I dont need this for work right now, but I may need it in the future, so I want to play around with it. Or do I have to take this to a unix sys admin forum? If so anyone know any good ones?

Threading interfaces seem to be very similiar across platforms. In oracle you achieve the same thing with dbms_job,dbms_lock,and dbms_alert.

anyone got any scripts or experience with this? This is not essential. Im just playing around.