|
we are still on 8i at work. That stuff is useful in
the future. what would you suggest for parallel extraction to from a DB to
files? I dont know perl and dont have time to learn it right now. I can do it in
C. yeah I know its easy, but Im in school too and there are so many hours in a
day.
how is unix redirection to a file for extraction?
real slow? I always get concerned about using C in a DB, since I have to
consider whether someone 2-3 years from now will be able to maintain it.
----- Original Message -----
Sent: Saturday, August 30, 2003 5:39
PM
Subject: Re: has anyone used threads to
parallelize bulk loads?
Or forget about 3GL programming altogether and
just query 9i external tables in parallel and pass the results into an INSERT
/*+ APPEND PARALLEL */? Combine it all with PL/SQL pipelined table
functions and you have parallel ETL -- �extraction� (using 9i external
tables), �transformation� (using PL/SQL pipelined table functions), and
�loading� (using INSERT APPEND) -- in a single SQL statement. Also,
don�t forget about 9i multi-table INSERT statements, conditional INSERT
statements, and of course the MERGE (a.k.a �up-sert�) statements. There
is a tremendous amount of flexibility and new features around ETL built into
Oracle these days...
I�ve been programming �C� for 20 years and there
are very few reasons to write another line of �C� code these days. I
still find reasons, but only rarely. And never related to
ETL...
on 8/30/03 1:54 PM, Ryan at [EMAIL PROTECTED]
wrote:
I find this useful when I have
large loads with alot of small to medium sized DML statements. Instead of
running them serially I use DBMS_JOB and break them up into pieces.
DBMS_JOB is limiting in that it can only do 36
at once per instance. If I want to do more, Id have to do it outside the
database. I have read about unix 'pthreads'. These appear to have the same
wait,notify,synchronize interface that java has. Im also on
Solaris.
Has anyone done anything like this? Any
features on Solaris that I can use? I know solaris has some built-in low
level C functions to handle locking(such as a mutex script). Any idea of
some sources I can use to look this up?
I dont
need this for work right now, but I may need it in the future, so I want to
play around with it. Or do I have to take this to a unix sys admin forum? If
so anyone know any good ones?
Threading
interfaces seem to be very similiar across platforms. In oracle you achieve
the same thing with dbms_job,dbms_lock,and dbms_alert.
anyone got any scripts or experience with this? This is not
essential. Im just playing around.
|
