Stefan,
If you are doing a data conversion exercise then there may be some other
things to consider:
1) Parallel loading into the staging tables - might not be significant if
this is only a small portion of the conversion effort.
2) Consider options such as NOLOGGING - if you are creating temp tables,
or even creating a final table then consider the NOLOGGING option. This
can often half a queries execution time and if your recovery to that step
failing would be "recreate the table from scratch" then you have nothing to
lose.
3) If you are doing a lot of sorting / grouping then give yourself plenty
of sort_area_size. If the machine has several GB of RAM then you don't
want sorts going out to disk once they hit a few hundred MB. Since you'll
have all of the ram to yourself there's no reason to not take all you can -
as long as you stay in physical ram.
4) Determine when an index will help your conversion processing. If the
index won't help your conversion, don't create it until you are finished.
Sometimes you'll even want to create a temporary index to help your
processing and then remove it again shortly after. Dropping indexes during
data conversion and recreating them later has often saved me a considerable
amount of time overall.
But to a degree you are right - there's no silver bullet, especially
without understanding the data volumes, types of queries, etc. There are,
however, a few settings like sort_area_size, nologging and parallel clauses
which would be set differently for your environment than a multi-user
system.
Regards,
Mark.
Stefan Jahnke
<Stefan.Jahnke To: Multiple recipients of list ORACLE-L
<[EMAIL PROTECTED]>
@bov.de> cc:
Sent by: Subject: AW: Tuning instance for batch
processing
[EMAIL PROTECTED]
om
14/02/2003
20:43
Please respond
to ORACLE-L
Mark and Dennis
Thanks for the replies. I'm actually doing a data conversion (migrating
mainframe data to Oracle). First step wil be loading all the files into a
staging db using external tables. Then, a lot of insert/update jobs will
run, and finally, we will do some queries / pl/sql stuff to check integrity
(on a logical level). Additionally, the machine will be mine for a whole
day, so I won't have to deal with side effects caused by other processes.
Looks like I should go ahead, start a test with plain vanilla
configuration,
run statspack, change some parameters or execute queries in parallel mode,
run statspack and so on. And I think all your other points are also pretty
important, like the use of bind variables, bulk inserts instead of loops
commiting single rows, distribute i/o load etc.
But, it looks like it comes down to: "There is no silver bullet, just
iterative testing" ?
Regards,
Stefan Jahnke
Consultant
BOV Aktiengesellschaft
Voice: +49 201 - 4513-298
Fax: +49 201 - 4513-149
mailto: [EMAIL PROTECTED]
Please remove nospam to contact me via email.
visit our website: http://www.bov.de
subscribe to our newsletter: http://www.bov.de/presse/newsletter.asp
Sicherheitsluecken mit IT-Security-Konzepten von BOV effizient schliessen!
Weitere Informationen unter +49 201/45 13-240 oder E-Mail an
mailto:[EMAIL PROTECTED].
Wie Sie wissen, koennen ueber das Internet versandte E-Mails leicht unter
fremden Namen erstellt oder manipuliert werden. Aus diesem Grunde bitten
wir um Verstaendnis dafuer, dass wir zu Ihrem und unserem Schutz die
rechtliche Verbindlichkeit der vorstehenden Erklaerungen und Aeusserungen
ausschliessen.
As you are probably aware, e-mails sent via the Internet can easily be
copied or manipulated by third parties. For this reason we would ask for
your understanding that, for your own protection and ours, we must decline
all legal responsibility for the validity of the statements and comments
given above.
-----Urspr�ngliche Nachricht-----
Von: Mark Richard [mailto:[EMAIL PROTECTED]]
Gesendet: Donnerstag, 13. Februar 2003 23:46
An: Multiple recipients of list ORACLE-L
Betreff: Re: Tuning instance for batch processing
Stefan,
Are you looking at tuning in terms of Oracle parameters or rewriting SQL?
I guess one of the first things to consider is: Will the server be doing
anything else at the same time? If the batch jobs are the only thing
running then you tuning should probably focus more on execution time than
execution efficiency. For example, look at things like parallel queries to
load the box and reduce overall execution time.
Other than that the usual rules apply:
1) Use trace and TKPROF to see what's happening.
2) Look at using bind variables in queries that are repeated often to
reduce the parse count.
3) Look at processing a group of records in a single statement rather than
using loops in program logic if at all possible.
4) Look at I/O - do you have the load balanced well across datafiles /
disks.
5) If running on UNIX run sar or top or something similar to determine if
the server is working hard. If not, perhaps you can run jobs in parallel.
5) Are you doing anything unnecessary (ie: executing a statement within a
loop that will return the same value each time).
Even though #5 seems a little silly it can occaisionally work a treat. I
was aked to tune a very ugly query once because a batch job was running for
2 hours. The query, whilst very ugly, ran in under 500ms. Looking at the
program logic showed it was being called about 12,000 times though and was
guaranteed to return the same value each time. Running it once and storing
the result in a variable reduced the batch to under 5 minutes.
Regards,
Mark.
Stefan Jahnke
<Stefan.Jahnke To: Multiple recipients of
list
ORACLE-L <[EMAIL PROTECTED]>
@bov.de> cc:
Sent by: Subject: Tuning instance for
batch processing
[EMAIL PROTECTED]
om
14/02/2003
01:49
Please respond
to ORACLE-L
Hi list
Does anyone know of any good white papers online (or maybe a Metalink Note
ID, I couldn't really find the right thing) about tuning an Oracle instance
for batch processing ?
Thanks,
Stefan Jahnke
Consultant
BOV Aktiengesellschaft
Voice: +49 201 - 4513-298
Fax: +49 201 - 4513-149
mailto: [EMAIL PROTECTED]
Please remove nospam to contact me via email.
visit our website: http://www.bov.de
subscribe to our newsletter: http://www.bov.de/presse/newsletter.asp
Sicherheitsluecken mit IT-Security-Konzepten von BOV effizient schliessen!
Weitere Informationen unter +49 201/45 13-240 oder E-Mail an
mailto:[EMAIL PROTECTED].
Wie Sie wissen, koennen ueber das Internet versandte E-Mails leicht unter
fremden Namen erstellt oder manipuliert werden. Aus diesem Grunde bitten
wir um Verstaendnis dafuer, dass wir zu Ihrem und unserem Schutz die
rechtliche Verbindlichkeit der vorstehenden Erklaerungen und Aeusserungen
ausschliessen.
As you are probably aware, e-mails sent via the Internet can easily be
copied or manipulated by third parties. For this reason we would ask for
your understanding that, for your own protection and ours, we must decline
all legal responsibility for the validity of the statements and comments
given above.
--
Please see the official ORACLE-L FAQ: http://www.orafaq.net
--
Author: Stefan Jahnke
INET: [EMAIL PROTECTED]
Fat City Network Services -- 858-538-5051 http://www.fatcity.com
San Diego, California -- Mailing list and web hosting services
---------------------------------------------------------------------
To REMOVE yourself from this mailing list, send an E-Mail message
to: [EMAIL PROTECTED] (note EXACT spelling of 'ListGuru') and in
the message BODY, include a line containing: UNSUB ORACLE-L
(or the name of mailing list you want to be removed from). You may
also send the HELP command for other information (like subscribing).
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<---->>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
Privileged/Confidential information may be contained in this message.
If you are not the addressee indicated in this message
(or responsible for delivery of the message to such person),
you may not copy or deliver this message to anyone.
In such case, you should destroy this message and kindly notify the sender
by reply e-mail or by telephone on (61 3) 9612-6999.
Please advise immediately if you or your employer does not consent to
Internet e-mail for messages of this kind.
Opinions, conclusions and other information in this message
that do not relate to the official business of
Transurban City Link Ltd
shall be understood as neither given nor endorsed by it.
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<---->>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>
--
Please see the official ORACLE-L FAQ: http://www.orafaq.net
--
Author: Mark Richard
INET: [EMAIL PROTECTED]
Fat City Network Services -- 858-538-5051 http://www.fatcity.com
San Diego, California -- Mailing list and web hosting services
---------------------------------------------------------------------
To REMOVE yourself from this mailing list, send an E-Mail message
to: [EMAIL PROTECTED] (note EXACT spelling of 'ListGuru') and in
the message BODY, include a line containing: UNSUB ORACLE-L
(or the name of mailing list you want to be removed from). You may
also send the HELP command for other information (like subscribing).
--
Please see the official ORACLE-L FAQ: http://www.orafaq.net
--
Author: Stefan Jahnke
INET: [EMAIL PROTECTED]
Fat City Network Services -- 858-538-5051 http://www.fatcity.com
San Diego, California -- Mailing list and web hosting services
---------------------------------------------------------------------
To REMOVE yourself from this mailing list, send an E-Mail message
to: [EMAIL PROTECTED] (note EXACT spelling of 'ListGuru') and in
the message BODY, include a line containing: UNSUB ORACLE-L
(or the name of mailing list you want to be removed from). You may
also send the HELP command for other information (like subscribing).
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<---->>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
Privileged/Confidential information may be contained in this message.
If you are not the addressee indicated in this message
(or responsible for delivery of the message to such person),
you may not copy or deliver this message to anyone.
In such case, you should destroy this message and kindly notify the sender
by reply e-mail or by telephone on (61 3) 9612-6999.
Please advise immediately if you or your employer does not consent to
Internet e-mail for messages of this kind.
Opinions, conclusions and other information in this message
that do not relate to the official business of
Transurban City Link Ltd
shall be understood as neither given nor endorsed by it.
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<---->>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
--
Please see the official ORACLE-L FAQ: http://www.orafaq.net
--
Author: Mark Richard
INET: [EMAIL PROTECTED]
Fat City Network Services -- 858-538-5051 http://www.fatcity.com
San Diego, California -- Mailing list and web hosting services
---------------------------------------------------------------------
To REMOVE yourself from this mailing list, send an E-Mail message
to: [EMAIL PROTECTED] (note EXACT spelling of 'ListGuru') and in
the message BODY, include a line containing: UNSUB ORACLE-L
(or the name of mailing list you want to be removed from). You may
also send the HELP command for other information (like subscribing).
