RE: [Fwd: UNIX Performance Issues]
Title: Message Rahul, Did you get a response on this? I'm not sure I fully understand the actual question - are you looking for specific commands you need to run to get the information, or advice on how to interpret it? Don't forget that you will really need to correlate many of these statistics to the Oracle pathology at the same time. This then causes a problem because your sample points will at the very least experience clock drift and become harder to compare over time. There are ways to solve it, though. Anyway, if you could elaborate a little, I can try to assist! Regards James --James MorleScale Abilities, Ltdhttp://www.scaleabilities.co.ukAuthor of "Scaling Oracle8i - Building Highly Scalable OLTP System Architectures" -Original Message-From: Mogens Nørgaard [mailto:[EMAIL PROTECTED]] Sent: 18 February 2002 22:11To: James MorleSubject: [Fwd: UNIX Performance Issues]Hi James,I've got no idea whether this is of interest or not to you, but you probably know a bit about this topic.Mogens Original Message Subject: UNIX Performance Issues Date: Thu, 14 Feb 2002 07:43:26 -0800 From: "Rahul Dandekar" [EMAIL PROTECTED] Reply-To: [EMAIL PROTECTED] Organization: Fat City Network Services, San Diego, California To: Multiple recipients of list ORACLE-L [EMAIL PROTECTED]DBAs, This might be littlebit (or completely!) UNIX related... But I am told to do the performance analysis of some 10-15 machines and generate some statistical data to find out bottlenecks and identify areas of tuning... Operating System : Solaris 2.6 I have been using sar, iostat, top... I actually plan to script these things and run these scripts at certain intervals and put the data in database (Oracle 8i) and then do the crunching... Inputs are appreciated... 1. I/O What is current I/O status. Is there a lot of I/O going on? 2. Paging Is there lot of swapping / paging happening? Which processes are getting swapped in/out continuously? Are the I/O waits due to swapping / paging or regular stuff like DB waiting to read from DB files? 3. CPU What is the CPU utulization? Which processes are using lot of CPU? 4. Memory What is the current picture of Real and Virtual Memory? What processes are using how much memory? Which processes are i n real memory and which are in virtual memory? Which processes are swapped in and out from/to real/virtual memory and how many times? 5. Network What is the percentage utilization of network pipe? What is the capacity (bandwidth) of the network device? What percentage of that bandwidth is getting used? Is the system waiting for data from outside network I/O? In short, is there any bandwidth problem with network device or network traffic. Thanks, ___ ______ ___ ___ / /\ / /\ / /\ / /\ / /\ / /::\ / /::\/ /:// /:// /:/ / /:/\:\ / /:/:| / /:// /:// /:/ / /::\ \:\ / /:/|:| / /::\ __ / /:/ ___ / /:/ /__/:/\:\_\:\ /__/::\|:| /__/:/\:\/ /\ /__/:/ / /\ /__/:/ \__\/~|::\/:/ \__\/\:\:| \__\/ \:\/:/ \ \:\ / /:/ \ \:\ | |:|::/ \__\::| \__\::/ \ \:\ /:/ \ \:\ | |:|\/ | |:| / /:/ \ \:\/:/ \ \:\ |__|:| |__|:| /__/:/ \ \::/ \ \:\ \__\| \__\| \__\/ \__\/ \__\/ -- Please see the official ORACLE-L FAQ: http://www.orafaq.com -- Author: Rahul Dandekar INET: [EMAIL PROTECTED] Fat City Network Services-- (858) 538-5051 FAX: (858) 538-5051 San Diego, California-- Public Internet access / Mailing Lists To REMOVE yourself from this mailing list, send an E-Mail message to: [EMAIL PROTECTED] (note EXACT spelling of 'ListGuru') and in the message BODY, include a line containing: UNSUB ORACLE-L (or the name of mailing list you want to be removed from). You may also send the HELP command for other information (like subscribing).
Re: [Fwd: UNIX Performance Issues]
Title: Message James, Interleaved, please find my reply +Rahul - Original Message - From: James Morle To: Multiple recipients of list ORACLE-L Sent: Tuesday, February 19, 2002 6:03 AM Subject: RE: [Fwd: UNIX Performance Issues] Rahul, Did you get a response on this? I'm not sure I fully understand the actual question - are you looking for specific commands you need to run to get the information, [Rahul] Yes. I would like to know which flags of the commonly used commands give good information. For general System stats, I use "sar -u" (same as default), for Memory / Virtual Memory I use "vmstat" and look for "r b w swap free pi po us sy id" columns. I am looking for general monitoring. And once we have this general information giving a overall picture, we could know if there is a problem and we could investigate further. I am specifically looking for IO and Network statistics. Is there any command which would give me approx IO of the system, say in last 5 minutes or current? How to get network statistics? I was littlebit confused with netstat. There are two main categories in my output : hme0 and Total. What does that mean? input hme0 output input (Total) outputpackets errs packets errs colls packets errs packets errs colls5757291 0 2447690 0 0 6071152 0 2761551 0 045 0 1 0 0 45 0 1 0 024 0 2 0 0 24 0 2 0 0 What I plan to do is to take snapshot of all these statistics at acertain frequency and put it in database.Later on I could generate reports based on this. Currently, I have a lot of "Camera"s like thistaking snapshots of my system. Others involveOracle stuff like DB Size Growth, Performance Ratios,UNIX File System usage, Replication Statistics, Growth of DB objects, a lot of monitors for application info (e.g. total # of clients, # of invoices generated per day). I generate trends based on this archival data for capacity planning and proactively anticipating chronic problems. or advice on how to interpret it? Don't forget that you will really need to correlate many of these statistics to the Oracle pathology at the same time. You said it! I want co-relation of Application Load, UNIX System Load and Database Statistics. And not just when the problem arises. So, that's what I am trying to develop. This then causes a problem because your sample points will at the very least experience clock drift and become harder to compare over time. There are ways to solve it, though. Anyway, if you could elaborate a little, I can try to assist! Regards James --James MorleScale Abilities, Ltdhttp://www.scaleabilities.co.ukAuthor of "Scaling Oracle8i - Building Highly Scalable OLTP System Architectures" -Original Message-From: Mogens Nørgaard [mailto:[EMAIL PROTECTED]] Sent: 18 February 2002 22:11To: James MorleSubject: [Fwd: UNIX Performance Issues]Hi James,I've got no idea whether this is of interest or not to you, but you probably know a bit about this topic.Mogens Original Message Subject: UNIX Performance Issues Date: Thu, 14 Feb 2002 07:43:26 -0800 From: "Rahul Dandekar" [EMAIL PROTECTED] Reply-To: [EMAIL PROTECTED] Organization: Fat City Network Services, San Diego, California To: Multiple recipients of list ORACLE-L [EMAIL PROTECTED]DBAs, This might be littlebit (or completely!) UNIX related... But I am told to do the performance analysis of some 10-15 machines and generate some statistical data to find out bottlenecks and identify areas of tuning... Operating System : Solaris 2.6 I have been using sar, iostat, top... I actually plan to script these things and run these scripts at certain intervals and put the data in database (Oracle 8i) and then do the crunching... Inputs are appreciated... 1. I/O What is current I/O status. Is there a lot of I/O going on? 2. Paging Is there lot of swapping / paging happening? Which processes are getting swapped in/out continuously? Are the I/O waits due to swapping / paging or regular stuff like DB waiting to read from DB files? 3. CPU What is the CPU utulization? Which processes are using lot of CPU? 4. Memory What is the current picture of Real and Virtual Memory? What processes are using how much memory? Which processes are i n real memory and which are in virtual memory? Which processes are swapped in and out from/to real/virtual memory and how many times? 5. Network What is the percentage utilization of network pipe? What is the capacity (bandwidth) of the network device? What percentage of that bandwidth is getting used? Is the system waiting for data from outside network I/O? In short, is there any
RE: [Fwd: UNIX Performance Issues]
Title: Message Rahul, Here's what I would do. 1) I would use "mpstat" for the processor statistics. This breaks the usage up by processor in SMP configurations. This can be useful to see the relative loading of each CPU, in particular the breakdown of kernel and user time. 2) Memory: Concentrate on Page Outs and Free Memory more than anything else. That will give you plenty of clues about memory starvation, and the relevence of your VM tuning. 3) I/O: User "sar -d". It's a bit annoying on a system with a lot of disks, because it returns a row for every device, even if no I/O occurred in the sample period. However, it makes it easier to parse. ;-) Notably, keep an eye on the Service Times (avserv?), Wait times (avwait), and the queue depth. The utilisation is a function of these (queuing theory), but you can store that too as a shortcut. You can give sar any sample period, so your 5 minute averages are no problem. 4) Network: "netstat 5" will report a row for every 5 seconds (for example), showing how many packets went in and out of each interface. Your question below is easily answered - you have two columns in your output; the first is for the named interface (hme0), the 100baseT network card. The second is a total of all cards - looks like you only have one. This total can also include the loopback interface (lo0), so look out for that. Good luck, you're doing the right thing. I've been working on some software to do just this for a couple of years. I'd love to hear how it goes! Regards James --James MorleScale Abilities, Ltdhttp://www.scaleabilities.co.ukAuthor of "Scaling Oracle8i - Building Highly Scalable OLTP System Architectures" -Original Message-From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]] On Behalf Of Rahul DandekarSent: 19 February 2002 12:59To: Multiple recipients of list ORACLE-LSubject: Re: [Fwd: UNIX Performance Issues] James, Interleaved, please find my reply +Rahul - Original Message - From: James Morle To: Multiple recipients of list ORACLE-L Sent: Tuesday, February 19, 2002 6:03 AM Subject: RE: [Fwd: UNIX Performance Issues] Rahul, Did you get a response on this? I'm not sure I fully understand the actual question - are you looking for specific commands you need to run to get the information, [Rahul] Yes. I would like to know which flags of the commonly used commands give good information. For general System stats, I use "sar -u" (same as default), for Memory / Virtual Memory I use "vmstat" and look for "r b w swap free pi po us sy id" columns. I am looking for general monitoring. And once we have this general information giving a overall picture, we could know if there is a problem and we could investigate further. I am specifically looking for IO and Network statistics. Is there any command which would give me approx IO of the system, say in last 5 minutes or current? How to get network statistics? I was littlebit confused with netstat. There are two main categories in my output : hme0 and Total. What does that mean? input hme0 output input (Total) outputpackets errs packets errs colls packets errs packets errs colls5757291 0 2447690 0 0 6071152 0 2761551 0 045 0 1 0 0 45 0 1 0 024 0 2 0 0 24 0 2 0 0 What I plan to do is to take snapshot of all these statistics at acertain frequency and put it in database.Later on I could generate reports based on this. Currently, I have a lot of "Camera"s like thistaking snapshots of my system. Others involveOracle stuff like DB Size Growth, Performance Ratios,UNIX File System usage, Replication Statistics, Growth of DB objects, a lot of monitors for application info (e.g. total # of clients, # of invoices generated per day). I generate trends based on this archival data for capacity planning and proactively anticipating chronic problems. or advice on how to interpret it? Don't forget that you will really need to correlate many of these statistics to the Oracle pathology at the same time. You said it! I want co-relation of Application Load, UNIX System Load and Database Statistics. And not just when the problem arises. So, that's what I am trying to develop. This then causes a problem because your sample points will at the very least experience clock drift and become harder to compare over time. There are ways to solve it, though. Anyway, if you could elaborate a little, I can try to assist! Regards James --James MorleScale Abilities, Ltdhttp://www.scaleabilities.co.ukAuthor of "Scaling Oracle8i - Building Highly Scalable OLTP System Architectures" -Orig
RE: [Fwd: UNIX Performance Issues]
Title: Message Hi, I was at the Sun benchmarking labs in Paris before Christmas, and they had a tool which someone on there was working on. It had a web based interface, and showed everything OS performance related that you could think of. It was also very configurable, and had lots of graphs, charts etc. One of the best thing about it was that it could record the past statistics, for trend analysis. And had good report generation tools. The problem is I didn't catch it's name, and don't know if it's released yet. Sorry. Also, take a look at "High performance oracle tuning with statspack". It has lots of scripts etc. doing the type of thing you want. Jim -Original Message-From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]]On Behalf Of James MorleSent: 19 February 2002 13:58To: Multiple recipients of list ORACLE-LSubject: RE: [Fwd: UNIX Performance Issues] Rahul, Here's what I would do. 1) I would use "mpstat" for the processor statistics. This breaks the usage up by processor in SMP configurations. This can be useful to see the relative loading of each CPU, in particular the breakdown of kernel and user time. 2) Memory: Concentrate on Page Outs and Free Memory more than anything else. That will give you plenty of clues about memory starvation, and the relevence of your VM tuning. 3) I/O: User "sar -d". It's a bit annoying on a system with a lot of disks, because it returns a row for every device, even if no I/O occurred in the sample period. However, it makes it easier to parse. ;-) Notably, keep an eye on the Service Times (avserv?), Wait times (avwait), and the queue depth. The utilisation is a function of these (queuing theory), but you can store that too as a shortcut. You can give sar any sample period, so your 5 minute averages are no problem. 4) Network: "netstat 5" will report a row for every 5 seconds (for example), showing how many packets went in and out of each interface. Your question below is easily answered - you have two columns in your output; the first is for the named interface (hme0), the 100baseT network card. The second is a total of all cards - looks like you only have one. This total can also include the loopback interface (lo0), so look out for that. Good luck, you're doing the right thing. I've been working on some software to do just this for a couple of years. I'd love to hear how it goes! Regards James --James MorleScale Abilities, Ltdhttp://www.scaleabilities.co.ukAuthor of "Scaling Oracle8i - Building Highly Scalable OLTP System Architectures" -Original Message-From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]] On Behalf Of Rahul DandekarSent: 19 February 2002 12:59To: Multiple recipients of list ORACLE-LSubject: Re: [Fwd: UNIX Performance Issues] James, Interleaved, please find my reply +Rahul - Original Message - From: James Morle To: Multiple recipients of list ORACLE-L Sent: Tuesday, February 19, 2002 6:03 AM Subject: RE: [Fwd: UNIX Performance Issues] Rahul, Did you get a response on this? I'm not sure I fully understand the actual question - are you looking for specific commands you need to run to get the information, [Rahul] Yes. I would like to know which flags of the commonly used commands give good information. For general System stats, I use "sar -u" (same as default), for Memory / Virtual Memory I use "vmstat" and look for "r b w swap free pi po us sy id" columns. I am looking for general monitoring. And once we have this general information giving a overall picture, we could know if there is a problem and we could investigate further. I am specifically looking for IO and Network statistics. Is there any command which would give me approx IO of the system, say in last 5 minutes or current? How to get network statistics? I was littlebit confused with netstat. There are two main categories in my output : hme0 and Total. What does that mean? input hme0 output input (Total) outputpackets errs packets errs colls packets errs packets errs colls5757291 0 2447690 0 0 6071152 0 2761551 0 045 0 1 0 0 45 0 1 0 024 0 2 0 0 24 0 2 0 0 What I plan to do is to take snapshot of all these statistics at acertain frequency and put it in database.Later on I could generate reports based on this. Currently, I have a lot of "Camera"s like thistaking snapshots of my system. Others involveOracle stuff like DB Size Growth, Performance Ratios,UNIX File System u
Re: [Fwd: UNIX Performance Issues]
Title: Message James, Getting interesting, isn't it? I have added my response... - Original Message - From: James Morle To: Multiple recipients of list ORACLE-L Sent: Tuesday, February 19, 2002 8:58 AM Subject: RE: [Fwd: UNIX Performance Issues] Rahul, Here's what I would do. 1) I would use "mpstat" for the processor statistics. This breaks the usage up by processor in SMP configurations. This can be useful to see the relative loading of each CPU, in particular the breakdown of kernel and user time. 2) Memory: Concentrate on Page Outs and Free Memory more than anything else. That will give you plenty of clues about memory starvation, and the relevence of your VM tuning. 3) I/O: User "sar -d". It's a bit annoying on a system with a lot of disks, because it returns a row for every device, even if no I/O occurred in the sample period. However, it makes it easier to parse. ;-) Notably, keep an eye on the Service Times (avserv?), Wait times (avwait), and the queue depth. The utilisation is a function of these (queuing theory), but you can store that too as a shortcut. You can give sar any sample period, so your 5 minute averages are no problem. How can I get the current I/O load on the system? I don't know exactly what metric I am looking for. But I want to establish some baseline metric for each machine and then hunt for spikes from the gathered data. The metric can be "I/O load on system bus in Mb/sec" (like the netstat info packets input and output). I don't want individual disk statistics. I just want a overall number, which I can snapshot. 4) Network: "netstat 5" will report a row for every 5 seconds (for example), showing how many packets went in and out of each interface. Your question below is easily answered - you have two columns in your output; the first is for the named interface (hme0), the 100baseT network card. The second is a total of all cards - looks like you only have one. This total can also include the loopback interface (lo0), so look out for that. If I have only one card then why the total and hme0 data are different (by about 10%)? Good luck, you're doing the right thing. I've been working on some software to do just this for a couple of years. I'd love to hear how it goes! +Rahul Regards James --James MorleScale Abilities, Ltdhttp://www.scaleabilities.co.ukAuthor of "Scaling Oracle8i - Building Highly Scalable OLTP System Architectures" -Original Message-From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]] On Behalf Of Rahul DandekarSent: 19 February 2002 12:59To: Multiple recipients of list ORACLE-LSubject: Re: [Fwd: UNIX Performance Issues] James, Interleaved, please find my reply +Rahul - Original Message - From: James Morle To: Multiple recipients of list ORACLE-L Sent: Tuesday, February 19, 2002 6:03 AM Subject: RE: [Fwd: UNIX Performance Issues] Rahul, Did you get a response on this? I'm not sure I fully understand the actual question - are you looking for specific commands you need to run to get the information, [Rahul] Yes. I would like to know which flags of the commonly used commands give good information. For general System stats, I use "sar -u" (same as default), for Memory / Virtual Memory I use "vmstat" and look for "r b w swap free pi po us sy id" columns. I am looking for general monitoring. And once we have this general information giving a overall picture, we could know if there is a problem and we could investigate further. I am specifically looking for IO and Network statistics. Is there any command which would give me approx IO of the system, say in last 5 minutes or current? How to get network statistics? I was littlebit confused with netstat. There are two main categories in my output : hme0 and Total. What does that mean? input hme0 output input (Total) outputpackets errs packets errs colls packets errs packets errs colls5757291 0 2447690 0 0 6071152 0 2761551 0 045 0 1 0 0 45 0 1 0 024 0 2 0 0 24 0 2 0 0 What I plan to do is to take snapshot of all these statistics at acertain frequency and put it in database.Later on I could generate reports based on this. Currently, I have a lot of "Camera"s like thistaking snapshots of my system. Others involveOracle stuff like DB Size Growth, Performance Ratios,UNIX File System usage, Replication Statistics, Growth of DB objects, a lot of monitors for application info (e.g. total # of clients
RE: [Fwd: UNIX Performance Issues]
Title: Message
Hi Rahul.
Interesting, as ever!
See below
James
--James MorleScale Abilities, Ltdhttp://www.scaleabilities.co.ukAuthor
of "Scaling Oracle8i - Building Highly Scalable OLTP System
Architectures"
-Original Message-From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED]] On Behalf Of Rahul DandekarSent:
19 February 2002 15:49To: Multiple recipients of list
ORACLE-LSubject: Re: [Fwd: UNIX Performance
Issues]
James,
Getting interesting, isn't it? I have added my
response...
- Original Message -
From:
James Morle
To: Multiple recipients of list ORACLE-L
Sent: Tuesday, February 19, 2002 8:58
AM
Subject: RE: [Fwd: UNIX Performance
Issues]
Rahul,
Here's what I would do.
1) I would use "mpstat" for the processor statistics.
This breaks the usage up by processor in SMP configurations. This can be
useful to see the relative loading of each CPU, in particular the breakdown
of kernel and user time.
2) Memory: Concentrate on Page Outs and Free Memory
more than anything else. That will give you plenty of clues about memory
starvation, and the relevence of your VM tuning.
3) I/O: User "sar -d". It's a bit annoying on a system
with a lot of disks, because it returns a row for every device, even if no
I/O occurred in the sample period. However, it makes it easier to parse. ;-)
Notably, keep an eye on the Service Times (avserv?), Wait times (avwait),
and the queue depth. The utilisation is a function of these (queuing
theory), but you can store that too as a shortcut. You can give sar any
sample period, so your 5 minute averages are no
problem.
How can I get the current I/O load on the system? I don't know
exactly what metric I am looking for.
But I want to establish some baseline metric for each machine
and then hunt for spikes from the
gathered data. The metric can be "I/O load on system bus in
Mb/sec" (like the netstat info packets
input and output). I don't want individual disk statistics. I
just want a overall number, which I
can snapshot.
I know what you're after, but it's just not going to work that way! A
network adapter is a single serial resource with a finite limit. An I/O
subsystem is an arbitrarily complex *set* ofresources with a
finitecapacity on each! For example,
if you were to just measure the aggregate I/O rate across your SAN (or
whatever), that may well return a good number. However, one disk in there
could be assuming 50% or more of the load due to hotspots. This disk would
probably be providing multi-SECOND response time, and because it's the hot
disk, will be slowing nearly everything down. Your aggregate stats would not
show this. You need per-disk, per-controller, and if you've got a very busy
system you might want to start worrying about backplane capacity. There's no
easy way to measure that one,
however.
4) Network: "netstat 5" will report a row for every 5
seconds (for example), showing how many packets went in and out of each
interface. Your question below is easily answered - you have two columns in
your output; the first is for the named interface (hme0), the 100baseT
network card. The second is a total of all cards - looks like you only have
one. This total can also include the loopback interface (lo0), so look out
for that.
If I have only one card then why the
total and hme0 data are different (by about 10%)?
I suspect it is reporting the lo0interface in the total, but not showing it individually. Check
out the options for netstat (I don't have Slowlaris in front of me right
now).
Good luck, you're doing the right thing. I've been
working on some software to do just this for a couple of years. I'd love to
hear how it goes!
+Rahul
Regards
James
--James MorleScale Abilities, Ltdhttp://www.scaleabilities.co.ukAuthor
of "Scaling Oracle8i - Building Highly Scalable OLTP System
Architectures"
-Original Message-From:
[EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]] On Behalf Of Rahul
DandekarSent: 19 February 2002 12:59To: Multiple
recipients of list ORACLE-LSubject: Re: [Fwd: UNIX Performance
Issues]
James,
Interleaved, please find my
reply
+Rahul
- Original Message -
From:
James Morle
To: Multiple recipients of list
ORACLE-L
Sent: Tuesday, February 19, 2002
6:03 AM
Subject: RE: [Fwd: UNIX Performance
Issues]
Rahul,
Did you get a response on this? I'm not sure I
fully understand the actual question - are you looking for specific
commands you need
RE: [Fwd: UNIX Performance Issues]
Title: Message Rahul, For a start go to ixora.com.au and run Steve Adams response_time_breakdown.sql script against each of your databases. Paste the contents into a document with a little commentary and you have a nice summary of any potential issues. I have automated the process and get a daily diff of the script in a report each day via email. Lets me know where my waits are occuring. If I see something terribly unusual I investigate. Another easy thing to do is implement automated Statspack snapshots during peak periods and you might spot some trouble there. This will at least qwell any fears of performance trouble for the time being (or help you spot some). SAR reports aren't a bad idea either because they are also easy to automate. - Ethan DBAs,This might be littlebit (or completely!) UNIX related... But I am toldto do the performance analysis of some 10-15 machines and generatesome statistical data to find out bottlenecks and identify areas oftuning...Operating System : Solaris 2.6I have been using sar, iostat, top...I actually plan to script these things and run these scripts at certainintervals and put the data in database (Oracle 8i) and then do thecrunching...Inputs are appreciated...1. I/O What is current I/O status. Is there a lot of I/O going on?2. Paging Is there lot of swapping / paging happening? Which processes are getting swapped in/out continuously? Are the I/O waits due to swapping / paging or regular stuff like DB waiting to read from DB files?3. CPU What is the CPU utulization? Which processes are using lot of CPU?4. Memory What is the current picture of Real and Virtual Memory? What processes are using how much memory? Which processes are in real memory and which are in virtual memory? Which processes are swapped in and out from/to real/virtual memory and how many times?5. Network What is the percentage utilization of network pipe? What is the capacity (bandwidth) of the network device? What percentage of that bandwidth is getting used? Is the system waiting for data from outside network I/O? In short, is there any bandwidth problem with network device or network traffic.Thanks, ___ ___ ___ ___ ___ / /\ / /\ / /\ / /\ / /\ / /::\ / /::\ / /:/ / /:/ / /:/ / /:/\:\ / /:/:| / /:/ / /:/ / /:/ / /::\ \:\ / /:/|:| / /::\ __ / /:/ ___ / /:/ /__/:/\:\_\:\ /__/::\|:| /__/:/\:\/ /\ /__/:/ / /\ /__/:/ \__\/~|::\/:/ \__\/\:\:| \__\/ \:\/:/ \ \:\ / /:/ \ \:\ | |:|::/ \__\::| \__\::/ \ \:\ /:/ \ \:\ | |:|\/ | |:| / /:/ \ \:\/:/ \ \:\ |__|:| |__|:| /__/:/ \ \::/ \ \:\ \__\| \__\| \__\/ \__\/ \__\/-- Please see the official ORACLE-L FAQ: http://www.orafaq.com-- Author: Rahul Dandekar INET: [EMAIL PROTECTED]Fat City Network Services -- (858) 538-5051 FAX: (858) 538-5051San Diego, California -- Public Internet access / Mailing ListsTo REMOVE yourself from this mailing list, send an E-Mail messageto: [EMAIL PROTECTED] (note EXACT spelling of 'ListGuru') and inthe message BODY, include a line containing: UNSUB ORACLE-L(or the name of mailing list you want to be removed from). You mayalso send the HELP command for other information (like subscribing).
