HITACHI was RE: I/O Performance/bottlenecks on EMC Symmetrix
Anyone aware of the related tools for HITACHI Storage Boxes? Mucho Appreciado in advance.. -Original Message- Sent: Friday, September 14, 2001 4:39 PM To: Multiple recipients of list ORACLE-L !! Please do not post Off Topic to this List !! We use EMC Symmetrics. When I got here, the folks who set things up initially where under the impression that you will never have to worry about placement/io with a Symm. Wrong! I underwent a painstaking process of relaying out the disks according to conventions DBA normally use taking into consideration index/data/volume of read/write to physical disks. Since you are presented with logical volumes, we use sym commands on each of our hosts connected to the symm to give us a graphical layout of all the logical volumes on the physical disks. EMC has a tool to do this called Resource View. EMC also has a tool called workload analyzer to analyze activity on each fa / disk, etc. And there is also a utility called Symm Optimizer to detect hot spots and optionally, replace that data to a different physical disk. You should find out if you have any of these tools available to you. HTH Michele Armstrong -Original Message- Sent: Thursday, September 13, 2001 7:55 AM To: Multiple recipients of list ORACLE-L !! Please do not post Off Topic to this List !! Hi All, Does anybody here on the list have experience with EMC/symmetrix storage units.? We have our databases on this machine and I have a feeling the the I/O performance is not very good. I can not proof it since I do not have any experience/data/access to that machine. We do however have a very cooperative UNIX group but they also lack experience with performance on this machine. Who can give me pointers about I/O throughput that can be reached, configuration pittfalls etc.. Example: RS6000 8CPU's and 4Gb memory with storage on EMC/symmetrix. Job takes about 2 hours to complete. F50 1 CPU 1Gb memory (TEST machine) local disks. same job takes 0.5 hours to complete. Jack = De informatie verzonden in dit e-mailbericht is vertrouwelijk en is uitsluitend bestemd voor de geadresseerde. Openbaarmaking, vermenigvuldiging, verspreiding en/of verstrekking van deze informatie aan derden is, behoudens voorafgaande schriftelijke toestemming van Ernst Young, niet toegestaan. Ernst Young staat niet in voor de juiste en volledige overbrenging van de inhoud van een verzonden e-mailbericht, noch voor tijdige ontvangst daarvan. Ernst Young kan niet garanderen dat een verzonden e-mailbericht vrij is van virussen, noch dat e-mailberichten worden overgebracht zonder inbreuk of tussenkomst van onbevoegde derden. Indien bovenstaand e-mailbericht niet aan u is gericht, verzoeken wij u vriendelijk doch dringend het e-mailbericht te retourneren aan de verzender en het origineel en eventuele kopieën te verwijderen en te vernietigen. Ernst Young hanteert bij de uitoefening van haar werkzaamheden algemene voorwaarden, waarin een beperking van aansprakelijkheid is opgenomen. De algemene voorwaarden worden u op verzoek kosteloos toegezonden. = The information contained in this communication is confidential and is intended solely for the use of the individual or entity to whom it is addressed. You should not copy, disclose or distribute this communication without the authority of Ernst Young. Ernst Young is neither liable for the proper and complete transmission of the information contained in this communication nor for any delay in its receipt. Ernst Young does not guarantee that the integrity of this communication has been maintained nor that the communication is free of viruses, interceptions or interference. If you are not the intended recipient of this communication please return the communication to the sender and delete and destroy all copies. In carrying out its engagements, Ernst Young applies general terms and conditions, which contain a clause that limits its liability. A copy of these terms and conditions is available on request free of charge. = -- Please see the official ORACLE-L FAQ: http://www.orafaq.com -- Author: INET: [EMAIL PROTECTED] Fat City Network Services-- (858) 538-5051 FAX: (858) 538-5051 San Diego, California-- Public Internet access / Mailing Lists To REMOVE yourself from this mailing list, send an E-Mail message to: [EMAIL PROTECTED] (note EXACT spelling of 'ListGuru') and in the message BODY, include a line containing: UNSUB ORACLE-L (or the name of mailing list you want to be removed from). You may also send the HELP command for other information (like subscribing). -- Please see the official ORACLE-L FAQ: http://www.orafaq.com -- Author: Armstrong, Michele INET: [EMAIL
Re: I/O Performance/bottlenecks on EMC Symmetrix
!! Please do not post Off Topic to this List !! Hi Jonathan, Thx for your input. If I look at v$filestat I see many columns. Searching the documents don't give me the explanation for them. Where do I get this info or maybe you can explain. AVGIOTM seems to be the column I need (units 10ms??), correct .? Jack Jonathan Lewis [EMAIL PROTECTED]@fatcity.com on 13-09-2001 22:20:50 Please respond to [EMAIL PROTECTED] Sent by: [EMAIL PROTECTED] To: Multiple recipients of list ORACLE-L [EMAIL PROTECTED] cc:(bcc: Jack van Zanen/nlzanen1/External/MEY/NL) !! Please do not post Off Topic to this List !! I have had, and heard of, poor performance from EMC boxes before now because of the big black box principal. You might check out James Morle's book (scaling Oracle 8i) for some thoughts. From an Oracle perspective, you may find that simply checking v$filestat will demonstrate quite clearly that the average read time off disk is very poor - the last big site I went to were getting read times of worse than 100 millisecs - when EMC were claiming to offer better than 20 millisecs. If this doesn't help, there is a C program on my web-site (under a Miscellanous or Performance article on choosing a block size) which allows you to create a file, and then start emulating random Oracle-read I/Os - this should give you a quick way of testing the real response time of the black box. Jonathan Lewis http://www.jlcomp.demon.co.uk Host to The Co-Operative Oracle Users' FAQ http://www.jlcomp.demon.co.uk/faq/ind_faq.html Author of: Practical Oracle 8i: Building Efficient Databases Screen saver or Life saver: http://www.ud.com Use spare CPU to assist in cancer research. -Original Message- To: Multiple recipients of list ORACLE-L [EMAIL PROTECTED] Date: 13 September 2001 16:30 !! Please do not post Off Topic to this List !! Hi, The other workload I would say was about the same (Relatively speaking) Both machines did have idle time on the CPU (I/O intensive job). The quicker test machine was a bit stretched on memory at the time (running about 8 databases) but not too bad. The only thing I can think of is pi** poor performance (3 p's you don't want in marketing) of the symmetrix disks. One point this symmetrix is loaded with 36 Gb disks and all of them are sliced to pieces and than allocated to filesystems. In theory you can be sharing disks with other high I/O apps (E-mail system etc..). I do not however have any insight in what is where physically. Jack -- Please see the official ORACLE-L FAQ: http://www.orafaq.com -- Author: Jonathan Lewis INET: [EMAIL PROTECTED] Fat City Network Services-- (858) 538-5051 FAX: (858) 538-5051 San Diego, California-- Public Internet access / Mailing Lists To REMOVE yourself from this mailing list, send an E-Mail message to: [EMAIL PROTECTED] (note EXACT spelling of 'ListGuru') and in the message BODY, include a line containing: UNSUB ORACLE-L (or the name of mailing list you want to be removed from). You may also send the HELP command for other information (like subscribing). = De informatie verzonden in dit e-mailbericht is vertrouwelijk en is uitsluitend bestemd voor de geadresseerde. Openbaarmaking, vermenigvuldiging, verspreiding en/of verstrekking van deze informatie aan derden is, behoudens voorafgaande schriftelijke toestemming van Ernst Young, niet toegestaan. Ernst Young staat niet in voor de juiste en volledige overbrenging van de inhoud van een verzonden e-mailbericht, noch voor tijdige ontvangst daarvan. Ernst Young kan niet garanderen dat een verzonden e-mailbericht vrij is van virussen, noch dat e-mailberichten worden overgebracht zonder inbreuk of tussenkomst van onbevoegde derden. Indien bovenstaand e-mailbericht niet aan u is gericht, verzoeken wij u vriendelijk doch dringend het e-mailbericht te retourneren aan de verzender en het origineel en eventuele kopieën te verwijderen en te vernietigen. Ernst Young hanteert bij de uitoefening van haar werkzaamheden algemene voorwaarden, waarin een beperking van aansprakelijkheid is opgenomen. De algemene voorwaarden worden u op verzoek kosteloos toegezonden. = The information contained in this communication is confidential and is intended solely for the use of the individual or entity to whom it is addressed. You should not copy, disclose or distribute this communication without the authority of Ernst Young. Ernst Young is neither liable for the proper and complete transmission of the information contained in this communication nor for any delay in its receipt. Ernst Young does not guarantee that the integrity of this communication has been maintained nor that the communication is free of viruses, interceptions or interference. If you are not the
Re: I/O Performance/bottlenecks on EMC Symmetrix
!! Please do not post Off Topic to this List !! The other workload I would say was about the same (Relatively speaking) Both machines did have idle time on the CPU (I/O intensive job). What are the dominant wait events - in Oracle? It sounds like its probably I/O, but you might want to verify - if for no other reason than to have some proof to justify some EMC time and/or a Sym reorg. The quicker test machine was a bit stretched on memory at the time (running about 8 databases) but not too bad. The only thing I can think of is pi** poor performance (3 p's you don't want in marketing) of the symmetrix disks. One point this symmetrix is loaded with 36 Gb disks and all of them are sliced to pieces and than allocated to filesystems. In theory you can be sharing disks with other high I/O apps (E-mail system etc..). If this is more than just theory, the message from John Hallis most likely hit the nail on the head. I'll lay odds of 3:1 for disk/cache contention in the Sym. I do not however have any insight in what is where physically. My condolences. EMC should be able to help here though, if you don't have another way (ecc, DBtuner, etc.). They should be able to come in, look at the Sym, and tell you what is going on - in cache, against disks, etc. - and also tell you whether it should be a problem or not. There is a common thread to all these good responses you are getting - I/O tuning is still critical, in spite of any vendor propaganda to the contrary. -Don Granaman [OraSaurus - Honk if you remember UFI!] -- Please see the official ORACLE-L FAQ: http://www.orafaq.com -- Author: Don Granaman INET: [EMAIL PROTECTED] Fat City Network Services-- (858) 538-5051 FAX: (858) 538-5051 San Diego, California-- Public Internet access / Mailing Lists To REMOVE yourself from this mailing list, send an E-Mail message to: [EMAIL PROTECTED] (note EXACT spelling of 'ListGuru') and in the message BODY, include a line containing: UNSUB ORACLE-L (or the name of mailing list you want to be removed from). You may also send the HELP command for other information (like subscribing).
Re: I/O Performance/bottlenecks on EMC Symmetrix
!! Please do not post Off Topic to this List !! Hi Don, wait_events that are dominant. db_file_scattered_reads, db_file_sequential_reads, db_file_parallel_write,sort_segment_request are the dominant wait (right under SQL*Net message from client rdbms ipc message) So yes I know I have an I/O problem. It's just that I'm stuck with an EMC storage solution that I can not look into. I need evidence to go to SA/management and say that disk layout is no good or something along that line. I do know for a fact that each individual disk is 36Gb and sliced in (i believe) 4,5Gb slices. You can therefore be sharing disks with other I/O intensive apps. As for monitoring tools, I'm the lowly DBA that has no business on UNIX so I need the UNIX people to do this for me. They will help as they are always cooperative, but also very busy, so I need to show them some facts and figures. TIA Jack Don Granaman [EMAIL PROTECTED]@fatcity.com on 14-09-2001 10:20:17 Please respond to [EMAIL PROTECTED] Sent by: [EMAIL PROTECTED] To: Multiple recipients of list ORACLE-L [EMAIL PROTECTED] cc:(bcc: Jack van Zanen/nlzanen1/External/MEY/NL) !! Please do not post Off Topic to this List !! The other workload I would say was about the same (Relatively speaking) Both machines did have idle time on the CPU (I/O intensive job). What are the dominant wait events - in Oracle? It sounds like its probably I/O, but you might want to verify - if for no other reason than to have some proof to justify some EMC time and/or a Sym reorg. The quicker test machine was a bit stretched on memory at the time (running about 8 databases) but not too bad. The only thing I can think of is pi** poor performance (3 p's you don't want in marketing) of the symmetrix disks. One point this symmetrix is loaded with 36 Gb disks and all of them are sliced to pieces and than allocated to filesystems. In theory you can be sharing disks with other high I/O apps (E-mail system etc..). If this is more than just theory, the message from John Hallis most likely hit the nail on the head. I'll lay odds of 3:1 for disk/cache contention in the Sym. I do not however have any insight in what is where physically. My condolences. EMC should be able to help here though, if you don't have another way (ecc, DBtuner, etc.). They should be able to come in, look at the Sym, and tell you what is going on - in cache, against disks, etc. - and also tell you whether it should be a problem or not. There is a common thread to all these good responses you are getting - I/O tuning is still critical, in spite of any vendor propaganda to the contrary. -Don Granaman [OraSaurus - Honk if you remember UFI!] -- Please see the official ORACLE-L FAQ: http://www.orafaq.com -- Author: Don Granaman INET: [EMAIL PROTECTED] Fat City Network Services-- (858) 538-5051 FAX: (858) 538-5051 San Diego, California-- Public Internet access / Mailing Lists To REMOVE yourself from this mailing list, send an E-Mail message to: [EMAIL PROTECTED] (note EXACT spelling of 'ListGuru') and in the message BODY, include a line containing: UNSUB ORACLE-L (or the name of mailing list you want to be removed from). You may also send the HELP command for other information (like subscribing). = De informatie verzonden in dit e-mailbericht is vertrouwelijk en is uitsluitend bestemd voor de geadresseerde. Openbaarmaking, vermenigvuldiging, verspreiding en/of verstrekking van deze informatie aan derden is, behoudens voorafgaande schriftelijke toestemming van Ernst Young, niet toegestaan. Ernst Young staat niet in voor de juiste en volledige overbrenging van de inhoud van een verzonden e-mailbericht, noch voor tijdige ontvangst daarvan. Ernst Young kan niet garanderen dat een verzonden e-mailbericht vrij is van virussen, noch dat e-mailberichten worden overgebracht zonder inbreuk of tussenkomst van onbevoegde derden. Indien bovenstaand e-mailbericht niet aan u is gericht, verzoeken wij u vriendelijk doch dringend het e-mailbericht te retourneren aan de verzender en het origineel en eventuele kopieën te verwijderen en te vernietigen. Ernst Young hanteert bij de uitoefening van haar werkzaamheden algemene voorwaarden, waarin een beperking van aansprakelijkheid is opgenomen. De algemene voorwaarden worden u op verzoek kosteloos toegezonden. = The information contained in this communication is confidential and is intended solely for the use of the individual or entity to whom it is addressed. You should not copy, disclose or distribute this communication without the authority of Ernst Young. Ernst Young is neither liable for the proper and complete transmission of the information contained in this communication nor for any delay in its receipt. Ernst Young does
Re: I/O Performance/bottlenecks on EMC Symmetrix
!! Please do not post Off Topic to this List !! Hi Don, I think I can get your don't believe everything you hear list to 10no make that 14. This is not specific to any storage vendor : 8) I/O access patterns on datafiles and redo logfiles are the same, hence can co-exist without any I/O issues. 9) A logical device with 16 drives will perform exactly like 4 logical devices with 4 drives each. So always create one huge logical volume with all of your drives. 10) Even if you use Parallel Query and Database Partitioning, you can still put everything on the same large logical device. Don't worry about localized I/O isolation it is not relevant. 11) Don't worry about availability issues with the one huge logical volume, even though you will affect every database component with the failure of 1 disk drive. That's because everything is mirrored. 12) We have benchmarked this new I/O methodology on a system with 1440 drives. We did another benchmark with 2400 drives and it worked really well. 13) I/O diagnostics can be done only at the file-level, as object-level I/O diagnostics is extremely difficult if not impossible. Thus, create one huge logical volume and put all of your database components on the same logical devices to eliminate hotspots. 14) We are XXX Corporation, the gods of disks, and we have invented an 7th fibonacci series inverse convoluted extremely complex heat and pressure sensitive algorithm, that will measure the angle of the sun rays coming through the window in your datacenter and take into consideration the time date of the day, determine the gravitational pull of the moon, to manage the cache in our storage array which will eliminate all I/O bottlenecks and cure cancer automatically 7x24xforever. Don't we love our jobs Gaja --- Don Granaman [EMAIL PROTECTED] wrote: !! Please do not post Off Topic to this List !! WOW! Is the other workload on these similar when this job runs? Are you sure the problem is the Symmetrix and not something in the OS or instance configuration? Does this job spend a lot of time waiting (in Oracle) on physical I/O - or on something else? (I guess if you don't have access to the machine, you can't find out though. The ultimate tuning challenge!) If the problem is actually Symmetrix I/O, I could only hazard a guess that it might be due to RAID-5 for something inappropriate (hot redo log files?) or extreme I/O contention in the layout. As far as pointers, pitfalls, and suggestions... I really have only one: don't believe everything you hear! For example: (top 10 list) 1) With EMC, RAID-5 won't matter. (it likely still will - for write-intensive stuff) 2) With EMC, you don't want to stripe. (you might - it can still make a big difference) 3) With the cache, I/O won't ever be a bottleneck. (until cache becomes saturated or ...) 4) [Corollary to #3] Throw out all that basic I/O tuning stuff you learned (but back it up to tape first!) 5) The best layout is always SAME - stripe and mirror everything across everything. 6) The check is in the mail. 7) This won't hurt a bit. [ORA-00051] Drat! I crashed before getting to ten! Sorry, I was up all night repairing a bridge... For more serious and less evasive answers, see Gaja's paper at http://www.quest.com/whitepapers/Raid1.pdf -Don Granaman [OraSaurus - Honk if you remember UFI!] - Original Message - To: Multiple recipients of list ORACLE-L [EMAIL PROTECTED] Sent: Thursday, September 13, 2001 6:55 AM !! Please do not post Off Topic to this List !! Hi All, Does anybody here on the list have experience with EMC/symmetrix storage units.? We have our databases on this machine and I have a feeling the the I/O performance is not very good. I can not proof it since I do not have any experience/data/access to that machine. We do however have a very cooperative UNIX group but they also lack experience with performance on this machine. Who can give me pointers about I/O throughput that can be reached, configuration pittfalls etc.. Example: RS6000 8CPU's and 4Gb memory with storage on EMC/symmetrix. Job takes about 2 hours to complete. F50 1 CPU 1Gb memory (TEST machine) local disks. same job takes 0.5 hours to complete. Jack === = Gaja Krishna Vaidyanatha Director, Storage Management Products, Quest Software, Inc. Co-author - Oracle Performance Tuning 101 http://www.osborne.com/database_erp/0072131454/0072131454.shtml __ Terrorist Attacks on U.S. - How can you help? Donate cash, emergency relief information http://dailynews.yahoo.com/fc/US/Emergency_Information/ -- Please see the official ORACLE-L FAQ: http://www.orafaq.com -- Author: Gaja Krishna Vaidyanatha INET: [EMAIL PROTECTED] Fat City Network Services-- (858) 538-5051 FAX: (858) 538-5051 San Diego, California
RE: I/O Performance/bottlenecks on EMC Symmetrix
Title: RE: I/O Performance/bottlenecks on EMC Symmetrix Rules of tuning databases. There is always a bottleneck. Once you solve the bottle neck, refer to rule number 1. Do not criticize someone until you walked a mile in their shoes, that way when you criticize them, you are a mile a way and have their shoes. Christopher R. Spence Oracle DBA Phone: (978) 322-5744 Fax: (707) 885-2275 Fuelspot 73 Princeton Street North, Chelmsford 01863 -Original Message- From: Hallas John [mailto:[EMAIL PROTECTED]] Sent: Thursday, September 13, 2001 10:01 AM To: Multiple recipients of list ORACLE-L Subject: RE: I/O Performance/bottlenecks on EMC Symmetrix At one site I worked using Oracle Financials we were having serious performance problems at what seemed to us random intervals. Spent months looking at the database after the Unix boys had said that there was no way we could have I/O problems with the throughput capabililities of EMC and the Symetrix set up we had. Eventually turned out that 3 systems were sharing the same disks and the disks had not been striped. Therefore other system were causing us performance problems. If you have an EMC support contract which I think you must have you, the SA's get all the free GUI tools that allow them to look at channels and logical/physical layout. Ask them about. John -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]] Sent: 13 September 01 12:55 To: Multiple recipients of list ORACLE-L Subject: I/O Performance/bottlenecks on EMC Symmetrix !! Please do not post Off Topic to this List !! Hi All, Does anybody here on the list have experience with EMC/symmetrix storage units.? We have our databases on this machine and I have a feeling the the I/O performance is not very good. I can not proof it since I do not have any experience/data/access to that machine. We do however have a very cooperative UNIX group but they also lack experience with performance on this machine. Who can give me pointers about I/O throughput that can be reached, configuration pittfalls etc.. Example: RS6000 8CPU's and 4Gb memory with storage on EMC/symmetrix. Job takes about 2 hours to complete. F50 1 CPU 1Gb memory (TEST machine) local disks. same job takes 0.5 hours to complete. Jack = De informatie verzonden in dit e-mailbericht is vertrouwelijk en is uitsluitend bestemd voor de geadresseerde. Openbaarmaking, vermenigvuldiging, verspreiding en/of verstrekking van deze informatie aan derden is, behoudens voorafgaande schriftelijke toestemming van Ernst Young, niet toegestaan. Ernst Young staat niet in voor de juiste en volledige overbrenging van de inhoud van een verzonden e-mailbericht, noch voor tijdige ontvangst daarvan. Ernst Young kan niet garanderen dat een verzonden e-mailbericht vrij is van virussen, noch dat e-mailberichten worden overgebracht zonder inbreuk of tussenkomst van onbevoegde derden. Indien bovenstaand e-mailbericht niet aan u is gericht, verzoeken wij u vriendelijk doch dringend het e-mailbericht te retourneren aan de verzender en het origineel en eventuele kopieën te verwijderen en te vernietigen. Ernst Young hanteert bij de uitoefening van haar werkzaamheden algemene voorwaarden, waarin een beperking van aansprakelijkheid is opgenomen. De algemene voorwaarden worden u op verzoek kosteloos toegezonden. = The information contained in this communication is confidential and is intended solely for the use of the individual or entity to whom it is addressed. You should not copy, disclose or distribute this communication without the authority of Ernst Young. Ernst Young is neither liable for the proper and complete transmission of the information contained in this communication nor for any delay in its receipt. Ernst Young does not guarantee that the integrity of this communication has been maintained nor that the communication is free of viruses, interceptions or interference. If you are not the intended recipient of this communication please return the communication to the sender and delete and destroy all copies. In carrying out its engagements, Ernst Young applies general terms and conditions, which contain a clause that limits its liability. A copy of these terms and conditions is available on request free of charge. = -- Please see the official ORACLE-L FAQ: http://www.orafaq.com -- Author: INET: [EMAIL PROTECTED] Fat City Network Services -- (858) 538-5051 FAX: (858) 538-5051 San Diego, California -- Public Internet access / Mailing Lists To REMOVE yourself from this mailing list, send an E-Mail message to: [EMAIL PROTECTED] (note EXACT spelling
Re: I/O Performance/bottlenecks on EMC Symmetrix
!! Please do not post Off Topic to this List !! It seems that the I/O wait statistics, along with the increased time it takes for the job to run, is pretty good circumstantial evidence (enough for the grand jury hearing). That should be sufficient motivation to start the ball rolling to collect more detailed information about the physical layout and what is going on inside the Sym. They (management, SA, whomever) can't really expect a definitive analysis (the whole trial argument) and a proposed solution in detail (sentencing recommendation???) since you don't have the level of access necessary to get that information. If you can get very specific wait information and identify the biggest bottlenecks - which datafiles, redo logs, etc. are the worse offenders - it might help build a stronger case. I hesitate a bit on this recommendation because overly specific information of that nature at this time might lead to only a partial solution - a fix to only the most severe immediate problems - rather than to do a more comprehensive review of the physical layout in the Symmetrix. Also, a comparison of total time waited on these I/O events and job run time between the test system and the EMC system should help. You might be able to see a direct correlation between the I/O waits and the run time. The company paid a premium for EMC storage and should be getting more out of it, not less. My experience has been that EMC is actually pretty good about helping out with gathering statistics, etc. - if you can get them in. (Your mileage may vary.) If it is any help... (and anecdotal evidence rarely is) less than a year ago, I took an EMC approved, big black box layout, performed a thorough I/O analysis on the database, and rebuilt the disks in the Sym according to more conventional I/O practices - striping, dedicating redo log disks, distributing contending objects between disks/stripe_sets, etc. The after configuration throughput was eight times greater than the before layout - on identical hardware. And this was with all the disks in question (not the entire Sym though) already being dedicated entirely to a single database. This wasn't an isolated case, just the most dramatic of several. I'm sure that others, especially Gaja, have such stories also. Again, I would seriously suggest reading his white paper at http://www.quest.com/whitepapers/Raid1.pdf . It has a wealth of information on this very topic and, I believe, it has some specific examples of problem layouts,solutions, and gains. (At least the presentation did.) Incidentally, it isn't absolutely necessary to dedicate entire disks to a single database. It is usually preferred - in my opinion, but if you don't dedicate disks, you should treat the I/O tuning exercise as if everything using any particular set of disks is a single database (even if some of it isn't database!). For example, you have DB01 and DB02 sharing a set of disks. Distribute the I/O between disks as if DB01 + DB02 are a single database. Completely independent I/O tuning for DB01 and DB02 probably won't cut it. Often, you have to win the motivational/political battle before you can even really begin the technical battle. It sounds like that is probably where you are now. So, I'll refrain from pollutimg the list with more long generic discussion on this topic. Good luck! -Don Granaman [OraSaurus - Honk if you remember UFI!] - Original Message - To: Multiple recipients of list ORACLE-L [EMAIL PROTECTED] Sent: Friday, September 14, 2001 4:00 AM !! Please do not post Off Topic to this List !! Hi Don, wait_events that are dominant. db_file_scattered_reads, db_file_sequential_reads, db_file_parallel_write,sort_segment_request are the dominant wait (right under SQL*Net message from client rdbms ipc message) So yes I know I have an I/O problem. It's just that I'm stuck with an EMC storage solution that I can not look into. I need evidence to go to SA/management and say that disk layout is no good or something along that line. I do know for a fact that each individual disk is 36Gb and sliced in (i believe) 4,5Gb slices. You can therefore be sharing disks with other I/O intensive apps. As for monitoring tools, I'm the lowly DBA that has no business on UNIX so I need the UNIX people to do this for me. They will help as they are always cooperative, but also very busy, so I need to show them some facts and figures. TIA Jack -- Please see the official ORACLE-L FAQ: http://www.orafaq.com -- Author: Don Granaman INET: [EMAIL PROTECTED] Fat City Network Services-- (858) 538-5051 FAX: (858) 538-5051 San Diego, California-- Public Internet access / Mailing Lists To REMOVE yourself from this mailing list, send an E-Mail message to: [EMAIL PROTECTED] (note EXACT spelling of 'ListGuru') and in the message BODY, include a line containing: UNSUB ORACLE-L (or the name of mailing list
Re: I/O Performance/bottlenecks on EMC Symmetrix
!! Please do not post Off Topic to this List !! Gaja, I REALLY like #14 :) Rachel From: Gaja Krishna Vaidyanatha [EMAIL PROTECTED] Reply-To: [EMAIL PROTECTED] To: Multiple recipients of list ORACLE-L [EMAIL PROTECTED] Subject: Re: I/O Performance/bottlenecks on EMC Symmetrix Date: Fri, 14 Sep 2001 02:25:24 -0800 !! Please do not post Off Topic to this List !! Hi Don, I think I can get your don't believe everything you hear list to 10no make that 14. This is not specific to any storage vendor : 8) I/O access patterns on datafiles and redo logfiles are the same, hence can co-exist without any I/O issues. 9) A logical device with 16 drives will perform exactly like 4 logical devices with 4 drives each. So always create one huge logical volume with all of your drives. 10) Even if you use Parallel Query and Database Partitioning, you can still put everything on the same large logical device. Don't worry about localized I/O isolation it is not relevant. 11) Don't worry about availability issues with the one huge logical volume, even though you will affect every database component with the failure of 1 disk drive. That's because everything is mirrored. 12) We have benchmarked this new I/O methodology on a system with 1440 drives. We did another benchmark with 2400 drives and it worked really well. 13) I/O diagnostics can be done only at the file-level, as object-level I/O diagnostics is extremely difficult if not impossible. Thus, create one huge logical volume and put all of your database components on the same logical devices to eliminate hotspots. 14) We are XXX Corporation, the gods of disks, and we have invented an 7th fibonacci series inverse convoluted extremely complex heat and pressure sensitive algorithm, that will measure the angle of the sun rays coming through the window in your datacenter and take into consideration the time date of the day, determine the gravitational pull of the moon, to manage the cache in our storage array which will eliminate all I/O bottlenecks and cure cancer automatically 7x24xforever. Don't we love our jobs Gaja --- Don Granaman [EMAIL PROTECTED] wrote: !! Please do not post Off Topic to this List !! WOW! Is the other workload on these similar when this job runs? Are you sure the problem is the Symmetrix and not something in the OS or instance configuration? Does this job spend a lot of time waiting (in Oracle) on physical I/O - or on something else? (I guess if you don't have access to the machine, you can't find out though. The ultimate tuning challenge!) If the problem is actually Symmetrix I/O, I could only hazard a guess that it might be due to RAID-5 for something inappropriate (hot redo log files?) or extreme I/O contention in the layout. As far as pointers, pitfalls, and suggestions... I really have only one: don't believe everything you hear! For example: (top 10 list) 1) With EMC, RAID-5 won't matter. (it likely still will - for write-intensive stuff) 2) With EMC, you don't want to stripe. (you might - it can still make a big difference) 3) With the cache, I/O won't ever be a bottleneck. (until cache becomes saturated or ...) 4) [Corollary to #3] Throw out all that basic I/O tuning stuff you learned (but back it up to tape first!) 5) The best layout is always SAME - stripe and mirror everything across everything. 6) The check is in the mail. 7) This won't hurt a bit. [ORA-00051] Drat! I crashed before getting to ten! Sorry, I was up all night repairing a bridge... For more serious and less evasive answers, see Gaja's paper at http://www.quest.com/whitepapers/Raid1.pdf -Don Granaman [OraSaurus - Honk if you remember UFI!] - Original Message - To: Multiple recipients of list ORACLE-L [EMAIL PROTECTED] Sent: Thursday, September 13, 2001 6:55 AM !! Please do not post Off Topic to this List !! Hi All, Does anybody here on the list have experience with EMC/symmetrix storage units.? We have our databases on this machine and I have a feeling the the I/O performance is not very good. I can not proof it since I do not have any experience/data/access to that machine. We do however have a very cooperative UNIX group but they also lack experience with performance on this machine. Who can give me pointers about I/O throughput that can be reached, configuration pittfalls etc.. Example: RS6000 8CPU's and 4Gb memory with storage on EMC/symmetrix. Job takes about 2 hours to complete. F50 1 CPU 1Gb memory (TEST machine) local disks. same job takes 0.5 hours to complete. Jack === = Gaja Krishna Vaidyanatha Director, Storage Management Products, Quest Software, Inc. Co-author - Oracle Performance Tuning 101 http://www.osborne.com/database_erp/0072131454/0072131454.shtml
RE: I/O Performance/bottlenecks on EMC Symmetrix
!! Please do not post Off Topic to this List !! corollary to rule 2 after a certain point, just stop. It ain't worth it anymore. From: Christopher Spence [EMAIL PROTECTED] Reply-To: [EMAIL PROTECTED] To: Multiple recipients of list ORACLE-L [EMAIL PROTECTED] Subject: RE: I/O Performance/bottlenecks on EMC Symmetrix Date: Fri, 14 Sep 2001 05:45:20 -0800 Rules of tuning databases. 1. There is always a bottleneck. 2. Once you solve the bottle neck, refer to rule number 1. Do not criticize someone until you walked a mile in their shoes, that way when you criticize them, you are a mile a way and have their shoes. Christopher R. Spence Oracle DBA Phone: (978) 322-5744 Fax:(707) 885-2275 Fuelspot 73 Princeton Street North, Chelmsford 01863 -Original Message- Sent: Thursday, September 13, 2001 10:01 AM To: Multiple recipients of list ORACLE-L At one site I worked using Oracle Financials we were having serious performance problems at what seemed to us random intervals. Spent months looking at the database after the Unix boys had said that there was no way we could have I/O problems with the throughput capabililities of EMC and the Symetrix set up we had. Eventually turned out that 3 systems were sharing the same disks and the disks had not been striped. Therefore other system were causing us performance problems. If you have an EMC support contract which I think you must have you, the SA's get all the free GUI tools that allow them to look at channels and logical/physical layout. Ask them about. John -Original Message- Sent: 13 September 01 12:55 To: Multiple recipients of list ORACLE-L !! Please do not post Off Topic to this List !! Hi All, Does anybody here on the list have experience with EMC/symmetrix storage units.? We have our databases on this machine and I have a feeling the the I/O performance is not very good. I can not proof it since I do not have any experience/data/access to that machine. We do however have a very cooperative UNIX group but they also lack experience with performance on this machine. Who can give me pointers about I/O throughput that can be reached, configuration pittfalls etc.. Example: RS6000 8CPU's and 4Gb memory with storage on EMC/symmetrix. Job takes about 2 hours to complete. F50 1 CPU 1Gb memory (TEST machine) local disks. same job takes 0.5 hours to complete. Jack = De informatie verzonden in dit e-mailbericht is vertrouwelijk en is uitsluitend bestemd voor de geadresseerde. Openbaarmaking, vermenigvuldiging, verspreiding en/of verstrekking van deze informatie aan derden is, behoudens voorafgaande schriftelijke toestemming van Ernst Young, niet toegestaan. Ernst Young staat niet in voor de juiste en volledige overbrenging van de inhoud van een verzonden e-mailbericht, noch voor tijdige ontvangst daarvan. Ernst Young kan niet garanderen dat een verzonden e-mailbericht vrij is van virussen, noch dat e-mailberichten worden overgebracht zonder inbreuk of tussenkomst van onbevoegde derden. Indien bovenstaand e-mailbericht niet aan u is gericht, verzoeken wij u vriendelijk doch dringend het e-mailbericht te retourneren aan de verzender en het origineel en eventuele kopieën te verwijderen en te vernietigen. Ernst Young hanteert bij de uitoefening van haar werkzaamheden algemene voorwaarden, waarin een beperking van aansprakelijkheid is opgenomen. De algemene voorwaarden worden u op verzoek kosteloos toegezonden. = The information contained in this communication is confidential and is intended solely for the use of the individual or entity to whom it is addressed. You should not copy, disclose or distribute this communication without the authority of Ernst Young. Ernst Young is neither liable for the proper and complete transmission of the information contained in this communication nor for any delay in its receipt. Ernst Young does not guarantee that the integrity of this communication has been maintained nor that the communication is free of viruses, interceptions or interference. If you are not the intended recipient of this communication please return the communication to the sender and delete and destroy all copies. In carrying out its engagements, Ernst Young applies general terms and conditions, which contain a clause that limits its liability. A copy of these terms and conditions is available on request free of charge. = -- Please see the official ORACLE-L FAQ: http://www.orafaq.com http://www.orafaq.com -- Author: INET: [EMAIL PROTECTED] Fat City Network Services-- (858) 538-5051 FAX: (858) 538-5051 San Diego, California-- Public Internet access / Mailing Lists To REMOVE yourself from this mailing list, send an E-Mail message to: [EMAIL
Re: I/O Performance/bottlenecks on EMC Symmetrix
!! Please do not post Off Topic to this List !! Absolutely! Every loop must have an exit! - Original Message - To: Multiple recipients of list ORACLE-L [EMAIL PROTECTED] Sent: Friday, September 14, 2001 12:45 PM !! Please do not post Off Topic to this List !! corollary to rule 2 after a certain point, just stop. It ain't worth it anymore. From: Christopher Spence [EMAIL PROTECTED] Reply-To: [EMAIL PROTECTED] To: Multiple recipients of list ORACLE-L [EMAIL PROTECTED] Subject: RE: I/O Performance/bottlenecks on EMC Symmetrix Date: Fri, 14 Sep 2001 05:45:20 -0800 Rules of tuning databases. 1. There is always a bottleneck. 2. Once you solve the bottle neck, refer to rule number 1. Do not criticize someone until you walked a mile in their shoes, that way when you criticize them, you are a mile a way and have their shoes. Christopher R. Spence Oracle DBA Phone: (978) 322-5744 Fax:(707) 885-2275 Fuelspot 73 Princeton Street North, Chelmsford 01863 -Original Message- Sent: Thursday, September 13, 2001 10:01 AM To: Multiple recipients of list ORACLE-L At one site I worked using Oracle Financials we were having serious performance problems at what seemed to us random intervals. Spent months looking at the database after the Unix boys had said that there was no way we could have I/O problems with the throughput capabililities of EMC and the Symetrix set up we had. Eventually turned out that 3 systems were sharing the same disks and the disks had not been striped. Therefore other system were causing us performance problems. If you have an EMC support contract which I think you must have you, the SA's get all the free GUI tools that allow them to look at channels and logical/physical layout. Ask them about. John -Original Message- Sent: 13 September 01 12:55 To: Multiple recipients of list ORACLE-L !! Please do not post Off Topic to this List !! Hi All, Does anybody here on the list have experience with EMC/symmetrix storage units.? We have our databases on this machine and I have a feeling the the I/O performance is not very good. I can not proof it since I do not have any experience/data/access to that machine. We do however have a very cooperative UNIX group but they also lack experience with performance on this machine. Who can give me pointers about I/O throughput that can be reached, configuration pittfalls etc.. Example: RS6000 8CPU's and 4Gb memory with storage on EMC/symmetrix. Job takes about 2 hours to complete. F50 1 CPU 1Gb memory (TEST machine) local disks. same job takes 0.5 hours to complete. Jack = De informatie verzonden in dit e-mailbericht is vertrouwelijk en is uitsluitend bestemd voor de geadresseerde. Openbaarmaking, vermenigvuldiging, verspreiding en/of verstrekking van deze informatie aan derden is, behoudens voorafgaande schriftelijke toestemming van Ernst Young, niet toegestaan. Ernst Young staat niet in voor de juiste en volledige overbrenging van de inhoud van een verzonden e-mailbericht, noch voor tijdige ontvangst daarvan. Ernst Young kan niet garanderen dat een verzonden e-mailbericht vrij is van virussen, noch dat e-mailberichten worden overgebracht zonder inbreuk of tussenkomst van onbevoegde derden. Indien bovenstaand e-mailbericht niet aan u is gericht, verzoeken wij u vriendelijk doch dringend het e-mailbericht te retourneren aan de verzender en het origineel en eventuele kopieën te verwijderen en te vernietigen. Ernst Young hanteert bij de uitoefening van haar werkzaamheden algemene voorwaarden, waarin een beperking van aansprakelijkheid is opgenomen. De algemene voorwaarden worden u op verzoek kosteloos toegezonden. = The information contained in this communication is confidential and is intended solely for the use of the individual or entity to whom it is addressed. You should not copy, disclose or distribute this communication without the authority of Ernst Young. Ernst Young is neither liable for the proper and complete transmission of the information contained in this communication nor for any delay in its receipt. Ernst Young does not guarantee that the integrity of this communication has been maintained nor that the communication is free of viruses, interceptions or interference. If you are not the intended recipient of this communication please return the communication to the sender and delete and destroy all copies. In carrying out its engagements, Ernst Young applies general terms and conditions, which contain a clause that limits its liability. A copy of these terms and conditions is available on request free of charge. = -- Please see the official ORACLE-L FAQ: http://www.orafaq.com http://www.orafaq.com
RE: I/O Performance/bottlenecks on EMC Symmetrix
!! Please do not post Off Topic to this List !! We use EMC Symmetrics. When I got here, the folks who set things up initially where under the impression that you will never have to worry about placement/io with a Symm. Wrong! I underwent a painstaking process of relaying out the disks according to conventions DBA normally use taking into consideration index/data/volume of read/write to physical disks. Since you are presented with logical volumes, we use sym commands on each of our hosts connected to the symm to give us a graphical layout of all the logical volumes on the physical disks. EMC has a tool to do this called Resource View. EMC also has a tool called workload analyzer to analyze activity on each fa / disk, etc. And there is also a utility called Symm Optimizer to detect hot spots and optionally, replace that data to a different physical disk. You should find out if you have any of these tools available to you. HTH Michele Armstrong -Original Message- Sent: Thursday, September 13, 2001 7:55 AM To: Multiple recipients of list ORACLE-L !! Please do not post Off Topic to this List !! Hi All, Does anybody here on the list have experience with EMC/symmetrix storage units.? We have our databases on this machine and I have a feeling the the I/O performance is not very good. I can not proof it since I do not have any experience/data/access to that machine. We do however have a very cooperative UNIX group but they also lack experience with performance on this machine. Who can give me pointers about I/O throughput that can be reached, configuration pittfalls etc.. Example: RS6000 8CPU's and 4Gb memory with storage on EMC/symmetrix. Job takes about 2 hours to complete. F50 1 CPU 1Gb memory (TEST machine) local disks. same job takes 0.5 hours to complete. Jack = De informatie verzonden in dit e-mailbericht is vertrouwelijk en is uitsluitend bestemd voor de geadresseerde. Openbaarmaking, vermenigvuldiging, verspreiding en/of verstrekking van deze informatie aan derden is, behoudens voorafgaande schriftelijke toestemming van Ernst Young, niet toegestaan. Ernst Young staat niet in voor de juiste en volledige overbrenging van de inhoud van een verzonden e-mailbericht, noch voor tijdige ontvangst daarvan. Ernst Young kan niet garanderen dat een verzonden e-mailbericht vrij is van virussen, noch dat e-mailberichten worden overgebracht zonder inbreuk of tussenkomst van onbevoegde derden. Indien bovenstaand e-mailbericht niet aan u is gericht, verzoeken wij u vriendelijk doch dringend het e-mailbericht te retourneren aan de verzender en het origineel en eventuele kopieën te verwijderen en te vernietigen. Ernst Young hanteert bij de uitoefening van haar werkzaamheden algemene voorwaarden, waarin een beperking van aansprakelijkheid is opgenomen. De algemene voorwaarden worden u op verzoek kosteloos toegezonden. = The information contained in this communication is confidential and is intended solely for the use of the individual or entity to whom it is addressed. You should not copy, disclose or distribute this communication without the authority of Ernst Young. Ernst Young is neither liable for the proper and complete transmission of the information contained in this communication nor for any delay in its receipt. Ernst Young does not guarantee that the integrity of this communication has been maintained nor that the communication is free of viruses, interceptions or interference. If you are not the intended recipient of this communication please return the communication to the sender and delete and destroy all copies. In carrying out its engagements, Ernst Young applies general terms and conditions, which contain a clause that limits its liability. A copy of these terms and conditions is available on request free of charge. = -- Please see the official ORACLE-L FAQ: http://www.orafaq.com -- Author: INET: [EMAIL PROTECTED] Fat City Network Services-- (858) 538-5051 FAX: (858) 538-5051 San Diego, California-- Public Internet access / Mailing Lists To REMOVE yourself from this mailing list, send an E-Mail message to: [EMAIL PROTECTED] (note EXACT spelling of 'ListGuru') and in the message BODY, include a line containing: UNSUB ORACLE-L (or the name of mailing list you want to be removed from). You may also send the HELP command for other information (like subscribing). -- Please see the official ORACLE-L FAQ: http://www.orafaq.com -- Author: Armstrong, Michele INET: [EMAIL PROTECTED] Fat City Network Services-- (858) 538-5051 FAX: (858) 538-5051 San Diego, California-- Public Internet access / Mailing Lists
Re: I/O Performance/bottlenecks on EMC Symmetrix
!! Please do not post Off Topic to this List !!EMC supplies very good tools and support. Ask your EMC rep to show you how to use them. [EMAIL PROTECTED] 09/13/01 07:55AM !! Please do not post Off Topic to this List !! Hi All, Does anybody here on the list have experience with EMC/symmetrix storage units.? We have our databases on this machine and I have a feeling the the I/O performance is not very good. I can not proof it since I do not have any experience/data/access to that machine. We do however have a very cooperative UNIX group but they also lack experience with performance on this machine. Who can give me pointers about I/O throughput that can be reached, configuration pittfalls etc.. Example: RS6000 8CPU's and 4Gb memory with storage on EMC/symmetrix. Job takes about 2 hours to complete. F50 1 CPU 1Gb memory (TEST machine) local disks. same job takes 0.5 hours to complete. Jack = De informatie verzonden in dit e-mailbericht is vertrouwelijk en is uitsluitend bestemd voor de geadresseerde. Openbaarmaking, vermenigvuldiging, verspreiding en/of verstrekking van deze informatie aan derden is, behoudens voorafgaande schriftelijke toestemming van Ernst Young, niet toegestaan. Ernst Young staat niet in voor de juiste en volledige overbrenging van de inhoud van een verzonden e-mailbericht, noch voor tijdige ontvangst daarvan. Ernst Young kan niet garanderen dat een verzonden e-mailbericht vrij is van virussen, noch dat e-mailberichten worden overgebracht zonder inbreuk of tussenkomst van onbevoegde derden. Indien bovenstaand e-mailbericht niet aan u is gericht, verzoeken wij u vriendelijk doch dringend het e-mailbericht te retourneren aan de verzender en het origineel en eventuele kopieën te verwijderen en te vernietigen. Ernst Young hanteert bij de uitoefening van haar werkzaamheden algemene voorwaarden, waarin een beperking van aansprakelijkheid is opgenomen. De algemene voorwaarden worden u op verzoek kosteloos toegezonden. = The information contained in this communication is confidential and is intended solely for the use of the individual or entity to whom it is addressed. You should not copy, disclose or distribute this communication without the authority of Ernst Young. Ernst Young is neither liable for the proper and complete transmission of the information contained in this communication nor for any delay in its receipt. Ernst Young does not guarantee that the integrity of this communication has been maintained nor that the communication is free of viruses, interceptions or interference. If you are not the intended recipient of this communication please return the communication to the sender and delete and destroy all copies. In carrying out its engagements, Ernst Young applies general terms and conditions, which contain a clause that limits its liability. A copy of these terms and conditions is available on request free of charge. = -- Please see the official ORACLE-L FAQ: http://www.orafaq.com -- Author: INET: [EMAIL PROTECTED] Fat City Network Services-- (858) 538-5051 FAX: (858) 538-5051 San Diego, California-- Public Internet access / Mailing Lists To REMOVE yourself from this mailing list, send an E-Mail message to: [EMAIL PROTECTED] (note EXACT spelling of 'ListGuru') and in the message BODY, include a line containing: UNSUB ORACLE-L (or the name of mailing list you want to be removed from). You may also send the HELP command for other information (like subscribing). -- Please see the official ORACLE-L FAQ: http://www.orafaq.com -- Author: Gene Sais INET: [EMAIL PROTECTED] Fat City Network Services-- (858) 538-5051 FAX: (858) 538-5051 San Diego, California-- Public Internet access / Mailing Lists To REMOVE yourself from this mailing list, send an E-Mail message to: [EMAIL PROTECTED] (note EXACT spelling of 'ListGuru') and in the message BODY, include a line containing: UNSUB ORACLE-L (or the name of mailing list you want to be removed from). You may also send the HELP command for other information (like subscribing).
RE: I/O Performance/bottlenecks on EMC Symmetrix
Title: RE: I/O Performance/bottlenecks on EMC Symmetrix At one site I worked using Oracle Financials we were having serious performance problems at what seemed to us random intervals. Spent months looking at the database after the Unix boys had said that there was no way we could have I/O problems with the throughput capabililities of EMC and the Symetrix set up we had. Eventually turned out that 3 systems were sharing the same disks and the disks had not been striped. Therefore other system were causing us performance problems. If you have an EMC support contract which I think you must have you, the SA's get all the free GUI tools that allow them to look at channels and logical/physical layout. Ask them about. John -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]] Sent: 13 September 01 12:55 To: Multiple recipients of list ORACLE-L Subject: I/O Performance/bottlenecks on EMC Symmetrix !! Please do not post Off Topic to this List !! Hi All, Does anybody here on the list have experience with EMC/symmetrix storage units.? We have our databases on this machine and I have a feeling the the I/O performance is not very good. I can not proof it since I do not have any experience/data/access to that machine. We do however have a very cooperative UNIX group but they also lack experience with performance on this machine. Who can give me pointers about I/O throughput that can be reached, configuration pittfalls etc.. Example: RS6000 8CPU's and 4Gb memory with storage on EMC/symmetrix. Job takes about 2 hours to complete. F50 1 CPU 1Gb memory (TEST machine) local disks. same job takes 0.5 hours to complete. Jack = De informatie verzonden in dit e-mailbericht is vertrouwelijk en is uitsluitend bestemd voor de geadresseerde. Openbaarmaking, vermenigvuldiging, verspreiding en/of verstrekking van deze informatie aan derden is, behoudens voorafgaande schriftelijke toestemming van Ernst Young, niet toegestaan. Ernst Young staat niet in voor de juiste en volledige overbrenging van de inhoud van een verzonden e-mailbericht, noch voor tijdige ontvangst daarvan. Ernst Young kan niet garanderen dat een verzonden e-mailbericht vrij is van virussen, noch dat e-mailberichten worden overgebracht zonder inbreuk of tussenkomst van onbevoegde derden. Indien bovenstaand e-mailbericht niet aan u is gericht, verzoeken wij u vriendelijk doch dringend het e-mailbericht te retourneren aan de verzender en het origineel en eventuele kopieën te verwijderen en te vernietigen. Ernst Young hanteert bij de uitoefening van haar werkzaamheden algemene voorwaarden, waarin een beperking van aansprakelijkheid is opgenomen. De algemene voorwaarden worden u op verzoek kosteloos toegezonden. = The information contained in this communication is confidential and is intended solely for the use of the individual or entity to whom it is addressed. You should not copy, disclose or distribute this communication without the authority of Ernst Young. Ernst Young is neither liable for the proper and complete transmission of the information contained in this communication nor for any delay in its receipt. Ernst Young does not guarantee that the integrity of this communication has been maintained nor that the communication is free of viruses, interceptions or interference. If you are not the intended recipient of this communication please return the communication to the sender and delete and destroy all copies. In carrying out its engagements, Ernst Young applies general terms and conditions, which contain a clause that limits its liability. A copy of these terms and conditions is available on request free of charge. = -- Please see the official ORACLE-L FAQ: http://www.orafaq.com -- Author: INET: [EMAIL PROTECTED] Fat City Network Services -- (858) 538-5051 FAX: (858) 538-5051 San Diego, California -- Public Internet access / Mailing Lists To REMOVE yourself from this mailing list, send an E-Mail message to: [EMAIL PROTECTED] (note EXACT spelling of 'ListGuru') and in the message BODY, include a line containing: UNSUB ORACLE-L (or the name of mailing list you want to be removed from). You may also send the HELP command for other information (like subscribing). ** This email and any attachments may be confidential and the subject of legal professional privilege. Any disclosure, use, storage or copying of this email without the consent of the sender is strictly prohibited. Please notify the sender immediately if you are not the intended recipient and then delete the email from your inbox and do not disclose the contents to another person, use
Re: I/O Performance/bottlenecks on EMC Symmetrix
!! Please do not post Off Topic to this List !! WOW! Is the other workload on these similar when this job runs? Are you sure the problem is the Symmetrix and not something in the OS or instance configuration? Does this job spend a lot of time waiting (in Oracle) on physical I/O - or on something else? (I guess if you don't have access to the machine, you can't find out though. The ultimate tuning challenge!) If the problem is actually Symmetrix I/O, I could only hazard a guess that it might be due to RAID-5 for something inappropriate (hot redo log files?) or extreme I/O contention in the layout. As far as pointers, pitfalls, and suggestions... I really have only one: don't believe everything you hear! For example: (top 10 list) 1) With EMC, RAID-5 won't matter. (it likely still will - for write-intensive stuff) 2) With EMC, you don't want to stripe. (you might - it can still make a big difference) 3) With the cache, I/O won't ever be a bottleneck. (until cache becomes saturated or ...) 4) [Corollary to #3] Throw out all that basic I/O tuning stuff you learned (but back it up to tape first!) 5) The best layout is always SAME - stripe and mirror everything across everything. 6) The check is in the mail. 7) This won't hurt a bit. [ORA-00051] Drat! I crashed before getting to ten! Sorry, I was up all night repairing a bridge... For more serious and less evasive answers, see Gaja's paper at http://www.quest.com/whitepapers/Raid1.pdf -Don Granaman [OraSaurus - Honk if you remember UFI!] - Original Message - To: Multiple recipients of list ORACLE-L [EMAIL PROTECTED] Sent: Thursday, September 13, 2001 6:55 AM !! Please do not post Off Topic to this List !! Hi All, Does anybody here on the list have experience with EMC/symmetrix storage units.? We have our databases on this machine and I have a feeling the the I/O performance is not very good. I can not proof it since I do not have any experience/data/access to that machine. We do however have a very cooperative UNIX group but they also lack experience with performance on this machine. Who can give me pointers about I/O throughput that can be reached, configuration pittfalls etc.. Example: RS6000 8CPU's and 4Gb memory with storage on EMC/symmetrix. Job takes about 2 hours to complete. F50 1 CPU 1Gb memory (TEST machine) local disks. same job takes 0.5 hours to complete. Jack = De informatie verzonden in dit e-mailbericht is vertrouwelijk en is uitsluitend bestemd voor de geadresseerde. Openbaarmaking, vermenigvuldiging, verspreiding en/of verstrekking van deze informatie aan derden is, behoudens voorafgaande schriftelijke toestemming van Ernst Young, niet toegestaan. Ernst Young staat niet in voor de juiste en volledige overbrenging van de inhoud van een verzonden e-mailbericht, noch voor tijdige ontvangst daarvan. Ernst Young kan niet garanderen dat een verzonden e-mailbericht vrij is van virussen, noch dat e-mailberichten worden overgebracht zonder inbreuk of tussenkomst van onbevoegde derden. Indien bovenstaand e-mailbericht niet aan u is gericht, verzoeken wij u vriendelijk doch dringend het e-mailbericht te retourneren aan de verzender en het origineel en eventuele kopieën te verwijderen en te vernietigen. Ernst Young hanteert bij de uitoefening van haar werkzaamheden algemene voorwaarden, waarin een beperking van aansprakelijkheid is opgenomen. De algemene voorwaarden worden u op verzoek kosteloos toegezonden. = The information contained in this communication is confidential and is intended solely for the use of the individual or entity to whom it is addressed. You should not copy, disclose or distribute this communication without the authority of Ernst Young. Ernst Young is neither liable for the proper and complete transmission of the information contained in this communication nor for any delay in its receipt. Ernst Young does not guarantee that the integrity of this communication has been maintained nor that the communication is free of viruses, interceptions or interference. If you are not the intended recipient of this communication please return the communication to the sender and delete and destroy all copies. In carrying out its engagements, Ernst Young applies general terms and conditions, which contain a clause that limits its liability. A copy of these terms and conditions is available on request free of charge. = -- Please see the official ORACLE-L FAQ: http://www.orafaq.com -- Author: INET: [EMAIL PROTECTED] Fat City Network Services-- (858) 538-5051 FAX: (858) 538-5051 San Diego, California-- Public Internet access / Mailing Lists To REMOVE yourself from this mailing list, send an E-Mail message to:
RE: I/O Performance/bottlenecks on EMC Symmetrix
Title: RE: I/O Performance/bottlenecks on EMC Symmetrix We saw this at a previous location and we found that we had logical volumes on the same disk.. Used for different applications, and even different systems!! For example-someone placed TEMP for DB1 and TEMP for DB2 on the same disk unknowingly This created just a small contention problem... This is a common problem when large sym cabinets are set up with "logical volumes split to different systems". Each LV may be defined such that you wont know that they are on the same disk. If you are using BCV's, and clustering-it get's even more detailed and confusing... Have the EMC rep come in, take a look at the "bin" file of your sym cabinet(s).. And see where the logical volumes are living.. Draw a map of the bin file with a graphical representation of each LV, and where they live... Greg -Original Message-From: Hallas John [mailto:[EMAIL PROTECTED]]Sent: Thursday, September 13, 2001 10:01 AMTo: Multiple recipients of list ORACLE-LSubject: RE: I/O Performance/bottlenecks on EMC Symmetrix At one site I worked using Oracle Financials we were having serious performance problems at what seemed to us random intervals. Spent months looking at the database after the Unix boys had said that there was no way we could have I/O problems with the throughput capabililities of EMC and the Symetrix set up we had. Eventually turned out that 3 systems were sharing the same disks and the disks had not been striped. Therefore other system were causing us performance problems. If you have an EMC support contract which I think you must have you, the SA's get all the free GUI tools that allow them to look at channels and logical/physical layout. Ask them about. John -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]] Sent: 13 September 01 12:55 To: Multiple recipients of list ORACLE-L Subject: I/O Performance/bottlenecks on EMC Symmetrix !! Please do not post Off Topic to this List !! Hi All, Does anybody here on the list have experience with EMC/symmetrix storage units.? We have our databases on this machine and I have a feeling the the I/O performance is not very good. I can not proof it since I do not have any experience/data/access to that machine. We do however have a very cooperative UNIX group but they also lack experience with performance on this machine. Who can give me pointers about I/O throughput that can be reached, configuration pittfalls etc.. Example: RS6000 8CPU's and 4Gb memory with storage on EMC/symmetrix. Job takes about 2 hours to complete. F50 1 CPU 1Gb memory (TEST machine) local disks. same job takes 0.5 hours to complete. Jack = De informatie verzonden in dit e-mailbericht is vertrouwelijk en is uitsluitend bestemd voor de geadresseerde. Openbaarmaking, vermenigvuldiging, verspreiding en/of verstrekking van deze informatie aan derden is, behoudens voorafgaande schriftelijke toestemming van Ernst Young, niet toegestaan. Ernst Young staat niet in voor de juiste en volledige overbrenging van de inhoud van een verzonden e-mailbericht, noch voor tijdige ontvangst daarvan. Ernst Young kan niet garanderen dat een verzonden e-mailbericht vrij is van virussen, noch dat e-mailberichten worden overgebracht zonder inbreuk of tussenkomst van onbevoegde derden. Indien bovenstaand e-mailbericht niet aan u is gericht, verzoeken wij u vriendelijk doch dringend het e-mailbericht te retourneren aan de verzender en het origineel en eventuele kopieën te verwijderen en te vernietigen. Ernst Young hanteert bij de uitoefening van haar werkzaamheden algemene voorwaarden, waarin een beperking van aansprakelijkheid is opgenomen. De algemene voorwaarden worden u op verzoek kosteloos toegezonden. = The information contained in this communication is confidential and is intended solely for the use of the individual or entity to whom it is addressed. You should not copy, disclose or distribute this communication without the authority of Ernst Young. Ernst Young is neither liable for the proper and complete transmission of the information contained in this communication nor for any delay in its receipt. Ernst Young does not guarantee that the integrity of this communication has been maintained nor that the communication is free of viruses, interceptions or interference. If you are not the intended recipient of this communication please return the communication to the sender and delete and destroy all copies. In carrying out its engagements, Ernst Young applies general terms and conditions, which contain a clause that limits its liability. A
RE: I/O Performance/bottlenecks on EMC Symmetrix
Title: RE: I/O Performance/bottlenecks on EMC Symmetrix Good list Don, and a dentist as well (repairing a bridge)!!! -Original Message- From: Don Granaman [mailto:[EMAIL PROTECTED]] Sent: 13 September 01 15:15 To: Multiple recipients of list ORACLE-L Subject: Re: I/O Performance/bottlenecks on EMC Symmetrix !! Please do not post Off Topic to this List !! WOW! Is the other workload on these similar when this job runs? Are you sure the problem is the Symmetrix and not something in the OS or instance configuration? Does this job spend a lot of time waiting (in Oracle) on physical I/O - or on something else? (I guess if you don't have access to the machine, you can't find out though. The ultimate tuning challenge!) If the problem is actually Symmetrix I/O, I could only hazard a guess that it might be due to RAID-5 for something inappropriate (hot redo log files?) or extreme I/O contention in the layout. As far as pointers, pitfalls, and suggestions... I really have only one: don't believe everything you hear! For example: (top 10 list) 1) With EMC, RAID-5 won't matter. (it likely still will - for write-intensive stuff) 2) With EMC, you don't want to stripe. (you might - it can still make a big difference) 3) With the cache, I/O won't ever be a bottleneck. (until cache becomes saturated or ...) 4) [Corollary to #3] Throw out all that basic I/O tuning stuff you learned (but back it up to tape first!) 5) The best layout is always SAME - stripe and mirror everything across everything. 6) The check is in the mail. 7) This won't hurt a bit. [ORA-00051] Drat! I crashed before getting to ten! Sorry, I was up all night repairing a bridge... For more serious and less evasive answers, see Gaja's paper at http://www.quest.com/whitepapers/Raid1.pdf -Don Granaman [OraSaurus - Honk if you remember UFI!] - Original Message - To: Multiple recipients of list ORACLE-L [EMAIL PROTECTED] Sent: Thursday, September 13, 2001 6:55 AM !! Please do not post Off Topic to this List !! Hi All, Does anybody here on the list have experience with EMC/symmetrix storage units.? We have our databases on this machine and I have a feeling the the I/O performance is not very good. I can not proof it since I do not have any experience/data/access to that machine. We do however have a very cooperative UNIX group but they also lack experience with performance on this machine. Who can give me pointers about I/O throughput that can be reached, configuration pittfalls etc.. Example: RS6000 8CPU's and 4Gb memory with storage on EMC/symmetrix. Job takes about 2 hours to complete. F50 1 CPU 1Gb memory (TEST machine) local disks. same job takes 0.5 hours to complete. Jack = De informatie verzonden in dit e-mailbericht is vertrouwelijk en is uitsluitend bestemd voor de geadresseerde. Openbaarmaking, vermenigvuldiging, verspreiding en/of verstrekking van deze informatie aan derden is, behoudens voorafgaande schriftelijke toestemming van Ernst Young, niet toegestaan. Ernst Young staat niet in voor de juiste en volledige overbrenging van de inhoud van een verzonden e-mailbericht, noch voor tijdige ontvangst daarvan. Ernst Young kan niet garanderen dat een verzonden e-mailbericht vrij is van virussen, noch dat e-mailberichten worden overgebracht zonder inbreuk of tussenkomst van onbevoegde derden. Indien bovenstaand e-mailbericht niet aan u is gericht, verzoeken wij u vriendelijk doch dringend het e-mailbericht te retourneren aan de verzender en het origineel en eventuele kopieën te verwijderen en te vernietigen. Ernst Young hanteert bij de uitoefening van haar werkzaamheden algemene voorwaarden, waarin een beperking van aansprakelijkheid is opgenomen. De algemene voorwaarden worden u op verzoek kosteloos toegezonden. = The information contained in this communication is confidential and is intended solely for the use of the individual or entity to whom it is addressed. You should not copy, disclose or distribute this communication without the authority of Ernst Young. Ernst Young is neither liable for the proper and complete transmission of the information contained in this communication nor for any delay in its receipt. Ernst Young does not guarantee that the integrity of this communication has been maintained nor that the communication is free of viruses, interceptions or interference. If you are not the intended recipient of this communication please return the communication to the sender and delete and destroy all copies. In carrying out its engagements, Ernst Young applies general terms and conditions, which contain a clause that limits its liability. A copy of these terms and conditions is available on request free of charge. = -- Please see the official ORACLE-L FAQ: http
Re: I/O Performance/bottlenecks on EMC Symmetrix
!! Please do not post Off Topic to this List !! Hi, The other workload I would say was about the same (Relatively speaking) Both machines did have idle time on the CPU (I/O intensive job). The quicker test machine was a bit stretched on memory at the time (running about 8 databases) but not too bad. The only thing I can think of is pi** poor performance (3 p's you don't want in marketing) of the symmetrix disks. One point this symmetrix is loaded with 36 Gb disks and all of them are sliced to pieces and than allocated to filesystems. In theory you can be sharing disks with other high I/O apps (E-mail system etc..). I do not however have any insight in what is where physically. Jack Don Granaman [EMAIL PROTECTED]@fatcity.com on 13-09-2001 16:15:25 Please respond to [EMAIL PROTECTED] Sent by: [EMAIL PROTECTED] To: Multiple recipients of list ORACLE-L [EMAIL PROTECTED] cc:(bcc: Jack van Zanen/nlzanen1/External/MEY/NL) !! Please do not post Off Topic to this List !! WOW! Is the other workload on these similar when this job runs? Are you sure the problem is the Symmetrix and not something in the OS or instance configuration? Does this job spend a lot of time waiting (in Oracle) on physical I/O - or on something else? (I guess if you don't have access to the machine, you can't find out though. The ultimate tuning challenge!) If the problem is actually Symmetrix I/O, I could only hazard a guess that it might be due to RAID-5 for something inappropriate (hot redo log files?) or extreme I/O contention in the layout. As far as pointers, pitfalls, and suggestions... I really have only one: don't believe everything you hear! For example: (top 10 list) 1) With EMC, RAID-5 won't matter. (it likely still will - for write-intensive stuff) 2) With EMC, you don't want to stripe. (you might - it can still make a big difference) 3) With the cache, I/O won't ever be a bottleneck. (until cache becomes saturated or ...) 4) [Corollary to #3] Throw out all that basic I/O tuning stuff you learned (but back it up to tape first!) 5) The best layout is always SAME - stripe and mirror everything across everything. 6) The check is in the mail. 7) This won't hurt a bit. [ORA-00051] Drat! I crashed before getting to ten! Sorry, I was up all night repairing a bridge... For more serious and less evasive answers, see Gaja's paper at http://www.quest.com/whitepapers/Raid1.pdf -Don Granaman [OraSaurus - Honk if you remember UFI!] - Original Message - To: Multiple recipients of list ORACLE-L [EMAIL PROTECTED] Sent: Thursday, September 13, 2001 6:55 AM !! Please do not post Off Topic to this List !! Hi All, Does anybody here on the list have experience with EMC/symmetrix storage units.? We have our databases on this machine and I have a feeling the the I/O performance is not very good. I can not proof it since I do not have any experience/data/access to that machine. We do however have a very cooperative UNIX group but they also lack experience with performance on this machine. Who can give me pointers about I/O throughput that can be reached, configuration pittfalls etc.. Example: RS6000 8CPU's and 4Gb memory with storage on EMC/symmetrix. Job takes about 2 hours to complete. F50 1 CPU 1Gb memory (TEST machine) local disks. same job takes 0.5 hours to complete. Jack = De informatie verzonden in dit e-mailbericht is vertrouwelijk en is uitsluitend bestemd voor de geadresseerde. Openbaarmaking, vermenigvuldiging, verspreiding en/of verstrekking van deze informatie aan derden is, behoudens voorafgaande schriftelijke toestemming van Ernst Young, niet toegestaan. Ernst Young staat niet in voor de juiste en volledige overbrenging van de inhoud van een verzonden e-mailbericht, noch voor tijdige ontvangst daarvan. Ernst Young kan niet garanderen dat een verzonden e-mailbericht vrij is van virussen, noch dat e-mailberichten worden overgebracht zonder inbreuk of tussenkomst van onbevoegde derden. Indien bovenstaand e-mailbericht niet aan u is gericht, verzoeken wij u vriendelijk doch dringend het e-mailbericht te retourneren aan de verzender en het origineel en eventuele kopieën te verwijderen en te vernietigen. Ernst Young hanteert bij de uitoefening van haar werkzaamheden algemene voorwaarden, waarin een beperking van aansprakelijkheid is opgenomen. De algemene voorwaarden worden u op verzoek kosteloos toegezonden. = The information contained in this communication is confidential and is intended solely for the use of the individual or entity to whom it is addressed. You should not copy, disclose or distribute this communication without the authority of Ernst Young. Ernst Young is neither liable for the proper and complete transmission of the information contained in this communication nor for any delay in its receipt. Ernst Young does not guarantee
RE: I/O Performance/bottlenecks on EMC Symmetrix
!! Please do not post Off Topic to this List !! Hi Given that you don't seem to have access to the hardware, I'd suggest timestamping. Choose say a few dozen points in the process and collect timestamps at each one. Compare the prod timestamps to the test timestamps, something may jump out, eg one call taking 90 minutes instead of 30 seconds. I tend to put a lot of this sort of thing into my production systems anyway, it's the first place I look when a job slows down. Your duhvelopers may be able to help. HTH Greg -Original Message- Sent: Thursday, 13 September 2001 16:15 To: Multiple recipients of list ORACLE-L !! Please do not post Off Topic to this List !! Hi, The other workload I would say was about the same (Relatively speaking) Both machines did have idle time on the CPU (I/O intensive job). The quicker test machine was a bit stretched on memory at the time (running about 8 databases) but not too bad. The only thing I can think of is pi** poor performance (3 p's you don't want in marketing) of the symmetrix disks. One point this symmetrix is loaded with 36 Gb disks and all of them are sliced to pieces and than allocated to filesystems. In theory you can be sharing disks with other high I/O apps (E-mail system etc..). I do not however have any insight in what is where physically. Jack Don Granaman [EMAIL PROTECTED]@fatcity.com on 13-09-2001 16:15:25 Please respond to [EMAIL PROTECTED] Sent by: [EMAIL PROTECTED] To: Multiple recipients of list ORACLE-L [EMAIL PROTECTED] cc:(bcc: Jack van Zanen/nlzanen1/External/MEY/NL) !! Please do not post Off Topic to this List !! WOW! Is the other workload on these similar when this job runs? Are you sure the problem is the Symmetrix and not something in the OS or instance configuration? Does this job spend a lot of time waiting (in Oracle) on physical I/O - or on something else? (I guess if you don't have access to the machine, you can't find out though. The ultimate tuning challenge!) If the problem is actually Symmetrix I/O, I could only hazard a guess that it might be due to RAID-5 for something inappropriate (hot redo log files?) or extreme I/O contention in the layout. As far as pointers, pitfalls, and suggestions... I really have only one: don't believe everything you hear! For example: (top 10 list) 1) With EMC, RAID-5 won't matter. (it likely still will - for write-intensive stuff) 2) With EMC, you don't want to stripe. (you might - it can still make a big difference) 3) With the cache, I/O won't ever be a bottleneck. (until cache becomes saturated or ...) 4) [Corollary to #3] Throw out all that basic I/O tuning stuff you learned (but back it up to tape first!) 5) The best layout is always SAME - stripe and mirror everything across everything. 6) The check is in the mail. 7) This won't hurt a bit. [ORA-00051] Drat! I crashed before getting to ten! Sorry, I was up all night repairing a bridge... For more serious and less evasive answers, see Gaja's paper at http://www.quest.com/whitepapers/Raid1.pdf -Don Granaman [OraSaurus - Honk if you remember UFI!] - Original Message - To: Multiple recipients of list ORACLE-L [EMAIL PROTECTED] Sent: Thursday, September 13, 2001 6:55 AM !! Please do not post Off Topic to this List !! Hi All, Does anybody here on the list have experience with EMC/symmetrix storage units.? We have our databases on this machine and I have a feeling the the I/O performance is not very good. I can not proof it since I do not have any experience/data/access to that machine. We do however have a very cooperative UNIX group but they also lack experience with performance on this machine. Who can give me pointers about I/O throughput that can be reached, configuration pittfalls etc.. Example: RS6000 8CPU's and 4Gb memory with storage on EMC/symmetrix. Job takes about 2 hours to complete. F50 1 CPU 1Gb memory (TEST machine) local disks. same job takes 0.5 hours to complete. Jack = De informatie verzonden in dit e-mailbericht is vertrouwelijk en is uitsluitend bestemd voor de geadresseerde. Openbaarmaking, vermenigvuldiging, verspreiding en/of verstrekking van deze informatie aan derden is, behoudens voorafgaande schriftelijke toestemming van Ernst Young, niet toegestaan. Ernst Young staat niet in voor de juiste en volledige overbrenging van de inhoud van een verzonden e-mailbericht, noch voor tijdige ontvangst daarvan. Ernst Young kan niet garanderen dat een verzonden e-mailbericht vrij is van virussen, noch dat e-mailberichten worden overgebracht zonder inbreuk of tussenkomst van onbevoegde derden. Indien bovenstaand e-mailbericht niet aan u is gericht, verzoeken wij u vriendelijk doch dringend het e-mailbericht te retourneren aan de verzender en het origineel en eventuele kopieën te verwijderen en te vernietigen. Ernst Young hanteert bij de uitoefening van haar werkzaamheden algemene voorwaarden,
RE: I/O Performance/bottlenecks on EMC Symmetrix
Title: RE: I/O Performance/bottlenecks on EMC Symmetrix John, I have to say that's a very interesting post. Two shops I've worked in utilizing an EMC Symmetrix claimed that there's no way you could ever end up with i/o problems and you shouldn't even consider it when creating a database on these disks. All conventional knowledge was out the window (tables indexes on separate disks, etc., etc.) I couldn't get by it, I just didn't believe it (like trying to convince myself of physics topics that just didn't make sense - just tell yourself it's true). You validated my gut feel. Thanks for sending this. Lisa Koivu Oracle Database Administrator Fairfield Resorts, Inc. 954-935-4117 -Original Message- From: Hallas John [SMTP:[EMAIL PROTECTED]] Sent: Thursday, September 13, 2001 10:01 AM To: Multiple recipients of list ORACLE-L Subject: RE: I/O Performance/bottlenecks on EMC Symmetrix At one site I worked using Oracle Financials we were having serious performance problems at what seemed to us random intervals. Spent months looking at the database after the Unix boys had said that there was no way we could have I/O problems with the throughput capabililities of EMC and the Symetrix set up we had. Eventually turned out that 3 systems were sharing the same disks and the disks had not been striped. Therefore other system were causing us performance problems. If you have an EMC support contract which I think you must have you, the SA's get all the free GUI tools that allow them to look at channels and logical/physical layout. Ask them about. John -Original Message- From: [EMAIL PROTECTED] [ mailto:[EMAIL PROTECTED]] Sent: 13 September 01 12:55 To: Multiple recipients of list ORACLE-L Subject: I/O Performance/bottlenecks on EMC Symmetrix !! Please do not post Off Topic to this List !! Hi All, Does anybody here on the list have experience with EMC/symmetrix storage units.? We have our databases on this machine and I have a feeling the the I/O performance is not very good. I can not proof it since I do not have any experience/data/access to that machine. We do however have a very cooperative UNIX group but they also lack experience with performance on this machine. Who can give me pointers about I/O throughput that can be reached, configuration pittfalls etc.. Example: RS6000 8CPU's and 4Gb memory with storage on EMC/symmetrix. Job takes about 2 hours to complete. F50 1 CPU 1Gb memory (TEST machine) local disks. same job takes 0.5 hours to complete. Jack = De informatie verzonden in dit e-mailbericht is vertrouwelijk en is uitsluitend bestemd voor de geadresseerde. Openbaarmaking, vermenigvuldiging, verspreiding en/of verstrekking van deze informatie aan derden is, behoudens voorafgaande schriftelijke toestemming van Ernst Young, niet toegestaan. Ernst Young staat niet in voor de juiste en volledige overbrenging van de inhoud van een verzonden e-mailbericht, noch voor tijdige ontvangst daarvan. Ernst Young kan niet garanderen dat een verzonden e-mailbericht vrij is van virussen, noch dat e-mailberichten worden overgebracht zonder inbreuk of tussenkomst van onbevoegde derden. Indien bovenstaand e-mailbericht niet aan u is gericht, verzoeken wij u vriendelijk doch dringend het e-mailbericht te retourneren aan de verzender en het origineel en eventuele kopieën te verwijderen en te vernietigen. Ernst Young hanteert bij de uitoefening van haar werkzaamheden algemene voorwaarden, waarin een beperking van aansprakelijkheid is opgenomen. De algemene voorwaarden worden u op verzoek kosteloos toegezonden. = The information contained in this communication is confidential and is intended solely for the use of the individual or entity to whom it is addressed. You should not copy, disclose or distribute this communication without the authority of Ernst Young. Ernst Young is neither liable for the proper and complete transmission of the information contained in this communication nor for any delay in its receipt. Ernst Young does not guarantee that the integrity of this communication has been maintained nor that the communication is free of viruses, interceptions or interference. If you are not the intended recipient of this communication please return the communication to the sender and delete and destroy all copies. In carrying out its engagements, Ernst Young applies general terms and conditions, which contain a clause that limits its liability. A copy of these terms and conditions is available on request free of charge. = -- Please see the official ORACLE-L FAQ: http://www.orafaq.com -- Author: INET: [EMAIL PROTECTED] Fat City Network Services -- (858) 538-5051 FAX: (858) 538-5051 San Diego, California -- Public Internet
Re: I/O Performance/bottlenecks on EMC Symmetrix
!! Please do not post Off Topic to this List !! I have had, and heard of, poor performance from EMC boxes before now because of the big black box principal. You might check out James Morle's book (scaling Oracle 8i) for some thoughts. From an Oracle perspective, you may find that simply checking v$filestat will demonstrate quite clearly that the average read time off disk is very poor - the last big site I went to were getting read times of worse than 100 millisecs - when EMC were claiming to offer better than 20 millisecs. If this doesn't help, there is a C program on my web-site (under a Miscellanous or Performance article on choosing a block size) which allows you to create a file, and then start emulating random Oracle-read I/Os - this should give you a quick way of testing the real response time of the black box. Jonathan Lewis http://www.jlcomp.demon.co.uk Host to The Co-Operative Oracle Users' FAQ http://www.jlcomp.demon.co.uk/faq/ind_faq.html Author of: Practical Oracle 8i: Building Efficient Databases Screen saver or Life saver: http://www.ud.com Use spare CPU to assist in cancer research. -Original Message- To: Multiple recipients of list ORACLE-L [EMAIL PROTECTED] Date: 13 September 2001 16:30 !! Please do not post Off Topic to this List !! Hi, The other workload I would say was about the same (Relatively speaking) Both machines did have idle time on the CPU (I/O intensive job). The quicker test machine was a bit stretched on memory at the time (running about 8 databases) but not too bad. The only thing I can think of is pi** poor performance (3 p's you don't want in marketing) of the symmetrix disks. One point this symmetrix is loaded with 36 Gb disks and all of them are sliced to pieces and than allocated to filesystems. In theory you can be sharing disks with other high I/O apps (E-mail system etc..). I do not however have any insight in what is where physically. Jack -- Please see the official ORACLE-L FAQ: http://www.orafaq.com -- Author: Jonathan Lewis INET: [EMAIL PROTECTED] Fat City Network Services-- (858) 538-5051 FAX: (858) 538-5051 San Diego, California-- Public Internet access / Mailing Lists To REMOVE yourself from this mailing list, send an E-Mail message to: [EMAIL PROTECTED] (note EXACT spelling of 'ListGuru') and in the message BODY, include a line containing: UNSUB ORACLE-L (or the name of mailing list you want to be removed from). You may also send the HELP command for other information (like subscribing).
