Re: [U2] 24 X 7 MV systems
Nick, Dawn, Tony, David, Glen, John, Jeff, Bill, everyone. Thanks for the posts on this thread. There is good material for me to share with the rest of our team and hopefully overcome inertia and set some new direction. Thank you. -Baker -Original Message- From: u2-users-boun...@listserver.u2ug.org [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Dawn Wolthuis Sent: Thursday, July 29, 2010 4:53 PM To: U2 Users List Subject: Re: [U2] 24 X 7 MV systems You said MV system rather than something more specific, so I would encourage you to look at Cache'. I think it has the best features in the MV space in this regard (as well as scalability and other ilities). --dawn This communication, its contents and any file attachments transmitted with it are intended solely for the addressee(s) and may contain confidential proprietary information. Access by any other party without the express written permission of the sender is STRICTLY PROHIBITED. If you have received this communication in error you may not copy, distribute or use the contents, attachments or information in any way. Please destroy it and contact the sender. ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
Re: [U2] 24 X 7 MV systems
1. When I buy electronic components, the first place I look is Mouser. I find it a kick that Mouser is running U2. 2. If it can be revealed... the servers there that run U2... are they all Windows Servers?... Are some, any, or all *nix machines? --Bill -Original Message- From: u2-users-boun...@listserver.u2ug.org [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Baker Hughes Sent: Friday, July 30, 2010 4:51 PM To: 'U2 Users List' Subject: Re: [U2] 24 X 7 MV systems Nick, Dawn, Tony, David, Glen, John, Jeff, Bill, everyone. Thanks for the posts on this thread. There is good material for me to share with the rest of our team and hopefully overcome inertia and set some new direction. Thank you. -Baker -Original Message- From: u2-users-boun...@listserver.u2ug.org [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Dawn Wolthuis Sent: Thursday, July 29, 2010 4:53 PM To: U2 Users List Subject: Re: [U2] 24 X 7 MV systems You said MV system rather than something more specific, so I would encourage you to look at Cache'. I think it has the best features in the MV space in this regard (as well as scalability and other ilities). --dawn This communication, its contents and any file attachments transmitted with it are intended solely for the addressee(s) and may contain confidential proprietary information. Access by any other party without the express written permission of the sender is STRICTLY PROHIBITED. If you have received this communication in error you may not copy, distribute or use the contents, attachments or information in any way. Please destroy it and contact the sender. ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
Re: [U2] 24 X 7 MV systems
Hi Glen, Thanks for the reference. FYI, you can configure Visage.DRS to operate over point to point SSL connections - individual deltas can also be zipped to reduce bandwidth, and the zip can also be password protected, but as your probably know this simplistic encryption (password protected ZIP files) can be brute-forced but SSL would provide adequate protection on the wire :-) Ross Ferris Stamina Software Visage Better by Design! -Original Message- From: u2-users-boun...@listserver.u2ug.org [mailto:u2-users- boun...@listserver.u2ug.org] On Behalf Of Glen Batchelor Sent: Tuesday, 27 July 2010 1:03 AM To: 'U2 Users List' Subject: Re: [U2] 24 X 7 MV systems Hey Stranger, The best way you'll get there is with a transaction/request based redundancy setup. Does U2 have anything that isn't trigger related? Even a block-level DRDB config won't help with databases since transactions in memory aren't committed to disk promptly enough for the replicator to get all of the new data pieces as the fault happens. I've been looking for a remote hosted failover solution myself (not for U2). A truly fault tolerant setup requires that the incoming requests be replicated to multiple machines before anything happens inside. You might think of the user app as a web service consumer that makes requests to a proxy. That proxy mux's the requests to multiple machines, compares all of the responses and then passes back the response of one machine based on failover priority. If one of the responses aren't the same then an error is sent to the admin. Unfortunately, this spits in the eye of MV which is designed to be a stand-alone central data store. Maybe you can do it with your own TCP packets, but if you don't use an encrypted media you may get into security trouble. Web services are horribly bloated, but the security layer is already in there. You should bug Ross Ferris and pick his brain about his DRS product versus other options for U2. DRS supposedly uses a small amount of bandwidth but it isn't encrypted AFAIK. Regards, Glen Batchelor IT Director/CIO/CTO All-Spec Industries phone: (910) 332-0424 fax: (910) 763-5664 E-mail: webmas...@all-spec.com Web: http://www.all-spec.com Blog: http://blog.all-spec.com -Original Message- From: u2-users-boun...@listserver.u2ug.org [mailto:u2-users- boun...@listserver.u2ug.org] On Behalf Of Baker Hughes Sent: Monday, July 26, 2010 9:38 AM To: 'U2 Users List' Subject: [U2] 24 X 7 MV systems Hey y'all, I'm interested in hearing from folks who are currently on, or have worked with fault tolerant MV systems. We'd like to host our Business Layer on the MV system and serve It to our e-commerce portals, instead of re-coding our business rules first in Basic, then in .Net In order to get there though we must meet the primary business requirement of zero downtime (not even 2 minutes to manually switch). We're not talking about different levels of Raid - it's assumed the storage array is up and available. If the MV system has a hiccup of more than a few seconds it needs to hot failover to a backup twin sister. Is anyone doing this or something close to it? When I worked in public safety, Stratus sold such an automatic hot failover. I'm sure the EnRoute folks are doing something like this still. Maybe Nick G. or Margaret M. is listening in today. Thanks, -Baker This communication, its contents and any file attachments transmitted with it are intended solely for the addressee(s) and may contain confidential proprietary information. Access by any other party without the express written permission of the sender is STRICTLY PROHIBITED. If you have received this communication in error you may not copy, distribute or use the contents, attachments or information in any way. Please destroy it and contact the sender. ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
Re: [U2] 24 X 7 MV systems
Hi Baker, As Glen and other have alluded, I believe you are unlikely to find a OS level replication technology that will work adequately, unless there is a way to force writes to the OS level immediately, which in turn is likely to have performance implications. We cheat with Visage.DRS - we don't force you to operate via a middleware layer, so it will simply plug in to an existing application without the need for any changes. If your application already uses transaction boundaries, that is great, but our data replication service doesn't need them. I thought UV had an inbuilt solution already though I seem to remember seeing a demo at a spectrum exhibition a few years ago, and I thought it looked OK? Not sure it necessarily allowed replication to multiple servers, but on the surface it looked as though the basics probably worked (then again, D3 Hotbackup looked good on the surface, but we had to write DRS when our largest client failed an IT disaster recovery audit) We haven't ported Visage.DRS to UV - hasn't been any demand we could see, but if there IS a need (translation: makes financial sense to invest resources) then it could be done Ross Ferris Stamina Software Visage Better by Design! -Original Message- From: u2-users-boun...@listserver.u2ug.org [mailto:u2-users- boun...@listserver.u2ug.org] On Behalf Of Baker Hughes Sent: Tuesday, 27 July 2010 2:14 AM To: 'U2 Users List' Subject: Re: [U2] 24 X 7 MV systems Glen, Thank you for the good observations, and suggestion to ping Ross, which I will do. It is the MV db paradigm which in this case is hampering us, which drives the solution to either the OS level or some middle ware solution. Pausing the db to flush memory is what keeps us just seconds from a full solution. Thank you. -Baker -Original Message- From: u2-users-boun...@listserver.u2ug.org [mailto:u2-users- boun...@listserver.u2ug.org] On Behalf Of Glen Batchelor Sent: Monday, July 26, 2010 10:03 AM To: 'U2 Users List' Subject: Re: [U2] 24 X 7 MV systems Hey Stranger, The best way you'll get there is with a transaction/request based redundancy setup. Does U2 have anything that isn't trigger related? Even a block-level DRDB config won't help with databases since transactions in memory aren't committed to disk promptly enough for the replicator to get all of the new data pieces as the fault happens. I've been looking for a remote hosted failover solution myself (not for U2). A truly fault tolerant setup requires that the incoming requests be replicated to multiple machines before anything happens inside. You might think of the user app as a web service consumer that makes requests to a proxy. That proxy mux's the requests to multiple machines, compares all of the responses and then passes back the response of one machine based on failover priority. If one of the responses aren't the same then an error is sent to the admin. Unfortunately, this spits in the eye of MV which is designed to be a stand-alone central data store. Maybe you can do it with your own TCP packets, but if you don't use an encrypted media you may get into security trouble. Web services are horribly bloated, but the security layer is already in there. You should bug Ross Ferris and pick his brain about his DRS product versus other options for U2. DRS supposedly uses a small amount of bandwidth but it isn't encrypted AFAIK. Regards, Glen Batchelor IT Director/CIO/CTO All-Spec Industries phone: (910) 332-0424 fax: (910) 763-5664 E-mail: webmas...@all-spec.com Web: http://www.all-spec.com Blog: http://blog.all-spec.com This communication, its contents and any file attachments transmitted with it are intended solely for the addressee(s) and may contain confidential proprietary information. Access by any other party without the express written permission of the sender is STRICTLY PROHIBITED. If you have received this communication in error you may not copy, distribute or use the contents, attachments or information in any way. Please destroy it and contact the sender. ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
Re: [U2] 24 X 7 MV systems
Now that you have opened the door, I will say that I believe that the Reality solution is probably the best that is available from any vendor in the MV space that I have seen ... quite awesome. Cache also has some strong options in this area, BUT I think there is a non-trivial amount of plumbing that needs to be put in place (and not sure if this has been fully tested with MV implementation I know it shouldn't make a difference, but and now, back to your regularly scheduled U2 discussions (sorry) Ross Ferris Stamina Software Visage Better by Design! -Original Message- From: u2-users-boun...@listserver.u2ug.org [mailto:u2-users- boun...@listserver.u2ug.org] On Behalf Of Baker Hughes Sent: Tuesday, 27 July 2010 2:26 AM To: 'U2 Users List' Subject: Re: [U2] 24 X 7 MV systems Jeff, Thank you for giving your experience with HA AIX and a cluster. We are also doing the HA but not clustering. The couple minutes time lag you mention and the possibility of broken transactions make one wonder if it's worth going that distance. I didn't know Stratus was still out there so thanks for that. What about Sequoia? That was also a very coveted system in the 911 offices back in the day. Awesome story about the Sequoia still tooling right along while the disk is on fire. So did the system switch itself over to 'B' or did y'all do it, when? I can't match that one, but even with Reality 7 (sorry to mention this on the U2 list) we could throw a manual switch (took me about 30 seconds to get from my office to the switch in the computer room) and when the dispatchers logged in, there were their sessions with screens looking identical to system A. That's why I gotta believe we can do better, 20 years later. Thank you. -Baker -Original Message- From: u2-users-boun...@listserver.u2ug.org [mailto:u2-users- boun...@listserver.u2ug.org] On Behalf Of Jeff Schasny Sent: Monday, July 26, 2010 10:24 AM To: U2 Users List Subject: Re: [U2] 24 X 7 MV systems We are running an IBM high availability cluster of AIX machines which do auto fail-over. There is a couple minutes of time lag involved and there can be broken transactions since the switchover is OS level and not applications based so this is probably not a good solution for you since it sounds like you are looking for a truly fault tolerant solution. Stratus is still out there and probably a good choice for your needs. I know they have at least one series of boxes which run Red Hat and therefore are U2 compatible. I worked on a fault tolerant Sequoia system running Pick OA many years ago supporting an alarm monitoring application. Amazing machines. True Story: The operations manager gets a call from the computer operator who tells him Theres smoke coming out of one of the disk drawers on the Sequoia 'A' system (we had a second redundant Sequoia 'B' system as well). A couple quick phone calls later 5 of us are huddled together in the computer room on various phones with Sequoia after hitting the raid disk drawer in question with a fire extinguisher, trying to decide if we should switch over to the backup system, when out of the little glass cubicle where the operators live comes the operator on duty. He walks up to the smoking system, pulls off a spinning magnetic tape, and mounts the next reel of a file restore he's doing for someone. We all look at each other and laugh because the system is still running along just fine while on fire. Baker Hughes wrote: Hey y'all, I'm interested in hearing from folks who are currently on, or have worked with fault tolerant MV systems. We'd like to host our Business Layer on the MV system and serve It to our e-commerce portals, instead of re-coding our business rules first in Basic, then in .Net In order to get there though we must meet the primary business requirement of zero downtime (not even 2 minutes to manually switch). We're not talking about different levels of Raid - it's assumed the storage array is up and available. If the MV system has a hiccup of more than a few seconds it needs to hot failover to a backup twin sister. Is anyone doing this or something close to it? When I worked in public safety, Stratus sold such an automatic hot failover. I'm sure the EnRoute folks are doing something like this still. Maybe Nick G. or Margaret M. is listening in today. Thanks, -Baker This communication, its contents and any file attachments transmitted with it are intended solely for the addressee(s) and may contain confidential proprietary information. Access by any other party without the express written permission of the sender is STRICTLY PROHIBITED. If you have received this communication in error you may not copy, distribute or use the contents, attachments or information in any way. Please destroy it and contact the sender. ___ U2-Users mailing list U2-Users@listserver.u2ug.org http
Re: [U2] 24 X 7 MV systems
You said MV system rather than something more specific, so I would encourage you to look at Cache'. I think it has the best features in the MV space in this regard (as well as scalability and other ilities). --dawn On Mon, Jul 26, 2010 at 8:38 AM, Baker Hughes baker.hug...@mouser.comwrote: Hey y'all, I'm interested in hearing from folks who are currently on, or have worked with fault tolerant MV systems. We'd like to host our Business Layer on the MV system and serve It to our e-commerce portals, instead of re-coding our business rules first in Basic, then in .Net In order to get there though we must meet the primary business requirement of zero downtime (not even 2 minutes to manually switch). We're not talking about different levels of Raid - it's assumed the storage array is up and available. If the MV system has a hiccup of more than a few seconds it needs to hot failover to a backup twin sister. Is anyone doing this or something close to it? When I worked in public safety, Stratus sold such an automatic hot failover. I'm sure the EnRoute folks are doing something like this still. Maybe Nick G. or Margaret M. is listening in today. Thanks, -Baker This communication, its contents and any file attachments transmitted with it are intended solely for the addressee(s) and may contain confidential proprietary information. Access by any other party without the express written permission of the sender is STRICTLY PROHIBITED. If you have received this communication in error you may not copy, distribute or use the contents, attachments or information in any way. Please destroy it and contact the sender. ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users -- Dawn M. Wolthuis Take and give some delight today ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
Re: [U2] 24 X 7 MV systems
On 26/07/10 23:17, Tony Gravagno wrote: As another Sequoia anecdote: An IT manager was surprised one day by a FedEx delivery of a motherboard. Apparently his system has lost a CPU and phoned to Support to get a new one, though no one at the company had noticed yet. The instructions were simply something like Remove bad board, insert new board. Twenty years later people are still wondering if it's possible to minimize downtime... Stratus anecdote this time ... I was at their UK office in Cannon Street (can't quite remember why) meeting a friend (Bill Hooper) who worked for them. He showed me round, and stopped at one machine. If I pull the CPU board out of this, the only people who notice anything wrong will be us, as this machine rings itself up to report a fault. It was their fault-reporting system, and it received phone calls like those made by your system. Cheers, Wol ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
[U2] 24 X 7 MV systems
Hey y'all, I'm interested in hearing from folks who are currently on, or have worked with fault tolerant MV systems. We'd like to host our Business Layer on the MV system and serve It to our e-commerce portals, instead of re-coding our business rules first in Basic, then in .Net In order to get there though we must meet the primary business requirement of zero downtime (not even 2 minutes to manually switch). We're not talking about different levels of Raid - it's assumed the storage array is up and available. If the MV system has a hiccup of more than a few seconds it needs to hot failover to a backup twin sister. Is anyone doing this or something close to it? When I worked in public safety, Stratus sold such an automatic hot failover. I'm sure the EnRoute folks are doing something like this still. Maybe Nick G. or Margaret M. is listening in today. Thanks, -Baker This communication, its contents and any file attachments transmitted with it are intended solely for the addressee(s) and may contain confidential proprietary information. Access by any other party without the express written permission of the sender is STRICTLY PROHIBITED. If you have received this communication in error you may not copy, distribute or use the contents, attachments or information in any way. Please destroy it and contact the sender. ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
Re: [U2] 24 X 7 MV systems
Baker, My company has a 2 hour window backup to a standby server using rsync based sbom backup. At 10:00, 12:15, 3:00 and 6:00 we pause, split, resume and backup from the detached mirror. Recovery is far from automated but it fits our business needs. Jeff - Industrial Piping Specialists, Tulsa, OK On 07/26/2010 08:38 AM, Baker Hughes wrote: Hey y'all, I'm interested in hearing from folks who are currently on, or have worked with fault tolerant MV systems. We'd like to host our Business Layer on the MV system and serve It to our e-commerce portals, instead of re-coding our business rules first in Basic, then in .Net In order to get there though we must meet the primary business requirement of zero downtime (not even 2 minutes to manually switch). We're not talking about different levels of Raid - it's assumed the storage array is up and available. If the MV system has a hiccup of more than a few seconds it needs to hot failover to a backup twin sister. Is anyone doing this or something close to it? When I worked in public safety, Stratus sold such an automatic hot failover. I'm sure the EnRoute folks are doing something like this still. Maybe Nick G. or Margaret M. is listening in today. Thanks, -Baker This communication, its contents and any file attachments transmitted with it are intended solely for the addressee(s) and may contain confidential proprietary information. Access by any other party without the express written permission of the sender is STRICTLY PROHIBITED. If you have received this communication in error you may not copy, distribute or use the contents, attachments or information in any way. Please destroy it and contact the sender. ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
Re: [U2] 24 X 7 MV systems
We are running an IBM high availability cluster of AIX machines which do auto fail-over. There is a couple minutes of time lag involved and there can be broken transactions since the switchover is OS level and not applications based so this is probably not a good solution for you since it sounds like you are looking for a truly fault tolerant solution. Stratus is still out there and probably a good choice for your needs. I know they have at least one series of boxes which run Red Hat and therefore are U2 compatible. I worked on a fault tolerant Sequoia system running Pick OA many years ago supporting an alarm monitoring application. Amazing machines. True Story: The operations manager gets a call from the computer operator who tells him Theres smoke coming out of one of the disk drawers on the Sequoia 'A' system (we had a second redundant Sequoia 'B' system as well). A couple quick phone calls later 5 of us are huddled together in the computer room on various phones with Sequoia after hitting the raid disk drawer in question with a fire extinguisher, trying to decide if we should switch over to the backup system, when out of the little glass cubicle where the operators live comes the operator on duty. He walks up to the smoking system, pulls off a spinning magnetic tape, and mounts the next reel of a file restore he's doing for someone. We all look at each other and laugh because the system is still running along just fine while on fire. Baker Hughes wrote: Hey y'all, I'm interested in hearing from folks who are currently on, or have worked with fault tolerant MV systems. We'd like to host our Business Layer on the MV system and serve It to our e-commerce portals, instead of re-coding our business rules first in Basic, then in .Net In order to get there though we must meet the primary business requirement of zero downtime (not even 2 minutes to manually switch). We're not talking about different levels of Raid - it's assumed the storage array is up and available. If the MV system has a hiccup of more than a few seconds it needs to hot failover to a backup twin sister. Is anyone doing this or something close to it? When I worked in public safety, Stratus sold such an automatic hot failover. I'm sure the EnRoute folks are doing something like this still. Maybe Nick G. or Margaret M. is listening in today. Thanks, -Baker This communication, its contents and any file attachments transmitted with it are intended solely for the addressee(s) and may contain confidential proprietary information. Access by any other party without the express written permission of the sender is STRICTLY PROHIBITED. If you have received this communication in error you may not copy, distribute or use the contents, attachments or information in any way. Please destroy it and contact the sender. ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users -- Jeff Schasny - Denver, Co, USA jschasny at gmail dot com ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
Re: [U2] 24 X 7 MV systems
Hi Baker: We have a warehouse that runs 24/7 and cannot be down, not even for backups. They are 100% Web based except for the handhelds which run on a telnet interface. The warehouse's backup machine is used by the warehouse's customers access to the their data via the internet. So the data must be current at all times and available should the main server suffer a hardware or software failure. Since we run on Unidata and Universe we had to have a solution that worked on both. We wrote replication technology that is handled in our middleware (U2WebLink) for all of our web applications. All changes to the database create a transaction record. There is a process always running that picks up those records sending them to the backup machine where the backup machine does the update. Our telnet programs create transactions as well and those are replicated to the other machine. We surround each write or delete in our Basic code with a call to a subroutine to capture the before and after images. In this case they manually switch the IP address to the new machine when we tested the worst case scenario, but this can be done with hardware. My client was not willing to purchase any more hardware to accomplish this switch over. Regards, Doug BTW: We are just finish up a Continuous Backup program that runs in Eclipse that continually copies data from one machine to another or can copy it up to the cloud company such as Amazon. You should be seeing announcement soon. -Original Message- From: u2-users-boun...@listserver.u2ug.org [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Baker Hughes Sent: Monday, July 26, 2010 7:38 AM To: 'U2 Users List' Subject: [U2] 24 X 7 MV systems Hey y'all, I'm interested in hearing from folks who are currently on, or have worked with fault tolerant MV systems. We'd like to host our Business Layer on the MV system and serve It to our e-commerce portals, instead of re-coding our business rules first in Basic, then in .Net In order to get there though we must meet the primary business requirement of zero downtime (not even 2 minutes to manually switch). We're not talking about different levels of Raid - it's assumed the storage array is up and available. If the MV system has a hiccup of more than a few seconds it needs to hot failover to a backup twin sister. Is anyone doing this or something close to it? When I worked in public safety, Stratus sold such an automatic hot failover. I'm sure the EnRoute folks are doing something like this still. Maybe Nick G. or Margaret M. is listening in today. Thanks, -Baker This communication, its contents and any file attachments transmitted with it are intended solely for the addressee(s) and may contain confidential proprietary information. Access by any other party without the express written permission of the sender is STRICTLY PROHIBITED. If you have received this communication in error you may not copy, distribute or use the contents, attachments or information in any way. Please destroy it and contact the sender. ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
Re: [U2] 24 X 7 MV systems
Glen, Thank you for the good observations, and suggestion to ping Ross, which I will do. It is the MV db paradigm which in this case is hampering us, which drives the solution to either the OS level or some middle ware solution. Pausing the db to flush memory is what keeps us just seconds from a full solution. Thank you. -Baker -Original Message- From: u2-users-boun...@listserver.u2ug.org [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Glen Batchelor Sent: Monday, July 26, 2010 10:03 AM To: 'U2 Users List' Subject: Re: [U2] 24 X 7 MV systems Hey Stranger, The best way you'll get there is with a transaction/request based redundancy setup. Does U2 have anything that isn't trigger related? Even a block-level DRDB config won't help with databases since transactions in memory aren't committed to disk promptly enough for the replicator to get all of the new data pieces as the fault happens. I've been looking for a remote hosted failover solution myself (not for U2). A truly fault tolerant setup requires that the incoming requests be replicated to multiple machines before anything happens inside. You might think of the user app as a web service consumer that makes requests to a proxy. That proxy mux's the requests to multiple machines, compares all of the responses and then passes back the response of one machine based on failover priority. If one of the responses aren't the same then an error is sent to the admin. Unfortunately, this spits in the eye of MV which is designed to be a stand-alone central data store. Maybe you can do it with your own TCP packets, but if you don't use an encrypted media you may get into security trouble. Web services are horribly bloated, but the security layer is already in there. You should bug Ross Ferris and pick his brain about his DRS product versus other options for U2. DRS supposedly uses a small amount of bandwidth but it isn't encrypted AFAIK. Regards, Glen Batchelor IT Director/CIO/CTO All-Spec Industries phone: (910) 332-0424 fax: (910) 763-5664 E-mail: webmas...@all-spec.com Web: http://www.all-spec.com Blog: http://blog.all-spec.com This communication, its contents and any file attachments transmitted with it are intended solely for the addressee(s) and may contain confidential proprietary information. Access by any other party without the express written permission of the sender is STRICTLY PROHIBITED. If you have received this communication in error you may not copy, distribute or use the contents, attachments or information in any way. Please destroy it and contact the sender. ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
Re: [U2] 24 X 7 MV systems
Jeff, Thank you for giving your experience with HA AIX and a cluster. We are also doing the HA but not clustering. The couple minutes time lag you mention and the possibility of broken transactions make one wonder if it's worth going that distance. I didn't know Stratus was still out there so thanks for that. What about Sequoia? That was also a very coveted system in the 911 offices back in the day. Awesome story about the Sequoia still tooling right along while the disk is on fire. So did the system switch itself over to 'B' or did y'all do it, when? I can't match that one, but even with Reality 7 (sorry to mention this on the U2 list) we could throw a manual switch (took me about 30 seconds to get from my office to the switch in the computer room) and when the dispatchers logged in, there were their sessions with screens looking identical to system A. That's why I gotta believe we can do better, 20 years later. Thank you. -Baker -Original Message- From: u2-users-boun...@listserver.u2ug.org [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Jeff Schasny Sent: Monday, July 26, 2010 10:24 AM To: U2 Users List Subject: Re: [U2] 24 X 7 MV systems We are running an IBM high availability cluster of AIX machines which do auto fail-over. There is a couple minutes of time lag involved and there can be broken transactions since the switchover is OS level and not applications based so this is probably not a good solution for you since it sounds like you are looking for a truly fault tolerant solution. Stratus is still out there and probably a good choice for your needs. I know they have at least one series of boxes which run Red Hat and therefore are U2 compatible. I worked on a fault tolerant Sequoia system running Pick OA many years ago supporting an alarm monitoring application. Amazing machines. True Story: The operations manager gets a call from the computer operator who tells him Theres smoke coming out of one of the disk drawers on the Sequoia 'A' system (we had a second redundant Sequoia 'B' system as well). A couple quick phone calls later 5 of us are huddled together in the computer room on various phones with Sequoia after hitting the raid disk drawer in question with a fire extinguisher, trying to decide if we should switch over to the backup system, when out of the little glass cubicle where the operators live comes the operator on duty. He walks up to the smoking system, pulls off a spinning magnetic tape, and mounts the next reel of a file restore he's doing for someone. We all look at each other and laugh because the system is still running along just fine while on fire. Baker Hughes wrote: Hey y'all, I'm interested in hearing from folks who are currently on, or have worked with fault tolerant MV systems. We'd like to host our Business Layer on the MV system and serve It to our e-commerce portals, instead of re-coding our business rules first in Basic, then in .Net In order to get there though we must meet the primary business requirement of zero downtime (not even 2 minutes to manually switch). We're not talking about different levels of Raid - it's assumed the storage array is up and available. If the MV system has a hiccup of more than a few seconds it needs to hot failover to a backup twin sister. Is anyone doing this or something close to it? When I worked in public safety, Stratus sold such an automatic hot failover. I'm sure the EnRoute folks are doing something like this still. Maybe Nick G. or Margaret M. is listening in today. Thanks, -Baker This communication, its contents and any file attachments transmitted with it are intended solely for the addressee(s) and may contain confidential proprietary information. Access by any other party without the express written permission of the sender is STRICTLY PROHIBITED. If you have received this communication in error you may not copy, distribute or use the contents, attachments or information in any way. Please destroy it and contact the sender. ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users -- Jeff Schasny - Denver, Co, USA jschasny at gmail dot com ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
Re: [U2] 24 X 7 MV systems
Baker, The A to B switchover on the Sequoias was manual and we did in fact do it after about 15 minutes. Only because it provided an excellent opportunity to do so. The Sequoia Field Engineer was on site with the new parts (it was one of the 2 power supplys for the drawer of Raid disks) within 30 minutes. As far as I know Sequoia is no more. I seem to remember they sold their hardware business unit to General Automation in the mid 1990's. Baker Hughes wrote: Jeff, Thank you for giving your experience with HA AIX and a cluster. We are also doing the HA but not clustering. The couple minutes time lag you mention and the possibility of broken transactions make one wonder if it's worth going that distance. I didn't know Stratus was still out there so thanks for that. What about Sequoia? That was also a very coveted system in the 911 offices back in the day. Awesome story about the Sequoia still tooling right along while the disk is on fire. So did the system switch itself over to 'B' or did y'all do it, when? I can't match that one, but even with Reality 7 (sorry to mention this on the U2 list) we could throw a manual switch (took me about 30 seconds to get from my office to the switch in the computer room) and when the dispatchers logged in, there were their sessions with screens looking identical to system A. That's why I gotta believe we can do better, 20 years later. Thank you. -Baker -Original Message- From: u2-users-boun...@listserver.u2ug.org [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Jeff Schasny Sent: Monday, July 26, 2010 10:24 AM To: U2 Users List Subject: Re: [U2] 24 X 7 MV systems We are running an IBM high availability cluster of AIX machines which do auto fail-over. There is a couple minutes of time lag involved and there can be broken transactions since the switchover is OS level and not applications based so this is probably not a good solution for you since it sounds like you are looking for a truly fault tolerant solution. Stratus is still out there and probably a good choice for your needs. I know they have at least one series of boxes which run Red Hat and therefore are U2 compatible. I worked on a fault tolerant Sequoia system running Pick OA many years ago supporting an alarm monitoring application. Amazing machines. True Story: The operations manager gets a call from the computer operator who tells him Theres smoke coming out of one of the disk drawers on the Sequoia 'A' system (we had a second redundant Sequoia 'B' system as well). A couple quick phone calls later 5 of us are huddled together in the computer room on various phones with Sequoia after hitting the raid disk drawer in question with a fire extinguisher, trying to decide if we should switch over to the backup system, when out of the little glass cubicle where the operators live comes the operator on duty. He walks up to the smoking system, pulls off a spinning magnetic tape, and mounts the next reel of a file restore he's doing for someone. We all look at each other and laugh because the system is still running along just fine while on fire. Baker Hughes wrote: Hey y'all, I'm interested in hearing from folks who are currently on, or have worked with fault tolerant MV systems. We'd like to host our Business Layer on the MV system and serve It to our e-commerce portals, instead of re-coding our business rules first in Basic, then in .Net In order to get there though we must meet the primary business requirement of zero downtime (not even 2 minutes to manually switch). We're not talking about different levels of Raid - it's assumed the storage array is up and available. If the MV system has a hiccup of more than a few seconds it needs to hot failover to a backup twin sister. Is anyone doing this or something close to it? When I worked in public safety, Stratus sold such an automatic hot failover. I'm sure the EnRoute folks are doing something like this still. Maybe Nick G. or Margaret M. is listening in today. Thanks, -Baker This communication, its contents and any file attachments transmitted with it are intended solely for the addressee(s) and may contain confidential proprietary information. Access by any other party without the express written permission of the sender is STRICTLY PROHIBITED. If you have received this communication in error you may not copy, distribute or use the contents, attachments or information in any way. Please destroy it and contact the sender. ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users -- Jeff Schasny - Denver, Co, USA jschasny at gmail dot com
Re: [U2] 24 X 7 MV systems
-Original Message- From: u2-users-boun...@listserver.u2ug.org [mailto:u2-users- boun...@listserver.u2ug.org] On Behalf Of Baker Hughes Sent: Monday, July 26, 2010 12:14 PM To: 'U2 Users List' Subject: Re: [U2] 24 X 7 MV systems Glen, Thank you for the good observations, and suggestion to ping Ross, which I will do. It is the MV db paradigm which in this case is hampering us, which drives the solution to either the OS level or some middle ware solution. Pausing the db to flush memory is what keeps us just seconds from a full solution. Thank you. -Baker Baker, Triggers may be the only sane option to tie into all of the services that modify the data. If you only had WRITE() to deal with then it'd be a no-brainer to just replace it with a custom write control and replication output routine. I know a small amount about U2, so I won't make any guesses on what's possible. As far as network failover, just rely on a local DNS service. You can use one local domain name and assign multiple A records. The resolver on the client machine should do the heavy lifting and resolve to the first IP that works. Just make sure that the failed node drops off of the network or you could run into latency issue with the resolver. Regards, Glen Batchelor IT Director/CIO/CTO All-Spec Industries phone: (910) 332-0424 fax: (910) 763-5664 E-mail: webmas...@all-spec.com Web: http://www.all-spec.com Blog: http://blog.all-spec.com ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
Re: [U2] 24 X 7 MV systems
From: Jeff Schasny As far as I know Sequoia is no more. I seem to remember they sold their hardware business unit to General Automation in the mid 1990's. To my understanding, the Sequoia platform became mvEnterprise, which is still maintained though not energetically marketed by TigerLogic Corp (home of D3 and mvBase). Without the same hardware base as Sequoia, I don't know how resilient the platform is to hardware issues. I've never heard of an RS6000 being as fault-resistant as Sequoia, even with HA AIX. (Note term fault-resistant compared to fault-tolerant.) As another Sequoia anecdote: An IT manager was surprised one day by a FedEx delivery of a motherboard. Apparently his system has lost a CPU and phoned to Support to get a new one, though no one at the company had noticed yet. The instructions were simply something like Remove bad board, insert new board. Twenty years later people are still wondering if it's possible to minimize downtime... As to Baker's original request: We'd like to host our Business Layer on the MV system and serve It to our e-commerce portals, instead of re-coding our business rules first in Basic, then in .Net You may be interested in my recent blogs (link below) on creating Web Services for MV BASIC. With what I've built so far, no changes to the BASIC code are required and there is no need to know anything about XML or .NET or SOAP or WSDL or any of that other stuff. To the point above, I've already implemented the basics for failovers, where failure to invoke a subroutine on any given server will cause a retry down a list of alternate servers - a notice will be sent to the IT admin but the client won't know anything has gone wrong. It's not tough code, it's just something that needs to be considered. Regards, Tony Gravagno Nebula Research and Development TG@ remove.pleaseNebula-RnD.com Nebula RD sells mv.NET and other Pick/MultiValue products worldwide, and provides related development services remove.pleaseNebula-RnD.com/blog Visit PickWiki.com! Contribute! http://Twitter.com/TonyGravagno ___ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
