Re: Backing up Q files but not the LOG files
Chris, Thanks a lot for this, it's quite useful. We believe we're OK for a local failover, the scenario we're trying to cater for is a computer center outage. In this scenario we want to recover all messages using a hot DR site. The circular logs were chosen previously to ease housekeeping and archiving, the logs are sized at about 2Gb per QM so they should be big enough. We don't have very large volumes or message sizes going through the system. Kulbir. Christopher Warneke [EMAIL PROTECTED] Sent by: MQSeries List [EMAIL PROTECTED] 18-Nov-2004 06:47 Please respond to MQSeries List [EMAIL PROTECTED] To: MQSERIES cc: Subject: Re: Backing up Q files but not the LOG files Kulbir, Persistent messages are written to the log, commited - then written to the queues. Without the log files, you risk losing inflight activity. Plain and simple. If the messages are not persistent - who cares if you loose them? If you have a redundant hardware solution, 2nd machine, mirrored DASD UPS, you likely have a robust enough configuration for all but total loss of the computer center. If this outage would have you moving operations to a DR site, how much inflight activity are you trying to protect? Given that you are running with circular logging, is it really necessary to try to come up at the DR site with all inflight activity recovered? If this is the case, is this DR site hot? If not, why wouldn't simple configuration backup suffice? I suspect that you are wasting time trying to nickle-and-dime a solution that will likely give you a nasty suprise, should you be in the position of needing the level of recovery that you are seeking. That, or you don't really need to recover the in-flight traffic. Good luck, Chris --- Kulbir S. Thind [EMAIL PROTECTED] wrote: We're trying to ensure we don't lose messages in Disaster Recovery situations. Our failover solution seems OK, we have a SAN configured to hold all queue manager details and that is accessible by another machine in the Veritas cluster if we need to failover. For DR we're looking to use another machine that is based 3,000 miles away, for this we need to minimise the amount of data we replicate, hence the reason why we're contemplating on replicating just the Q files and not LOG files. Cheers. Christopher Warneke [EMAIL PROTECTED] Sent by: MQSeries List [EMAIL PROTECTED] 17-Nov-2004 20:26 Please respond to MQSeries List [EMAIL PROTECTED] To: MQSERIES cc: Subject:Re: Backing up Q files but not the LOG files What I don't understand here, is the purpose of the replication. Are we trying to retain backups of the qmgr's objects, or both the objects and the messages? If you are only trying to retain copies of the objects... http://www-306.ibm.com/software/integration/support/supportpacs/category.html#cat2 MS03: WebSphere MQ - Save Queue Manager object definitions using PCFs Otherwise, investigate a mirrored DSAD solution, or; if you are using AIX, the HACMP supportpac... MC63: WebSphere MQ for AIX - Implementing with HACMP Maybe explain what your goal is, that has you trying this experiment. I'd like to understand what you are trying to accomplish. Thank you, Chris --- Kulbir S. Thind [EMAIL PROTECTED] wrote: Hi, We're thinking about setting up replication for our queue manager to ensure that just the Q files are backed up but not the LOG files for the queue manager. This is being contemplated as it appears the log files are updated far more than the Q files as the LOG files are doing internal MQSeries checks, etc. We need to reduce the amount of data that we're replicating. My first thoughts on this were that this would not work as the Q files would be out of sync with the LOG files. However we performed the following: Created a queue manager Started the queue manager Created some queues Put some message on the queues Ended the queue manager Took a copy of the LOG files used by the queue manager Started the queue manager Put some more messages on the queues Created another queue object Ended the queue manager Restored the LOG files that were backed up in step 6 Started the queue manager At this point I was expecting issues but I found that the queue manager started without problems and it recognised the queue that was created in step 9. Does this mean if we took a copy of the LOG files and restored them to a queue manager at a later point retaining the latest Q files we would have no problems? Has anyone tried this or no of any problems? Cheers, Kulbir. Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com Archive: http://vm.akh-wien.ac.at/MQSeries.archive Instructions for managing your mailing list subscription are provided in the Listserv General
Re: Backing up Q files but not the LOG files
I gueess it would all come down to your Business description of DR. For some companies the are glad to be back up. In other cases they want to be back up as near to, if not exactly, like production when they went down. This is where a replication of the Logs and Queue Data directories are required. Just backing up the OBJECTS, which BTW isn't the only thing you need to do, is just going to give you back your QMGR. But without the state of data when it went down what good does that do your business. This is where the SLA requirements by them are very important. It transcribes down to for them. you want a car with all the optionsget out the checkbook. These requirements are SOMETIMES give anbd take depending on the business requirements and hand in hand with the size of the pocketbook. bobbee From: Christopher Warneke [EMAIL PROTECTED] Reply-To: MQSeries List [EMAIL PROTECTED] To: [EMAIL PROTECTED] Subject: Re: Backing up Q files but not the LOG files Date: Wed, 17 Nov 2004 12:26:08 -0800 What I don't understand here, is the purpose of the replication. Are we trying to retain backups of the qmgr's objects, or both the objects and the messages? If you are only trying to retain copies of the objects... http://www-306.ibm.com/software/integration/support/supportpacs/category.html#cat2 MS03: WebSphere MQ - Save Queue Manager object definitions using PCFs Otherwise, investigate a mirrored DSAD solution, or; if you are using AIX, the HACMP supportpac... MC63: WebSphere MQ for AIX - Implementing with HACMP Maybe explain what your goal is, that has you trying this experiment. I'd like to understand what you are trying to accomplish. Thank you, Chris --- Kulbir S. Thind [EMAIL PROTECTED] wrote: Hi, We're thinking about setting up replication for our queue manager to ensure that just the Q files are backed up but not the LOG files for the queue manager. This is being contemplated as it appears the log files are updated far more than the Q files as the LOG files are doing internal MQSeries checks, etc. We need to reduce the amount of data that we're replicating. My first thoughts on this were that this would not work as the Q files would be out of sync with the LOG files. However we performed the following: Created a queue manager Started the queue manager Created some queues Put some message on the queues Ended the queue manager Took a copy of the LOG files used by the queue manager Started the queue manager Put some more messages on the queues Created another queue object Ended the queue manager Restored the LOG files that were backed up in step 6 Started the queue manager At this point I was expecting issues but I found that the queue manager started without problems and it recognised the queue that was created in step 9. Does this mean if we took a copy of the LOG files and restored them to a queue manager at a later point retaining the latest Q files we would have no problems? Has anyone tried this or no of any problems? Cheers, Kulbir. Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com Archive: http://vm.akh-wien.ac.at/MQSeries.archive Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com Archive: http://vm.akh-wien.ac.at/MQSeries.archive
Re: Backing up Q files but not the LOG files
Hi, I would most definitely ask your management the following question (and get their answer in writing - email): 'Would the business unit(s) be ok if during a DR the queue files became unused (messages were lost) and ONLY the MQ objects (queues, channels, etc..) were created on the DR box?' Because if management's doesn't ok it, it will be YOUR butt on the line and not theirs. Regards, Roger Lacroix Capitalware Inc. http://www.capitalware.biz Quoting Kulbir S. Thind [EMAIL PROTECTED]: Chris, Thanks a lot for this, it's quite useful. We believe we're OK for a local failover, the scenario we're trying to cater for is a computer center outage. In this scenario we want to recover all messages using a hot DR site. The circular logs were chosen previously to ease housekeeping and archiving, the logs are sized at about 2Gb per QM so they should be big enough. We don't have very large volumes or message sizes going through the system. Kulbir. Christopher Warneke [EMAIL PROTECTED] Sent by: MQSeries List [EMAIL PROTECTED] 18-Nov-2004 06:47 Please respond to MQSeries List [EMAIL PROTECTED] To: MQSERIES cc: Subject:Re: Backing up Q files but not the LOG files Kulbir, Persistent messages are written to the log, commited - then written to the queues. Without the log files, you risk losing inflight activity. Plain and simple. If the messages are not persistent - who cares if you loose them? If you have a redundant hardware solution, 2nd machine, mirrored DASD UPS, you likely have a robust enough configuration for all but total loss of the computer center. If this outage would have you moving operations to a DR site, how much inflight activity are you trying to protect? Given that you are running with circular logging, is it really necessary to try to come up at the DR site with all inflight activity recovered? If this is the case, is this DR site hot? If not, why wouldn't simple configuration backup suffice? I suspect that you are wasting time trying to nickle-and-dime a solution that will likely give you a nasty suprise, should you be in the position of needing the level of recovery that you are seeking. That, or you don't really need to recover the in-flight traffic. Good luck, Chris --- Kulbir S. Thind [EMAIL PROTECTED] wrote: We're trying to ensure we don't lose messages in Disaster Recovery situations. Our failover solution seems OK, we have a SAN configured to hold all queue manager details and that is accessible by another machine in the Veritas cluster if we need to failover. For DR we're looking to use another machine that is based 3,000 miles away, for this we need to minimise the amount of data we replicate, hence the reason why we're contemplating on replicating just the Q files and not LOG files. Cheers. Christopher Warneke [EMAIL PROTECTED] Sent by: MQSeries List [EMAIL PROTECTED] 17-Nov-2004 20:26 Please respond to MQSeries List [EMAIL PROTECTED] To: MQSERIES cc: Subject:Re: Backing up Q files but not the LOG files What I don't understand here, is the purpose of the replication. Are we trying to retain backups of the qmgr's objects, or both the objects and the messages? If you are only trying to retain copies of the objects... http://www-306.ibm.com/software/integration/support/supportpacs/category.html#cat2 MS03: WebSphere MQ - Save Queue Manager object definitions using PCFs Otherwise, investigate a mirrored DSAD solution, or; if you are using AIX, the HACMP supportpac... MC63: WebSphere MQ for AIX - Implementing with HACMP Maybe explain what your goal is, that has you trying this experiment. I'd like to understand what you are trying to accomplish. Thank you, Chris --- Kulbir S. Thind [EMAIL PROTECTED] wrote: Hi, We're thinking about setting up replication for our queue manager to ensure that just the Q files are backed up but not the LOG files for the queue manager. This is being contemplated as it appears the log files are updated far more than the Q files as the LOG files are doing internal MQSeries checks, etc. We need to reduce the amount of data that we're replicating. My first thoughts on this were that this would not work as the Q files would be out of sync with the LOG files. However we performed the following: Created a queue manager Started the queue manager Created some queues Put some message on the queues Ended the queue manager Took a copy of the LOG files used by the queue manager Started the queue manager Put some more messages on the queues Created another queue object Ended the queue manager Restored the LOG files that were backed up in step 6 Started the queue manager At this point I
Re: Backing up Q files but not the LOG files
A M E N to that brother! ! ! From: Roger Lacroix [EMAIL PROTECTED] Reply-To: MQSeries List [EMAIL PROTECTED] To: [EMAIL PROTECTED] Subject: Re: Backing up Q files but not the LOG files Date: Thu, 18 Nov 2004 09:27:35 -0500 Hi, I would most definitely ask your management the following question (and get their answer in writing - email): 'Would the business unit(s) be ok if during a DR the queue files became unused (messages were lost) and ONLY the MQ objects (queues, channels, etc..) were created on the DR box?' Because if management's doesn't ok it, it will be YOUR butt on the line and not theirs. Regards, Roger Lacroix Capitalware Inc. http://www.capitalware.biz Quoting Kulbir S. Thind [EMAIL PROTECTED]: Chris, Thanks a lot for this, it's quite useful. We believe we're OK for a local failover, the scenario we're trying to cater for is a computer center outage. In this scenario we want to recover all messages using a hot DR site. The circular logs were chosen previously to ease housekeeping and archiving, the logs are sized at about 2Gb per QM so they should be big enough. We don't have very large volumes or message sizes going through the system. Kulbir. Christopher Warneke [EMAIL PROTECTED] Sent by: MQSeries List [EMAIL PROTECTED] 18-Nov-2004 06:47 Please respond to MQSeries List [EMAIL PROTECTED] To: MQSERIES cc: Subject:Re: Backing up Q files but not the LOG files Kulbir, Persistent messages are written to the log, commited - then written to the queues. Without the log files, you risk losing inflight activity. Plain and simple. If the messages are not persistent - who cares if you loose them? If you have a redundant hardware solution, 2nd machine, mirrored DASD UPS, you likely have a robust enough configuration for all but total loss of the computer center. If this outage would have you moving operations to a DR site, how much inflight activity are you trying to protect? Given that you are running with circular logging, is it really necessary to try to come up at the DR site with all inflight activity recovered? If this is the case, is this DR site hot? If not, why wouldn't simple configuration backup suffice? I suspect that you are wasting time trying to nickle-and-dime a solution that will likely give you a nasty suprise, should you be in the position of needing the level of recovery that you are seeking. That, or you don't really need to recover the in-flight traffic. Good luck, Chris --- Kulbir S. Thind [EMAIL PROTECTED] wrote: We're trying to ensure we don't lose messages in Disaster Recovery situations. Our failover solution seems OK, we have a SAN configured to hold all queue manager details and that is accessible by another machine in the Veritas cluster if we need to failover. For DR we're looking to use another machine that is based 3,000 miles away, for this we need to minimise the amount of data we replicate, hence the reason why we're contemplating on replicating just the Q files and not LOG files. Cheers. Christopher Warneke [EMAIL PROTECTED] Sent by: MQSeries List [EMAIL PROTECTED] 17-Nov-2004 20:26 Please respond to MQSeries List [EMAIL PROTECTED] To: MQSERIES cc: Subject:Re: Backing up Q files but not the LOG files What I don't understand here, is the purpose of the replication. Are we trying to retain backups of the qmgr's objects, or both the objects and the messages? If you are only trying to retain copies of the objects... http://www-306.ibm.com/software/integration/support/supportpacs/category.html#cat2 MS03: WebSphere MQ - Save Queue Manager object definitions using PCFs Otherwise, investigate a mirrored DSAD solution, or; if you are using AIX, the HACMP supportpac... MC63: WebSphere MQ for AIX - Implementing with HACMP Maybe explain what your goal is, that has you trying this experiment. I'd like to understand what you are trying to accomplish. Thank you, Chris --- Kulbir S. Thind [EMAIL PROTECTED] wrote: Hi, We're thinking about setting up replication for our queue manager to ensure that just the Q files are backed up but not the LOG files for the queue manager. This is being contemplated as it appears the log files are updated far more than the Q files as the LOG files are doing internal MQSeries checks, etc. We need to reduce the amount of data that we're replicating. My first thoughts on this were that this would not work as the Q files would be out of sync with the LOG files. However we performed the following: Created a queue manager Started the queue manager Created some queues Put some message on the queues Ended the queue manager Took a copy of the LOG files used by the queue manager Started the queue
Re: Backing up Q files but not the LOG files
Kulbir, Sounds like what you really need is a redundant DASD solution and a DR center that is close enough to you to mirror disks at both sites. EMC and IBM both have technologies that support having the data centers some distance from each other. Shared Queues on z/OS can be 27km apart (I've heard that IBM has extended this, not sure how far). QSGs employ m/f coupling facilities so, my bet is that the Mirrored DASD solutions require closer proximity, for performance sake. Years ago, 2001, I worked for a bank that had a primary data center in Jersey City, NJ and a secondary data center in Manhattan, NYC. Several miles of fiberconn connected the 2 data centers. EMC DASD was mirrored at both sites. Both OS/390 and AIX 4.3.3 machines were supported. Failover worked quite well - the solution was without significant performance impact. I'm not sure that the solution that you're attempting to implement will perform well over 3k miles. What kind of DASD solution are you employing? What vendor/product? Do you have databases that are being replicated across both sites as well? Lemmeknow, Chris --- Kulbir S. Thind [EMAIL PROTECTED] wrote: Chris, Thanks a lot for this, it's quite useful. We believe we're OK for a local failover, the scenario we're trying to cater for is a computer center outage. In this scenario we want to recover all messages using a hot DR site. The circular logs were chosen previously to ease housekeeping and archiving, the logs are sized at about 2Gb per QM so they should be big enough. We don't have very large volumes or message sizes going through the system. Kulbir. Christopher Warneke [EMAIL PROTECTED] Sent by: MQSeries List [EMAIL PROTECTED] 18-Nov-2004 06:47 Please respond to MQSeries List [EMAIL PROTECTED] To: MQSERIES cc: Subject:Re: Backing up Q files but not the LOG files Kulbir, Persistent messages are written to the log, commited - then written to the queues. Without the log files, you risk losing inflight activity. Plain and simple. If the messages are not persistent - who cares if you loose them? If you have a redundant hardware solution, 2nd machine, mirrored DASD UPS, you likely have a robust enough configuration for all but total loss of the computer center. If this outage would have you moving operations to a DR site, how much inflight activity are you trying to protect? Given that you are running with circular logging, is it really necessary to try to come up at the DR site with all inflight activity recovered? If this is the case, is this DR site hot? If not, why wouldn't simple configuration backup suffice? I suspect that you are wasting time trying to nickle-and-dime a solution that will likely give you a nasty suprise, should you be in the position of needing the level of recovery that you are seeking. That, or you don't really need to recover the in-flight traffic. Good luck, Chris --- Kulbir S. Thind [EMAIL PROTECTED] wrote: We're trying to ensure we don't lose messages in Disaster Recovery situations. Our failover solution seems OK, we have a SAN configured to hold all queue manager details and that is accessible by another machine in the Veritas cluster if we need to failover. For DR we're looking to use another machine that is based 3,000 miles away, for this we need to minimise the amount of data we replicate, hence the reason why we're contemplating on replicating just the Q files and not LOG files. Cheers. Christopher Warneke [EMAIL PROTECTED] Sent by: MQSeries List [EMAIL PROTECTED] 17-Nov-2004 20:26 Please respond to MQSeries List [EMAIL PROTECTED] To: MQSERIES cc: Subject:Re: Backing up Q files but not the LOG files What I don't understand here, is the purpose of the replication. Are we trying to retain backups of the qmgr's objects, or both the objects and the messages? If you are only trying to retain copies of the objects... http://www-306.ibm.com/software/integration/support/supportpacs/category.html#cat2 MS03: WebSphere MQ - Save Queue Manager object definitions using PCFs Otherwise, investigate a mirrored DSAD solution, or; if you are using AIX, the HACMP supportpac... MC63: WebSphere MQ for AIX - Implementing with HACMP Maybe explain what your goal is, that has you trying this experiment. I'd like to understand what you are trying to accomplish. Thank you, Chris --- Kulbir S. Thind [EMAIL PROTECTED] wrote: Hi, We're thinking about setting up replication for our queue manager to ensure that just the Q files are backed up but not the LOG files for the queue manager. This is being contemplated as it appears the log files are updated far more than the Q files as the LOG files are doing internal MQSeries
Re: Backing up Q files but not the LOG files
IBM's fiber connections can now be 32 miles long. We mirror (not through ficon) our mainframes to both Long Island and Maryland. Ò¿Ó Bob Juch Citigroup MQ Mainframe Support Team Weehawken, NJ 201-974-2147 mailto:[EMAIL PROTECTED] -Original Message- From: MQSeries List [mailto:[EMAIL PROTECTED] Behalf Of Christopher Warneke Sent: Thursday, November 18, 2004 10:58 AM To: [EMAIL PROTECTED] Subject: Re: Backing up Q files but not the LOG files Kulbir, Sounds like what you really need is a redundant DASD solution and a DR center that is close enough to you to mirror disks at both sites. EMC and IBM both have technologies that support having the data centers some distance from each other. Shared Queues on z/OS can be 27km apart (I've heard that IBM has extended this, not sure how far). QSGs employ m/f coupling facilities so, my bet is that the Mirrored DASD solutions require closer proximity, for performance sake. Years ago, 2001, I worked for a bank that had a primary data center in Jersey City, NJ and a secondary data center in Manhattan, NYC. Several miles of fiberconn connected the 2 data centers. EMC DASD was mirrored at both sites. Both OS/390 and AIX 4.3.3 machines were supported. Failover worked quite well - the solution was without significant performance impact. I'm not sure that the solution that you're attempting to implement will perform well over 3k miles. What kind of DASD solution are you employing? What vendor/product? Do you have databases that are being replicated across both sites as well? Lemmeknow, Chris Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com Archive: http://vm.akh-wien.ac.at/MQSeries.archive
Re: Backing up Q files but not the LOG files
I believe you also do it between Delaware, and Wehawkin, NJ. Which makes me wonder about the 32 mile statement?? I know the replication was there. I may be incorrect about the geog. bobbee From: Juch, Robert [EMAIL PROTECTED] Reply-To: MQSeries List [EMAIL PROTECTED] To: [EMAIL PROTECTED] Subject: Re: Backing up Q files but not the LOG files Date: Thu, 18 Nov 2004 11:44:31 -0500 IBM's fiber connections can now be 32 miles long. We mirror (not through ficon) our mainframes to both Long Island and Maryland. R?S Bob Juch Citigroup MQ Mainframe Support Team Weehawken, NJ 201-974-2147 mailto:[EMAIL PROTECTED] -Original Message- From: MQSeries List [mailto:[EMAIL PROTECTED] Behalf Of Christopher Warneke Sent: Thursday, November 18, 2004 10:58 AM To: [EMAIL PROTECTED] Subject: Re: Backing up Q files but not the LOG files Kulbir, Sounds like what you really need is a redundant DASD solution and a DR center that is close enough to you to mirror disks at both sites. EMC and IBM both have technologies that support having the data centers some distance from each other. Shared Queues on z/OS can be 27km apart (I've heard that IBM has extended this, not sure how far). QSGs employ m/f coupling facilities so, my bet is that the Mirrored DASD solutions require closer proximity, for performance sake. Years ago, 2001, I worked for a bank that had a primary data center in Jersey City, NJ and a secondary data center in Manhattan, NYC. Several miles of fiberconn connected the 2 data centers. EMC DASD was mirrored at both sites. Both OS/390 and AIX 4.3.3 machines were supported. Failover worked quite well - the solution was without significant performance impact. I'm not sure that the solution that you're attempting to implement will perform well over 3k miles. What kind of DASD solution are you employing? What vendor/product? Do you have databases that are being replicated across both sites as well? Lemmeknow, Chris Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com Archive: http://vm.akh-wien.ac.at/MQSeries.archive Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com Archive: http://vm.akh-wien.ac.at/MQSeries.archive
Re: Backing up Q files but not the LOG files
Thank you Bob. Chris --- Juch, Robert [EMAIL PROTECTED] wrote: IBM's fiber connections can now be 32 miles long. We mirror (not through ficon) our mainframes to both Long Island and Maryland. R?S Bob Juch Citigroup MQ Mainframe Support Team Weehawken, NJ 201-974-2147 mailto:[EMAIL PROTECTED] -Original Message- From: MQSeries List [mailto:[EMAIL PROTECTED] Behalf Of Christopher Warneke Sent: Thursday, November 18, 2004 10:58 AM To: [EMAIL PROTECTED] Subject: Re: Backing up Q files but not the LOG files Kulbir, Sounds like what you really need is a redundant DASD solution and a DR center that is close enough to you to mirror disks at both sites. EMC and IBM both have technologies that support having the data centers some distance from each other. Shared Queues on z/OS can be 27km apart (I've heard that IBM has extended this, not sure how far). QSGs employ m/f coupling facilities so, my bet is that the Mirrored DASD solutions require closer proximity, for performance sake. Years ago, 2001, I worked for a bank that had a primary data center in Jersey City, NJ and a secondary data center in Manhattan, NYC. Several miles of fiberconn connected the 2 data centers. EMC DASD was mirrored at both sites. Both OS/390 and AIX 4.3.3 machines were supported. Failover worked quite well - the solution was without significant performance impact. I'm not sure that the solution that you're attempting to implement will perform well over 3k miles. What kind of DASD solution are you employing? What vendor/product? Do you have databases that are being replicated across both sites as well? Lemmeknow, Chris Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com Archive: http://vm.akh-wien.ac.at/MQSeries.archive Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com Archive: http://vm.akh-wien.ac.at/MQSeries.archive
Re: Backing up Q files but not the LOG files
Bob, You've got to fix that sticky e key! We don't use ficon for the replication but use high-speed lines. Not that it matters, but the Delaware mainframes are in Maryland. Ò¿Ó Bob Juch Citigroup MQ Mainframe Support Team Weehawken, NJ 201-974-2147 mailto:[EMAIL PROTECTED] -Original Message- From: MQSeries List [mailto:[EMAIL PROTECTED] Behalf Of Robert Broderick Sent: Thursday, November 18, 2004 1:44 PM To: [EMAIL PROTECTED] Subject: Re: Backing up Q files but not the LOG files I believe you also do it between Delaware, and Wehawkin, NJ. Which makes me wonder about the 32 mile statement?? I know the replication was there. I may be incorrect about the geog. bobbee From: Juch, Robert [EMAIL PROTECTED] Reply-To: MQSeries List [EMAIL PROTECTED] To: [EMAIL PROTECTED] Subject: Re: Backing up Q files but not the LOG files Date: Thu, 18 Nov 2004 11:44:31 -0500 IBM's fiber connections can now be 32 miles long. We mirror (not through ficon) our mainframes to both Long Island and Maryland. R?S Bob Juch Citigroup MQ Mainframe Support Team Weehawken, NJ 201-974-2147 mailto:[EMAIL PROTECTED] Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com Archive: http://vm.akh-wien.ac.at/MQSeries.archive
Re: Backing up Q files but not the LOG files
Chris, We're using Veritas Volume Replicator (VVR) and we're replicating the Oracle database as well. Regards, Kulbir. Christopher Warneke [EMAIL PROTECTED] Sent by: MQSeries List [EMAIL PROTECTED] 18-Nov-2004 15:57 Please respond to MQSeries List [EMAIL PROTECTED] To: MQSERIES cc: Subject: Re: Backing up Q files but not the LOG files Kulbir, Sounds like what you really need is a redundant DASD solution and a DR center that is close enough to you to mirror disks at both sites. EMC and IBM both have technologies that support having the data centers some distance from each other. Shared Queues on z/OS can be 27km apart (I've heard that IBM has extended this, not sure how far). QSGs employ m/f coupling facilities so, my bet is that the Mirrored DASD solutions require closer proximity, for performance sake. Years ago, 2001, I worked for a bank that had a primary data center in Jersey City, NJ and a secondary data center in Manhattan, NYC. Several miles of fiberconn connected the 2 data centers. EMC DASD was mirrored at both sites. Both OS/390 and AIX 4.3.3 machines were supported. Failover worked quite well - the solution was without significant performance impact. I'm not sure that the solution that you're attempting to implement will perform well over 3k miles. What kind of DASD solution are you employing? What vendor/product? Do you have databases that are being replicated across both sites as well? Lemmeknow, Chris --- Kulbir S. Thind [EMAIL PROTECTED] wrote: Chris, Thanks a lot for this, it's quite useful. We believe we're OK for a local failover, the scenario we're trying to cater for is a computer center outage. In this scenario we want to recover all messages using a hot DR site. The circular logs were chosen previously to ease housekeeping and archiving, the logs are sized at about 2Gb per QM so they should be big enough. We don't have very large volumes or message sizes going through the system. Kulbir. Christopher Warneke [EMAIL PROTECTED] Sent by: MQSeries List [EMAIL PROTECTED] 18-Nov-2004 06:47 Please respond to MQSeries List [EMAIL PROTECTED] To: MQSERIES cc: Subject:Re: Backing up Q files but not the LOG files Kulbir, Persistent messages are written to the log, commited - then written to the queues. Without the log files, you risk losing inflight activity. Plain and simple. If the messages are not persistent - who cares if you loose them? If you have a redundant hardware solution, 2nd machine, mirrored DASD UPS, you likely have a robust enough configuration for all but total loss of the computer center. If this outage would have you moving operations to a DR site, how much inflight activity are you trying to protect? Given that you are running with circular logging, is it really necessary to try to come up at the DR site with all inflight activity recovered? If this is the case, is this DR site hot? If not, why wouldn't simple configuration backup suffice? I suspect that you are wasting time trying to nickle-and-dime a solution that will likely give you a nasty suprise, should you be in the position of needing the level of recovery that you are seeking. That, or you don't really need to recover the in-flight traffic. Good luck, Chris --- Kulbir S. Thind [EMAIL PROTECTED] wrote: We're trying to ensure we don't lose messages in Disaster Recovery situations. Our failover solution seems OK, we have a SAN configured to hold all queue manager details and that is accessible by another machine in the Veritas cluster if we need to failover. For DR we're looking to use another machine that is based 3,000 miles away, for this we need to minimise the amount of data we replicate, hence the reason why we're contemplating on replicating just the Q files and not LOG files. Cheers. Christopher Warneke [EMAIL PROTECTED] Sent by: MQSeries List [EMAIL PROTECTED] 17-Nov-2004 20:26 Please respond to MQSeries List [EMAIL PROTECTED] To: MQSERIES cc: Subject:Re: Backing up Q files but not the LOG files What I don't understand here, is the purpose of the replication. Are we trying to retain backups of the qmgr's objects, or both the objects and the messages? If you are only trying to retain copies of the objects... http://www-306.ibm.com/software/integration/support/supportpacs/category.html#cat2 MS03: WebSphere MQ - Save Queue Manager object definitions using PCFs Otherwise, investigate a mirrored DSAD solution, or; if you are using AIX, the HACMP supportpac... MC63: WebSphere MQ for AIX - Implementing with HACMP Maybe explain what your goal is, that has you trying this experiment. I'd like to understand what you are trying to accomplish. Thank you, Chris --- Kulbir S. Thind [EMAIL PROTECTED] wrote: Hi
Re: Backing up Q files but not the LOG files
Kulbir, You didn't say what platform you're doing this on. I do just mainframes, but expect you'd have the same problems on smaller ones. If you don't replicate the log files and you have any persistent messages on your queues when you go through your exercise below, then you'll have problems. MQ needs the log files then. If you get out of sync, you'll never be able to bring up your queue manager. Ò¿ÓBob JuchCitigroupMQ Mainframe Support TeamWeehawken, NJ201-974-2147mailto:[EMAIL PROTECTED] -Original Message-From: MQSeries List [mailto:[EMAIL PROTECTED]On Behalf Of Kulbir S. ThindSent: Wednesday, November 17, 2004 12:05 PMTo: [EMAIL PROTECTED]Subject: Backing up "Q" files but not the "LOG" filesHi, We're thinking about setting up replication for our queue manager to ensure that just the Q files are backed up but not the LOG files for the queue manager. This is being contemplated as it appears the log files are updated far more than the Q files as the LOG files are doing internal MQSeries checks, etc. We need to reduce the amount of data that we're replicating. My first thoughts on this were that this would not work as the Q files would be out of sync with the LOG files. However we performed the following: Created a queue manager Started the queue manager Created some queues Put some message on the queues Ended the queue manager Took a copy of the LOG files used by the queue manager Started the queue manager Put some more messages on the queues Created another queue object Ended the queue manager Restored the LOG files that were backed up in step 6 Started the queue managerAt this point I was expecting issues but I found that the queue manager started without problems and it recognised the queue that was created in step 9. Does this mean if we took a copy of the LOG files and restored them to a queue manager at a later point retaining the latest Q files we would have no problems? Has anyone tried this or no of any problems? Cheers, Kulbir.
Re: Backing up Q files but not the LOG files
You should check with IBM, however... The technique will work, but if an object is broken, then it will likely be broken forever. Also, there's no guarantee the queue file will accurately reflect the state of the queue. That is, there may be messages that would have been backed out and synced with the log, but not yet on the queue file. The point is if IBM didn't think the log file was needed, then they wouldn't have created it. You may be better off keeping the LOG files and not the queue files. Then at least, you can rcrmqobj the queue files once the queue manager is running. Also, if you issued the rcdmqobj after your step 12, it should restore the queue back to step 6 You are using Linear logs right ? Kulbir S. Thind [EMAIL PROTECTED]To: [EMAIL PROTECTED] SK.COM cc: Sent by: MQSeriesSubject: Backing up Q files but not the LOG files List [EMAIL PROTECTED] n.ac.at 11/17/2004 12:04 PM Please respond to MQSeries List Hi, We're thinking about setting up replication for our queue manager to ensure that just the Q files are backed up but not the LOG files for the queue manager. This is being contemplated as it appears the log files are updated far more than the Q files as the LOG files are doing internal MQSeries checks, etc. We need to reduce the amount of data that we're replicating. My first thoughts on this were that this would not work as the Q files would be out of sync with the LOG files. However we performed the following: 1. Created a queue manager 2. Started the queue manager 3. Created some queues 4. Put some message on the queues 5. Ended the queue manager 6. Took a copy of the LOG files used by the queue manager 7. Started the queue manager 8. Put some more messages on the queues 9. Created another queue object 10. Ended the queue manager 11. Restored the LOG files that were backed up in step 6 12. Started the queue manager At this point I was expecting issues but I found that the queue manager started without problems and it recognised the queue that was created in step 9. Does this mean if we took a copy of the LOG files and restored them to a queue manager at a later point retaining the latest Q files we would have no problems? Has anyone tried this or no of any problems? Cheers, Kulbir. Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com Archive: http://vm.akh-wien.ac.at/MQSeries.archive
Re: Backing up Q files but not the LOG files
Kulbir, Again, I do mainframes. On the mainframe, if you have an out-of-sync condition between your page datasets and your logs, the queue manager will abend with a s6C6. I'm not sure if a queue manager would stay up on an small platform or not. If you don't replicate the queue datasets and only the logs, you would then need EVERY log ever created to restore the queues. Ò¿Ó Bob Juch Citigroup MQ Mainframe Support Team Weehawken, NJ 201-974-2147 mailto:[EMAIL PROTECTED] -Original Message- From: MQSeries List [mailto:[EMAIL PROTECTED] Behalf Of [EMAIL PROTECTED] Sent: Wednesday, November 17, 2004 1:21 PM To: [EMAIL PROTECTED] Subject: Re: Backing up Q files but not the LOG files You should check with IBM, however... The technique will work, but if an object is broken, then it will likely be broken forever. Also, there's no guarantee the queue file will accurately reflect the state of the queue. That is, there may be messages that would have been backed out and synced with the log, but not yet on the queue file. The point is if IBM didn't think the log file was needed, then they wouldn't have created it. You may be better off keeping the LOG files and not the queue files. Then at least, you can rcrmqobj the queue files once the queue manager is running. Also, if you issued the rcdmqobj after your step 12, it should restore the queue back to step 6 You are using Linear logs right ? Kulbir S. Thind [EMAIL PROTECTED]To: [EMAIL PROTECTED] SK.COM cc: Sent by: MQSeriesSubject: Backing up Q files but not the LOG files List [EMAIL PROTECTED] n.ac.at 11/17/2004 12:04 PM Please respond to MQSeries List Hi, We're thinking about setting up replication for our queue manager to ensure that just the Q files are backed up but not the LOG files for the queue manager. This is being contemplated as it appears the log files are updated far more than the Q files as the LOG files are doing internal MQSeries checks, etc. We need to reduce the amount of data that we're replicating. My first thoughts on this were that this would not work as the Q files would be out of sync with the LOG files. However we performed the following: 1. Created a queue manager 2. Started the queue manager 3. Created some queues 4. Put some message on the queues 5. Ended the queue manager 6. Took a copy of the LOG files used by the queue manager 7. Started the queue manager 8. Put some more messages on the queues 9. Created another queue object 10. Ended the queue manager 11. Restored the LOG files that were backed up in step 6 12. Started the queue manager At this point I was expecting issues but I found that the queue manager started without problems and it recognised the queue that was created in step 9. Does this mean if we took a copy of the LOG files and restored them to a queue manager at a later point retaining the latest Q files we would have no problems? Has anyone tried this or no of any problems? Cheers, Kulbir. Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com Archive: http://vm.akh-wien.ac.at/MQSeries.archive Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com Archive: http://vm.akh-wien.ac.at/MQSeries.archive
Re: Backing up Q files but not the LOG files
What I don't understand here, is the purpose of the replication. Are we trying to retain backups of the qmgr's objects, or both the objects and the messages? If you are only trying to retain copies of the objects... http://www-306.ibm.com/software/integration/support/supportpacs/category.html#cat2 MS03: WebSphere MQ - Save Queue Manager object definitions using PCFs Otherwise, investigate a mirrored DSAD solution, or; if you are using AIX, the HACMP supportpac... MC63: WebSphere MQ for AIX - Implementing with HACMP Maybe explain what your goal is, that has you trying this experiment. I'd like to understand what you are trying to accomplish. Thank you, Chris --- Kulbir S. Thind [EMAIL PROTECTED] wrote: Hi, We're thinking about setting up replication for our queue manager to ensure that just the Q files are backed up but not the LOG files for the queue manager. This is being contemplated as it appears the log files are updated far more than the Q files as the LOG files are doing internal MQSeries checks, etc. We need to reduce the amount of data that we're replicating. My first thoughts on this were that this would not work as the Q files would be out of sync with the LOG files. However we performed the following: Created a queue manager Started the queue manager Created some queues Put some message on the queues Ended the queue manager Took a copy of the LOG files used by the queue manager Started the queue manager Put some more messages on the queues Created another queue object Ended the queue manager Restored the LOG files that were backed up in step 6 Started the queue manager At this point I was expecting issues but I found that the queue manager started without problems and it recognised the queue that was created in step 9. Does this mean if we took a copy of the LOG files and restored them to a queue manager at a later point retaining the latest Q files we would have no problems? Has anyone tried this or no of any problems? Cheers, Kulbir. Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com Archive: http://vm.akh-wien.ac.at/MQSeries.archive
Re: Backing up Q files but not the LOG files
Thanks for the replies. We're testing this on Windows but are contemplating on doing this on Sun Solaris. We're using circular logging. I'm surprised the sequence of events worked, especially as we were using persistent messages. The reason we don't want to replicate the LOGs is because they seem to be involved in a lot more disk I/O and therefore we need to replicate more. Can anyone think of scenarios that this approach would definitely not work in or scenarios I should try? Thanks. Juch, Robert [EMAIL PROTECTED] Sent by: MQSeries List [EMAIL PROTECTED] 17-Nov-2004 19:40 Please respond to MQSeries List [EMAIL PROTECTED] To: MQSERIES cc: Subject: Re: Backing up Q files but not the LOG files Kulbir, Again, I do mainframes. On the mainframe, if you have an out-of-sync condition between your page datasets and your logs, the queue manager will abend with a s6C6. I'm not sure if a queue manager would stay up on an small platform or not. If you don't replicate the queue datasets and only the logs, you would then need EVERY log ever created to restore the queues. Ò¿Ó Bob Juch Citigroup MQ Mainframe Support Team Weehawken, NJ 201-974-2147 mailto:[EMAIL PROTECTED] -Original Message- From: MQSeries List [mailto:[EMAIL PROTECTED] Behalf Of [EMAIL PROTECTED] Sent: Wednesday, November 17, 2004 1:21 PM To: [EMAIL PROTECTED] Subject: Re: Backing up Q files but not the LOG files You should check with IBM, however... The technique will work, but if an object is broken, then it will likely be broken forever. Also, there's no guarantee the queue file will accurately reflect the state of the queue. That is, there may be messages that would have been backed out and synced with the log, but not yet on the queue file. The point is if IBM didn't think the log file was needed, then they wouldn't have created it. You may be better off keeping the LOG files and not the queue files. Then at least, you can rcrmqobj the queue files once the queue manager is running. Also, if you issued the rcdmqobj after your step 12, it should restore the queue back to step 6 You are using Linear logs right ? Kulbir S. Thind [EMAIL PROTECTED]To: [EMAIL PROTECTED] SK.COM cc: Sent by: MQSeriesSubject: Backing up Q files but not the LOG files List [EMAIL PROTECTED] n.ac.at 11/17/2004 12:04 PM Please respond to MQSeries List Hi, We're thinking about setting up replication for our queue manager to ensure that just the Q files are backed up but not the LOG files for the queue manager. This is being contemplated as it appears the log files are updated far more than the Q files as the LOG files are doing internal MQSeries checks, etc. We need to reduce the amount of data that we're replicating. My first thoughts on this were that this would not work as the Q files would be out of sync with the LOG files. However we performed the following: 1. Created a queue manager 2. Started the queue manager 3. Created some queues 4. Put some message on the queues 5. Ended the queue manager 6. Took a copy of the LOG files used by the queue manager 7. Started the queue manager 8. Put some more messages on the queues 9. Created another queue object 10. Ended the queue manager 11. Restored the LOG files that were backed up in step 6 12. Started the queue manager At this point I was expecting issues but I found that the queue manager started without problems and it recognised the queue that was created in step 9. Does this mean if we took a copy of the LOG files and restored them to a queue manager at a later point retaining the latest Q files we would have no problems? Has anyone tried this or no of any problems? Cheers, Kulbir. Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com Archive: http://vm.akh-wien.ac.at/MQSeries.archive Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com Archive: http://vm.akh-wien.ac.at/MQSeries.archive
Re: Backing up Q files but not the LOG files
We're trying to ensure we don't lose messages in Disaster Recovery situations. Our failover solution seems OK, we have a SAN configured to hold all queue manager details and that is accessible by another machine in the Veritas cluster if we need to failover. For DR we're looking to use another machine that is based 3,000 miles away, for this we need to minimise the amount of data we replicate, hence the reason why we're contemplating on replicating just the Q files and not LOG files. Cheers. Christopher Warneke [EMAIL PROTECTED] Sent by: MQSeries List [EMAIL PROTECTED] 17-Nov-2004 20:26 Please respond to MQSeries List [EMAIL PROTECTED] To: MQSERIES cc: Subject: Re: Backing up Q files but not the LOG files What I don't understand here, is the purpose of the replication. Are we trying to retain backups of the qmgr's objects, or both the objects and the messages? If you are only trying to retain copies of the objects... http://www-306.ibm.com/software/integration/support/supportpacs/category.html#cat2 MS03: WebSphere MQ - Save Queue Manager object definitions using PCFs Otherwise, investigate a mirrored DSAD solution, or; if you are using AIX, the HACMP supportpac... MC63: WebSphere MQ for AIX - Implementing with HACMP Maybe explain what your goal is, that has you trying this experiment. I'd like to understand what you are trying to accomplish. Thank you, Chris --- Kulbir S. Thind [EMAIL PROTECTED] wrote: Hi, We're thinking about setting up replication for our queue manager to ensure that just the Q files are backed up but not the LOG files for the queue manager. This is being contemplated as it appears the log files are updated far more than the Q files as the LOG files are doing internal MQSeries checks, etc. We need to reduce the amount of data that we're replicating. My first thoughts on this were that this would not work as the Q files would be out of sync with the LOG files. However we performed the following: Created a queue manager Started the queue manager Created some queues Put some message on the queues Ended the queue manager Took a copy of the LOG files used by the queue manager Started the queue manager Put some more messages on the queues Created another queue object Ended the queue manager Restored the LOG files that were backed up in step 6 Started the queue manager At this point I was expecting issues but I found that the queue manager started without problems and it recognised the queue that was created in step 9. Does this mean if we took a copy of the LOG files and restored them to a queue manager at a later point retaining the latest Q files we would have no problems? Has anyone tried this or no of any problems? Cheers, Kulbir. Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com Archive: http://vm.akh-wien.ac.at/MQSeries.archive
Re: Backing up Q files but not the LOG files
Kulbir, Persistent messages are written to the log, commited - then written to the queues. Without the log files, you risk losing inflight activity. Plain and simple. If the messages are not persistent - who cares if you loose them? If you have a redundant hardware solution, 2nd machine, mirrored DASD UPS, you likely have a robust enough configuration for all but total loss of the computer center. If this outage would have you moving operations to a DR site, how much inflight activity are you trying to protect? Given that you are running with circular logging, is it really necessary to try to come up at the DR site with all inflight activity recovered? If this is the case, is this DR site hot? If not, why wouldn't simple configuration backup suffice? I suspect that you are wasting time trying to nickle-and-dime a solution that will likely give you a nasty suprise, should you be in the position of needing the level of recovery that you are seeking. That, or you don't really need to recover the in-flight traffic. Good luck, Chris --- Kulbir S. Thind [EMAIL PROTECTED] wrote: We're trying to ensure we don't lose messages in Disaster Recovery situations. Our failover solution seems OK, we have a SAN configured to hold all queue manager details and that is accessible by another machine in the Veritas cluster if we need to failover. For DR we're looking to use another machine that is based 3,000 miles away, for this we need to minimise the amount of data we replicate, hence the reason why we're contemplating on replicating just the Q files and not LOG files. Cheers. Christopher Warneke [EMAIL PROTECTED] Sent by: MQSeries List [EMAIL PROTECTED] 17-Nov-2004 20:26 Please respond to MQSeries List [EMAIL PROTECTED] To: MQSERIES cc: Subject:Re: Backing up Q files but not the LOG files What I don't understand here, is the purpose of the replication. Are we trying to retain backups of the qmgr's objects, or both the objects and the messages? If you are only trying to retain copies of the objects... http://www-306.ibm.com/software/integration/support/supportpacs/category.html#cat2 MS03: WebSphere MQ - Save Queue Manager object definitions using PCFs Otherwise, investigate a mirrored DSAD solution, or; if you are using AIX, the HACMP supportpac... MC63: WebSphere MQ for AIX - Implementing with HACMP Maybe explain what your goal is, that has you trying this experiment. I'd like to understand what you are trying to accomplish. Thank you, Chris --- Kulbir S. Thind [EMAIL PROTECTED] wrote: Hi, We're thinking about setting up replication for our queue manager to ensure that just the Q files are backed up but not the LOG files for the queue manager. This is being contemplated as it appears the log files are updated far more than the Q files as the LOG files are doing internal MQSeries checks, etc. We need to reduce the amount of data that we're replicating. My first thoughts on this were that this would not work as the Q files would be out of sync with the LOG files. However we performed the following: Created a queue manager Started the queue manager Created some queues Put some message on the queues Ended the queue manager Took a copy of the LOG files used by the queue manager Started the queue manager Put some more messages on the queues Created another queue object Ended the queue manager Restored the LOG files that were backed up in step 6 Started the queue manager At this point I was expecting issues but I found that the queue manager started without problems and it recognised the queue that was created in step 9. Does this mean if we took a copy of the LOG files and restored them to a queue manager at a later point retaining the latest Q files we would have no problems? Has anyone tried this or no of any problems? Cheers, Kulbir. Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com Archive: http://vm.akh-wien.ac.at/MQSeries.archive Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com Archive: http://vm.akh-wien.ac.at/MQSeries.archive
