So you are saying you suspended your PPRC and then did a "failover test" . What does that mean? You IPL'd off the PPRC secondaries (that were now broken mirror? ) or you flashed the whole thing so something else? Or is it the primary copy that is corrupted?
If you corrupted or lost data in the process, you probably should be opening a ticket with IBM to see if they can figure out why. Marcy From: The IBM z/VM Operating System [mailto:[email protected]] On Behalf Of Perez, Steve S Sent: Sunday, March 06, 2011 5:35 PM To: [email protected] Subject: Re: [IBMVM] zLinux OS disk read-only Output from q v dasd below. The OS disk is address 200. We still had the problem. Nothing was in the Operator log to indicate any write inhibit messages. To correct it, we decided to IPL the z/VM LPAR this weekend. When the z/Linux guest machine came up, the Filesystem was corrupted. That same z/Linux guest machine we recovered during our test D/R had the same issue with a corrupt Filesystem. With that in mind, we suspect that after suspending PPRC/Global Mirror, the Failover to test our D/R process may have had some hand in corrupting the Filesystem. -----q v dasd DASD 0199 3390 540RES R/O 5 CYL ON DASD 50B4 SUBCHANNEL = 0009 DASD 0200 3390 LX53B5 R/W 10016 CYL ON DASD 53B5 SUBCHANNEL = 000A DASD 0201 3390 LX54B4 R/W 10016 CYL ON DASD 54B4 SUBCHANNEL = 000B DASD 0202 3390 LX54B5 R/W 3333 CYL ON DASD 54B5 SUBCHANNEL = 000C DASD 0203 3390 LX59B5 R/W 6666 CYL ON DASD 59B5 SUBCHANNEL = 000D DASD 0204 3390 LX5AB4 R/W 10016 CYL ON DASD 5AB4 SUBCHANNEL = 000E DASD 0205 3390 LX5AB5 R/W 10016 CYL ON DASD 5AB5 SUBCHANNEL = 000F ---snippet from z/Linux guest Scanning and configuring dmraid supported devices Scanning logical volumes Reading all physical volumes. This may take a while... Found volume group "vgIBM" using metadata type lvm2 Found volume group "VolGroup01" using metadata type lvm2 Activating logical volumes 2 logical volume(s) in volume group "VolGroup01" now active Creating root device. Mounting root filesystem. EXT3-fs: INFO: recovery required on readonly filesystem. EXT3-fs: write access will be enabled during recovery. kjournald starting. Commit interval 5 seconds EXT3-fs warning (device dm-0): ext3_clear_journal_err: Filesystem error recorded from previous mount: IO failure EXT3-fs warning (device dm-0): ext3_clear_journal_err: Marking fs in need of fil esystem check. EXT3-fs: recovery complete. EXT3-fs: mounted filesystem with ordered data mode. Setting up other filesystems. Setting up new root fs no fstab.sys, mounting internal defaults Switching to new root and running init. unmounting old /dev unmounting old /proc unmounting old /sys SELinux: Disabled at runtime. type=1404 audit(1299410056.218:2): selinux=0 auid=4294967295 ses=4294967295 "INIT: version 2.86 booting"" Welcome to Red Hat Enterprise Linux Server" Setting clock (localtime): Sun Mar 6 05:14:16 CST 2011 Ý OK ¨"" Starting udev: Ý OK ¨"" Thanks, Steve. ________________________________________ From: The IBM z/VM Operating System [mailto:[email protected]] On Behalf Of Marcy Cortes Sent: Tuesday, March 01, 2011 3:50 PM To: [email protected] Subject: Re: zLinux OS disk read-only Do a vmcp q v dasd If it shows r/w and is still not working, log the guest off and back on. If it works then, that would indicate it is some kind of RH problem and Linux was confused. If it still does not work, check the VM Operator log for any write inhibit HCP* error messages. That would indicate some problem with the HW stop you did. Marcy From: The IBM z/VM Operating System [mailto:[email protected]] On Behalf Of Perez, Steve S Sent: Tuesday, March 01, 2011 1:41 PM To: [email protected] Subject: Re: [IBMVM] zLinux OS disk read-only I issued a LINK RR against it and did a Q LINKS and it shows no other link access to that disk. Would it be possible that when we paused PPRC and suspended Global Mirror on the z/OS LPAR (shared volumes between all LPARS) that it may have accessed the dasd the minidisk is on in write mode and caused the access mode on the z/VM LPAR to go into a READ-MODE? Is that probable? Steve. ________________________________________ From: The IBM z/VM Operating System [mailto:[email protected]] On Behalf Of Mark Pace Sent: Tuesday, March 01, 2011 2:57 PM To: [email protected] Subject: Re: zLinux OS disk read-only M Multiple-write access. Write access is established unless another user holds a write, a stable (SR, SW, SM) or an exclusive (ER, EW) mode access to the disk. Looks like some other VM has that disk linked in write mode. On Tue, Mar 1, 2011 at 3:53 PM, Perez, Steve S <[email protected]> wrote: The disk is defined as follows. This is an excerpt from the CP directory: IPL 200 ..... LINK RHMASTER 199 199 RR MDISK 200 3390 1 10016 LX53B5 M Unfortunately, the console log did not get spooled so I don't know what the log would have indicated for that disk when the guest machine came up. That's on my follow-up list. The guest machine is IPL'd off of its OS (disk 200) disk when it comes up (in its CP Directory) so I need to find a way to spool the console when it starts and not later after it has gone through its initialization. Thanks, Steve -----Original Message----- From: The IBM z/VM Operating System [mailto:[email protected]] On Behalf Of RPN01 Sent: Tuesday, March 01, 2011 2:33 PM To: [email protected] Subject: Re: zLinux OS disk read-only How is the disk defined in the CP Directory entry (i.e. What is the mode of the disk), and what is in the console log when the user was logged in that could give a clue about the status of the disk when the user was initialized? The mode will tell you the condition(s) that could lead to it being read only (other users having it read/write or even read only), and the log may even tell you which or how many users gummed up the works, or when things when oval on you. In any case, it had to have happened at some point, and there has to be a footprint, if you keep your logs. -- Robert P. Nix Mayo Foundation .~. RO-OC-1-18 200 First Street SW /V\ 507-284-0844 Rochester, MN 55905 /( )\ ----- ^^-^^ "In theory, theory and practice are the same, but in practice, theory and practice are different." On 3/1/11 2:23 PM, "Steve Perez" <[email protected]> wrote: > Hello All, > > Has anyone run into a situation where the zLinux OS disk has become > READ- > > ONLY access? We are running z/Linux under z/VM 5.4 Redhat 5.4. > > My zLinux Admin were doing compares between the production environment > > versus the Test D/R environment and noticed it. He issued the > following > > on the prod zLinux guest environment: > > # mount -o remount,rw /dev/VolGroup01/LogVol00 > mount: block device /dev/VolGroup01/LogVol00 is write-protected, > mounting > > read-only > > Since we are testing our D/R process at the moment for the z/VM LPAR > we > > are unsure at this point whether that is a contributing factor. It > shoul d not be but we can't rule it out. We paused our PPRC/Global > mirroring fro m the z/OS side before starting the D/R activities to > perform recovery of > > the z/VM & z/Linux. The problem was found while in the middle of > verifying/comparing environments on the zLinux side. I can link to > the > > minidisk that is used to IPL that zLinux guest and it shows R/W when I > > issue Q LINKS. All other minidisks owned by that zLinux guest are R/W a > s > well. From my perspective (z/VM) all looks good. > > Any input would be appreciated, if anything to rule out that PPRC/GM > woul d have contributed to this. > > Thanks. > Steve. ****************************************************************************************** This message may contain confidential or proprietary information intended only for the use of the addressee(s) named above or may contain information that is legally privileged. If you are not the intended addressee, or the person responsible for delivering it to the intended addressee, you are hereby notified that reading, disseminating, distributing or copying this message is strictly prohibited. If you have received this message by mistake, please immediately notify us by replying to the message and delete the original message and any copies immediately thereafter. Thank you. ****************************************************************************************** CLLD -- Mark D Pace Senior Systems Engineer Mainline Information Systems
