Re: vmrelocate and quiescence time
Even CTCs to the same LPAR have cables on them. There are no internal wrap CTCs. Annoying, yet true. The cables can be point to point or to a switch. Regards, Alan Altmark IBM >> On Aug 16, 2020, at 5:36 PM, Grzegorz Powiedziuk wrote: >> >> On Sun, Aug 16, 2020 at 1:10 AM Alan Altmark >> wrote: >> >> We chose 10 seconds as the default because longer than that tends to cause >> applications to get upset, as you have discovered. You said you were >> using virtual CTCs, so that means you're in the same LPAR, not just same >> CPC. > > Apologize, I confused real vs virtual. I thought that real refers to actual > cables hanging across mainframes etc. > So this is a regular 1st level z/VMs in LPARs setup with linux guests as > VMS. CTC not virtualized in z/VM but defined I believe in HCDS (these tasks > are being done by the hardware team so I don't know the details) > thanks > Gregory > > -- > For LINUX-390 subscribe / signoff / archive access instructions, > send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit > http://www2.marist.edu/htbin/wlvindex?LINUX-390 -- For LINUX-390 subscribe / signoff / archive access instructions, send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit http://www2.marist.edu/htbin/wlvindex?LINUX-390
Re: ancient history - MP3000 CTC network on SLES9
On Aug 16, 2020, at 1:41 PM, Mark Post wrote: > If you mean you're running SLES9 GA, with no service packs, then some of > what I'm going to say may be wrong. I only have a SLES9 SP4 guest > running. However, if you look in /sbin/, there should be a > dasd_configure command that may or may not produce different results > than what you're seeing. dasd_configure is what yast uses under the covers to bring the device online (with `echo 1 >online`), among other things. I've played with it a bit and it doesn't matter whether it's dasd_configure in yast or me running dasd_configure, or me doing the echo >online, it always hangs up the same way, when it gets to echo >online >> CNTLUNIT CUNUMBER=232F,PATH=(10),CUADD=2,UNIT=SCTC, X >> >> UNITADD=((00,16)) >> >> CNTLUNIT CUNUMBER=233E,PATH=(11),CUADD=3,UNIT=SCTC, X >> > > I don't know that it makes any real difference, but why is this 233E, > and not 233F? Because that wasn't the whole story. In order to get a CTC and CNC devices distributed among the LPARs, with each being able to address each other LPAR (by CTC or CNC as necessary), this is what I have (and believe to be required - though, yes, this is definining more than just one CTC/CNC connecting two LPARs): CNTLUNIT CUNUMBER=231F,PATH=(10),CUADD=1,UNIT=SCTC, X UNITADD=((00,16)) CNTLUNIT CUNUMBER=232F,PATH=(10),CUADD=2,UNIT=SCTC, X UNITADD=((00,16)) CNTLUNIT CUNUMBER=232E,PATH=(11),CUADD=2,UNIT=SCTC, X UNITADD=((00,16)) CNTLUNIT CUNUMBER=233E,PATH=(11),CUADD=3,UNIT=SCTC, X UNITADD=((00,16)) * IODEVICE ADDRESS=(2310,16),UNIT=SCTC,CUNUMBR=(231F), X UNITADD=00,STADET=Y,PART=(IZXD,IZXL) IODEVICE ADDRESS=(2320,16),UNIT=SCTC,CUNUMBR=(232E), X UNITADD=00,STADET=Y,PART=(IZOD) IODEVICE ADDRESS=(2320,16),UNIT=SCTC,CUNUMBR=(232F), X UNITADD=00,STADET=Y,PART=(IZXL) IODEVICE ADDRESS=(2330,16),UNIT=SCTC,CUNUMBR=(233E), X UNITADD=00,STADET=Y,PART=(IZOD,IZXD) > Doesn't this mean that 2330 is TCPIP's read channel? (I'm not a z/VM > networking type, so I don't know.) > >> = LINK IZXL2320 CTC 1 IZXL /* I've tried this with both CTC 1 and CTC >> 0 */ > > Or does this statement say which of the two is used for read versus write? The CTC 1 makes 2330 the write channel, and 2331 the read. CTC 0 would make 2330 the read. If I've read the VM TCP guide correctly. I tried it both ways because I don't know if the IOCP does the crossover for me or not. >> List of first 10 CTC Channels that were detected: >> Device Channel type >> 0.0.0e20 3088/01 >> 0.0.0e21 3088/01 >> 0.0.2310 3088/1f >> 0.0.2311 3088/1f > > OK, I'm not seeing the device addresses you said you were going to use. > That is, 2320 and 2321. Where did 2310 and 2311 come from? I cut out some of the list, and it's a list of only the first 10 detected channels. 2320 would be the 19th (of 34 total defined). > Well, to me, where you're going wrong is not defining the Linux system > as a guest of your z/VM system. :) Well, if this were an official project and not a learning one, I'd agree unreservedly. Eventually I'll have linux guests under VM, too. But not at this moment. > Beyond that, I think that you need to be using the same CTC devices in > both LPARs. Otherwise, how is the system supposed to know that traffic > on 2320 goes to 2331, and 2321 to 2330? I believe (this is the part of the EMIF CTC section in the IOCP User's Guide I think I understand pretty clearly) the CUADD and third digit of the real device number (23x0) indicates the LPAR number of the other end. My (less clear) understanding is that IOCP generates pseudo control units based on this, which perform the role of the 'CP COUPLE' command. I have assumed, but am not yet confident, that the last digits are what tie both ends together. e.g. in LPAR 3, 2310 would connect to 2330 in LPAR 1, and 2320 to 2330 in LPAR 2. -- "A corresponding pair of CTC devices can have different device numbers, but they must use the same unit address." ok bear. -- until further notice -- For LINUX-390 subscribe / signoff / archive access instructions, send email to
Re: vmrelocate and quiescence time
On Sun, Aug 16, 2020 at 1:16 AM Alan Altmark wrote: > On Saturday, 08/15/2020 at 02:25 GMT, Grzegorz Powiedziuk > wrote: > > Thank you for verification! If timestamps are correct then this step > > literally takes a very brief moment. So I suspect that that final memory > > pass takes so much time, or am I wrong? > > So how much usually does it take to vmrelocate a 100-200G vm? Just an > > estimate ... do I have to worry with my 30 seconds? > > As you've noted "it depends". A 30 second relocation doesn't bother me, > but a 30 second quiesce time does. > > For Very Large Guests, it's typically better to use application clusters > so that you don't need to move the guests. Just take one down and let the > other members take the load. E.g. I wouldn't use LGR on an Oracle db, > given it's failover capabilities. Let the standby Oracle instance take > over. > > What is a very large guest nowadays? Is 128G considered to be very large? Would 30s quiesce time be expected for a 128G VM or rather for a 512G guest. I am trying to figure out if I have a problem I need to solve or we just have to accept it because that's normal. thanks! -- For LINUX-390 subscribe / signoff / archive access instructions, send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit http://www2.marist.edu/htbin/wlvindex?LINUX-390
Re: vmrelocate and quiescence time
On Sun, Aug 16, 2020 at 1:10 AM Alan Altmark wrote: > > We chose 10 seconds as the default because longer than that tends to cause > applications to get upset, as you have discovered. You said you were > using virtual CTCs, so that means you're in the same LPAR, not just same > CPC. > Apologize, I confused real vs virtual. I thought that real refers to actual cables hanging across mainframes etc. So this is a regular 1st level z/VMs in LPARs setup with linux guests as VMS. CTC not virtualized in z/VM but defined I believe in HCDS (these tasks are being done by the hardware team so I don't know the details) thanks Gregory -- For LINUX-390 subscribe / signoff / archive access instructions, send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit http://www2.marist.edu/htbin/wlvindex?LINUX-390
Re: ancient history - MP3000 CTC network on SLES9
On 8/15/20 5:45 PM, r.stricklin wrote: -snip- > I have the install working (to some degree) with the MP3000 emulated > LCS3174/MPTN arrangement, but I'm having trouble with attempts to bring the > (emulated) 3390 volumes online hanging. The first attempt to `echo 1 > >/sys/bus/ccw/devices/0.0.0980/online` blocks and never returns, even though > there are messages from the kernel that indicate the device has been probed > (it lists geometry, etc.). If you mean you're running SLES9 GA, with no service packs, then some of what I'm going to say may be wrong. I only have a SLES9 SP4 guest running. However, if you look in /sbin/, there should be a dasd_configure command that may or may not produce different results than what you're seeing. -snip- > I am trying to create a CTC network between SLES9 in partition 3 (IZXL) and > VM/ESA V2R4 TCPIP in partition 2 (IZXD). VM has an already-functioning > ethernet IP link via the emulated LCS. > >devs (r,w)ip peer > VM:2331,2330 192.168.176.179 192.168.176.79 > Linux: 2320,2321 192.168.176.79 192.168.176.179 > > SLES9 will configure the CTC but fails to ping its peer, and the install > can't proceed. I'm not sure what I'm missing; it may likely be some EMIF CTC > detail. My incomplete understanding is that 2320/2330 should be either end of > one CTC connecting LPARs 2 and 3, and nothing more need be done to connect > them, but I'm not really sure. > > > Relevant IOCDF entries for the EMIF CTC definitions - > >CHPID PATH=(10),TYPE=CTC,SHARED,PART=((IZOD,IZXD,IZXL),(=)) > >CHPID PATH=(11),TYPE=CNC,SHARED,PART=((IZOD,IZXD,IZXL),(=)) > >CNTLUNIT CUNUMBER=232F,PATH=(10),CUADD=2,UNIT=SCTC, X > > UNITADD=((00,16)) > >CNTLUNIT CUNUMBER=233E,PATH=(11),CUADD=3,UNIT=SCTC, X > I don't know that it makes any real difference, but why is this 233E, and not 233F? > UNITADD=((00,16)) > > IODEVICE ADDRESS=(2320,16),UNIT=SCTC,CUNUMBR=(232F), X > > UNITADD=00,STADET=Y,PART=(IZXL) > > IODEVICE ADDRESS=(2330,16),UNIT=SCTC,CUNUMBR=(233E), X > > UNITADD=00,STADET=Y,PART=(IZOD,IZXD) > > > Relevant data from the VM side - >q ctc >CTCA 0E20 ATTACHED TO TCPIP0E20 >CTCA 0E21 ATTACHED TO TCPIP0E21 >CTCA 2330 ATTACHED TO TCPIP2330 >CTCA 2331 ATTACHED TO TCPIP2331 >Ready; T=0.01/0.01 08:57:53 > > PROFILE TCPIP (excerpted) - > = DEVICE UNIT0 LCS E20 > = LINK MPTN2 ETHERNET 2 UNIT0 > = DEVICE IZXL CTC 2330 Doesn't this mean that 2330 is TCPIP's read channel? (I'm not a z/VM networking type, so I don't know.) > = LINK IZXL2320 CTC 1 IZXL /* I've tried this with both CTC 1 and CTC 0 > */ Or does this statement say which of the two is used for read versus write? > = HOME > =192.168.5.78 MPTN2 > =192.168.176.179 IZXL2320 > = GATEWAY > = 192.168.0.0=MPTN1 DEFAULTSIZE 0.0.255.0 0.0.5.0 > = ; IZXL CTC > = 192.168.176.79 =IZXL2320 1500 HOST > = START UNIT0 > = START IZXL2320 > > Linux console log (exerpted) - > >Please select the type of your network device: >4) Channel to Channel >Enter your choice (0-10): > 4 >Loading CTC module: >CTC driver Version: 1.58.2.1 initialized >List of first 10 CTC Channels that were detected: >Device Channel type >0.0.0e20 3088/01 >0.0.0e21 3088/01 >0.0.2310 3088/1f >0.0.2311 3088/1f OK, I'm not seeing the device addresses you said you were going to use. That is, 2320 and 2321. Where did 2310 and 2311 come from? >Device address for read channel (0.0.0e20): > 0.0.2320 >Device address for write channel (0.0.0e21): > 0.0.2321 I don't know how this worked if the installer only presented 2310 and 2311 to you for choosing. >Select protocol number for CTC: >0) Compatibility mode, also for non-Linux peers other > than OS/3909 and z/OS (this is the default mode) >1) Extended mode >3) Compatibility mode with OS/390 and z/OS >Enter your choice (0): > 0 >ctc0: read: ch-0.0.2320, write: ch-0.0.2321, proto: 0 >ctc0 detected. >ctc0 is available, continuing with network setup. >ifconfig ctc0 192.168.176.79 pointopoint 192.168.176.179 mtu 1500 >Trying to ping my IP address: >PING 192.168.176.79 (192.168.176.79) 56(84) bytes of data. >64 bytes from 192.168.176.79: icmp_seq=1 ttl=64 time=0.171ms >3 packets transmitted, 3 received, 0% packet loss, time 1998ms >Waiting 6 seconds for connection with remote side. >Waiting 3 seconds for connection with remote side. >Waiting 4
