Re: diagnose 8 / interesting dilemma

[Paul Schuster]

Thank you for all of the responses and ideas.

I guess if your 'guest operating system' is something
like 'CMS' then it is quite easy to get consecutive pages of
real storage addresses.  But not quite so easy with z/os.

I guess if you really want to do 'query virtual dasd' under
z/os pre z/vm 6.4 you would need to do it via multiple calls to
diagnose 8 and doing a 'query virtual -' into the 4k
buffer and then
further parse out the DASD addresses. Or use Ed Jaffe's methodology.

Thank you.

Paul

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

AW: Re: Does >4K PGFIX make sure the frames are contiguous? (was: diagnose 8 / interesting dilemma)

[Peter Hunkeler]

> Data chaining was the original solution to this problem.


It was probably a bad idea to mention I/O in my post (I know about data 
chainging and IDAW). I was thinking more about the possible need to have more 
than 4k of contiguous real storage. Like Tony mentioned, new interfaces such as 
SMC-R/D might need this. There might be internal interfaces for this, right. It 
just triggered my curiosity.


But the net seems to be, my assumption is wrong. If an area >4k is PGFIXed, 
then I need to find the real address of every page and cannot assume the real 
storage is contiguous.


--
Peter Hunkeler

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: diagnose 8 / interesting dilemma

[Peter Relson]

> What happens if AMODE 31 code issues LRA and the real address
> is above the bar?  Program check? 
yes. The ramifications of using the mod-2G part of the real address were 
quite obvious. The architecture reflected that understanding by making 
this case program check.

>Or does LRA simply (always?) load a 64-bit register? 
no

>Is there an LRAG instruction?
yes

Peter Relson 
z/OS Core Technology Design


--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Does >4K PGFIX make sure the frames are contiguous? (was: diagnose 8 / interesting dilemma)

[Binyamin Dissen]

Data chaining was the original solution to this problem.

On Wed, 19 Oct 2016 00:27:56 +0200 Peter Hunkeler  wrote:

:>
:>Below discussion triggered a question I could not answer by RTFM. I had never 
thought about this before in this detail, but now that I do, I wonder if the 
following is correct.
:>
:>Program allocates >4k of virtual storage. The real frames backing it may or 
may not be contiguous. The program wants to use this area with some interface 
(I/O, etc) that will use real addresses when accessing the storage, so it does 
a PGFIX. PGFIX would move the possibly non-contiguous frames guaranteeing the 
area is backed by contiguous frames. After all, the interface using real 
addresses would expect the the frames to be contiguous. 
:> 
:>
:>Am I wrong? I would have expected to find this documented with the PGFIX or 
PGSER FIX macros, but did not find it.  

--
Binyamin Dissen 
http://www.dissensoftware.com

Director, Dissen Software, Bar & Grill - Israel


Should you use the mailblocks package and expect a response from me,
you should preauthorize the dissensoftware.com domain.

I very rarely bother responding to challenge/response systems,
especially those from irresponsible companies.

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Does >4K PGFIX make sure the frames are contiguous? (was: diagnose 8 / interesting dilemma)

[Paul Gilmartin]

On Tue, 18 Oct 2016 19:34:10 -0400, Tony Harminc wrote:
>
>If you want to do I/O to your real storage, that is what IDAWs are
>for. Perhaps there are undocumented (or at least not publicly
>documented) IBM facilities -- I'm guessing things like crypto,
>compression, newer non traditional I/O -- that take real addresses and
>need more than a page at a time, but I don't know of any. In this
>sense, VM's Diagnose interface is an outlier.
> 
I understand that for some time IEBCOPY could not deal with VIO because
IEBCOPY dealt only with real addresses and VIO dealt only with virtual
addresses.  I was slightly surprised when the conflict was resolved by
modifying VIO rather than IEBCOPY.  I suppose this required something
like an inverse LRA.

-- gil

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Does >4K PGFIX make sure the frames are contiguous? (was: diagnose 8 / interesting dilemma)

[Tony Harminc]

On 18 October 2016 at 18:27, Peter Hunkeler  wrote:
> Below discussion triggered a question I could not answer by RTFM. I had never 
> thought about this before in this detail,
> but now that I do, I wonder if the following is correct.
>
> Program allocates >4k of virtual storage. The real frames backing it may or 
> may not be contiguous. The program wants to
> use this area with some interface (I/O, etc) that will use real addresses 
> when accessing the storage, so it does a PGFIX.
> PGFIX would move the possibly non-contiguous frames guaranteeing the area is 
> backed by contiguous frames. After all,
> the interface using real addresses would expect the the frames to be 
> contiguous.
>
> Am I wrong? I would have expected to find this documented with the PGFIX or 
> PGSER FIX macros, but did not find it.

I do not believe there is such a service documented. As Jim Mulder
pointed out, there are 1M pages available above the bar, but that's
relatively new stuff. And there are unexposed RSM services to manage
pairs of frames (and quads, I think he said).

If you want to do I/O to your real storage, that is what IDAWs are
for. Perhaps there are undocumented (or at least not publicly
documented) IBM facilities -- I'm guessing things like crypto,
compression, newer non traditional I/O -- that take real addresses and
need more than a page at a time, but I don't know of any. In this
sense, VM's Diagnose interface is an outlier.

Tony H.

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Does >4K PGFIX make sure the frames are contiguous? (was: diagnose 8 / interesting dilemma)

[Peter Hunkeler]

Below discussion triggered a question I could not answer by RTFM. I had never 
thought about this before in this detail, but now that I do, I wonder if the 
following is correct.

Program allocates >4k of virtual storage. The real frames backing it may or may 
not be contiguous. The program wants to use this area with some interface (I/O, 
etc) that will use real addresses when accessing the storage, so it does a 
PGFIX. PGFIX would move the possibly non-contiguous frames guaranteeing the 
area is backed by contiguous frames. After all, the interface using real 
addresses would expect the the frames to be contiguous. 
 

Am I wrong? I would have expected to find this documented with the PGFIX or 
PGSER FIX macros, but did not find it.  


--
Peter Hunkeler





 
On 10/17/2016 08:23 AM, Paul Schuster wrote: 
> I am issuing DIAGNOSE 8 on my z/os image under VM (z/vm) to do a QUERY 
> VIRTUAL DASD.  It works—up to a certain point: 
>  
> The QUERY VIRTUAL DASD command returns (for me) 38617 (decimal) bytes, 
> according to the CC=0 after the DIAGNOSE 8 command.  My buffer is large 
> enough to accommodate this.  I have tried several different sub-pools of 
> storage.  I PGSER FIX the buffer pages.  I do a SYSEVENT DONTSWAP.  I do a 
> LRA of the virtual address of the start of the buffer.  The DIAGNOSE 
> completes CC=0. But, in my buffer, I am only seeing the first page (4095) 
> bytes of the output.   
>  
> My question: I don’t see any documented restriction in the VM manuals that 
> limits the DIAGNOSE 8 output buffer to 4K (rather the limitation is the 
> architecture limit depending on your amode.) The z/vm manuals say the buffer 
> can cross page boundaries.  So is there a way to force the real storage 
> addresses of the page-fixed pages to be consecutive?  According to the 
> diagnose 8 doc., the buffer needs to be in guest-real storage, hence the LRA. 
>  And it is working for the first 4k page. 
>  
> Thank you for any insight you can provide.  
 
As you already guessed, the memory you get is virtual, so the pages are not 
consecutive. The LRA will give you the address of the first page, but the 2nd, 
3rd and so on will be somewhere else. Please note that your code will even 
cause random memory overwrites in your z/OS as the diag8 will write to the real 
address of your LRA, the real address of your LRA+4096 and so on. 
 


--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: diagnose 8 / interesting dilemma

[Tony Harminc]

On 18 October 2016 at 13:15, Paul Gilmartin
<000433f07816-dmarc-requ...@listserv.ua.edu> wrote:
>   I find:
> z/OS 2.2.0
> z/OS MVS
> z/OS MVS Programming: Assembler Services Guide
> Understanding 31-bit addressing
> Understanding the use of central storage
> Central storage considerations for user programs
> Load real address (LRA) instruction

As suggested, you need to look at the Principles of Operation. LRA is
a matter of hardware, not MVS. Of course the MVS environment
potentially affects some things, but LRA is entirely defined by the
hardware.

> Woe be unto anyone who did LRA; LA in AMODE 24  (But why?.  And it
> doesn't discuss AR mode.  Perhaps elsewhere, but my curiosity is exhausted.

POO.

> Which leaves a historical question.  Was AMODE 31 antedated by the
> availability of real storage >16 MiB,

Assuming you mean the hardware addressing mode, rather than the MVS
Assembler and Binder's notion of AMODE for modules, Yes. But in a
quirky way. In the pre-XA era, there was a time when 64 MB of real
storage could be addressed by DAT tables, but not by channels or
instructions running with DAT off.

> or was AMODE 64 antedated by the availability of real storage >2GiB?

How is this an "or"? You sound like a lawyer at bar: "My client did
not kill Mr X, or in the alternative, if he did kill Mr X he did it
with justification." Maybe I just need to apply De Morgan to your
sentence...

> How did LRA(G) operate in those eras?

LRA provided a 26-bit real address (padded on the left to 32 bits) in
the 64 MiB era.

> And in a virtual guest with shadow page tables, does LRA return the
> real real address or a virtual real address?

Always a virtual real address.

> I suppose it's all emulated by CP anyway, so it depends.

No, it doesn't depend. What sense would it make to return a real real
address to LRA? If you want a real real address you need to use a
conceptually different level of query; a meta-query, one might say.
Normally this would be a Diagnose.

> Or maybe SIE.  My head hurts.

With (XA+) or without (370 pre-XA) SIE, the result is the same. With
SIE it's all handled by the "hardware". Pre SIE, since LRA is
privileged, and the guest always runs in real problem state, CP
provides the answer.

Tony H.

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: diagnose 8 / interesting dilemma

[Paul Gilmartin]

On Mon, 17 Oct 2016 10:08:50 -0700, Ed Jaffe wrote:
>>
>> Hmmm... What happens if AMODE 31 code issues LRA and the real address
>> is above the bar?  Program check?  Or does LRA simply (always?) load a
>> 64-bit register?  Is there an LRAG instruction?
>
>Why not take at least a cursory glance at Principles of Operation before
>asking such questions publicly?
> 
You're right; thanks.  GIMF.  (Oops; the Urban Dictionary says that doesn't
mean what I expected.)  I find:
z/OS 2.2.0
z/OS MVS
z/OS MVS Programming: Assembler Services Guide
Understanding 31-bit addressing
Understanding the use of central storage
Central storage considerations for user programs
Load real address (LRA) instruction

Woe be unto anyone who did LRA; LA in AMODE 24  (But why?.  And it
doesn't discuss AR mode.  Perhaps elsewhere, but my curiosity is exhausted.

Which leaves a historical question.  Was AMODE 31 antedated by the
availability of real storage >16 MiB, or was AMODE 64 antedated by
the availability of real storage >2GiB?  How did LRA(G) operate in
those eras?

And in a virtual guest with shadow page tables, does LRA return the
real real address or a virtual real address?  I suppose it's all emulated
by CP anyway, so it depends.  Or maybe SIE.  My head hurts.

Thanks,
gil

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: diagnose 8 / interesting dilemma

[Jim Mulder]

> Of course there's the question of whether z/VM yet supports the large
> pages (EDAT-1) needed to make this work. I don't believe it does as of
> 6.3, though I may have missed an announcement.

  Good point, and the second time in a month that I have embarassingly 
forgotten about that (Peter Relson will remember the first time). 

 z/VM 6.4 will support 1MB pages for guests.

http://www-01.ibm.com/common/ssi/ShowDoc.wss?docURL=/common/ssi/rep_ca/9/897/ENUS216-009/index.html=en_locale=en
 


Jim Mulder z/OS Diagnosis, Design, Development, Test  IBM Corp. 
Poughkeepsie NY



--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: diagnose 8 / interesting dilemma

[Tony Harminc]

On 17 October 2016 at 13:47, Jim Mulder  wrote:

>  There is IARV64 GETSTOR with TYPE=FIXED and PAGEFRAMESIZE=1M.

It's perhaps not 100% clear how it uses the 64-bit registers, but z/VM
does seem to support AMODE 64 for the caller of Diagnose 8, so I think
this could work. I suppose these days a megabyte of fixed storage here
or there doesn't count for much.

Of course there's the question of whether z/VM yet supports the large
pages (EDAT-1) needed to make this work. I don't believe it does as of
6.3, though I may have missed an announcement.

Tony H.

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: diagnose 8 / interesting dilemma

[Mark Post]

>>> On 10/17/2016 at 02:23 AM, Paul Schuster  wrote: 
> I am issuing DIAGNOSE 8 on my z/os image under VM (z/vm) to do a QUERY 
> VIRTUAL DASD.  It works*up to a certain point:
> 
> The QUERY VIRTUAL DASD command returns (for me) 38617 (decimal) bytes, 
> according to the CC=0 after the DIAGNOSE 8 command.  My buffer is large 
> enough to accommodate this.  I have tried several different sub-pools of 
> storage.  I PGSER FIX the buffer pages.  I do a SYSEVENT DONTSWAP.  I do a 
> LRA of the virtual address of the start of the buffer.  The DIAGNOSE 
> completes CC=0. But, in my buffer, I am only seeing the first page (4095) 
> bytes of the output.  
> 
> My question: I don*t see any documented restriction in the VM manuals that 
> limits the DIAGNOSE 8 output buffer to 4K (rather the limitation is the 
> architecture limit depending on your amode.) The z/vm manuals say the buffer 
> can cross page boundaries.  So is there a way to force the real storage 
> addresses of the page-fixed pages to be consecutive?  According to the 
> diagnose 8 doc., the buffer needs to be in guest-real storage, hence the LRA. 
>  And it is working for the first 4k page.
> 
> Thank you for any insight you can provide.

I have no idea how well the concepts will transfer, but Linux has had a command 
that issues DIAGNOSE 8 calls for a very long time now.  That command was 
originally "hcp" by Neale Ferguson, but then IBM created a different version 
called "vmcp".  Those commands work, so you might be able to figure out what 
they're doing and apply that to your program.


Mark Post

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: diagnose 8 / interesting dilemma

[Jim Mulder]

> I am issuing DIAGNOSE 8 on my z/os image under VM (z/vm) to do a 
> QUERY VIRTUAL DASD.  It works?up to a certain point:
> 
> The QUERY VIRTUAL DASD command returns (for me) 38617 (decimal) 
> bytes, according to the CC=0 after the DIAGNOSE 8 command.  My 
> buffer is large enough to accommodate this.  I have tried several 
> different sub-pools of storage.  I PGSER FIX the buffer pages.  I do
> a SYSEVENT DONTSWAP.  I do a LRA of the virtual address of the start
> of the buffer.  The DIAGNOSE completes CC=0. But, in my buffer, I am
> only seeing the first page (4095) bytes of the output. 
> 
> My question: I don?t see any documented restriction in the VM 
> manuals that limits the DIAGNOSE 8 output buffer to 4K (rather the 
> limitation is the architecture limit depending on your amode.) The 
> z/vm manuals say the buffer can cross page boundaries.  So is there 
> a way to force the real storage addresses of the page-fixed pages to
> be consecutive?  According to the diagnose 8 doc., the buffer needs 
> to be in guest-real storage, hence the LRA.  And it is working for 
> the first 4k page.

  RSM internally manages double frames (used for access lists,
and before z/Architecture, for segment tables) and quadframes
(used for segment tables and region tables).  There are no
external interfaces for double frames or quadframes.

 There is IARV64 GETSTOR with TYPE=FIXED and PAGEFRAMESIZE=1M. 

Jim Mulder z/OS Diagnosis, Design, Development, Test  IBM Corp. 
Poughkeepsie NY



--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: diagnose 8 / interesting dilemma

[Ed Jaffe]

On 10/17/2016 9:16 AM, Paul Gilmartin wrote:


Hmmm... What happens if AMODE 31 code issues LRA and the real address
is above the bar?  Program check?  Or does LRA simply (always?) load a
64-bit register?  Is there an LRAG instruction?


Why not take at least a cursory glance at Principles of Operation before 
asking such questions publicly?


--
Edward E Jaffe
Phoenix Software International, Inc
831 Parkview Drive North
El Segundo, CA 90245
http://www.phoenixsoftware.com/

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: diagnose 8 / interesting dilemma

[Paul Gilmartin]

On Mon, 17 Oct 2016 12:12:38 +0200, Christian Borntraeger wrote:
>
>As you already guessed, the memory you get is virtual, so the pages are not 
>consecutive. The LRA will give you the address of the first page, but the 2nd, 
>3rd and so on will be somewhere else. Please note that your code will even 
>cause random memory overwrites in your z/OS as the diag8 will write to the 
>real address of your LRA, the real address of your LRA+4096 and so on.
>
Ouch!  But can't physical pages nowadays be much larger than 4096 bytes?  Might
the real address be above the bar?  Does DIAG 8 support buffers above the bar?
Otherwise it would be prudent design for DIAG 8 to reject requests when the
reply buffer might cross physical page boundaries?

Do some OSes support allocation of contiguous pages?  I suppose V=R is
obsolete nowadays.

Hmmm... What happens if AMODE 31 code issues LRA and the real address
is above the bar?  Program check?  Or does LRA simply (always?) load a
64-bit register?  Is there an LRAG instruction?

-- gil

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: diagnose 8 / interesting dilemma

[Christian Borntraeger]

On 10/17/2016 08:23 AM, Paul Schuster wrote:
> I am issuing DIAGNOSE 8 on my z/os image under VM (z/vm) to do a QUERY 
> VIRTUAL DASD.  It works—up to a certain point:
> 
> The QUERY VIRTUAL DASD command returns (for me) 38617 (decimal) bytes, 
> according to the CC=0 after the DIAGNOSE 8 command.  My buffer is large 
> enough to accommodate this.  I have tried several different sub-pools of 
> storage.  I PGSER FIX the buffer pages.  I do a SYSEVENT DONTSWAP.  I do a 
> LRA of the virtual address of the start of the buffer.  The DIAGNOSE 
> completes CC=0. But, in my buffer, I am only seeing the first page (4095) 
> bytes of the output.  
> 
> My question: I don’t see any documented restriction in the VM manuals that 
> limits the DIAGNOSE 8 output buffer to 4K (rather the limitation is the 
> architecture limit depending on your amode.) The z/vm manuals say the buffer 
> can cross page boundaries.  So is there a way to force the real storage 
> addresses of the page-fixed pages to be consecutive?  According to the 
> diagnose 8 doc., the buffer needs to be in guest-real storage, hence the LRA. 
>  And it is working for the first 4k page.
> 
> Thank you for any insight you can provide. 

As you already guessed, the memory you get is virtual, so the pages are not 
consecutive. The LRA will give you the address of the first page, but the 2nd, 
3rd and so on will be somewhere else. Please note that your code will even 
cause random memory overwrites in your z/OS as the diag8 will write to the real 
address of your LRA, the real address of your LRA+4096 and so on.

I do not know if there is an interface to get consective real memory as I am 
not an Z/OS expert. 

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

diagnose 8 / interesting dilemma

[Paul Schuster]

I am issuing DIAGNOSE 8 on my z/os image under VM (z/vm) to do a QUERY VIRTUAL 
DASD.  It works—up to a certain point:

The QUERY VIRTUAL DASD command returns (for me) 38617 (decimal) bytes, 
according to the CC=0 after the DIAGNOSE 8 command.  My buffer is large enough 
to accommodate this.  I have tried several different sub-pools of storage.  I 
PGSER FIX the buffer pages.  I do a SYSEVENT DONTSWAP.  I do a LRA of the 
virtual address of the start of the buffer.  The DIAGNOSE completes CC=0. But, 
in my buffer, I am only seeing the first page (4095) bytes of the output.  

My question: I don’t see any documented restriction in the VM manuals that 
limits the DIAGNOSE 8 output buffer to 4K (rather the limitation is the 
architecture limit depending on your amode.) The z/vm manuals say the buffer 
can cross page boundaries.  So is there a way to force the real storage 
addresses of the page-fixed pages to be consecutive?  According to the diagnose 
8 doc., the buffer needs to be in guest-real storage, hence the LRA.  And it is 
working for the first 4k page.

Thank you for any insight you can provide. 
Paul

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN