Re: UNIX "BLKSIZE"?

[Jon Perryman]

On Tue, 14 Nov 2023 20:14:37 -0600, Paul Gilmartin  wrote:

>What's XSAM?  Cite an IBM or general IT source.

Tony used the term first and you want to rag on me. It's a lowercase "x" which 
I take as meaning the group of Sequential Access Methods (e.g. QSAM, BSAM, 
VSAM, ...).

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: UNIX "BLKSIZE"?

[Paul Gilmartin]

On Tue, 14 Nov 2023 19:04:44 -0600, Jon Perryman wrote:
>
>xSAM works because UNIX files are 1 logical record and it's I/O methods makes 
>each byte of that logical record easily addressable. A single read can read 
>starting at any byte location desired for 1 byte, 1,000 bytes or to the end of 
>that record. It's up to xSAM to determine the most suitable interface to the 
>file.
>
What's XSAM?  Cite an IBM or general IT source.

In z/OS 3.1 documentation I find 4 mentions in Using Data Sets, all directory
names in examples.

-- 
gil

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: UNIX "BLKSIZE"?

[Jon Perryman]

On Mon, 13 Nov 2023 19:09:26 -0500, Tony Harminc  wrote:

>Not all z/OS UNIX filesystems are VSAM linear datasets, nor do such
>datasets directly match the UNIX file semantics.

ZFS and HFS are filesystems, not files so why would they match UNIX file 
semantics? Files in z/OS filesystems match file semantics within a Unix 
filesystem. 

>> This is conjecture because logic for BSAM communications to the Unix file
>> subsystem is not made publicly. 
>
>I think you're muddling interface levels
>clearly the xSAM interfaces do not talk directly to an underlying VSAM
>filestore, if there is one.

I said UNIX file subsystem. Of course xSAM doesn't use VSAM because all UNIX 
file reads and writes are thru a single address space which manages 
everything.in a single address space. 

>>A Unix file is always 1 logical record. BLKSIZE has no meaning in this 
>>context. 

xSAM works because UNIX files are 1 logical record and it's I/O methods makes 
each byte of that logical record easily addressable. A single read can read 
starting at any byte location desired for 1 byte, 1,000 bytes or to the end of 
that record. It's up to xSAM to determine the most suitable interface to the 
file.

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: UNIX "BLKSIZE"?

[Paul Gilmartin]

On Mon, 13 Nov 2023 19:09:26 -0500, Tony Harminc wrote:
>>
>> >>On Mon, 13 Nov 2023 01:10:14 -0600, Jon Perryman >> wrote:
>> >>z/OS Unix filesystems are linear datasets.
>
>Some are.
> 
Exceptions might be sockets, descriptors, pipes, and NFS.  And not all
UNIX files belong to filesystems.

>>No.  BLKSIZE is not ignored.
>>
I was envisioning unblocked RECFM (or BLKSIZE=LRECL) as achieving
the effect of setvbuf( ..., _IOLBF )


o {_IOLBF} shall cause input/output to be line buffered.

-- 
gil

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: UNIX "BLKSIZE"?

[Tony Harminc]

On Mon, 13 Nov 2023 at 13:54, Jon Perryman  wrote:

> On Mon, 13 Nov 2023 09:11:19 -0600, Paul Gilmartin 
> wrote:
>
> >>On Mon, 13 Nov 2023 01:10:14 -0600, Jon Perryman 
> wrote:
> >>z/OS Unix filesystems are linear datasets.
>

Some are.

>No.  BLKSIZE is not ignored.
>
> Incorrect, BLKSIZE has no meaning for VSAM linear datasets. By definition,
> "All linear data set bytes are data bytes" which is how a Unix filesystem
> functions.
>

Not all z/OS UNIX filesystems are VSAM linear datasets, nor do such
datasets directly match the UNIX file semantics.

A Unix file is always 1 logical record. BLKSIZE has no meaning in this
> context. While BLKSIZE is stored in the DCB the subsystem ignores it. Any
> part of that logical record can be processed or in its entirety (e.g. read
> or write at offset 1000 for 100 bytes).
>
> > for BSAM it affects how many times the WRITE/CHECK is issued,
>
> This is conjecture because logic for BSAM communications to the Unix file
> subsystem is not made publicly. I suspect that CHECK is a simple BR14 but
> maybe IBM bypassed this logic. BSAM CHECK has no meaning because the
> subsystem will determine to do VSAM CHECK because it must remain compatible
> with UNIX. As for the WRITE, we don't know if every program write causes a
> transfer to the file subsystem or if BSAM buffers according to the BLKSIZE
> before sending to the subsystem. I'm not willing to venture a guess because
> there are pro's and con's to both methods.
>

I think you're muddling interface levels. I'd suggest a high-level read
through of the z/OS UNIX System Services File System Interface Reference.
There's a good introductory picture at
https://www.ibm.com/docs/en/zos/2.5.0?topic=overview-system-structure. Very
clearly the xSAM interfaces do not talk directly to an underlying VSAM
filestore, if there is one.

Tony H.

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: UNIX "BLKSIZE"?

[Seymour J Metz]

While BLKSIZE has no meaning for linear data sets, it *does* have meaning for 
the BSAM compatibiliy interface.


-- 
Shmuel (Seymour J.) Metz
http://mason.gmu.edu/~smetz3
עַם יִשְׂרָאֵל חַי




From: IBM Mainframe Discussion List  on behalf of Jon 
Perryman 
Sent: Monday, November 13, 2023 1:54 PM
To: IBM-MAIN@LISTSERV.UA.EDU
Subject: Re: UNIX "BLKSIZE"?

On Mon, 13 Nov 2023 09:11:19 -0600, Paul Gilmartin  wrote:

>>On Mon, 13 Nov 2023 01:10:14 -0600, Jon Perryman  wrote:
>>z/OS Unix filesystems are linear datasets.

>No.  BLKSIZE is not ignored.

Incorrect, BLKSIZE has no meaning for VSAM linear datasets. By definition, "All 
linear data set bytes are data bytes" which is how a Unix filesystem functions.

> It is merged into the DCB by OPEN,
> or supplied by SDB if not otherwise specified.

A Unix file is always 1 logical record. BLKSIZE has no meaning in this context. 
While BLKSIZE is stored in the DCB the subsystem ignores it. Any part of that 
logical record can be processed or in its entirety (e.g. read or write at 
offset 1000 for 100 bytes).

> for BSAM it affects how many times the WRITE/CHECK is issued,

This is conjecture because logic for BSAM communications to the Unix file 
subsystem is not made publicly. I suspect that CHECK is a simple BR14 but maybe 
IBM bypassed this logic. BSAM CHECK has no meaning because the subsystem will 
determine to do VSAM CHECK because it must remain compatible with UNIX. As for 
the WRITE, we don't know if every program write causes a transfer to the file 
subsystem or if BSAM buffers according to the BLKSIZE before sending to the 
subsystem. I'm not willing to venture a guess because there are pro's and con's 
to both methods.

> and probably has some effect on performance for QSAM.

I'm guessing that performance benefit is insignificant. More important is the 
adverse effect of buffering by BSAM / QSAM. It violates the UNIX standard of 
concurrent writes to a file. Most people don't think about the impact of this 
logic which says you have a file open 20 times, the records will be interleaved 
according to the time the write occurred. Instead of immediately adding each 
record, BSAM / QSAM buffering would delay adding the records to the file 
causing them to be out of sequence. I simply can't see IBM enabling BSAM / QSAM 
buffering that would confuse Unix programmers, but I could be wrong.

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: UNIX "BLKSIZE"?

[Paul Gilmartin]

On Mon, 13 Nov 2023 12:54:31 -0600, Jon Perryman wrote:
>...
>> and probably has some effect on performance for QSAM.
>
>I'm guessing that performance benefit is insignificant. More important is the 
>adverse effect of buffering by BSAM / QSAM. It violates the UNIX standard of 
>concurrent writes to a file. Most people don't think about the impact of this 
>logic which says you have a file open 20 times, the records will be 
>interleaved according to the time the write occurred. Instead of immediately 
>adding each record, BSAM / QSAM buffering would delay adding the records to 
>the file causing them to be out of sequence. I simply can't see IBM enabling 
>BSAM / QSAM buffering that would confuse Unix programmers, but I could be 
>wrong.
>
I suppose part of the answer is here:


-The access method buffers writes beyond the buffering that your program 
sees. This means that after your program issues BSAM WRITE and CHECK, QSAM PUT 
with BUFNO=1 or a VSAM PUT, the data probably is not yet on the disk. If the 
file is not a FIFO, your program can issue the BSAM or QSAM SYNCDEV or CLOSE 
macro to force immediate writing. This interferes with good performance.

-- 
gil

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: UNIX "BLKSIZE"?

[Jon Perryman]

On Mon, 13 Nov 2023 09:11:19 -0600, Paul Gilmartin  wrote:

>>On Mon, 13 Nov 2023 01:10:14 -0600, Jon Perryman  wrote:
>>z/OS Unix filesystems are linear datasets. 

>No.  BLKSIZE is not ignored. 

Incorrect, BLKSIZE has no meaning for VSAM linear datasets. By definition, "All 
linear data set bytes are data bytes" which is how a Unix filesystem functions.

> It is merged into the DCB by OPEN, 
> or supplied by SDB if not otherwise specified. 

A Unix file is always 1 logical record. BLKSIZE has no meaning in this context. 
While BLKSIZE is stored in the DCB the subsystem ignores it. Any part of that 
logical record can be processed or in its entirety (e.g. read or write at 
offset 1000 for 100 bytes).   

> for BSAM it affects how many times the WRITE/CHECK is issued, 

This is conjecture because logic for BSAM communications to the Unix file 
subsystem is not made publicly. I suspect that CHECK is a simple BR14 but maybe 
IBM bypassed this logic. BSAM CHECK has no meaning because the subsystem will 
determine to do VSAM CHECK because it must remain compatible with UNIX. As for 
the WRITE, we don't know if every program write causes a transfer to the file 
subsystem or if BSAM buffers according to the BLKSIZE before sending to the 
subsystem. I'm not willing to venture a guess because there are pro's and con's 
to both methods. 

> and probably has some effect on performance for QSAM.

I'm guessing that performance benefit is insignificant. More important is the 
adverse effect of buffering by BSAM / QSAM. It violates the UNIX standard of 
concurrent writes to a file. Most people don't think about the impact of this 
logic which says you have a file open 20 times, the records will be interleaved 
according to the time the write occurred. Instead of immediately adding each 
record, BSAM / QSAM buffering would delay adding the records to the file 
causing them to be out of sequence. I simply can't see IBM enabling BSAM / QSAM 
buffering that would confuse Unix programmers, but I could be wrong.

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: UNIX "BLKSIZE"?

[Paul Gilmartin]

On Mon, 13 Nov 2023 01:10:14 -0600, Jon Perryman  wrote:
>
>>Suppose I am writing with QSAM/BSAM to a UNIX file allocated
>>RECFM=FB,LRECL=80,FILEDATA=BINARY,PATH=...
>>Should I specify:
>>o BLKSIZE=80 for minimum latency for other jobs doing "tail -f"?
>
>z/OS Unix filesystems are linear datasets. Unless things have changed, BLKSIZE 
>is ignored. LRECL is not stored and must be specified.
> 
No.  BLKSIZE is not ignored.  It is merged into the DCB by OPEN, or supplied
by SDB if not otherwise specified.  for BSAM it affects how many times the
WRITE/CHECK is issued, and probably has some effect on performance
for QSAM.

-- 
gil

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: UNIX "BLKSIZE"?

[Jon Perryman]

On Sat, 11 Nov 2023 21:44:51 -0600, Paul Gilmartin  wrote:

>Suppose I am writing with QSAM/BSAM to a UNIX file allocated
>RECFM=FB,LRECL=80,FILEDATA=BINARY,PATH=...
>Should I specify:
>o BLKSIZE=80 for minimum latency for other jobs doing "tail -f"?

z/OS Unix filesystems are linear datasets. Unless things have changed, BLKSIZE 
is ignored. LRECL is not stored and must be specified.

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

UNIX "BLKSIZE"?

[Paul Gilmartin]

Suppose I am writing with QSAM/BSAM to a UNIX file allocated
RECFM=FB,LRECL=80,FILEDATA=BINARY,PATH=...
Should I specify:
o BLKSIZE=80 for minimum latency for other jobs doing "tail -f"?
o BLKSIZE=32720 for fewest costly access method calls?
o BLKSIZE=0 and trust SDB (which will probably use 1/2 3390)?

-- 
gil

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN