On 05/23/2016 01:22 AM, Mark Post wrote:
On 5/22/2016 at 10:45 AM, WF Konynenberg <[email protected]> wrote:
In Linux/Unix, a file is simply a byte stream. Any format information
about a structured file should be accessible to the application by some
means, so it can know how to parse the file. Since the CMS file system
stores structured files of different types/modes, this meta information
in the file system should somehow be made available to applications in
Linux/Unix when accessing the file system through cms-fuse. There are
Sounds like you're starting to argue Alan's case here. Have the file system
driver do the heavy lifting for you. I don't think that's likely to happen.
I'm not so sure.
I'm not suggesting that cmsfs-fuse actually *handle* the various CMS
file formats ("do the heavy lifting"), convert the format, and somehow
magically make the files appear as some comparable common Linux file
format to Linux programs.
I'm just saying that any information that's available in the CMS file
system "out of band" that is somehow relevant to an application trying
to read/write any given file must be made available to the Linux
programs in some way or other. By simply dropping such additional info
on the floor, it may become rather difficult for Linux programs to
correctly interpret the raw file data presented by cmfsf-fuse, and it
may be impossible to create a new file with correct meta data.
Cmsfs-fuse could provide this out-of-band info in various ways. There's
no "perfect" solution. It just somehow needs to make this info
available to the Linux program. I'm guessing that cmsfs-fuse currently
already somehow "grabs" the raw record data from the CMS files, in
whatever file format it happens to be, and presents this raw data in
Linux as a stream of bytes with some minimal logical structure. It
doesn't need to do anything smarter than that. It should just make sure
that if there is "out of band" info in the CMS FS describing the record
format and other details, this info is somehow presented to the Linux
programs for processing.
Note that many (binary) file formats in Linux contain some of this "meta
data" "in band", i.e. as part of a header in the file itself, which
describes relevant details of the file format, thus allowing the program
to correctly interpret the rest of the file. The actual data part of
the file might then be structured as blocks, records, or whatever, and
it is up to the program to correctly handle this.
various ways to achieve that, e.g. represent each CMS file by 2 files in
Linux/Unix (similar, I think, to how "resource forks" are handled in
Apple systems), one containing the data, one containing the meta data,
or, alternatively, add a header to each file that represents the meta
data from the file system (though this might break some existing
applications that currently expect to find only the actual data in the
file and somehow "magically" know the meta data...), etc, etc. Finding
the most suitable model will likely require some creative thought on the
part of the developers. Perhaps the representation model can be made to
depend on a mount option, with the default being backward compatible
lack of metadata.
Just simply saying "sorry, this critical meta data from the CMS file
system is not available in Linux" basically means unnecessarily
crippling Linux access to the CMS file system.
Again, Alan's original suggestion was to teach cmsfs-fuse about such things.
I suppose one could use a combination of cmsfs-fuse and Rick Troth's wonderful
CMSFS package.
s390vsl204:~ # cmsfs-fuse /dev/dasdg /mnt
s390vsl204:~ # l /mnt
total 34024
drwxrwx--- 2 root root 512 May 4 18:56 ./
drwxr-xr-x 1 root root 172 May 18 19:00 ../
-rw-rw---- 1 root root 160 May 4 17:07 FTPBOOT.PARM
-rw-rw---- 1 root root 34812720 May 4 18:56 INITRAMF.IMG
-rw-rw---- 1 root root 2000 Oct 1 2012 KAKKA2.EXEC
-rw-rw---- 1 root root 2000 Oct 1 2012 KAKKA3.EXEC
-rw-rw---- 1 root root 2000 Oct 1 2012 LINUX.EXEC
-rw-rw---- 1 root root 2000 Oct 1 2012 S11S2GM.EXEC
s390vsl204:~ # cmsfslst -f /dev/dasdg
FILENAME FILETYPE FM FORMAT LRECL RECS BLOCKS DATE TIME
DIRECTOR P0 F 64 8 1 5/04/2016 19:56:38
ALLOCMAP P0 F 4096 2 2 5/04/2016 19:56:38
FTPBOOT PARM B1 F 80 2 1 5/04/2016 18:07:40
S11S2GM EXEC Z1 F 80 25 1 10/01/2012 6:23:36
KAKKA2 EXEC Z1 F 80 25 1 10/01/2012 6:27:42
KAKKA3 EXEC Z1 F 80 25 1 10/01/2012 6:28:18
LINUX EXEC Z1 F 80 25 1 10/01/2012 6:27:00
INITRAMF IMG B1 F 80 435159 8500 5/04/2016 19:56:38
I don't think the file command is going to be extended to handle this, however.
So if cmsfslst can find all the relevant bits & bytes in the CMS file
system that indicate what sort of file this is, and how it is to be
read/parsed by programs, then cmsfs-fuse should be able to somehow (by
whatever feasible means) pass the info from these bits & bytes to the
Linux program accessing the file system through cmfsf-fuse.
Actually parsing the file format would still be up to the Linux program,
just as it is for any regular Linux program, because Linux quite
fundamentally doesn't "do" "file formats" at the kernel level, it only
sees a sequence of bytes.
So if, say, FTPBOOT.PARM contains a set 2 of 80 byte card deck records
containing EBCDIC text, then in principle cmsfs-fuse should present
exactly that, a sequence of 80 byte card decks containing EBCDIC text,
with some meta data provided in some way indicating this fact (FM=B1,
FORMAT=F, LRECL=80, ..., RECS is implicit).
If a Linux user wants to actually read this file as a regular linux text
file, it is up to the user to convert the file as appropriate with a
suitable tool (e.g. cms2unix, mirroring dos2unix), prior to handing it
to some Linux text viewing program.
The same applies to handling data from other operating systems. When
accessing native DOS/Windows data on a PC, the text format needs to be
converted from DOS to Unix format, for example.
Some networking tools, such as FTP, have some very limited built-in
capability to perform transparent on-the-fly text format conversion
between different operating systems, when the user indicates that a file
is a text file. For cmfsf-fuse, there might be a few limited cases
where it might make some sense to ask it to do a similar basic text
format conversion, but generally it shouldn't touch the data.
Also, if you want a linux program to be able to actually write a CMS
file, with all the appropriate file type/mode bits set in the file
system meta data, then it obviously needs to be able to somehow provide
this information to cmsfs-fuse.
Willy
----------------------------------------------------------------------
For LINUX-390 subscribe / signoff / archive access instructions,
send email to [email protected] with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
----------------------------------------------------------------------
For more information on Linux on System z, visit
http://wiki.linuxvm.org/
