Re: IBM python documentation? [was: RE: IBM SMF Export with Python]

[David Crayford]

On 4/10/22 01:56, Farley, Peter x23353 wrote:

I have read that tagging behavior page and it makes sense when applied to files stored in 
the Unix file system.  MVS datasets could just as easily just be considered always tagged 
as cp1047 (with an option to change the CCSID to 037 or one of the European EBCIDIC code 
pages) and then you just go on from there.  Connecting to the MVS file system "under 
the covers" is the runtime implementor's job, not the application developer's job.

As to providing a package to specifically access MVS datasets, my first question is why IBM didn't 
provide one in the first place, or better yet build it into the python I/O subsystem from the 
start?  Why does the community need to provide such a basic functionality?  "zoautil_py" 
is one such implementation, but it is extraordinarily "heavyweight" and expensive to use, 
and is not an acceptable solution in a world where every second counts and costs real money.


I am quite sure an MVS I/O package will happen just like it did for 
golang and Node.js. Yet again, open source code documented in Github and 
the respective package manager repositories. You may not like that but 
it's how these things work.


https://github.com/ibmruntimes/go-recordio



The lack of easy-to-use and efficient MVS file access may well discourage more 
widespread use of python on z/OS for everyday computing tasks.


Maybe not as much as you think. The tooling coming online will require 
Python so it's mandatory. The vast majority on this forum would rather 
use REXX anyway!





And yes, I can accept that my expectations are not realistic.  It is just a shame that an 
easy-to-use and efficient access to MVS datasets isn't and won't be there unless the 
"community" provides it.  IMHO it should have been part-and-parcel of the 
original language implementation, but I know what my opinions and expectations are worth.


The community involves IBM and the team that ported Python, Golang, 
Node.js etc. The team from IBM Toronto have setup and open source 
community that are actively ported FOSS to z/OS. Interestingly, this 
includes the zsh which I'm delighted about.




Peter

-Original Message-
From: IBM Mainframe Discussion List  On Behalf Of 
David Crayford
Sent: Monday, October 3, 2022 1:48 AM
To: IBM-MAIN@LISTSERV.UA.EDU
Subject: Re: IBM python documentation? [was: RE: IBM SMF Export with Python]

On 3/10/22 07:18, Farley, Peter x23353 wrote:

Where, for instance, in the documentation that you linked to is there information on 
whether the python "open()" function for files will or will not support direct 
access to MVS datasets without transferring them down to the Unix file system?  I.E.,  
are either of these calls to the open() function valid or not:

Myfile = open("//'MVS.DATA.SET.NAME", "rt", ccsid="1047")

Myfile = open("//:DDNAME", "rt", ccsid="1047") # This of course
assumes that a call to "bpxunix" (or by other means) has been used to
establish a DD allocation in the AS where the python program is
executing

This will *never* happen. If you read the doc WRT tagging behaviors you will 
understand why 
https://urldefense.com/v3/__https://www.ibm.com/docs/en/python-zos/3.9?topic=translation-tagging-behaviors__;!!Ebr-cpPeAnfNniQ8HSAI-g_K5b7VKg!O2czdpcmZt_bBNIuRr4UXedStf34QpBfTBkZ-wQNwNFosWZR2zrDSP1nat8ZFfmZnRA9s7TVhDN6jaUYFy-CaGg$
  .
Rocket's Python port supported reading MVS data sets but it was not a 
documented feature and was used with great care. Patching the Python core I/O 
libraries to support MVS data sets is not a great idea. It's much better to 
write a specific library that *only* support MVS data sets and not z/OS UNIX 
files. A good reference implementation is the JZOS ZFile class. If you open an 
RFE IBM may do this. Or maybe the community can contribute? Either way it's 
going to be implemented as a specific package.

Porting tools to support MVS data sets in trivial. They use open() because it's 
a low-level API and read/write support sockets, FIFO's, pipes etc. It's not 
just a case of replacing open with fopen(). Python implements a fdopen()
https://urldefense.com/v3/__https://docs.python.org/3/library/os.html*os.fdopen__;Iw!!Ebr-cpPeAnfNniQ8HSAI-g_K5b7VKg!O2czdpcmZt_bBNIuRr4UXedStf34QpBfTBkZ-wQNwNFosWZR2zrDSP1nat8ZFfmZnRA9s7TVhDN6jaUY6MzWLWc$
   method for wrapping
file descriptors.
--


This message and any attachments are intended only for the use of the addressee 
and may contain information that is privileged and confidential. If the reader 
of the message is not the intended recipient or an authorized representative of 
the intended recipient, you are hereby notified that any dissemination of this 
communication is strictly prohibited. If you have received this communication 
in error, please notify us immediately

Re: IBM python documentation? [was: RE: IBM SMF Export with Python]

[Farley, Peter x23353]

I have read that tagging behavior page and it makes sense when applied to files 
stored in the Unix file system.  MVS datasets could just as easily just be 
considered always tagged as cp1047 (with an option to change the CCSID to 037 
or one of the European EBCIDIC code pages) and then you just go on from there.  
Connecting to the MVS file system "under the covers" is the runtime 
implementor's job, not the application developer's job.

As to providing a package to specifically access MVS datasets, my first 
question is why IBM didn't provide one in the first place, or better yet build 
it into the python I/O subsystem from the start?  Why does the community need 
to provide such a basic functionality?  "zoautil_py" is one such 
implementation, but it is extraordinarily "heavyweight" and expensive to use, 
and is not an acceptable solution in a world where every second counts and 
costs real money.  The lack of easy-to-use and efficient MVS file access may 
well discourage more widespread use of python on z/OS for everyday computing 
tasks.

And yes, I can accept that my expectations are not realistic.  It is just a 
shame that an easy-to-use and efficient access to MVS datasets isn't and won't 
be there unless the "community" provides it.  IMHO it should have been 
part-and-parcel of the original language implementation, but I know what my 
opinions and expectations are worth.

Peter

-Original Message-
From: IBM Mainframe Discussion List  On Behalf Of 
David Crayford
Sent: Monday, October 3, 2022 1:48 AM
To: IBM-MAIN@LISTSERV.UA.EDU
Subject: Re: IBM python documentation? [was: RE: IBM SMF Export with Python]

On 3/10/22 07:18, Farley, Peter x23353 wrote:
> Where, for instance, in the documentation that you linked to is there 
> information on whether the python "open()" function for files will or will 
> not support direct access to MVS datasets without transferring them down to 
> the Unix file system?  I.E.,  are either of these calls to the open() 
> function valid or not:
>
> Myfile = open("//'MVS.DATA.SET.NAME", "rt", ccsid="1047")
>
> Myfile = open("//:DDNAME", "rt", ccsid="1047") # This of course 
> assumes that a call to "bpxunix" (or by other means) has been used to 
> establish a DD allocation in the AS where the python program is 
> executing

This will *never* happen. If you read the doc WRT tagging behaviors you will 
understand why 
https://urldefense.com/v3/__https://www.ibm.com/docs/en/python-zos/3.9?topic=translation-tagging-behaviors__;!!Ebr-cpPeAnfNniQ8HSAI-g_K5b7VKg!O2czdpcmZt_bBNIuRr4UXedStf34QpBfTBkZ-wQNwNFosWZR2zrDSP1nat8ZFfmZnRA9s7TVhDN6jaUYFy-CaGg$
  . 
Rocket's Python port supported reading MVS data sets but it was not a 
documented feature and was used with great care. Patching the Python core I/O 
libraries to support MVS data sets is not a great idea. It's much better to 
write a specific library that *only* support MVS data sets and not z/OS UNIX 
files. A good reference implementation is the JZOS ZFile class. If you open an 
RFE IBM may do this. Or maybe the community can contribute? Either way it's 
going to be implemented as a specific package.

Porting tools to support MVS data sets in trivial. They use open() because it's 
a low-level API and read/write support sockets, FIFO's, pipes etc. It's not 
just a case of replacing open with fopen(). Python implements a fdopen() 
https://urldefense.com/v3/__https://docs.python.org/3/library/os.html*os.fdopen__;Iw!!Ebr-cpPeAnfNniQ8HSAI-g_K5b7VKg!O2czdpcmZt_bBNIuRr4UXedStf34QpBfTBkZ-wQNwNFosWZR2zrDSP1nat8ZFfmZnRA9s7TVhDN6jaUY6MzWLWc$
   method for wrapping 
file descriptors.
--


This message and any attachments are intended only for the use of the addressee 
and may contain information that is privileged and confidential. If the reader 
of the message is not the intended recipient or an authorized representative of 
the intended recipient, you are hereby notified that any dissemination of this 
communication is strictly prohibited. If you have received this communication 
in error, please notify us immediately by e-mail and delete the message and any 
attachments from your system.


--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: IBM python documentation?

[Phil Smith III]

Shmuel asked:
> BTW, has usage shifted from ISO-8859-1 (Latin 1) to ISO-8859-15 (Latin-9) to 
> accomodate the Euro €?



Not that I’ve noticed, though it may be that they’re really doing -15 and 
calling it -1. Have not looked at Encoding headers to see.

 

Not to trivialize it—it’s an important character!—but for the most part it’s 
names and the like that people get bent about (accented characters etc.), which 
I think is why it either hasn’t happened or isn’t visible as such.


--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: IBM python documentation? [was: RE: IBM SMF Export with Python]

[David Crayford]

On 3/10/22 13:47, David Crayford wrote:

Porting tools to support MVS data sets in trivial.

I meant Porting tools to support MVS data sets is non trivial.

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: IBM python documentation? [was: RE: IBM SMF Export with Python]

[David Crayford]

On 3/10/22 07:18, Farley, Peter x23353 wrote:

Where, for instance, in the documentation that you linked to is there information on 
whether the python "open()" function for files will or will not support direct 
access to MVS datasets without transferring them down to the Unix file system?  I.E.,  
are either of these calls to the open() function valid or not:

Myfile = open("//'MVS.DATA.SET.NAME", "rt", ccsid="1047")

Myfile = open("//:DDNAME", "rt", ccsid="1047") # This of course assumes that a call to 
"bpxunix" (or by other means) has been used to establish a DD allocation in the AS where the python program 
is executing


This will *never* happen. If you read the doc WRT tagging behaviors you 
will understand why 
https://www.ibm.com/docs/en/python-zos/3.9?topic=translation-tagging-behaviors. 
Rocket's Python port supported reading MVS data sets but it was not a 
documented feature and was used with great care. Patching the Python 
core I/O libraries to support MVS data sets is not a great idea. It's 
much better to write a specific library that *only* support MVS data 
sets and not z/OS UNIX files. A good reference implementation is the 
JZOS ZFile class. If you open an RFE IBM may do this. Or maybe the 
community can contribute? Either way it's going to be implemented as a 
specific package.


Porting tools to support MVS data sets in trivial. They use open() 
because it's a low-level API and read/write support sockets, FIFO's, 
pipes etc. It's not just a case of replacing open with fopen(). Python 
implements a fdopen() 
https://docs.python.org/3/library/os.html#os.fdopen method for wrapping 
file descriptors.


--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: IBM python documentation? [was: RE: IBM SMF Export with Python]

[David Crayford]

The only z/OS specific library is a utility package for tagging files 
which is documented here 
https://www.ibm.com/docs/en/python-zos/3.9?topic=package-documentation-zos-util.


Doc for disutils and building packages is here 
https://www.ibm.com/docs/en/python-zos/3.9?topic=information-using-distutils-module.


The README includes information about what additional packages have been 
included with the SDK. It's the same set as Rocket shipped with their 
Python. Maybe I'm missing a trick here because IMO IBM have done a great 
job with Python.


Post-Installation instructions:
-
Note:
In order for Python to operate correctly,
the following environment variables must be set:
    export LIBPATH=/usr/lpp/IBM/cyp/v3r10/pyz/lib
    export _BPXK_AUTOCVT='ON'
    export _CEE_RUNOPTS='FILETAG(AUTOCVT,AUTOTAG) POSIX(ON)'
    # if you are using a bash shell, you should also set these:
    export _TAG_REDIR_ERR=txt
    export _TAG_REDIR_IN=txt
    export _TAG_REDIR_OUT=txt
Also please ensure that your PATH environment variable contains the 'bin'
directory for python.  e.g.:
   export PATH=/usr/lpp/IBM/cyp/v3r10/pyz/bin:$PATH
2. Python location
==
You will find the python executable in
the /usr/lpp/IBM/cyp/v3r10/pyz/bin/ directory.
For example:
    $ /usr/lpp/IBM/cyp/v3r10/pyz/bin/python3 --version
    or
    $ python3 --version
    Python 3.10.5
3. List of Bundled Packages
==
IBM Open Enterprise SDK for Python includes several packages in addition to
the packages included in the default CPython distribution.
Package  Version
 
cffi 1.14.6
cryptography 3.3.2
ebcdic   1.1.1
numpy    1.21.2
pip  22.1.2
pycparser    2.20
setuptools   58.3.0
six  1.16.0
zos_util 1.0.0


On 3/10/22 07:40, David Crayford wrote:


On 3/10/22 07:18, Farley, Peter x23353 wrote:
As the OP of the other thread on python documentation, let me answer 
that what I expected from IBM for python documentation was the same 
sort of Language Reference and Programmers Guide manuals as they 
provide for all of the other major programming languages that they 
support.  My understanding at the moment is that that hasn't been 
done and is not likely to be done.


Where, for instance, in the documentation that you linked to is there 
information on whether the python "open()" function for files will or 
will not support direct access to MVS datasets without transferring 
them down to the Unix file system?  I.E., are either of these calls 
to the open() function valid or not:


Where is the language reference and programming language for Java on 
z/OS? There isn't one, so why would Python be any different? The only 
language references and guides you will find for Java is for products 
such as DB2 or for z/OS specific toolkits such as JZOS.


There are no z/OS specific modifications to Python other than patches 
for the build. If you want to open an MVS data set then write a 
package using cutils to directly reference the C RTL without writing a 
single line of C code. Once you have a library put it on pypi so 
everybody can download it using "pip". Why do you expect IBM to 
provide the function? They are working on a completely new set of 
tooling and have limited resources.


https://medium.com/python-supply/working-with-foreign-functions-ee3b5e40f2ca 





Myfile = open("//'MVS.DATA.SET.NAME", "rt", ccsid="1047")

Myfile = open("//:DDNAME", "rt", ccsid="1047") # This of course 
assumes that a call to "bpxunix" (or by other means) has been used to 
establish a DD allocation in the AS where the python program is 
executing


In other words, where are the z/OS-specific enhancements to the 
standard python functions and libraries documented (assuming there 
are any such enhancements, which I am beginning to doubt)?  For other 
IBM-supported programming languages, that information would probably 
be available in the Language Reference manual and practical examples 
and performance-enhancing advice would be provided in the Programmers 
Guide manual.


The first link that you provided is fine for the systems programmer 
who installs the language on z/OS, but has basically nothing relevant 
for the python application programmer.


The "zoautil_py" library does provide some "direct access" to MVS 
datasets for python programs, but the package is quite "heavyweight" 
and consumes a large amount of elapsed and CPU time to use, as recent 
experimentation I have performed clearly showed.  I found that 
downloading an MVS dataset to the Unix file system with the "zowe" 
CLI interface and accessing that file with "normal" python file 
access functions proved far less expensive than using the 
"zoautil_py" functions to access the same data.  Nowhere in the 
IBM-provided ZOAU documentation did I find any reference to 
performance characteristics or performance-enhancing advice.


That separate python libr

Re: IBM python documentation? [was: RE: IBM SMF Export with Python]

[David Crayford]

On 3/10/22 07:18, Farley, Peter x23353 wrote:

As the OP of the other thread on python documentation, let me answer that what 
I expected from IBM for python documentation was the same sort of Language 
Reference and Programmers Guide manuals as they provide for all of the other 
major programming languages that they support.  My understanding at the moment 
is that that hasn't been done and is not likely to be done.

Where, for instance, in the documentation that you linked to is there information on 
whether the python "open()" function for files will or will not support direct 
access to MVS datasets without transferring them down to the Unix file system?  I.E.,  
are either of these calls to the open() function valid or not:


Where is the language reference and programming language for Java on 
z/OS? There isn't one, so why would Python be any different? The only 
language references and guides you will find for Java is for products 
such as DB2 or for z/OS specific toolkits such as JZOS.


There are no z/OS specific modifications to Python other than patches 
for the build. If you want to open an MVS data set then write a package 
using cutils to directly reference the C RTL without writing a single 
line of C code. Once you have a library put it on pypi so everybody can 
download it using "pip". Why do you expect IBM to provide the function? 
They are working on a completely new set of tooling and have limited 
resources.


https://medium.com/python-supply/working-with-foreign-functions-ee3b5e40f2ca



Myfile = open("//'MVS.DATA.SET.NAME", "rt", ccsid="1047")

Myfile = open("//:DDNAME", "rt", ccsid="1047") # This of course assumes that a call to 
"bpxunix" (or by other means) has been used to establish a DD allocation in the AS where the python program 
is executing

In other words, where are the z/OS-specific enhancements to the standard python 
functions and libraries documented (assuming there are any such enhancements, 
which I am beginning to doubt)?  For other IBM-supported programming languages, 
that information would probably be available in the Language Reference manual 
and practical examples and performance-enhancing advice would be provided in 
the Programmers Guide manual.

The first link that you provided is fine for the systems programmer who 
installs the language on z/OS, but has basically nothing relevant for the 
python application programmer.

The "zoautil_py" library does provide some "direct access" to MVS datasets for python programs, but the package is 
quite "heavyweight" and consumes a large amount of elapsed and CPU time to use, as recent experimentation I have performed 
clearly showed.  I found that downloading an MVS dataset to the Unix file system with the "zowe" CLI interface and accessing that 
file with "normal" python file access functions proved far less expensive than using the "zoautil_py" functions to 
access the same data.  Nowhere in the IBM-provided ZOAU documentation did I find any reference to performance characteristics or 
performance-enhancing advice.

That separate python libraries for z/OS maintain separate python documentation of their 
functionality is OK, but then the IBM-specific library websites need to provide the same sort 
of automatic linking to the library documentation that the standard python documentation 
websites use.  I.E., the IBM-specific libraries need to have their documentation directly 
available on the ".readthedocs.io" website and linked to from 
their specific page on the Pypi indexing website and not just at their github location.


What IBM-specific library? There isn't one!



ISTM that IBM is a day late and a dollar short on application-level 
documentation for this language on their systems.


I disagree. Maybe your expectations are unrealistic.



Peter

-Original Message-
From: IBM Mainframe Discussion List  On Behalf Of 
David Crayford
Sent: Saturday, October 1, 2022 9:35 PM
To: IBM-MAIN@LISTSERV.UA.EDU
Subject: Re: IBM SMF Export with Python

On 2/10/22 01:31, Charles Mills wrote:

We LOVE Python.

When is IBM going to love it enough to ship it and document it as though they 
loved it?

You can order Python via zShop and install it via SMP/E or download a pax file. 
That's choice right there.

What's wrong with the doc? https://www.ibm.com/docs/en/python-zos/3.10 has all 
the information I need to install, customize and build extensions.

For z/OS specific Python libraries the doc is either shipped with the product 
so as ZAOU or for open source projects with the code in a repository. For 
example, the Python DB2 library and doc can be found on Github 
https://github.com/ibmdb/python-ibmdb  . z/OS specific doc here 
https://github.com/ibmdb/node-ibm_db*configure-odbc-driver-on-zos  . This is 
standard for any open source project. I understand that may be uncomfortable to 
folks that have no experience working with open source.

Another example is https://github.com/ambitus/pyzkiln   which is maintained by 
I

Re: IBM python documentation? [was: RE: IBM SMF Export with Python]

[Farley, Peter x23353]

As the OP of the other thread on python documentation, let me answer that what 
I expected from IBM for python documentation was the same sort of Language 
Reference and Programmers Guide manuals as they provide for all of the other 
major programming languages that they support.  My understanding at the moment 
is that that hasn't been done and is not likely to be done.

Where, for instance, in the documentation that you linked to is there 
information on whether the python "open()" function for files will or will not 
support direct access to MVS datasets without transferring them down to the 
Unix file system?  I.E.,  are either of these calls to the open() function 
valid or not:

Myfile = open("//'MVS.DATA.SET.NAME", "rt", ccsid="1047")

Myfile = open("//:DDNAME", "rt", ccsid="1047") # This of course assumes that a 
call to "bpxunix" (or by other means) has been used to establish a DD 
allocation in the AS where the python program is executing

In other words, where are the z/OS-specific enhancements to the standard python 
functions and libraries documented (assuming there are any such enhancements, 
which I am beginning to doubt)?  For other IBM-supported programming languages, 
that information would probably be available in the Language Reference manual 
and practical examples and performance-enhancing advice would be provided in 
the Programmers Guide manual.

The first link that you provided is fine for the systems programmer who 
installs the language on z/OS, but has basically nothing relevant for the 
python application programmer.

The "zoautil_py" library does provide some "direct access" to MVS datasets for 
python programs, but the package is quite "heavyweight" and consumes a large 
amount of elapsed and CPU time to use, as recent experimentation I have 
performed clearly showed.  I found that downloading an MVS dataset to the Unix 
file system with the "zowe" CLI interface and accessing that file with "normal" 
python file access functions proved far less expensive than using the 
"zoautil_py" functions to access the same data.  Nowhere in the IBM-provided 
ZOAU documentation did I find any reference to performance characteristics or 
performance-enhancing advice.

That separate python libraries for z/OS maintain separate python documentation 
of their functionality is OK, but then the IBM-specific library websites need 
to provide the same sort of automatic linking to the library documentation that 
the standard python documentation websites use.  I.E., the IBM-specific 
libraries need to have their documentation directly available on the 
".readthedocs.io" website and linked to from their specific page 
on the Pypi indexing website and not just at their github location.

ISTM that IBM is a day late and a dollar short on application-level 
documentation for this language on their systems.

Peter

-Original Message-
From: IBM Mainframe Discussion List  On Behalf Of 
David Crayford
Sent: Saturday, October 1, 2022 9:35 PM
To: IBM-MAIN@LISTSERV.UA.EDU
Subject: Re: IBM SMF Export with Python

On 2/10/22 01:31, Charles Mills wrote:
> We LOVE Python.
>
> When is IBM going to love it enough to ship it and document it as though they 
> loved it?

You can order Python via zShop and install it via SMP/E or download a pax file. 
That's choice right there.

What's wrong with the doc? https://www.ibm.com/docs/en/python-zos/3.10 has all 
the information I need to install, customize and build extensions.

For z/OS specific Python libraries the doc is either shipped with the product 
so as ZAOU or for open source projects with the code in a repository. For 
example, the Python DB2 library and doc can be found on Github 
https://github.com/ibmdb/python-ibmdb  . z/OS specific doc here 
https://github.com/ibmdb/node-ibm_db*configure-odbc-driver-on-zos  . This is 
standard for any open source project. I understand that may be uncomfortable to 
folks that have no experience working with open source.

Another example is https://github.com/ambitus/pyzkiln   which is maintained by 
IBMers from Poughkeepsie but part of the OMP. Anybody interested in writing 
Python libraries that include calling HLASM should take a look at that repo.

Honestly, I don't see what the issue is with doc? It's no different to Java. If 
I need doc for the JRE I read the Oracle doc. If I need doc for Python I head 
on over to the Python doc. Everything I need for z/OS is covered.

>
> (Rhetorical question. I know that David does not know the answer.)
>
> Charles
>
>
> -Original Message-
> From: IBM Mainframe Discussion List [mailto:IBM-MAIN@LISTSERV.UA.EDU] 
> On Behalf Of David Crayford
> Sent: Friday, September 30, 2022 11:09 PM
> To: IBM-MAIN@LISTSERV.UA.EDU
> Subject: IBM SMF Export with Python
>
> I know Python is not well received on this forum but as an example of 
> IBMs investment they have announced a new Python toolkit to process 
> SMF data using Jupyter notesbooks, an industry standard visualization 
> too

Re: IBM python documentation?

[Paul Gorlinsky]

My motto has always been "Be reasonable! Do it my way."

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: IBM python documentation?

[Matt Hogstrom]

I’ve always said everyone wants there to be one standard: the problem is they 
all want it to be theirs

Matt Hogstrom
+1 (919) 656-0564

> On Oct 1, 2022, at 22:20, Phil Smith III  wrote:
> 
> But it feels kinda like https://xkcd.com/927/ .

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: IBM python documentation?

[Seymour J Metz]

BTW, has usage shifted from ISO-8859-1 (Latin 1) to ISO-8859-15 (Latin-9) to 
accomodate the Euro €?


--
Shmuel (Seymour J.) Metz
http://mason.gmu.edu/~smetz3


From: IBM Mainframe Discussion List [IBM-MAIN@LISTSERV.UA.EDU] on behalf of 
David Crayford [dcrayf...@gmail.com]
Sent: Saturday, October 1, 2022 2:01 AM
To: IBM-MAIN@LISTSERV.UA.EDU
Subject: Re: IBM python documentation?

On 27/9/22 19:26, Seymour J Metz wrote:
> By "enhanced ASCII" do you mean ISO-8859-1? UTF-8? "E. None of the above."?

ISO8859-1 is more reliable. There are issues with UTF-8. There is a
great blog post by an IBM DE who is working on porting FOSS to z/OS
https://secure-web.cisco.com/138beZNdimezxlHRsxawGBduyA_QNJd0ziH82OWJkgr5V7VLESiEF0calqTxs2UbYZeCcXSMrtebjg2XnbFsVQkmqrvUVvqQ257-iRELKvJoyzXW3gxdzHTLR80w-DIA2AQqeD2QF04I55bYPX5YTUFeziwJDJ-xYxi3LczU4HztNY2hxo_sT4thnBRI-AVyrfPOf-5RxzhPSCUcsT_GWFPq6pL9ApMx7WT45RW-dPpLbHbNc2JySA8sxOdgoVM1oE3ZPfVnZj-XBzMiXnp0BdxlA0h3oKOFNAJs5XG1FyLENxoysSfcOBdoIzbalF_WkwVp4tlyCqk37xCbk5a3bvSMPZB8iF0GX0Lf-OTOKgkxqYCwVoQ8Jdpne72wwGKkyunij3l9fX-xhyCNSI1_oyEAC49X9IHFN1RzGa4TCvb5hKDQf0D-1pdemKTyT3FPF/https%3A%2F%2Fmakingdeveloperslivesbetter.wordpress.com%2F2022%2F01%2F07%2Fis-z-os-ascii-or-ebcdic-yes%2F



>
>
> --
> Shmuel (Seymour J.) Metz
> http://mason.gmu.edu/~smetz3
>
> 
> From: IBM Mainframe Discussion List [IBM-MAIN@LISTSERV.UA.EDU] on behalf of 
> David Crayford [dcrayf...@gmail.com]
> Sent: Monday, September 26, 2022 6:41 PM
> To: IBM-MAIN@LISTSERV.UA.EDU
> Subject: Re: IBM python documentation?
>
> On 27/9/22 00:34, Farley, Peter x23353 wrote:
>> Thanks for the reminder about the zoautil_py functions, I did know about 
>> them but there are gaps -- e.g. ZOAU has no direct support for simple 
>> sequential processing of VSAM files (e.g., to download a copy of VSAM data 
>> to a flat file in the Unix Services environment for analysis there).  I have 
>> used a Rexx script that executes the FileManager batch utility program 
>> FMNMAIN to perform such functions.  The FileManager batch program proved to 
>> be the most compatible in generating correct Unix Services output files for 
>> various flavors of VSAM input files where IDCAMS REPRO or DFSORT "OPTION 
>> COPY" outputs were less so.
>>
>> Re: "ADDRESS TSO" needs - Rexx is not necessarily needed, as there is also 
>> the Unix Services command "tso" (or "tsocmd" for authorized services) to 
>> execute a single TSO command.  The python subprocess.run() function could 
>> handle executing that command and capturing the output as needed.
>>
>> For more complicated "ADDRESS TSO" needs, yes one can execute a Rexx script 
>> in the Unix Services environment which uses "ADDRESS TSO" to execute a Rexx 
>> script also stored in that environment (or in an MVS library assigned to 
>> SYSEXEC) that then actually uses "ADDRESS TSO" functions.  A bit Rube 
>> Goldberg, but it does work, as my recent experimentation with getting 
>> LISTDSI output from the Unix Services environment proved.  Conceivably one 
>> could write that first Rexx script to set up an ISPF environment (all the 
>> needed DD's, etc.) and execute the second (functional) script in an ISPF 
>> environment (I haven't tried that yet; here there be dragons).
> Should have used my lua port :) It supports all access methods and has a
> listdsi function that I wrote - io.listdsi(dsname)
>
>
>> Re: Writing an MVS I/O package for IBM python - ISTM that without 
>> documentation of the runtime architecture of the IBM python port (and also 
>> possibly access to the source of the port) it is not likely to succeed.  OCO 
>> strikes again.
> It's certainly not OCO. IBM ship all the header files to write
> extensions including a zos.h. What you need to know is that it's 64-bit,
> enhanced ASCII and is compiled using the xlclang compiler. Extensions
> are shared objects so you need to following compiler options: -q64
> -qascii -qexportall. You may not need "exportall" if IBM have defined
> PyMODINIT_FUNC for the platform.
>
>
>> The new DSFS functionality may or may not relieve the need for an 
>> MVS-specific I/O package.  That remains to be seen.
>>
>> Peter
>>
>> -Original Message-
>> From: IBM Mainframe Discussion List  On Behalf Of 
>> Matt Hogstrom
>> Sent: Monday, September 26, 2022 8:28 AM
>> To: IBM-MAIN@LISTSERV.UA.EDU
>> Subject: Re: IBM python documentation?
>>
>> If IIRC IBM loosely paired the IBM ZOAU to “assist Python”.  
>> https:

Re: IBM python documentation?

[Seymour J Metz]

A search for UTF-EBCDIC on the IETF web page got no hits, so regardless of its 
status at the Unicode Consortium, it doesn't seem to have gotten traction other 
than instructions on IBM Z.


--
Shmuel (Seymour J.) Metz
http://mason.gmu.edu/~smetz3


From: IBM Mainframe Discussion List [IBM-MAIN@LISTSERV.UA.EDU] on behalf of 
Tony Harminc [t...@harminc.net]
Sent: Saturday, October 1, 2022 4:20 PM
To: IBM-MAIN@LISTSERV.UA.EDU
Subject: Re: IBM python documentation?

On Sat, 1 Oct 2022 at 13:40, Phil Smith III  wrote:

> Jay Maynard wrote:
> >OK, so what kind of issues are there with UTF-8? Especially since it's
> >pretty much the standard everywhere, these days?
>
> Yeah, that caught my eye too. I suspect the answer is that *mixing* UTF-8
> and EBCDIC gets complicated because you cannot always convert: e.g., if you
> have  in the same string, UTF-anything
> can handle it, but you cannot convert that string to EBCDIC because those
> two characters are in different EBCDIC code pages.
>
> Combine that with UTF-8 normalization and variable-length characters and
> it's bewildering for EBCDIC-based minds.
>
> This does NOT really reflect deficiencies in UTF-8 but rather just
> difficulties switching between EBCDIC and UTF-8.
>
> ISO8859-1 is cleaner (for cases where it's sufficient!) because it CAN map
> 1:1 to EBCDIC. Of course it's not sufficient in many, many cases in a
> global
> economy.
>

There *was* something called UTF-EBCDIC, but IBM and UNICODE seem to have
abandoned it shortly after it was written.

https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Funicode.org%2Freports%2Ftr16%2Ftr16-7.2.html&data=05%7C01%7Csmetz3%40gmu.edu%7Ca2c2238aeb204b8fe97608daa3ea7a81%7C9e857255df574c47a0c00546460380cb%7C0%7C0%7C638002524732265567%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=yrqEKpQ56BgWULVPg97g9YcVo2woJYWyacKHooC8UfA%3D&reserved=0

I've never understood why there was so little interest shown in it; it
seemed to me to fill the bill very nicely.

Tony H.

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: IBM python documentation?

[Seymour J Metz]

To add to the importance of proper UTF-8, the IETF has mandated UTF-8 as the 
direction going forward, and several protocols have been upgraded for UTF-8 
support.


--
Shmuel (Seymour J.) Metz
http://mason.gmu.edu/~smetz3


From: IBM Mainframe Discussion List [IBM-MAIN@LISTSERV.UA.EDU] on behalf of 
David Crayford [dcrayf...@gmail.com]
Sent: Saturday, October 1, 2022 10:38 PM
To: IBM-MAIN@LISTSERV.UA.EDU
Subject: Re: IBM python documentation?

On 2/10/22 01:40, Phil Smith III wrote:
> Jay Maynard wrote:
>> OK, so what kind of issues are there with UTF-8? Especially since it's
>> pretty much the standard everywhere, these days?
>
>
> Yeah, that caught my eye too. I suspect the answer is that *mixing* UTF-8
> and EBCDIC gets complicated because you cannot always convert: e.g., if you
> have  in the same string, UTF-anything
> can handle it, but you cannot convert that string to EBCDIC because those
> two characters are in different EBCDIC code pages.
>
>

All good points. There are other issues when setting _BPX_AUTOCVT=ALL
and things start to break depending on your system configuration.
If you set _BPX_AUTOCVT=ALL and use Rocket ported tools they will break.
There are other issues not related to UTF-8 to file tagging and text
conversion in general. Unnamed pipes only do auto-conversion on one side
of the pipe so any ported tools using unnamed pipes will have to convert
to named pipes. The language specification for Golang, Python, JSON,
YAML etc all mandate UTF-8 but all the ported files on z/OS are tagged
ISO8859-1.

Using the ISPF editor with tagged files can get interesting. It doesn't
work at all with UTF-8 if AUTOCVT(ALL) is set in BPXPRMxx. If
AUTOCVT(ON) is set and the user is using an emulator with a codepage
that is not IBM-1047 then unpredictable results will occur. We had a
case opened with IBM when a customer using a German code page created
malformed YAML files using the ISPF editor. It took ages to diagnose and
the PMR was closed WAD. IBM asked to to open an RFE to implement a new
configuration option to prevent this unwanted translation occurring.

>
> Combine that with UTF-8 normalization and variable-length characters and
> it's bewildering for EBCDIC-based minds.
>
>
>
> This does NOT really reflect deficiencies in UTF-8 but rather just
> difficulties switching between EBCDIC and UTF-8.
>
>
>
> ISO8859-1 is cleaner (for cases where it's sufficient!) because it CAN map
> 1:1 to EBCDIC. Of course it's not sufficient in many, many cases in a global
> economy.
>
>
>
> ...phsiii
>
>
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: IBM python documentation?

[Mike Schwab]

Any text in any code page can be translated into UTF-8 (since it is a
merge of all existing ASCII/EBCDIC code pages).

A particular UTF-8 string can be converted into a code page
IF, AND ONLY IF,
all characters are from that code page.
If you have any characters outside that code page, you get a bad character.

On Sat, Oct 1, 2022 at 10:09 PM Phil Smith III  wrote:
>
> David Crayford expounded on some issues with UTF-8 *on z/OS* and 
> _BPX_AUTOCVT=ALL. All legitimate, all real problems, but really
> z/OS issues, not UTF-8 issues. That is, these don't reflect problems with 
> UTF-8 itself. What we see all the time is data that's
> ISO8859-1 and is treated as 7-bit ASCII (which mostly works) or UTF-8 (which 
> sorta works). The problem, of course, is that every
> character in this note works fine in that scenario, but as soon as you get 
> into some interesting glyphs, Bad Things happen. And then
> it's "Your product isn't handling this right"--no, you lied to it and are 
> paying the price, sorry! Calling a tail a leg doesn't make
> it one, even if you can whack things with both of them.
>
>
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN



-- 
Mike A Schwab, Springfield IL USA
Where do Forest Rangers go to get away from it all?

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: IBM python documentation?

[David Crayford]

100% agreement. And it's not just interesting glyphs I've had problems 
with. Any code point > 0x7F can get interesting.


On 2/10/22 11:09, Phil Smith III wrote:

David Crayford expounded on some issues with UTF-8 *on z/OS* and 
_BPX_AUTOCVT=ALL. All legitimate, all real problems, but really
z/OS issues, not UTF-8 issues. That is, these don't reflect problems with UTF-8 
itself. What we see all the time is data that's
ISO8859-1 and is treated as 7-bit ASCII (which mostly works) or UTF-8 (which 
sorta works). The problem, of course, is that every
character in this note works fine in that scenario, but as soon as you get into 
some interesting glyphs, Bad Things happen. And then
it's "Your product isn't handling this right"--no, you lied to it and are 
paying the price, sorry! Calling a tail a leg doesn't make
it one, even if you can whack things with both of them.


--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN


--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: IBM python documentation?

[Phil Smith III]

David Crayford expounded on some issues with UTF-8 *on z/OS* and 
_BPX_AUTOCVT=ALL. All legitimate, all real problems, but really
z/OS issues, not UTF-8 issues. That is, these don't reflect problems with UTF-8 
itself. What we see all the time is data that's
ISO8859-1 and is treated as 7-bit ASCII (which mostly works) or UTF-8 (which 
sorta works). The problem, of course, is that every
character in this note works fine in that scenario, but as soon as you get into 
some interesting glyphs, Bad Things happen. And then
it's "Your product isn't handling this right"--no, you lied to it and are 
paying the price, sorry! Calling a tail a leg doesn't make
it one, even if you can whack things with both of them.


--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: IBM python documentation?

[David Crayford]

On 2/10/22 01:40, Phil Smith III wrote:

Jay Maynard wrote:

OK, so what kind of issues are there with UTF-8? Especially since it's
pretty much the standard everywhere, these days?
  


Yeah, that caught my eye too. I suspect the answer is that *mixing* UTF-8
and EBCDIC gets complicated because you cannot always convert: e.g., if you
have  in the same string, UTF-anything
can handle it, but you cannot convert that string to EBCDIC because those
two characters are in different EBCDIC code pages.

  


All good points. There are other issues when setting _BPX_AUTOCVT=ALL 
and things start to break depending on your system configuration.
If you set _BPX_AUTOCVT=ALL and use Rocket ported tools they will break. 
There are other issues not related to UTF-8 to file tagging and text
conversion in general. Unnamed pipes only do auto-conversion on one side 
of the pipe so any ported tools using unnamed pipes will have to convert 
to named pipes. The language specification for Golang, Python, JSON, 
YAML etc all mandate UTF-8 but all the ported files on z/OS are tagged 
ISO8859-1.


Using the ISPF editor with tagged files can get interesting. It doesn't 
work at all with UTF-8 if AUTOCVT(ALL) is set in BPXPRMxx. If 
AUTOCVT(ON) is set and the user is using an emulator with a codepage 
that is not IBM-1047 then unpredictable results will occur. We had a 
case opened with IBM when a customer using a German code page created 
malformed YAML files using the ISPF editor. It took ages to diagnose and 
the PMR was closed WAD. IBM asked to to open an RFE to implement a new 
configuration option to prevent this unwanted translation occurring.




Combine that with UTF-8 normalization and variable-length characters and
it's bewildering for EBCDIC-based minds.

  


This does NOT really reflect deficiencies in UTF-8 but rather just
difficulties switching between EBCDIC and UTF-8.

  


ISO8859-1 is cleaner (for cases where it's sufficient!) because it CAN map
1:1 to EBCDIC. Of course it's not sufficient in many, many cases in a global
economy.

  


...phsiii


--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN


--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: IBM python documentation?

[Phil Smith III]

Tony Harminc wrote:
>There *was* something called UTF-EBCDIC, but IBM and UNICODE seem to have
>abandoned it shortly after it was written.

>  
> https://unicode.org/reports/tr16/tr16-7.2.html

>I've never understood why there was so little interest shown in it; it
>seemed to me to fill the bill very nicely.

 

At the risk of sounding.something, what bill is that? If you're going to muck 
with encoding, go to UTF-8 and be done with it. I'm
not being flip-I have to deal with ASCII/EBCDIC/UTF-n in a format-preserving 
data protection environment, which is even more complex
(normalization, for one thing), so I'm by no means minimizing the pain. But it 
feels kinda like https://xkcd.com/927/ .


--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: IBM python documentation?

[Tony Harminc]

On Sat, 1 Oct 2022 at 13:40, Phil Smith III  wrote:

> Jay Maynard wrote:
> >OK, so what kind of issues are there with UTF-8? Especially since it's
> >pretty much the standard everywhere, these days?
>
> Yeah, that caught my eye too. I suspect the answer is that *mixing* UTF-8
> and EBCDIC gets complicated because you cannot always convert: e.g., if you
> have  in the same string, UTF-anything
> can handle it, but you cannot convert that string to EBCDIC because those
> two characters are in different EBCDIC code pages.
>
> Combine that with UTF-8 normalization and variable-length characters and
> it's bewildering for EBCDIC-based minds.
>
> This does NOT really reflect deficiencies in UTF-8 but rather just
> difficulties switching between EBCDIC and UTF-8.
>
> ISO8859-1 is cleaner (for cases where it's sufficient!) because it CAN map
> 1:1 to EBCDIC. Of course it's not sufficient in many, many cases in a
> global
> economy.
>

There *was* something called UTF-EBCDIC, but IBM and UNICODE seem to have
abandoned it shortly after it was written.

https://unicode.org/reports/tr16/tr16-7.2.html

I've never understood why there was so little interest shown in it; it
seemed to me to fill the bill very nicely.

Tony H.

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: IBM python documentation?

[Phil Smith III]

Jay Maynard wrote:
>OK, so what kind of issues are there with UTF-8? Especially since it's
>pretty much the standard everywhere, these days?

 

Yeah, that caught my eye too. I suspect the answer is that *mixing* UTF-8
and EBCDIC gets complicated because you cannot always convert: e.g., if you
have  in the same string, UTF-anything
can handle it, but you cannot convert that string to EBCDIC because those
two characters are in different EBCDIC code pages.

 

Combine that with UTF-8 normalization and variable-length characters and
it's bewildering for EBCDIC-based minds.

 

This does NOT really reflect deficiencies in UTF-8 but rather just
difficulties switching between EBCDIC and UTF-8.

 

ISO8859-1 is cleaner (for cases where it's sufficient!) because it CAN map
1:1 to EBCDIC. Of course it's not sufficient in many, many cases in a global
economy.

 

...phsiii 


--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: IBM python documentation?

[Jay Maynard]

OK, so what kind of issues are there with UTF-8? Especially since it's
pretty much the standard everywhere, these days?

On Sat, Oct 1, 2022 at 1:01 AM David Crayford  wrote:

> On 27/9/22 19:26, Seymour J Metz wrote:
> > By "enhanced ASCII" do you mean ISO-8859-1? UTF-8? "E. None of the
> above."?
>
> ISO8859-1 is more reliable. There are issues with UTF-8. There is a
> great blog post by an IBM DE who is working on porting FOSS to z/OS
>
> https://makingdeveloperslivesbetter.wordpress.com/2022/01/07/is-z-os-ascii-or-ebcdic-yes/.
>
>
>
>
> >
> >
> > --
> > Shmuel (Seymour J.) Metz
> > http://mason.gmu.edu/~smetz3
> >
> > 
> > From: IBM Mainframe Discussion List [IBM-MAIN@LISTSERV.UA.EDU] on
> behalf of David Crayford [dcrayf...@gmail.com]
> > Sent: Monday, September 26, 2022 6:41 PM
> > To: IBM-MAIN@LISTSERV.UA.EDU
> > Subject: Re: IBM python documentation?
> >
> > On 27/9/22 00:34, Farley, Peter x23353 wrote:
> >> Thanks for the reminder about the zoautil_py functions, I did know
> about them but there are gaps -- e.g. ZOAU has no direct support for simple
> sequential processing of VSAM files (e.g., to download a copy of VSAM data
> to a flat file in the Unix Services environment for analysis there).  I
> have used a Rexx script that executes the FileManager batch utility program
> FMNMAIN to perform such functions.  The FileManager batch program proved to
> be the most compatible in generating correct Unix Services output files for
> various flavors of VSAM input files where IDCAMS REPRO or DFSORT "OPTION
> COPY" outputs were less so.
> >>
> >> Re: "ADDRESS TSO" needs - Rexx is not necessarily needed, as there is
> also the Unix Services command "tso" (or "tsocmd" for authorized services)
> to execute a single TSO command.  The python subprocess.run() function
> could handle executing that command and capturing the output as needed.
> >>
> >> For more complicated "ADDRESS TSO" needs, yes one can execute a Rexx
> script in the Unix Services environment which uses "ADDRESS TSO" to execute
> a Rexx script also stored in that environment (or in an MVS library
> assigned to SYSEXEC) that then actually uses "ADDRESS TSO" functions.  A
> bit Rube Goldberg, but it does work, as my recent experimentation with
> getting LISTDSI output from the Unix Services environment proved.
> Conceivably one could write that first Rexx script to set up an ISPF
> environment (all the needed DD's, etc.) and execute the second (functional)
> script in an ISPF environment (I haven't tried that yet; here there be
> dragons).
> > Should have used my lua port :) It supports all access methods and has a
> > listdsi function that I wrote - io.listdsi(dsname)
> >
> >
> >> Re: Writing an MVS I/O package for IBM python - ISTM that without
> documentation of the runtime architecture of the IBM python port (and also
> possibly access to the source of the port) it is not likely to succeed.
> OCO strikes again.
> > It's certainly not OCO. IBM ship all the header files to write
> > extensions including a zos.h. What you need to know is that it's 64-bit,
> > enhanced ASCII and is compiled using the xlclang compiler. Extensions
> > are shared objects so you need to following compiler options: -q64
> > -qascii -qexportall. You may not need "exportall" if IBM have defined
> > PyMODINIT_FUNC for the platform.
> >
> >
> >> The new DSFS functionality may or may not relieve the need for an
> MVS-specific I/O package.  That remains to be seen.
> >>
> >> Peter
> >>
> >> -Original Message-
> >> From: IBM Mainframe Discussion List  On
> Behalf Of Matt Hogstrom
> >> Sent: Monday, September 26, 2022 8:28 AM
> >> To: IBM-MAIN@LISTSERV.UA.EDU
> >> Subject: Re: IBM python documentation?
> >>
> >> If IIRC IBM loosely paired the IBM ZOAU to “assist Python”.
> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcommunity.ibm.com%2Fcommunity%2Fuser%2Fibmz-and-linuxone%2Fblogs%2Faustin-wells1%2F2020%2F12%2F11%2Fusing-ibm-open-enterprise-python-for-zos-and-zoau&data=05%7C01%7Csmetz3%40gmu.edu%7Ccfa1d3f0cb6144907fa608daa0104277%7C9e857255df574c47a0c00546460380cb%7C0%7C0%7C637998288955467761%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=CSIi3EDh0%2Bl60cgC%2B2ZSh6zbr6omoi4FwR125vs76UE%3D&reserved=0
> >>
> >> The challenge I see currently with Python is that is runs i

Re: IBM python documentation?

[David Crayford]

On 27/9/22 19:26, Seymour J Metz wrote:

By "enhanced ASCII" do you mean ISO-8859-1? UTF-8? "E. None of the above."?


ISO8859-1 is more reliable. There are issues with UTF-8. There is a 
great blog post by an IBM DE who is working on porting FOSS to z/OS 
https://makingdeveloperslivesbetter.wordpress.com/2022/01/07/is-z-os-ascii-or-ebcdic-yes/. 







--
Shmuel (Seymour J.) Metz
http://mason.gmu.edu/~smetz3


From: IBM Mainframe Discussion List [IBM-MAIN@LISTSERV.UA.EDU] on behalf of 
David Crayford [dcrayf...@gmail.com]
Sent: Monday, September 26, 2022 6:41 PM
To: IBM-MAIN@LISTSERV.UA.EDU
Subject: Re: IBM python documentation?

On 27/9/22 00:34, Farley, Peter x23353 wrote:

Thanks for the reminder about the zoautil_py functions, I did know about them but there 
are gaps -- e.g. ZOAU has no direct support for simple sequential processing of VSAM 
files (e.g., to download a copy of VSAM data to a flat file in the Unix Services 
environment for analysis there).  I have used a Rexx script that executes the FileManager 
batch utility program FMNMAIN to perform such functions.  The FileManager batch program 
proved to be the most compatible in generating correct Unix Services output files for 
various flavors of VSAM input files where IDCAMS REPRO or DFSORT "OPTION COPY" 
outputs were less so.

Re: "ADDRESS TSO" needs - Rexx is not necessarily needed, as there is also the Unix Services 
command "tso" (or "tsocmd" for authorized services) to execute a single TSO command.  The 
python subprocess.run() function could handle executing that command and capturing the output as needed.

For more complicated "ADDRESS TSO" needs, yes one can execute a Rexx script in the Unix Services 
environment which uses "ADDRESS TSO" to execute a Rexx script also stored in that environment (or 
in an MVS library assigned to SYSEXEC) that then actually uses "ADDRESS TSO" functions.  A bit Rube 
Goldberg, but it does work, as my recent experimentation with getting LISTDSI output from the Unix Services 
environment proved.  Conceivably one could write that first Rexx script to set up an ISPF environment (all 
the needed DD's, etc.) and execute the second (functional) script in an ISPF environment (I haven't tried 
that yet; here there be dragons).

Should have used my lua port :) It supports all access methods and has a
listdsi function that I wrote - io.listdsi(dsname)



Re: Writing an MVS I/O package for IBM python - ISTM that without documentation 
of the runtime architecture of the IBM python port (and also possibly access to 
the source of the port) it is not likely to succeed.  OCO strikes again.

It's certainly not OCO. IBM ship all the header files to write
extensions including a zos.h. What you need to know is that it's 64-bit,
enhanced ASCII and is compiled using the xlclang compiler. Extensions
are shared objects so you need to following compiler options: -q64
-qascii -qexportall. You may not need "exportall" if IBM have defined
PyMODINIT_FUNC for the platform.



The new DSFS functionality may or may not relieve the need for an MVS-specific 
I/O package.  That remains to be seen.

Peter

-Original Message-
From: IBM Mainframe Discussion List  On Behalf Of 
Matt Hogstrom
Sent: Monday, September 26, 2022 8:28 AM
To: IBM-MAIN@LISTSERV.UA.EDU
Subject: Re: IBM python documentation?

If IIRC IBM loosely paired the IBM ZOAU to “assist Python”.  
https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcommunity.ibm.com%2Fcommunity%2Fuser%2Fibmz-and-linuxone%2Fblogs%2Faustin-wells1%2F2020%2F12%2F11%2Fusing-ibm-open-enterprise-python-for-zos-and-zoau&data=05%7C01%7Csmetz3%40gmu.edu%7Ccfa1d3f0cb6144907fa608daa0104277%7C9e857255df574c47a0c00546460380cb%7C0%7C0%7C637998288955467761%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=CSIi3EDh0%2Bl60cgC%2B2ZSh6zbr6omoi4FwR125vs76UE%3D&reserved=0

The challenge I see currently with Python is that is runs in a USS environment 
so if you need to “ADDRESS TSO” you need to call a REXX from Python in USS and 
do the other work.  Its probably not going to happen but a “System” Python 
might fill that gap.

Matt Hogstrom
m...@hogstrom.org

“It may be cognitive, but, it ain’t intuitive."
— Hogstrom


On Sep 25, 2022, at 11:15 PM, David Crayford  wrote:

On 26/9/22 10:43, Charles Mills wrote:

It's trivial to write an MVS I/O package if you have a C compiler.

One might ask then why IBM has not done so.

I would suggest that they have not had a requirement. IBM use Python in their 
analytics products and for new stuff like Ansible. Same with golang, they need 
it for Kubernetes and OpenShift for z/CX containers. I doubt very much if many 
customers have tried golang. It's a great language now it supports generics. As 
fast as C++ wi

Re: IBM python documentation?

[Seymour J Metz]

By "enhanced ASCII" do you mean ISO-8859-1? UTF-8? "E. None of the above."?


--
Shmuel (Seymour J.) Metz
http://mason.gmu.edu/~smetz3


From: IBM Mainframe Discussion List [IBM-MAIN@LISTSERV.UA.EDU] on behalf of 
David Crayford [dcrayf...@gmail.com]
Sent: Monday, September 26, 2022 6:41 PM
To: IBM-MAIN@LISTSERV.UA.EDU
Subject: Re: IBM python documentation?

On 27/9/22 00:34, Farley, Peter x23353 wrote:
> Thanks for the reminder about the zoautil_py functions, I did know about them 
> but there are gaps -- e.g. ZOAU has no direct support for simple sequential 
> processing of VSAM files (e.g., to download a copy of VSAM data to a flat 
> file in the Unix Services environment for analysis there).  I have used a 
> Rexx script that executes the FileManager batch utility program FMNMAIN to 
> perform such functions.  The FileManager batch program proved to be the most 
> compatible in generating correct Unix Services output files for various 
> flavors of VSAM input files where IDCAMS REPRO or DFSORT "OPTION COPY" 
> outputs were less so.
>
> Re: "ADDRESS TSO" needs - Rexx is not necessarily needed, as there is also 
> the Unix Services command "tso" (or "tsocmd" for authorized services) to 
> execute a single TSO command.  The python subprocess.run() function could 
> handle executing that command and capturing the output as needed.
>
> For more complicated "ADDRESS TSO" needs, yes one can execute a Rexx script 
> in the Unix Services environment which uses "ADDRESS TSO" to execute a Rexx 
> script also stored in that environment (or in an MVS library assigned to 
> SYSEXEC) that then actually uses "ADDRESS TSO" functions.  A bit Rube 
> Goldberg, but it does work, as my recent experimentation with getting LISTDSI 
> output from the Unix Services environment proved.  Conceivably one could 
> write that first Rexx script to set up an ISPF environment (all the needed 
> DD's, etc.) and execute the second (functional) script in an ISPF environment 
> (I haven't tried that yet; here there be dragons).

Should have used my lua port :) It supports all access methods and has a
listdsi function that I wrote - io.listdsi(dsname)


> Re: Writing an MVS I/O package for IBM python - ISTM that without 
> documentation of the runtime architecture of the IBM python port (and also 
> possibly access to the source of the port) it is not likely to succeed.  OCO 
> strikes again.

It's certainly not OCO. IBM ship all the header files to write
extensions including a zos.h. What you need to know is that it's 64-bit,
enhanced ASCII and is compiled using the xlclang compiler. Extensions
are shared objects so you need to following compiler options: -q64
-qascii -qexportall. You may not need "exportall" if IBM have defined
PyMODINIT_FUNC for the platform.


>
> The new DSFS functionality may or may not relieve the need for an 
> MVS-specific I/O package.  That remains to be seen.
>
> Peter
>
> -Original Message-
> From: IBM Mainframe Discussion List  On Behalf Of 
> Matt Hogstrom
> Sent: Monday, September 26, 2022 8:28 AM
> To: IBM-MAIN@LISTSERV.UA.EDU
> Subject: Re: IBM python documentation?
>
> If IIRC IBM loosely paired the IBM ZOAU to “assist Python”.  
> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcommunity.ibm.com%2Fcommunity%2Fuser%2Fibmz-and-linuxone%2Fblogs%2Faustin-wells1%2F2020%2F12%2F11%2Fusing-ibm-open-enterprise-python-for-zos-and-zoau&data=05%7C01%7Csmetz3%40gmu.edu%7Ccfa1d3f0cb6144907fa608daa0104277%7C9e857255df574c47a0c00546460380cb%7C0%7C0%7C637998288955467761%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=CSIi3EDh0%2Bl60cgC%2B2ZSh6zbr6omoi4FwR125vs76UE%3D&reserved=0
>
> The challenge I see currently with Python is that is runs in a USS 
> environment so if you need to “ADDRESS TSO” you need to call a REXX from 
> Python in USS and do the other work.  Its probably not going to happen but a 
> “System” Python might fill that gap.
>
> Matt Hogstrom
> m...@hogstrom.org
>
> “It may be cognitive, but, it ain’t intuitive."
> — Hogstrom
>
>> On Sep 25, 2022, at 11:15 PM, David Crayford  wrote:
>>
>> On 26/9/22 10:43, Charles Mills wrote:
>>>> It's trivial to write an MVS I/O package if you have a C compiler.
>>> One might ask then why IBM has not done so.
>> I would suggest that they have not had a requirement. IBM use Python in 
>> their analytics products and for new stuff like Ansible. Same with golang, 
>> they need it for Kubernetes and OpenShift for z/CX containers. I doubt very 
>> much if many customers ha

Re: IBM python documentation?

[David Crayford]

On 27/9/22 00:34, Farley, Peter x23353 wrote:

Thanks for the reminder about the zoautil_py functions, I did know about them but there 
are gaps -- e.g. ZOAU has no direct support for simple sequential processing of VSAM 
files (e.g., to download a copy of VSAM data to a flat file in the Unix Services 
environment for analysis there).  I have used a Rexx script that executes the FileManager 
batch utility program FMNMAIN to perform such functions.  The FileManager batch program 
proved to be the most compatible in generating correct Unix Services output files for 
various flavors of VSAM input files where IDCAMS REPRO or DFSORT "OPTION COPY" 
outputs were less so.

Re: "ADDRESS TSO" needs - Rexx is not necessarily needed, as there is also the Unix Services 
command "tso" (or "tsocmd" for authorized services) to execute a single TSO command.  The 
python subprocess.run() function could handle executing that command and capturing the output as needed.

For more complicated "ADDRESS TSO" needs, yes one can execute a Rexx script in the Unix Services 
environment which uses "ADDRESS TSO" to execute a Rexx script also stored in that environment (or 
in an MVS library assigned to SYSEXEC) that then actually uses "ADDRESS TSO" functions.  A bit Rube 
Goldberg, but it does work, as my recent experimentation with getting LISTDSI output from the Unix Services 
environment proved.  Conceivably one could write that first Rexx script to set up an ISPF environment (all 
the needed DD's, etc.) and execute the second (functional) script in an ISPF environment (I haven't tried 
that yet; here there be dragons).


Should have used my lua port :) It supports all access methods and has a 
listdsi function that I wrote - io.listdsi(dsname)




Re: Writing an MVS I/O package for IBM python - ISTM that without documentation 
of the runtime architecture of the IBM python port (and also possibly access to 
the source of the port) it is not likely to succeed.  OCO strikes again.


It's certainly not OCO. IBM ship all the header files to write 
extensions including a zos.h. What you need to know is that it's 64-bit, 
enhanced ASCII and is compiled using the xlclang compiler. Extensions 
are shared objects so you need to following compiler options: -q64 
-qascii -qexportall. You may not need "exportall" if IBM have defined 
PyMODINIT_FUNC for the platform.





The new DSFS functionality may or may not relieve the need for an MVS-specific 
I/O package.  That remains to be seen.

Peter

-Original Message-
From: IBM Mainframe Discussion List  On Behalf Of 
Matt Hogstrom
Sent: Monday, September 26, 2022 8:28 AM
To: IBM-MAIN@LISTSERV.UA.EDU
Subject: Re: IBM python documentation?

If IIRC IBM loosely paired the IBM ZOAU to “assist Python”.  
https://community.ibm.com/community/user/ibmz-and-linuxone/blogs/austin-wells1/2020/12/11/using-ibm-open-enterprise-python-for-zos-and-zoau

The challenge I see currently with Python is that is runs in a USS environment 
so if you need to “ADDRESS TSO” you need to call a REXX from Python in USS and 
do the other work.  Its probably not going to happen but a “System” Python 
might fill that gap.

Matt Hogstrom
m...@hogstrom.org

“It may be cognitive, but, it ain’t intuitive."
— Hogstrom


On Sep 25, 2022, at 11:15 PM, David Crayford  wrote:

On 26/9/22 10:43, Charles Mills wrote:

It's trivial to write an MVS I/O package if you have a C compiler.

One might ask then why IBM has not done so.

I would suggest that they have not had a requirement. IBM use Python in their 
analytics products and for new stuff like Ansible. Same with golang, they need 
it for Kubernetes and OpenShift for z/CX containers. I doubt very much if many 
customers have tried golang. It's a great language now it supports generics. As 
fast as C++ with many advantages.



Charles

-Original Message-
From: IBM Mainframe Discussion List [mailto:IBM-MAIN@LISTSERV.UA.EDU]
On Behalf Of David Crayford
Sent: Sunday, September 25, 2022 6:57 PM
To: IBM-MAIN@LISTSERV.UA.EDU
Subject: Re: IBM python documentation?

On 26/9/22 07:34, Farley, Peter x23353 wrote:

I know Rocket's port of python has some documented enhancements to support MVS dataset access among 
other things, but I have failed to find any documentation on the IBM websites for an IBM-produced 
"python Programmers Guide" (or similar) that would describe and provide examples for any 
"IBM-specific" functional enhancements to the base language facilities.

Is there any such documentation?  Or are the python.org
documentation websites the only reference material available for the
IBM port of python? (i.e., no functional enhancements at all are
provided in the IBM port)

Correct! It's trivial to write an MVS I/O package if you have a C compiler.

--

This message and any attachments are intended only for the use of the address

Re: IBM python documentation?

[Colin Paice]

I'm working on a Python external function(s) which can process datasets and
//DDNAMEs etc ...  using C interface... I'll post here when it is available.
Colin

On Mon, 26 Sept 2022 at 17:35, Farley, Peter x23353 <
031df298a9da-dmarc-requ...@listserv.ua.edu> wrote:

> Thanks for the reminder about the zoautil_py functions, I did know about
> them but there are gaps -- e.g. ZOAU has no direct support for simple
> sequential processing of VSAM files (e.g., to download a copy of VSAM data
> to a flat file in the Unix Services environment for analysis there).  I
> have used a Rexx script that executes the FileManager batch utility program
> FMNMAIN to perform such functions.  The FileManager batch program proved to
> be the most compatible in generating correct Unix Services output files for
> various flavors of VSAM input files where IDCAMS REPRO or DFSORT "OPTION
> COPY" outputs were less so.
>
> Re: "ADDRESS TSO" needs - Rexx is not necessarily needed, as there is also
> the Unix Services command "tso" (or "tsocmd" for authorized services) to
> execute a single TSO command.  The python subprocess.run() function could
> handle executing that command and capturing the output as needed.
>
> For more complicated "ADDRESS TSO" needs, yes one can execute a Rexx
> script in the Unix Services environment which uses "ADDRESS TSO" to execute
> a Rexx script also stored in that environment (or in an MVS library
> assigned to SYSEXEC) that then actually uses "ADDRESS TSO" functions.  A
> bit Rube Goldberg, but it does work, as my recent experimentation with
> getting LISTDSI output from the Unix Services environment proved.
> Conceivably one could write that first Rexx script to set up an ISPF
> environment (all the needed DD's, etc.) and execute the second (functional)
> script in an ISPF environment (I haven't tried that yet; here there be
> dragons).
>
> Re: Writing an MVS I/O package for IBM python - ISTM that without
> documentation of the runtime architecture of the IBM python port (and also
> possibly access to the source of the port) it is not likely to succeed.
> OCO strikes again.
>
> The new DSFS functionality may or may not relieve the need for an
> MVS-specific I/O package.  That remains to be seen.
>
> Peter
>
> -Original Message-
> From: IBM Mainframe Discussion List  On Behalf
> Of Matt Hogstrom
> Sent: Monday, September 26, 2022 8:28 AM
> To: IBM-MAIN@LISTSERV.UA.EDU
> Subject: Re: IBM python documentation?
>
> If IIRC IBM loosely paired the IBM ZOAU to “assist Python”.
> https://community.ibm.com/community/user/ibmz-and-linuxone/blogs/austin-wells1/2020/12/11/using-ibm-open-enterprise-python-for-zos-and-zoau
>
>
> The challenge I see currently with Python is that is runs in a USS
> environment so if you need to “ADDRESS TSO” you need to call a REXX from
> Python in USS and do the other work.  Its probably not going to happen but
> a “System” Python might fill that gap.
>
> Matt Hogstrom
> m...@hogstrom.org
>
> “It may be cognitive, but, it ain’t intuitive."
> — Hogstrom
>
> > On Sep 25, 2022, at 11:15 PM, David Crayford 
> wrote:
> >
> > On 26/9/22 10:43, Charles Mills wrote:
> >>> It's trivial to write an MVS I/O package if you have a C compiler.
> >> One might ask then why IBM has not done so.
> >
> > I would suggest that they have not had a requirement. IBM use Python in
> their analytics products and for new stuff like Ansible. Same with golang,
> they need it for Kubernetes and OpenShift for z/CX containers. I doubt very
> much if many customers have tried golang. It's a great language now it
> supports generics. As fast as C++ with many advantages.
> >
> >
> >> Charles
> >>
> >> -Original Message-
> >> From: IBM Mainframe Discussion List [mailto:IBM-MAIN@LISTSERV.UA.EDU]
> >> On Behalf Of David Crayford
> >> Sent: Sunday, September 25, 2022 6:57 PM
> >> To: IBM-MAIN@LISTSERV.UA.EDU
> >> Subject: Re: IBM python documentation?
> >>
> >> On 26/9/22 07:34, Farley, Peter x23353 wrote:
> >>> I know Rocket's port of python has some documented enhancements to
> support MVS dataset access among other things, but I have failed to find
> any documentation on the IBM websites for an IBM-produced "python
> Programmers Guide" (or similar) that would describe and provide examples
> for any "IBM-specific" functional enhancements to the base language
> facilities.
> >>>
> >>> Is there any such documentation?  Or are the python.org
> >>> documentation websites t

Re: IBM python documentation?

[Farley, Peter x23353]

Thanks for the reminder about the zoautil_py functions, I did know about them 
but there are gaps -- e.g. ZOAU has no direct support for simple sequential 
processing of VSAM files (e.g., to download a copy of VSAM data to a flat file 
in the Unix Services environment for analysis there).  I have used a Rexx 
script that executes the FileManager batch utility program FMNMAIN to perform 
such functions.  The FileManager batch program proved to be the most compatible 
in generating correct Unix Services output files for various flavors of VSAM 
input files where IDCAMS REPRO or DFSORT "OPTION COPY" outputs were less so.

Re: "ADDRESS TSO" needs - Rexx is not necessarily needed, as there is also the 
Unix Services command "tso" (or "tsocmd" for authorized services) to execute a 
single TSO command.  The python subprocess.run() function could handle 
executing that command and capturing the output as needed.

For more complicated "ADDRESS TSO" needs, yes one can execute a Rexx script in 
the Unix Services environment which uses "ADDRESS TSO" to execute a Rexx script 
also stored in that environment (or in an MVS library assigned to SYSEXEC) that 
then actually uses "ADDRESS TSO" functions.  A bit Rube Goldberg, but it does 
work, as my recent experimentation with getting LISTDSI output from the Unix 
Services environment proved.  Conceivably one could write that first Rexx 
script to set up an ISPF environment (all the needed DD's, etc.) and execute 
the second (functional) script in an ISPF environment (I haven't tried that 
yet; here there be dragons).

Re: Writing an MVS I/O package for IBM python - ISTM that without documentation 
of the runtime architecture of the IBM python port (and also possibly access to 
the source of the port) it is not likely to succeed.  OCO strikes again.

The new DSFS functionality may or may not relieve the need for an MVS-specific 
I/O package.  That remains to be seen.

Peter

-Original Message-
From: IBM Mainframe Discussion List  On Behalf Of 
Matt Hogstrom
Sent: Monday, September 26, 2022 8:28 AM
To: IBM-MAIN@LISTSERV.UA.EDU
Subject: Re: IBM python documentation?

If IIRC IBM loosely paired the IBM ZOAU to “assist Python”.  
https://community.ibm.com/community/user/ibmz-and-linuxone/blogs/austin-wells1/2020/12/11/using-ibm-open-enterprise-python-for-zos-and-zoau
  

The challenge I see currently with Python is that is runs in a USS environment 
so if you need to “ADDRESS TSO” you need to call a REXX from Python in USS and 
do the other work.  Its probably not going to happen but a “System” Python 
might fill that gap.

Matt Hogstrom
m...@hogstrom.org

“It may be cognitive, but, it ain’t intuitive."
— Hogstrom

> On Sep 25, 2022, at 11:15 PM, David Crayford  wrote:
> 
> On 26/9/22 10:43, Charles Mills wrote:
>>> It's trivial to write an MVS I/O package if you have a C compiler.
>> One might ask then why IBM has not done so.
> 
> I would suggest that they have not had a requirement. IBM use Python in their 
> analytics products and for new stuff like Ansible. Same with golang, they 
> need it for Kubernetes and OpenShift for z/CX containers. I doubt very much 
> if many customers have tried golang. It's a great language now it supports 
> generics. As fast as C++ with many advantages.
> 
> 
>> Charles
>> 
>> -Original Message-
>> From: IBM Mainframe Discussion List [mailto:IBM-MAIN@LISTSERV.UA.EDU] 
>> On Behalf Of David Crayford
>> Sent: Sunday, September 25, 2022 6:57 PM
>> To: IBM-MAIN@LISTSERV.UA.EDU
>> Subject: Re: IBM python documentation?
>> 
>> On 26/9/22 07:34, Farley, Peter x23353 wrote:
>>> I know Rocket's port of python has some documented enhancements to support 
>>> MVS dataset access among other things, but I have failed to find any 
>>> documentation on the IBM websites for an IBM-produced "python Programmers 
>>> Guide" (or similar) that would describe and provide examples for any 
>>> "IBM-specific" functional enhancements to the base language facilities.
>>> 
>>> Is there any such documentation?  Or are the python.org 
>>> documentation websites the only reference material available for the 
>>> IBM port of python? (i.e., no functional enhancements at all are 
>>> provided in the IBM port)
>> Correct! It's trivial to write an MVS I/O package if you have a C compiler.
--

This message and any attachments are intended only for the use of the addressee 
and may contain information that is privileged and confidential. If the reader 
of the message is not the intended recipient or an authorized representative of 
the intended recipient, you are hereby notified that any dissemination of this 
communication is 

Re: IBM python documentation?

[Matt Hogstrom]

If IIRC IBM loosely paired the IBM ZOAU to “assist Python”.  
https://community.ibm.com/community/user/ibmz-and-linuxone/blogs/austin-wells1/2020/12/11/using-ibm-open-enterprise-python-for-zos-and-zoau

The challenge I see currently with Python is that is runs in a USS environment 
so if you need to “ADDRESS TSO” you need to call a REXX from Python in USS and 
do the other work.  Its probably not going to happen but a “System” Python 
might fill that gap.

Matt Hogstrom
m...@hogstrom.org

“It may be cognitive, but, it ain’t intuitive."
— Hogstrom

> On Sep 25, 2022, at 11:15 PM, David Crayford  wrote:
> 
> On 26/9/22 10:43, Charles Mills wrote:
>>> It's trivial to write an MVS I/O package if you have a C compiler.
>> One might ask then why IBM has not done so.
> 
> I would suggest that they have not had a requirement. IBM use Python in their 
> analytics products and for new stuff like Ansible. Same with golang, they 
> need it for Kubernetes and OpenShift for z/CX containers. I doubt very much 
> if many customers have tried golang. It's a great language now it supports 
> generics. As fast as C++ with many advantages.
> 
> 
>> Charles
>> 
>> -Original Message-
>> From: IBM Mainframe Discussion List [mailto:IBM-MAIN@LISTSERV.UA.EDU] On 
>> Behalf Of David Crayford
>> Sent: Sunday, September 25, 2022 6:57 PM
>> To: IBM-MAIN@LISTSERV.UA.EDU
>> Subject: Re: IBM python documentation?
>> 
>> On 26/9/22 07:34, Farley, Peter x23353 wrote:
>>> I know Rocket's port of python has some documented enhancements to support 
>>> MVS dataset access among other things, but I have failed to find any 
>>> documentation on the IBM websites for an IBM-produced "python Programmers 
>>> Guide" (or similar) that would describe and provide examples for any 
>>> "IBM-specific" functional enhancements to the base language facilities.
>>> 
>>> Is there any such documentation?  Or are the python.org documentation 
>>> websites the only reference material available for the IBM port of python? 
>>> (i.e., no functional enhancements at all are provided in the IBM port)
>> Correct! It's trivial to write an MVS I/O package if you have a C compiler.
>> 
>> --
>> For IBM-MAIN subscribe / signoff / archive access instructions,
>> send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
>> 
>> --
>> For IBM-MAIN subscribe / signoff / archive access instructions,
>> send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
> 
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to lists...@listserv.ua.edu <mailto:lists...@listserv.ua.edu> with 
> the message: INFO IBM-MAIN


--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: IBM python documentation?

[Wayne Bickerdike]

Shades of Monty Python:

IBM:
"Can I help you sir?"
Customer:
"I'm looking for an MVS IO package for Python".
IBM:
"You're the fifth bloke in here today asking for that, there's no demand
for it.."

On Mon, Sep 26, 2022 at 1:15 PM David Crayford  wrote:

> On 26/9/22 10:43, Charles Mills wrote:
> >> It's trivial to write an MVS I/O package if you have a C compiler.
> > One might ask then why IBM has not done so.
>
> I would suggest that they have not had a requirement. IBM use Python in
> their analytics products and for new stuff like Ansible. Same with
> golang, they need it for Kubernetes and OpenShift for z/CX containers. I
> doubt very much if many customers have tried golang. It's a great
> language now it supports generics. As fast as C++ with many advantages.
>
>
> > Charles
> >
> > -Original Message-
> > From: IBM Mainframe Discussion List [mailto:IBM-MAIN@LISTSERV.UA.EDU]
> On Behalf Of David Crayford
> > Sent: Sunday, September 25, 2022 6:57 PM
> > To: IBM-MAIN@LISTSERV.UA.EDU
> > Subject: Re: IBM python documentation?
> >
> > On 26/9/22 07:34, Farley, Peter x23353 wrote:
> >> I know Rocket's port of python has some documented enhancements to
> support MVS dataset access among other things, but I have failed to find
> any documentation on the IBM websites for an IBM-produced "python
> Programmers Guide" (or similar) that would describe and provide examples
> for any "IBM-specific" functional enhancements to the base language
> facilities.
> >>
> >> Is there any such documentation?  Or are the python.org documentation
> websites the only reference material available for the IBM port of python?
> (i.e., no functional enhancements at all are provided in the IBM port)
> > Correct! It's trivial to write an MVS I/O package if you have a C
> compiler.
> >
> > --
> > For IBM-MAIN subscribe / signoff / archive access instructions,
> > send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
> >
> > --
> > For IBM-MAIN subscribe / signoff / archive access instructions,
> > send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
>
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
>


-- 
Wayne V. Bickerdike

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: IBM python documentation?

[David Crayford]

On 26/9/22 10:43, Charles Mills wrote:

It's trivial to write an MVS I/O package if you have a C compiler.

One might ask then why IBM has not done so.


I would suggest that they have not had a requirement. IBM use Python in 
their analytics products and for new stuff like Ansible. Same with 
golang, they need it for Kubernetes and OpenShift for z/CX containers. I 
doubt very much if many customers have tried golang. It's a great 
language now it supports generics. As fast as C++ with many advantages.




Charles

-Original Message-
From: IBM Mainframe Discussion List [mailto:IBM-MAIN@LISTSERV.UA.EDU] On Behalf 
Of David Crayford
Sent: Sunday, September 25, 2022 6:57 PM
To: IBM-MAIN@LISTSERV.UA.EDU
Subject: Re: IBM python documentation?

On 26/9/22 07:34, Farley, Peter x23353 wrote:

I know Rocket's port of python has some documented enhancements to support MVS dataset access among 
other things, but I have failed to find any documentation on the IBM websites for an IBM-produced 
"python Programmers Guide" (or similar) that would describe and provide examples for any 
"IBM-specific" functional enhancements to the base language facilities.

Is there any such documentation?  Or are the python.org documentation websites 
the only reference material available for the IBM port of python? (i.e., no 
functional enhancements at all are provided in the IBM port)

Correct! It's trivial to write an MVS I/O package if you have a C compiler.

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN


--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: IBM python documentation?

[Charles Mills]

> It's trivial to write an MVS I/O package if you have a C compiler.

One might ask then why IBM has not done so.

Charles

-Original Message-
From: IBM Mainframe Discussion List [mailto:IBM-MAIN@LISTSERV.UA.EDU] On Behalf 
Of David Crayford
Sent: Sunday, September 25, 2022 6:57 PM
To: IBM-MAIN@LISTSERV.UA.EDU
Subject: Re: IBM python documentation?

On 26/9/22 07:34, Farley, Peter x23353 wrote:
> I know Rocket's port of python has some documented enhancements to support 
> MVS dataset access among other things, but I have failed to find any 
> documentation on the IBM websites for an IBM-produced "python Programmers 
> Guide" (or similar) that would describe and provide examples for any 
> "IBM-specific" functional enhancements to the base language facilities.
>
> Is there any such documentation?  Or are the python.org documentation 
> websites the only reference material available for the IBM port of python? 
> (i.e., no functional enhancements at all are provided in the IBM port)
Correct! It's trivial to write an MVS I/O package if you have a C compiler.

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: IBM python documentation?

[David Crayford]

On 26/9/22 07:34, Farley, Peter x23353 wrote:

I know Rocket's port of python has some documented enhancements to support MVS dataset access among 
other things, but I have failed to find any documentation on the IBM websites for an IBM-produced 
"python Programmers Guide" (or similar) that would describe and provide examples for any 
"IBM-specific" functional enhancements to the base language facilities.

Is there any such documentation?  Or are the python.org documentation websites 
the only reference material available for the IBM port of python? (i.e., no 
functional enhancements at all are provided in the IBM port)

Correct! It's trivial to write an MVS I/O package if you have a C compiler.

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN