On 6/6/2023 12:02 am, Farley, Peter wrote:
Which is fine for those who have access to those tools; many of us do not have that
luxury, nor the permission nor the disk space to "try them out" for ourselves.
That's a really unfortunate situation. I encourage you to talk to your
management and try to convince them to change their policy. All the cool
new tools and technologies, like dynamic languages, compilers,
libraries, frameworks, and more, often require USS (UNIX System
Services). By not giving you access to USS, they're basically keeping
you stuck in the past and preventing your growth. It's important to make
them understand that adapting and learning new things is essential for
professional development, no matter how old you are.
-----Original Message-----
From: IBM Mainframe Discussion List <[email protected]> On Behalf Of
Lionel B. Dyck
Sent: Monday, June 5, 2023 11:44 AM
To: [email protected]
Subject: Re: z/OS 3.1: Now UNIXR Certified
Peter - I believe you'll find that the Co:Z folks, at coztoolkit.com, have
tools that may provide the performance improvements that you are looking for.
The ZIGI tool will take advantage of their getpds and putpds commands if they
are available and achieve significantly improved performance compared to native
cp.
Lionel B. Dyck <><
Website:
https://urldefense.com/v3/__https://www.lbdsoftware.com__;!!Ebr-cpPeAnfNniQ8HSAI-g_K5b7VKg!LAIovDokCs-oEgMZ6WCSnTOL8NyW8GN8r0RbiJPkLNybyfjLR0U7xWJijkAwrCmc7-VI_n5W5CsQYcDqizc$
Github:
https://urldefense.com/v3/__https://github.com/lbdyck__;!!Ebr-cpPeAnfNniQ8HSAI-g_K5b7VKg!LAIovDokCs-oEgMZ6WCSnTOL8NyW8GN8r0RbiJPkLNybyfjLR0U7xWJijkAwrCmc7-VI_n5W5CsQyKQdnzg$
“Worry more about your character than your reputation. Character is what you
are, reputation merely what others think you are.” - - - John Wooden
-----Original Message-----
From: IBM Mainframe Discussion List <[email protected]> On Behalf Of
Farley, Peter
Sent: Monday, June 5, 2023 10:37 AM
To: [email protected]
Subject: Re: z/OS 3.1: Now UNIXR Certified
Where the performance of the USS file system falls down is in the transfer between z/OS files/data stores and
USS files/data stores. Practical experimentation has shown me that the "cp" and "cat"
commands are the fastest way to sequentially transfer non-DB2 data using only a pre-written utility (and both
do support VSAM files), but that is not to say those are fast processes compared to directly reading and
writing files in the "other system". When the business-important permanent data store is not
originally in the USS file system you must deal with that transfer time in the application design or write
your own data transfer code to achieve an efficiently running system.
Neither zowe nor zoau provide good performance in the data transfer process. The zowe transfer time is
reasonable for one-offs (but still not as fast as "cp" or "cat") but zoau is horribly
slow no matter how it is used. The DB2CLI is a significant time consumer; the python-ibmdb module does it
far faster, though that product is still officially in "beta" mode.
As usual, the best way to optimize data access time is good design and a
thorough knowledge of the technical benefits and flaws of your system.
-----Original Message-----
From: IBM Mainframe Discussion List <[email protected]> On Behalf Of
David Crayford
Sent: Monday, June 5, 2023 7:06 AM
To: [email protected]
Subject: Re: z/OS 3.1: Now UNIXR Certified
One compelling reason to embrace zFS is its potential for modernization and
facilitating the development of contemporary tools. While acknowledging the
significance of QSAM, VSAM KSDS, and other older technologies, it is crucial to
recognize the advancements made in data structure formats for disk files since
the days of VSAM. In the present era, LSM-trees have gained popularity for
their application in NoSQL key-value stores, blazing-fast TSDBs, and highly
optimized logging systems.
Attempting to implement an LSM-tree using VSAM would be an arduous endeavor,
bordering on a nightmare. Even with the assistance of Media Manager, it remains
a Herculean task to reconcile these two disparate technologies.
I dedicated a couple of hours to porting RocksDB, and the results have been
nothing short of exceptional. It operates seamlessly on z/OS, demonstrating its
prowess and resilience. Another noteworthy aspect of LSM-trees is their
inherent ability to merge and compact while in operation, eliminating the need
for reorgs.
https://urldefense.com/v3/__https://github.com/facebook/rocksdb__;!!Ebr-cpPeAnfNniQ8HSAI-g_K5b7VKg!LzT4O_YlJ0b4g_9FS6BO4zVHzZ55N3E8k4QPWtQx_hnCZfUjLh3mDAsW-PYPhMIqZnaz3WNQfEhFE3UKfk3uzfY$
https://urldefense.com/v3/__https://www.youtube.com/watch?v=I6jB0nM9SKU__;!!Ebr-cpPeAnfNniQ8HSAI-g_K5b7VKg!LzT4O_YlJ0b4g_9FS6BO4zVHzZ55N3E8k4QPWtQx_hnCZfUjLh3mDAsW-PYPhMIqZnaz3WNQfEhFE3UKiaUQef4$
On 5/6/2023 5:55 pm, David Crayford wrote:
On 2/6/2023 11:31 pm, René Jansen wrote:
What I remember of it is that he was convinced it was a lot slower.
He was mistaken! I've tested it out, and QSAM is no match for zFS. You
can find the details in this gist:
https://urldefense.com/v3/__https://gist.github.com/daveyc/14b45d6d70d8dd9af1848e539d78881f__;!!Ebr-cpPeAnfNniQ8HSAI-g_K5b7VKg!LzT4O_YlJ0b4g_9FS6BO4zVHzZ55N3E8k4QPWtQx_hnCZfUjLh3mDAsW-PYPhMIqZnaz3WNQfEhFE3UKrdRm7Qs$
.
Adding an fsync() call after writing each record barely incurs any
overhead. zFS, operating with highly optimized Media Manager APIs,
handles it efficiently. Additionally, zFS functions as a caching file
system.
I have observed a certain degree of snobbery among many
traditionalists when it comes to USS. I can recall an incident from
approximately 15 years ago when I advocated for the use of sqlite in
one of our products. My boss dismissed the idea, expressing concerns
that customers might be deterred by using the UNIX file system.
Consequently, we opted for a VSAM KSDS, despite its inherent
limitations. Interestingly, it is worth noting that there are now
numerous IBM z/OS products that embrace sqlite, with some even
integrating it with HLASM.
So I told him that nobody forced him not to use QSAM for datasets just because
it ran in USS. And it think that is a great asset of it. Just because Unix
forces you to have a hierarchical directory system does not mean, in USS, that
you need to use it for all I/O.
René.
On 2 Jun 2023, at 17:03, Seymour J Metz<[email protected]> wrote:
Dubbing is part of the setup overhead for a task, and only occurs once, so
except for very short tasks it is just noise in measuring performance.
As for the general overhead of Unix System Services, the Devil is in the
details. For a comparison to be reasonable, the two programs have to be using
the services in a comparable fashion. Was your COBOL programmer really
comparing the overhead of conventional access methods to Unix file I/O, or were
the numbers drowned out by, e.g., differences in application logic?
--
Shmuel (Seymour J.) Metz
https://urldefense.com/v3/__http://mason.gmu.edu/*smetz3__;fg!!Ebr-c
pPeAnfNniQ8HSAI-g_K5b7VKg!LzT4O_YlJ0b4g_9FS6BO4zVHzZ55N3E8k4QPWtQx_h
nCZfUjLh3mDAsW-PYPhMIqZnaz3WNQfEhFE3UKsIUJTkY$
--
This message and any attachments are intended only for the use of the addressee
and may contain information that is privileged and confidential. If the reader
of the message is not the intended recipient or an authorized representative of
the intended recipient, you are hereby notified that any dissemination of this
communication is strictly prohibited. If you have received this communication
in error, please notify us immediately by e-mail and delete the message and any
attachments from your system.
----------------------------------------------------------------------
For IBM-MAIN subscribe / signoff / archive access instructions, send email to
[email protected] with the message: INFO IBM-MAIN
----------------------------------------------------------------------
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to [email protected] with the message: INFO IBM-MAIN
This message and any attachments are intended only for the use of the addressee
and may contain information that is privileged and confidential. If the reader
of the message is not the intended recipient or an authorized representative of
the intended recipient, you are hereby notified that any dissemination of this
communication is strictly prohibited. If you have received this communication
in error, please notify us immediately by e-mail and delete the message and any
attachments from your system.
----------------------------------------------------------------------
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to [email protected] with the message: INFO IBM-MAIN
----------------------------------------------------------------------
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to [email protected] with the message: INFO IBM-MAIN
