Rex Pommier write:
>Moving the data to the mainframe isn't an option here.
I'm talking about *not* moving *any* data. ("Doc, it hurts when I move
data." / "Don't move data.")
There is a concept known as federated Apache Spark analytics. I recommend
looking into it. IBM Open Data Analytics for z/OS is the Spark ingredient
for z/OS-based data.
IBM Data Virtualization Manager for z/OS is another possible approach (or
in combination), and again none of the data moves. But you still perform
the same analytics, reporting, etc. that you do today. That part shouldn't
have to change.
I happened to listen to a really excellent presentation a couple days ago,
given by a presenter from one of the U.S. national laboratories. He had a
simple chart illustrating the extreme divergence between processing
improvements and storage bandwidth improvements (memory bandwidth in their
case, but the same principle applies to other I/O types). Both have
improved over the years, but the former has improved a lot more than the
latter -- by about three orders of magnitude over the time interval he
illustrated. He then discussed some of the techniques they're using to work
with (rather than try to fight) these computing trends. In summary, they're
trying to make every bit count (yes, bits in their case), to keep data
movements in check. It's all very logical. We (industry here) championed a
lot of ELT/ETL starting about 30 years ago, back when that divergence
hadn't happened. That approach worked pretty well back then. But it's
getting progressively more painful now, and one of the symptoms is the "ETL
problem," the increasing burdens of moving data. This problem is popping up
all over the place as data volumes and data uses explode.
So, it's time for a rethink, to mark the solution approach to market in
2018 and beyond.
--------------------------------------------------------------------------------------------------------
Timothy Sipples
IT Architect Executive, Industry Solutions, IBM Z & LinuxONE,
Multi-Geography
E-Mail: [email protected]
----------------------------------------------------------------------
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to [email protected] with the message: INFO IBM-MAIN
