DataStage was originally created by VMARK, using UniVerse 9.4 as its underlying engine and UniVerse Objects for communication between its client and "server" components. It also made use of BCI for communicating with other databases using ODBC protocols, and GCI for communication with C-callable database client software. (It also leveraged VMARK's experience at migrating from one database to another (to UniVerse!), for example in its understanding that the process needs not just data but, even more important, metadata. It (DataStage) also made use of the BASIC programming language in UniVerse to allow users to create their own transformation functions and callable subroutines.
As VMARK merged with UniData to become Ardent, which was then acquired by Informix, DataStage remained a UniVerse application, with DataStage 5.2 being the last such version, being built on UniVerse 9.6. Some of the DW pieces from the UniData side of things became adjoint products (Quality Manger, PRISM Warehouse Executive to name two), while some of that technology was incorporated into DataStage. When Informix, by that time controlled by the ex-Ardent board (and that's an interesting story in itself!), sold their databases to IBM, arrangements were put in place that both companies would begin from the same base, but development would occur separately as the separate products' requirements dictated. In particular, it became necessary that the DataStage Engine and UniVerse could run on the same machine. To achieve this, the "home" directory for DataStage is now recorded in /.dshome on UNIX platforms, rather than /.uvhome, the RPC daemon runs as dsprcd, the deadlock daemon runs as dsdlockd, and so on. Shared memory identifiers use a different key; where UniVerse keys have the form 0xace..., DataStage keys have the form 0xdae... There are a few other changes at this "magic number" level, one implication of which is that BASIC code compiled on the one will not run on the other. No less significant (for ETL performance) are some things that were removed from the DataStage Engine as unnecessary. For example, it does not have a spooler subsystem, it does not have networking software like UV/Net. Some of the configuration parameters have different default values, and there are some configuration parameters in DataStage that simply do not exist in UniVerse. Probably the major change in DataStage is the ability to cache hashed files in memory. One of the most heavy-duty tasks in ETL processing is reference lookups; if the source can be guaranteed to be in memory these can be very fast. Since the split, it has been possible to cache hashed files in memory for writing, too; when populating the hashed file writes to random groups occur into memory then, when they are all done, the file is flushed to disk one group at a time, in group order. Very efficient. Currently (version 7.0 of DataStage), there is still support for the query langugages, paragraphs, even PROCs. If you were to move to a DataStage site, you could find your way around. Although the preferred design approach is to use the GUI and not to code things - this approach is seen as being more easy to maintain, plus there are benefits in ease of documentation. There have been a lot of non-UniVerse things done to DataStage since the split with UniVerse; in particular there is a second engine, called Orchestrate, which Ascential got when it acquired Torrent Systems. This is a mechanism for automatically generating parallel processing in an SMP or MPP environment. In a fairly busy burst of acquisition, using some of the funds from IBM, they also acquired MetaRecon data profiling software from MetaGenix, INTEGRITY data cleansing software from Vality, and Mercator most recently of all. These technologies are integrated into the overall scheme of what Ascential calls "Enterprise Data Integration" in which DataStage is only one pillar. Underpinning it all is MetaStage, a technology for managing metadata in a semantic fashion so that metadata can be shared with many consumers and providers of metadata, such as CASE tools, database management systems, and business intelligence tools. There have been two major releases of DataStage since UniVerse was sold to IBM, versions 6.0 and 7.0. Version 7.1 is due out in a couple of months. It is still recognizably UniVerse under the covers, though there are rumours that a new Engine is under development. We have already seen the beginnings of the likely direction for this in version 7.0, in which there is a component (interlude?) in Orchestrate that allows BASIC code to be executed, by loading the run machine. I suspect (personal opinion/educated guess only) that the new DataStage Engine will not be obviously UniVerse-based, but will continue to be able to do many of the things that UniVerse can do, but not all, and quite a few that UniVerse can't do. In summary I would disagree that DataStage Engine is a stripped down or simplified version of UniVerse. Certainly a couple of unnecessary things have been removed (or not put in, such as triggers on base files). On the other hand, quite a few things have been put in, that aren't in - aren't required in - UniVerse. They're different products and, as development of each progresses, I would expect to see greater divergence in functionality. After all, they do quite different jobs. It is to be hoped that the DataStage engineers (some of whom are ex UniVerse) and the IBM U2 engineers continue to exchange ideas. Ultimately, however, it is not engineers who decide product directions. Recall the Golgafrinchan "B" Ark. ----- Original Message ----- From: [EMAIL PROTECTED] Date: Sat, 31 Jan 2004 21:02:16 EST To: [EMAIL PROTECTED] Subject: Re: [ot] Peoplesoft migrates to Ascential Could anyone elucidate a bit on what exactly was meant by saying that Datastage is a stipped-down, or more simplified, or more purified or whatever (!) version of Universe? Like maybe an example, or some details about how exactly Universe was modded into Datastage? Thanks Will _______________________________________________ u2-users mailing list [EMAIL PROTECTED] http://www.oliver.com/mailman/listinfo/u2-users
