I created an ETL and installed it at a client site, now the company I currently work for has been marketing it recently.? It is called "The SQLizer".? It takes the Unidata/Universe files it SQLizes and splits off the associations/MV fields into their own tables as part of the normalization process.?
When I started I thought I'd use ODBC, but it turns out it is very easy to just dump the Uni files into text files and then have a short perl program do the explodes as needed. To transport and load, I am using the TMI Listener which is bundled with the SQLizer.? Target databases are SQL Server, MySQL, and Oracle.? SQLizer is being marketed for just under $10k. It can load SQL "on demand" (such as overnight batches) or keep relatively current using index-based triggers.? Since it is all batch based (phantom) it does not take up licenses (unless your site's phantoms *have* to take up a license due to being iphatoms.)? All it's calls to sockets are in perl, it is almost like calling ftp to run from Basic, so the phantoms stay phantoms. I have some questions for folks: ? 1) Are subvalue marks commonly used?? Right now the current users don't have an issue, the few tables that use it we have been splitting SVMs out into columns, but in general I was wondering how prevalent the use of SVMs has been. 2) Do folks generally have good metadata at their sites?? The U2 dictionaries are often unreliable and I have a data scanner to see what the data actually is, but I was wondering if folks typically setup their own metadata. 3) Do folks typically want overnight updates or to keep things relatively current throughout the day? 4) How much transformation would folks like to do on the U2 side?? I have the ability to trigger updates and then run a transformation program as part of the pre-SQLization, as well as the ability to do some on-the-fly I-descriptors, but do folks typically just want things SQLized to do the manipulations on the other side? 5) What other features are folks looking for? Thanks for any info you can provide! Steve... -- Steve Kneizys [email protected] -----Original Message----- From: Tony G <[email protected]> To: [email protected] Sent: Thu, Jul 30, 2009 3:49 am Subject: Re: [U2] What ETL? Somehow when someone mentions BI there are suggestions for Reporting tools, and with notes about ETL come notes about BI and Reporting. In my mind (what there is of it) there is a vast difference between these concepts. I think the confusion comes in when a product like MITS, for example, incorporates its own ETL functionality to accomplish what it does. Some basic reporting tools do the same in the name of performance. But the ETL performed by these tools is generally proprietary and cannot be used in other contexts. Now when you're talking about a real ETL platform like DataStage, you should be able to use data from platform X with any platform Y. A product like MITS or Cognos or Informer or any other could code their front-end processes to use DataStage as a data source. This would open them for use, even with non-MV platforms like Oracle or DB2 - and of course products like Cognos do exactly that - but our MV-centric colleagues generally don't think in those directions. Speaking of DataStage, I was discussing an association with a company a while back for providing mainstream BI tools for MV (that option is still considered from time to time and interested parties are welcome to contact me). As we can all relate, I had to spend a lot of time explaining the Pick/MV concepts which were completely unfamiliar to them. In our discussion we decided that the best way to use common BI tools with MV was not to link directly to MV at all as a data source, but to use a middle-tier ETL tool, I could provide the extraction from MV and they could extract from the generic middle-tier using common queries and tools. Ironically when we were discussing what tools they already used, they mentioned DataStage. Maybe they were unique but it seems to me that DataStage could be considered a poster-child as a successful MV application, but somehow that marketing value seems completely untapped. *sigh* Tony Gravagno Nebula Research and Development TG@ remove.pleaseNebula-RnD.com _______________________________________________ U2-Users mailing list [email protected] http://listserver.u2ug.org/mailman/listinfo/u2-users _______________________________________________ U2-Users mailing list [email protected] http://listserver.u2ug.org/mailman/listinfo/u2-users
