Hello all, I'm new to both this list, XSL, and Xalan so forgive any mistakes or ignorance on my part.
The Problem: Basically, I have a large amount of data that exists in an XML format (w/ DTD). I need to get the XML data into a relational database which has an existing schema. From what I've read so far it seems like I can use XSLT to generate SQL statements that then get sent to the database via JDBC or whatever. The real problem is the amount of data I have to deal with. To start off there will be ~ 10 million records followed by weekly updates of ~ 10k records or so. Each XML file will have to generate many INSERT, UPDATE, and DELETE SQL statements all wrapped in a transaction, so that the whole file can be rejected if something goes wrong. Question #1: Is Xalan or even XSLT right for this job? It seems like this is a fairly common problem there must be existing tools. Question #2: Does anyone have any comments or suggestions for dealing with large data sets and databases? Optimizations, things to watch out for and what not. Does Xalan have to load the whole XML file into memory in order to work with it? Question #3: Another potential problem is that while parsing the document we won't know if an entity should generate an UPDATE or an INSERT without first checking the database to see if that record already exists. How will Xalan handle this? Thanks in advance for the abundance of wisdom, charity, and encouragement that I'm about to receive from the good people of Xalan land. =) -ishmael
