The current approach is this 1.Export MySQL (or origin database) to XML files 2.Import the XML files in one at a time and either succeed or fail the entire file.
Because of the SAX parser you're limited to in best case scenario to what you can fit into memory (because of the data OFBiz's data import logic, you're limited to one file at a time, even if two or more will fit in memory). Therefore you're guessing if referential integrity is maintained (exists) in subsequent XML data files. In the case of Postgres, once OFBiz creates the database schema, postgres handles referential integrity constraints. Because of this, it's not enough just to have the entity engine ignore its error on referential integrity with a dummy key, the dummy key actually has to be written to the destination database(which in my experience did not happen, I didn't look any deeper into this as a solution because of the other remaining issues). By reading all of the XML data files into an XML database first, children elements can be added to each record with error information. You'll also be able to trace through referential integrity to ensure it's maintained in subsequent records and then actually import the data with the PK records going in first. Additionally, you can test an entire data set for importation, report back all the errors and let the user make adjustments as is needed. It should be very interesting and fairly simple to implement as the solution is more logic based instead of code based. It would be great if someone could review OFBIZ-851 and maybe add it as a specialpurpose app so that others can play with it and contribute ideas. --- Jonathon -- Improov <[EMAIL PROTECTED]> wrote: > Chris, > > You mean go from MySQL to Xindice to PostgreSQL? Yeah, I know, data > migration can be a pain, even > without data-mapping efforts to go with it (ie, same structure > migrated to same structure).
