Concerning SCDs, another question comes to mind. I have a large fact table having foreign keys to some big dimensions. I want to load this fact table incrementally (no truncation of the fact table) in order to keep the ETL process short. So the dimensions have to be loaded incrementally, too. Changed dimension records (type 1 ATM) have to keep their surrogate keys, so that only new fact records have to be loaded in each batch.
With the current design of AW-ETL, this seems impossible. SCDs always delete the old record and insert a new record using the bulk loader. I can use the surrogate key generator on a field, but it doesn't seem to honor surrogate keys on existing dimension records. This pertains to type 2 mode - the type 1 mode only works when truncating and reloading the complete dimension table. I would migrate to type 2 SCDs here if that helped. I really don't understand how currently type 2 SCDs are supposed to work with AW-ETL. The only possibility I can imagine is to have the natural key in the fact table. Then you could constrain on the natural key and the effective dates in the dimension table using a date in the fact table. Really complicated! Some hints would be greatly appreciated. Regards Martin _______________________________________________ Activewarehouse-discuss mailing list Activewarehouse-discuss@rubyforge.org http://rubyforge.org/mailman/listinfo/activewarehouse-discuss
