[WiX-users] rfc: break WiX 3.0 backwards-compatibility with WiX 2.0 library (wixlib) files
WiX 3.0 currently has the ability to read library (.wixlib) files generated by the WiX 2.0 version of the toolset. However, weve recently identified several reasons why wed like to stop maintaining backwards-compatibility with the 2.0 format. The overall goal here is to make the changes necessary so that we never need to touch the wixobj or wixlib file formats ever again. All the proposed changes should make the contents of the wixobj and wixlib files so generic that all future improvements can be made by merely working with the existing concept of unreal tables instead of special 1-off xml and unreal columns. Drawbacks The one obvious drawback of this change is that customers using WiX 2.0 and 3.0 will not be able to share wixlibs from 2.0 to 3.0 as they may have done before. All 2.0 libraries must be converted to the 3.0 format (or re-built in 3.0) to work. For most groups, we anticipate that this will not be a problem since the move to 3.0 should only be done for a new product release and mixing versions of WiX in your build process is currently not advisable (its not a scenario we test very often). From 3.0 onward, however, we should be able to keep a consistent file format for wixobj/wixlib/wixout files. Advantages 1. Remove unreal columns from real tables Currently WiX internally uses a concept of unreal columns to associate additional information with standard MSI tables. For example, to associate a file path with a File row, WiX has a special Source column in the File table which is considered an unreal column. This basically means that wixobj and wixlib files carry the column but the final MSI file does not. The big danger with using this method of persisting additional information about standard tables is that should the MSI team ever decide to add additional columns to the tables, WiX will need to add hacks to ignore its unreal columns since all columns are addressed by their index (not the name of the column). In order to prepare for the possibility of the MSI team adding new columns to existing standard tables, wed like to remove the unreal column concept. This doesnt mean that metadata can no longer be associated with standard tables it just means it needs to be stored in a separate table like a WixFile table with a foreign key matching a File table entry. 2. Prefix wix-specific unreal tables with Wix There are currently several WiX-specific tables used between candle and light which do not actually appear in any MSI files. However, these tables do reside in the same namespace as normal tables that will be put in the MSI file. Some of these tables include: - FeatureGroup supports ComponentGroup authoring concepts - ComponentGroup support ComponentGroup authoring concept - Merge supports merge modules - Actions supports scheduling for standard and custom actions - SuppressAction supports suppression of actions - CustomTables supports custom tables without needing an extension - EnsureTables supports ensuring a table exists in an MSI file - RowData contains row information for CustomTables - UI supports UI elements The danger is that should MSI or any other group decide to use one of these names for a table in their setup package, a collision will occur and WiX will not be able to represent it properly. In order to prepare for this scenario, wed like to preface all WiX-specific table names with Wix similar to how MSI deals with collisions since MSI 2.0 but prefixing all their tables with Msi. This prefix will essentially become a namespace for WiX-specific tables and should not collide with other products. This change will not affect custom action tables like IIsWebSite, SecureObj, XmlFile, etc These must now stay consistent since they ship in MSI files to avoid breaking scenarios in which customers already use these tables. 3. Remove custom xml in wixobj wixlib files Currently WiX passes special information between the compiler and linker in the form of special xml in the wixobj and wixlib files. Over time, we came to recognize that usage of this custom xml was not very extensible and cause a lot of problems whenever a change needed to be made because it basically invalidated the entire wixobj and wixlib file formats. Going forward, we have recognized that the best way to avoid this problem is to persist information between the compiler and linker using unreal tables. This is why WiX 3.0 recently stopped using complexReference elements in the wixobj files and instead switching to unreal WixComplexReference tables. The tables basically allow us to add columns as necessary without breaking backwards-compatibly with previous versions of the wix toolset. This change removing custom xml from the wixobj and wixlib files can be made without sacrificing backwards-compatibility with WiX 2.0 wixlib files. However, Ive called it out here because this change is what prompted us to begin looking at breaking backwards-compatibility. Basically, we found
[WiX-users] rfc: break WiX 3.0 backwards-compatibility with WiX 2.0 library (wixlib) files
WiX 3.0 currently has the ability to read library (.wixlib) files generated by the WiX 2.0 version of the toolset. However, weve recently identified several reasons why wed like to stop maintaining backwards-compatibility with the 2.0 format. The overall goal here is to make the changes necessary so that we never need to touch the wixobj or wixlib file formats ever again. All the proposed changes should make the contents of the wixobj and wixlib files so generic that all future improvements can be made by merely working with the existing concept of unreal tables instead of special 1-off xml and unreal columns. Drawbacks The one obvious drawback of this change is that customers using WiX 2.0 and 3.0 will not be able to share wixlibs from 2.0 to 3.0 as they may have done before. All 2.0 libraries must be converted to the 3.0 format (or re-built in 3.0) to work. For most groups, we anticipate that this will not be a problem since the move to 3.0 should only be done for a new product release and mixing versions of WiX in your build process is currently not advisable (its not a scenario we test very often). From 3.0 onward, however, we should be able to keep a consistent file format for wixobj/wixlib/wixout files. Advantages 1. Remove unreal columns from real tables Currently WiX internally uses a concept of unreal columns to associate additional information with standard MSI tables. For example, to associate a file path with a File row, WiX has a special Source column in the File table which is considered an unreal column. This basically means that wixobj and wixlib files carry the column but the final MSI file does not. The big danger with using this method of persisting additional information about standard tables is that should the MSI team ever decide to add additional columns to the tables, WiX will need to add hacks to ignore its unreal columns since all columns are addressed by their index (not the name of the column). In order to prepare for the possibility of the MSI team adding new columns to existing standard tables, wed like to remove the unreal column concept. This doesnt mean that metadata can no longer be associated with standard tables it just means it needs to be stored in a separate table like a WixFile table with a foreign key matching a File table entry. 2. Prefix wix-specific unreal tables with Wix There are currently several WiX-specific tables used between candle and light which do not actually appear in any MSI files. However, these tables do reside in the same namespace as normal tables that will be put in the MSI file. Some of these tables include: - FeatureGroup supports ComponentGroup authoring concepts - ComponentGroup support ComponentGroup authoring concept - Merge supports merge modules - Actions supports scheduling for standard and custom actions - SuppressAction supports suppression of actions - CustomTables supports custom tables without needing an extension - EnsureTables supports ensuring a table exists in an MSI file - RowData contains row information for CustomTables - UI supports UI elements The danger is that should MSI or any other group decide to use one of these names for a table in their setup package, a collision will occur and WiX will not be able to represent it properly. In order to prepare for this scenario, wed like to preface all WiX-specific table names with Wix similar to how MSI deals with collisions since MSI 2.0 but prefixing all their tables with Msi. This prefix will essentially become a namespace for WiX-specific tables and should not collide with other products. This change will not affect custom action tables like IIsWebSite, SecureObj, XmlFile, etc These must now stay consistent since they ship in MSI files to avoid breaking scenarios in which customers already use these tables. 3. Remove custom xml in wixobj wixlib files Currently WiX passes special information between the compiler and linker in the form of special xml in the wixobj and wixlib files. Over time, we came to recognize that usage of this custom xml was not very extensible and cause a lot of problems whenever a change needed to be made because it basically invalidated the entire wixobj and wixlib file formats. Going forward, we have recognized that the best way to avoid this problem is to persist information between the compiler and linker using unreal tables. This is why WiX 3.0 recently stopped using complexReference elements in the wixobj files and instead switching to unreal WixComplexReference tables. The tables basically allow us to add columns as necessary without breaking backwards-compatibly with previous versions of the wix toolset. This change removing custom xml from the wixobj and wixlib files can be made without sacrificing backwards-compatibility with WiX 2.0 wixlib files. However, Ive called it out here because this change is what prompted us to begin looking at breaking backwards-compatibility. Basically, we found
