Re: [Python-Dev] Python merge module
On Thu, Feb 3, 2011 at 00:30, Martin v. Löwis mar...@v.loewis.de wrote: Another challenge with shared location merge modules is upgrades: the Python installer currently doesn't use stable component IDs; I think this would cause problems for users of the merge module. Providing stable component IDs is a challenge since it's difficult to version the files on disk. Not sure why Michael thinks that a private location merge module would provide no benefits to the user of the merge module. I hadn't thought it through fully, but the preceding paragraph really gets to the core of the problem. The maintenance nightmare is security updates for private location installations by third parties. The only MSI-friendly way to update that code is through releasing an updated merge module and having the consuming application also release an update that uses it. Since Python's components use fresh GUIDs each time, this would require a major upgrade; minor upgrades would cause Windows Installer to throw fits. Technically this is a problem with the component generation in Python, and for that in particular, a move to WiX could be very helpful. They have stable component code generation which keys off of location, name, platform, etc., but only works for single-file components. For contrast, I don't see a shared-location merge module as offering benefits beyond a silently redistributable msi package. The shared location is better about following component code rules (re-use in private areas is an allowed gray area), and there are people out there who consider the reference counting through merge modules to be superior. I find the resulting complexity in the consuming package's installation to be more of a down-side. I work on open source projects myself, and we always provide both a merge module and a normal msi installer. It's very little extra work (in WiX at least) to create both. But what's the quality of these? Ideally, I'd like to create a single merge module which, at the option of the user of the merge module, produces either a shared or a private installation. Is that still only little extra work in Wix? I've never tried to make a configurable merge module in WiX, but I think that's the only option if you want a single merge module to allow both. It should be a one-time authoring overhead with [1]. Using them is pretty straightforward within the Merge elements [2]. [1] http://wix.sourceforge.net/manual-wix3/wix_xsd_configuration.htm [2] http://wix.sourceforge.net/manual-wix2/wix_xsd_configurationdata.htm Michael ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python merge module
On 3 February 2011 15:38, Michael Urman mur...@gmail.com wrote: Technically this is a problem with the component generation in Python, and for that in particular, a move to WiX could be very helpful. They have stable component code generation which keys off of location, name, platform, etc., but only works for single-file components. At work we keep the required stable UUIDs in an ConfigParser-format file checked in to the VCS for that purpose. FWIW our build system uses WiX (v2) currently and if I where to redo it, I'd change to msilib and not WiX v3. But never change working systems. Python.org supplied merge modules would be nice though, if the upgrade/security issue can be resolved. Regards Floris -- Debian GNU/Linux -- The Power of Freedom www.debian.org | www.gnu.org | www.kernel.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python merge module
I hadn't thought it through fully, but the preceding paragraph really gets to the core of the problem. The maintenance nightmare is security updates for private location installations by third parties. The only MSI-friendly way to update that code is through releasing an updated merge module and having the consuming application also release an update that uses it. Since Python's components use fresh GUIDs each time, this would require a major upgrade; minor upgrades would cause Windows Installer to throw fits. That's exactly right and why I said earlier that patches w/ merge modules are a pain/nightmare. But I also said that the security patching w/ a merge module would then depend on the party who has integrated the merge module into their product -- not on you guys. I don't think most (or any) parties are trying to do incremental minor updates w/ python right now anyway. With us, we just want a single, well-tested and known python implementation installed w/ our product -- we basically don't trust the user to get the right version and install it correctly. You obviously can't hold their hand every step of the way, at some point you have to let go -- but this is one thing that we could possibly control. Technically this is a problem with the component generation in Python, and for that in particular, a move to WiX could be very helpful. They have stable component code generation which keys off of location, name, platform, etc., but only works for single-file components. The general recommendation regarding msi packages is that you always, always do single-file components (one of the major reasons being for patching purposes). If you use WiX' heat application to autogenerate WiX files from directories, you'll see that it always produces single-file components. For contrast, I don't see a shared-location merge module as offering benefits beyond a silently redistributable msi package. The shared location is better about following component code rules (re-use in private areas is an allowed gray area), and there are people out there who consider the reference counting through merge modules to be superior. I find the resulting complexity in the consuming package's installation to be more of a down-side. I think a merge module (shared or private) generally reduces complexity unless you already have a bootstrapper for other packages. Including one in WiX is very simple (if you're already familiar w/ msi's): Feature Id=Full MergeRef Id=Python / /Feature Directory Id=TARGETDIR Merge Id=Python SourceFile=python.msm Language=1033 DiskId=1 / /Directory If you use something like the VS installer projects, you just have to use the GUI to add a reference to it and you're done. A shared merge module might pose a problem if you want to have multiple python versions installed. At the least, you'd need to change the component guid w/ each major release (e.g. 2.x - 3.x, but not 2.7 - 2.8 - 2.9, etc.). I'd recommend w/ sticking to a private one that doesn't modify the PATH (should you choose to create one) and then you're free to keep or change the component guids. Can python operate inside a sandboxed C:\Program Files\My Product Here\ directory? I've never tried to make a configurable merge module in WiX, but I think that's the only option if you want a single merge module to allow both. It should be a one-time authoring overhead with [1]. Using them is pretty straightforward within the Merge elements [2]. [1] http://wix.sourceforge.net/manual-wix3/wix_xsd_configuration.htm [2] http://wix.sourceforge.net/manual-wix2/wix_xsd_configurationdata.htm I wouldn't try to make it configurable (nor does it really need to be) beyond what it already should do -- that is, just be able to set the base target directory, which is already done for you. And as I just suggested, I wouldn't go for both -- declare the merge module to be private and that's that. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python merge module
At work we keep the required stable UUIDs in an ConfigParser-format file checked in to the VCS for that purpose. FWIW our build system uses WiX (v2) currently and if I where to redo it, I'd change to msilib and not WiX v3. But never change working systems. No need to do that if you're using heat -- I haven't used WiX v2 so I can't speak to its relative merits, but v3.5 (which was just officially released) is pretty good and much more feature complete (according to rob mensching's blog). I'd recommend re-evaluating it. I'm not a Microsoft fanboy, btw. But I am getting my group (or trying to, at least) to migrate away from older, stale installer projects (e.g. Visual Studio will be dropping support for any further installer project improvements in the future) and into WiX because that's where the momentum is and it also keeps up-to-date w/ the latest msi changes. That and I was tired of the install looking like an intern did it (no offense to interns -- I was one once. (c: ). A good, polished product should (IMO) also have a good, polished installer. Especially since that's one of the customer's first views/impressions of your product. Python.org supplied merge modules would be nice though, if the upgrade/security issue can be resolved. Good to know I'm not entirely alone. (c: ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python merge module
Technically this is a problem with the component generation in Python, and for that in particular, a move to WiX could be very helpful. They have stable component code generation which keys off of location, name, platform, etc., but only works for single-file components. That would be no reason to move to Wix. Integrating the same algorithm in msilib is trivial. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python merge module
Am 03.02.2011 18:58, schrieb Floris Bruynooghe: On 3 February 2011 15:38, Michael Urman mur...@gmail.com wrote: Technically this is a problem with the component generation in Python, and for that in particular, a move to WiX could be very helpful. They have stable component code generation which keys off of location, name, platform, etc., but only works for single-file components. At work we keep the required stable UUIDs in an ConfigParser-format file checked in to the VCS for that purpose. That's also the path I'd take. But I'm unsure how versioning would work, in particular if I have per-directory components, and files get added (which typically shouldn't, but might happen in bugfix releases). Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python merge module
The general recommendation regarding msi packages is that you always, always do single-file components (one of the major reasons being for patching purposes). Can you please cite a source for that recommendation? Preferably some MSDN documentation. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python merge module
Can you please cite a source for that recommendation? Preferably some MSDN documentation. http://msdn.microsoft.com/en-us/library/aa368269(v=vs.85).aspx http://wix.sourceforge.net/manual-wix3/add_a_file.htm Specifically, starting in bold, where it says In general, you should restrict yourself to a single file per component. The Windows Installer is designed to support thousands of components in a single installer, so unless you have a very good reason, keep to one file per component. Every component must have its own unique GUID. Failure to follow these two basic rules can lead to many problems down the road when it comes to servicing. A little before that, it states: The component element describes a set of resources (usually files, registry entries, and shortcuts) that need to be installed as a single unit. This is separate from whether the set of items consist of a logical feature the user can select to install ... While it may not seem like a big deal when you are first authoring your installer, components play a critical role when you decide to build patches at a later date. I didn't say it's a must, but experience lends you to following the recommendation. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Python merge module
Is there or will there be support for python merge modules so we can include python in our installer? Also, the discussions I saw about windows installers not removing the path on uninstall is completely false as regards the installers that wix creates, at least. I've modified the path many times and explicitly tested that it was removed on uninstall. I can't speak for the msilib package and what it does - perhaps it's doing the wrong thing? But has there been thought towards migrating away from msilib and using platform standard tools such as wix (used by ms office, visual studio, etc.)? ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python merge module
Am 02.02.2011 20:01, schrieb Hoyt, David: Is there or will there be support for python merge modules so we can include python in our installer? I haven't planned any. Contributions are welcome. But has there been thought towards migrating away from msilib and using platform standard tools such as wix (used by ms office, visual studio, etc.)? Are you sure visual studio uses wix? I wouldn't call it a platform standard tool. The Installer COM object is the platform standard mechanism, and that's what msilib uses. I really see no need to move away from that - it can create arbitrary MSI files. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python merge module
Is there or will there be support for python merge modules so we can include python in our installer? I haven't planned any. Contributions are welcome. But has there been thought towards migrating away from msilib and using platform standard tools such as wix (used by ms office, visual studio, etc.)? Are you sure visual studio uses wix? The visual studio regularly contributes to the wix toolset. They and the .net framework team contributed a lot of the wix bootstrapper (burn) foundational code. I wouldn't call it a platform standard tool. It's becoming that as more and more projects are adopting it. It was considered to be shipped w/ VS 2010 but didn't make it due to management/time issues. The Installer COM object is the platform standard mechanism, and that's what msilib uses. Why maintain a lib when there's (better), free alternatives out there that are maintained by Microsoft itself? (okay, a group at Microsoft that works on their free time, but has significant contributions by many teams at Microsoft -- thus they have a vested interested in its success). I really see no need to move away from that - it can create arbitrary MSI files. Can it create merge modules? Transform files (e.g. localization)? Bootstrappers? There's a lot more to ms installers than the msi itself. Wouldn't it be easier to maintain an XML file than an entire lib dedicated to something that someone else has already solved? ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python merge module
On Wed, Feb 2, 2011 at 15:27, Hoyt, David ho...@llnl.gov wrote: The Installer COM object is the platform standard mechanism, and that's what msilib uses. Why maintain a lib when there's (better), free alternatives out there that are maintained by Microsoft itself? (okay, a group at Microsoft that works on their free time, but has significant contributions by many teams at Microsoft -- thus they have a vested interested in its success). I really see no need to move away from that - it can create arbitrary MSI files. Can it create merge modules? Transform files (e.g. localization)? Bootstrappers? There's a lot more to ms installers than the msi itself. Wouldn't it be easier to maintain an XML file than an entire lib dedicated to something that someone else has already solved? If Python was starting at ground zero, and the choices were to create a library or to use WiX, the answer might have been different. However with a mature enough library to suit all the needs that anyone has been willing to author, it's certainly more work to create the WiX install and maintain it than it is to merely maintain the existing scripts. As far as the possibility of distributing Python as a merge module? I'd recommend against it. Shared location merge modules are a maintenance nightmare, and private location merge modules may not offer the benefit you need. Better to just install the main Python msi as part of a suite with your installer, whether you build that installer in WiX and burn, or not. Michael ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python merge module
The Installer COM object is the platform standard mechanism, and that's what msilib uses. Why maintain a lib when there's (better), free alternatives out there that are maintained by Microsoft itself? Using msilib is easier than using Wix. It's also more flexible. All you have to know is how the MSI schema works. I really see no need to move away from that - it can create arbitrary MSI files. Can it create merge modules? Transform files (e.g. localization)? It could easily be extended to do so, in a straight-forward manner. Bootstrappers? Not sure what this is - if it is what I think it is, there is no need to (actually, there is no need to create the other two types of files, either, as it stands). There's a lot more to ms installers than the msi itself. Wouldn't it be easier to maintain an XML file than an entire lib dedicated to something that someone else has already solved? Definitely not. Python is easier than XML. If you think otherwise, feel free to provide a proof in the form of a Wix installation generator for Python. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python merge module
Martin v. Löwis mar...@v.loewis.de wrote: The Installer COM object is the platform standard mechanism, and that's what msilib uses. I really see no need to move away from that - it can create arbitrary MSI files. I've used it to package UpLib for Windows -- see http://uplib.parc.com/hg/uplib/file/e29e36f751f7/win32/build-msi-installer.py. I've generalized some of Martin's code to create a Packager class which supports non-Python pre and post install scripts. I found it much easier to use than WiX, which I also tried. Bill ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python merge module
Using msilib is easier than using Wix. It's also more flexible. IMO, no. It's simply not. All you have to know is how the MSI schema works. Same with WiX. It could easily be extended to do so, in a straight-forward manner. Other packaging apps already have it - no work needed. (actually, there is no need to create the other two types of files, either, as it stands). Sure there is. Transform files + bootstrappers give you localizable installs in one flat file. Definitely not. Python is easier than XML. I disagree. If you think otherwise, feel free to provide a proof in the form of a Wix installation generator for Python. If you think otherwise, why don't you provide proof and get the WiX guys to switch to Python and use your msilib package? The point is that your argument is nonsensical. I'm really not trying to start a flame war here (my original post only asked if there was thought towards migrating away from msilib). There's legitimate need/desire for a merge module to make it easier to package a specific Python version. I can (and am) including it in the bootstrapper, but it would make things much cleaner to simply have a merge module available. If the answer's no and we don't ever intend to, then that's perfectly fine. (c: I really don't care to argue the merits or virtues of WiX vs. msilib and I apologize if I came across as doing that -- I simply was interested to know if there's any serious thought in the matter and it turns out that people are perfectly happy w/ msilib. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python merge module
If Python was starting at ground zero, and the choices were to create a library or to use WiX, the answer might have been different. However with a mature enough library to suit all the needs that anyone has been willing to author, it's certainly more work to create the WiX install and maintain it than it is to merely maintain the existing scripts. Sounds reasonable. As far as the possibility of distributing Python as a merge module? I'd recommend against it. Shared location merge modules are a maintenance nightmare, and private location merge modules may not offer the benefit you need. Better to just install the main Python msi as part of a suite with your installer, whether you build that installer in WiX and burn, or not. I'm not familiar w/ shared location merge modules vs. private location merge modules. Are you referring to the possibility of having multiple python apps trying to use their own python versions? How relocatable is python? The maintenance nightmare isn't on your end (in my case), it would be on me to provide patches/upgrades. I do agree that patches w/ merge modules are a pain/nightmare. But what about providing both a merge module and msi so developers have a choice? I work on open source projects myself, and we always provide both a merge module and a normal msi installer. It's very little extra work (in WiX at least) to create both. But I can tell from this discussion that it would require changes in msilib and since I don't have the time to work on it, it would most likely not happen. Perhaps I could generate enough fervor in the community for a merge module, though, to drive msilib development. :p At any rate, where could I find the script used to generate the msi package? Perhaps I can translate it into WiX if I thought it'd be used (but why bother if no one's interested?). ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python merge module
I found it much easier to use than WiX, which I also tried. I also used to use the Visual Studio installer projects until I needed something a lot more robust (e.g., customized UI + localizable strings). msilib does the job people need it to do and that's fine. I'm really not trying to argue the merits of WiX vs. msilib. Was just wondering if it had been considered... ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python merge module
On Wed, 02 Feb 2011 17:30:43 -0800, Hoyt, David ho...@llnl.gov wrote: Definitely not. Python is easier than XML. I disagree. Just as an FYI, I believe that most people in the Python community find XML much more of a pain than Python. Many of us (especially those of us who are not web developers) avoid XML whenever possible, and when we do have to deal with it we tend to abstract it behind easier to work with Python code. The obvious exception to that would be web templating languages, but I personally prefer to avoid those, as well :) -- R. David Murray www.bitdance.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python merge module
I'm really not trying to start a flame war here (my original post only asked if there was thought towards migrating away from msilib). There's legitimate need/desire for a merge module to make it easier to package a specific Python version. Please recognize that this question is entirely independent of the question whether Wix is used or not. I'm personally +0 on providing a merge module (although others are apparently opposed to that idea). As for msilib vs. Wix: yes, I did put thought into this (actually originally also when writing msilib, which slightly predates Wix in time). However, I don't see any need to switch, and I actually do believe that Wix couldn't provide a feature-by-feature full replacement of the current packaging code (but I might be wrong). Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python merge module
As far as the possibility of distributing Python as a merge module? I'd recommend against it. Shared location merge modules are a maintenance nightmare, and private location merge modules may not offer the benefit you need. Better to just install the main Python msi as part of a suite with your installer, whether you build that installer in WiX and burn, or not. I'm not familiar w/ shared location merge modules vs. private location merge modules. Are you referring to the possibility of having multiple python apps trying to use their own python versions? How relocatable is python? Fairly relocatable. If there is was a merge module, I'd really prefer it to be shared. The challenge here is site-packages: different users of the merge module would need different copies of site-packages (or else installing additional packages might break existing software). Another challenge with shared location merge modules is upgrades: the Python installer currently doesn't use stable component IDs; I think this would cause problems for users of the merge module. Providing stable component IDs is a challenge since it's difficult to version the files on disk. Not sure why Michael thinks that a private location merge module would provide no benefits to the user of the merge module. The maintenance nightmare isn't on your end (in my case), it would be on me to provide patches/upgrades. I do agree that patches w/ merge modules are a pain/nightmare. But what about providing both a merge module and msi so developers have a choice? I certainly wouldn't stop providing an MSI. The next question would be whether the MSI then also installs into the shared location, or whether it would have a private copy of Python. I work on open source projects myself, and we always provide both a merge module and a normal msi installer. It's very little extra work (in WiX at least) to create both. But what's the quality of these? Ideally, I'd like to create a single merge module which, at the option of the user of the merge module, produces either a shared or a private installation. Is that still only little extra work in Wix? At any rate, where could I find the script used to generate the msi package? Perhaps I can translate it into WiX if I thought it'd be used (but why bother if no one's interested?). It's in Tools/msi/msi.py. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
