Re: Dynamic language install and CMAKE_INSTALL_PREFIX
On Thu, Dec 5, 2013 at 3:26 PM, Justin Ross wrote: > On Thu, Dec 5, 2013 at 1:57 PM, Rafael Schloming wrote: > > The primary benefit is that the build will adapt itself to the user's > > environment. If the user has some custom python/ruby interpreter in their > > path, it will configure and built itself against it and get the user up > and > > running right away. This makes for a very simple and idiot proof README > for > > someone wanting to get up and running quickly from a source build, and > for > > much the same reason it is also very handy for testing. I depend on it > > myself quite a bit since I have a number of differently configured VMs > that > > I use for install testing, and for each one I can simply log in and use > the > > same incantation and be confident I'm running/testing the code that I > > expect. There is also a second order testing benefit since having a dirt > > simple and robust build option lets us give a source tarball to other > > people to test easily and not have to explain to them how to set up > custom > > search paths for each language before they can get bootstrapped into > > running test code. > > > > The drawbacks that have been pointed out are that when you do specify a > > CMAKE_INSTALL_PREFIX, it is unintuitive for things to be placed outside > > that prefix, and this can happen if the QUERIED_LOCATION for a given > > interpreter happens to be outside the specified prefix. It's also been > > pointed out that if you happen to have an rpm installed version of proton > > then with the existing trunk behaviour you could could end up > accidentally > > overwriting it since rpm installs proton code into the QUERIED_LOCATION > > also. > > Yes, indeed. These are coming from me. I want: > > 1. A relatively easy way to do test builds under a single prefix. > What's there now requires a tediously long command line that repeats > the prefix for each binding, and my scripts would surely break if > someone added a new binding. > > 2. A safely isolated default prefix: It's more than "you could end up > accidentally overwriting". It's really quite likely, because the > out-of-the-box, no-extra-steps behavior will write to OS-reserved > locations. Down below you talk about doing harm. *This* is harm. > > > Based on my reading, the change you've pointed to removes the ability to > > install directly to the QUERIED_LOCATION and instead uses the > > CONSTRUCTED_LOCATION. It also adds a consistent control interface for > > providing a custom location, i.e. the {LANG}_INSTALL_PREFIX variables. > > Assuming I've read this correctly, I have the following comments. > > > > First, I'm not ok with losing the ability to install directly to the > > queried location. I don't mind if it's not the default, but I want a > simple > > and easy way to get back that behaviour as it is of significant value in > > the scenarios I've mentioned above. > > As a note in passing (since I'm not really proposing you change back): > most autotools- or cmake-based projects don't have this behavior, and > we all get along fine. For instance, qpid-cpp has for a long time. > That's because in the end it really isn't difficult to explain to > users that they need to edit their interpreter's search path to match > the install path they choose. All the value you get in those test > scenarios is still easily had by doing this. Indeed, that's why all > those other projects haven't felt pressure to add query-based install > paths. > Most projects are established enough to already be part of every distribution and can therefore depend on rpm style install behaviours to provide a seamless experience for the user. As such, they don't need to think of their build system as performing any sort of installation function at all other than the bare minimum necessary to bootstrap into distros. Proton on the other hand is a) not in every distribution yet, and b) wants to be able to reach environments that don't necessarily use one of the popular distribution systems. Given that, I would argue that proton's build system has the perhaps somewhat old fashioned requirement of also functioning as an installer, and being able to easily install into the usual locations fits with that role. Even disregarding the installation experience, the bottom line is that without this I will need to hardcode different install scripts for each differently configured environment I currently test with, or short of that try to implement the same functionality with my own script outside of the build system. This will result in a super long command line that breaks with each new binding quite similar to what you were complaining about above. > > > Second, I think it's important to realize that the CONSTRUCTED_LOCATION > is > > almost guaranteed to be meaningless and quite possibly harmful if it is > at > > all different from the QUERIED_LOCATION. To understand this you can take > a > > look at the values from example 1 above. The queried interp
Re: Dynamic language install and CMAKE_INSTALL_PREFIX
On Thu, Dec 05, 2013 at 02:41:53PM -0500, Rafael Schloming wrote: > On Thu, Dec 5, 2013 at 2:29 PM, Darryl L. Pierce wrote: > > On Thu, Dec 05, 2013 at 01:57:22PM -0500, Rafael Schloming wrote: > > > > > So overall I'd say this change should have some kind of switch to control > > > whether the QUERIED_LOCATION is used directly, and I'd argue that for the > > > CUSTOM_LOCATION we should just pass through directly what the user > > supplies > > > and not attempt to merge it with the queried value. As for the > > > CONSTRUCTED_LOCATION, it's worth noting that we don't necessarily need to > > > compute that either, we could just pick an arbitrary location, e.g. > > > ${CMAKE_INSTALL_PREFIX}/lib64/proton-bindings or some such thing. > > > > I find the simplicity in this scenario to be very attractive. It also > > avoids situations like what I saw with the PHP ini directory location, > > to use project-defined directories for defaults. > > > > What I've seen, for each of the language bindings, is a need to know: > > > > 1. the directory to install platform-independent modules, > > 2. the directory to install platform-specific modules, and > > 3. the directory to install configuration (PHP only) > > > > So perhaps ${LANG}_LIBDIR, ${LANG}_ARCHDIR and ${LANG}_CONFDIR? In an > > RPM specfile we could define each one using the provide language's macro > > and it would be a fairly easy integration point. And if you don't define > > them then they work as you suggest above. > > Sounds good to me. We could also use one for docs. The python bindings have > documentation, and hopefully the other bindings will eventually need > somewhere for docs as well. For anything outside of the inline documentation, I would think we'd use omething like $CMAKE_INSTALL_PREFIX/share/doc/proton/$LANG for an install location it's not defined currently). -- Darryl L. Pierce, Sr. Software Engineer @ Red Hat, Inc. Delivering value year after year. Red Hat ranks #1 in value among software vendors. http://www.redhat.com/promo/vendor/ pgpi6PNcijqoJ.pgp Description: PGP signature
Re: Dynamic language install and CMAKE_INSTALL_PREFIX
On Thu, Dec 5, 2013 at 1:57 PM, Rafael Schloming wrote: > The primary benefit is that the build will adapt itself to the user's > environment. If the user has some custom python/ruby interpreter in their > path, it will configure and built itself against it and get the user up and > running right away. This makes for a very simple and idiot proof README for > someone wanting to get up and running quickly from a source build, and for > much the same reason it is also very handy for testing. I depend on it > myself quite a bit since I have a number of differently configured VMs that > I use for install testing, and for each one I can simply log in and use the > same incantation and be confident I'm running/testing the code that I > expect. There is also a second order testing benefit since having a dirt > simple and robust build option lets us give a source tarball to other > people to test easily and not have to explain to them how to set up custom > search paths for each language before they can get bootstrapped into > running test code. > > The drawbacks that have been pointed out are that when you do specify a > CMAKE_INSTALL_PREFIX, it is unintuitive for things to be placed outside > that prefix, and this can happen if the QUERIED_LOCATION for a given > interpreter happens to be outside the specified prefix. It's also been > pointed out that if you happen to have an rpm installed version of proton > then with the existing trunk behaviour you could could end up accidentally > overwriting it since rpm installs proton code into the QUERIED_LOCATION > also. Yes, indeed. These are coming from me. I want: 1. A relatively easy way to do test builds under a single prefix. What's there now requires a tediously long command line that repeats the prefix for each binding, and my scripts would surely break if someone added a new binding. 2. A safely isolated default prefix: It's more than "you could end up accidentally overwriting". It's really quite likely, because the out-of-the-box, no-extra-steps behavior will write to OS-reserved locations. Down below you talk about doing harm. *This* is harm. > Based on my reading, the change you've pointed to removes the ability to > install directly to the QUERIED_LOCATION and instead uses the > CONSTRUCTED_LOCATION. It also adds a consistent control interface for > providing a custom location, i.e. the {LANG}_INSTALL_PREFIX variables. > Assuming I've read this correctly, I have the following comments. > > First, I'm not ok with losing the ability to install directly to the > queried location. I don't mind if it's not the default, but I want a simple > and easy way to get back that behaviour as it is of significant value in > the scenarios I've mentioned above. As a note in passing (since I'm not really proposing you change back): most autotools- or cmake-based projects don't have this behavior, and we all get along fine. For instance, qpid-cpp has for a long time. That's because in the end it really isn't difficult to explain to users that they need to edit their interpreter's search path to match the install path they choose. All the value you get in those test scenarios is still easily had by doing this. Indeed, that's why all those other projects haven't felt pressure to add query-based install paths. > Second, I think it's important to realize that the CONSTRUCTED_LOCATION is > almost guaranteed to be meaningless and quite possibly harmful if it is at > all different from the QUERIED_LOCATION. To understand this you can take a > look at the values from example 1 above. The queried interpreter is the > system interpreter installed under /usr, but the binding code is installed > under /usr/local. In the best case scenario, this code will never be found > because nothing under /usr/local is in the default python search path. In > the worst case scenario there may be other python interpreters installed > under /usr/local that will find and attempt to load the code but will fail > because the code was built against a differently configured python (the > python version could be different, or it could even be the same version but > with a different build configuration). I think this is getting things quite backward. It's not meaningless, because it's a well-known location. It's an *expected* target for integration. There's tons of existing practice built around it. It's not really harmful, either. The default prefix puts it in a well isolated place (isolated from /usr and /opt) and furthermore a place that is reserved for such installs. If you install your interpreter to /usr/local, you can reasonably expect it to pick up library code under /usr/local. That's fair play. It's also fair to expect people to add /usr/local library code to their system library path as an explicit opt-in step. These are well understood rules of the road. The danger isn't in following them; it's in violating them. > Third, the custom location doesn't actually give you full c
Re: Dynamic language install and CMAKE_INSTALL_PREFIX
On Thu, Dec 5, 2013 at 2:29 PM, Darryl L. Pierce wrote: > On Thu, Dec 05, 2013 at 01:57:22PM -0500, Rafael Schloming wrote: > > > So overall I'd say this change should have some kind of switch to control > > whether the QUERIED_LOCATION is used directly, and I'd argue that for the > > CUSTOM_LOCATION we should just pass through directly what the user > supplies > > and not attempt to merge it with the queried value. As for the > > CONSTRUCTED_LOCATION, it's worth noting that we don't necessarily need to > > compute that either, we could just pick an arbitrary location, e.g. > > ${CMAKE_INSTALL_PREFIX}/lib64/proton-bindings or some such thing. > > I find the simplicity in this scenario to be very attractive. It also > avoids situations like what I saw with the PHP ini directory location, > to use project-defined directories for defaults. > > What I've seen, for each of the language bindings, is a need to know: > > 1. the directory to install platform-independent modules, > 2. the directory to install platform-specific modules, and > 3. the directory to install configuration (PHP only) > > So perhaps ${LANG}_LIBDIR, ${LANG}_ARCHDIR and ${LANG}_CONFDIR? In an > RPM specfile we could define each one using the provide language's macro > and it would be a fairly easy integration point. And if you don't define > them then they work as you suggest above. > Sounds good to me. We could also use one for docs. The python bindings have documentation, and hopefully the other bindings will eventually need somewhere for docs as well. --Rafael > > > Wherever we end up, we should also probably abstract the behaviour into a > > macro so that the behaviour is easier to keep consistent between > bindings, > > and so that new bindings pick up the same behaviour automatically. > > +1 > > -- > Darryl L. Pierce, Sr. Software Engineer @ Red Hat, Inc. > Delivering value year after year. > Red Hat ranks #1 in value among software vendors. > http://www.redhat.com/promo/vendor/ > >
Re: Dynamic language install and CMAKE_INSTALL_PREFIX
On Thu, Dec 05, 2013 at 01:57:22PM -0500, Rafael Schloming wrote: > So overall I'd say this change should have some kind of switch to control > whether the QUERIED_LOCATION is used directly, and I'd argue that for the > CUSTOM_LOCATION we should just pass through directly what the user supplies > and not attempt to merge it with the queried value. As for the > CONSTRUCTED_LOCATION, it's worth noting that we don't necessarily need to > compute that either, we could just pick an arbitrary location, e.g. > ${CMAKE_INSTALL_PREFIX}/lib64/proton-bindings or some such thing. I find the simplicity in this scenario to be very attractive. It also avoids situations like what I saw with the PHP ini directory location, to use project-defined directories for defaults. What I've seen, for each of the language bindings, is a need to know: 1. the directory to install platform-independent modules, 2. the directory to install platform-specific modules, and 3. the directory to install configuration (PHP only) So perhaps ${LANG}_LIBDIR, ${LANG}_ARCHDIR and ${LANG}_CONFDIR? In an RPM specfile we could define each one using the provide language's macro and it would be a fairly easy integration point. And if you don't define them then they work as you suggest above. > Wherever we end up, we should also probably abstract the behaviour into a > macro so that the behaviour is easier to keep consistent between bindings, > and so that new bindings pick up the same behaviour automatically. +1 -- Darryl L. Pierce, Sr. Software Engineer @ Red Hat, Inc. Delivering value year after year. Red Hat ranks #1 in value among software vendors. http://www.redhat.com/promo/vendor/ pgpmrakI63iJX.pgp Description: PGP signature
Re: Dynamic language install and CMAKE_INSTALL_PREFIX
As I understand it, the following process is roughly what we're going through for each binding to determine possible/actual install locations: 1. Query the (python/perl/ruby/php/...) interpreter to find the appropriate directory that is in the interpreters search path by default, e.g. site-packages for python. Let's call this the QUERIED_LOCATION. 2. Modify the QUERIED_LOCATION by substituting the CMAKE_INSTALL_PREFIX for the interpreter's own install prefix. Let's call this the CONSTRUCTED_LOCATION. 3. In the case where {LANG}_INSTALL_PREFIX is specified, step (2) is modified to substitute {LANG}_INSTALL_PREFIX rather than CMAKE_INSTALL_PREFIX. Let's call this the CUSTOM_LOCATION. Just in case things aren't clear from above, here are some example values for python: Example 1: CMAKE_INSTALL_PREFIX=/usr/local === QUERIED_LOCATION=/usr/lib64/python2.6/site-packages CONSTRUCTED_LOCATION=/usr/local/lib64/python2.6/site-packages CUSTOM_LOCATION=*N/A* Example 2: CMAKE_INSTALL_PREFIX=/usr, PYTHON_INSTALL_PREFIX=/usr/local === QUERIED_LOCATION=/usr/lib64/python2.6/site-packages CONSTRUCTED_LOCATION=/usr/lib64/python2.6/site-packages CUSTOM_LOCATION=/usr/local/lib64/python2.6/site-packages Example 3: CMAKE_INSTALL_PREFIX=/home/rhs/proton, PYTHON_INSTALL_PREFIX=/home/rhs/modules === QUERIED_LOCATION=/usr/lib64/python2.6/site-packages CONSTRUCTED_LOCATION=/home/rhs/proton/lib64/python2.6/site-packages CUSTOM_LOCATION=/home/rhs/modules/lib64/python2.6/site-packages The existing trunk behaviour is to simply use the QUERIED_LOCATION directly. This will place installed code where it will be found by precisely the interpreter it was built against without any users being required to set up custom search paths. This has a number of benefits and a couple of drawbacks also. The primary benefit is that the build will adapt itself to the user's environment. If the user has some custom python/ruby interpreter in their path, it will configure and built itself against it and get the user up and running right away. This makes for a very simple and idiot proof README for someone wanting to get up and running quickly from a source build, and for much the same reason it is also very handy for testing. I depend on it myself quite a bit since I have a number of differently configured VMs that I use for install testing, and for each one I can simply log in and use the same incantation and be confident I'm running/testing the code that I expect. There is also a second order testing benefit since having a dirt simple and robust build option lets us give a source tarball to other people to test easily and not have to explain to them how to set up custom search paths for each language before they can get bootstrapped into running test code. The drawbacks that have been pointed out are that when you do specify a CMAKE_INSTALL_PREFIX, it is unintuitive for things to be placed outside that prefix, and this can happen if the QUERIED_LOCATION for a given interpreter happens to be outside the specified prefix. It's also been pointed out that if you happen to have an rpm installed version of proton then with the existing trunk behaviour you could could end up accidentally overwriting it since rpm installs proton code into the QUERIED_LOCATION also. Based on my reading, the change you've pointed to removes the ability to install directly to the QUERIED_LOCATION and instead uses the CONSTRUCTED_LOCATION. It also adds a consistent control interface for providing a custom location, i.e. the {LANG}_INSTALL_PREFIX variables. Assuming I've read this correctly, I have the following comments. First, I'm not ok with losing the ability to install directly to the queried location. I don't mind if it's not the default, but I want a simple and easy way to get back that behaviour as it is of significant value in the scenarios I've mentioned above. Second, I think it's important to realize that the CONSTRUCTED_LOCATION is almost guaranteed to be meaningless and quite possibly harmful if it is at all different from the QUERIED_LOCATION. To understand this you can take a look at the values from example 1 above. The queried interpreter is the system interpreter installed under /usr, but the binding code is installed under /usr/local. In the best case scenario, this code will never be found because nothing under /usr/local is in the default python search path. In the worst case scenario there may be other python interpreters installed under /usr/local that will find and attempt to load the code but will fail because the code was built against a differently configured python (the python version could be different, or it could even be the same version but with a different build configuration). Third, the custom location doesn't actually give you full control over where the module is installed because it appends a portion of the queried location. This strikes