Re: Perl language bindings are now pushed...
On Wed, 2012-12-05 at 17:51 -0500, Rafael Schloming wrote: On Wed, Dec 5, 2012 at 4:38 PM, Darryl L. Pierce dpie...@redhat.com wrote: I disagree: There are two scenarios that we care about: 1. The install prefix of proton is the same as the install prefix of the perl/php/etc. In this case everything just works either way round. No problem with stripping of the prefix because we just add it back on installation in any case. Because this is the case I don't understand at all reason why we wouldn't do this for configuration files as well. This is the user case that Rafi is concerned about. 2. The specified install prefix is different from the the installed prefix of perl/php/etc. In this case the reason (for me) of doing make install is NOT to try to run the just produced modules - obviously without some extra work the language environment won't know where to find the extensions. The purpose is to get a whole sense of all the artifacts that have been produced by the build. If they are spread around the file system it becomes difficult to see at a glance whether something you were expecting to be there has been left out for instance. This case is the developer case. As far as I can see allowing the developer case to work has no effect on whether the user case works - since in that case stripping the prefix is pretty much a null op since it gets added back anyway. I guess this whole discussion hinges on what you think the outcome of a make install is. For a simple user I agree it should produce something that can run and I think that is the case. For a developer/package builder I'd say that the point is to produce an image of all the artifacts produced by the build. Andrew
Re: Perl language bindings are now pushed...
On Thu, Dec 6, 2012 at 1:05 PM, Andrew Stitcher astitc...@redhat.comwrote: On Wed, 2012-12-05 at 17:51 -0500, Rafael Schloming wrote: On Wed, Dec 5, 2012 at 4:38 PM, Darryl L. Pierce dpie...@redhat.com wrote: I disagree: Are you disagreeing with me or Darryl or both? ;-) There are two scenarios that we care about: 1. The install prefix of proton is the same as the install prefix of the perl/php/etc. In this case everything just works either way round. No problem with stripping of the prefix because we just add it back on installation in any case. Because this is the case I don't understand at all reason why we wouldn't do this for configuration files as well. This is the user case that Rafi is concerned about. No actually, the user case I described is where perl and php have *different* install prefixes, e.g. one is installed in /usr and one is installed in /usr/local. In this case munging with the install prefix of proton is guaranteed to do the wrong thing for at least one of php or perl. 2. The specified install prefix is different from the the installed prefix of perl/php/etc. In this case the reason (for me) of doing make install is NOT to try to run the just produced modules - obviously without some extra work the language environment won't know where to find the extensions. The purpose is to get a whole sense of all the artifacts that have been produced by the build. If they are spread around the file system it becomes difficult to see at a glance whether something you were expecting to be there has been left out for instance. This case is the developer case. I think you're missing the mixed install root case, of which this is a subset, and I think this is actually the general case, not purely a developer use case. If I have system binaries for perl and php, but had to update ruby to a later build because of some bug fix that hasn't hit the distro yet, then I might well have perl and php in /usr and ruby in /usr/local. As far as I can see allowing the developer case to work has no effect on whether the user case works - since in that case stripping the prefix is pretty much a null op since it gets added back anyway. The best way to cater to the developer scenario you mention is to simply look at install_manifest.txt. This file is generated whenever you type make install and will provide a much more accurate check on whether the install is behaving as it should, and because this is generated automatically by cmake, it will in pretty much any conceivable scheme as long as we don't write our own install macro. I guess this whole discussion hinges on what you think the outcome of a make install is. For a simple user I agree it should produce something that can run and I think that is the case. For a developer/package builder I'd say that the point is to produce an image of all the artifacts produced by the build. I don't think we have a conflict here. The semantics I've described provide users exactly what they want even in the general case mixed install root scenario, and the install_manifest.txt provides us developers exactly what we want (and is also very handy for users who want to be able to uninstall). --Rafael
Re: Perl language bindings are now pushed...
On Thu, 2012-12-06 at 13:24 -0500, Rafael Schloming wrote: ... The best way to cater to the developer scenario you mention is to simply look at install_manifest.txt. This file is generated whenever you type make install and will provide a much more accurate check on whether the install is behaving as it should, and because this is generated automatically by cmake, it will in pretty much any conceivable scheme as long as we don't write our own install macro. Does it work where make install fails? If not it isn't useful in exactly the cases I care about! Andrew
Re: Perl language bindings are now pushed...
On Thu, 2012-12-06 at 13:35 -0500, Andrew Stitcher wrote: On Thu, 2012-12-06 at 13:24 -0500, Rafael Schloming wrote: ... The best way to cater to the developer scenario you mention is to simply look at install_manifest.txt. This file is generated whenever you type make install and will provide a much more accurate check on whether the install is behaving as it should, and because this is generated automatically by cmake, it will in pretty much any conceivable scheme as long as we don't write our own install macro. Does it work where make install fails? If not it isn't useful in exactly the cases I care about! To reply to myself: install_manifest.txt is a *result* of a *successful* make install so is not useful to the developer case. As that fails in your proposed world stopping the file being created at all. Andrew
Re: Perl language bindings are now pushed...
On Thu, Dec 6, 2012 at 1:35 PM, Andrew Stitcher astitc...@redhat.comwrote: On Thu, 2012-12-06 at 13:24 -0500, Rafael Schloming wrote: ... The best way to cater to the developer scenario you mention is to simply look at install_manifest.txt. This file is generated whenever you type make install and will provide a much more accurate check on whether the install is behaving as it should, and because this is generated automatically by cmake, it will in pretty much any conceivable scheme as long as we don't write our own install macro. Does it work where make install fails? If not it isn't useful in exactly the cases I care about! I don't know. (If it doesn't, I'd file a cmake bug.) Either way though I think with what I've suggested you have plenty of good options as a developer, e.g. just override stuff to go where you want it to. True it might be slightly more onerous for you, however at least it's possible. With the current state of things I don't believe it's possible to install properly in the mixed root scenarios I've mentioned, and at a minimum it's way more difficult for our users. --Rafael
Re: Perl language bindings are now pushed...
On Wed, Dec 5, 2012 at 8:18 AM, Darryl L. Pierce dpie...@redhat.com wrote: On Tue, Dec 04, 2012 at 05:24:25PM -0500, Andrew Stitcher wrote: On Thu, 2012-11-29 at 17:16 -0500, Darryl L. Pierce wrote: I've pushed the Perl language bindings as well as the send/recv examples for using the qpid::proton::Messenger and qpid::proton::Message classes. These changes break make install for a developer build installing in a non system location: ... -- Installing: /home/andrew/Work/proton/install/lib64/python2.7/site-packages/proton.pyo -- Installing: /home/andrew/Work/proton/install/lib64/python2.7/site-packages/_cproton.so -- Removed runtime path from /home/andrew/Work/proton/install/lib64/python2.7/site-packages/_cproton.so -- Installing: /home/andrew/Work/proton/install/lib64/ruby/cproton.so -- Removed runtime path from /home/andrew/Work/proton/install/lib64/ruby/cproton.so -- Installing: /usr/lib64/perl5/libcproton_perl.so CMake Error at bindings/perl/cmake_install.cmake:44 (FILE): file INSTALL cannot copy file /home/andrew/Work/proton/bld/bindings/perl/libcproton_perl.so to /usr/lib64/perl5/libcproton_perl.so. Call Stack (most recent call first): bindings/cmake_install.cmake:39 (INCLUDE) cmake_install.cmake:137 (INCLUDE) It seems like this new stuff just ignores CMAKE_INSTALL_PREFIX. Yeah, it appears that the CMake configuration for Proton doesn't do what Qpid's configuration did as far as prepending the install prefix to directories. None of our language bindings seem to be doing this in Proton. The other builds don't manually prepend the install prefix, however they still honor it where appropriate by using relative paths. Cmake will automatically prepend the install prefix for you if you just use a relative path. I'm not sure why the qpid build goes to the trouble of manually prepending it. IMHO that's a bit misleading as you generally only need to use absolute paths if you're trying to install something outside of the install root. --Rafael
Re: Perl language bindings are now pushed...
On Wed, Dec 05, 2012 at 08:18:06AM -0500, Darryl L. Pierce wrote: It seems like this new stuff just ignores CMAKE_INSTALL_PREFIX. Yeah, it appears that the CMake configuration for Proton doesn't do what Qpid's configuration did as far as prepending the install prefix to directories. None of our language bindings seem to be doing this in Proton. More specifically, PHP is doing the same as Perl and not honoring the install prefix. I've modified the Perl CMake environment to correct this behavior. But the PHP bindings are giving me a particular pain. Specifically, the INI directory. Is there a good example of how to set the prefix? -- Darryl L. Pierce, Sr. Software Engineer @ Red Hat, Inc. Delivering value year after year. Red Hat ranks #1 in value among software vendors. http://www.redhat.com/promo/vendor/ pgpanK6wCjVxC.pgp Description: PGP signature
Re: Perl language bindings are now pushed...
On Wed, Dec 05, 2012 at 08:55:32AM -0500, Rafael Schloming wrote: The other builds don't manually prepend the install prefix, however they still honor it where appropriate by using relative paths. Cmake will automatically prepend the install prefix for you if you just use a relative path. I'm not sure why the qpid build goes to the trouble of manually prepending it. IMHO that's a bit misleading as you generally only need to use absolute paths if you're trying to install something outside of the install root. The Python bindings using the value returned by: python -c from distutils.sysconfig import get_python_lib; print get_python_lib(True, prefix='') which returns a relative path. Perl uses: perl -V:installarchlib which returns an absolute patch. So we're going to have to monkey with the path in the Perl case, either by removing the leading / on *nix (and whatever is necessary on Windows), or else continue to manually find the prefix [1] and replace it, which is what we're now doing. [1] perl -V:installprefix -- Darryl L. Pierce, Sr. Software Engineer @ Red Hat, Inc. Delivering value year after year. Red Hat ranks #1 in value among software vendors. http://www.redhat.com/promo/vendor/ pgpCL6P5HxByk.pgp Description: PGP signature
Re: Perl language bindings are now pushed...
On Wed, Dec 05, 2012 at 11:12:02AM -0500, Rafael Schloming wrote: On Wed, Dec 5, 2012 at 9:52 AM, Darryl L. Pierce dpie...@redhat.com wrote: On Wed, Dec 05, 2012 at 08:18:06AM -0500, Darryl L. Pierce wrote: It seems like this new stuff just ignores CMAKE_INSTALL_PREFIX. Yeah, it appears that the CMake configuration for Proton doesn't do what Qpid's configuration did as far as prepending the install prefix to directories. None of our language bindings seem to be doing this in Proton. More specifically, PHP is doing the same as Perl and not honoring the install prefix. I've modified the Perl CMake environment to correct this behavior. But the PHP bindings are giving me a particular pain. Specifically, the INI directory. Is there a good example of how to set the prefix? The PHP bindings do honor the install prefix, just possibly not in the way you're thinking. The PHP bindings build provides a number of user visible configuration options to explicitly control install locations, e.g.: PHP_EXT_DIR, PHP_INCLUDE_DIR, PHP_INI_DIR, etc If you set these locations to a relative path then the build will install relative to the CMAKE_INSTALL_PREFIX. The *default* values for these locations are created by interrogating the PHP install that the binding is built against. Okay, then a developer has to explicitly override the INI directory each time. For the EXT and INCLUDE directories it's easy to do what we're doing in Perl now to override the prefix. That feels a little clumsy to me since I don't really want to have to specify it each time I do a build. -- Darryl L. Pierce, Sr. Software Engineer @ Red Hat, Inc. Delivering value year after year. Red Hat ranks #1 in value among software vendors. http://www.redhat.com/promo/vendor/ pgpLulMisDZuA.pgp Description: PGP signature
Re: Perl language bindings are now pushed...
On Wed, 2012-12-05 at 11:12 -0500, Rafael Schloming wrote: On Wed, Dec 5, 2012 at 9:52 AM, Darryl L. Pierce dpie...@redhat.com wrote: On Wed, Dec 05, 2012 at 08:18:06AM -0500, Darryl L. Pierce wrote: It seems like this new stuff just ignores CMAKE_INSTALL_PREFIX. Yeah, it appears that the CMake configuration for Proton doesn't do what Qpid's configuration did as far as prepending the install prefix to directories. None of our language bindings seem to be doing this in Proton. More specifically, PHP is doing the same as Perl and not honoring the install prefix. I've modified the Perl CMake environment to correct this behavior. But the PHP bindings are giving me a particular pain. Specifically, the INI directory. Is there a good example of how to set the prefix? The PHP bindings do honor the install prefix, just possibly not in the way you're thinking. The PHP bindings build provides a number of user visible configuration options to explicitly control install locations, e.g.: PHP_EXT_DIR, PHP_INCLUDE_DIR, PHP_INI_DIR, etc If you set these locations to a relative path then the build will install relative to the CMAKE_INSTALL_PREFIX. The *default* values for these locations are created by interrogating the PHP install that the binding is built against. I disagree, I'd say that the PHP make install is equally broken. When you interrogate php-config you should also get the installation prefix[1] and then subtract it, just like with the other bindings. [Darryl, thanks for fixing the perl make install] Andrew [1] Use php-config --prefix
Re: Perl language bindings are now pushed...
On Wed, Dec 5, 2012 at 2:50 PM, Andrew Stitcher astitc...@redhat.comwrote: On Wed, 2012-12-05 at 11:12 -0500, Rafael Schloming wrote: On Wed, Dec 5, 2012 at 9:52 AM, Darryl L. Pierce dpie...@redhat.com wrote: On Wed, Dec 05, 2012 at 08:18:06AM -0500, Darryl L. Pierce wrote: It seems like this new stuff just ignores CMAKE_INSTALL_PREFIX. Yeah, it appears that the CMake configuration for Proton doesn't do what Qpid's configuration did as far as prepending the install prefix to directories. None of our language bindings seem to be doing this in Proton. More specifically, PHP is doing the same as Perl and not honoring the install prefix. I've modified the Perl CMake environment to correct this behavior. But the PHP bindings are giving me a particular pain. Specifically, the INI directory. Is there a good example of how to set the prefix? The PHP bindings do honor the install prefix, just possibly not in the way you're thinking. The PHP bindings build provides a number of user visible configuration options to explicitly control install locations, e.g.: PHP_EXT_DIR, PHP_INCLUDE_DIR, PHP_INI_DIR, etc If you set these locations to a relative path then the build will install relative to the CMAKE_INSTALL_PREFIX. The *default* values for these locations are created by interrogating the PHP install that the binding is built against. I disagree, I'd say that the PHP make install is equally broken. When you interrogate php-config you should also get the installation prefix[1] and then subtract it, just like with the other bindings. I don't see why this is desirable, this just makes the overall build broken if you ever choose a prefix that is different from that reported by php. I think the correct thing to do is default to installing extension modules in the system configured directories, but allow people (experts really) to override that behaviour if they choose to. --Rafael
Re: Perl language bindings are now pushed...
On Wed, Dec 5, 2012 at 4:38 PM, Darryl L. Pierce dpie...@redhat.com wrote: On Wed, Dec 05, 2012 at 03:08:24PM -0500, Rafael Schloming wrote: Okay, then a developer has to explicitly override the INI directory each time. For the EXT and INCLUDE directories it's easy to do what we're doing in Perl now to override the prefix. That feels a little clumsy to me since I don't really want to have to specify it each time I do a build. I'm not sure I follow you. These are all configured variables. You only ever have to set any of them once. To keep things from lingering, each time I do a build I go into my OOT build directory and do: rm -rf * cmake ../proton-c I don't want to have to specify the PHP_INI_DIR when I do the above. I suppose I could have an alias defined for it (maybe have config.sh source a separate, non-versioned file containing developer aliases?). But that could very well gum up the plumbing. :) You may not have to. It's not necessary for building, only if you want to do a make install. I suppose my usage isn't quite the same as what Andrew mentioned. For me, I don't install to non-standard places, so my above scenario won't be affected. IOW, I don't do make install and run from those installed pieces and instead run against things in my CPROTON_BUILD directory. I guess for a developer who's doing as Andrew mentioned, they'll have to override PHP_INI_DIR when they install, which should be less frequent than my in-place builds. So the main point of this stuff is to make proton easy to install for users. I'm all for making our lives as developers easier, but not at the expense of users, and I think not having the bindings behave as I've described will significantly complicates things for users. Imagine a system with python installed in /usr, and perl and php both installed in /usr/local. Say a user wants to install proton and the relevant bindings onto this system. If all the bindings default their location based on interrogating the installed interpreters, this is easy. The user can choose to install proton itself into either /usr or /usr/local, and all the bindings will just go into the right place. Now consider what happens if we automatically munge the prefix. There really isn't a good way to explain to a user how to install on this system. They either need to choose to put proton into /usr and munge their perl and php installs to look for extensions under /usr as well as /usr/local, or possibly somehow override undocumented variables in the cmake build to get the extension modules into the right place. I don't even know if this is possible as the overridden variables might well be munged. I think this scenario really points to two issues (1) the defaults really should be based on interrogation of the interpreters that the bindings are built against if we want a robust and simple README file, and (2) all the bindings should support proper manual overriding of the installed location of each binding. I believe (2) is only the case for the PHP binding currently, or at least it was the case before the php binding was modified to do the munging. Unless I'm missing something, I'd like to suggest we rollback that change and make the other bindings match the PHP behaviour of defaulting based on interrogation and providing proper configuration variables for manual override. I'm aware this might make certain developer scenarios mildly more onereous, but with manual overrides in place they should all be possible, and I believe the benefit for users is significant. --Rafael
Re: Perl language bindings are now pushed...
On Thu, 2012-11-29 at 17:16 -0500, Darryl L. Pierce wrote: I've pushed the Perl language bindings as well as the send/recv examples for using the qpid::proton::Messenger and qpid::proton::Message classes. These changes break make install for a developer build installing in a non system location: ... -- Installing: /home/andrew/Work/proton/install/lib64/python2.7/site-packages/proton.pyo -- Installing: /home/andrew/Work/proton/install/lib64/python2.7/site-packages/_cproton.so -- Removed runtime path from /home/andrew/Work/proton/install/lib64/python2.7/site-packages/_cproton.so -- Installing: /home/andrew/Work/proton/install/lib64/ruby/cproton.so -- Removed runtime path from /home/andrew/Work/proton/install/lib64/ruby/cproton.so -- Installing: /usr/lib64/perl5/libcproton_perl.so CMake Error at bindings/perl/cmake_install.cmake:44 (FILE): file INSTALL cannot copy file /home/andrew/Work/proton/bld/bindings/perl/libcproton_perl.so to /usr/lib64/perl5/libcproton_perl.so. Call Stack (most recent call first): bindings/cmake_install.cmake:39 (INCLUDE) cmake_install.cmake:137 (INCLUDE) It seems like this new stuff just ignores CMAKE_INSTALL_PREFIX. Andrew
Perl language bindings are now pushed...
I've pushed the Perl language bindings as well as the send/recv examples for using the qpid::proton::Messenger and qpid::proton::Message classes. -- Darryl L. Pierce, Sr. Software Engineer @ Red Hat, Inc. Delivering value year after year. Red Hat ranks #1 in value among software vendors. http://www.redhat.com/promo/vendor/ pgpx2SFeVXJm4.pgp Description: PGP signature
