Bugs item #14943, was opened at 2007-10-22 10:57 You can respond by visiting: http://rubyforge.org/tracker/?func=detail&atid=575&aid=14943&group_id=126
Category: RubyGems installer (setup.rb) Group: v0.9.5 Status: Open Resolution: None Priority: 1 Submitted By: Marcus Rueckert (darix) Assigned to: Eric Hodel (drbrain) Summary: rubygems should use the same arch specific subdirs in lib for the native extensions as ruby Initial Comment: quoting my my mail to ruby-core [1] [[[ there is another feature that gem doesnt handle nicely atm: with the standard ruby directory layout you can share the tree via nfs for multiple architectures as the native extensions are in an arch specific path. within an installed gem they are directly inside the "lib" subdir. i wonder if gem should use the arch specific subdirs below its "lib" subdir aswell. ]]] Eric asked me in a follow up to open a ticket here. :) Looking forward to the fix :) [1] http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-core/12724 ---------------------------------------------------------------------- >Comment By: James Tucker (raggi) Date: 2010-11-26 23:25 Message: VERSION should include prefix, suffix and full version. RubyCores idea that the contents of 1.9.1 are compatible with 1.9.x is a completely invalid assumption. I spoke to Matz about this at RubyConf. He suggested we speak to nobu. ---------------------------------------------------------------------- Comment By: Marcus Rueckert (darix) Date: 2010-11-25 19:59 Message: would be an option. but then you still need 2 search paths: 1. for pure ruby gems 2. one for gems with native portions. i just want consistent behavior, especially now that rubygems is in the core. although the $(Gem.dir)/gems/foo-1.0/lib/<arch> method had the advantage for me that i could share the pure ruby stuff between all archs in my packages. but the other way would work for me aswell. ---------------------------------------------------------------------- Comment By: Luis Lavena (luislavena) Date: 2010-11-25 18:49 Message: Hello Marcus, There has been several conversations about the pollution of LOAD_PATH and a proposal of solution by Eric Hodel at ruby-core: http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-core/33206 I think making RubyGems add another PATH inside the same gem will be a problem. Instead, what if the default path where gems are installed contains the platform information? So Gem.dir and Gem.path uses ~/.gem/ENGINE/VERSION/PLATFORM Where ENGINE is ruby/jruby VERSION is 1.8 or 1.9.1 and PLATFORM is the one computed by Gem::Platform.local That will not solve the coexistence of the same codebase for two different platforms, but at least will avoid having another performance discussion at Ruby-Core Thoughts? ---------------------------------------------------------------------- Comment By: Marcus Rueckert (darix) Date: 2010-11-25 18:32 Message: 1. i think this ticket is still very valid. 2. i didnt mean that gems should be able to describe native libraries. while it would be nice it is not in the scope of this bug, so it should mostlikely be tracked in a different bug. If you open it, please CC me. I am sure i can give some tips and tricks how we solve it in the linux packaging world to achieve better cross distro support. ---------------------------------------------------------------------- Comment By: Ryan Davis (zenspider) Date: 2010-11-12 22:46 Message: This ticket has been deemed stale and we're closing it in order to catch up with our ticket list. If you think it is still valid, please reopen. ---------------------------------------------------------------------- Comment By: Hongli Lai (hongli) Date: 2010-01-20 22:37 Message: My experience with package management (Dev-C++, Autopackage, Phusion Passenger) so far is that dependency rules, especially for cross-platform software, can get very complex. I'm not sure whether it's at all possible to cover all cases without a turing-complete dependency specification language. Autopackage's dependency checking mechanism is turing-complete (shell scripts) and it works wonderfully well, though few people understand it because it's so powerful. But maybe it's possible to have a "good enough" system that works for most people, and let developers who need more power write their own thing, kinda like what we did for Phusion Passenger by splitting native dependency checking to passenger-install-*-module. For DebGem we didn't try to make RubyGems support native dependencies. We just manually maintained a per-gem native dependency list. I agree with not bothering with C++ compatibility. Most native extensions are written in C anyways. Developers can always write their own thing to handle this. For example the next version of Phusion Passenger will no longer specify extconf.rb in the gem spec; instead it will automatically compile the native extension when it's require()'ed into an Ruby version-, OS- and architecture-specific subdirectory under $HOME. As for Rubists being pitiful at dependency management: most Rubyists are on OS X which has no proper dependency management, so most people are not only unfamiliar with native dependency management but also wouldn't benefit from it. :) If RubyGems is to support native dependencies it should make it extremely easy for gem writers to specify them correctly. ---------------------------------------------------------------------- Comment By: James Tucker (raggi) Date: 2010-01-20 18:33 Message: I mean, whilst GCC versions might be ok for C++, what about say, a lua plugin or a php plugin, etc, etc. Tracking external dependencies and binary level compatibility becomes a nightmare with more VMs / runtimes, combined too with platforms, heh. Maybe, if we could pull in a more advanced extconf, that allowed arbitrary addition of dependency information, by reasonably formed keys, maybe such as: dependencies << [:debian, :package, :gcc, gcc_version] dependencies << [:debian, :package, :lua, lua_version] etc Then compute that based on some hash of the total dependency set (merged with rubys). So then you have a ruby dependency hash, from rbconfig (plus expanded info), and a gem dependency hash from install-time expansion data. I've actually missed something here, which is to tell the dependencies list how to check if those dependencies are still available... Need to store some code for that really, so eval strings (nominally probably calls to Kernel#`)... ---------------------------------------------------------------------- Comment By: James Tucker (raggi) Date: 2010-01-20 18:28 Message: Hongli Lai - agreed completely. A hash of some larger set of attributes may be most appropriate. As far as dependencies like C++ are concerned, there's little we can do to be complete in that area. I try to encourage a move toward proper dependency management, but rubyists are extremely pitiful at this traditionally. ---------------------------------------------------------------------- Comment By: James Tucker (raggi) Date: 2010-01-20 18:26 Message: I support the idea of putting a gem in an arch specific subdir. I am strongly against this subdir being inside the gems installed path. 1. some platform specific gems pack different *ruby* code in their gemfiles for specific archs 2. some gems do not use the commonly accepted paths, or have multiple lib paths, i wouldn't want selection of the path to become arbitrary ---------------------------------------------------------------------- Comment By: Hongli Lai (hongli) Date: 2010-01-20 16:27 Message: I support this change. This would make it possible for multiple Ruby interpreters, possibly on multiple machines with different platforms, to share the same gem directory. However I don't think Gem::Platform in its current form is adequate. While Gem::Platform is a good enough description of the platform, it isn't a good enough description of Ruby extension binary compatibility, which is what matters more. A few examples: - OS X Snow Leopard broke the ABI by switching to x86_64 by default, but uname and therefore RUBY_PLATFORM still report i386 as platform. - Ruby breaks API compatibility even between teeny releases, e.g. RSTRING_PTR vs RSTRING(x)->ptr. The former didn't exist until 1.8.6. On 1.8.7 and later it became required so that the latter doesn't work anymore. It doesn't look like the ABI changes but the Ruby core team doesn't seem to make any ABI guarantees. The switch from 1.8 to 1.9 most definitely breaks some ABI though. For Phusion Passenger we use the following code to identify Ruby extension binary compatibility: http://pastie.org/786648 But even this doesn't tell us the entire story. For example it doesn't include the C++ ABI version, which can be a problem for Ruby extensions written in C++ like EventMachine. GCC breaks the C++ ABI regularly, sometimes even multiple times within the same minor release. For example GCC 3.1 broke the C++ ABI about 3 times if I recall correctly. ---------------------------------------------------------------------- Comment By: Daniel Berger (djberg96) Date: 2009-02-02 04:56 Message: Luis, Right, modification to the extension builder would be required as far as I know. Dan ---------------------------------------------------------------------- Comment By: Luis Lavena (luislavena) Date: 2009-01-31 13:03 Message: Sorry not to properly introduce my question. Let's say I build a gem with VC6 (i386-mswin32), the gem platform is x86-mswin32-60 This means, my lib folder will be like this: lib/my_lib.rb lib/x86-mswin32-60/my_ext.so my_lib.rb contains this: require 'my_ext' now, I'm in a IRB session: require 'rubygems' require 'my_lib' It is supposed to work out. But what about this: require 'rubygems' require 'my_ext' Right now, since lib is added to LOAD_PATH on gem activation, there is no problem with that. With the change, it indicates that: If a gem contains a platform folder for libdir and the platform matches the current one, it should append that folder into LOAD_PATH. That is the correct assumption? I believe changes to Extension Builders bundled in rubygems will be required. ---------------------------------------------------------------------- Comment By: Daniel Berger (djberg96) Date: 2009-01-27 17:52 Message: Luis, I'm not sure what you mean by "require of the extension directly". Can you explain? Dan ---------------------------------------------------------------------- Comment By: Luis Lavena (luislavena) Date: 2009-01-27 17:01 Message: So this will mean "lib/#{Gem::Platform}" will be added to the $LOAD_PATH if exist, correct? What happens when users doing the require of the extension directly? it triggers the activation mechanism? I kind of like this, but maybe I'm missing a drawback that will against us in the long run? ---------------------------------------------------------------------- Comment By: Daniel Berger (djberg96) Date: 2009-01-26 20:24 Message: For example, if you install a C extension gem called 'foo' on Windows you want it to install as: C:/ruby/lib/ruby/gems/1.8/gems/foo/lib/i386-msvcr80/foo.so And Rubygems should add this sitearch directory to its search path? Is that about the sum of it? Regards, Dan ---------------------------------------------------------------------- Comment By: 7rans (transami) Date: 2007-11-06 10:27 Message: After some trial and error with extensions, I m starting to agree with this. It's more flexible. ---------------------------------------------------------------------- You can respond by visiting: http://rubyforge.org/tracker/?func=detail&atid=575&aid=14943&group_id=126 _______________________________________________ Rubygems-developers mailing list http://rubyforge.org/projects/rubygems Rubygems-developers@rubyforge.org http://rubyforge.org/mailman/listinfo/rubygems-developers
