Bugs item #27109, was opened at 2009-09-16 01:44 You can respond by visiting: http://rubyforge.org/tracker/?func=detail&atid=575&aid=27109&group_id=126
Category: None Group: None Status: Closed Resolution: Rejected Priority: 3 Submitted By: Adam Salter (aqsalter) Assigned to: Nobody (None) Summary: Should be able to specify gem source in gemspec file Initial Comment: Since there are now at least 3 different 'major' gem hosts - rubyforge, github, gemcutter - it would be nice if a gem dependency's source could be specified in the gemspec file. It might be really nice if _all_ the options supported by 'gem install' were supported but at least being able to specify a non-rubyforge source would be great. ---------------------------------------------------------------------- >Comment By: James Tucker (raggi) Date: 2009-09-18 08:25 Message: You're not understanding what I've told you. Your description of your problem is incorrect. "When somebody tried to install my gem it would fail with a 'dependency not found'. Now, hopefully the user would read my documentation which states that my gem requires gemcutter, but that's not exactly a perfect solution." So, let me get this straight. Your gem is only available on gemcutter, and you're telling me rubygems will report 'dependency not found' when someone tries to install it without the gemcutter source. I don't believe that. I repeat, on my setup, where I have rubyforge, github, and gemcutter permanently in my source list, when I install a gem, it will find dependencies on any one of those three gem servers - as long as the name matches. I've also told you, that gem sources will come and go. The problem you incorrectly describe will happen 100% guaranteed in the future if you bind gems to specific sources, as the sources WILL go away before the gems do. Binding gems to sources also REQUIRES access to the internet to install the gems, for example, you would be stopping local installs of gems, or gem servers living inside walled gardens. As far as "having the right gem" is concerned, this is one of my BIG BIG BIG issues with all these gemservers popping up. Gem::Security exists for *some* of this purpose but it is seldom used properly by gem authors, just as you are not using rubyforge properly, because you're lazy. It is not (contrary to recent hype) difficult to publish gems to rubyforge, the rubyforge gem (available on rubyforge and gemcutter) allows one to do so with a single command. It also allows one to manage file packages for ruby libraries suitable for those who do not use rubygems (of which there are still some people). The --source argument semantics seem to be the actual cause of your issue, but I'd suggest you're doing it wrong. You have an implicit configuration dependency on the user having two sources in their gem configuration. Your users must have rubyforge and gemcutter as individual sources in their configuration. You made this requirement. Don't introduce coupling as a solution to requirements YOU invented. Github is a complete misrepresentation of "a good way" to release gems in the sense of documentation failing, due to the username coupling. I've disagreed with their approach from the start, and I allowing people to be this lazy *already* causes massive issues for the sinatra community who jumped on that bandwagon like crazy. Their dependency stacks are very broken and sometimes completely unresolvable. That's their fault for not doing it right. End of story. gem install gemname The above should work, forever, as long as the user running the command has valid, well populated gem sources in their gem configuration. Those sources may be on the internet, they may be on a lan, they may be your mates laptop next to you running `gem server`. Coupling will only break many very valid use cases that *you* are just unaware of. The bug in your approach is using the --source argument. Stop using this in documentation, and start using your tools right, and stop telling your users to do it the wrong way too. Please please please try to understand several things: 1. Your gem requires configuration of rubygems. 2. Your gem is only available at an immature source because of you and you alone. 3. Your dependencies are not available at the immature source because the source is immature, and not a complete mirror. 4. Duplicating gem names is a security risk, and Gem::Security is already in existence to solve this issue, but poorly educated users think that coupling with specific sources is security (MiTM anyone?) 5. Sources will die - and your idea of MD5 publishings will fail when the same gem, wiht the same name and version, is rebuilt removing the source specific dependency you describe (bug). 6. All of your problems can be solved by proper use of the tools already provided, and correcting your documentation. ---------------------------------------------------------------------- Comment By: Adam Salter (aqsalter) Date: 2009-09-18 01:34 Message: As I said I'm no longer really trying to chase this. I have solved my own issue in this regard, but I want to clarify a few things after thinking some more about this. OK, so firstly, you can install a gem from a specific source right? Why not then have dependencies from a specific source? Remember this all arose because I couldn't publish my gem to rubyforge and have a gem published to gemcutter as a dependency, and have it 'just work'(tm). When somebody tried to install my gem it would fail with a 'dependency not found'. Now, hopefully the user would read my documentation which states that my gem requires gemcutter, but that's not exactly a perfect solution. I _do_ understand that gems are not ideally source specific, otherwise it becomes "github-hpricot-1.2.3" vs "gemcutter-hpricot-1.2.3", but maybe they _should_ be... can't two gems of the same name be published with the same version to different sources? yes, potentially they can. The first solution to allowing people to publish gems to multiple sources is to allow them to have gems with multiple source dependencies. The _other_ solution is to act like there aren't multiple sources. Ie Rubyforge is the canonical and only gem source. If you want your gem to work you _have_ to publish to rubyforge... which is pretty much summed up by "If you find a gem that isn't on rubyforge... submit that as a bug report." I have a philosophical aversion to raising a bug report on a gem because it wasn't published to rubyforge. So I've published my gem to the same source, thereby requiring the user to install gemcutter. I'm assured my gem will work as long as it doesn't depend on a gem from a third gem source (which in this case it doesn't). I also realised that some confusion my have arose from my suggestion because I didn't clarify exactly what I was asking for in code... Iif no source is specified (as currently) every dependency will be resolved as currently, _so no current gems will break_. I think this might have been a misunderstanding. But for any single dependency that you (as the gem owner) know will not be resolved automatically you can specify a source. Obviously rubygem would try this source and raise an error (or maybe just warning and then try the default) if not found. No current gems broken. But also my own gem would 'just work'. Obviously if the gem server that my gem relies on disappears I would have to fix my gem. Yes I know that.... What is the difference between that and having 'gem install adamsalter-hpricot --source=http://gems.github.com' in my docs no longer work? The internet changes and websites come up and go down. Life goes on. Technically multiple gem sources seems to be a problem. Do we fully support multiple gem sources? In which case there is no way to be 100% sure that a gem is the right one without md5 hashing and checking the hashes. Name and version doesn't ensure you have the right gem. Perhaps if you do put a source in a dependency you also have to put a md5 hash? That way if it was mirrored the correct gem could still be found. Or even you could just have an extra option of md5 hash, using that you don't need the version either. (AT&T could still mirror multiple sources and be confident everything was going to work) Thanks everybody for all your great work on RubyGems. It is a great system for managing installation of ruby code. ---------------------------------------------------------------------- Comment By: James Tucker (raggi) Date: 2009-09-17 08:07 Message: Just for accuracies sake, I notice a commit went in for support of prerelease gems into gemcutter last night. I'm not sure if it's complete or deployed though, and this isn't the only thing. ---------------------------------------------------------------------- Comment By: James Tucker (raggi) Date: 2009-09-17 08:03 Message: Adam - what you don't seem to understand is that sources are completely separate from gems. My gems are available from gemcutter, rubyforge and github, for example. You could do: gem install subload --source http://gems.rubyforge.org/ gem install subload --source http://gems.gemcutter.org gem install raggi-subload --source http://gems.github.com Any of which will work. Rubygems can store these, and my .gemrc looks like this: :sources: - http://gems.rubyforge.org/ - http://gemcutter.org - http://gems.github.com So when I type gem install <some_gem> it will try rubyforge first, then gemcutter.org, then gems.github.com. Dependencies could be on any or *all* of the three sources, I see no reason to to put restriction on this flexibility. All that would lead to is a "weakest link" problem, and over time as gem servers are deprecated or come into disservice those gems that have dependencies forcefully relying on them will be broken due to this feature you request. The resolution at this point would be to fetch the gem, unpack it, strip any signatures, open the gemspec, remove the dependency source, then repack the gem. Why would we ever want to do this? As for gemcutter, it's not finished, and still has issues, for example rspec is currently publishing prerelease gems to it, but they are not marked as prerelease by gemcutter, because the product is unfinished. Fact of life, new things take time to be completed. ---------------------------------------------------------------------- Comment By: Adam Salter (aqsalter) Date: 2009-09-17 05:39 Message: OK guys. I'll leave it in your capable hands. If, a) no two differing gems can have the same name (I actually really like github's solution to this) and b) other gem sources are at least able to compete as 'legitimate' external gem sources (by which adding a source to the .gemrc seems as good a solution as any) I guess I can't really complain. I probably won't publish my gem to rubyforge as everything will work for me as setup on gemcutter. The users have to install gemcutter to install my gem anyway. I just raised this issue here because it seemed like a solution to a problem I was trying to forsee (and that I haven't actually encountered yet, admittedly). Cheers, ---------------------------------------------------------------------- Comment By: John Barnette (jbarnette) Date: 2009-09-17 04:40 Message: Specifying dependency sources would also break a significant number of companies who run combined internal gem servers. AT&T, for example, runs an internal gem server which mirrors *all* GitHub and Rubyforge gems, and all deploys use the internal mirror. If gems could specify dependency sources, mirrors like this would have to find a way to override those sources. Cross-source dependency resolution is definitely something we need to smooth out, but explicit dependency sources isn't the way to do it. ---------------------------------------------------------------------- Comment By: Adam Salter (aqsalter) Date: 2009-09-17 03:50 Message: OK. Just to clarify. I do understand that it will complicate things, but I still think the large gem publishers will always find ways to make life easier for their users. i.e. publish to rubyforge... but no... I can't say I agree that no gem should ever have a specific source dependency. Small gem publishers might come and go, but gems already break because of version changes... it's the same thing (in my mind). The larger projects will probably never use source dependencies, but source dependency helps encourage change and competition for smaller gems (again in my mind). As the gem grows they will have to work out any issues.... Hopefully haven't offended anybody. ---------------------------------------------------------------------- Comment By: Adam Salter (aqsalter) Date: 2009-09-17 03:36 Message: I understand the want to have gems work by default, but really think this is overcomplicating a simple issue. People are and will continue to want to publish gems to various gem providers. It's the way people are. This all started because I asked for somebody to publish a gem as a dependency and that gem got published to gemcutter. I was essentially forced to publish my gem to gemcutter so that I could use that dependency. I like the idea that other gem sources should be allowed to flourish. The (best) major ones will always win out... I don't see this gem apocalypse you envision. It is _not_ a bug (in my mind) that somebody didn't release their gem to rubyforge. No matter how easy or friendly it is. ---------------------------------------------------------------------- Comment By: James Tucker (raggi) Date: 2009-09-16 10:47 Message: Ryan - lol, sorry :-( It's useful, as a minimal starting point, for me. My main gripe of hoe is the task names, which i know is totally minor, but anyway, that's another discussion for somewhere else... ---------------------------------------------------------------------- Comment By: Ryan Davis (zenspider) Date: 2009-09-16 09:38 Message: Don't fool yourself. That isn't even close to what Hoe is capable of. :P ---------------------------------------------------------------------- Comment By: James Tucker (raggi) Date: 2009-09-16 09:20 Message: Here is a rakefile that's largely as capable as hoe.rb, but self contained, and a single file. This would help with releasing to rubyforge, and also supports maintaining a .gemspec for github, making releasing on both systems trivial. http://github.com/raggi/subload/blob/master/Rakefile ---------------------------------------------------------------------- Comment By: James Tucker (raggi) Date: 2009-09-16 09:17 Message: Several things here: 1. Rubyforge is the authorative source for rubygems. 2. Gemcutter although functional is not finished, and the discussion of merging it with the rubyforge source is underway - but only when it's ready. 3. Gemcutter requires gems be authorised for name clobber 4. Github won't publish gems that have the same name as a rubyforge gem 5. Gem security policies could be used to solve the auth problem 6. Contrary to popular (uneducated) belief, it is remarkably easy to publish gems to rubyforge using the rubyforge gem. 7. Despite it's php roots, rubyforge is very useful. 8. Gem files are not tied to a source, and never should be, that's centralisation and coupling that will be utterly destructive in future. 9. Github gem hosting is likely to leave soon, according to chris. 10. If you find a gem that isn't on rubyforge, then I recommend you submit that as a bug report. In my opinion that's *very* bad release policy. ---------------------------------------------------------------------- Comment By: Adam Salter (aqsalter) Date: 2009-09-16 09:07 Message: perhaps you're right... I do think that fully supporting other gem sources is a good idea. I regularly need a gem that isn't on rubyforge as a dependency. There is also the issue whereby a gem of the same name exists on two providers. I don't want my users to have their install break in strange ways because they got the wrong/different gem... It is a real issue. There are multiple gem providers and not all people are publishing to rubyforge as a default choice. Would you like me to raise it on the mailing list? ---------------------------------------------------------------------- Comment By: James Tucker (raggi) Date: 2009-09-16 08:23 Message: Add the sources to your .gemrc and you're done. This is a really bad idea in the long run, as at least one of these services will disappear this year, then you'll have a lot of broken gems. ---------------------------------------------------------------------- You can respond by visiting: http://rubyforge.org/tracker/?func=detail&atid=575&aid=27109&group_id=126 _______________________________________________ Rubygems-developers mailing list http://rubyforge.org/projects/rubygems Rubygems-developers@rubyforge.org http://rubyforge.org/mailman/listinfo/rubygems-developers
