Re: [rust-dev] Announcing the new Rust package manager, Cargo
Strahinja: We are definitely going to be smart with dependency resolution. We are going to allow libs to specify their dependencies using semver (http://semver.org/) as well as version bounds (for example, foo = 2.0.0 - any version of foo greater than 2.0.0, bar ~ 2.1.0, any version of bar greater than 2.1.0 and less than 2.2 -- exact syntax is still TBD). Ideally, if pulling in shared dependencies that satisfy the bounds, only one version of the dep will be present (reduce binary bloat). We should also be able to handle the incompatible transitive dependency case by getting both versions of the library, like you said. Davis: The term native library is pretty loaded, so hopefully I will be able to answer your question. First, we do not want to reinvent Make or apt-get. So, Cargo will not, itself, be able to build C / C++ libraries nor will it be able to pull in system packages (via apt-get, homebrew, etc...). However, it will definitely be extensible. So, if you have C library dependencies that you want to be able to compile, you can include a makefile and tell cargo when and how to invoke it. Also, if you want to ensure that certain system packages are present, you will also be able to tell cargo when and how to do this. As for moving cargo to rust-lang, that is the plan. We haven't done so yet as we are still still in spike mode and code is pretty messy, but we want to move it to rust-lang's github very soon. Cheers, Carl On Tue, Mar 18, 2014 at 8:37 PM, Davis Silverman sinistersn...@gmail.com wrote: This is great news! I wonder, what will the story be with native libraries? Will/could it be possible to package other dependencies, say, in C, to build also? Also, are there plans to move the repository to rust-lang? I'm extremely happy and can't wait to hear more. Thanks so much! ___ Rust-dev mailing list Rust-dev@mozilla.org https://mail.mozilla.org/listinfo/rust-dev ___ Rust-dev mailing list Rust-dev@mozilla.org https://mail.mozilla.org/listinfo/rust-dev
Re: [rust-dev] Announcing the new Rust package manager, Cargo
This is wonderful news! Thank you for taking on such an important and difficult task! I'm wondering will Cargo leverage one of Rust's amazing features, which is that library symbols include the crate hash and version in their namehttps://github.com/mozilla/rust/wiki/Doc-crate-hashes#symbol-hashes . This means that finally we have a language that can handle the following case: lib A depends on lib B, which depends on lib C v1. Lib A now gets a dep on lib D, which depends on lib C v2. With Rust we can just link all this together without issues whereas with something like C++ you'd end up very unhappy. I'm hoping that Cargo will support such use cases and won't insist on just one version of a dependency in the dependency tree, as package managers commonly do. On Mon Mar 17 2014 at 6:27:09 PM, Yehuda Katz wyc...@gmail.com wrote: Hello Rustlers, I'll be writing here more with more details soon. For now, a few quick comments: - I'm really glad that Mozilla and the Rust team are prioritizing package management. An open source language ecosystem really lives or dies based on how easy it is to share code, and writing a world-class package manager (as brson put it) takes time, especially when you account for the inevitable and important iteration that comes from real-world usage. - There's a lot about package management that's well-understood and somewhat language agnostic. On the other hand, there are things that are specific to native code or even more specific to Rust that a Rust package manager need to account for. My goal is to use well-known best practices for the former, and avoid reinventing the wheel, without losing track of what makes Rust unique or different. Carl and I are both the creators of the predominant Ruby package manager (bundler) and day-to-day production users of Rust (really!) at the company we founded. We think that mix should enable us to balance both of those priorities. - Over the next month or two, we plan to prioritize getting to regular, working milestones with Cargo. These milestones will not always reflect our plan for the final workflow that we expect with Cargo, but having real-world working code is very important when building something the size and scope of Cargo. We plan to share design documents (both on the internal architecture and expected workflows) as we work. We started work 10 days ago, and we already have a primitive cargo compiling one of our libraries based on its Cargo manifest, but so far only via very simple plumbing commands that don't reflect the actual workflow we intend. In general, some guiding principles for the project: - It should be possible for new users of Rust to use Cargo to experiment with Rust and its library ecosystem and have success quickly. - Users of Cargo should get deterministic builds out of the box. If I build an artifact on one machine, it should be possible to build that same artifact on another machine with exactly the same source of every dependency, and exactly the same options. - Users of Cargo should be able to update dependencies with as minimal impact on other dependencies as possible. If I update a utility library, Cargo should avoid updates to other, unrelated libraries by design. - Cargo should support cross-compilation out of the box. As long as your version of Rust and its standard library are compiled for the expected targets, a single line of configuration should be enough to get builds for those targets - Cargo should support the common lifecycle for packages: a package starts out as a part of an existing project, moves to Github for easier sharing across multiple projects and eventually the open source community, and finally is published to a central repository with a version number. This means that all three kinds of sources (local file system, github, and central package repository) should be supported by the default distribution of Cargo. We plan to publish more detail really soon, as well as more information on what we've already built. Please feel free to ask questions :) Yehuda Katz (ph) 718.877.1325 On Mon, Mar 17, 2014 at 5:48 PM, Brian Anderson bander...@mozilla.comwrote: Dearest Rustlers, Today I'm super-excited to announce that Mozilla has arranged to develop a world-class package manager for Rust. Yehuda Katz and Carl Lerche, from Tilde Inc., have previously built the popular Ruby package manager, Bundler, and now they are going to build Rust's package manager, Cargo. They are *experts* at the management of packages, and will deliver a tool that builds off both their experience in Ruby and ours with pevious iterations of rustpkg. The two of them will be talking about this project in more detail very soon. Please welcome them into your hearts. Regards, Brian
[rust-dev] Announcing the new Rust package manager, Cargo
Dearest Rustlers, Today I'm super-excited to announce that Mozilla has arranged to develop a world-class package manager for Rust. Yehuda Katz and Carl Lerche, from Tilde Inc., have previously built the popular Ruby package manager, Bundler, and now they are going to build Rust's package manager, Cargo. They are *experts* at the management of packages, and will deliver a tool that builds off both their experience in Ruby and ours with pevious iterations of rustpkg. The two of them will be talking about this project in more detail very soon. Please welcome them into your hearts. Regards, Brian ___ Rust-dev mailing list Rust-dev@mozilla.org https://mail.mozilla.org/listinfo/rust-dev
Re: [rust-dev] Announcing the new Rust package manager, Cargo
You had my curiosity. But now you have my attention. On Mon, Mar 17, 2014 at 5:48 PM, Brian Anderson bander...@mozilla.com wrote: Dearest Rustlers, Today I'm super-excited to announce that Mozilla has arranged to develop a world-class package manager for Rust. Yehuda Katz and Carl Lerche, from Tilde Inc., have previously built the popular Ruby package manager, Bundler, and now they are going to build Rust's package manager, Cargo. They are *experts* at the management of packages, and will deliver a tool that builds off both their experience in Ruby and ours with pevious iterations of rustpkg. The two of them will be talking about this project in more detail very soon. Please welcome them into your hearts. Regards, Brian ___ Rust-dev mailing list Rust-dev@mozilla.org https://mail.mozilla.org/listinfo/rust-dev ___ Rust-dev mailing list Rust-dev@mozilla.org https://mail.mozilla.org/listinfo/rust-dev
Re: [rust-dev] Announcing the new Rust package manager, Cargo
I have long touted Bundler as one of the greatest tools for package management currently in existence and have missed its presence when working in C++ and many other languages. This is a very welcome development indeed for the Rust ecosystem and I'm excited to see what becomes of it. On Mon, Mar 17, 2014 at 6:07 PM, François-Xavier Bourlet bomb...@gmail.com wrote: You had my curiosity. But now you have my attention. On Mon, Mar 17, 2014 at 5:48 PM, Brian Anderson bander...@mozilla.com wrote: Dearest Rustlers, Today I'm super-excited to announce that Mozilla has arranged to develop a world-class package manager for Rust. Yehuda Katz and Carl Lerche, from Tilde Inc., have previously built the popular Ruby package manager, Bundler, and now they are going to build Rust's package manager, Cargo. They are *experts* at the management of packages, and will deliver a tool that builds off both their experience in Ruby and ours with pevious iterations of rustpkg. The two of them will be talking about this project in more detail very soon. Please welcome them into your hearts. Regards, Brian ___ Rust-dev mailing list Rust-dev@mozilla.org https://mail.mozilla.org/listinfo/rust-dev ___ Rust-dev mailing list Rust-dev@mozilla.org https://mail.mozilla.org/listinfo/rust-dev ___ Rust-dev mailing list Rust-dev@mozilla.org https://mail.mozilla.org/listinfo/rust-dev
Re: [rust-dev] Announcing the new Rust package manager, Cargo
Hello Rustlers, I'll be writing here more with more details soon. For now, a few quick comments: - I'm really glad that Mozilla and the Rust team are prioritizing package management. An open source language ecosystem really lives or dies based on how easy it is to share code, and writing a world-class package manager (as brson put it) takes time, especially when you account for the inevitable and important iteration that comes from real-world usage. - There's a lot about package management that's well-understood and somewhat language agnostic. On the other hand, there are things that are specific to native code or even more specific to Rust that a Rust package manager need to account for. My goal is to use well-known best practices for the former, and avoid reinventing the wheel, without losing track of what makes Rust unique or different. Carl and I are both the creators of the predominant Ruby package manager (bundler) and day-to-day production users of Rust (really!) at the company we founded. We think that mix should enable us to balance both of those priorities. - Over the next month or two, we plan to prioritize getting to regular, working milestones with Cargo. These milestones will not always reflect our plan for the final workflow that we expect with Cargo, but having real-world working code is very important when building something the size and scope of Cargo. We plan to share design documents (both on the internal architecture and expected workflows) as we work. We started work 10 days ago, and we already have a primitive cargo compiling one of our libraries based on its Cargo manifest, but so far only via very simple plumbing commands that don't reflect the actual workflow we intend. In general, some guiding principles for the project: - It should be possible for new users of Rust to use Cargo to experiment with Rust and its library ecosystem and have success quickly. - Users of Cargo should get deterministic builds out of the box. If I build an artifact on one machine, it should be possible to build that same artifact on another machine with exactly the same source of every dependency, and exactly the same options. - Users of Cargo should be able to update dependencies with as minimal impact on other dependencies as possible. If I update a utility library, Cargo should avoid updates to other, unrelated libraries by design. - Cargo should support cross-compilation out of the box. As long as your version of Rust and its standard library are compiled for the expected targets, a single line of configuration should be enough to get builds for those targets - Cargo should support the common lifecycle for packages: a package starts out as a part of an existing project, moves to Github for easier sharing across multiple projects and eventually the open source community, and finally is published to a central repository with a version number. This means that all three kinds of sources (local file system, github, and central package repository) should be supported by the default distribution of Cargo. We plan to publish more detail really soon, as well as more information on what we've already built. Please feel free to ask questions :) Yehuda Katz (ph) 718.877.1325 On Mon, Mar 17, 2014 at 5:48 PM, Brian Anderson bander...@mozilla.comwrote: Dearest Rustlers, Today I'm super-excited to announce that Mozilla has arranged to develop a world-class package manager for Rust. Yehuda Katz and Carl Lerche, from Tilde Inc., have previously built the popular Ruby package manager, Bundler, and now they are going to build Rust's package manager, Cargo. They are *experts* at the management of packages, and will deliver a tool that builds off both their experience in Ruby and ours with pevious iterations of rustpkg. The two of them will be talking about this project in more detail very soon. Please welcome them into your hearts. Regards, Brian ___ Rust-dev mailing list Rust-dev@mozilla.org https://mail.mozilla.org/listinfo/rust-dev ___ Rust-dev mailing list Rust-dev@mozilla.org https://mail.mozilla.org/listinfo/rust-dev
Re: [rust-dev] Announcing the new Rust package manager, Cargo
Great news! 2014年3月18日 上午9:27于 Yehuda Katz wyc...@gmail.com写道： Hello Rustlers, I'll be writing here more with more details soon. For now, a few quick comments: - I'm really glad that Mozilla and the Rust team are prioritizing package management. An open source language ecosystem really lives or dies based on how easy it is to share code, and writing a world-class package manager (as brson put it) takes time, especially when you account for the inevitable and important iteration that comes from real-world usage. - There's a lot about package management that's well-understood and somewhat language agnostic. On the other hand, there are things that are specific to native code or even more specific to Rust that a Rust package manager need to account for. My goal is to use well-known best practices for the former, and avoid reinventing the wheel, without losing track of what makes Rust unique or different. Carl and I are both the creators of the predominant Ruby package manager (bundler) and day-to-day production users of Rust (really!) at the company we founded. We think that mix should enable us to balance both of those priorities. - Over the next month or two, we plan to prioritize getting to regular, working milestones with Cargo. These milestones will not always reflect our plan for the final workflow that we expect with Cargo, but having real-world working code is very important when building something the size and scope of Cargo. We plan to share design documents (both on the internal architecture and expected workflows) as we work. We started work 10 days ago, and we already have a primitive cargo compiling one of our libraries based on its Cargo manifest, but so far only via very simple plumbing commands that don't reflect the actual workflow we intend. In general, some guiding principles for the project: - It should be possible for new users of Rust to use Cargo to experiment with Rust and its library ecosystem and have success quickly. - Users of Cargo should get deterministic builds out of the box. If I build an artifact on one machine, it should be possible to build that same artifact on another machine with exactly the same source of every dependency, and exactly the same options. - Users of Cargo should be able to update dependencies with as minimal impact on other dependencies as possible. If I update a utility library, Cargo should avoid updates to other, unrelated libraries by design. - Cargo should support cross-compilation out of the box. As long as your version of Rust and its standard library are compiled for the expected targets, a single line of configuration should be enough to get builds for those targets - Cargo should support the common lifecycle for packages: a package starts out as a part of an existing project, moves to Github for easier sharing across multiple projects and eventually the open source community, and finally is published to a central repository with a version number. This means that all three kinds of sources (local file system, github, and central package repository) should be supported by the default distribution of Cargo. We plan to publish more detail really soon, as well as more information on what we've already built. Please feel free to ask questions :) Yehuda Katz (ph) 718.877.1325 On Mon, Mar 17, 2014 at 5:48 PM, Brian Anderson bander...@mozilla.comwrote: Dearest Rustlers, Today I'm super-excited to announce that Mozilla has arranged to develop a world-class package manager for Rust. Yehuda Katz and Carl Lerche, from Tilde Inc., have previously built the popular Ruby package manager, Bundler, and now they are going to build Rust's package manager, Cargo. They are *experts* at the management of packages, and will deliver a tool that builds off both their experience in Ruby and ours with pevious iterations of rustpkg. The two of them will be talking about this project in more detail very soon. Please welcome them into your hearts. Regards, Brian ___ Rust-dev mailing list Rust-dev@mozilla.org https://mail.mozilla.org/listinfo/rust-dev ___ Rust-dev mailing list Rust-dev@mozilla.org https://mail.mozilla.org/listinfo/rust-dev ___ Rust-dev mailing list Rust-dev@mozilla.org https://mail.mozilla.org/listinfo/rust-dev
Re: [rust-dev] Announcing the new Rust package manager, Cargo
This is incredible news. Thank you for agreeing to help us out in our time of need! I have a couple of questions: - For those of us who are less familiar with Bundler, could you offer a quick run-down of how it works? - Are there any things you learned from doing Bundler that you would do differently the next time round? That is, in addition to the specific changes you would make to tailor it to the Rust language. ~Brendan On 18 Mar 2014, at 12:26 pm, Yehuda Katz wyc...@gmail.com wrote: Hello Rustlers, I'll be writing here more with more details soon. For now, a few quick comments: • I'm really glad that Mozilla and the Rust team are prioritizing package management. An open source language ecosystem really lives or dies based on how easy it is to share code, and writing a world-class package manager (as brson put it) takes time, especially when you account for the inevitable and important iteration that comes from real-world usage. • There's a lot about package management that's well-understood and somewhat language agnostic. On the other hand, there are things that are specific to native code or even more specific to Rust that a Rust package manager need to account for. My goal is to use well-known best practices for the former, and avoid reinventing the wheel, without losing track of what makes Rust unique or different. Carl and I are both the creators of the predominant Ruby package manager (bundler) and day-to-day production users of Rust (really!) at the company we founded. We think that mix should enable us to balance both of those priorities. • Over the next month or two, we plan to prioritize getting to regular, working milestones with Cargo. These milestones will not always reflect our plan for the final workflow that we expect with Cargo, but having real-world working code is very important when building something the size and scope of Cargo. We plan to share design documents (both on the internal architecture and expected workflows) as we work. We started work 10 days ago, and we already have a primitive cargo compiling one of our libraries based on its Cargo manifest, but so far only via very simple plumbing commands that don't reflect the actual workflow we intend. In general, some guiding principles for the project: • It should be possible for new users of Rust to use Cargo to experiment with Rust and its library ecosystem and have success quickly. • Users of Cargo should get deterministic builds out of the box. If I build an artifact on one machine, it should be possible to build that same artifact on another machine with exactly the same source of every dependency, and exactly the same options. • Users of Cargo should be able to update dependencies with as minimal impact on other dependencies as possible. If I update a utility library, Cargo should avoid updates to other, unrelated libraries by design. • Cargo should support cross-compilation out of the box. As long as your version of Rust and its standard library are compiled for the expected targets, a single line of configuration should be enough to get builds for those targets • Cargo should support the common lifecycle for packages: a package starts out as a part of an existing project, moves to Github for easier sharing across multiple projects and eventually the open source community, and finally is published to a central repository with a version number. This means that all three kinds of sources (local file system, github, and central package repository) should be supported by the default distribution of Cargo. We plan to publish more detail really soon, as well as more information on what we've already built. Please feel free to ask questions :) Yehuda Katz (ph) 718.877.1325 On Mon, Mar 17, 2014 at 5:48 PM, Brian Anderson bander...@mozilla.com wrote: Dearest Rustlers, Today I'm super-excited to announce that Mozilla has arranged to develop a world-class package manager for Rust. Yehuda Katz and Carl Lerche, from Tilde Inc., have previously built the popular Ruby package manager, Bundler, and now they are going to build Rust's package manager, Cargo. They are *experts* at the management of packages, and will deliver a tool that builds off both their experience in Ruby and ours with pevious iterations of rustpkg. The two of them will be talking about this project in more detail very soon. Please welcome them into your hearts. Regards, Brian ___ Rust-dev mailing list Rust-dev@mozilla.org https://mail.mozilla.org/listinfo/rust-dev ___ Rust-dev mailing list Rust-dev@mozilla.org https://mail.mozilla.org/listinfo/rust-dev ___ Rust-dev mailing list Rust-dev@mozilla.org https://mail.mozilla.org/listinfo/rust-dev
Re: [rust-dev] Announcing the new Rust package manager, Cargo
On Mon, Mar 17, 2014 at 8:24 PM, Brendan Zabarauskas bjz...@yahoo.com.auwrote: This is incredible news. Thank you for agreeing to help us out in our time of need! I have a couple of questions: - For those of us who are less familiar with Bundler, could you offer a quick run-down of how it works? The most important thing about bundler is that it's structured around allowing you to define the list of dependencies that a particular project needs, and handles fetching them and making sure they are available to the project *across multiple machines*. In other words, a major goal of bundler is that you should be able to clone a Ruby repository, run `bundle install` at its root, and have all of its Ruby dependencies taken care of. This includes both packages published to the Rubygems repository and packages that are still being developed (or versions of packages not yet published) that live on Github. The unusual features of Bundler that I think are most important are: - The lockfile, which stores the actual versions of every dependency that the dependency resolver found, including SHAs for git dependencies. This file is generated every time you run `bundle install`, is meant to be checked into version control, and guarantees that the code that I run on my machine is the same as the code that you run on your machine, and the code that is run in tests and production (obviously, modulo any explicit test- or production-specific changes). - Conservative updating, which builds on the lockfile to guarantee that changes that you make to the primary manifest (known as the Gemfile) only have the smallest possible effect on the list of packages you were using before. This is a solution to the rebuild the universe problem that many package managers have: when you make a small tweak to the list of packages you were using (even a small utility library), you are suddenly opted in to the latest versions of every package that match your original dependency range. In principle, Semver should moot this problem; in practice, changing the actual code you're running as little as possible is a good thing. Bundler, in general, also insisted on proper dependency resolution (using real CS), which makes the above two features possible. TL;DR Bundler Workflow: ``` $ git clone https://github.com/my/repo $ cd repo $ bundle install # uses the lockfile to use last-known-good versions of everything $ bundle exec some-executable # run some-executable from the app's packages $ vim Gemfile # edit the Gemfile to add a new gem or update an existing one $ bundle install # re-calculate dependencies and fetch any new ones locally $ git add Gemfile.lock # commit the (poorly named) lockfile to version control $ git push # now other machines and production can use these dependencies ``` Rust obviously has compilation to contend with, but a lot of this basic workflow (especially around maintaining a list of last-known-good dependencies) applies here as well. - Are there any things you learned from doing Bundler that you would do differently the next time round? That is, in addition to the specific changes you would make to tailor it to the Rust language. Yes. I'll reply to this tomorrow :) ~Brendan On 18 Mar 2014, at 12:26 pm, Yehuda Katz wyc...@gmail.com wrote: Hello Rustlers, I'll be writing here more with more details soon. For now, a few quick comments: * I'm really glad that Mozilla and the Rust team are prioritizing package management. An open source language ecosystem really lives or dies based on how easy it is to share code, and writing a world-class package manager (as brson put it) takes time, especially when you account for the inevitable and important iteration that comes from real-world usage. * There's a lot about package management that's well-understood and somewhat language agnostic. On the other hand, there are things that are specific to native code or even more specific to Rust that a Rust package manager need to account for. My goal is to use well-known best practices for the former, and avoid reinventing the wheel, without losing track of what makes Rust unique or different. Carl and I are both the creators of the predominant Ruby package manager (bundler) and day-to-day production users of Rust (really!) at the company we founded. We think that mix should enable us to balance both of those priorities. * Over the next month or two, we plan to prioritize getting to regular, working milestones with Cargo. These milestones will not always reflect our plan for the final workflow that we expect with Cargo, but having real-world working code is very important when building something the size and scope of Cargo. We plan to share design documents (both on the internal architecture and expected workflows) as we work. We started work 10 days ago, and we already have a primitive cargo compiling one of our
