On Thu, 2008-12-11 at 16:23 -0600, Shawn Walker wrote: > >> * overhead of a few actions to every single manifest in that repository > > > > That would be typically <10 actions. > > multiplied by the number of packages that the system is managing.
Yes, but if that overhead brings about doom then doom is near already. The biggest resource overhead that I can see is the disk space needed for the source tarballs. > The increased publishing overhead, however, is unique to adding these > additional resources. It also incurs additional penalties in > repository > management (time to synchronise the repositories, extra bandwidth and > system resources, etc.). I accept that, it is a disadvantage. > >> While I understand what you're trying to accomplish, my gut feeling is > >> that the depot server is not the right place to host this information. > > > > I think it is. To simplify, the depot is for hosting files grouped > > into packages and nicely versioned so they can be used to construct > > consistent images. To host the information for rebuilding these > > packages, we need to host files grouped into packages that are > > versioned the same way as the binary packages. > > I still believe it is not the right place to store this information. > Quite frankly, I wouldn't want us to be using what resources we have for > serving this content instead of the content we really care about: packages. So, that's mostly disk space and publishing/sync bandwidth. That should not dictate architecture. (If it's not logical to store these files in the IPS repo that _is_ a good reason.) > I also think that this content will end up being duplicated, as I would > imagine that you would need a copy of all of this content somewhere else > anyway (such as a source code control repository). At least for JDS and SFE, the source tarballs are not mirrored on opensolaris.org (or anywhere on the internet), and this is a pain for those who build our sources. We do have a tarball store inside Sun, but external contributors have to rely on various download servers that are outside outside our control. Storing tarballs in SVN would really really be inefficient and definitely the wrong thing to do. The spec files and patches are in SVN, so they would really end up being duplicated. However, I would like to be able to create source packages for code that is not (or not yet) under version control, or uses a version control mechanism that the build tool doesn't implement. (If we only include pointers files in an SCM repo, the build tool has to support each SCM so that it can encode the necessary information to get to the file and to be able to get those files for a rebuild.) > It would seem more logical to me to have pointers to wherever it is that > you're already hosting that content instead of duplicating it. Okay, so an alternative solution could be * mirror the source tarballs somewhere * store as metadata: - the scm type - the scm location - revision/changeset id - list of files to get from the scm - pointer to the tarball store * implement tools for adding tarballs to the tarball store * implement support for each scm in the build tool I don't think it's quite as nice and simple as using the same tool (pkg) for storing both the sources and the binaries. It would be a lot easier to run a separate pkg.depotd for source packages only and add the fmri of the source package to binary package as metadata. Laca _______________________________________________ pkg-discuss mailing list [email protected] http://mail.opensolaris.org/mailman/listinfo/pkg-discuss
