Proposal: Adopt Conary as the standard packaging and distribution
framework for OpenSolaris.
The excellent open source packaging and build system Conary has been
suggested a few times as the packaging system for OpenSolaris.org.
I have one a preliminary review of Conary, and feel that it would make
an excellent choice for an OpenSolaris packaging and distribution
system. I would like endorsement to start a project and get a mailing
list created.
I have included some introductory information below:
I am proposing myself as project leader. (And anyone else who is interested)
Description:
--------------
The Conary software provisioning system is a means to package,
deliver, and maintain software.
Conary provides new features to package management, addressing some of
the common issues of system administration. Employing repositories, a
branch structure with shadowing, and change sets, Conary offers the
advantages of a revision control system to improve system maintenance
tasks such as updating current software, installing new software,
maintaining configurations during software updates, and resolving
dependency issues.
These features extend beyond previous software packaging technologies,
creating a system in which many traditional software maintenance tasks
are automated and efficient, significantly reducing the time spent on
system management.
How?
-------
A Conary repository is a database that contains source and binary
files organized in packages which consist of any number of components.
This database is accessible over a network (including over the
Internet if needed) by Conary-based systems, and it is the means by
which these systems can install and maintain their software. The
repository contains multiple packages in multiple versions along
multiple development branches, and Conary ensures that system
maintenance uses the appropriate version on the appropriate branch
when updating a system. Access to a package in the database requires
enough information to distinguish the exact version required, but much
of this information is maintained in configuration files and installed
package information to further automate the update process.
Repository Contents
-------------------------
A repository provides the packages that can be installed on a
Conary-based system. Other types of content in the repository include
components, groups, and filesets, all of which represent a type of
trove, or versioned item in the repository.
Packages
------------
A package represents a piece of software that can be installed on a
Conary-based system. When software is packaged for Conary, building
the package creates components, and the package contains those
components.
When a user installs a package, Conary installs all the components of
that package. Conary will also indicate any dependencies the package
requires, and the user can add an option to let Conary automatically
resolve those dependencies. When Conary installs a package with this
option, it additionally installs only the components necessary to
resolve the dependency.
When a user updates installed packages, Conary only retrieves files
that have changed since the last update on the package. For example,
if the component foo:runtime has five files, and only one of those
files has changed between two versions, then only that one file will
be updated when updating between the two versions.
>From the repository's perspective, Conary aims for similar efficiency.
The new version of a package does not contain duplicates of files in
previous versions. Only if a file has changed will its update
information be saved to the repository.
Components
---------------
A component is a grouping of files that are needed to install some
specific part of software, such as the application, libraries, or
documentation. When a package is built, Conary automatically assigns
files to components (though this assignment can be overridden if
desired). Components are identified using the package name to which it
applies, a colon (:), and a suffix describing the component (such as
package_name:lib and package_name:docs).
A source component (:source) contains files and is named like other
components, but Conary treats source components differently during
development and maintenance of packages. Though it is named using the
original package for which it was built, a source component is
versioned differently, and it is not considered part of any package. A
source component may be used to build multiple packages, and it not
installable like other components.
Troves
--------
A trove is a collection of files or a collection other troves. The
term trove is used often in Conary discussions. A trove could refer to
a component, a package, a group, or a fileset.
A trove is identified by name, version, and flavor; these three values
make the trove unique in the repository. When a file in a trove is
first saved to a repository, it is assigned a unique identifier so
that it can be tracked by that identifier instead of by name. The file
identifier allows Conary to handle continued updates to the file,
including filename changes, while preserving its unique existence in
the trove.
Groups
A group contains a grouping of other kinds of troves (components,
packages, filesets, or groups) that are found in the same repository.
Groups are mainly used to pull together packages and components that
should be installed together to create a complete infrastructure for
software. A group name has the prefix group-. The Conary
QuickReference provides instructions for installing and updating
groups on a Conary-based system.
Groups are versioned like other troves and can be updated in the same
way. The advantage of creating a group of packages and components is
that these troves can be updated and maintained together. The group
identifies specific versions of each of its contents. Installing a
group means installing those specific versions of packages and
components. Because of this, installing or updating a group may not
result in receiving the latest available version of its contents
because a package or component in the group may have a later version
available outside of the group. Instead, installing and updating by
group ensures that the group maintains the versions that were
intended. Groups allow a developer to distribute software that is
assured to work together. If packages or components in a group are
updated, the group update will only affect those updated troves.
For example, suppose you have installed group-foobar on your
Conary-based system which consists of packages foo-1.2.1 and
bar-2.1.1. Suppose you are told that newer version foo-2.0.1 is now
available, and you perform a system-wide update on your system to
ensure you have the latest version. Because foo was installed as part
of the group and not on its own, your version of foo remains 1.2.1.
Then, suppose group-foobar is updated to use bar-2.2.1. Another system
update results in a group update; while foo remains in version 1.2.1,
bar is updated to version 2.2.1.
Filesets
A fileset contains only files from components in the repository.
Filesets can be created to designate a custom group of files in the
repository outside of the component, package, and group purposes. A
fileset name has the prefix fileset-. This type of trove is rarely
used outside of low-level Conary development.
Repository Access
Conary uses client-side repository access features that enhance
Conary's functionality.
Repository Map
-------------------
A repository map setting on a Conary-based system maps a hostname from
a label to a full URL for a networked repository. When software is
installed using Conary, the label is part of the information that
identifies the software and its source. If the repository changes
locations, the repository map can be updated to ensure that Conary
finds the relocated repository when a user runs software updates.
Repository map settings are maintained in a Conary configuration file
on the Conary-based system.
Install Label Path
Though it is an option, Conary does not require a user to specify a
label when installing a trove from a repository. When a user names the
trove (with or without specifying a revision), the repositories listed
in the system's install label path are searched in order to locate
that trove and fulfill the request. As needed, repositories can be
appended to or removed from this path. Install label path settings are
maintained in the installLabelPath line of a Conary configuration file
on the Conary-based system.
Mirrors
Conary provides the option to mirror a single repository onto another
Conary system. This can be useful when backing up a repository or when
making privately-developed software packages available in a publically
accessible repository. Conary users can create a standalone repository
and mirror to that repository. Additionally, project owners on an
rBuilder Online (rBO) project can create a mirror of the rBO
repository identified with that project. On the software appliance
scale, appliance developers and maintainers can use the rPath
Appliance Platform Update Service appliance to manage mirrors provided
to multiple consumers (such as software vendors or customers).
Branch Structure
Conary uses a branch structure to simplify update paths for software.
This structure is aimed at eliminating the guesswork when identifying:
* Whether the package is "newer" or "older"
* The appropriate upstream development path the current package
has followed to its current version
Conary identifies each component with a version string that identifies
the originating respository, revision number, and branch structure to
which it belongs. When a user performs system update activities,
Conary uses this string as the update path to follow for each trove.
Packaging technologies without this branch structure may have
uncertainties in the update path resulting in broken packages after
updates are applied.
Versions
-----------
As previously described, Conary structures packaged software in
components, and each component contains one or more files. The version
string provides a globally unique identifier for the component. See
Conary Version String which describes the version string and its
parts, including the commonly used branch, label, and revision values.
Flavors
Along with the name and version, the flavor of a trove makes it unique
among troves that can be installed on a Conary-based system. In the
same way that the version identifies where and when in a sequence the
trove was built, the flavor identifies how it was built. This
information is part of the build instructions and includes selecting
options in the packaged software, such as designating an architecture
or ensuring support for a particular database. Flavors are typically
invisible to users installing software because Conary automatically
selects the most appropriate flavor for a package for the target
system. Conary provides an option to override this automatic choice to
users wishing to select a particular flavor of the package.
Branch Affinity
------------------
Conary supports divergent development by allowing that development to
occur on different branches. Such development in the Conary repository
creates a branched structure and a need to identify which branch to
use when updating packages installed from the repository. When a trove
is installed on a Conary-based system, the updates for that trove will
come from the same branch in the respository from which it was
installed. This concept, called branch affinity, ensures the update
path is consistent, even when the development is not linear. The
identifiers that Conary creates for ensuring branch affinity are
hidden from users, and it is not necessary to know them when updating
software on a Conary-based system.
Conary recognizes an appended version string with a slash and branch
labels as a unique identifier for a development branch. When the
branch is created, the branch name becomes the label for that branch.
This new branch inherits the namespace, so it can be referred to not
only with the longer slash-separated strings, but also with the branch
label in place of the original tag. In the example, 1 is the branch
name, and the full label for the branch is conary.rpath.com at rpl:1.
/conary.rpath.com at rpl:devel//1/1.5.0-2-3
or
conary.rpath.com at rpl:1
If a package is installed from this branch, the branch name
(namespace:tag, such as rpl:1 in the example) ensures that Conary uses
the correct update path in the repository.
Historically, Conary had separate branch and shadow creations.
Explicit branch creation was deprecated so that shadows alone create a
repository's branch structure.
Shadows
Conary provides the shadow concept as a means of creating parallel
development branches. Shadows can keep track of upstream changes from
a parent branch, and developers can choose to merge changes from any
new revision on the parent at any time. In this way, developers can be
responsible for their own changes without maintaining the entire
development stream.
Changesets
---------------
A Conary changeset is a file that describes modifications to be made
to a system in order to apply a Conary update. A changeset can include
how particular files have changed as well as how their associated
troves have changed. Conary uses changesets to make the update process
more efficient and less intrusive to a Conary-based system. By
applying only the changes that are needed, files that have not changed
are not needlessly updated, and locally-changed files are merged with
updated files instead of overwritten.
A changeset represents the difference between two versions. An
absolute changeset is the difference between having no version
installed on a system and installing the desired version. A relative
changeset is the difference between a currently installed version and
the desired version. A relative changeset could imply either an
upgrade to a newer version or a downgrade to an older version.
Changesets are typically invisible to the user making updates to a
Conary-based system. During an update operation, Conary creates the
changeset dynamically before downloading and applying that changeset.
The changeset may be cached by the repository server, but it is not
saved as a separate file on the target system. In some instances, a
system administrator or software developer may generate a static
changeset file to distribute a desired set of changes.
One use of a static changeset is a local changeset, which is a
relative changeset containing the differences between the trove as
installed and configured on the local system and the trove as it
exists in the repository. System administrators can distribute certain
configuration and permission changes on a trove by creating and
distributing a local changeset. The local changeset can be distributed
to target machines with the same trove version, and it can be
committed to a branch of the repository from which target machines can
install the necessary update.
Rollbacks
-----------
As a result of the use of changesets in Conary, a Conary-based system
provides the user an opportunity to roll back changes that have been
made to the system as the result of a Conary update. Conary maintains
a list of rollbacks, and the system administrator can roll back a
trove or the entire system when necessary to bring the system back to
a particular state.
--
- Brian Gupta
http://opensolaris.org/os/project/nycosug/
--
- Brian Gupta
http://opensolaris.org/os/project/nycosug/
