[Foresight-devel] Re: conary repository setup instructions

Michael K. Johnson Sat, 22 Jun 2013 06:22:30 -0700

On Sat, Jun 22, 2013 at 07:14:26AM +0200, Tomas Forsman wrote:
> On 06/21/2013 10:46 PM, Michael K. Johnson wrote:
> I can't see anything and thunderbird is saying:
>
> This attachment appears to be empty.
> Please check with the person who sent this.
> Often company firewalls or antivirus programs will destroy attachments
>
> Possible to re-attach it again?


It was the list settings. I have changed them slightly and will try
again...

Title: Conary:Repository - rPathWiki

Conary:Repository

From rPathWiki

Installing and Running a Local Conary Repository

If you would like to test Conary packaging in a private environment, or have another requirement for access to a repository that is not publicly available, you can install and run a local networked Conary repository.

This guide describes the installation of software components and configuration necessary to enable a private repository, and how to access the repository once it is running.

These instructions apply to Conary 1.0.14 or later.

Software

Before you start, you should make sure all the necessary software is installed. The required software depends on the following factors:

The underlying database you want your Conary repository to use
The manner in which the repository is to be run

The SQLite, MySQL and PostgreSQL databases are supported by Conary. All Conary-based distributions will already include the necessary SQLite software, as SQLite is used by Conary for the system's Conary database.

Using PostgreSQL requires the postgresql, postgresql-server and python-pgsql packages; and pgbouncer is highly recommended.

Using MySQL requires the mysql, mysql-server and MySQL-python packages.

The Conary repository can run in single-threaded standalone mode for debugging and lightly-used repositories, but for production use should be run behind an nginx proxy using gunicorn, or as part of apache through mod_python.

To use the mod_python setup, first install the httpd, mod_python, conary-repository, and elementtree packages. Installing conary-web-common as well makes for a nicer web interface.

Configuration

nginx/gunicorn wsgi

Configuration files are provided in the Conary source code in the extra/repos/ directory. (This description should be updated with instructions on how to deploy them, as this is now the recommended way to deploy a standalone Conary repository.)

mod_python

To use the mod_python setup, create a configuration file in /etc/httpd/conf.d/ containing the following (replacing <path-to-repository-prefix> with your intended repository location):

Alias /conary/ "/<path-to-repository-prefix>/config/repository.cnr/"

<Directory /<path-to-repository-prefix>/config>
    AddHandler python-program .cnr
    PythonHandler conary.server.apachehooks
</Directory>

# You may also find it useful to redirect /conary to /conary/, but if you
# do, make sure you have trailing slashes in your repository Map entries.
RedirectMatch /conary$ /conary/

repository configuration files

Next, create your repository.cnr file in /<path-to-repository-prefix>/config/ (i.e., the path you listed under Alias in the file you just created in /etc/httpd/conf.d/. This file follows a conaryrc-like syntax with the following options:

`commitAction`	This can be any action you would like the server to perform after each commit, such as email notification.
`forceSSL`	Do not allow authenticated operations unless those operations are done over SSL.
`repositoryDB`	The database location for the repository. Currently MySQL and SQLite are supported. To specify a MySQL database, the configuration value is `mysql <database-user>@<host>/<database-name>`. For SQLite, use `sqlite /<path-to-sqlite-file>/sqldb` (assuming the usual SQLite name of `sqldb`).
`changesetCacheDir`	The location for the cached changesets. Conary can cache changeset files as they are created. this directive configures where conary will cache these changesets. The user that will be running the conary repository must be able to write to this location.
`contentsDir`	The location to store file contents. The user that will be running the conary repository must be able to write to this location.
`logFile`	The location of the conary xmlrpc log. The user that will be running the conary repository must be able to write to this location.
`readOnlyRepository`	Set this directive's value True to ensure no write operations will be performed on the repository.
`repositoryMap`	OBSOLETE This value is no longer needed. The client settings are used when performing repository operations.
`serverName`	The name of the repository server, or a list of names to which this repository answers.
`tmpDir`	The path to use for repository-related temporary files.

Here's a useful example to copy to /<path-to-repository-prefix>/config/repository.cnr (replacing the fields on the repositoryDB line with appropriate values for your situation, <path-to-repository-prefix> with your intended repository location, foo.example.com with your desired value for serverName, and [email protected] with the email address to receive commit emails):

repositoryDB <database-type> <database-specific-info>
contentsDir /<path-to-repository-prefix>/contents
tmpDir /<path-to-repository-prefix>/tmp

# this is the hostname part of the label, not the hostname of the machine
# so on both conary.rpath.com and on conary-commits.rpath.com, this
# is "serverName conary.rpath.com".  This can also be a list if your repository
# serves more than one label.

serverName foo.example.com

# The following commit Action sends mails on commits

commitAction /usr/lib/python2.4/site-packages/conary/commitaction --module "/usr/lib/python2.4/site-packages/conary/changemail --user %(user)s  --email [email protected]"

Make sure that the web server user (normally apache) can write to the appropriate files and directories; particularly the repository Dir and tmpDir.

Creating the Database

sqlite

Due to SQLite's design, it is not necessary to explicitly create the database, as the first database access performed (when creating an administrative user in the next step) will cause the database to be created. You need to make sure that the user you are using for running the standalone server has write permissions to the database path you are using in the repositoryDB setting. For sqlite, the repositoryDB setting would be something like:

repositoryDB sqlite /path/to/repository/sqldb

This will use the sqldb file created under the /path/to/repository directory.

postgresql

Postgresql 9.1 or later is recommended.

Beyond usual Postgresql tuning, allowing for large transactions, make sure to set bytea_output = 'escape' in postgresql.conf.

# createdb -U postgres --owner repositoryOwner -T template0 name_of_database

Especially if you host many databases, it is strongly recommended to use pgbouncer. A typical pgbouncer.ini configuration will look like:

[pgbouncer]
user = pgbouncer
pidfile = /var/run/pgbouncer/pgbouncer.pid
listen_addr = SOME_INTERNAL_IP_NOT_THE_INTERNET
listen_port = 6432
auth_type = md5
auth_file = /etc/pgbouncer.users
admin_users = pgbouncer
pool_mode = session
default_pool_size = 20
server_reset_query = RESET ALL; SET SESSION AUTHORIZATION DEFAULT;
server_idle_timeout = 45

[databases]
name_of_database =
...

Then the /etc/pgbouncer.users file will need to define the users and passwords used to access the database in the format:

"repositoryOwner" "repositoryOwnerPassword"

(The quotation marks are literal and are included in the file.)

Assuming you have your postgresql database name_of_database created, your database or pgbouncer is running on dbhost, and the username you need to connect is repositoryOwner, your repositoryDB entry would look something like this:

repositoryDB pgpool repositoryOwner@dbhost/name_of_database

You can supply a passsword for dbuser this way:

repositoryDB pgpool repositoryOwner:repositoryOwnerPassword@dbhost/name_of_database

For reference, the database connection string has the following specification: dbuser[:password]@dbhost[:port]/dbname, with the password and port settings being optional.

mysql

It is not generally recommended to use mysql for the database. For large transactions, it has in practice not performed as well as postgresql.

In the case of MySQL, you must create the database using the CREATE DATABASE <database-name>; SQL statement. The recommended way to create the database is:

CREATE DATABASE <dbname> CHARACTER SET latin1 COLLATE latin1_bin;

Also, we recommend you use InnoDB as the storage engine for the mysql database. The following configuration options are required by the repository schema (add these to the [mysqld] section of your /etc/my.cnf and restart the the mysql server):

# server mode configuration
sql_mode = TRADITIONAL,NO_AUTO_VALUE_ON_ZERO,ONLY_FULL_GROUP_BY
lower_case_table_names = 1
default-table-type = INNODB
character_set_server = latin1
collation_server = latin1_bin

Assuming you have your mysql database dbname created, your MySQL database is running on dbhost, and the username you need to connect is dbuser, your repositoryDB entry would look something like this:

repositoryDB mysql dbuser@dbhost/dbname

You can supply a passsword for dbuser this way:

repositoryDB mysql dbuser:password@dbhost/dbname

For reference, the database connection string has the following specification: dbuser[:password]@dbhost[:port]/dbname, with the password and port settings being optional.

Converting between databases

The /usr/share/conary/migration/db2db.py script in Conary can convert from one supported database type to another.

Creating Users

To run a repository, you'll need to define authorized users. Use server.py to create the first (administrative) user:

/usr/lib/python2.4/site-packages/conary/server/server.py --add-user admin --admin --config-file <path to your .cnr file>

It will prompt you for the password, and create that user as an administrative user (which lets the user create new users on the system). Once the repository is up, you can point a web browser at the repository and use the web interface to add other users (including other administrative users).

For anonymous access to the repository, create a user with name anonymous and password anonymous that has read-only access.

/usr/lib/python2.4/site-packages/conary/server/server.py --add-user anonymous --config-file <path to your .cnr file>

Running this command as root can create files with permissions inaccessible to the web server. If you are using SQLite, make sure the database file is writable by the apache user. Also make sure that the permissions are correct on contentsDir, tmpDir and /<path-to-repository-prefix> as conary must be able to write to these directories.

Each user will need to submit at least one valid OpenPGP (GPG) public key. This can be done from the web interface.

Starting and Accessing the Service

Conary requires that its web server support SSL. To enable SSL in apache, install mod_ssl:

conary update mod_ssl --resolve

After installing mod_ssl, use the following command to restart apache:

/sbin/service httpd restart

You can now use the repository's web interface by pointing a browser to foo.example.com/conary/ and access your new repository from Conary as foo.example.com.

_______________________________________________
Foresight-devel mailing list
[email protected]
https://lists.foresightlinux.org/mailman/listinfo/foresight-devel