Hi DCers, As we've discussed, we've set up a Subversion code repository on the Precursor archive machine (lsstarchive.ncsa.uiuc.edu). Attached is documentation detailing how to get going on it; it's also in docushare in our DC1 folder (Collection-472). We would very much like to have your feedback on both this documentation and the directory structure we've set up. If you have recommendations for the structure, I ask that you try to get those back to us by Tuesday (6/6), so that we can make changes accordingly before we get a lot of check-ins. We can still make changes later than that, but nevertheless, it would be convenient to establish a common understanding of the structure early.
To review the structure, I recommend that you do a full check out of the main branch (the so-called "trunk"); see the first example in section 2.3 of the attached documentation. Finally, although the main driver for this repository at the moment are the data challenges, we intended this to be open to all new DM code. However, I would like to let the DC folks shake it out first (say, over the next week) before we add too many more users. thanks, Ray
LSST Software Repository A Subversion repository has been set up for LSST software. This document describes the basics of accessing the archive as well as the overall structure. It also provides a quick survival guide for those unfamiliar with Subversion. Contents: 1. The Repository Installation at NCSA 1.1. Installing Subversion 1.2. Repository Structure 1.3. Access to LSST Subversion Repository 2. Subversion Survival Guide 2.1. Helpful Documentation 2.2. Differences with CVS 2.3. Common Usage 1. The Repository Installation at NCSA The LSST Software repository is located on lsstarchive.ncsa.uiuc.edu, one of NCSA's Data Cluster nodes. To add to this repository, you will need ssh access to this machine. If you do not have an account, please contact the NCSA LSST team ([EMAIL PROTECTED]) to have an account added. Mention that you need Subversion access so you can be added to the subversion users group on the system. 1.1. Installing Subversion The client program you will use is called svn, so if you do not have this program on your system, you will need to install it. The Subversion source code is available from http://subversion.tigris.org/ and builds quite readily under Linux. This site also provides pointers to binary distributions as well. 1.2. Repository Structure The Repository is laid out with three main catagories at the top--infrastructure, middleware, and applications--with some predefined sub-categories below them. Developers are welcome to add additional sub-categories if appropriate. We recommend that developers contribute code in the form of packages, added into one of the (preferably existing) sub-categories. For example, a pipeline queue library might be added as a directory (representing the package) named "SimpleQueuing" to the middleware/pipeline/appframework directory. We further recommend the following structural pattern within the package: README.txt -- a brief description of the package and overview of build instructions. src/cpp -- directory for C/C++ source code for the package src/java -- directory for Java source code for the package src/python -- directory for Python source code for the package src/scripts -- directory for shell scripts (e.g. that launch Java applications). doc -- directory for package documentation conf -- directory for configuration files If any of the above directories are not used, then they need not exist. Similarly, other directories might be added--e.g., for other implementation languages. Files used to build the package (e.g. Makefile, build.xml) should exist in the top directory of the package--i.e., as siblings of README.txt. 1.3. Access to LSST Subversion Repository Access to the LSST subversion repository is through use of ssh. The general syntax to access the LSST repository is: % svn command svn+ssh://lsstarchive.ncsa.uiuc.edu/path/to/code For those familiar with CVS, there is no mechanism for setting the root directory of the repository in subversion, but you can set an environment variable so you needn't type the string all the time, e.g., % setenv SVN svn+ssh://lsstarchive.ncsa.uiuc.edu % svn command $SVN/path/to/code The svn+ssh prefix tells subversion to use the ssh protocol to connect and once connected, run the subversion server to access the repository. Note that you will be prompted multiple times for your ssh password. This can be avoided by either using ssh-agent to manage your ssh session or by installing a passphrase-less public key into the .ssh/authorized_users file in your home directory on the lsstarchive machine. 2. Subversion Survival Guide 2.1. Helpful Documentation The Subversion command-line client svn interface is modeled after CVS. See also svn help for more commands and options, as well as the on-line book at http://svnbook.red-bean.com/nightly/en/svn-book.html Section 3 ("Guided Tour") is especially useful for getting started. 2.2. Differences with CVS If you are familiar with CVS, you should note these behavioral differences: o There is no equivalent to CVSROOT o With Subversion, revision numbers are assigned globally, not on a per-file basis. Everytime a commit is done, the revision of the entire repository increments. o A commit command is guaranteed atomic, regardless of the number of files being committed; if anything goes wrong, none of the files will be committed. o When update conflicts are detected, additional local versions are created to help you sort out the conflict (see below). 2.3. Common Usage In the examples given below, it is assumed that the SVN environment variable has been set to svn+ssh://lsstarchive.ncsa.uiuc.edu/ as suggested above. 1. Checkout files svn checkout URL [PATH] This downloads the part of the repository pointed to by URL into the local directory given by PATH. Example: Check out everything for normal development and put it into a local directory called dev. Note that the trunk represents the leading edge of main development, excluding special branches or tagged sets. $ svn checkout $SVN/trunk dev Example: Checkout a local copy of the pipeline construction code from the middleware tree of the repository. $ svn checkout $SVN/trunk/middleware/pipeline/construction pcs 2. Adding and removing files or directories svn add PATH svn delete PATH svn commit -m "LOG-MESSAGE" [--recursive] PATH ... The first two commands above update your local copy of the code tree, while the third line commits the local changes to the repository. Example: add a new source file and delete an old one. $ svn add foo.py $ svn delete bar.py $ svn commit -m "prefering foo.py over bar.py" foo.py bar.py Here's another way to do the commit that catches all pending changes under a directory: $ svn commit -m "prefering foo.py over bar.py" --recursive . An easy way to add an entire package tree to the repository is with the import command: svn import PATH URL This recursively imports a copy of PATH to repository. Example: add in a new package, code and all: $ svn import mypkg $SVN/trunk/middleware/appframe/mypkg 3. Updating existing files svn update [PATH ...] ... svn commit -m "UPDATE-MESSAGE" [PATH ...] The update command downloads any changes to the files or directories pointed to by PATH since the last time you checked out or updated them. The behavior is much like in CVS. Example: update and commit some files $ svn update foo.c bar.c choo.c U foo.c G bar.c C choo.c In this example, an updated version of foo.c was downloaded. bar.c was also updated; however, svn noticed that your local version included uncommitted changes that you made, and these changes were merged into the updated version. With choo.c, the merge was not successful, and special tags were inserted that mark where you need to resolve conflicts between your local changes and the changes downloaded from the repository. The difference in behavior from CVS is that in the case of a conflict, svn may create additional versions of the file to help with resolving the conflicts: choo.c # the updated version with conflicts tagged choo.c.mine # your pre-updated version choo.c.r1 # the committed version prior to your updates choo.c.r2 # the committed version to be merged with your version After adding resolving conflicts and making new changes, you can commit: $ svn commit -m "bug fix: memory leak" foo.c bar.c choo.c If svn detects that someone else has checked in any changes since you did your update, the entire commit for all three files will fail (unlike with CVS). In this case, you should run the update command again on those files and resolve an conflicts before trying another commit. 4. Checking the status of directory and files It's often useful to peruse the differences between your local files and those in the repository without actually doing an update. These command are useful for doing this. svn status [PATH ...] lists all files that are not in sync with repository and tags its state (e.g. needs updating, in conflict, etc.). svn diff [PATH ...] shows differences between your local version and the version in the repository. svn revert [--recursive] [PATH ...] undo any uncommitted changes, adds, deletes, copies, and moves of the listed files or directories. For more information on these commands see Section 3 (Guided Tour/Examine Your Changes) and Section 9 (Subversion Complete Reference).
_______________________________________________ LSST-data mailing list [email protected] http://www.lsstmail.org/mailman/listinfo/lsst-data
