1.4.7 is relatively ancient. The issue is that we really want to be working with the 1.5.* release, because there are critical features that any Enterprise using OpenAFS is going to need, and some key features I intend to leverage.
Even if someone starts putting 1.5.* rpms out there, I want to be in total control of it for my own development environment. I expect to be patching it myself, and applying bleeding edge patches from others, so the ability to build it ourselves is essential. Hell, I've already submitted 3 patches to gerrit.openafs.org already -- there are sure to be more to follow.... On Fri, Oct 8, 2010 at 1:48 PM, Baldwin Sung <[email protected]> wrote: > Looks like OpenAFS does have a repository you can add into CentOS. Couldn't > find one for 1.5.x too quickly. > > * > http://dl.openafs.org/dl/openafs/1.4.7/openafs-repository-1.4.7-1.noarch.rpm > * > > On Oct 8, 2010, at 1:31 PM, Phillip Moore wrote: > > > This entire exercise has suggested that we're doing something very > inefficient with the way we build these VMs. > > The original goal of efs_virtual_machine was to automate the entire setup > of these machines, assuming that the starting point is a VM "template" that > is a purely vanilla installation of the distribution. The first thing the > script does is install a bunch of packages, of course. > > I'm pretty sure none of you have rebuilt things anywhere close to the > number of times I have. What I've found is that the slowest part of the > process is the software installation. Once that's done, all the OS > configuration is pretty quick. > > Now that I've added OpenAFS to the mix, it's a LOT more complicated, > because OpenAFS packages are not part of the default network of "yum" > servers for CentOS, and you need to install packages that are specific to > the kernel patch level of the local machine. This is really tedious, and > really slow. > > I think it makes sense to factor out the package installation, and do that > ONCE, per VM template. If we want to upgrade the OpenAFS kernel version, > then we can upgrade the templates first. > > So... I'm going to rebuild my VM templates today, and take notes on HOW I > did it (we've always left that as an exercise for the developer, and it's > not trivial). This will deal with another nit that's been annoying me. > When you try to tun efs_virtual_machine on a couple of the platforms, you > have to first install a couple of packages anyway (perl on Debian, IIRC, and > a few things on FreeBSD). I think it makes sense to create VM templates > with minimally installed software, but my point is that the way we do it now > is *too* minimal. > > My notes are going to be specific to how I do this in VMWare Fusion on > MacOS X, and while the OS install part will be generic, the VMware part > won't. I'll leave it up to others to patch those notes with instructions > specific to the other VMware products. > > The big win here will be that we can deal with the OpenAFS stuff ONCE, and > then forget about it, until we need to upgrade. Then, we'll be upgrading > the templates, once per platform. > > On Fri, Oct 8, 2010 at 12:39 PM, Phillip Moore > <[email protected]>wrote: > >> After I got the test.efs setup done with 1.4.12, I immediately wanted to >> rebuild it all with 1.5.77, since there are some key features in the 1.5.* >> code that EFS is going to leverage. In particular, I plan to require the >> newer "fs mkmount" syntax that let's me avoid creating temporary volume >> mounts (man, do I wish I had that back when I wrote AFS/VMS...) >> >> This has been a LOT of work, and not everything's automated yet, but >> here's what I've accomplished. In Jerry's boot.efs environment, I built >> openafs/core/1.5.77-build001, and managed to get a complete build of the >> user-mode binaries, but I built with --disable-kernel-module for that >> release. The goal is to use openafs/core/* releases for the servers I'll be >> building, but that comes later (I have a really sick idea for leveraging >> /efs/boot, but as usual, I digress...) >> >> In order to build the test.efs setup, I needed a set of 1.5.77 rpms. >> Building the rpms is a little tricky. First, you have to build the srpm, >> and then you use that srpm to build the binary rpms. This is partially >> automated in openafs/rpms/1.5.77-build001, but I quickly learned that you >> have to build a kmod-openafs-* binary rpm for EVERY SINGLE kernel patch >> level. Simon Wilkerson wrote a script that mass-produces these, but I >> haven't figure out how it works yet. The stuff in the install tree in >> openafs/rpms is what was built automatically, but it only includes the >> kmod-openafs rpm for the kernel version that is used on the compile servers. >> >> As soon as I tried to use that kernel rpm on MY CentOS 5.4 machines, I >> discovered that Jerry's setup runs: >> >> 2.6.18-164.el5 >> >> but mine are running: >> >> 2.6.18-164.15.1.el5 >> >> This required building another kmod-openafs rpm for that specific kernel. >> What that means is that the simple approach I took for openafs/rpms will >> not work. This is why Simon wrote the script he did, and I will need to >> figure out how to setup the build environment, which mostly involves >> collecting all the kernel-devel-* rpms needed to populate /usr/src/kernels >> with the right directories. I'll know more when I dig into it. >> >> In order to make the test.efs environment reproducible, which is my >> immediate goal, I've uploaded the rpms I'm working with to: >> >> ftp-osl.osuosl.org:~openefs/data/packages/openafs/1.5.77/$platform >> >> The $platform values are x86-32.rhel.5 and x86-64.rhel.5 of course. This >> will be an entirely different effort for Debian/Ubuntu (although that will >> probably be much easier, since Russ Allbery owns that code, and he does a >> fantastic job of managing OpenAFS for at least Debian), and the other OSes, >> as well of course. >> >> The setup process for getting test.efs configured is significantly more >> complicated now. For one thing, in order to NOT require everyone else to >> setup AFS, I've decoupled the setup into a separate script. For another >> thing, you have to run it twice, and it costs you another pair of reboots, >> too. >> >> There's lots of todo items for test.efs still, and I'm nailing these one >> by one. I do NOT make any PAM changes at all, so authentication is still >> manual, but since the test suite automated this, it's just the human users >> who have to deal with it for now. >> >> Next, I'm going to start extending efs-core and laying the framework for >> the OpenAFS support. One nice thing is that I will be able to use the same >> test.efs setup to test AFS and non-AFS, so I will be able to make sure that >> basic NFS support doesn't break. >> >> I said I'd have the test.efs done by the end of this week -- mission >> accomplished. >> >> By the end of NEXT week, I'm hoping to know a lot more about how efs-core >> has to change in order to support both. >> > > _______________________________________________ > EFS-dev mailing list > [email protected] > http://mailman.openefs.org/mailman/listinfo/efs-dev > > > > _______________________________________________ > EFS-dev mailing list > [email protected] > http://mailman.openefs.org/mailman/listinfo/efs-dev > >
_______________________________________________ EFS-dev mailing list [email protected] http://mailman.openefs.org/mailman/listinfo/efs-dev
