Replies in-line: > general advice: > > - make sure the network connectivity between your three AFS "database > servers" is always up...they depend on the network to communicate with > each other, and if they are always up and always reliable, it will > enhance the perceived performance of afs
Undertood. The links are pretty stable. > - if most of the users do mostly reading, and mostly smallish files, you > might get by using volume replication at each site, leaving the RW > volume wherever it is, and relying on some tricks to keep the user > confused during writes...i think, in general, the idea of moving RW > volumes all the time is a bad one...it will put big loads on your > fileservers if you have lots of users For the most part, yes. We try to keep the large stuff (databases) separately and have the db apps take care of synchronization in their own way. I thought as much, but I'm getting very poor performance even for reading. It seems that the clients don't do a good job of prioritizing the local fileserver first. Some posts mention lower-numbered IP addresses getting priority, but I'm not sure how that would work for servers on different subnets. Shouldn't the choice have something to do with lower latency? > - check the top of your afs tree and make sure root.afs, root.cell and > probably most other volumes at the top of the tree contain only > mountpoints, and are all replicated, and don't change them much...in > other words, you want stability at the top of the tree, to reduce the > changes that clients have to pay attention to These are all mounted R/O, right? Should mostly-static volumes be mounted R/O and then only access the R/W vol through /afs/.domain/ for the occasional updates? What about for the software repository scenario below? The way I understand the volume access is that even if a volume is mounted R/W, its R/O vol is accessed until a write operation, in which case all further access (including reads) are from the R/W vol. If so, then a user should ideally be able to use the local server for most reads. We haven't made any changes to the default cache options, so maybe this would be a good next step? > - do your vos operations, backups, etc. in a coordinated way so you can > predict when they are happening, and don't end up hosing your servers by > doing multiple simulataneous operations on the same volumes How do I know when to release? Do people do this automagically with scripting? With what triggers / schedules? > how many users? what kinds of files and what kinds of usage? how much > sharing? these can make a difference in how you set things up, too. (i > worked on a cell once with clients in sweden, south america, and on the > european continent...one server at each site, whenever the south > american site became master db server, followed by transatlantic link > going down...the european users weren't happy....basically, as long as > you have good network connectivity between client systems and server > systems, there is no need for physical proximity...) We've got about 5 users, so a pretty small setup. They're mostly very non-technical, and expect it to Just Work (tm). The files regular users are concerned with are mostly documents, images, and presentations. Their heavy sharing is usually confined to a small set of small files (usually less that 1MB each); sets change as projects change (weekly/monthly). Still, it takes about 1 minute to transfer 1MB across the VPN -- not insignificant. -Chaz > Chaz Chandler wrote: >> Hello all! >> >> I am attempting to implement OpenAFS across a VPN with limited bandwidth >> between sites but relatively mobile users who expect to have their data >> available (and writable) at whichever site they are currently located. >> >> The issue I am running up against is how to organize the AFS volume >> structure so that things like user home dirs, collaboration/group dirs, >> and writable system items (like a Windows profile, for instance) are >> optimally maintained in AFS. >> >> The set-up is: >> 1) Each site has an OpenAFS server (each with vl, pt, bu, vol, >> fileservers & salvager). Currently, v1.4.8 on Linux 2.6. >> 2) Clients computers are a mix of Windows XP, OpenBSD, and Linux. 1.5 >> clients for windows, 1.4 clients for Linux, and native openbsd clients. >> 3) All sites are connected in a full mesh VPN (max of about 30KB/s for >> each link) >> 4) There's about 600GB of data at the moment. Although most of it >> doesn't need to be writable most of the time, things that are frequently >> written are not currently segregated from static or infrequently-written >> files/dirs. Perhaps only a few gigs change on a weekly basis. >> 5) Users move from site to site, but once there usually spend several >> weeks. However, two sites are physically very close and users move >> between them more frequently (sometimes daily) although the link >> bandwidth is the same as the others. >> 6) We have a pretty standard AFS volume layout: separate volumes for >> each user, a few large volumes with relatively static content, a few >> volumes for groups to share. >> 7) Currently, volume releases are done manually. >> 8) When a user changes locations for a long stretch, we move their R/W >> user volume to the new location (electronically, not physically), a >> process which is labor- and time-intensive and usually has at least one >> snafu along the way. >> 9) We have been unable to come up with a working implementation of >> roaming windows profiles on AFS. >> >> I'm seeking recommendations on: >> 1) How others have set up a regular release schedule to keep a large >> amount of data synced over a slow network (custom scripts, I assume, but >> is there a repository of these things and what are the general mechanics >> and best practices here?) >> 2) What sort of volume layout would one recommend, and how should >> frequently-updated data be stored? Take, for instance, three examples: >> - A software repository: large volume with relatively static contents, >> occasionally has large additions or subtractions when a new piece of >> software is added or an old one removed. Ideally, these updates should >> be able to be accomplished from any location. Users don't need to write >> to it, but may need to read from it frequently at LAN speeds. >> - A collaboration dir: several users read and write a small amount (10s >> of MBs) on a daily basis from different locations simultaneously, but >> they expect LAN-type performance. >> - A user dir: large amounts of data updated from a single location, but >> user may move to any other site at any time, potentially with up to a >> day of transit time in which a volume could be moved to the destination >> site. >> 3) Any concrete recommendations on how to properly implement windows >> integration with AFS (especially folder redirection and roaming profiles >> on AFS). Yes, I've read the '04 and '05 best practices, however they >> are now quite old and did not work for me. >> >> I've been lurking on this list for a while now and have come to the >> conclusion that while there are a few very knowledgeable and experienced >> folks in the AFS community, there are not any good, current, and >> comprehensive AFS information repositories out there. The list archives >> are the best option, but I find them almost impossible to use unless I >> know the exact phrase I'm looking for. Is there something I'm missing? >> >> Cheers, >> -Chaz >> _______________________________________________ >> OpenAFS-info mailing list >> [email protected] >> https://lists.openafs.org/mailman/listinfo/openafs-info >> > > _______________________________________________ > OpenAFS-info mailing list > [email protected] > https://lists.openafs.org/mailman/listinfo/openafs-info _______________________________________________ OpenAFS-info mailing list [email protected] https://lists.openafs.org/mailman/listinfo/openafs-info
