On 8/26/2016 9:43 AM, Karl-Philipp Richter wrote: > > Are there any plans to use name resolution in OpenAFS? It's a major > technology that exists for decades and for a reason. It'd make all our > lives much easier.
Kalle, There are two types of servers that the AFS cache manager communicates with. The first is the Volume Location servers (port 7003) whose endpoints are determined by querying the client's CellServDB file. The CellServDB file format is (approximately): >cellname optional-linked-cellname # Description of Cell ipv4-addr-of-vlserver # DNS-hostname-of-vlserver ... >nextcell ... .... Those not familiar with the CellServDB format mistakenly assume that the '#' is a comment. It is not, its a mandatory field separator. The list of server records (ipv4-addr # dns-hostname) is optional and if the AFS cache manager is started with the -afsdb command-line parameter then cells which cannot be resolved from the CellServDB file will be resolved via a combination of * DNS SRV records or DNS AFSDB records * DNS A records It is important to note that the behavior of the OpenAFS *nix cache manager and the Windows cache manager and Arla differ. For example, whereas the OpenAFS *nix cache manager only uses the ipv4 addresses from the CellServDB file, the Windows cache manager always performs DNS A or CNAME lookups on the DNS-hostname. Whereas the *nix cache manager, only resolves volume location servers for a cell at startup, the Windows cache manager tracks the DNS TTL and applies a configurable TTL to the CellServDB entries. Part of the reason for the difference in behavior in this regard is because the *nix kernel module is not the entity that reads the CellServDB file or performs DNS lookups. The userland "afsd" process parses the CellServDB info and pushes the results into the kernel. Similarly to how an end-user can do so using the "fs newcell" command. The volume location servers maintain a database, VLDB, which stores the mapping of volume names and numeric identifiers to locations (file servers and partitions.) It also maintains a record of file server identifiers and the list of ipv4 addresses on which the file server can be contacted. This latter list is updated each time the file server restarts which is why it is necessary for the file server to be restarted each time its ipv4 address configuration changes. Public NAT addresses are registered by a file server by including it in the NetInfo file on the file server. Private NAT addresses are excluded from the registration by including them in the NetRestrict file. Each time the file server registration alters the contents of the matching VLDB record, a counter for that file server is incremented. This counter combined with the file server identifier are delivered to cache managers as the location of a volume. The Volume Location service also provides an RPC that permits cache managers to query the set of ipv4 addresses associated with a given file server identifier and counter. It is this mechanism that permits the AFS cache manager to cache the locations of file servers. The underlying issues that you are facing are: 1. Each time your public NAT address changes you have to update each of the server CellServDB and NetInfo files and restart the affected servers. If one of those servers is a Ubik server (VLDB, PRDB, etc.) then it is critical that the CellServDB on each of the servers be updated to include the updated address. The cell must then be restarted. For Ubik it is critical that all of the servers in a quorum see the same set of ipv4 addresses for all of the servers in the quorum. It is not safe to rely upon the DNS responses that might be altered by a NAT gateway and if DNS-SEC is used the NAT could not alter them. If the cell uses DNS records SRV or AFSDB to distribute the location of the Ubik database servers to clients, then those records must be updated. File servers contact the Volume Location and Protection servers and do so by using the ipv4 addresses in the server CellServDB. Servers in OpenAFS never use the DNS hostnames to contact one another. 2. Each time your public NAT address for a VL server changes you have to do one of the following on each *nix cache manager: a. update CellServDB and restart b. "fs newcell" the updated VL server info for the cell Knowing that the IP addresses of your cell are unstable you can automate these operations using scripts that execute periodically to check the public NAT address. However, even if you have such scripts in place there is a strong likelihood that there will be outages since DNS resolvers are permitted to cache records for the full TTL of the response. Therefore, it is possible that the automated script would fail to notice the address change for up to a TTL. If OpenAFS supported IPv6 (see Simon Wilkinson's talk "The Road to IPv6" from the 2015 Best Practices Workshop) http://workshop.openafs.org/afsbpw15/talks/thursday/wilkinson-road-to-afs.pdf you could obtain your own IPv6 tunnel from Hurricane Electric and avoid the IPv4 NAT nonsense when IPv6 is available to your client systems. Of course, OpenAFS doesn't. Alternatives include requiring the use of a VPN to access the cell and ensuring that only private addresses are published for cell servers. As indicated above, DNS can be used by clients to locate the database services: Volume Location, Protection, etc. DNS cannot be used by servers to locate other servers. Nor can DNS be used by clients to locate file servers. When AuriStor added IPv6 support to its product one of the goals was support for 6to4 gateways such as Microsoft Direct Access. These gateways rely upon DNS queries to obtain a AAAA record for a host that only has an IPv4 address. The AFS3 model of embedding IPv4 addresses in the VL_GetAddrsU RPC response posed a challenge because an IPv6 only client must implement the necessary logic to identify the correct ipv6 prefix to use in converting the private IPv4 address to an IPv6 address that will be routed through the 6to4 gateway. Other protocols such as FTP that embed addresses in the control stream have similar problems. Although it might be tempting to extend the VL RPCs to permit registration of file servers by DNS host name instead of by IP address, a better approach might be to add split horizon support to the VL service such that file servers can register both the public and private addresses with the VL service but the VL service would only disclose one based upon which address the client's query is received from. It would not be safe to convert the Ubik servers to rely upon DNS host names. Not with the current Ubik protocol specification. Jeffrey Altman
<<attachment: jaltman.vcf>>
smime.p7s
Description: S/MIME Cryptographic Signature
