Re: [OpenAFS] client behind NAT firewall
On Wed, 06 Aug 2014 15:33:02 -0400
Dale Pontius pont...@btv.ibm.com wrote:
On 08/05/2014 11:08 AM, Simon Wilkinson wrote:
The complication is that firewalls/NATs only preserve these mappings
for a finite length of time. We attempt to keep them open through
regular fileserver pings, but sometimes that isn't enough. When a
mapping expires, the client is unable to receive callbacks until it
next contacts the fileserver.
...
Obviously this was client side, but I find it hard to believe that
keeping a connection mapped for the 2 hours mentioned elsewhere would be
necessary.
Interestingly enough, the time to live for callbacks varies based on
the number of users. If your fileserver doesn't do pings, I think 2
hours might be a little short for some volumes. From viced/callback.c:
/* Time to live for call backs depends upon number of users of the file.
* TimeOuts is indexed by this number/8 (using TimeOut macro). Times
* in this table are for the workstation; server timeouts, add
* ServerBias */
static int TimeOuts[] = {
/* Note: don't make the first entry larger than 4 hours (see above) */
4 * 60 * 60,/* 0-7 users */
1 * 60 * 60,/* 8-15 users */
30 * 60,/* 16-23 users */
15 * 60,/* 24-31 users */
15 * 60,/* 32-39 users */
10 * 60,/* 40-47 users */
10 * 60,/* 48-55 users */
10 * 60,/* 56-63 users */
}; /* Anything more: MinTimeOut */
/* minimum time given for a call back */
#ifndef INTERPRET_DUMP
static int MinTimeOut = (7 * 60);
#endif
___
OpenAFS-info mailing list
OpenAFS-info@openafs.org
https://lists.openafs.org/mailman/listinfo/openafs-info
Re: [OpenAFS] client behind NAT firewall
On 08/05/2014 11:08 AM, Simon Wilkinson wrote: The complication is that firewalls/NATs only preserve these mappings for a finite length of time. We attempt to keep them open through regular fileserver pings, but sometimes that isn't enough. When a mapping expires, the client is unable to receive callbacks until it next contacts the fileserver. I fiddled with this many years ago, only on the client side. On the advice of a friend I adjusted the connection timeout rules from the default to 15 minutes. I'm under the impression that afs will give up / re-establish after 5 minutes, and 15 minutes was a bit of guardband against that. I was happy enough running the client this way, but shortly after an area move got me into an office with an additional port, and I quite NATting through one box to another. Obviously this was client side, but I find it hard to believe that keeping a connection mapped for the 2 hours mentioned elsewhere would be necessary. -- Dale Pontius Senior Engineer IBM Corporation Phone: (802) 769-6850 Tie-Line: 446-6850 email: pont...@us.ibm.com This e-mail and its attachments, if any, may contain confidential and privileged material for the sole use of the intended recipient. Any review, use, distribution or disclosure by others is strictly prohibited. If you are not the intended recipient (or authorized to receive for the recipient), please contact the sender by reply e-mail and delete all copies of this message from your system without copying it and notify sender of the misdirection by reply e-mail. ___ OpenAFS-info mailing list OpenAFS-info@openafs.org https://lists.openafs.org/mailman/listinfo/openafs-info
Re: [OpenAFS] client behind NAT firewall
On Tue, 2014-08-05 at 09:30 +0200, Alex wrote: Now, I didn't find in the admin guide or wiki[1] some useful information about client's firewall, but I could find some information on the Internet saying that client doesn't work without opening 7001 for incoming UDP [2]. This should be open for callbacks (if I understood correctly). I also tested the client behind NAT with some public cells and it worked well. So, does a client work behind a firewall NAT even without opening inbound ports? If not, is there any solution for this? You will get basic client functionality even without opening the port. What you won't get is notifications from the server that something the server knows to be cached on the client has been modified elsewhere and the client should flush its cached information (this is the callback). In most cases, clients already discard this cached information after some amount of time; additionally, if you are mostly using read-only volumes then the cached information would only be invalidated by a new volume release. In addition, even if you open the port, most NAT implementations don't retain UDP NAT mappings for long enough to be useful for callback breaks (generally their expected use case for UDP is DNS). So you might be able to get by with just running fs checkvolumes periodically in a cron job to make up for missing callback breaks on volume releases. For the most reliable operation, however, you should check that the NAT gateway can remember UDP NAT mappings *specifically on client port 7001* for at least 2 hours and open 7001/udp in the firewall so the client can receive callback breaks. -- brandon s allbery kf8nh sine nomine associates allber...@gmail.com ballb...@sinenomine.net unix openafs kerberos infrastructure xmonadhttp://sinenomine.net :�� T���j)b� b�өzpJ)ߢ�^��좸!��l��b��(���~�+Y���b�ا~�~ȧ~
Re: [OpenAFS] client behind NAT firewall
On 2014-08-05, at 15:08, Brandon Allbery ballb...@sinenomine.net wrote: On Tue, 2014-08-05 at 09:30 +0200, Alex wrote: Now, I didn't find in the admin guide or wiki[1] some useful information about client's firewall, but I could find some information on the Internet saying that client doesn't work without opening 7001 for incoming UDP [2]. This should be open for callbacks (if I understood correctly). I also tested the client behind NAT with some public cells and it worked well. So, does a client work behind a firewall NAT even without opening inbound ports? If not, is there any solution for this? You will get basic client functionality even without opening the port. What you won't get is notifications from the server that something the server knows to be cached on the client has been modified elsewhere and the client should flush its cached information (this is the callback). And what those who modify such content elsewhere get is that they have to wait while the server tries to break the callback. I believe we even have seen file creation fail due to this, although it probably shouldn't. In most cases, clients already discard this cached information after some amount of time; additionally, if you are mostly using read-only volumes then the cached information would only be invalidated by a new volume release. In addition, even if you open the port, most NAT implementations don't retain UDP NAT mappings for long enough to be useful for callback breaks (generally their expected use case for UDP is DNS). So you might be able to get by with just running fs checkvolumes periodically in a cron job to make up for missing callback breaks on volume releases. For the most reliable operation, however, you should check that the NAT gateway can remember UDP NAT mappings *specifically on client port 7001* for at least 2 hours and open 7001/udp in the firewall so the client can receive callback breaks. -- Stephan Wiesand DESY - DV - Platanenallee 6 15738 Zeuthen, Germany ___ OpenAFS-info mailing list OpenAFS-info@openafs.org https://lists.openafs.org/mailman/listinfo/openafs-info
Re: [OpenAFS] client behind NAT firewall
On 2014-08-05, at 9:30, Alex euergetiko...@gmail.com wrote: Please help me to make a decision here. I am trying to determine whether Openafs is the right choice for us and it is not clear for me if modifying client's firewall is mandatory or not. The situation is like this: -all Openafs servers are behind the same NAT firewall. Firewall rules can be changed. I'm not that NAT savvy... how could this possibly work (more than one server)? -client computers are behind another NAT firewall. Firewall rules cannot be changed. -some clients are on Windows, some on Linux. -- Stephan Wiesand DESY - DV - Platanenallee 6 15738 Zeuthen, Germany ___ OpenAFS-info mailing list OpenAFS-info@openafs.org https://lists.openafs.org/mailman/listinfo/openafs-info
Re: [OpenAFS] client behind NAT firewall
On 08/05/14 15:08, Brandon Allbery wrote: On Tue, 2014-08-05 at 09:30 +0200, Alex wrote: Now, I didn't find in the admin guide or wiki[1] some useful information about client's firewall, but I could find some information on the Internet saying that client doesn't work without opening 7001 for incoming UDP [2]. This should be open for callbacks (if I understood correctly). I also tested the client behind NAT with some public cells and it worked well. So, does a client work behind a firewall NAT even without opening inbound ports? If not, is there any solution for this? You will get basic client functionality even without opening the port. What you won't get is notifications from the server that something the server knows to be cached on the client has been modified elsewhere and the client should flush its cached information (this is the callback). In most cases, clients already discard this cached information after some amount of time; additionally, if you are mostly using read-only volumes then the cached information would only be invalidated by a new volume release. In addition, even if you open the port, most NAT implementations don't retain UDP NAT mappings for long enough to be useful for callback breaks (generally their expected use case for UDP is DNS). So you might be able to get by with just running fs checkvolumes periodically in a cron job to make up for missing callback breaks on volume releases. For the most reliable operation, however, you should check that the NAT gateway can remember UDP NAT mappings *specifically on client port 7001* for at least 2 hours and open 7001/udp in the firewall so the client can receive callback breaks. Thank you all for answering, I guess we should test it more carefully to check how it will work. Parallel access is a must for us.The main concern is the possibility that one client overwrites modifications of another one who is editing the file in the same time. cheers, Alex ___ OpenAFS-info mailing list OpenAFS-info@openafs.org https://lists.openafs.org/mailman/listinfo/openafs-info
Re: [OpenAFS] client behind NAT firewall
On Tue, 2014-08-05 at 16:12 +0200, Alex wrote: Parallel access is a must for us.The main concern is the possibility that one client overwrites modifications of another one who is editing the file in the same time. This is going to bite you if you don't have callbacks working. -- brandon s allbery kf8nh sine nomine associates allber...@gmail.com ballb...@sinenomine.net unix openafs kerberos infrastructure xmonadhttp://sinenomine.net :�� T���j)b� b�өzpJ)ߢ�^��좸!��l��b��(���~�+Y���b�ا~�~ȧ~
Re: [OpenAFS] client behind NAT firewall
On 8/5/2014 3:30 AM, Alex wrote: Hi all, Please help me to make a decision here. I am trying to determine whether Openafs is the right choice for us and it is not clear for me if modifying client's firewall is mandatory or not. The situation is like this: -all Openafs servers are behind the same NAT firewall. Firewall rules can be changed. How many OpenAFS servers and how many public IP addresses on the NAT? Can you provide static mappings for the servers? OpenAFS services are required to be visible on explicit port numbers file: 7000/udp protection: 7002/udp volume location: 7003/udp volume mgmt 7005/udp at the very least the file and volume location services must be visible to clients. If there are more than one instance of each AFS service and the NAT only supports one public IP address, then you will have a problem because only one instance of each service can be accessed by clients. There will be other issues regarding volume management and there are likely to be delays when clients attempt to access data if the service's private addresses are registered for the file servers with the location service. It is possible to run OpenAFS services from behind a NAT. The your-file-system.com cell has many db and file service instances behind NATs with static public/private address mappings. All of the service instances were originally OpenAFS but are now YFS. The YFS tool chain does a much better job of support volume management across NAT boundaries. -client computers are behind another NAT firewall. Firewall rules cannot be changed. Clients by default use port 7001/udp to communicate with db and file servers. In AFS, the caching coherency model is built upon a push messaging model. File servers provide the client's cache manager some data with a lease expiration time and promises to notify the cache manager if the data is going to change before the lease expires. However, the file server is not particular about the port number the client uses. If the port is mapped from 7001/udp to 56039/udp that is fine provided the NAT maintains the port mapping while traffic continues to flow periodically. The IETF recommends sending udp keep alive packets at least every 30 seconds to keep NAT port mappings open but not all NAT devices do so. Some NAT devices explicitly do not do so for security reasons. In particular, they might cap the number of inbound UDP packets received in response. These devices are broken and will fail to work with any number of other udp based streaming media protocols. The cache managers will probe known file servers every ten minutes when the file system is idle. Configuring the NAT to maintain the port mappings for up to 15 minutes is recommended. The cache managers (in particular the Windows CMs) work very hard to ensure that lost connections due to NATs do not prevent access to data or corrupt the cache. Windows applications (unlike UNIX) almost always maintain file locks whose leases must be extended every five minutes. The Windows CM is more likely to notice and be forced to recover from a temporary connectivity loss. As Stephan indicated, an inability for the file server to notify cache managers of changes has a larger impact on clients attempting to change files for which active leases (callback promises) have been issued. The clients make the change must wait for the callback delivery attempt to timeout before the file server can complete the request. The file server thread assigned to that RPC will be blocked and idle until the timeout completes. If there are many threads in such as state, the file server will be unable to accept new RPCs until the backlog clears. It is also worth noting that the OpenAFS file servers have to perform extra work when there are many clients accessing the file server from the same IP address and different port numbers. The file server has a harder time determining whether a client has moved port numbers or is in fact a unique client. Whether OpenAFS works for you is going to be somewhat dependent upon the workload and the specific behavior of the NAT devices involved. -some clients are on Windows, some on Linux. smime.p7s Description: S/MIME Cryptographic Signature
Re: [OpenAFS] client behind NAT firewall
On 5 Aug 2014, at 14:08, Brandon Allbery ballb...@sinenomine.net wrote: On Tue, 2014-08-05 at 09:30 +0200, Alex wrote: Now, I didn't find in the admin guide or wiki[1] some useful information about client's firewall, but I could find some information on the Internet saying that client doesn't work without opening 7001 for incoming UDP [2]. This should be open for callbacks (if I understood correctly). I also tested the client behind NAT with some public cells and it worked well. So, does a client work behind a firewall NAT even without opening inbound ports? If not, is there any solution for this? You will get basic client functionality even without opening the port. What you won't get is notifications from the server that something the server knows to be cached on the client has been modified elsewhere and the client should flush its cached information (this is the callback). This isn't actually the case, because of the complexities of imposing connections on top of UDP. Because UDP is a connectionless protocol, a firewall that blocked all inbound UDP packets would prevent any two way UDP conversations from occurring. Instead what firewalls and NATs do is open a hole/mapping for inbound packets that matches the path taken by the outbound ones. Because the cache manager listens for incoming callback breaks on the same port as it uses to send packets to the fileserver, this means that a firewall hole for callbacks is created as a side effect of the original connection. This is how AFS continues to work in the face of the Mac OS X firewall, for example The complication is that firewalls/NATs only preserve these mappings for a finite length of time. We attempt to keep them open through regular fileserver pings, but sometimes that isn't enough. When a mapping expires, the client is unable to receive callbacks until it next contacts the fileserver. Cheers, Simon___ OpenAFS-info mailing list OpenAFS-info@openafs.org https://lists.openafs.org/mailman/listinfo/openafs-info
Re: [OpenAFS] client behind NAT firewall
On Tue, 2014-08-05 at 16:08 +0100, Simon Wilkinson wrote: The complication is that firewalls/NATs only preserve these mappings for a finite length of time. We attempt to keep them open through regular fileserver pings, but sometimes that isn't enough. When a mapping expires, the client is unable to receive callbacks until it next contacts the fileserver. Perhaps you should have read the following paragraph -- brandon s allbery kf8nh sine nomine associates allber...@gmail.com ballb...@sinenomine.net unix openafs kerberos infrastructure xmonadhttp://sinenomine.net
Re: [OpenAFS] client behind NAT firewall
On 5 Aug 2014, at 16:09, Brandon Allbery ballb...@sinenomine.net wrote: On Tue, 2014-08-05 at 16:08 +0100, Simon Wilkinson wrote: The complication is that firewalls/NATs only preserve these mappings for a finite length of time. We attempt to keep them open through regular fileserver pings, but sometimes that isn't enough. When a mapping expires, the client is unable to receive callbacks until it next contacts the fileserver. Perhaps you should have read the following paragraph I did. You talked about opening firewall holes, which isn't necessary for callback traffic. S.___ OpenAFS-info mailing list OpenAFS-info@openafs.org https://lists.openafs.org/mailman/listinfo/openafs-info
Re: [OpenAFS] client behind NAT firewall
On 08/05/14 16:55, Jeffrey Altman wrote: -all Openafs servers are behind the same NAT firewall. Firewall rules can be changed. How many OpenAFS servers and how many public IP addresses on the NAT? to simplify for now, and for testing, we will use just one machine with one IP behind the firewall, as our main concern is how clients behave on parallel access. Can you provide static mappings for the servers? OpenAFS services are required to be visible on explicit port numbers file: 7000/udp protection: 7002/udp volume location:7003/udp volume mgmt 7005/udp at the very least the file and volume location services must be visible to clients. If there are more than one instance of each AFS service and the NAT only supports one public IP address, then you will have a problem because only one instance of each service can be accessed by clients. Is it possible to have one public IP and two different static routes, for instance 7003/ goes to volume location database 7000/udp goes to file server. How BOS server will behave in that case? Alex ___ OpenAFS-info mailing list OpenAFS-info@openafs.org https://lists.openafs.org/mailman/listinfo/openafs-info
