Re: Does Tomcat/Java get around the problem of 64K maximum client source ports?
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Eric, On 3/27/20 16:39, Eric Robinson wrote: > Thanks for all the feedback, André, Christopher, and John. Let me > see if I can quickly answer everyone's comments.> Since there is a > TCB for each connection, and the OS knows which TCBs are associated > with which processes, I don't see any problem using the same local > port on different sockets. When a packet arrives from a remote > server, the stack looks at the full socket details, checks for a > matching TCB, and routes the packet to the appropriate process. > There's no confusion (except when using tools that don't show > process names, like netstat without -p).> Using > 64K local source > ports seems like a useful capability in high-load environments > where tomcat is doing a lot of back-end access (i.e., where JSPs > and class files frequently call back-end services). With hashing > and indexing, having giant connection tables does not seem like an > unrealistic amount of processing load to me. Linux-based stateful > firewalls have to keep track of a lot more connections than that, > with rule processing and even layer-7 inspection at the same time, > on relatively low-powered hardware. The likelihood that you end up with more than one connection with identical source addr:port + dest addr+port is quite high in this situation. You have to be VERY careful that you don't overlap or you'll find that you have mass chaos on your hands. > FYI, I don't have 1800 tomcat instances on one server. I have > about 100 instances on each of 18 servers. So is this just of academic interest to you, then? 100 client connections per host doesn't seem like you have a problem to solve. > That said, I agree that the real focus should probably be on the > JDBC driver. I asked the question here because it seemed like a > good place to start. Any ideas where I could go to chat with JDBC > developers? MySQL is open-source. They have forums, mailing lists, a public bug-tracker, etc. You can also download the source and read through it to find out how things are being done. - -chris >> -Original Message- From: Christopher Schultz >> Sent: Friday, March 27, 2020 1:42 >> PM To: users@tomcat.apache.org Subject: Re: Does Tomcat/Java get >> around the problem of 64K maximum client source ports? >> > André, > > On 3/27/20 11:01, André Warnier (tomcat/perl) wrote: >>>> On 27.03.2020 14:27, André Warnier (tomcat/perl) wrote: >>>>> On 26.03.2020 20:42, Eric Robinson wrote: >>>>>>> -Original Message- From: Olaf Kock >>>>>>> Sent: Thursday, March 26, 2020 >>>>>>> 2:06 PM To: users@tomcat.apache.org Subject: Re: Does >>>>>>> Tomcat/Java get around the problem of 64K maximum >>>>>>> client source ports? >>>>>>> >>>>>>> Hi Eric, >>>>>>> >>>>>>> On 26.03.20 18:58, Eric Robinson wrote: >>>>>>>> Greetings, >>>>>>>> >>>>>>>> Many people say the maximum number of client ports is >>>>>>>> 64K. However, >>>>>>> TCP connections only require unique sockets, which are >>>>>>> defined as... >>>>>>>> >>>>>>>> local_IP:local_port -> remote_ip:remote_port >>>>>>>> >>>>>>>> Theoretically, it is possible for a client process to >>>>>>>> keep using the same local >>>>>>> source port, as long as the connections are to a >>>>>>> unique destinations. For example on a local machine, >>>>>>> the following connections should be possible... >>>>>>>> >>>>>>>> 192.168.5.100:1400 -> 192.168.5.200:3306 >>>>>>>> 192.168.5.100:1400 -> 192.168.5.201:3306 >>>>>>>> 192.168.5.100:1400 -> 192.168.5.202:3306 >>>>>>>> 192.168.5.100:1400 -> 192.168.5.203:3306 >>>>>>>> >>>>>>>> I've seen this demonstrated successfully here: >>>>>>>> >>>>>>>> https://serverfault.com/questions/326819/does-the-tcp-source-po rt- > >>>>>>>> have >>>>>>>> >>>>>>>> > -to-be-unique-per-host >>>>>>>> >>>>>>>> As someone on that page pointed out, while it is >>>>>>>> possible, it does not >>>>>>>
RE: Does Tomcat/Java get around the problem of 64K maximum client source ports?
> -Original Message- > From: André Warnier (tomcat/perl) > Sent: Saturday, March 28, 2020 5:35 PM > To: users@tomcat.apache.org > Subject: Re: Does Tomcat/Java get around the problem of 64K maximum > client source ports? > > On 27.03.2020 21:39, Eric Robinson wrote: > > FYI, I don't have 1800 tomcat instances on one server. I have about 100 > instances on each of 18 servers. > > When one of these (attempted) connections fails, do you not get some error > message which gives a clue as to what the failure is due to ? > (should be a log somewhere, no ?) > Hi André -- Yes, it does log a connection failure message. It's been a while since the last time it happened so I don't recall the exact wording of the error, but the gist of it is that it could create a TCP connection. > Also, just for info : > in the past, I have run into problems under Linux (no more connections > accepted, neither incoming nor outgoing) whenever the actual number of > TCP connections went above a certain number (maybe it was 64K). > A TCP connection goes through a number of states (which you see with a > netstat display), such as "ESTABLISHED" but also "TIME_WAIT", > "CLOSE_WAIT" etc.. In some of these states, the connection no longer has > any link to any process, but the connection still counts against the limit (of > the OS/TCP stack). > > The case I'm talking about was a bit like yours : a webapp running under > tomcat was making a connection to a remote host, but this connection was > wrapped inside an object of some kind. When the webapp no longer needed > the connection, it just discarded the wrapping object, which was left without > references to it, and thus candidate for destruction at some point. But the > discarded object never explicitly closed the underlying connection. > > Over a period of time, this left an accumulation of (no longer used) > connections in the "CLOSE_WAIT" state (closed by the remote host side, but > not by the webapp side), which just sat there until a GC happened, at which > time the destruction of these objects really happened, and some implicit > close was done at the OS level, which eliminated these pending underlying > CLOSE_WAIT connections. > And since the available heap was quite large, it took a long time before a GC > happened, which allowed such CLOSE_WAIT connections to accumulate in > the hundreds or thousands before being "recycled". > Until a certain number was reached, and then the host became all but > unreachable and very slow. > That was a long time ago, and thus a lot of Java versions and Linux versions > ago, so maybe something happened since then to avoid such a situation. > But maybe also, you are suffering of some similar phenomenon. > You could try to use netstat some more, and when you are having the > problem, you should count at ALL the TCP connections, including the ones in > CLOSE_WAIT, and just check if you do not have an obscene number of them > in total. There is definitely some limit number past which the OS starts > acting > funny. > > (Note : unlike for TIME_WAIT e.g., there is no time limit for a connection in > the CLOSE_WAIT state; it will stay in that state as long as the client side > has > not explicitly closed it, in some kind of zombie half-life) See e.g. : > https://users.cs.northwestern.edu/~agupta/cs340/project2/TCPIP_State_Tr > ansition_Diagram.pdf > I'm familiar with the issue you described above. In the past, we addressed it by decreasing the TIME_WAIT timer and/or by enabling TCP reuse. That helps some. > > > - > To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org > For additional commands, e-mail: users-h...@tomcat.apache.org Disclaimer : This email and any files transmitted with it are confidential and intended solely for intended recipients. If you are not the named addressee you should not disseminate, distribute, copy or alter this email. Any views or opinions presented in this email are solely those of the author and might not represent those of Physician Select Management. Warning: Although Physician Select Management has taken reasonable precautions to ensure no viruses are present in this email, the company cannot accept responsibility for any loss or damage arising from the use of this email or attachments.
Re: Does Tomcat/Java get around the problem of 64K maximum client source ports?
On 27.03.2020 21:39, Eric Robinson wrote: FYI, I don't have 1800 tomcat instances on one server. I have about 100 instances on each of 18 servers. When one of these (attempted) connections fails, do you not get some error message which gives a clue as to what the failure is due to ? (should be a log somewhere, no ?) Also, just for info : in the past, I have run into problems under Linux (no more connections accepted, neither incoming nor outgoing) whenever the actual number of TCP connections went above a certain number (maybe it was 64K). A TCP connection goes through a number of states (which you see with a netstat display), such as "ESTABLISHED" but also "TIME_WAIT", "CLOSE_WAIT" etc.. In some of these states, the connection no longer has any link to any process, but the connection still counts against the limit (of the OS/TCP stack). The case I'm talking about was a bit like yours : a webapp running under tomcat was making a connection to a remote host, but this connection was wrapped inside an object of some kind. When the webapp no longer needed the connection, it just discarded the wrapping object, which was left without references to it, and thus candidate for destruction at some point. But the discarded object never explicitly closed the underlying connection. Over a period of time, this left an accumulation of (no longer used) connections in the "CLOSE_WAIT" state (closed by the remote host side, but not by the webapp side), which just sat there until a GC happened, at which time the destruction of these objects really happened, and some implicit close was done at the OS level, which eliminated these pending underlying CLOSE_WAIT connections. And since the available heap was quite large, it took a long time before a GC happened, which allowed such CLOSE_WAIT connections to accumulate in the hundreds or thousands before being "recycled". Until a certain number was reached, and then the host became all but unreachable and very slow. That was a long time ago, and thus a lot of Java versions and Linux versions ago, so maybe something happened since then to avoid such a situation. But maybe also, you are suffering of some similar phenomenon. You could try to use netstat some more, and when you are having the problem, you should count at ALL the TCP connections, including the ones in CLOSE_WAIT, and just check if you do not have an obscene number of them in total. There is definitely some limit number past which the OS starts acting funny. (Note : unlike for TIME_WAIT e.g., there is no time limit for a connection in the CLOSE_WAIT state; it will stay in that state as long as the client side has not explicitly closed it, in some kind of zombie half-life) See e.g. : https://users.cs.northwestern.edu/~agupta/cs340/project2/TCPIP_State_Transition_Diagram.pdf - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
RE: Does Tomcat/Java get around the problem of 64K maximum client source ports?
Thanks for all the feedback, André, Christopher, and John. Let me see if I can quickly answer everyone's comments. Since there is a TCB for each connection, and the OS knows which TCBs are associated with which processes, I don't see any problem using the same local port on different sockets. When a packet arrives from a remote server, the stack looks at the full socket details, checks for a matching TCB, and routes the packet to the appropriate process. There's no confusion (except when using tools that don't show process names, like netstat without -p). Using > 64K local source ports seems like a useful capability in high-load environments where tomcat is doing a lot of back-end access (i.e., where JSPs and class files frequently call back-end services). With hashing and indexing, having giant connection tables does not seem like an unrealistic amount of processing load to me. Linux-based stateful firewalls have to keep track of a lot more connections than that, with rule processing and even layer-7 inspection at the same time, on relatively low-powered hardware. FYI, I don't have 1800 tomcat instances on one server. I have about 100 instances on each of 18 servers. That said, I agree that the real focus should probably be on the JDBC driver. I asked the question here because it seemed like a good place to start. Any ideas where I could go to chat with JDBC developers? --Eric > -Original Message- > From: Christopher Schultz > Sent: Friday, March 27, 2020 1:42 PM > To: users@tomcat.apache.org > Subject: Re: Does Tomcat/Java get around the problem of 64K maximum > client source ports? > > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA256 > > André, > > On 3/27/20 11:01, André Warnier (tomcat/perl) wrote: > > On 27.03.2020 14:27, André Warnier (tomcat/perl) wrote: > >> On 26.03.2020 20:42, Eric Robinson wrote: > >>>> -Original Message- From: Olaf Kock > >>>> Sent: Thursday, March 26, 2020 2:06 PM > >>>> To: users@tomcat.apache.org Subject: Re: Does Tomcat/Java get > >>>> around the problem of 64K maximum client source ports? > >>>> > >>>> Hi Eric, > >>>> > >>>> On 26.03.20 18:58, Eric Robinson wrote: > >>>>> Greetings, > >>>>> > >>>>> Many people say the maximum number of client ports is 64K. > >>>>> However, > >>>> TCP connections only require unique sockets, which are defined > >>>> as... > >>>>> > >>>>> local_IP:local_port -> remote_ip:remote_port > >>>>> > >>>>> Theoretically, it is possible for a client process to keep using > >>>>> the same local > >>>> source port, as long as the connections are to a unique > >>>> destinations. For example on a local machine, the following > >>>> connections should be possible... > >>>>> > >>>>> 192.168.5.100:1400 -> 192.168.5.200:3306 192.168.5.100:1400 > >>>>> -> 192.168.5.201:3306 192.168.5.100:1400 -> > >>>>> 192.168.5.202:3306 192.168.5.100:1400 -> > >>>>> 192.168.5.203:3306 > >>>>> > >>>>> I've seen this demonstrated successfully here: > >>>>> > >>>>> https://serverfault.com/questions/326819/does-the-tcp-source-port- > have > >>>>> > >>>>> > - -to-be-unique-per-host > >>>>> > >>>>> As someone on that page pointed out, while it is possible, it does > >>>>> not > >>>> commonly occur in practice "because most TCP APIs don't provide a > >>>> way to create more than one connection with the same source port, > >>>> unless they have different source IP addresses." This leads to the > >>>> 64K maximum client port range, but it is really a limitation of the > >>>> APIs, not TCP. > >>>>> > >>>>> So how does tomcat handle things? Is it limited to a maximum 64K > >>>>> client > >>>> source ports, or is it 64K per destination, as it should be? > >>>> > >>>> To be honest, I can't remember to have seen a web- or application > >>>> server that accepts 64K concurrent connections at all, let alone to > >>>> a single client. > >>>> > >>>> I can't come up with any reason to do so, I'd assume that there's a > >>>> DOS attack if I get 100 concurrent incoming
Re: Does Tomcat/Java get around the problem of 64K maximum client source ports?
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 André, On 3/27/20 11:01, André Warnier (tomcat/perl) wrote: > On 27.03.2020 14:27, André Warnier (tomcat/perl) wrote: >> On 26.03.2020 20:42, Eric Robinson wrote: >>>> -Original Message- From: Olaf Kock >>>> Sent: Thursday, March 26, 2020 2:06 PM >>>> To: users@tomcat.apache.org Subject: Re: Does Tomcat/Java get >>>> around the problem of 64K maximum client source ports? >>>> >>>> Hi Eric, >>>> >>>> On 26.03.20 18:58, Eric Robinson wrote: >>>>> Greetings, >>>>> >>>>> Many people say the maximum number of client ports is 64K. >>>>> However, >>>> TCP connections only require unique sockets, which are >>>> defined as... >>>>> >>>>> local_IP:local_port -> remote_ip:remote_port >>>>> >>>>> Theoretically, it is possible for a client process to keep >>>>> using the same local >>>> source port, as long as the connections are to a unique >>>> destinations. For example on a local machine, the following >>>> connections should be possible... >>>>> >>>>> 192.168.5.100:1400 -> 192.168.5.200:3306 192.168.5.100:1400 >>>>> -> 192.168.5.201:3306 192.168.5.100:1400 -> >>>>> 192.168.5.202:3306 192.168.5.100:1400 -> >>>>> 192.168.5.203:3306 >>>>> >>>>> I've seen this demonstrated successfully here: >>>>> >>>>> https://serverfault.com/questions/326819/does-the-tcp-source-port- have >>>>> >>>>> - -to-be-unique-per-host >>>>> >>>>> As someone on that page pointed out, while it is possible, >>>>> it does not >>>> commonly occur in practice "because most TCP APIs don't >>>> provide a way to create more than one connection with the >>>> same source port, unless they have different source IP >>>> addresses." This leads to the 64K maximum client port range, >>>> but it is really a limitation of the APIs, not TCP. >>>>> >>>>> So how does tomcat handle things? Is it limited to a >>>>> maximum 64K client >>>> source ports, or is it 64K per destination, as it should be? >>>> >>>> To be honest, I can't remember to have seen a web- or >>>> application server that accepts 64K concurrent connections at >>>> all, let alone to a single client. >>>> >>>> I can't come up with any reason to do so, I'd assume that >>>> there's a DOS attack if I get 100 concurrent incoming >>>> connections from a single IP, and a programming error if the >>>> server sets up more than 1K outgoing connections >>>> >>>> Maybe I'm missing the obvious, or have only administered >>>> meaningless installations, but I fail to see the real world >>>> relevance of this question. >>>> >>>> >>> >>> I don't blame you for being puzzled, but this not about tomcat >>> accepting connections. It's about tomcat acting as the client, >>> where MySQL is the server. I'm referring to client connections >>> from tomcat to MySQL. We have about 1800 instances of tomcat >>> running. This question comes up once in a while when tomcat >>> can't connect to MySQL. Trust me, it can be an issue. >>> >>> --Eric >>> >> >> The question is : is there even any Java API (method) (or even OS >> API) which allows a client to open a (client, non-LISTEN) socket >> AND specify the client IP address and/or port ? >> >> I mean, if there is none, then the question may be interesting in >> the absolute, but ultimately pointless. >> >> I believe that the IP address of client packets, will be >> determined by which /route/ the OS determines that the target >> server address can be reached (which will determine through which >> network interface the packets "go out", which will determine the >> sender IP address inserted in the packets). I don't think that >> the application-level software (here a java webapp) can determine >> this in advance. And the client port will be decided by the >> OS-level TCP stack, in function of which ones are not yet in use >> (which a java webapp can also not determine in advance). >> >> Example of creating a client socket : Socket echoSocke
Re: Does Tomcat/Java get around the problem of 64K maximum client source ports?
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 John, On 3/27/20 13:37, john.e.gr...@wellsfargo.com.INVALID wrote: > A few random thoughts. > > First, this is really a MySQL driver thing, not a Tomcat thing. > If you want to know why the driver does what it does, look at the > driver source. There is actually a Socket constructor that allows > you to specify the local port. I don't know why you would do that, > but it's there. > > Second, it does appear to be possible for there to be multiple > connections using the same source port as long as they go to > different destinations. Just read through the various links that > people have provided. Honestly it blows my mind that that's even > possible, but that just shows how little I know about networking. > How does the client OS route return packets correctly if they have > the same source port? Because the connection is defined by *both* endpoints in a TCP connection. The client knows its own source port and the destination port as well. Imagine two clients: saddr sport daddr sport ::1 1400 x 3306 ::1 1400 y 3306 All is well, there: the OS knows where to route data returning to ::1/1400/y/3306 distinctly from ::1/1400/x/3306. But if you have two clients wanting to connect to the same destination (e.g. x:3306) then they must have distinct source ports. This is why most clients just use a random OS-assigned port. - -chris -BEGIN PGP SIGNATURE- Comment: Using GnuPG with Thunderbird - https://www.enigmail.net/ iQIzBAEBCAAdFiEEMmKgYcQvxMe7tcJcHPApP6U8pFgFAl5+R5QACgkQHPApP6U8 pFjLsQ/+IdSoJIPFhNfLmQnxEKEQA13Arr9vUE4A+83uh2+tHmo0nwW9VBzWYRLT hEciA++PSBKBDimaeNrjQBmNEmMfLnfWBZfc0TjDCAUEYvvBXslYyw9CySya3ISC d79hCJj55oAiua+eJ7ZN/MyRgeldwF/3CsP6lwpoH/rR9KI8/oKqt4vV64a7BuTW gSIlzKJ5BQCtGZIDZmf7PeL5d48GJnw0jdREFC8Bf8NYjS/uj8Cccy8XH7zPcraP S8krhrqemH8e6UQE3XID+qyatkHXBo5AVC16soQ0+jwDp0oxCsV0Faxk+8KollKL aWti+di9DOQkKSYHq0IgyE82sxxm8FL5vAKABpE/i9864PqwBBv9hIfk4Lyiqirx zuI3kVEEixcgVnv6Wdrmj1oGdhIcV6kHD/cXlTgMRTcGLBqrMFFfj4YdgUUHwOId ZaL1UMlH0sB9NiZtSwxnEFslHbqV6dYZjpWdBnTRjiYWbTsz5/ZLLMLxvg9V+cr2 fCc+/B+Co6q9DnUyipfDfcHfvXDoPFseMmCfXH8I/4DPD8dmAxzM9BoviL9GSnAG T0AVfKtB/HeGeZ67jZ9iIqpNVhCjxRrb1XZmoa4RUTU5J8i1xxBqpnpcYtCQXoJm q7UJpJAVOHg68NQgxh3YK7wemb9nSacWO587rhCNjkIoVKkMDCQ= =K9in -END PGP SIGNATURE- - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
RE: Does Tomcat/Java get around the problem of 64K maximum client source ports?
Chris, André, Eric, etc, A few random thoughts. First, this is really a MySQL driver thing, not a Tomcat thing. If you want to know why the driver does what it does, look at the driver source. There is actually a Socket constructor that allows you to specify the local port. I don't know why you would do that, but it's there. Second, it does appear to be possible for there to be multiple connections using the same source port as long as they go to different destinations. Just read through the various links that people have provided. Honestly it blows my mind that that's even possible, but that just shows how little I know about networking. How does the client OS route return packets correctly if they have the same source port? John - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Does Tomcat/Java get around the problem of 64K maximum client source ports?
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Eric, On 3/26/20 13:58, Eric Robinson wrote: > Many people say the maximum number of client ports is 64K. > However, TCP connections only require unique sockets, which are > defined as... > > local_IP:local_port -> remote_ip:remote_port > > Theoretically, it is possible for a client process to keep using > the same local source port, as long as the connections are to a > unique destinations. For example on a local machine, the following > connections should be possible... > > 192.168.5.100:1400 -> 192.168.5.200:3306 192.168.5.100:1400 -> > 192.168.5.201:3306 192.168.5.100:1400 -> 192.168.5.202:3306 > 192.168.5.100:1400 -> 192.168.5.203:3306 I think you must be confusing something, here. The left-hand side of that list looks like it cannot happen. Each client connection requires its own source-port, so port 1400 cannot be used by many simultaneous connections. It's also strange for me to see client port numbers so low. Typically, client ports are much higher. It's also very uncommon for the client to specify a source-port, so the OS's TCP/IP stack usually allocates an available port, which is why they end up being higher-numbered ports. Have a look here for a very informative answer: https://superuser.com/a/1118742/408230 Note that the maximum number of client port is DEFINITELY fixed at 64k, because of the data types used by the C library. This happens when you bind() a socket to a port: int bind(int sockfd, const struct sockaddr *addr, socklen_t addrlen); struct sockaddr is: struct sockaddr { sa_family_t sa_family; charsa_data[14]; } and, for the AF_INET socket address family (from ): struct sockaddr_in { sa_family_tsin_family; /* address family: AF_INET */ in_port_t sin_port; /* port in network byte order */ struct in_addr sin_addr; /* internet address */ }; Continuing down the road, in_port_t (from /* Type to represent a port. */ typedef uint16_t in_port_t; So the port number is unsigned 16-bit. you can't squeeze more than 64 ports out of that. Note that you aren't limited to 64k sockets, as you have intimated: ports can be re-used. > I've seen this demonstrated successfully here: > > https://serverfault.com/questions/326819/does-the-tcp-source-port-have - -to-be-unique-per-host > > As someone on that page pointed out, while it is possible, it does > not commonly occur in practice "because most TCP APIs don't > provide a way to create more than one connection with the same > source port, unless they have different source IP addresses." This > leads to the 64K maximum client port range, but it is really a > limitation of the APIs, not TCP. > > So how does tomcat handle things? Is it limited to a maximum 64K > client source ports, or is it 64K per destination, as it should > be? The short answer is that Tomcat doesn't handle this. There are very few outgoing connections that Tomcat makes, and database connections aren't on that list. What you are seeing is the JDBC driver's connection being handled by Java, which ultimately is being handled by the JRE and the OS working together. I see you are using port 3306 which is MySQL and/or MariaDB. I don't know of any driver which bothers to set the local port, so it's again strange to me to see port 1400 getting ... any use at all. - -chris -BEGIN PGP SIGNATURE- Comment: Using GnuPG with Thunderbird - https://www.enigmail.net/ iQIzBAEBCAAdFiEEMmKgYcQvxMe7tcJcHPApP6U8pFgFAl5+KvEACgkQHPApP6U8 pFi5rxAAp8HoqhQFOX6FtR3r+f1wkcdx7eLVc68e1NngB05hXdLTiXtmRy+XRXYy 8XahJkFvqWkKpKKcRopn714g+P44qMcXCtLtgQ3iTRB9dwp5uuYKN5z6QbaAzWfq vPb8UXESFoxCiXVBuV5Q8H+9utvS1OaST8exkwuAi33dkuh/Es4gFWpmbrKVhExl s4TNwq2UCTB5d6ybNM8AZzeAiNxkrop9+zi3AcgcVN6KMUS/dPWen3YTchUyTpdc 821gOKVrpy3nvPdCJv2dtoZ7G343D/Ice+ZYaDEe+oooOcPu88siOzna5vVDyUSM z7o3XsQ/zHIihfDwG5F1z0upWEGPPy5RD9VwOGvlCCDrG3dCHpYrgYfcOm3jA1kh 6Bu1y8vIy2qX7ZWEaHPOyAMpZKsodugpGR6Piov78OsEQWBLW5pD5E1sOBkXa2TO QSoadpWSjTVlTfDvZvTM4l5KDA+Q8NFmBPvoViejXIQCsshT4Hz4fEnBZTSPzQLf 6bxJrp9+5Y9f2ZFM3Iss4NR6PfCTE5LbWGJZOY+IWzN1p2UUJtEw1nazbWc57kRx ZLZgwz8D0IiNnooU3NA6rr+hqC+FPYgszGOS3BJUyfUn7PetZbtoOA5ouZn7S5jQ kq18P3k9z9rc/BKSaSnYdX7NmUavkqfM0Nw+Dwrmds3fdchUVoY= =DXcE -END PGP SIGNATURE- - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Does Tomcat/Java get around the problem of 64K maximum client source ports?
On 27.03.2020 14:27, André Warnier (tomcat/perl) wrote: On 26.03.2020 20:42, Eric Robinson wrote: -Original Message- From: Olaf Kock Sent: Thursday, March 26, 2020 2:06 PM To: users@tomcat.apache.org Subject: Re: Does Tomcat/Java get around the problem of 64K maximum client source ports? Hi Eric, On 26.03.20 18:58, Eric Robinson wrote: Greetings, Many people say the maximum number of client ports is 64K. However, TCP connections only require unique sockets, which are defined as... local_IP:local_port -> remote_ip:remote_port Theoretically, it is possible for a client process to keep using the same local source port, as long as the connections are to a unique destinations. For example on a local machine, the following connections should be possible... 192.168.5.100:1400 -> 192.168.5.200:3306 192.168.5.100:1400 -> 192.168.5.201:3306 192.168.5.100:1400 -> 192.168.5.202:3306 192.168.5.100:1400 -> 192.168.5.203:3306 I've seen this demonstrated successfully here: https://serverfault.com/questions/326819/does-the-tcp-source-port-have -to-be-unique-per-host As someone on that page pointed out, while it is possible, it does not commonly occur in practice "because most TCP APIs don't provide a way to create more than one connection with the same source port, unless they have different source IP addresses." This leads to the 64K maximum client port range, but it is really a limitation of the APIs, not TCP. So how does tomcat handle things? Is it limited to a maximum 64K client source ports, or is it 64K per destination, as it should be? To be honest, I can't remember to have seen a web- or application server that accepts 64K concurrent connections at all, let alone to a single client. I can't come up with any reason to do so, I'd assume that there's a DOS attack if I get 100 concurrent incoming connections from a single IP, and a programming error if the server sets up more than 1K outgoing connections Maybe I'm missing the obvious, or have only administered meaningless installations, but I fail to see the real world relevance of this question. I don't blame you for being puzzled, but this not about tomcat accepting connections. It's about tomcat acting as the client, where MySQL is the server. I'm referring to client connections from tomcat to MySQL. We have about 1800 instances of tomcat running. This question comes up once in a while when tomcat can't connect to MySQL. Trust me, it can be an issue. --Eric The question is : is there even any Java API (method) (or even OS API) which allows a client to open a (client, non-LISTEN) socket AND specify the client IP address and/or port ? I mean, if there is none, then the question may be interesting in the absolute, but ultimately pointless. I believe that the IP address of client packets, will be determined by which /route/ the OS determines that the target server address can be reached (which will determine through which network interface the packets "go out", which will determine the sender IP address inserted in the packets). I don't think that the application-level software (here a java webapp) can determine this in advance. And the client port will be decided by the OS-level TCP stack, in function of which ones are not yet in use (which a java webapp can also not determine in advance). Example of creating a client socket : Socket echoSocket = new Socket(hostName, portNumber); The hostname (or IP address of ditto) and port numbers passed as arguments, are the IP:port of the server you are connecting /to/, not the ones of the local socket. Addendum : https://stackoverflow.com/questions/11129212/tcp-can-two-different-sockets-share-a-port/11129641 From which I gather that, although it may be possible (in some languages or using some API) to open several client connections (to different target IP/port) using the same local port number, you may still have other issues when doing this : - the host's TCP/IP stack has to keep track of all the simultaneous connections, and if the list gets very large, this may become very resource-intensive - there may be a limit at the OS level, to how many sockets are allowed at the same time - to each connection, corresponds some kind of a "file descriptor". There may be limits, at the process level (in this case the JVM running tomcat), or even at the host level, to how many of those may be in use at any one time (So if you were running on a host one JVM for a tomcat instance, it would compete with other processes running on the same host and also using file descriptors) It seems also thus possible that the reason why you are running against a problem not being able to make additional connections to a MySql server, is not necessarily the usage of all 64K port numbers, but one of the other limitations above. ---
Re: Does Tomcat/Java get around the problem of 64K maximum client source ports?
On 26.03.2020 20:42, Eric Robinson wrote: -Original Message- From: Olaf Kock Sent: Thursday, March 26, 2020 2:06 PM To: users@tomcat.apache.org Subject: Re: Does Tomcat/Java get around the problem of 64K maximum client source ports? Hi Eric, On 26.03.20 18:58, Eric Robinson wrote: Greetings, Many people say the maximum number of client ports is 64K. However, TCP connections only require unique sockets, which are defined as... local_IP:local_port -> remote_ip:remote_port Theoretically, it is possible for a client process to keep using the same local source port, as long as the connections are to a unique destinations. For example on a local machine, the following connections should be possible... 192.168.5.100:1400 -> 192.168.5.200:3306 192.168.5.100:1400 -> 192.168.5.201:3306 192.168.5.100:1400 -> 192.168.5.202:3306 192.168.5.100:1400 -> 192.168.5.203:3306 I've seen this demonstrated successfully here: https://serverfault.com/questions/326819/does-the-tcp-source-port-have -to-be-unique-per-host As someone on that page pointed out, while it is possible, it does not commonly occur in practice "because most TCP APIs don't provide a way to create more than one connection with the same source port, unless they have different source IP addresses." This leads to the 64K maximum client port range, but it is really a limitation of the APIs, not TCP. So how does tomcat handle things? Is it limited to a maximum 64K client source ports, or is it 64K per destination, as it should be? To be honest, I can't remember to have seen a web- or application server that accepts 64K concurrent connections at all, let alone to a single client. I can't come up with any reason to do so, I'd assume that there's a DOS attack if I get 100 concurrent incoming connections from a single IP, and a programming error if the server sets up more than 1K outgoing connections Maybe I'm missing the obvious, or have only administered meaningless installations, but I fail to see the real world relevance of this question. I don't blame you for being puzzled, but this not about tomcat accepting connections. It's about tomcat acting as the client, where MySQL is the server. I'm referring to client connections from tomcat to MySQL. We have about 1800 instances of tomcat running. This question comes up once in a while when tomcat can't connect to MySQL. Trust me, it can be an issue. --Eric The question is : is there even any Java API (method) (or even OS API) which allows a client to open a (client, non-LISTEN) socket AND specify the client IP address and/or port ? I mean, if there is none, then the question may be interesting in the absolute, but ultimately pointless. I believe that the IP address of client packets, will be determined by which /route/ the OS determines that the target server address can be reached (which will determine through which network interface the packets "go out", which will determine the sender IP address inserted in the packets). I don't think that the application-level software (here a java webapp) can determine this in advance. And the client port will be decided by the OS-level TCP stack, in function of which ones are not yet in use (which a java webapp can also not determine in advance). Example of creating a client socket : Socket echoSocket = new Socket(hostName, portNumber); The hostname (or IP address of ditto) and port numbers passed as arguments, are the IP:port of the server you are connecting /to/, not the ones of the local socket. - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
RE: Does Tomcat/Java get around the problem of 64K maximum client source ports?
> -Original Message- > From: Olaf Kock > Sent: Thursday, March 26, 2020 2:06 PM > To: users@tomcat.apache.org > Subject: Re: Does Tomcat/Java get around the problem of 64K maximum > client source ports? > > Hi Eric, > > On 26.03.20 18:58, Eric Robinson wrote: > > Greetings, > > > > Many people say the maximum number of client ports is 64K. However, > TCP connections only require unique sockets, which are defined as... > > > > local_IP:local_port -> remote_ip:remote_port > > > > Theoretically, it is possible for a client process to keep using the same > > local > source port, as long as the connections are to a unique destinations. For > example on a local machine, the following connections should be possible... > > > > 192.168.5.100:1400 -> 192.168.5.200:3306 > > 192.168.5.100:1400 -> 192.168.5.201:3306 > > 192.168.5.100:1400 -> 192.168.5.202:3306 > > 192.168.5.100:1400 -> 192.168.5.203:3306 > > > > I've seen this demonstrated successfully here: > > > > https://serverfault.com/questions/326819/does-the-tcp-source-port-have > > -to-be-unique-per-host > > > > As someone on that page pointed out, while it is possible, it does not > commonly occur in practice "because most TCP APIs don't provide a way to > create more than one connection with the same source port, unless they > have different source IP addresses." This leads to the 64K maximum client > port range, but it is really a limitation of the APIs, not TCP. > > > > So how does tomcat handle things? Is it limited to a maximum 64K client > source ports, or is it 64K per destination, as it should be? > > To be honest, I can't remember to have seen a web- or application server > that accepts 64K concurrent connections at all, let alone to a single client. > > I can't come up with any reason to do so, I'd assume that there's a DOS attack > if I get 100 concurrent incoming connections from a single IP, and a > programming error if the server sets up more than 1K outgoing connections > > Maybe I'm missing the obvious, or have only administered meaningless > installations, but I fail to see the real world relevance of this question. > > I don't blame you for being puzzled, but this not about tomcat accepting connections. It's about tomcat acting as the client, where MySQL is the server. I'm referring to client connections from tomcat to MySQL. We have about 1800 instances of tomcat running. This question comes up once in a while when tomcat can't connect to MySQL. Trust me, it can be an issue. --Eric Disclaimer : This email and any files transmitted with it are confidential and intended solely for intended recipients. If you are not the named addressee you should not disseminate, distribute, copy or alter this email. Any views or opinions presented in this email are solely those of the author and might not represent those of Physician Select Management. Warning: Although Physician Select Management has taken reasonable precautions to ensure no viruses are present in this email, the company cannot accept responsibility for any loss or damage arising from the use of this email or attachments. - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Does Tomcat/Java get around the problem of 64K maximum client source ports?
Hi Eric, On 26.03.20 18:58, Eric Robinson wrote: > Greetings, > > Many people say the maximum number of client ports is 64K. However, TCP > connections only require unique sockets, which are defined as... > > local_IP:local_port -> remote_ip:remote_port > > Theoretically, it is possible for a client process to keep using the same > local source port, as long as the connections are to a unique destinations. > For example on a local machine, the following connections should be > possible... > > 192.168.5.100:1400 -> 192.168.5.200:3306 > 192.168.5.100:1400 -> 192.168.5.201:3306 > 192.168.5.100:1400 -> 192.168.5.202:3306 > 192.168.5.100:1400 -> 192.168.5.203:3306 > > I've seen this demonstrated successfully here: > > https://serverfault.com/questions/326819/does-the-tcp-source-port-have-to-be-unique-per-host > > As someone on that page pointed out, while it is possible, it does not > commonly occur in practice "because most TCP APIs don't provide a way to > create more than one connection with the same source port, unless they have > different source IP addresses." This leads to the 64K maximum client port > range, but it is really a limitation of the APIs, not TCP. > > So how does tomcat handle things? Is it limited to a maximum 64K client > source ports, or is it 64K per destination, as it should be? To be honest, I can't remember to have seen a web- or application server that accepts 64K concurrent connections at all, let alone to a single client. I can't come up with any reason to do so, I'd assume that there's a DOS attack if I get 100 concurrent incoming connections from a single IP, and a programming error if the server sets up more than 1K outgoing connections Maybe I'm missing the obvious, or have only administered meaningless installations, but I fail to see the real world relevance of this question. - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org