RE: FTPServer handling of multiple concurrent connections.
L.S. Is there an ETA for this change? Greetings, Peter van der Velde -Original Message- From: David Latorre [mailto:dvl...@gmail.com] Sent: donderdag 1 april 2010 12:57 To: dev@mina.apache.org Subject: Re: FTPServer handling of multiple concurrent connections. 2010/3/31 Niklas Gustavsson nik...@protocol7.com: On Wed, Mar 31, 2010 at 6:32 PM, Sai Pullabhotla sai.pullabho...@jmethods.com wrote: Since I did not hear back anything on this, I will ask again :). Are you guys okay with the proposed short term solution? +1 to option 1 and to defaulting to max users. Sounds good :-) /niklas
Re: FTPServer handling of multiple concurrent connections.
I'm pretty sure it is checked into the trunk. Regards, Sai Pullabhotla On Fri, Apr 23, 2010 at 7:35 AM, Peter van der Velde peter.vanderve...@anachron.com wrote: L.S. Is there an ETA for this change? Greetings, Peter van der Velde -Original Message- From: David Latorre [mailto:dvl...@gmail.com] Sent: donderdag 1 april 2010 12:57 To: dev@mina.apache.org Subject: Re: FTPServer handling of multiple concurrent connections. 2010/3/31 Niklas Gustavsson nik...@protocol7.com: On Wed, Mar 31, 2010 at 6:32 PM, Sai Pullabhotla sai.pullabho...@jmethods.com wrote: Since I did not hear back anything on this, I will ask again :). Are you guys okay with the proposed short term solution? +1 to option 1 and to defaulting to max users. Sounds good :-) /niklas
RE: FTPServer handling of multiple concurrent connections.
Great, I will fetch the trunk then. Thanks! Greetings, Peter. -Original Message- From: Sai Pullabhotla [mailto:sai.pullabho...@jmethods.com] Sent: vrijdag 23 april 2010 15:24 To: dev@mina.apache.org Subject: Re: FTPServer handling of multiple concurrent connections. I'm pretty sure it is checked into the trunk. Regards, Sai Pullabhotla On Fri, Apr 23, 2010 at 7:35 AM, Peter van der Velde peter.vanderve...@anachron.com wrote: L.S. Is there an ETA for this change? Greetings, Peter van der Velde -Original Message- From: David Latorre [mailto:dvl...@gmail.com] Sent: donderdag 1 april 2010 12:57 To: dev@mina.apache.org Subject: Re: FTPServer handling of multiple concurrent connections. 2010/3/31 Niklas Gustavsson nik...@protocol7.com: On Wed, Mar 31, 2010 at 6:32 PM, Sai Pullabhotla sai.pullabho...@jmethods.com wrote: Since I did not hear back anything on this, I will ask again :). Are you guys okay with the proposed short term solution? +1 to option 1 and to defaulting to max users. Sounds good :-) /niklas
Re: FTPServer handling of multiple concurrent connections.
2010/3/31 Niklas Gustavsson nik...@protocol7.com: On Wed, Mar 31, 2010 at 6:32 PM, Sai Pullabhotla sai.pullabho...@jmethods.com wrote: Since I did not hear back anything on this, I will ask again :). Are you guys okay with the proposed short term solution? +1 to option 1 and to defaulting to max users. Sounds good :-) /niklas
Re: FTPServer handling of multiple concurrent connections.
Since I did not hear back anything on this, I will ask again :). Are you guys okay with the proposed short term solution? Regards, Sai Pullabhotla On Tue, Mar 30, 2010 at 10:10 AM, Sai Pullabhotla sai.pullabho...@jmethods.com wrote: Since changing everything over to MINA could be quite a bit of work, and the issue we have is somewhat serious, we should come up with a short term solution first and release a patch. Perhaps this patch would do the following: Option 1: Have Max Threads as a configurable option at the server level. Each listener would share the same thread pool. Option 2: Have Max Threads as a configurable option at the listener (acceptor) level. I prefer option 1 as most people would look at the server level rather than listener level. In other words, my server should be able to handle 200 concurrent users. If Max Threads is not specified, may be we should default it to maxUsers/connections that we have. Not sure what the current keep-alive time for the threads is, but perhaps having a shorter keep-alive time may help in some cases, or make it as a configurable option. What do you think? Regards, Sai Pullabhotla On Tue, Mar 30, 2010 at 8:33 AM, Niklas Gustavsson nik...@protocol7.com wrote: On Tue, Mar 30, 2010 at 2:22 PM, David Latorre dvl...@gmail.com wrote: I would rather go for a solution that make it impossible to block FTPServer rather than making it 'more difficult'. For this, we might limit the total number of data connections which wouldn't be perfect but might help... or maybe we can enforce a rule that MaxUsers MaxThreads. Depending on your user profiles, I think there might be different configurations that suites you best. So, I think we should supply both the max concurrent users option (that we do today) and a max threads option. Then, we need to provide reasonable defaults and documentation on how to best use them. - We could have a filter that limited bandwidth usage, although I don't think there is anything like that in FTPServer. We already do bandwidth limitation in our blocking data connections. /niklas
Re: FTPServer handling of multiple concurrent connections.
On Wed, Mar 31, 2010 at 6:32 PM, Sai Pullabhotla sai.pullabho...@jmethods.com wrote: Since I did not hear back anything on this, I will ask again :). Are you guys okay with the proposed short term solution? +1 to option 1 and to defaulting to max users. /niklas
FTPServer handling of multiple concurrent connections.
Hello, When inspecting http://issues.apache.org/jira/browse/FTPSERVER-359 I noticed that we might have an issue with threading in Ftpserver. I think we are using a 'fixed size thread pool' with max-threads=16 for the Executor threadpool in which commands are executed. This would mean that if we had 16 data transfers currently running , FTPServer would be blocked until some of these data transfers ended- Is my understanding correct? Is this behaviour desirable for us? I don't think that 16 data transfers are that many, and some users might need to send huge files.
Re: FTPServer handling of multiple concurrent connections.
I don't think that was intended. If this is in fact an issue, we should probably consider adding a configuration option such as maxThreads as the default max we choose may not be the best in all cases. Regards, Sai Pullabhotla On Tue, Mar 30, 2010 at 3:29 AM, David Latorre dvl...@gmail.com wrote: Hello, When inspecting http://issues.apache.org/jira/browse/FTPSERVER-359 I noticed that we might have an issue with threading in Ftpserver. I think we are using a 'fixed size thread pool' with max-threads=16 for the Executor threadpool in which commands are executed. This would mean that if we had 16 data transfers currently running , FTPServer would be blocked until some of these data transfers ended- Is my understanding correct? Is this behaviour desirable for us? I don't think that 16 data transfers are that many, and some users might need to send huge files.
Re: FTPServer handling of multiple concurrent connections.
On Tue, Mar 30, 2010 at 10:33 AM, Sai Pullabhotla sai.pullabho...@jmethods.com wrote: I don't think that was intended. If this is in fact an issue, we should probably consider adding a configuration option such as maxThreads as the default max we choose may not be the best in all cases. I think David is correct, and I agree with your proposed solution (defaulting the value higher than 16 thou). We should also look into moving the data connection over to using MINA, if someone got the time. /niklas
Re: FTPServer handling of multiple concurrent connections.
What would be the benefit of using MINA compared to the traditional sockets, especially for passive connections? Regards, Sai Pullabhotla On Tue, Mar 30, 2010 at 7:33 AM, Niklas Gustavsson nik...@protocol7.com wrote: On Tue, Mar 30, 2010 at 10:33 AM, Sai Pullabhotla sai.pullabho...@jmethods.com wrote: I don't think that was intended. If this is in fact an issue, we should probably consider adding a configuration option such as maxThreads as the default max we choose may not be the best in all cases. I think David is correct, and I agree with your proposed solution (defaulting the value higher than 16 thou). We should also look into moving the data connection over to using MINA, if someone got the time. /niklas
Re: FTPServer handling of multiple concurrent connections.
On Tue, Mar 30, 2010 at 1:39 PM, Sai Pullabhotla sai.pullabho...@jmethods.com wrote: What would be the benefit of using MINA compared to the traditional sockets, especially for passive connections? That we don't get one thread for each one socket. Perhaps this is not such a big issue, at least not as far as your not planning for thousands of concurrent data connections. /niklas
Re: FTPServer handling of multiple concurrent connections.
2010/3/30 Niklas Gustavsson nik...@protocol7.com: On Tue, Mar 30, 2010 at 10:33 AM, Sai Pullabhotla sai.pullabho...@jmethods.com wrote: I don't think that was intended. If this is in fact an issue, we should probably consider adding a configuration option such as maxThreads as the default max we choose may not be the best in all cases. I think David is correct, and I agree with your proposed solution (defaulting the value higher than 16 thou). We should also look into moving the data connection over to using MINA, if someone got the time. I would rather go for a solution that make it impossible to block FTPServer rather than making it 'more difficult'. For this, we might limit the total number of data connections which wouldn't be perfect but might help... or maybe we can enforce a rule that MaxUsers MaxThreads. Regarding the move of the data connection to MINA, I understand Sai's position that is using blocking-IO here is probably lighter ( It would be nice if we could have the same 'MINA pipeline' accepting on several ports...). Still I see an advantage if we used MINA - filters: - We could have different filters for ASCII mode, binary mode, 'Z' mode or other modes of encoding we or the users might think of (Although it is easy to have custom streams that do this for us with the current implementation). - We can include a fine grained monitoring of current transfers, a filter might inform via JMX or other means of e.g., the amount of data transferred - A filter could limit the quota for a user, for a session, for a week/month for this user in 'real time'. There are other ways to do this as well... - We could have a filter that limited bandwidth usage, although I don't think there is anything like that in FTPServer. Still, several of the possibilities we can think of seem not that useful since we should have multiple 'chains' for data connections. /niklas
Re: FTPServer handling of multiple concurrent connections.
On Tue, Mar 30, 2010 at 2:22 PM, David Latorre dvl...@gmail.com wrote: I would rather go for a solution that make it impossible to block FTPServer rather than making it 'more difficult'. For this, we might limit the total number of data connections which wouldn't be perfect but might help... or maybe we can enforce a rule that MaxUsers MaxThreads. Depending on your user profiles, I think there might be different configurations that suites you best. So, I think we should supply both the max concurrent users option (that we do today) and a max threads option. Then, we need to provide reasonable defaults and documentation on how to best use them. - We could have a filter that limited bandwidth usage, although I don't think there is anything like that in FTPServer. We already do bandwidth limitation in our blocking data connections. /niklas
Re: FTPServer handling of multiple concurrent connections.
Since changing everything over to MINA could be quite a bit of work, and the issue we have is somewhat serious, we should come up with a short term solution first and release a patch. Perhaps this patch would do the following: Option 1: Have Max Threads as a configurable option at the server level. Each listener would share the same thread pool. Option 2: Have Max Threads as a configurable option at the listener (acceptor) level. I prefer option 1 as most people would look at the server level rather than listener level. In other words, my server should be able to handle 200 concurrent users. If Max Threads is not specified, may be we should default it to maxUsers/connections that we have. Not sure what the current keep-alive time for the threads is, but perhaps having a shorter keep-alive time may help in some cases, or make it as a configurable option. What do you think? Regards, Sai Pullabhotla On Tue, Mar 30, 2010 at 8:33 AM, Niklas Gustavsson nik...@protocol7.com wrote: On Tue, Mar 30, 2010 at 2:22 PM, David Latorre dvl...@gmail.com wrote: I would rather go for a solution that make it impossible to block FTPServer rather than making it 'more difficult'. For this, we might limit the total number of data connections which wouldn't be perfect but might help... or maybe we can enforce a rule that MaxUsers MaxThreads. Depending on your user profiles, I think there might be different configurations that suites you best. So, I think we should supply both the max concurrent users option (that we do today) and a max threads option. Then, we need to provide reasonable defaults and documentation on how to best use them. - We could have a filter that limited bandwidth usage, although I don't think there is anything like that in FTPServer. We already do bandwidth limitation in our blocking data connections. /niklas
