Re: [twsocket] FTP Server example
Hello Angus, ARMSL If by exe you mean the same project and executable file, and you are not ARMSL already using threads, then the two applications are totally related, ARMSL only one will be running at a time. Yes, one delphi project producing one exe. ARMSL Again you don't actually say what you mean by 'database' , most people ARMSL would assume this means a database server engine like MS SQL, MySQL or ARMSL DB2 which you won't be compiling into your own executable. But maybe you ARMSL have a lightweight internal database or some non-SQL engine? I'm talking about building the FTP server functionality into a compiled NexusDB database server. So the server engine (which in itself is already multithreaded) will contain the FTPServer component. Can I create a thread and instantiate the FTPserver component within that thread ? Will this help ? Many thanks for your answers so far. David David Rose Navigator Systems Ltd, Network Business Centre, 3 Gloucester Street, Bath, BA1 2SE,UK Main Tel: +44 (0)207 183 0011 Fax: +44 (0)1225 464984 Company Email: da...@navigator.co.uk (Private e-mail on request) CONFIDENTIALITY NOTICE The contents of this e-mail are confidential to the ordinary user of the e-mail address to which it was addressed and may also be privileged. If you are not the addressee of this e-mail you may not copy, forward, disclose or otherwise use it or any part of it in any form whatsoever. If you have received this e-mail in error please e-mail the sender by replying to this message. -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] FTP Server example
David Rose wrote: Can I create a thread and instantiate the FTPserver component within that thread ? Will this help ? Yes! -- Arno Garrels -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] FTP Server example
Can I create a thread and instantiate the FTPserver component within that thread ? Will this help ? Yes, you can create the FTP server within a thread, but there are currently no ICS demos doing so. Unless you expect a performance issue, it's probably not worth the trouble. Angus -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] FTP Server example
Hello Angus, Many thanks. Is there anything specific to running this component in a thread I need to be aware of (I'm aware of general thread principles) ARMSL Unless you expect a performance issue, it's probably not worth the ARMSL trouble. I think you're right. I won't do anything about it until it turns out to be a problem. Though I'll make sure I know what I need to do IYSWIM. -- regards David Thursday, February 11, 2010, 11:59:00 AM, you wrote: Can I create a thread and instantiate the FTPserver component within that thread ? Will this help ? ARMSL Yes, you can create the FTP server within a thread, but there are ARMSL currently no ICS demos doing so. ARMSL Unless you expect a performance issue, it's probably not worth the ARMSL trouble. ARMSL Angus ARMSL -- ARMSL To unsubscribe or change your settings for TWSocket mailing list ARMSL please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket ARMSL Visit our website at http://www.overbyte.be David Rose Navigator Systems Ltd, Network Business Centre, 3 Gloucester Street, Bath, BA1 2SE,UK Main Tel: +44 (0)207 183 0011 Fax: +44 (0)1225 464984 Company Email: da...@navigator.co.uk (Private e-mail on request) CONFIDENTIALITY NOTICE The contents of this e-mail are confidential to the ordinary user of the e-mail address to which it was addressed and may also be privileged. If you are not the addressee of this e-mail you may not copy, forward, disclose or otherwise use it or any part of it in any form whatsoever. If you have received this e-mail in error please e-mail the sender by replying to this message. -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] FTP Server example
Many thanks. Is there anything specific to running this component in a thread I need to be aware of (I'm aware of general thread principles) I've not used a thread for a server myself, but it's purely event driven so should be straight forward. Just don't try updating visual components or log files in the main thread, except via Windows messages or a similar technology. Angus -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] FTP Server example
Many thanks. Is there anything specific to running this component in a thread I need to be aware of (I'm aware of general thread principles) Make sure your thread has a message pump otherwise no event will be handled. Have a look at OverbyteIcssDll1.dpr sample project. It is a DLL using a thread to run ICS component. The thread part is just the same being in a DLL or EXE. -- francois.pie...@overbyte.be The author of the freeware multi-tier middleware MidWare The author of the freeware Internet Component Suite (ICS) http://www.overbyte.be -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] FTP Server example
Hello Angus, I don't think I explained properly. The FTP server and Database are completely unrelated except for the fact that they share the same exe. In an ideal world, they'd be two completely independent exe's but the spec of this program means that one exe has got to be both a database engine and an FTP server. However the FTP server isn't doing very much I have a client program myapp.exe which when it starts logs into the FTP server to see if there is an updated version of itself, if there is, then it downloads it, if there's not, then it logs off the ftp server and gets on with being a database client app. So all the FTP server is doing is supplying a directory listing to myprogram.exe and then with an updated exe if one exists. The FTP server and database NEVER talk to each other. -- regards David Tuesday, February 9, 2010, 2:44:00 PM, you wrote: ARMSL Your server application also needs to take care with file lock timeouts, ARMSL network timeouts, etc, to avoid blocking the main thread. Could you elaborate. I know very little about this subject. ARMSL Provided you do not make database calls from any FTP server events, you ARMSL will not have a problem. Generally, making simple database calls is ARMSL painless since they usually respond within 20ms and that is the time ARMSL during which the FTP server will stop responding to all clients. But you ARMSL would not want to do a complex database query that took any longer ARMSL without using a thread. ARMSL But if the database needs to be re-opened and there is a network problem ARMSL (since real databases are never on the same platform as an FTP server) ARMSL there may be a 30 second timeout before the connection fails, and that ARMSL means the FTP server also stops for 30 seconds. ARMSL One of my applications used async database events, both opening and ARMSL executing statements, with a queue for new statements that are needed ARMSL before previous ones have completed, with a trigger emptying the queue, ARMSL but it all gets very complicated, particularly the error handling. ARMSL My FTP servers also log to a database, but this is very fast since it ARMSL mostly does not need the response to be interpreted. ARMSL The same issues apply to file event logging, opening a file on a network ARMSL share may have a long timeout, during which the server is blocked. ARMSL Angus ARMSL -- ARMSL To unsubscribe or change your settings for TWSocket mailing list ARMSL please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket ARMSL Visit our website at http://www.overbyte.be David Rose Navigator Systems Ltd, Network Business Centre, 3 Gloucester Street, Bath, BA1 2SE,UK Main Tel: +44 (0)207 183 0011 Fax: +44 (0)1225 464984 Company Email: da...@navigator.co.uk (Private e-mail on request) CONFIDENTIALITY NOTICE The contents of this e-mail are confidential to the ordinary user of the e-mail address to which it was addressed and may also be privileged. If you are not the addressee of this e-mail you may not copy, forward, disclose or otherwise use it or any part of it in any form whatsoever. If you have received this e-mail in error please e-mail the sender by replying to this message. -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] FTP Server example
I don't think I explained properly. The FTP server and Database are completely unrelated except for the fact that they share the same exe. If by exe you mean the same project and executable file, and you are not already using threads, then the two applications are totally related, only one will be running at a time. Again you don't actually say what you mean by 'database' , most people would assume this means a database server engine like MS SQL, MySQL or DB2 which you won't be compiling into your own executable. But maybe you have a lightweight internal database or some non-SQL engine? If you actually mean database client code accessing an external database, all my previous comments stand, the FTP server will block during database access. I have a client program myapp.exe Irrelevant to this discussion. Angus -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] FTP Server example
ARMSL Your server application also needs to take care with file lock timeouts, ARMSL network timeouts, etc, to avoid blocking the main thread. Could you elaborate. I know very little about this subject. I just need the simplest way to act as a simple FTP server. I will be controlling both the files it's serving (max size will be about 10MB) and the client software connecting, but the server is part of a database engine, so I need to be sure that some FTP action isn't going to hold up the database functions. Basically the FTP server part is used by our client autoupdate mechanism. The Database Server program downloads any updates to our client program from our internet based server and presents the downloaded info to the Admin. If the Admin decides that he wants the clients updated to this new release, he 'publishes' the update files - which basically means making sure the FTP server component is aware of them. Next time a client starts, it checks the local FTP site (i.e the database engine program) to see if there is an update, and if so, it downloads it and updates itself. Most system have about 5-10 clients (though some have a 100) and I suppose the worst time will be the Monday morning following an update when all the clients try to update themselves at once. David Rose Navigator Systems Ltd, Network Business Centre, 3 Gloucester Street, Bath, BA1 2SE,UK Main Tel: +44 (0)207 183 0011 Fax: +44 (0)1225 464984 Company Email: da...@navigator.co.uk (Private e-mail on request) CONFIDENTIALITY NOTICE The contents of this e-mail are confidential to the ordinary user of the e-mail address to which it was addressed and may also be privileged. If you are not the addressee of this e-mail you may not copy, forward, disclose or otherwise use it or any part of it in any form whatsoever. If you have received this e-mail in error please e-mail the sender by replying to this message. -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] FTP Server example
ARMSL Your server application also needs to take care with file lock timeouts, ARMSL network timeouts, etc, to avoid blocking the main thread. Could you elaborate. I know very little about this subject. Provided you do not make database calls from any FTP server events, you will not have a problem. Generally, making simple database calls is painless since they usually respond within 20ms and that is the time during which the FTP server will stop responding to all clients. But you would not want to do a complex database query that took any longer without using a thread. But if the database needs to be re-opened and there is a network problem (since real databases are never on the same platform as an FTP server) there may be a 30 second timeout before the connection fails, and that means the FTP server also stops for 30 seconds. One of my applications used async database events, both opening and executing statements, with a queue for new statements that are needed before previous ones have completed, with a trigger emptying the queue, but it all gets very complicated, particularly the error handling. My FTP servers also log to a database, but this is very fast since it mostly does not need the response to be interpreted. The same issues apply to file event logging, opening a file on a network share may have a long timeout, during which the server is blocked. Angus -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] FTP Server example
David Rose wrote: If my main app (which contains an TFtpServer component) is working hard processing data and a client requests a large file, does the main app stop processing its data whilst the file is being sent ? No, it would not. However such an application should excecute the TFtpServer in the context of one separate worker thread. The worker thread must have a working message pump and the component has to be created and freed in TThread.Execute method. What if 25 users request the same large file ? They are all served at the same time. Depending on network speed and CPU power/load their download might however take a bit longer. -- Arno Garrels -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] FTP Server example
If my main app (which contains an TFtpServer component) is working hard processing data and a client requests a large file, does the main app stop processing its data whilst the file is being sent ? There are two distincts things there: 1) hard processing 2) Sending large file Answers: 1) While you are processing, unless you place processing in a separate thread, the FTP server will be stopped or at least very slowed down (depending on how often you call the message pump). 2) While sendinglarge file, there is no performance hit: the componentis asynchronous and will send the large file without blocking. What if 25 users request the same large file ? No problem. Actually, if you have a very fast network (Gigabit) and hard disk system (ie 15000 rpm SCSI), and a small CPU, then the I/O will be able to overflow the processing power of your CPU. Usually the network is the most limiting factor. If you have 10 or 100 mbps bandwidth and a decent processor, there will be no real slow down while files are transfered. Does it answers your question ? -- francois.pie...@overbyte.be The author of the freeware multi-tier middleware MidWare The author of the freeware Internet Component Suite (ICS) http://www.overbyte.be -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] FTP Server example
Hello Francois, Ok, thanks, but is there any reason why the processing of the delivery of the file takes place in the OnTerminate event, rather than at the end of the execute method i.e after the call to the lengthy process. -- regards David Thursday, January 28, 2010, 4:56:24 PM, you wrote: I've been looking at the FTPServ1 sample app and I don't quite understand the worker thread part of the Getprocessing routine . Why is the data sent in the OnTerminate event rather than the execute method ? FP The idea is to have the FTP server trigger some lengthy processing (here a FP simple 10 sec sleep for the demo, but could be a very long database query) FP to produce some data in response to a ftp get. This shows that the server FP keep servicing other clients while processing take place in a thread. FP -- FP francois.pie...@overbyte.be FP The author of the freeware multi-tier middleware MidWare FP The author of the freeware Internet Component Suite (ICS) FP http://www.overbyte.be FP -- FP To unsubscribe or change your settings for TWSocket mailing list FP please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket FP Visit our website at http://www.overbyte.be David Rose Navigator Systems Ltd, Network Business Centre, 3 Gloucester Street, Bath, BA1 2SE,UK Main Tel: +44 (0)207 183 0011 Fax: +44 (0)1225 464984 Company Email: da...@navigator.co.uk (Private e-mail on request) CONFIDENTIALITY NOTICE The contents of this e-mail are confidential to the ordinary user of the e-mail address to which it was addressed and may also be privileged. If you are not the addressee of this e-mail you may not copy, forward, disclose or otherwise use it or any part of it in any form whatsoever. If you have received this e-mail in error please e-mail the sender by replying to this message. -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] FTP Server example
David Rose wrote: Ok, thanks, but is there any reason why the processing of the delivery of the file takes place in the OnTerminate event, rather than at the end of the execute method i.e after the call to the lengthy process. The component (derived from TWSocketServer) is designed to run socket IO in context of a single thread. That's possible since ICS uses non-blocking winsock API. If you do not have to serve more than around 100-200 clients on a multi-processor system all _pumping data concurrently at the same time_ the application won't benefit from multiple socket threads. Only if socket IO actually was a bottleneck it might make sense to use a different thread design, something like around 100 clients per thread, for example. -- Arno Garrels -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] FTP Server example
Ok, thanks, but is there any reason why the processing of the delivery of the file takes place in the OnTerminate event, rather than at the end of the execute method i.e after the call to the lengthy process. OnTerminate event is handled in the context of the calling thread, that is the thread which runs the component. Placing code at the end of the execute method make it executed in the context of the worker thread and as such would require locked access to the component. Using the OnTerminate event handler greatly simplify the code. -- francois.pie...@overbyte.be The author of the freeware multi-tier middleware MidWare The author of the freeware Internet Component Suite (ICS) http://www.overbyte.be -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] FTP Server example
AG Only if socket IO actually was a bottleneck it might make sense AG to use a different thread design, something like around 100 AG clients per thread, for example. You'll have to bare with my questions on this as this is a subject I don't really know yet. If my main app (which contains an TFtpServer component) is working hard processing data and a client requests a large file, does the main app stop processing its data whilst the file is being sent ? What if 25 users request the same large file ? If it does stop, then what do I need to do to make sure that it doesn't ? David Rose Navigator Systems Ltd, Network Business Centre, 3 Gloucester Street, Bath, BA1 2SE,UK Main Tel: +44 (0)207 183 0011 Fax: +44 (0)1225 464984 Company Email: da...@navigator.co.uk (Private e-mail on request) CONFIDENTIALITY NOTICE The contents of this e-mail are confidential to the ordinary user of the e-mail address to which it was addressed and may also be privileged. If you are not the addressee of this e-mail you may not copy, forward, disclose or otherwise use it or any part of it in any form whatsoever. If you have received this e-mail in error please e-mail the sender by replying to this message. -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] FTP Server example
David Rose wrote: You'll have to bare with my questions on this as this is a subject I don't really know yet. If my main app (which contains an TFtpServer component) is working hard processing data and a client requests a large file, does the main app stop processing its data whilst the file is being sent ? What if 25 users request the same large file ? If it does stop, then what do I need to do to make sure that it doesn't ? TFtpServer component will create a new thread for each client, and each client will be handled in multiple instances of the same procedure. The same file can be sent to each individual user. ICS is non blocking so the application won't stop while a client is downloading a ginormous file. Hope that helps. Lester -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] FTP Server example
TFtpServer component will create a new thread for each client, and each client will be handled in multiple instances of the same procedure. The ICS FTP server does not create a thread for each client, all clients are handled using the same thread. Some commands that require potentially lengthy processing are handled using a temporary thread, specifically some directory listing commands, MD5 and CRC32 calculations. A temporary thread is used since these commands can take a couple of minutes with gigabyte sized files during which all other actions in the server would be blocked. If you extend the FTP server with custom commands that take more than a second or so to process, you should also use a thread for that processing. Your server application also needs to take care with file lock timeouts, network timeouts, etc, to avoid blocking the main thread. Angus -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] FTP Server example
I've been looking at the FTPServ1 sample app and I don't quite understand the worker thread part of the Getprocessing routine . Why is the data sent in the OnTerminate event rather than the execute method ? The idea is to have the FTP server trigger some lengthy processing (here a simple 10 sec sleep for the demo, but could be a very long database query) to produce some data in response to a ftp get. This shows that the server keep servicing other clients while processing take place in a thread. -- francois.pie...@overbyte.be The author of the freeware multi-tier middleware MidWare The author of the freeware Internet Component Suite (ICS) http://www.overbyte.be -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
[twsocket] FTP Server example
I've been looking at the FTPServ1 sample app and I don't quite understand the worker thread part of the Getprocessing routine . Why is the data sent in the OnTerminate event rather than the execute method ? regards David David Rose Navigator Systems Ltd, Network Business Centre, 3 Gloucester Street, Bath, BA1 2SE,UK Main Tel: +44 (0)207 183 0011 Fax: +44 (0)7000 NAVFAX (628329) Company Email: da...@navigator.co.uk (Private e-mail on request) CONFIDENTIALITY NOTICE The contents of this e-mail are confidential to the ordinary user of the e-mail address to which it was addressed and may also be privileged. If you are not the addressee of this e-mail you may not copy, forward, disclose or otherwise use it or any part of it in any form whatsoever. If you have received this e-mail in error please e-mail the sender by replying to this message. -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
