Re: [twsocket] Using the FTP Client
Maybe "background loop" is the wrong term. I have a timer generating an event every 50mS. This routine processes the data from the USB and FTP. Well, this is slightly better but still use CPU for nothing most of the time. The master idea is to have all thread sleeping almost all time and waken up only when something has to be processed. Waking a thread up every 50 mS and then checking for some condition which is probably false most of the time is a waste of CPU and introduce a delay of 50 mS in every action, even if the system is doing nothing at the wake up time. Consider using windows events (or mutex or semaphore). It's quite easy once you've got the idea. -- francois.pie...@overbyte.be The author of the freeware multi-tier middleware MidWare The author of the freeware Internet Component Suite (ICS) http://www.overbyte.be - Original Message - From: "Graham Powell" To: "'ICS support mailing'" Sent: Friday, January 28, 2011 1:10 PM Subject: Re: [twsocket] Using the FTP Client Maybe "background loop" is the wrong term. I have a timer generating an event every 50mS. This routine processes the data from the USB and FTP. Graham -Original Message- From: twsocket-boun...@elists.org [mailto:twsocket-boun...@elists.org] On Behalf Of Francois PIETTE Sent: 28 January 2011 09:42 To: ICS support mailing Subject: Re: [twsocket] Using the FTP Client While the FTP data is being processed, the USB is continually monitored and FTP processing is interrupted to service the USB. Reading that rings the bell in my head ! If you are using polling, then this is bad design. Use two separate threads to handle USB and to handle FTP. Each thread should work purely asynchronous (as ICS does) or work blocked on a read or write (for the USB). Never use a polling loop. The USB is read by a high priority thread and stored in a buffer. The background loop monitors the USB buffer and the buffered streams from the FTP and processes accordingly. OK, but what is the "background loop" ? If it is a pure loop checking for boolean or integer variable flags, then it is bad design: you consume a lot of CPU for nothing. If it is a loop using windows events (I mean http://msdn.microsoft.com/en-us/library/ms682396(v=vs.85).aspx), semaphore or mutex, then it is probably OK. -- francois.pie...@overbyte.be The author of the freeware multi-tier middleware MidWare The author of the freeware Internet Component Suite (ICS) http://www.overbyte.be -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] Using the FTP Client
> Maybe "background loop" is the wrong term. I have a timer > generating anevent every 50mS. This routine processes the data from > the USB and FTP. 50ms is virtually a continuous loop, it's very short. I'd make it 500ms and see if application performance improves. Or use a thread and create an event. Angus -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] Using the FTP Client
Maybe "background loop" is the wrong term. I have a timer generating an event every 50mS. This routine processes the data from the USB and FTP. Graham -Original Message- From: twsocket-boun...@elists.org [mailto:twsocket-boun...@elists.org] On Behalf Of Francois PIETTE Sent: 28 January 2011 09:42 To: ICS support mailing Subject: Re: [twsocket] Using the FTP Client >> While the FTP data is being processed, the USB is continually >> monitored and FTP processing is interrupted to service the USB. > > Reading that rings the bell in my head ! > If you are using polling, then this is bad design. > Use two separate threads to handle USB and to handle FTP. Each thread > should work purely asynchronous (as ICS does) or work blocked on a > read or write (for the USB). Never use a polling loop. > The USB is read by a high priority thread and stored in a buffer. The > background loop monitors the USB buffer and the buffered streams from > the FTP and processes accordingly. OK, but what is the "background loop" ? If it is a pure loop checking for boolean or integer variable flags, then it is bad design: you consume a lot of CPU for nothing. If it is a loop using windows events (I mean http://msdn.microsoft.com/en-us/library/ms682396(v=vs.85).aspx), semaphore or mutex, then it is probably OK. -- francois.pie...@overbyte.be The author of the freeware multi-tier middleware MidWare The author of the freeware Internet Component Suite (ICS) http://www.overbyte.be -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] Using the FTP Client
While the FTP data is being processed, the USB is continually monitored and FTP processing is interrupted to service the USB. Reading that rings the bell in my head ! If you are using polling, then this is bad design. Use two separate threads to handle USB and to handle FTP. Each thread should work purely asynchronous (as ICS does) or work blocked on a read or write (for the USB). Never use a polling loop. The USB is read by a high priority thread and stored in a buffer. The background loop monitors the USB buffer and the buffered streams from the FTP and processes accordingly. OK, but what is the "background loop" ? If it is a pure loop checking for boolean or integer variable flags, then it is bad design: you consume a lot of CPU for nothing. If it is a loop using windows events (I mean http://msdn.microsoft.com/en-us/library/ms682396(v=vs.85).aspx), semaphore or mutex, then it is probably OK. -- francois.pie...@overbyte.be The author of the freeware multi-tier middleware MidWare The author of the freeware Internet Component Suite (ICS) http://www.overbyte.be -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] Using the FTP Client
The USB is read by a high priority thread and stored in a buffer. The background loop monitors the USB buffer and the buffered streams from the FTP and processes accordingly. Graham -Original Message- From: twsocket-boun...@elists.org [mailto:twsocket-boun...@elists.org] On Behalf Of Francois PIETTE Sent: 27 January 2011 20:48 To: ICS support mailing Subject: Re: [twsocket] Using the FTP Client > While the FTP data is being processed, the USB is continually > monitored and FTP processing is interrupted to service the USB. Reading that rings the bell in my head ! If you are using polling, then this is bad design. Use two separate threads to handle USB and to handle FTP. Each thread should work purely asynchronous (as ICS does) or work blocked on a read or write (for the USB). Never use a polling loop. -- francois.pie...@overbyte.be The author of the freeware multi-tier middleware MidWare The author of the freeware Internet Component Suite (ICS) http://www.overbyte.be -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] Using the FTP Client
While the FTP data is being processed, the USB is continually monitored and FTP processing is interrupted to service the USB. Reading that rings the bell in my head ! If you are using polling, then this is bad design. Use two separate threads to handle USB and to handle FTP. Each thread should work purely asynchronous (as ICS does) or work blocked on a read or write (for the USB). Never use a polling loop. -- francois.pie...@overbyte.be The author of the freeware multi-tier middleware MidWare The author of the freeware Internet Component Suite (ICS) http://www.overbyte.be -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] Using the FTP Client
> It is all one application. While having one application is very convenient, is there not some way to decouple the two different tasks into separate applications, perhaps with a socket being used for communication, or maybe just shared memory? This would remove any possible thread synchronise issues. > We write the software and manufacture the teleprompter hardware. I > also have my own iNEWS server. Absolutely the best way to test it all! Angus -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] Using the FTP Client
It is all one application. The FTP client uses only Async commands and loads the data to a stream. The USB component, based on CreateFile etc. has a high priority thread for reading data and stores the data in a string embedded in a Critical Section Acquire and Release. In this instance this is better than using PostMessage to basically generate an event into the main application thread. I now have a background timer in the main thread which monitors the incoming USB data string and if there is any FTP stream data to be processed. We write the software and manufacture the teleprompter hardware. I also have my own iNEWS server. Graham -Original Message- From: twsocket-boun...@elists.org [mailto:twsocket-boun...@elists.org] On Behalf Of Angus Robertson - Magenta Systems Ltd Sent: 27 January 2011 16:16 To: twsocket@elists.org Subject: Re: [twsocket] Using the FTP Client > It is only when the FTP is getting data that it can go pear shaped. > While the FTP data is being processed, the USB is continually > monitored and FTP processing is interrupted to service the USB. Are FTP and USB servicing running as separate applications? FTP is presumably writing a simple file and not trying to immediately resend the data to USB? It may be the USB stuff needs 100% CPU and can not tolerate anything else running. You are using async FTP, or at least FTP in a thread which is how I got 250 sessions running? You really just need to keep Task Manager running while the problem is happening, and see how many seconds of CPU each different application is using, the hog should become obvious. > All this works on my system So you have the teleprompter for development testing? It is often hard to replicate customer environments. Angus -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] Using the FTP Client
Maybe you'd try to extract USB interaction into separate thread and assign to it a higher priority? -- Anton -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] Using the FTP Client
> It is only when the FTP is getting data that it can go pear shaped. > While the FTP data is being processed, the USB is continually > monitored and FTP processing is interrupted to service the USB. Are FTP and USB servicing running as separate applications? FTP is presumably writing a simple file and not trying to immediately resend the data to USB? It may be the USB stuff needs 100% CPU and can not tolerate anything else running. You are using async FTP, or at least FTP in a thread which is how I got 250 sessions running? You really just need to keep Task Manager running while the problem is happening, and see how many seconds of CPU each different application is using, the hog should become obvious. > All this works on my system So you have the teleprompter for development testing? It is often hard to replicate customer environments. Angus -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] Using the FTP Client
The USB device is not a disk drive, but a teleprompter. Data is sent when requested from the device. So single bytes from the device result in large amounts of data being sent from the PC to the USB device. All this works. It is only when the FTP is getting data that it can go pear shaped. While the FTP data is being processed, the USB is continually monitored and FTP processing is interrupted to service the USB. All this works on my system and I just need some ideas as to what can cause this at the customer site. >From what you say, you are ruling out the FTP client itself. I shall try and get some ICS logging files to compare. Graham -Original Message- From: twsocket-boun...@elists.org [mailto:twsocket-boun...@elists.org] On Behalf Of Angus Robertson - Magenta Systems Ltd Sent: 27 January 2011 14:31 To: twsocket@elists.org Subject: Re: [twsocket] Using the FTP Client > somehow the application can at times no longer get data from the USB > quick enough. Disk I/O in Windows can be blocking, you normally only see it with bad disk sectors when the event log shows lots of errors in red when it times out waiting for a response from the drive. If this is the sort of error the USB memory key is causing, Windows will be upset. > Is it possible that the FTP transactions can, for whatever network > reasons> hog the CPU resources so that my application cannot get > access to the USB data? No, FTP uses very little CPU. I was testing the new bandwidth throttling a few months ago, with both ICS FTP client and server, and using one thread per FTP client I was successfully running 250 simultaneous FTP sessions (using my TMagFtp component), that is proper sessions each indexing files and downloading multiple real files at the same time. I even had 100 session to my public FTP server over the internet. Durations did increase as number of sessions went up, but never stopped, and that was on my old Core2 PC with 32-bit Windows and little memory, my newer PC is much more beefy and it's all 64-bit now, but not had time to test again. Angus -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] Using the FTP Client
> somehow the application can at times no longer get data from the USB > quick enough. Disk I/O in Windows can be blocking, you normally only see it with bad disk sectors when the event log shows lots of errors in red when it times out waiting for a response from the drive. If this is the sort of error the USB memory key is causing, Windows will be upset. > Is it possible that the FTP transactions can, for whatever network > reasons> hog the CPU resources so that my application cannot get > access to the USB data? No, FTP uses very little CPU. I was testing the new bandwidth throttling a few months ago, with both ICS FTP client and server, and using one thread per FTP client I was successfully running 250 simultaneous FTP sessions (using my TMagFtp component), that is proper sessions each indexing files and downloading multiple real files at the same time. I even had 100 session to my public FTP server over the internet. Durations did increase as number of sessions went up, but never stopped, and that was on my old Core2 PC with 32-bit Windows and little memory, my newer PC is much more beefy and it's all 64-bit now, but not had time to test again. Angus -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] Using the FTP Client
With FTP off or connected but not receiving data then all is fine. No problem with USB. Haven't looked at CPU usage, will possibly try that. Graham -Original Message- From: twsocket-boun...@elists.org [mailto:twsocket-boun...@elists.org] On Behalf Of Anton S. Sent: 27 January 2011 13:24 To: twsocket@elists.org Subject: Re: [twsocket] Using the FTP Client >Is it possible that the FTP transactions can, for whatever network >reasons, hog the CPU resources so that my application cannot get access to the USB data? Is it possible to completely turn off all FTP exchanges and try only with USB? You may also check CPU usage during FTP sessions. -- Anton -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] Using the FTP Client
>Is it possible that the FTP transactions can, for whatever network reasons, hog the CPU resources so that my application cannot get access to the USB data? Is it possible to completely turn off all FTP exchanges and try only with USB? You may also check CPU usage during FTP sessions. -- Anton -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
[twsocket] Using the FTP Client
OK all you clever people out there this is my current problem. I have an application that connects to a newsroom computer system (NCS) via FTP. I have a server in my office and use the ICS FTP client to make the connection. LIST commands, RETR commands etc. are going on pretty well all the time. In addition to this, the application is sending and receiving data via USB to some proprietary hardware. With my simple network which just consists of a few computers on a Workgroup, every thing is perfect. FTP data is loading to the application and USB data is flying back and forth. However at the customers site (big TV news station), all changes. The symptom I get is that when I turn on the FTP and start getting data, somehow the application can at times no longer get data from the USB quick enough. I found that I was sending more data up the USB than was necessary and when this was fixed the system at the customers site worked perfectly. Now the bad news. The customer needs proof that it is the FTP or something to do with the network that is causing the problem. Somehow I need to be able to replicate his problem on my system. I have tried playing with the ftpBandwidthControl and the BandwidthLimit, but cannot seem to make it do much. The customer suggested I should be able to throttle the data. Is this the same thing or something else I can try. Is it possible that the FTP transactions can, for whatever network reasons, hog the CPU resources so that my application cannot get access to the USB data? Any help much appreciated. Graham -- To unsubscribe or change your settings for TWSocket mailing list please goto http://lists.elists.org/cgi-bin/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] Using the ftp-client
> Have you specified a filename for local and remote ? Thanks, I missed that part, I had just set the name for local. Now it works. Regards Bjørnar -- To unsubscribe or change your settings for TWSocket mailing list please goto http://www.elists.org/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
Re: [twsocket] Using the ftp-client
> I want to conect and upload a file and quit, and found the > Transfer-prcedure. All seems fine, I get logged in and the password is > correct, but I get the following error-message when sending the file: > "< 500 'STOR' not understood" > > Is this a known problem? Have you specified a filename for local and remote ? Do you have the right permissions on the FTP server ? -- Contribute to the SSL Effort. Visit http://www.overbyte.be/eng/ssl.html -- [EMAIL PROTECTED] Author of ICS (Internet Component Suite, freeware) Author of MidWare (Multi-tier framework, freeware) http://www.overbyte.be -- To unsubscribe or change your settings for TWSocket mailing list please goto http://www.elists.org/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
[twsocket] Using the ftp-client
I tried the FTP-client of ICS for the first time today. I want to conect and upload a file and quit, and found the Transfer-prcedure. All seems fine, I get logged in and the password is correct, but I get the following error-message when sending the file: "< 500 'STOR' not understood" Is this a known problem? I'm not wery familiar with ftp. I tried using WinSCP to upload files, and that worked (not sure if this program uses the ftp-protocol though). Regards Bjørnar -- To unsubscribe or change your settings for TWSocket mailing list please goto http://www.elists.org/mailman/listinfo/twsocket Visit our website at http://www.overbyte.be
