Re: Reliable old script loses data on new Cygwin installation
On Tue, 13 Nov 2007, Thomas Baker wrote: The /bin/pdksh script sequence that is causing problems is: [snip] The obvious question is: do you still get the error if you replace pdksh with bash, or is the problem pdksh-specific? When you reduce this to a minimal testcase, are there bits of that testcase that require pdksh to be used? Igor -- http://cs.nyu.edu/~pechtcha/ |\ _,,,---,,_[EMAIL PROTECTED] | [EMAIL PROTECTED] ZZZzz /,`.-'`'-. ;-;;,_Igor Peshansky, Ph.D. (name changed!) |,4- ) )-,_. ,\ ( `'-' old name: Igor Pechtchanski '---''(_/--' `-'\_) fL a.k.a JaguaR-R-R-r-r-r-.-.-. Meow! Belief can be manipulated. Only knowledge is dangerous. -- Frank Herbert -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Problem reports: http://cygwin.com/problems.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/
Re: Reliable old script loses data on new Cygwin installation
On Thu, Nov 22, 2007 at 12:56:33PM -0500, Igor Peshansky wrote: The /bin/pdksh script sequence that is causing problems is: [snip] The obvious question is: do you still get the error if you replace pdksh with bash, or is the problem pdksh-specific? When you reduce this to a minimal testcase, are there bits of that testcase that require pdksh to be used? It appears that pdksh (or Cygwin) per se had nothing to do with the problem. As I reported in [1], the problem was resolved when I discovered that the anti-virus software (Anti-Vir) was checking each time a file is written to disk. When I switched off this extra check (so that Anti-Vir now checks only when files are read), the script ran perceptibly faster and data is no longer lost. I originally wrote the script using the MKS Toolkit Korn shell with DOS 3.3, and it uses some features which may be Korn shell-specific (e.g., redirecting to numbered file descriptors with constructs such as 3|) -- I have not really tried to port it to bash. Tom [1] http://cygwin.com/ml/cygwin/2007-11/msg00313.html -- Tom Baker - [EMAIL PROTECTED] - [EMAIL PROTECTED] -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Problem reports: http://cygwin.com/problems.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/
Re: Reliable old script loses data on new Cygwin installation
On Wed, Nov 14, 2007 at 06:03:42PM -0500, Larry Hall (Cygwin) wrote: Lewis Hyatt wrote: These machines have an anti-virus program, but the same one I have been using for the past two or three years. The filenames either have no extension, or .txt. That's most likely the problem anyway, what happens if you turn it off? This is possible, particularly if the AV program has been updated. If so, turning it off is typically not enough to avoid the problems introduced by it though. You may need to uninstall it (and reboot) to completely rule it out. Okay - I'll try this too. Thank you, Tom Baker -- Tom Baker - [EMAIL PROTECTED] - [EMAIL PROTECTED] -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Problem reports: http://cygwin.com/problems.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/
Re: Reliable old script loses data on new Cygwin installation
On Wed, Nov 14, 2007 at 10:07:09AM +0100, Corinna Vinschen wrote: I would be grateful if someone could suggest a way to test this. Would it make sense (and is it possible) to replace the Cygwin kernel or the coreutils package (because of mv) with earlier versions and see if the script works then? You should first create a minimal, self-contained script which allows to reproduce the effect reliably on your machines. A really minimal script. Self-contained. So that everybody can run it to try to reproduce the effect without relying on your environment. Okay. Then you could try to change the versions of coreutils and gawk. At no point in this thread did you mention different Cygwin versions. Are you running 1.5.24 on all of them including your laptop? If so, there's no point in experimenting with different Cygwin DLLs. On my laptop, I am using (uname -srv): CYGWIN_NT-5.1 1.5.22(0.156/4/2) 2006-11-13 17:01 (FWIW, I agree with Igor Pechtchanski's suggestion [1] that the Cygwin FAQ include more detail on how to check version [2].) [1] http://www.cygwin.com/ml/cygwin-xfree/2004-02/msg00121.html [2] http://www.cygwin.com/faq/faq.html#faq.what.version If you found the difference, you could send the observation here together with your really minimal, self-contained script, and a quick description what to do to reproduce the effect and how it can be observed. Thanks, understood - I'll give this a try. Tom Baker -- Tom Baker - [EMAIL PROTECTED] - [EMAIL PROTECTED] -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Problem reports: http://cygwin.com/problems.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/
Re: Reliable old script loses data on new Cygwin installation
On Nov 14 08:22, Thomas Baker wrote: I would be grateful if someone could suggest a way to test this. Would it make sense (and is it possible) to replace the Cygwin kernel or the coreutils package (because of mv) with earlier versions and see if the script works then? You should first create a minimal, self-contained script which allows to reproduce the effect reliably on your machines. A really minimal script. Self-contained. So that everybody can run it to try to reproduce the effect without relying on your environment. Then you could try to change the versions of coreutils and gawk. At no point in this thread did you mention different Cygwin versions. Are you running 1.5.24 on all of them including your laptop? If so, there's no point in experimenting with different Cygwin DLLs. If you found the difference, you could send the observation here together with your really minimal, self-contained script, and a quick description what to do to reproduce the effect and how it can be observed. Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Project Co-Leader cygwin AT cygwin DOT com Red Hat -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Problem reports: http://cygwin.com/problems.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/
Re: Reliable old script loses data on new Cygwin installation
These machines have an anti-virus program, but the same one I have been using for the past two or three years. The filenames either have no extension, or .txt. That's most likely the problem anyway, what happens if you turn it off? -Lewis -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Problem reports: http://cygwin.com/problems.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/
Re: Reliable old script loses data on new Cygwin installation
Lewis Hyatt wrote: These machines have an anti-virus program, but the same one I have been using for the past two or three years. The filenames either have no extension, or .txt. That's most likely the problem anyway, what happens if you turn it off? This is possible, particularly if the AV program has been updated. If so, turning it off is typically not enough to avoid the problems introduced by it though. You may need to uninstall it (and reboot) to completely rule it out. -- Larry Hall http://www.rfk.com RFK Partners, Inc. (508) 893-9779 - RFK Office 216 Dalton Rd. (508) 893-9889 - FAX Holliston, MA 01746 _ A: Yes. Q: Are you sure? A: Because it reverses the logical flow of conversation. Q: Why is top posting annoying in email? -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Problem reports: http://cygwin.com/problems.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/
Re: Reliable old script loses data on new Cygwin installation
Morgan Gangwere wrote: /!\ WOOGA WOOGA /!\ Possible Stupid human Error! /!\ OOOPS OOOPS /!\ if the copy of xp is new enough, its got a funky little tool called the Windows Malware Detection And removal Tool (a freind of mine who is an MS MVP has had the same problem as you just under a different circumstance.) I took an sp1 machine, created some text files that i wanted to search for and got no problems on an athlon machine (2.4Ghz) I then upgraded to sp2 -- same machine, same 200gb sata hdd and everything no problems then i let it run its updates BAM trace shows: its the windows malware detector (aka windows defender or whatever you want to call it.) also, avg can do this too try moving 200 files with mv -r /tmp/std-output/* /logs/$$/, each about 30kb each One other thing I wanted to add to this was make sure the drives are not overheating. Some drives will actually shut down or begin to act erratically if they get too hot. I discovered this when doing work with my external drive (notebook ide using usb adapter). P.S. - The reason for this issue was because I had the drive near an exhaust vent for my laptop -- mistake :( It is normally fast enough to handle high speed burns or file transfers as necessary. -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Problem reports: http://cygwin.com/problems.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/
Re: Reliable old script loses data on new Cygwin installation
On Mon, Nov 12, 2007 at 04:03:14AM -0600, René Berber wrote: [snip] However, the other problem (see below) has occurred -- sporadically -- on three different machines, all running German or English-language versions of XP, two with SATA disks and one with an ATA disk, all with freshly downloaded installations of cygwin.[snip] OK, that's interesting, 3 machines one using non SATA disk. Same irregular pattern of 10 to 20 runs without problem? I haven't run it enough to see clear patterns, but yes, the error occurs in an irregular manner. Since then I have tested this on a fourth machine -- my old machine, with newly installed XP and freshly downloaded Cywin but a slower hard disk -- and I get the same irregular errors there too! No. The actual storage of data is done by the operating system, not by Cygwin, the operating system provides the data even if it hasn't been stored on disk (it could still be in a memory buffer). Okay, thank you for the explanation. The speed is not the problem, it could be the usual suspect: an anti-virus, unlikely because the data written is not executable but it could be adding an extraneous delay between data written and data read. I'll ask the guy who installed the operating systems whether this looks possible. The only solid evidence is the error message from Windows, and it says device, that means that the hard disk is having problems (it could be the driver) but not your script or any program. Did you do anything special to the hard disks on all 3 machines? something like run an optimizer or tune parameters? I'll find out. The other problem could be related to the first, if the disk is failing then creating/moving a file will fail, I'm not sure if permission denied will be the error message but I could test that later (I can make my SATA disk fail using a program that allows the async unbuffered I/O). The /bin/pdksh script sequence that is causing problems is: 1. gawk '$1 !~ /blahblah/' huh2 |/tmp/shf2080.tmp (no error messages) 2. mv /tmp/shf2080.tmp huh2 mv: cannot create regular file `huh2': Permission denied 3. (some other commands in a looping construct) 4. gawk '$1 !~ /blahblah/' huh2 |/tmp/shf2080.tmp gawk: cmd. line:1: fatal: cannot open file `huh2' for reading (No such file or directory) Sometimes the sequence runs correct. Sometimes it results in a file huh2 with length 0. But the sequence has _sometimes_ resulted in the creation of files such as: - 2007-10-28 20:31 4010 german which _appear_ to have a length of 4010, but where od german shows: 000 00 00 00 00 00 00 00 00 * 0007640 00 00 00 00 00 0007652 I'll get answers to the questions above... Tom Baker -- Tom Baker - [EMAIL PROTECTED] - [EMAIL PROTECTED] -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Problem reports: http://cygwin.com/problems.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/
Re: Reliable old script loses data on new Cygwin installation
On Mon, Nov 12, 2007 at 09:38:03PM -0500, Morgan Gangwere wrote: if the copy of xp is new enough, its got a funky little tool called the Windows Malware Detection And removal Tool (a freind of mine who is an MS MVP has had the same problem as you just under a different circumstance.) I'll find out - thanks. Tom Baker -- Tom Baker - [EMAIL PROTECTED] - [EMAIL PROTECTED] -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Problem reports: http://cygwin.com/problems.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/
Re: Reliable old script loses data on new Cygwin installation
On Tue, Nov 13, 2007 at 09:58:31AM -0500, Robert Pendell wrote: One other thing I wanted to add to this was make sure the drives are not overheating. Some drives will actually shut down or begin to act erratically if they get too hot. I discovered this when doing work with my external drive (notebook ide using usb adapter). P.S. - The reason for this issue was because I had the drive near an exhaust vent for my laptop -- mistake :( It is normally fast enough to handle high speed burns or file transfers as necessary. Thank you for the suggestion, but little script does its work in 30 seconds or so, so the problem is occurring when the computer has been moving a few little files around for just a few minutes at most. Tom Baker -- Tom Baker - [EMAIL PROTECTED] - [EMAIL PROTECTED] -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Problem reports: http://cygwin.com/problems.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/
Re: Reliable old script loses data on new Cygwin installation
On Tue, Nov 13, 2007 at 06:06:21PM +0100, Thomas Baker wrote: if the copy of xp is new enough, its got a funky little tool called the Windows Malware Detection And removal Tool (a freind of mine who is an MS MVP has had the same problem as you just under a different circumstance.) I confirmed that there is no Malware tool on these machines, so that cannot be the explanation. Tom -- Tom Baker - [EMAIL PROTECTED] - [EMAIL PROTECTED] -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Problem reports: http://cygwin.com/problems.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/
Re: Reliable old script loses data on new Cygwin installation
On Tue, Nov 13, 2007 at 05:59:01PM +0100, Thomas Baker wrote: The speed is not the problem, it could be the usual suspect: an anti-virus, unlikely because the data written is not executable but it could be adding an extraneous delay between data written and data read. I'll ask the guy who installed the operating systems whether this looks possible. These machines have an anti-virus program, but the same one I have been using for the past two or three years. The filenames either have no extension, or .txt. The only solid evidence is the error message from Windows, and it says device, that means that the hard disk is having problems (it could be the driver) but not your script or any program. Did you do anything special to the hard disks on all 3 machines? something like run an optimizer or tune parameters? I'll find out. The guy who set up the machines did not fiddle with the hard disk at all. Two of the machines were brand-new. On these, he did not touch the hardware - only swapped the installed German-language version of XP for my own version of XP in English. Then I installed Cygwin from the net. I'm a bit at a loss as to what to do next. I use this one script so extensively (50 times per day?) that I would sooner move to a Linux or Macintosh system than continue on Windows without Cygwin - my whole working style depends on it. I work on a laptop with Cygwin and an up-to-date version of XP, and thankfully the script continues to run fine there. Having (seemingly) eliminated SATA/ATA, fast/slow, and old-XP/up-to-date-XMP as possible explanations for the errors, the only really obvious remaining difference between my laptop (where the script works) and the four desktop computers (where the script does not work reliably) is that the four desktop computers have the latest version of Cygwin and the laptop has a version that I have not updated in awhile (and now I do not dare!). I would be grateful if someone could suggest a way to test this. Would it make sense (and is it possible) to replace the Cygwin kernel or the coreutils package (because of mv) with earlier versions and see if the script works then? The version of pdksh is the same between the laptop and the desktops, so I assume this can be eliminated as the cause? The script itself is not long, and the data sets are also just a few pages long, so I could prepare a tar file with a simple test installation if anyone would like to try it on their machines. Tom -- Tom Baker - [EMAIL PROTECTED] - [EMAIL PROTECTED] -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Problem reports: http://cygwin.com/problems.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/
Re: Reliable old script loses data on new Cygwin installation
René Berber wrote: [snip] I have searched FAQs and mailing lists for problems with timeout and the like but find nothing obviously relevant. [snip] I have seen that problem and it has nothing to do with Cygwin. The problem is with SATA drives and Window's asynchronous unbuffered disk I/O, and my Adaptec 1210SA SATA card and driver (actually, I know the driver is the culprit, but newer drivers are so bad that they can't even be installed). With application that use asynchronous unbuffered disk I/O the disk stops responding after a while, and Windows pops up an error panel that shows the timeout message. How does is happen in your setting? I don't know. Notice that I said applications, Windows doesn't use the problematic mode, I've only seen a couple of applications using it when you configure them to use the fastest disk I/O possible, one used it all the time (Azureus?) so it was unusable with that disk. Could the problem be caused by something else that runs at the same time as your script? Thank you, René. At first we did think it was a problem with the SATA disk -- and it does seem like a plausible explanation for the error message: Windows - Device TimeOut The specified I/O operation on \Device\Harddisk7\DR10 was not completed before the time-out period expired. However, the other problem (see below) has occurred -- sporadically -- on three different machines, all running German or English-language versions of XP, two with SATA disks and one with an ATA disk, all with freshly downloaded installations of cygwin. The line that causes the problem is: gawk '$1 !~ /^LINX/ $3' |/tmp/sht2080.tmp; mv /tmp/sht2080.tmp huh2 ... What I get is error messages like the following: mv: cannot create regular file `huh2': Permission denied gawk: cmd. line:1: fatal: cannot open file `huh2' for reading (No such file or directory) gawk: cmd. line:1: fatal: cannot open file `huh2' for reading (No such file or directory) What I then find is that data has been lost. If I interrupt the script right after the error message I find files (such as huh2) that have a length of zero -- OR I find a file listed with a correct-looking length but garbage contents. For example, the text file (before running the script): - 2007-10-28 20:25 4010 german comes out the other end looking like - 2007-10-28 20:31 4010 german but od german shows the _entire_ contents of the file to be: 000 00 00 00 00 00 00 00 00 * 0007640 00 00 00 00 00 0007652 It seems plausible (to me as a non-expert) that an asynchronous unbuffered disk could be responsible for this problem too. However, I am getting this error on _also_ on an older machine with an ATA disk. The three test machines on which the problem is occurring have two things in common: -- They all have some version of XP with the most recent Cygwin installation plus Firefox, OpenOffice, and Java and nothing else. -- They are all faster than the machines I have been using over the years. A colleague of mine suspects that the Korn shell script on Cygwin is running so fast that occasionally the next command is being executed before the buffer is written to disk. Is it possible that the shell is somehow creating the file german (above), with its file name and length, a split second before the contents are written to disk, then the next command is being run too quickly, the script gets tripped up but keeps running, and data is lost? As this is happening both on a SATA disk and an ATA disk, I can't help wondering whether cygwin is perhaps too efficient for the faster hardware. My colleague suggests I modify the script to add 500 milliseconds of wait time between gawk '$1 !~ /^LINX/ $3' |/tmp/sht2080.tmp and mv /tmp/sht2080.tmp huh2 However, he says that this could conceivably solve the problem for this script, but if the problem is that Cygwin is too fast for the hardware I could still get this problem while using, say, mv. Can this explanation be ruled out? Tom -- Tom Baker - [EMAIL PROTECTED] - [EMAIL PROTECTED] -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Problem reports: http://cygwin.com/problems.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/
Re: Reliable old script loses data on new Cygwin installation
Thomas Baker wrote: [snip] However, the other problem (see below) has occurred -- sporadically -- on three different machines, all running German or English-language versions of XP, two with SATA disks and one with an ATA disk, all with freshly downloaded installations of cygwin.[snip] OK, that's interesting, 3 machines one using non SATA disk. Same irregular pattern of 10 to 20 runs without problem? A colleague of mine suspects that the Korn shell script on Cygwin is running so fast that occasionally the next command is being executed before the buffer is written to disk. Is it possible that the shell is somehow creating the file german (above), with its file name and length, a split second before the contents are written to disk, then the next command is being run too quickly, the script gets tripped up but keeps running, and data is lost? No. The actual storage of data is done by the operating system, not by Cygwin, the operating system provides the data even if it hasn't been stored on disk (it could still be in a memory buffer). As this is happening both on a SATA disk and an ATA disk, I can't help wondering whether cygwin is perhaps too efficient for the faster hardware.[snip] However, he says that this could conceivably solve the problem for this script, but if the problem is that Cygwin is too fast for the hardware I could still get this problem while using, say, mv. Can this explanation be ruled out? Move is different, it usually only renames the file, but does a copy if necessary. But again, it is the operating system the one that does the actual change, what any program does is usually to a buffer in memory which is marked to be stored by the operating system. The speed is not the problem, it could be the usual suspect: an anti-virus, unlikely because the data written is not executable but it could be adding an extraneous delay between data written and data read. The only solid evidence is the error message from Windows, and it says device, that means that the hard disk is having problems (it could be the driver) but not your script or any program. Did you do anything special to the hard disks on all 3 machines? something like run an optimizer or tune parameters? The other problem could be related to the first, if the disk is failing then creating/moving a file will fail, I'm not sure if permission denied will be the error message but I could test that later (I can make my SATA disk fail using a program that allows the async unbuffered I/O). -- René Berber -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Problem reports: http://cygwin.com/problems.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/
Re: Reliable old script loses data on new Cygwin installation
/!\ WOOGA WOOGA /!\ Possible Stupid human Error! /!\ OOOPS OOOPS /!\ if the copy of xp is new enough, its got a funky little tool called the Windows Malware Detection And removal Tool (a freind of mine who is an MS MVP has had the same problem as you just under a different circumstance.) I took an sp1 machine, created some text files that i wanted to search for and got no problems on an athlon machine (2.4Ghz) I then upgraded to sp2 -- same machine, same 200gb sata hdd and everything no problems then i let it run its updates BAM trace shows: its the windows malware detector (aka windows defender or whatever you want to call it.) also, avg can do this too try moving 200 files with mv -r /tmp/std-output/* /logs/$$/, each about 30kb each -- Morgan gangwere Please Excuse TOFU. Gmail/Mobile has no Power. Space does not reflect society, it expresses it. -- Castells, M., Space of Flows, Space of Places: Materials for a Theory of Urbanism in the Information Age, in The Cybercities Reader, S. Graham, Editor. 2004, Routledge: London. p. 82-93. -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Problem reports: http://cygwin.com/problems.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/
Reliable old script loses data on new Cygwin installation
Dear all, I have been using a Korn shell script twenty times per day for more than ten years -- I published an earlier version in UnixWorld in 1994 and the latest version last year in lifehacker.com [1]. When I now run the script on brand-new Cygwin installations, it loses data. I have tested the script on fresh data many times and find that it may run correctly twenty times and then fail. Then it will run correctly ten more times and fail, but at a different point in processing the data. The line of the script that is causing the failure is: gawk '$'$Searchfield' !~ /'$Searchkey'/' $3 |$Tmpfile; mv $Tmpfile $3 (it gets STDOUT from a previous line), where: Tmpdir=/tmp Tmpfile=$Tmpdir/shf$$.tmp(i.e., /tmp/shf2080.tmp) i.e.: gawk '$1 !~ /^LINX/ $3' |/tmp/sht2080.tmp; mv /tmp/sht2080.tmp huh2 What I get is error messages like the following: mv: cannot create regular file `huh2': Permission denied gawk: cmd. line:1: fatal: cannot open file `huh2' for reading (No such file or directory) gawk: cmd. line:1: fatal: cannot open file `huh2' for reading (No such file or directory) What I then find is that data has been lost. If I interrupt the script right after the error message I find files (such as huh2) that have a length of zero -- OR I find a file listed with a correct-looking length but garbage contents. For example, the text file (before running the script): - 2007-10-28 20:25 4010 german comes out the other end looking like - 2007-10-28 20:31 4010 german but od german shows the _entire_ contents of the file to be: 000 00 00 00 00 00 00 00 00 * 0007640 00 00 00 00 00 0007652 I have used this script on many machines and platforms over the years (Linux, Solaris, DOS 3.3 with MKS Toolkit, Win95, Win2000, XP). It is now failing on two different WinXP computers on which I have just installed Cygwin from the Web. -- one brand-new (with an English-language XP and a SATA hard disk) and one two years old (with a German-language-XP and an ATA hard disk). One of the machines with the new Cygwin installation also lost lots of data when I scheduled it to move a large subdirectory (using mv) in the middle of the night. The message I found on my return was: Windows - Device TimeOut The specified I/O operation on \Device\Harddisk7\DR10 was not completed before the time-out period expired. Comparing the packages installed on my laptop (which has no problems with mv) with the packages on the two computers with the new Cygwin installation, I see: Older (works) Newer installation (broken) coreutils 6.4-1 6.9-5 gawk3.1.5-4 3.1.6-1 pdksh 5.2.14-3 5.2.14-3 The two computers on which I am getting errors have the following in common: -- Their processors and hard disks are faster than *most* of the computers I have been using (both are generic home-office PCs from Siemens-Fujitsu). -- Both now have the latest version of Cygwin from the Web. I have searched FAQs and mailing lists for problems with timeout and the like but find nothing obviously relevant. A colleague of mine who knows alot more about these things is scratching his head on this. Is it possible that something in the more recent versions of Cygwin packages is causing this problem? Many thanks, Tom Baker [1] http://lifehacker.com/software/command-line/getting-things-done-with-rulebased-list-processing-217063.php -- Tom Baker - [EMAIL PROTECTED] - [EMAIL PROTECTED] -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Problem reports: http://cygwin.com/problems.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/
Reliable old script loses data on new Cygwin installation
Dear all, I have been using a Korn shell script twenty times per day for more than ten years -- I published an earlier version in UnixWorld in 1994 and the latest version last year in lifehacker.com [1]. When I now run the script on brand-new Cygwin installations, it loses data. I have tested the script on fresh data many times and find that it may run correctly twenty times and then fail. Then it will run correctly ten more times and fail, but at a different point in processing the data. The line of the script that is causing the failure is: gawk '$'$Searchfield' !~ /'$Searchkey'/' $3 |$Tmpfile; mv $Tmpfile $3 (it gets STDOUT from a previous line), where: Tmpdir=/tmp Tmpfile=$Tmpdir/shf$$.tmp(i.e., /tmp/shf2080.tmp) i.e.: gawk '$1 !~ /^LINX/ $3' |/tmp/sht2080.tmp; mv /tmp/sht2080.tmp huh2 What I get is error messages like the following: mv: cannot create regular file `huh2': Permission denied gawk: cmd. line:1: fatal: cannot open file `huh2' for reading (No such file or directory) gawk: cmd. line:1: fatal: cannot open file `huh2' for reading (No such file or directory) What I then find is that data has been lost. If I interrupt the script right after the error message I find files (such as huh2) that have a length of zero -- OR I find a file listed with a correct-looking length but garbage contents. For example, the text file (before running the script): - 2007-10-28 20:25 4010 german comes out the other end looking like - 2007-10-28 20:31 4010 german but od german shows the _entire_ contents of the file to be: 000 00 00 00 00 00 00 00 00 * 0007640 00 00 00 00 00 0007652 I have used this script on many machines and platforms over the years (Linux, Solaris, DOS 3.3 with MKS Toolkit, Win95, Win2000, XP). It is now failing on two different WinXP computers on which I have just installed Cygwin from the Web. -- one brand-new (with an English-language XP and a SATA hard disk) and one two years old (with a German-language-XP and an ATA hard disk). One of the machines with the new Cygwin installation also lost lots of data when I scheduled it to move a large subdirectory (using mv) in the middle of the night. The message I found on my return was: Windows - Device TimeOut The specified I/O operation on \Device\Harddisk7\DR10 was not completed before the time-out period expired. Comparing the packages installed on my laptop (which has no problems with mv) with the packages on the two computers with the new Cygwin installation, I see: Older (works) Newer installation (broken) coreutils 6.4-1 6.9-5 gawk3.1.5-4 3.1.6-1 pdksh 5.2.14-3 5.2.14-3 The two computers on which I am getting errors have the following in common: -- Their processors and hard disks are faster than *most* of the computers I have been using (both are generic home-office PCs from Siemens-Fujitsu). -- Both now have the latest version of Cygwin from the Web. I have searched FAQs and mailing lists for problems with timeout and the like but find nothing obviously relevant. A colleague of mine who knows alot more about these things is scratching his head on this. Is it possible that something in the more recent versions of Cygwin packages is causing this problem? Many thanks, Tom Baker [1] http://lifehacker.com/software/command-line/getting-things-done-with-rulebased-list-processing-217063.php -- Tom Baker - [EMAIL PROTECTED] - [EMAIL PROTECTED] -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Problem reports: http://cygwin.com/problems.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/
Re: Reliable old script loses data on new Cygwin installation
Thomas Baker wrote: [snip] I have searched FAQs and mailing lists for problems with timeout and the like but find nothing obviously relevant. [snip] I have seen that problem and it has nothing to do with Cygwin. The problem is with SATA drives and Window's asynchronous unbuffered disk I/O, and my Adaptec 1210SA SATA card and driver (actually, I know the driver is the culprit, but newer drivers are so bad that they can't even be installed). With application that use asynchronous unbuffered disk I/O the disk stops responding after a while, and Windows pops up an error panel that shows the timeout message. How does is happen in your setting? I don't know. Notice that I said applications, Windows doesn't use the problematic mode, I've only seen a couple of applications using it when you configure them to use the fastest disk I/O possible, one used it all the time (Azureus?) so it was unusable with that disk. Could the problem be caused by something else that runs at the same time as your script? Hope this helps. -- René Berber -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Problem reports: http://cygwin.com/problems.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/
