Looking at the code and with some debugging I put in, it seems that
retries is not incremented at line 282 in ftpclass.cc due to the 426
returned for the STOR of the truncated file. Of course, with my limited
knowledge of the code, I could be way out to lunch on this analysis.
Alexander: do you see what I'm talking about with regards to retries not
incrementing and thus the looping condition?
As an alternative to fixing the retries not incrementing, which would be
preferable, I have patched ftpclass.cc with the following:
--- ftpclass.cc.bak Thu Apr 15 11:58:29 2010
+++ ftpclass.cc Thu Apr 15 11:59:54 2010
@@ -264,7 +264,8 @@
state=EOF_STATE; // retry
return;
}
- if(is5XX(act) && !Transient5XX(act))
+ if((is5XX(act) && !Transient5XX(act))
+ || is4XX(act))
{
SetError(NO_FILE,all_lines);
return;
I've taken out the persist/reconnect lines in the command stream and
max-retries is back to 1 but this makes no difference since retries is
not incrementing anyway.
The result is now this:
1 lftp-ourpatch <<Ctrl-D
2 debug 10
3 set cmd:trace yes
4 set cmd:at-exit "kill all"
5 set cmd:fail-exit no
6 set cmd:move-background no
7 set cmd:save-cwd-history no
8 set cmd:save-rl-history no
9 set cmd:verbose no
10 set dns:fatal-timeout 10s
11 set ftp:passive-mode on
12 set net:max-retries 1
13 #set net:max-retries 3
14 #set net:persist-retries 1
15 #set net:reconnect-interval-base 60
16 #set net:reconnect-interval-max 60
17 #set net:reconnect-interval-multiplier 1
18 set xfer:log no
19 set cmd:trace yes
20 set cmd:default-protocol ftp
21 open ibm01 || exit 255
22 set cmd:trace no
23 user ${IDPASSWD}
24 set cmd:trace yes
25 ls > /dev/null || exit 254
26 put -a file_will_not_truncate.rpt -o
SRAYROVER/SRY260E18.S000000000 || exit 251
27 put -a file_will_not_truncate.rpt -o
SRAYROVER/SRY260E18.S000000000 || exit 251
28 put -a file_will_truncate.rpt -o
SRAYROVER/SRY260E18.S000000001 || exit 252
29 quit
30 Ctrl-D
31 notice: cannot open //.netrc: No such file or directory
32 ---- Resolving host address...
33 ---- 1 address found: 10.0.0.1
34 notice: cannot open //.netrc: No such file or directory
35 ---- dns cache hit
36 ---- Connecting to ibm01 (10.0.0.1) port 21
37 <--- 220-QTCP at IBM01.
38 <--- 220 Connection will close if idle more than 5 minutes.
39 ---> FEAT
40 <--- 211-Feature listing follows:
41 <--- AUTH TLS
42 <--- CCC
43 <--- PBSZ
44 <--- PROT
45 <--- 211 End of feature listing.
46 ---> AUTH TLS
47 <--- 431 Requested security mechanism not available at this
time.
48 ---> USER ftp1
49 <--- 331 Enter password.
50 ---> PASS mypassword
51 <--- 230 FTP1 logged on.
52 ---> PWD
53 <--- 257 "QGPL" is current library.
54 ---> PASV
55 <--- 227 Entering Passive Mode (10,0,0,1,59,234).
56 ---- Connecting data socket to (10.0.0.1) port 15338
57 ---- Data connection established
58 ---> LIST
59 <--- 125 List started.
60 <--- 250 List completed.
61 ---- Got EOF on data connection
62 ---- Closing data socket
First put works as expected:
63 copy: get hit eof
64 copy: waiting for put confirmation
65 copy: put confirmed store
66 copy: get is finished - all done
67 copy: get hit eof
68 copy: waiting for put confirmation
69 ---> PASV
70 <--- 227 Entering Passive Mode (10,0,0,1,151,9).
71 ---- Connecting data socket to (10.0.0.1) port 38665
72 ---- Data connection established
73 ---> ALLO 206
74 <--- 202 ALLO subcommand not required.
75 ---> STOR SRAYROVER/SRY260E18.S000000000
76 <--- 150 Sending file to member S000000000 in file SRY260E18 in
library SRAYROVER.
77 ---- Closing data socket
78 <--- 226 File transfer completed successfully.
Second put works as expected:
79 copy: put confirmed store
80 copy: get is finished - all done
81 copy: get hit eof
82 copy: waiting for put confirmation
83 ---> PASV
84 <--- 227 Entering Passive Mode (10,0,0,1,159,193).
85 ---- Connecting data socket to (10.0.0.1) port 40897
86 ---- Data connection established
87 ---> STOR SRAYROVER/SRY260E18.S000000000
88 <--- 150 Sending file to member S000000000 in file SRY260E18 in
library SRAYROVER.
89 ---- Closing data socket
90 <--- 226 File transfer completed successfully.
Third put fails as expected and the || exit 252 executes as we desire:
91 copy: put confirmed store
92 copy: get is finished - all done
93 copy: get hit eof
94 copy: waiting for put confirmation
95 ---> PASV
96 <--- 227 Entering Passive Mode (10,0,0,1,247,192).
97 ---- Connecting data socket to (10.0.0.1) port 63424
98 ---- Data connection established
99 ---> STOR SRAYROVER/SRY260E18.S000000001
100 <--- 150 Sending file to member S000000001 in file SRY260E18 in
library SRAYROVER.
101 ---- Closing data socket
102 <--- 426-Records written to file SRY260E18 in library SRAYROVER
have been truncated. Data in file may not be valid.
103 <--- 426 Data transfer ended.
104 put: Access failed: 426-Records written to file SRY260E18 in
library SRAYROVER have been truncated. Data in file may not be valid.
Data transfer ended. (SRAYROVER/SRY260E18.S000000001)
105 ---> QUIT
106 ---- Closing control socket
107 + set cmd:at-exit "kill all"
108 + set cmd:fail-exit no
109 + set cmd:move-background no
110 + set cmd:save-cwd-history no
111 + set cmd:save-rl-history no
112 + set cmd:verbose no
113 + set dns:fatal-timeout 10s
114 + set ftp:passive-mode on
115 + set net:max-retries 1
116 + set xfer:log no
117 + set cmd:trace yes
118 + set cmd:default-protocol ftp
119 + open ibm01
120 + set cmd:trace no
121 + ls
122 + put -a file_will_not_truncate.rpt -o
SRAYROVER/SRY260E18.S000000000
123 + put -a file_will_not_truncate.rpt -o
SRAYROVER/SRY260E18.S000000000
124 + put -a file_will_truncate.rpt -o
SRAYROVER/SRY260E18.S000000001
125 + exit 252
126 + kill all
127 echo lftp exist status $?
128 lftp exist status 252
Alexander: I would value your comments on the validity of this
workaround and the retries=0 findings.
Thanks!
Curtis
