Re: [MBS] Resuming uploads with SFTP

[Jerry Fritschle]

> On Nov 19, 2017, at 5:24 PM, Christian Schmitz 
>  wrote:
> 
> Well, I think you could move to CURLSFileInfoMBS class for FTP listing 
> instead.
> It does parsing better than FTPParse class.

Thank you.  I tried your example.  It works great with FTP.  Ironically, that 
is where I don’t need it for the purposes of this project.  Again, my aim is 
just to get the remote size in advance,  so I can tell SFTP to skip, and not 
try to append, any completed file.

For SFTP, curl returns Error 18 (closed with file length remaining).  If I take 
out the wildcard stuff, I get error 0 but blank FileInfo—same with FTP.

Now, I have no problem getting the string from OutputData.  So after splitting 
it into an array at the endOfLines (called “Stuff” here, with f as the local 
file), I can do this:
 
Dim x, u as integer
Dim canUpload as Boolean
u=ubound(Stuff)

For x=0 to u

If instr(Stuff(x),f.name)>0 then 
//exists on server
dim s as string=str(f.length)
if instr(Stuff(x),s)=0 then
//needs appending
canUpload=true
else
//it finished
 canUpload=False
end if
exit
else
//haven’t started it yet
canUpload=true
end if

Next

If canUpload=true then
//perform
else
//skip and go to next file in batch
End if

Now, this actually works in my tests, but I also feel that it would get me a 
special exhibit at the World Museum of Kludges, and I’m not that desperate to 
be famous. :-)  The other approach is to keep my own log in ApplicationData, 
and work from that when trying to resume.
-
"Drama is life with the dull bits cut out."
-Alfred Hitchcock

Jerry Fritschle
jerryfritsc...@icloud.com



___
Mbsplugins_monkeybreadsoftware.info mailing list
mbsplugins@monkeybreadsoftware.info
https://ml01.ispgateway.de/mailman/listinfo/mbsplugins_monkeybreadsoftware.info

Re: [MBS] Resuming uploads with SFTP

[Christian Schmitz]

> Am 19.11.2017 um 23:37 schrieb Jerry Fritschle :
> 
> I am still at this point:  that if I try to rerun an interrupted batch upload 
> with SFTP, it will corrupt any file that is already complete, when it tries 
> to append.  So I have thought of a couple of workarounds.

Sounds like we have a bug in resume.
But of course you may decide to not update if file is complete.

> FTParseMBS still appears in the online documentation.  However, it is also 
> said to derive from the Network plugin, and if I look there it is not listed. 
>  So is this just a case of the documentation needing to be updated?  If so, I 
> have another way to do this.

Well, I think you could move to CURLSFileInfoMBS class for FTP listing instead.
It does parsing better than FTPParse class.

Old documentation stays online as I don’t delete files usually.
So the links go away and over time the content is no longer found.

Sincerely
Christian

-- 
Read our blog about news on our plugins:

http://www.mbsplugins.de/


___
Mbsplugins_monkeybreadsoftware.info mailing list
mbsplugins@monkeybreadsoftware.info
https://ml01.ispgateway.de/mailman/listinfo/mbsplugins_monkeybreadsoftware.info

Re: [MBS] Resuming uploads with SFTP

[Jerry Fritschle]

I am still at this point:  that if I try to rerun an interrupted batch upload 
with SFTP, it will corrupt any file that is already complete, when it tries to 
append.  So I have thought of a couple of workarounds.

One of these was to get a directory listing in advance, parse the sizes, and 
skip over anything that appears complete (at least in the SFTP branching code, 
since, again, FTP does not need this.)  For this I was going to use 
FTPparseMBS, which I’ve used in the past.  It has been a few years, as most of 
my need for it went away when I transitioned my FTP operations to libcurl.  
However, it appears to be gone now.

FTParseMBS still appears in the online documentation.  However, it is also said 
to derive from the Network plugin, and if I look there it is not listed.  So is 
this just a case of the documentation needing to be updated?  If so, I have 
another way to do this.



> On Nov 17, 2017, at 11:23 AM, Jerry Fritschle  wrote:
> 
> What fails is that, unlike FTP, if the file is already complete on the 
> server, it gets corrupted in the manner described (i.e, becomes an empty 
> file.)

-
"Drama is life with the dull bits cut out."
-Alfred Hitchcock

Jerry Fritschle
jerryfritsc...@icloud.com



___
Mbsplugins_monkeybreadsoftware.info mailing list
mbsplugins@monkeybreadsoftware.info
https://ml01.ispgateway.de/mailman/listinfo/mbsplugins_monkeybreadsoftware.info

Re: [MBS] Resuming uploads with SFTP

[Jerry Fritschle]

Okay, in the light of day and being a little fresher, I see what is going on.

Under SFTP, the resumption DOES work correctly, and the large test file, 
intentionally broken, is completed.

What fails is that, unlike FTP, if the file is already complete on the server, 
it gets corrupted in the manner described (i.e, becomes an empty file.)

With a batch (i.e, folder) upload, I don’t expect the end user to know exactly 
where the break occurred. With FTP we can restart the batch, and it simply 
passes over each completed file till it finds the point left off.

Gives me some ideas, and also gives me some incentive to put in a window to 
read debug data (this particular project is actually pretty new.)  Should have 
done that in the first place :-)

On Nov 17, 2017, at 1:54 AM, Christian Schmitz  
wrote:
> 
> As far as I see in library code, seeking for resume is implemented.
> Of course server needs to respond to file size query. If that fails, it 
> starts a new upload.

-
"Drama is life with the dull bits cut out."
-Alfred Hitchcock

Jerry Fritschle
jerryfritsc...@icloud.com



___
Mbsplugins_monkeybreadsoftware.info mailing list
mbsplugins@monkeybreadsoftware.info
https://ml01.ispgateway.de/mailman/listinfo/mbsplugins_monkeybreadsoftware.info

Re: [MBS] Resuming uploads with SFTP

[Christian Schmitz]

> 
> From what I’m gathering about SFTP,  I am wondering whether this is even 
> possible with libcurl. It is my understanding that its SFTP command 
> vocabulary is relatively basic and limited.  Using built-in SFTP in the 
> terminal, I am successful with the “reput” command.  So calling that from a 
> shell, on Mac at least, is a possibility.

As far as I see in library code, seeking for resume is implemented.
Of course server needs to respond to file size query. If that fails, it starts 
a new upload.

Sincerely
Christian

-- 
Read our blog about news on our plugins:

http://www.mbsplugins.de/


___
Mbsplugins_monkeybreadsoftware.info mailing list
mbsplugins@monkeybreadsoftware.info
https://ml01.ispgateway.de/mailman/listinfo/mbsplugins_monkeybreadsoftware.info

[MBS] Resuming uploads with SFTP

[Jerry Fritschle]

I am working on adding SFTP compatibility to my products that, through 
CURLSMBS, employ FTP.  I am testing against an up-to-date CrushFTP server that 
I administer myself, which is accessible from the internet.

For my needs, I have worked everything out except for successfully being able 
to resume broken uploads.  Under regular FTP all is well, but trying it with 
SFTP results in the remote files being corrupted, with a byte count of 0.  The 
ability to resume is important, because my apps are primarily for moving large 
image files.  I’m interested in the security angle for files that may contain 
personal information.

From what I’m gathering about SFTP,  I am wondering whether this is even 
possible with libcurl. It is my understanding that its SFTP command vocabulary 
is relatively basic and limited.  Using built-in SFTP in the terminal, I am 
successful with the “reput” command.  So calling that from a shell, on Mac at 
least, is a possibility.

I appreciate hearing from anyone with experience in this.  I can also create an 
account on my server for anyone who wants to play. But I mainly just want to 
know whether it can be done via the CURL plugin before I spend any more time on 
it.

Thanks.

Jerry

-
"Drama is life with the dull bits cut out."
-Alfred Hitchcock

Jerry Fritschle
jerryfritsc...@icloud.com



___
Mbsplugins_monkeybreadsoftware.info mailing list
mbsplugins@monkeybreadsoftware.info
https://ml01.ispgateway.de/mailman/listinfo/mbsplugins_monkeybreadsoftware.info