Re: Fetching directories inclusive subdirectories on HTTP server via fetch or othe FreeBSD-own tools?
O. Hartmann wrote: > I tried 'omi' but I find that the tool does not travers deeper into a > dir than level one, so subdirs seem to be left out. I will try wget, > although this tool would not be the first choice. Well, omi _does_ recurse into subdirectories, but it might fail if the FTP server has an unusual output format, or otherwise behaves in unexpected ways. Sometimes "-o nostat" helps to work around it. Especially if the FTP server runs on some non-UNIX environment and the directory output does not look like an "ls -l", omi has trouble parsing it. I've seen FTP servers running on VMS or Novell Netware that looked _really_ weird. These are not supported by omi, I'm afraid. Best regards Oliver -- Oliver Fromme, secnetix GmbH & Co. KG, Marktplatz 29, 85567 Grafing b. M. Handelsregister: Registergericht Muenchen, HRA 74606, Geschäftsfuehrung: secnetix Verwaltungsgesellsch. mbH, Handelsregister: Registergericht Mün- chen, HRB 125758, Geschäftsführer: Maik Bachmann, Olaf Erb, Ralf Gebhart FreeBSD-Dienstleistungen, -Produkte und mehr: http://www.secnetix.de/bsd "I made up the term 'object-oriented', and I can tell you I didn't have C++ in mind." -- Alan Kay, OOPSLA '97 ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"
Re: Fetching directories inclusive subdirectories on HTTP server via fetch or othe FreeBSD-own tools?
On 2/4/09 10:41, O. Hartmann wrote: > Oliver Fromme wrote: >> O. Hartmann wrote: >> > I run into a problem I can not solve. I need to fetch a whole >> directory > tree from a public remote site. The top level directory >> and its > subdirectories are accessible via ftp:// and http:// so I >> tried fetch, > but fetch does only retrieve data on file basis and >> does not copy a > whole directory tree recursively. The remote site >> does not offer > sftp/sshd for that purpose. >> > > Is there a simple way to perform such a task with FreeBSD's own >> tools (I > try to avoid installing 'wget' and sibblings)? I need to >> keep it simple, > task should be performed via cronjob. >> >> I'm afraid you can't do that with FreeBSD base tools. >> >> An alternative to wget would be "omi" (ports/ftp/omi) >> which is a simple FTP mirroring tool, written in C >> without any dependencies. Usage is simple: >> >> $ omi -s server.name.com -r /remote/dir -l ./local/dir >> >> Note that, by default, it tries to synchronize the local >> dir perfectly, i.e. if the remote dir is empty, it will >> wipe out the local dir. (The option "-P 0" will prevent >> omi from removing anything.) >> >> Best regards >>Oliver >> > > Thanks for so much answers. > > I tried 'omi' but I find that the tool does not travers deeper into a > dir than level one, so subdirs seem to be left out. I will try wget, > although this tool would not be the first choice. > I seem to remember that nctp3 had a decent -R recurse option to its get (or its ncftpget non interactive mode) if you want to avoid wget. Vince > > Thanks, > Oliver > ___ > freebsd-questions@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-questions > To unsubscribe, send any mail to > "freebsd-questions-unsubscr...@freebsd.org" ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"
Re: Fetching directories inclusive subdirectories on HTTP server via fetch or othe FreeBSD-own tools?
Oliver Fromme wrote: O. Hartmann wrote: > I run into a problem I can not solve. I need to fetch a whole directory > tree from a public remote site. The top level directory and its > subdirectories are accessible via ftp:// and http:// so I tried fetch, > but fetch does only retrieve data on file basis and does not copy a > whole directory tree recursively. The remote site does not offer > sftp/sshd for that purpose. > > Is there a simple way to perform such a task with FreeBSD's own tools (I > try to avoid installing 'wget' and sibblings)? I need to keep it simple, > task should be performed via cronjob. I'm afraid you can't do that with FreeBSD base tools. An alternative to wget would be "omi" (ports/ftp/omi) which is a simple FTP mirroring tool, written in C without any dependencies. Usage is simple: $ omi -s server.name.com -r /remote/dir -l ./local/dir Note that, by default, it tries to synchronize the local dir perfectly, i.e. if the remote dir is empty, it will wipe out the local dir. (The option "-P 0" will prevent omi from removing anything.) Best regards Oliver Thanks for so much answers. I tried 'omi' but I find that the tool does not travers deeper into a dir than level one, so subdirs seem to be left out. I will try wget, although this tool would not be the first choice. Thanks, Oliver ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"
RE: Fetching directories inclusive subdirectories on HTTP server via fetch or othe FreeBSD-own tools?
> "O. Hartmann" wrote: > > I need to fetch a whole directory tree from a public remote site. > > The top level directory and its subdirectories are accessible via > > ftp:// and http:// so I tried fetch, but fetch does only retrieve > > data on file basis and does not copy a whole directory tree > > recursively. The remote site does not offer sftp/sshd for that > > purpose. Wget --ftp-user=USER --ftp-password=PASS -r -l=0 ftp://address/directory L8rs! Marci ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"
Re: Fetching directories inclusive subdirectories on HTTP server via fetch or othe FreeBSD-own tools?
> I need to fetch a whole directory tree from a public remote site. > The top level directory and its subdirectories are accessible via > ftp:// and http:// so I tried fetch, but fetch does only retrieve > data on file basis and does not copy a whole directory tree > recursively. The remote site does not offer sftp/sshd for that > purpose. wget -r It's in the ports. Bests, Olivier ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"
Re: Fetching directories inclusive subdirectories on HTTP server via fetch or othe FreeBSD-own tools?
"O. Hartmann" wrote:
> I need to fetch a whole directory tree from a public remote site.
> The top level directory and its subdirectories are accessible via
> ftp:// and http:// so I tried fetch, but fetch does only retrieve
> data on file basis and does not copy a whole directory tree
> recursively. The remote site does not offer sftp/sshd for that
> purpose.
There's at least one ftp server (wuftpd) that will do it for you
on the server end. All you have to do is "get {dirname}.tar".
(The .tar file won't appear in the directory listing, because
it's created on the fly.)
It might be worth a try, to see if the server of interest happens
to be set up this way.
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"
Re: Fetching directories inclusive subdirectories on HTTP server via fetch or othe FreeBSD-own tools?
O. Hartmann wrote: > I run into a problem I can not solve. I need to fetch a whole directory > tree from a public remote site. The top level directory and its > subdirectories are accessible via ftp:// and http:// so I tried fetch, > but fetch does only retrieve data on file basis and does not copy a > whole directory tree recursively. The remote site does not offer > sftp/sshd for that purpose. > > Is there a simple way to perform such a task with FreeBSD's own tools (I > try to avoid installing 'wget' and sibblings)? I need to keep it simple, > task should be performed via cronjob. I'm afraid you can't do that with FreeBSD base tools. An alternative to wget would be "omi" (ports/ftp/omi) which is a simple FTP mirroring tool, written in C without any dependencies. Usage is simple: $ omi -s server.name.com -r /remote/dir -l ./local/dir Note that, by default, it tries to synchronize the local dir perfectly, i.e. if the remote dir is empty, it will wipe out the local dir. (The option "-P 0" will prevent omi from removing anything.) Best regards Oliver -- Oliver Fromme, secnetix GmbH & Co. KG, Marktplatz 29, 85567 Grafing b. M. Handelsregister: Registergericht Muenchen, HRA 74606, Geschäftsfuehrung: secnetix Verwaltungsgesellsch. mbH, Handelsregister: Registergericht Mün- chen, HRB 125758, Geschäftsführer: Maik Bachmann, Olaf Erb, Ralf Gebhart FreeBSD-Dienstleistungen, -Produkte und mehr: http://www.secnetix.de/bsd "That's what I love about GUIs: They make simple tasks easier, and complex tasks impossible." -- John William Chambless ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"
Re: Fetching directories inclusive subdirectories on HTTP server via fetch or othe FreeBSD-own tools?
On 1/4/09 17:07, O. Hartmann wrote: > Hello. > > I run into a problem I can not solve. I need to fetch a whole > directory tree from a public remote site. The top level directory and > its subdirectories are accessible via ftp:// and http:// so I tried > fetch, but fetch does only retrieve data on file basis and does not > copy a whole directory tree recursively. The remote site does not > offer sftp/sshd for that purpose. > > Is there a simple way to perform such a task with FreeBSD's own tools > (I try to avoid installing 'wget' and sibblings)? I need to keep it > simple, task should be performed via cronjob. > > Please eMail me. > Without writing a script to parse the output of fetch I dont think its possible (fetch the index via http, parse for dirs, decend into dir, rinse and repeat.) Looking at http://www.freebsd.org/doc/en/articles/hubs/mirror-howto.html which I would expect to suggest FreeBSDs own tools, suggests various ports to use. Vince > Thanks in advance, > > Oliver > ___ > freebsd-questions@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-questions > To unsubscribe, send any mail to > "freebsd-questions-unsubscr...@freebsd.org" ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"
Fetching directories inclusive subdirectories on HTTP server via fetch or othe FreeBSD-own tools?
Hello. I run into a problem I can not solve. I need to fetch a whole directory tree from a public remote site. The top level directory and its subdirectories are accessible via ftp:// and http:// so I tried fetch, but fetch does only retrieve data on file basis and does not copy a whole directory tree recursively. The remote site does not offer sftp/sshd for that purpose. Is there a simple way to perform such a task with FreeBSD's own tools (I try to avoid installing 'wget' and sibblings)? I need to keep it simple, task should be performed via cronjob. Please eMail me. Thanks in advance, Oliver ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"
