Re: Fetching directories inclusive subdirectories on HTTP server via fetch or othe FreeBSD-own tools?
O. Hartmann ohart...@zedat.fu-berlin.de wrote:
I need to fetch a whole directory tree from a public remote site.
The top level directory and its subdirectories are accessible via
ftp:// and http:// so I tried fetch, but fetch does only retrieve
data on file basis and does not copy a whole directory tree
recursively. The remote site does not offer sftp/sshd for that
purpose.
There's at least one ftp server (wuftpd) that will do it for you
on the server end. All you have to do is get {dirname}.tar.
(The .tar file won't appear in the directory listing, because
it's created on the fly.)
It might be worth a try, to see if the server of interest happens
to be set up this way.
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Fetching directories inclusive subdirectories on HTTP server via fetch or othe FreeBSD-own tools?
I need to fetch a whole directory tree from a public remote site. The top level directory and its subdirectories are accessible via ftp:// and http:// so I tried fetch, but fetch does only retrieve data on file basis and does not copy a whole directory tree recursively. The remote site does not offer sftp/sshd for that purpose. wget -r It's in the ports. Bests, Olivier ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
RE: Fetching directories inclusive subdirectories on HTTP server via fetch or othe FreeBSD-own tools?
O. Hartmann ohart...@zedat.fu-berlin.de wrote: I need to fetch a whole directory tree from a public remote site. The top level directory and its subdirectories are accessible via ftp:// and http:// so I tried fetch, but fetch does only retrieve data on file basis and does not copy a whole directory tree recursively. The remote site does not offer sftp/sshd for that purpose. Wget --ftp-user=USER --ftp-password=PASS -r -l=0 ftp://address/directory L8rs! Marci ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Fetching directories inclusive subdirectories on HTTP server via fetch or othe FreeBSD-own tools?
Oliver Fromme wrote: O. Hartmann ohart...@zedat.fu-berlin.de wrote: I run into a problem I can not solve. I need to fetch a whole directory tree from a public remote site. The top level directory and its subdirectories are accessible via ftp:// and http:// so I tried fetch, but fetch does only retrieve data on file basis and does not copy a whole directory tree recursively. The remote site does not offer sftp/sshd for that purpose. Is there a simple way to perform such a task with FreeBSD's own tools (I try to avoid installing 'wget' and sibblings)? I need to keep it simple, task should be performed via cronjob. I'm afraid you can't do that with FreeBSD base tools. An alternative to wget would be omi (ports/ftp/omi) which is a simple FTP mirroring tool, written in C without any dependencies. Usage is simple: $ omi -s server.name.com -r /remote/dir -l ./local/dir Note that, by default, it tries to synchronize the local dir perfectly, i.e. if the remote dir is empty, it will wipe out the local dir. (The option -P 0 will prevent omi from removing anything.) Best regards Oliver Thanks for so much answers. I tried 'omi' but I find that the tool does not travers deeper into a dir than level one, so subdirs seem to be left out. I will try wget, although this tool would not be the first choice. Thanks, Oliver ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Fetching directories inclusive subdirectories on HTTP server via fetch or othe FreeBSD-own tools?
On 2/4/09 10:41, O. Hartmann wrote: Oliver Fromme wrote: O. Hartmann ohart...@zedat.fu-berlin.de wrote: I run into a problem I can not solve. I need to fetch a whole directory tree from a public remote site. The top level directory and its subdirectories are accessible via ftp:// and http:// so I tried fetch, but fetch does only retrieve data on file basis and does not copy a whole directory tree recursively. The remote site does not offer sftp/sshd for that purpose. Is there a simple way to perform such a task with FreeBSD's own tools (I try to avoid installing 'wget' and sibblings)? I need to keep it simple, task should be performed via cronjob. I'm afraid you can't do that with FreeBSD base tools. An alternative to wget would be omi (ports/ftp/omi) which is a simple FTP mirroring tool, written in C without any dependencies. Usage is simple: $ omi -s server.name.com -r /remote/dir -l ./local/dir Note that, by default, it tries to synchronize the local dir perfectly, i.e. if the remote dir is empty, it will wipe out the local dir. (The option -P 0 will prevent omi from removing anything.) Best regards Oliver Thanks for so much answers. I tried 'omi' but I find that the tool does not travers deeper into a dir than level one, so subdirs seem to be left out. I will try wget, although this tool would not be the first choice. I seem to remember that nctp3 had a decent -R recurse option to its get (or its ncftpget non interactive mode) if you want to avoid wget. Vince Thanks, Oliver ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Fetching directories inclusive subdirectories on HTTP server via fetch or othe FreeBSD-own tools?
O. Hartmann wrote: I tried 'omi' but I find that the tool does not travers deeper into a dir than level one, so subdirs seem to be left out. I will try wget, although this tool would not be the first choice. Well, omi _does_ recurse into subdirectories, but it might fail if the FTP server has an unusual output format, or otherwise behaves in unexpected ways. Sometimes -o nostat helps to work around it. Especially if the FTP server runs on some non-UNIX environment and the directory output does not look like an ls -l, omi has trouble parsing it. I've seen FTP servers running on VMS or Novell Netware that looked _really_ weird. These are not supported by omi, I'm afraid. Best regards Oliver -- Oliver Fromme, secnetix GmbH Co. KG, Marktplatz 29, 85567 Grafing b. M. Handelsregister: Registergericht Muenchen, HRA 74606, Geschäftsfuehrung: secnetix Verwaltungsgesellsch. mbH, Handelsregister: Registergericht Mün- chen, HRB 125758, Geschäftsführer: Maik Bachmann, Olaf Erb, Ralf Gebhart FreeBSD-Dienstleistungen, -Produkte und mehr: http://www.secnetix.de/bsd I made up the term 'object-oriented', and I can tell you I didn't have C++ in mind. -- Alan Kay, OOPSLA '97 ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Fetching directories inclusive subdirectories on HTTP server via fetch or othe FreeBSD-own tools?
Hello. I run into a problem I can not solve. I need to fetch a whole directory tree from a public remote site. The top level directory and its subdirectories are accessible via ftp:// and http:// so I tried fetch, but fetch does only retrieve data on file basis and does not copy a whole directory tree recursively. The remote site does not offer sftp/sshd for that purpose. Is there a simple way to perform such a task with FreeBSD's own tools (I try to avoid installing 'wget' and sibblings)? I need to keep it simple, task should be performed via cronjob. Please eMail me. Thanks in advance, Oliver ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Fetching directories inclusive subdirectories on HTTP server via fetch or othe FreeBSD-own tools?
On 1/4/09 17:07, O. Hartmann wrote: Hello. I run into a problem I can not solve. I need to fetch a whole directory tree from a public remote site. The top level directory and its subdirectories are accessible via ftp:// and http:// so I tried fetch, but fetch does only retrieve data on file basis and does not copy a whole directory tree recursively. The remote site does not offer sftp/sshd for that purpose. Is there a simple way to perform such a task with FreeBSD's own tools (I try to avoid installing 'wget' and sibblings)? I need to keep it simple, task should be performed via cronjob. Please eMail me. Without writing a script to parse the output of fetch I dont think its possible (fetch the index via http, parse for dirs, decend into dir, rinse and repeat.) Looking at http://www.freebsd.org/doc/en/articles/hubs/mirror-howto.html which I would expect to suggest FreeBSDs own tools, suggests various ports to use. Vince Thanks in advance, Oliver ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: Fetching directories inclusive subdirectories on HTTP server via fetch or othe FreeBSD-own tools?
O. Hartmann ohart...@zedat.fu-berlin.de wrote: I run into a problem I can not solve. I need to fetch a whole directory tree from a public remote site. The top level directory and its subdirectories are accessible via ftp:// and http:// so I tried fetch, but fetch does only retrieve data on file basis and does not copy a whole directory tree recursively. The remote site does not offer sftp/sshd for that purpose. Is there a simple way to perform such a task with FreeBSD's own tools (I try to avoid installing 'wget' and sibblings)? I need to keep it simple, task should be performed via cronjob. I'm afraid you can't do that with FreeBSD base tools. An alternative to wget would be omi (ports/ftp/omi) which is a simple FTP mirroring tool, written in C without any dependencies. Usage is simple: $ omi -s server.name.com -r /remote/dir -l ./local/dir Note that, by default, it tries to synchronize the local dir perfectly, i.e. if the remote dir is empty, it will wipe out the local dir. (The option -P 0 will prevent omi from removing anything.) Best regards Oliver -- Oliver Fromme, secnetix GmbH Co. KG, Marktplatz 29, 85567 Grafing b. M. Handelsregister: Registergericht Muenchen, HRA 74606, Geschäftsfuehrung: secnetix Verwaltungsgesellsch. mbH, Handelsregister: Registergericht Mün- chen, HRB 125758, Geschäftsführer: Maik Bachmann, Olaf Erb, Ralf Gebhart FreeBSD-Dienstleistungen, -Produkte und mehr: http://www.secnetix.de/bsd That's what I love about GUIs: They make simple tasks easier, and complex tasks impossible. -- John William Chambless ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
