Hi Chris, You had any luck figuring out what is causing this issue with the FTP site?
Hamish On 29 July 2010 14:46, Hamish McWilliam <[email protected]> wrote: > Hi Chris, > >> I did have to turn on support in VSFTPd for "ls -R" because it was >> considered a possible resource drain or DOS target on large directories but >> you are right that the extended mode passive stuff does not work. It does >> not seem to be a firewall issue on the server and I don't think our upstream >> Juniper SSG intrusion device is interfering. > > FWIW I get the same behavior when using passive or "active" FTP modes. > >> Can I ask what you are trying to accomplish? A mirror process? Can we get >> around this problem via an anonymous RSYNC service pointing at the anonymous >> FTP root? Passwordless SSH access for a mirror script that uses "rsync -avz >> -e ssh ..." or similar? > > I'm trying to update the EMBL-EBI mirror of the EMBOSS software > (ftp://ftp.ebi.ac.uk/pub/software/unix/EMBOSS/), this uses Mirror > (ftp://ftp.ebi.ac.uk/pub/software/unix/mirror.tar.Z) to do the > updates. As you can tell from the files in the directory this mirror > no longer works, since it cannot get the listing of files. In fact it > fails silently so I can't tell when the problem started. > >> vsftpd fits our needs for secure anonymous FTP server, if there is an easier >> route to getting your needs sorted I might want to look at that rather than >> going down deep into vsftp internals or swapping out the server software. > > From a few experiments with our vsftpd setup (CentOS 5.x, vsftpd > 2.0.5) this looks to be supported by default, other than the -R thing. > > From a little Google it could be some interaction with the other > components (http://www.ncftp.com/ncftpd/doc/misc/ftp_and_firewalls.html) > where they don't recognize the FTP command correctly and choose the > wrong mode, but it seems unlikely. Does it work if you run directly > against the box? > > Hamish > >> Hamish McWilliam wrote: >>> >>> Hi Chris, >>> >>>> I can't seem to replicate this at all with any of my available FTP >>>> clients >>>> or browser-based FTP clients. It works for me from both Mac OS X clients >>>> as >>>> well as a CentOS 5.4 based Linux system. >>>> >>>> Is there an FTP client / OS combo that seems in particular not to work? >>> >>> This affects all clients I've tried. >>> >>> From your transcript, you are not running the commands required to see >>> the issue: >>> >>> - 'ls' or 'dir' without options works >>> - 'ls' or 'dir' with options hangs and eventually gives a timeout. FTP >>> mirror systems often use -lRat as options to 'ls' to get all the files >>> for a particular branch. >>> >>> $ ftp emboss.open-bio.org >>> Connected to emboss.open-bio.org. >>> 220 (vsFTPd 2.0.1) >>> 530 Please login with USER and PASS. >>> 530 Please login with USER and PASS. >>> KERBEROS_V4 rejected as an authentication type >>> Name (emboss.open-bio.org:hpm): ftp >>> 331 Please specify the password. >>> Password: >>> 230 Login successful. >>> Remote system type is UNIX. >>> Using binary mode to transfer files. >>> ftp> cd /pub/EMBOSS/fixes >>> 250 Directory successfully changed. >>> ftp> ls >>> 227 Entering Passive Mode (208,94,50,58,51,71) >>> 150 Here comes the directory listing. >>> -rw-rw-r-- 1 501 503 385 Jul 21 09:25 README.fixes >>> -rwxrwxr-x 1 501 503 506140 Jul 21 09:25 configure >>> -rw-rw-r-- 1 501 503 5405 Jul 21 09:25 mysql.m4 >>> drwxrwsr-x 4 501 503 4096 Jul 21 09:26 patches >>> 226 Directory send OK. >>> ftp> ls -lRat >>> 227 Entering Passive Mode (208,94,50,58,31,88) >>> *HANG* >>> >>> Compare with the equivalent session against the EMBL-EBI FTP site: >>> >>> $ ftp ftp.ebi.ac.uk >>> Connected to ftp.ebi.ac.uk. >>> 220- >>> 220- ftp1.ebi.ac.uk FTP server >>> 220- >>> 220- WARNING: please note that the private part of this ftp service >>> 220- has been migrated to ftp-private.ebi.ac.uk on 3rd June 2010. >>> 220 >>> 530 Please login with USER and PASS. >>> 530 Please login with USER and PASS. >>> KERBEROS_V4 rejected as an authentication type >>> Name (ftp.ebi.ac.uk:hpm): ftp >>> 331 Please specify the password. >>> Password: >>> 230 Login successful. >>> Remote system type is UNIX. >>> Using binary mode to transfer files. >>> ftp> cd /pub/software/unix/EMBOSS/fixes >>> 250 Directory successfully changed. >>> ftp> ls >>> 227 Entering Passive Mode (193,62,193,4,238,107) >>> 150 Here comes the directory listing. >>> -r--r--r-- 1 ftp ftp 180 Jan 15 2010 README.fixes >>> 226 Directory send OK. >>> ftp> ls -lRat >>> 227 Entering Passive Mode (193,62,193,4,36,65) >>> 150 Here comes the directory listing. >>> .: >>> drwxr-xr-x 3 ftp ftp 4096 Jan 18 2010 .. >>> drwxr-xr-x 2 ftp ftp 4096 Jan 18 2010 . >>> -r--r--r-- 1 ftp ftp 180 Jan 15 2010 README.fixes >>> 226 Directory send OK. >>> ftp> bye >>> 221 Goodbye. >>> >>> All the best, >>> >>> Hamish >>> >>>> Here is one example: >>>> >>>>> dag-static:~ dag$ ftp emboss.open-bio.org >>>>> Connected to emboss.open-bio.org. >>>>> 220 (vsFTPd 2.0.1) >>>>> Name (emboss.open-bio.org:dag): anonymous >>>>> 331 Please specify the password. >>>>> Password: >>>>> 230 Login successful. >>>>> Remote system type is UNIX. >>>>> Using binary mode to transfer files. >>>>> ftp> dir >>>>> 229 Entering Extended Passive Mode (|||54472|) >>>>> 150 Here comes the directory listing. >>>>> drwxr-xr-x 8 14 50 4096 May 22 2006 pub >>>>> 226 Directory send OK. >>>>> ftp> cd pub/EMBOSS >>>>> 250 Directory successfully changed. >>>>> ftp> ls >>>>> 229 Entering Extended Passive Mode (|||38343|) >>>>> 150 Here comes the directory listing. >>>>> -rw-rw-r-- 1 501 503 389856 Jul 21 09:25 >>>>> CBSTOOLS-1.0.0.tar.gz >>>>> -rw-rw-r-- 1 501 503 426218 Jul 21 09:25 >>>>> DOMAINATRIX-0.1.0.tar.gz >>>>> -rw-rw-r-- 1 501 503 441025 Jul 21 09:25 >>>>> DOMALIGN-0.1.0.tar.gz >>>>> -rw-rw-r-- 1 501 503 452787 Jul 21 09:25 >>>>> DOMSEARCH-0.1.0.tar.gz >>>>> -rw-rw-r-- 1 501 503 23572243 Jul 19 13:41 >>>>> EMBOSS-6.3.1.tar.gz >>>>> lrwxrwxrwx 1 501 503 19 Jul 19 13:42 >>>>> EMBOSS-latest.tar.gz -> EMBOSS-6.3.1.tar.gz >>>>> -rw-rw-r-- 1 501 503 373798 Jul 21 09:25 EMNU-1.05.tar.gz >>>>> -rw-rw-r-- 1 501 503 415096 Jul 21 09:25 >>>>> ESIM4-1.0.0.tar.gz >>>>> -rw-rw-r-- 1 501 503 569581 Jul 21 09:25 >>>>> HMMER-2.3.2.tar.gz >>>>> -rw-rw-r-- 1 501 503 350791 Jul 21 09:25 >>>>> IPRSCAN-4.3.1.tar.gz >>>>> drwxrwsr-x 7 501 503 4096 Feb 01 2006 Jemboss >>>>> -rw-rw-r-- 1 501 503 513418 Jul 21 09:25 >>>>> MEMENEW-4.0.0.tar.gz >>>>> -rw-rw-r-- 1 501 503 823636 Jul 21 09:25 >>>>> MIRA-2.8.2.tar.gz >>>>> -rw-rw-r-- 1 501 503 435315 Jul 21 09:25 MSE-3.0.0.tar.gz >>>>> -rw-rw-r-- 1 501 503 328540 Jul 21 09:25 >>>>> MYEMBOSS-6.3.0.tar.gz >>>>> -rw-rw-r-- 1 501 503 359488 Jul 21 09:25 >>>>> MYEMBOSSDEMO-6.3.0.tar.gz >>>>> -rw-rw-r-- 1 501 503 1667760 Jul 21 09:25 >>>>> PHYLIPNEW-3.69.tar.gz >>>>> -rw-rw-r-- 1 501 503 571008 Jul 21 09:25 >>>>> SIGNATURE-0.1.0.tar.gz >>>>> -rw-rw-r-- 1 501 503 531604 Jul 21 09:25 >>>>> STRUCTURE-0.1.0.tar.gz >>>>> -rw-rw-r-- 1 501 503 386287 Jul 21 09:25 >>>>> TOPO-2.0.0.tar.gz >>>>> -rw-rw-r-- 1 501 503 652433 Jul 21 09:25 >>>>> VIENNA-1.7.2.tar.gz >>>>> drwxrwsr-x 3 522 503 4096 Aug 21 2006 contrib >>>>> drwxrwsr-x 2 501 503 4096 Nov 11 2005 doc >>>>> drwxrwsr-x 3 501 503 4096 Jul 21 09:25 fixes >>>>> drwxrwsr-x 14 501 503 4096 Jul 19 13:39 old >>>>> drwxrwsr-x 2 501 503 4096 Jul 06 2005 tutorials >>>>> drwxrwsr-x 4 501 503 4096 Jul 19 13:36 windows >>>>> 226 Directory send OK. >>>>> ftp> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> Hamish McWilliam wrote: >>>>> >>>>> Hi Chris, >>>>> >>>>> I'm also seeing problems with the FTP site, but using Mirror rather >>>>> than Transmit, it looks like the server does not like options being >>>>> specified to the LIST command: >>>>> >>>>> Scanning remote directory /pub/EMBOSS >>>>> ---> CWD /pub/EMBOSS >>>>> 250 Directory successfully changed. >>>>> ---> TYPE A >>>>> 200 Switching to ASCII mode. >>>>> ---> PORT 172,21,22,1,171,245 >>>>> 200 PORT command successful. Consider using PASV. >>>>> ---> PASV >>>>> 227 Entering Passive Mode (208,94,50,58,104,178) >>>>> ---> LIST -lat >>>>> timed out >>>>> Cannot get remote directory listing because: timed out >>>>> Cannot get remote directory details (/pub/EMBOSS) >>>>> disconnecting from emboss.open-bio.org >>>>> >>>>> Trying it with a command line client I get the response to hang if I >>>>> try using any options to ls or dir, without options they are fine. >>>>> >>>>> All the best, >>>>> >>>>> Hamish >>>>> >>>>> On 29 May 2010 03:44, Koen van der Drift<[email protected]> >>>>> wrote: >>>>>> >>>>>> Hi Chris, >>>>>> >>>>>> Did you have a chance to look at this? Just tried again, and Transmit >>>>>> still >>>>>> won't let me access the emboss ftp site. >>>>>> >>>>>> Thanks, >>>>>> >>>>>> - Koen. >>>>>> >>>>>> >>>>>> On Apr 23, 2010, at 11:44 AM, Chris Dagdigian wrote: >>>>>> >>>>>>> In the last few months the open-bio.org servers switched datacenters, >>>>>>> IP >>>>>>> addresses and firewall/IDS appliances. Lots of juicy things to look at >>>>>>> and >>>>>>> debug. >>>>>>> >>>>>>> Koen - if you have a chance can you send me the IP address that you >>>>>>> are >>>>>>> using to connect from? I might be able to find some relevant log >>>>>>> entries >>>>>>> with that info. >>>>>>> >>>>>>> -Chris >>>>>>> >>>>>>> >>>>>>> >>>>>>> Koen van der Drift wrote: >>>>>>>> >>>>>>>> Just for the record, it used to work with Transmit, this is only from >>>>>>>> the last few months. >>>>>>>> >>>>>>>> - Koen. >>>>>>>> >>>>>>>> On Thu, Apr 22, 2010 at 7:28 PM, Chris Dagdigian<[email protected]> >>>>>>>> wrote: >>>>>>>>> >>>>>>>>> Might be an issue with the Juniper Netscreen firewall/IDS security >>>>>>>>> appliance >>>>>>>>> that sits upstream of the EMBOSS FTP server. I'll take a look at the >>>>>>>>> security logs and alerts. >>>>>>>>> >>>>>>>>> -Chris >>>>>>>>> >>>>>>>>> >>>>>>>>> Koen van der Drift wrote: >>>>>>>>>> >>>>>>>>>> Hi, >>>>>>>>>> >>>>>>>>>> For a while now I am unable to access the emboss ftp site using the >>>>>>>>>> OS >>>>>>>>>> X >>>>>>>>>> client Transmit. Loggin in works fine, but it chokes on the LIST >>>>>>>>>> command. I have no problems accessing it from the command line. I >>>>>>>>>> have >>>>>>>>>> added the output from Transmit below. I don't know if this is a >>>>>>>>>> Transmit >>>>>>>>>> or emboss issue, but just wanted to let you know. >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> >>>>>>>>>> - Koen. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Transmit 3.6.9 Session Transcript >>>>>>>>>> LibNcFTP 3.2.1 (August 13, 2007) compiled for UNIX >>>>>>>>>> Uname: Darwin|exile.local|9.8.0|Darwin Kernel Version 9.8.0: Wed >>>>>>>>>> Jul >>>>>>>>>> 15 >>>>>>>>>> 16:57:01 PDT 2009; root:xnu-1228.15.4~1/RELEASE_PPC|Power Macintosh >>>>>>>>>> 220: (vsFTPd 2.0.1) >>>>>>>>>> Connected to emboss.open-bio.org. >>>>>>>>>> Cmd: USER anonymous >>>>>>>>>> 331: Please specify the password. >>>>>>>>>> Cmd: PASS NcFTP@ >>>>>>>>>> 230: Login successful. >>>>>>>>>> Cmd: TYPE A >>>>>>>>>> 200: Switching to ASCII mode. >>>>>>>>>> Logged in to emboss.open-bio.org as anonymous. >>>>>>>>>> Cmd: SYST >>>>>>>>>> 215: UNIX Type: L8 >>>>>>>>>> Cmd: PWD >>>>>>>>>> 257: "/" >>>>>>>>>> Cmd: CWD /pub/EMBOSS/fixes >>>>>>>>>> 250: Directory successfully changed. >>>>>>>>>> Cmd: PWD >>>>>>>>>> 257: "/pub/EMBOSS/fixes" >>>>>>>>>> Cmd: PASV >>>>>>>>>> 227: Entering Passive Mode (208,94,50,58,83,232) >>>>>>>>>> Cmd: LIST -a >>>>>>>>>> Could not read reply from control connection -- timed out. >>>>>>>>>> (SReadline >>>>>>>>>> 1) >>>>>>>>>> 220: (vsFTPd 2.0.1) >>>>>>>>>> Connected to emboss.open-bio.org. >>>>>>>>>> Cmd: USER anonymous >>>>>>>>>> 331: Please specify the password. >>>>>>>>>> Cmd: PASS NcFTP@ >>>>>>>>>> 230: Login successful. >>>>>>>>>> Logged in to emboss.open-bio.org as anonymous. >>>>>>>>>> Cmd: SYST >>>>>>>>>> 215: UNIX Type: L8 >>>>>>>>>> Cmd: PWD >>>>>>>>>> 257: "/" >>>>>>>>>> Cmd: CWD /pub/EMBOSS/fixes >>>>>>>>>> 250: Directory successfully changed. >>>>>>>>>> Cmd: PWD >>>>>>>>>> 257: "/pub/EMBOSS/fixes" >>>>>>>>>> Cmd: PASV >>>>>>>>>> 227: Entering Passive Mode (208,94,50,58,222,100) >>>>>>>>>> Cmd: LIST -a >>>>>>>>>> Could not read reply from control connection -- timed out. >>>>>>>>>> (SReadline >>>>>>>>>> 1) >>>>>>>>>> >>>>>>>>>> _______________________________________________ >>>>>>>>>> EMBOSS mailing list >>>>>>>>>> [email protected] >>>>>>>>>> http://lists.open-bio.org/mailman/listinfo/emboss >>>>>> >>>>>> _______________________________________________ >>>>>> EMBOSS mailing list >>>>>> [email protected] >>>>>> http://lists.open-bio.org/mailman/listinfo/emboss >>>>>> >>>>> >>>>> >>> >>> >>> >> > > > > -- > ---- > "Saying the internet has changed dramatically over the last five years > is cliché – the internet is always changing dramatically" - Craig > Labovitz, Arbor Networks. > -- ---- "Saying the internet has changed dramatically over the last five years is cliché – the internet is always changing dramatically" - Craig Labovitz, Arbor Networks. _______________________________________________ EMBOSS mailing list [email protected] http://lists.open-bio.org/mailman/listinfo/emboss
