I was in error. The value displayed is BLOCKS not BYTES. So adjusting the REGEX to handle the "no slash" form should be the only adjustment required.
Here is the response from the MadGoat FTP author: >> What units does the HGFTP (2.5-3) display for file size when >> responding to a LIST? Is it using DIRECTORY/UNITS= or just DIRECTORY >> and having that pick up whatever the local SET PROCESS/UNITS setting >> is? > > The directory display is created by HGFTP itself, mimicking the output of > DIRECTORY. The file size shown is always the number of blocks used. >> We are having problems using the Java Jakarta Commons Net FTP client >> parsing listings from ftp.accuweather.com <ftp://ftp.accuweather.com/> >> . The size is apparently being displayed as BYTES on this system and >> they are only doing a REGEX for bytes "used/allocated". > That was a while ago! 8-) AFAIK, *all* of the VMS FTP servers display the > file size only in blocks used. > HGFTP can be configured to emulate a UNIX listing display, but you'd have > to talk to the AccuWeather people about setting it up that way. > Hunter > ------ > Hunter Goatley, Process Software, http://www.process.com/ > <[EMAIL PROTECTED]> http://www.goatley.com/hunter/ Alastair Young Senior Network Engineer Technical Support & Operations M E T A T V [EMAIL PROTECTED] Tel : 415.380.6220 Cell: 925.784.0812 -----Original Message----- From: Steve Cohen [mailto:[EMAIL PROTECTED] Sent: Wednesday, May 19, 2004 4:47 PM To: Alastair Young Cc: [EMAIL PROTECTED] Subject: Re: VMSFTPEntryParser.java,v 1.23 does not work with MadGoat FTP server Moving this discussion onto the commons-dev list, where it really belongs. Mr. Young, if you're not a member, we'll keep you in it. If I read you correctly, what we are doing is definitely wrong, but it isn't clear what the right way should be. IF there is a slash, an APPROXIMATE size in bytes is available by multiplying the "numerator" of this "fraction" by 512. But if there is no slash and just a single number, it can be either blocks or bytes. What is the right thing to do in THAT case? ??? Not a pretty picture! On Wednesday 19 May 2004 12:36 pm, you wrote: > The MadGoat documentation indicates that it can serve listings in Unix > form, but this is set up by a LOGICAL (aka environmental variable) on > the server. This isn't our server, so we don't have the option of making > that change, and if we did, it would break our legacy production code > which is pulling stuff off the same server. > > > > I haven't used VMS since 1990 (VMS 4.7?) but I have found the current > VMS docs online at HP. See extracts below. > > > > Digging in the VMS documentation indicates that the size column defaults > to blocks "used/allocated" but other valid forms are "used", "allocated" > or "bytes". I have dropped a line to Hunter Goatley, author of MadGoat > FTP, asking him if the server is specifying the format or just calling > the DIRECTORY command and inheriting the setting. If the latter, we can > assume that no slash = BYTES, as the ALLOCATION and USED values would > not occur. > > > > From the VMS documentation for the DIRECTORY command at > http://h71000.www7.hp.com/DOC/732FINAL/9996/9996pro_016.html#index_x_316 > > > > /SIZE[=option] > > /NOSIZE (default) > > Displays the size in blocks of each file. If you omit the option > parameter, the default lists the file size in blocks used (USED). > Specify one of the following options: > > > ALL > > Lists the file size both in blocks allocated and blocks used. > > > ALLOCATION > > Lists the file size in blocks allocated. > > > UNITS[=option] > > Allows you to override the current default specified by SET > PROCESS/UNITS so that you can display file size in your choice of blocks > or bytes. > > The following keywords are valid options with the UNITS keyword: BLOCKS, > BYTES. > > If you specify UNITS with no option, the default value is not changed. > > > USED > > Lists the file size in blocks used. > > > > Also for SET PROCESS > http://h71000.www7.hp.com/DOC/732FINAL/9996/9996pro_061.html#index_x_100 > 1 > > > > /UNITS[=keyword] > > Specifies whether the amount of disk space reported by certain utilities > is to be displayed in blocks or bytes. Keyword options are: > > > Option > > Description > > > BLOCKS > > Displays disk space in blocks. > > > BYTES > > Displays disk space in bytes. > > Blocks is the default until /UNITS is set to BYTES. If you specify > /UNITS with no keyword, disk space is reported in blocks. > > Displays that are affected by changing the value of /UNITS include > output from certain forms of the following commands: COPY, DELETE, > DIRECTORY, PURGE, SHOW DEVICE, SHOW MEMORY, and SHOW QUOTA. Note that > input to these commands can be specified only in blocks. The DIRECTORY, > SHOW DEVICES, and SHOW MEMORY commands have a qualifier that lets you > override the default SET PROCESS/UNITS setting for a single command. > > > > Alastair Young > > Senior Network Engineer > > Technical Support & Operations > > M E T A T V > > [EMAIL PROTECTED] > > Tel : 415.380.6220 > > Cell: 925.784.0812 > > > > -----Original Message----- > From: Ojeda, Winston [mailto:[EMAIL PROTECTED] > Sent: Wednesday, May 19, 2004 6:11 AM > To: Steve Cohen; Alastair Young; [EMAIL PROTECTED]; [EMAIL PROTECTED] > Subject: RE: VMSFTPEntryParser.java,v 1.23 does not work with MadGoat > FTP server > > > > When I wrote the VMSListParser many years ago it was more in terms of a > > test to see what the Netcomponents library could do for us. > > Little did I know, wound up being the pillar of everything we do around > > here. > > > > Anyways, you are right. Under the VMS Operating System, every file is > > described in terms of blocks for size. One block is exactly 512 bytes. > > A regular directory listing will show two values; ie. 2709/2715. > > The nominator, which is what the parser uses, is the used blocks out of > > the allocated blocks which is the denominator. > > Now, as you can see, the problem with the current implementation of the > > ListParser for VMS is that to find "bytes" you will need to multiply the > > numerator by 512 which will net you a upward rounded number. This is an > > aproximation and cannot/should not be trusted for critical operations. > > > > The only way to do this right, that I know of, is to find the last used > > block and see how many bytes are populated in it. That will net you the > > "true" size of the file. The only way we have found is through a very > > complex data server written in java to provide access to native VMS > > calls. > > > > Maybe someone else can come up with a better way to do this. > > > > I don't know about MadGoat too much, but some other VMS ftp servers > > already serve directory listings in UNIX mode. If MadGoat is indeed > > serving a size in bytes, the VMSListParser can be modifed to accept both > > formats. > > > > > > > > Winston Ojeda > > Software Engineer > > Information Systems > > West Allis, Wisconsin > > [EMAIL PROTECTED] > > www.QG.com > > > > > > -----Original Message----- > > From: Steve Cohen [mailto:[EMAIL PROTECTED] > > Sent: Tuesday, May 18, 2004 11:09 PM > > To: Alastair Young; Ojeda, Winston; [EMAIL PROTECTED]; [EMAIL PROTECTED] > > Subject: Re: VMSFTPEntryParser.java,v 1.23 does not work with MadGoat > > FTP server > > > > > > I would say that yes, we can probably modify our regex to work the way > > this > > server does. Basically we are just ignoring the second number. Please > > log > > this as a bug in bugzilla. > > > > However, after glancing at this, I am a little concerned that we are not > > > > correctly interpreting the size for the regular servers. It seems like > > the > > second number must mean something. If the first number is indeed the > > size, > > then all of our original samples are pointing at very small files. If I > > had > > to bet, I would say that we're misinterpreting what we're parsing. > > > > Can you, Mr. Young, or someone else confirm the meaning of the two > > numbers in > > the regular VMS FTP listing? > > On Tuesday 18 May 2004 5:51 pm, Alastair Young wrote: > > We are trying to port our current script based download from > > > > ftp.accuweather.com <ftp://ftp.accuweather.com/> to a java app using > > > > the commons net ftp client. > > > > > > > > > > > > > > > > Unfortunately the VMSFTPEntryParser does not match the file listings > > > > returned by the ftp server at that site. > > > > > > > > > > > > > > > > They are running MadGoat V2.5-3 > > > > > > > > > > > > > > > > Connected to ftp.accuweather.com (207.242.93.48). > > > > > > > > 220 ftp.AccuWeather.com MadGoat FTP server V2.5-3 ready. > > > > > > > > Name (ftp.accuweather.com:ayoung): metatv > > > > > > > > 331 Username "metatv" Okay, need password. > > > > > > > > Password: > > > > > > > > 230-Welcome to AccuWeather, Inc. > > > > > > > > 230-User "METATV" logged in, 18-MAY-2004 22:04:16 +0000, proceed. > > > > > > > > 230 Connection closes if idle for 5 min. > > > > > > > > Remote system type is VMS. > > > > > > > > > > > > > > > > > > > > > > > > The file listings look like > > > > > > > > > > > > > > > > ftp> dir *.xml > > > > > > > > 227 Entering passive mode (207,242,93,48,17,149). > > > > > > > > 150 LIST of *.XML Started; Opening data connection. > > > > > > > > > > > > > > > > BBSDATA2:[METATV] > > > > > > > > > > > > > > > > 5DAY.XML;2615 21961 18-MAY-2004 21:17 [CLIENTS,METATV] > > > > (RWED,RWED,RE,RE) > > > > > > > > HOROSCOPES.XML;146 16 18-MAY-2004 10:22 [CLIENTS,METATV] > > > > (RWED,RWED,RE,RE) > > > > > > > > NEWENG_TIDES.XML;146 > > > > > > > > 17 18-MAY-2004 10:22 [CLIENTS,METATV] > > > > (RWED,RWED,RE,RE) > > > > > > > > POLLEN.XML;704 12308 18-MAY-2004 10:23 [CLIENTS,METATV] > > > > (RWED,RWED,RE,) > > > > > > > > RISE.XML;146 1064 18-MAY-2004 10:22 [CLIENTS,METATV] > > > > (RWED,RWED,RE,RE) > > > > > > > > SOAPS.XML;1 16 18-MAY-2004 10:22 [CLIENTS,METATV] > > > > (RWED,RWED,RE,RE) > > > > > > > > USCUR_NEW_UVI.XML;14175 > > > > > > > > 17230 18-MAY-2004 21:12 [CLIENTS,METATV] > > > > (RWED,RWED,RE,RE) > > > > > > > > US_CUR.XML;16136 7469 18-MAY-2004 21:13 [CLIENTS,METATV] > > > > (RWED,RWED,RE,RE) > > > > > > > > US_CUR_NEW.XML;16135 > > > > > > > > 7868 18-MAY-2004 21:13 [CLIENTS,METATV] > > > > (RWED,RWED,RE,RE) > > > > > > > > > > > > > > > > Total of 9 Files, 67949 Blocks. > > > > > > > > 226 File transfer Okay; Closing data connection. > > > > > > > > ftp> > > > > > > > > > > > > > > > > Looking at the columns it appears VMSFTPEntryParser is looking for n/n > > > > > > > > in the second column and using the number before the slash as the file > > > > > > > > size. The accuweather listing does not have the /n. > > > > > > > > > > > > > > > > I did VMS long ago - I *think* the n/n represents blocks > > > > used/allocated. The single number in the MadGoat is clearly in bytes. > > > > > > > > > > > > > > > > Any way this FTP server can be supported? > > > > > > > > > > > > > > > > > > > > > > > > Alastair Young > > > > > > > > Senior Network Engineer > > > > > > > > Technical Support & Operations > > > > > > > > <http://www.metatv.com/> <http://www.metatv.com/> > > > > > > > > [EMAIL PROTECTED] <mailto:[EMAIL PROTECTED]> > > > > > > > > Tel : 415.380.6220 > > > > > > > > Cell: 925.784.0812 --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
