Just to clarify Winston's initial observations, calculating the number of bytes from blocks used will not return the number of bytes in the file. It is rounded up to the nearest block.
This is why we had to just through some hoops here. We required the exact number of bytes, so we had to use VMS system calls and couldn't rely of VMS ftp listings to determine the file sizes. That said, I aggree, the slash processing of the RE could be added to support both types of ftp file listings. Care to take a stab at it? >>>>> On Thu, 20 May 2004 09:40:49 -0700, Alastair Young <[EMAIL PROTECTED]> said: > I was in error. The value displayed is BLOCKS not BYTES. So > adjusting the REGEX to handle the "no slash" form should be the only > adjustment required. > Here is the response from the MadGoat FTP author: >>> What units does the HGFTP (2.5-3) display for file size when >>> responding to a LIST? Is it using DIRECTORY/UNITS= or just >>> DIRECTORY and having that pick up whatever the local SET >>> PROCESS/UNITS setting is? >> The directory display is created by HGFTP itself, mimicking the >> output > of > DIRECTORY. The file size shown is always the number of blocks > used. >>> We are having problems using the Java Jakarta Commons Net FTP >>> client parsing listings from ftp.accuweather.com > <ftp://ftp.accuweather.com/> >>> . The size is apparently being displayed as BYTES on this system >>> and they are only doing a REGEX for bytes "used/allocated". >> That was a while ago! 8-) AFAIK, *all* of the VMS FTP servers >> display > the > file size only in blocks used. >> HGFTP can be configured to emulate a UNIX listing display, but >> you'd > have > to talk to the AccuWeather people about setting it up that > way. >> Hunter ------ Hunter Goatley, Process Software, >> http://www.process.com/ <[EMAIL PROTECTED]> >> http://www.goatley.com/hunter/ > Alastair Young Senior Network Engineer Technical Support & > Operations M E T A T V [EMAIL PROTECTED] Tel : 415.380.6220 > Cell: 925.784.0812 > -----Original Message----- From: Steve Cohen > [mailto:[EMAIL PROTECTED] Sent: Wednesday, May 19, 2004 4:47 > PM To: Alastair Young Cc: [EMAIL PROTECTED] Subject: > Re: VMSFTPEntryParser.java,v 1.23 does not work with MadGoat FTP > server > Moving this discussion onto the commons-dev list, where it really > belongs. Mr. Young, if you're not a member, we'll keep you in it. > If I read you correctly, what we are doing is definitely wrong, but > it isn't clear what the right way should be. > IF there is a slash, an APPROXIMATE size in bytes is available by > multiplying the "numerator" of this "fraction" by 512. But if there > is no slash and just a single number, it can be either blocks or > bytes. What is the right thing to do in THAT case? ??? > Not a pretty picture! > On Wednesday 19 May 2004 12:36 pm, you wrote: >> The MadGoat documentation indicates that it can serve listings in >> Unix form, but this is set up by a LOGICAL (aka environmental >> variable) on the server. This isn't our server, so we don't have >> the option of > making >> that change, and if we did, it would break our legacy production >> code which is pulling stuff off the same server. >> >> >> >> I haven't used VMS since 1990 (VMS 4.7?) but I have found the >> current VMS docs online at HP. See extracts below. >> >> >> >> Digging in the VMS documentation indicates that the size column > defaults >> to blocks "used/allocated" but other valid forms are "used", > "allocated" >> or "bytes". I have dropped a line to Hunter Goatley, author of >> MadGoat FTP, asking him if the server is specifying the format or >> just calling the DIRECTORY command and inheriting the setting. If >> the latter, we > can >> assume that no slash = BYTES, as the ALLOCATION and USED values >> would not occur. >> >> >> >> From the VMS documentation for the DIRECTORY command at >> > http://h71000.www7.hp.com/DOC/732FINAL/9996/9996pro_016.html#index_x_316 >> >> >> >> /SIZE[=option] >> >> /NOSIZE (default) >> >> Displays the size in blocks of each file. If you omit the option >> parameter, the default lists the file size in blocks used (USED). >> Specify one of the following options: >> >> >> ALL >> >> Lists the file size both in blocks allocated and blocks used. >> >> >> ALLOCATION >> >> Lists the file size in blocks allocated. >> >> >> UNITS[=option] >> >> Allows you to override the current default specified by SET >> PROCESS/UNITS so that you can display file size in your choice of > blocks >> or bytes. >> >> The following keywords are valid options with the UNITS keyword: > BLOCKS, >> BYTES. >> >> If you specify UNITS with no option, the default value is not >> changed. >> >> >> USED >> >> Lists the file size in blocks used. >> >> >> >> Also for SET PROCESS >> > http://h71000.www7.hp.com/DOC/732FINAL/9996/9996pro_061.html#index_x_100 >> 1 >> >> >> >> /UNITS[=keyword] >> >> Specifies whether the amount of disk space reported by certain > utilities >> is to be displayed in blocks or bytes. Keyword options are: >> >> >> Option >> >> Description >> >> >> BLOCKS >> >> Displays disk space in blocks. >> >> >> BYTES >> >> Displays disk space in bytes. >> >> Blocks is the default until /UNITS is set to BYTES. If you specify >> /UNITS with no keyword, disk space is reported in blocks. >> >> Displays that are affected by changing the value of /UNITS include >> output from certain forms of the following commands: COPY, DELETE, >> DIRECTORY, PURGE, SHOW DEVICE, SHOW MEMORY, and SHOW QUOTA. Note >> that input to these commands can be specified only in blocks. The > DIRECTORY, >> SHOW DEVICES, and SHOW MEMORY commands have a qualifier that lets >> you override the default SET PROCESS/UNITS setting for a single >> command. >> >> >> >> Alastair Young >> >> Senior Network Engineer >> >> Technical Support & Operations >> >> M E T A T V >> >> [EMAIL PROTECTED] >> >> Tel : 415.380.6220 >> >> Cell: 925.784.0812 >> >> >> >> -----Original Message----- From: Ojeda, Winston >> [mailto:[EMAIL PROTECTED] Sent: Wednesday, May 19, 2004 6:11 AM >> To: Steve Cohen; Alastair Young; [EMAIL PROTECTED]; >> [EMAIL PROTECTED] Subject: RE: VMSFTPEntryParser.java,v 1.23 does >> not work with MadGoat FTP server >> >> >> >> When I wrote the VMSListParser many years ago it was more in terms >> of > a >> test to see what the Netcomponents library could do for us. >> >> Little did I know, wound up being the pillar of everything we do > around >> here. >> >> >> >> Anyways, you are right. Under the VMS Operating System, every file >> is >> >> described in terms of blocks for size. One block is exactly 512 >> bytes. >> >> A regular directory listing will show two values; ie. 2709/2715. >> >> The nominator, which is what the parser uses, is the used blocks >> out > of >> the allocated blocks which is the denominator. >> >> Now, as you can see, the problem with the current implementation of > the >> ListParser for VMS is that to find "bytes" you will need to >> multiply > the >> numerator by 512 which will net you a upward rounded number. This >> is > an >> aproximation and cannot/should not be trusted for critical >> operations. >> >> >> >> The only way to do this right, that I know of, is to find the last > used >> block and see how many bytes are populated in it. That will net >> you > the >> "true" size of the file. The only way we have found is through a >> very >> >> complex data server written in java to provide access to native VMS >> >> calls. >> >> >> >> Maybe someone else can come up with a better way to do this. >> >> >> >> I don't know about MadGoat too much, but some other VMS ftp servers >> >> already serve directory listings in UNIX mode. If MadGoat is indeed >> >> serving a size in bytes, the VMSListParser can be modifed to accept > both >> formats. >> >> >> >> >> >> >> >> Winston Ojeda >> >> Software Engineer >> >> Information Systems >> >> West Allis, Wisconsin >> >> [EMAIL PROTECTED] >> >> www.QG.com >> >> >> >> >> >> -----Original Message----- >> >> From: Steve Cohen [mailto:[EMAIL PROTECTED] >> >> Sent: Tuesday, May 18, 2004 11:09 PM >> >> To: Alastair Young; Ojeda, Winston; [EMAIL PROTECTED]; > [EMAIL PROTECTED] >> Subject: Re: VMSFTPEntryParser.java,v 1.23 does not work with >> MadGoat >> >> FTP server >> >> >> >> >> >> I would say that yes, we can probably modify our regex to work the >> way >> >> this >> >> server does. Basically we are just ignoring the second number. > Please >> log >> >> this as a bug in bugzilla. >> >> >> >> However, after glancing at this, I am a little concerned that we >> are > not >> >> >> >> correctly interpreting the size for the regular servers. It seems > like >> the >> >> second number must mean something. If the first number is indeed >> the >> >> size, >> >> then all of our original samples are pointing at very small files. >> If > I >> had >> >> to bet, I would say that we're misinterpreting what we're parsing. >> >> >> >> Can you, Mr. Young, or someone else confirm the meaning of the two >> >> numbers in >> >> the regular VMS FTP listing? >> >> On Tuesday 18 May 2004 5:51 pm, Alastair Young wrote: >> > We are trying to port our current script based download from >> > >> > ftp.accuweather.com <ftp://ftp.accuweather.com/> to a java app > using >> > > the commons net ftp client. >> > >> > >> > >> > >> > >> > >> > >> > Unfortunately the VMSFTPEntryParser does not match the file >> listings >> > >> > returned by the ftp server at that site. >> > >> > >> > >> > >> > >> > >> > >> > They are running MadGoat V2.5-3 >> > >> > >> > >> > >> > >> > >> > >> > Connected to ftp.accuweather.com (207.242.93.48). >> > >> > >> > >> > 220 ftp.AccuWeather.com MadGoat FTP server V2.5-3 ready. >> > >> > >> > >> > Name (ftp.accuweather.com:ayoung): metatv >> > >> > >> > >> > 331 Username "metatv" Okay, need password. >> > >> > >> > >> > Password: >> > >> > >> > >> > 230-Welcome to AccuWeather, Inc. >> > >> > >> > >> > 230-User "METATV" logged in, 18-MAY-2004 22:04:16 +0000, proceed. >> > >> > >> > >> > 230 Connection closes if idle for 5 min. >> > >> > >> > >> > Remote system type is VMS. >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > The file listings look like >> > >> > >> > >> > >> > >> > >> > >> > ftp> dir *.xml >> > >> > >> > >> > 227 Entering passive mode (207,242,93,48,17,149). >> > >> > >> > >> > 150 LIST of *.XML Started; Opening data connection. >> > >> > >> > >> > >> > >> > >> > >> > BBSDATA2:[METATV] >> > >> > >> > >> > >> > >> > >> > >> > 5DAY.XML;2615 21961 18-MAY-2004 21:17 [CLIENTS,METATV] >> > >> > (RWED,RWED,RE,RE) >> > >> > >> > >> > HOROSCOPES.XML;146 16 18-MAY-2004 10:22 [CLIENTS,METATV] >> > >> > (RWED,RWED,RE,RE) >> > >> > >> > >> > NEWENG_TIDES.XML;146 >> > >> > >> > >> > 17 18-MAY-2004 10:22 [CLIENTS,METATV] >> > >> > (RWED,RWED,RE,RE) >> > >> > >> > >> > POLLEN.XML;704 12308 18-MAY-2004 10:23 [CLIENTS,METATV] >> > >> > (RWED,RWED,RE,) >> > >> > >> > >> > RISE.XML;146 1064 18-MAY-2004 10:22 [CLIENTS,METATV] >> > >> > (RWED,RWED,RE,RE) >> > >> > >> > >> > SOAPS.XML;1 16 18-MAY-2004 10:22 [CLIENTS,METATV] >> > >> > (RWED,RWED,RE,RE) >> > >> > >> > >> > USCUR_NEW_UVI.XML;14175 >> > >> > >> > >> > 17230 18-MAY-2004 21:12 [CLIENTS,METATV] >> > >> > (RWED,RWED,RE,RE) >> > >> > >> > >> > US_CUR.XML;16136 7469 18-MAY-2004 21:13 [CLIENTS,METATV] >> > >> > (RWED,RWED,RE,RE) >> > >> > >> > >> > US_CUR_NEW.XML;16135 >> > >> > >> > >> > 7868 18-MAY-2004 21:13 [CLIENTS,METATV] >> > >> > (RWED,RWED,RE,RE) >> > >> > >> > >> > >> > >> > >> > >> > Total of 9 Files, 67949 Blocks. >> > >> > >> > >> > 226 File transfer Okay; Closing data connection. >> > >> > >> > >> > ftp> >> > >> > >> > >> > >> > >> > >> > >> > Looking at the columns it appears VMSFTPEntryParser is looking >> for > n/n >> > >> > >> > >> > in the second column and using the number before the slash as the > file >> > >> > >> > >> > size. The accuweather listing does not have the /n. >> > >> > >> > >> > >> > >> > >> > >> > I did VMS long ago - I *think* the n/n represents blocks >> > >> > used/allocated. The single number in the MadGoat is clearly in > bytes. >> > >> > >> > >> > >> > >> > >> > >> > Any way this FTP server can be supported? >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > Alastair Young >> > >> > >> > >> > Senior Network Engineer >> > >> > >> > >> > Technical Support & Operations >> > >> > >> > >> > <http://www.metatv.com/> <http://www.metatv.com/> >> > >> > >> > >> > [EMAIL PROTECTED] <mailto:[EMAIL PROTECTED]> >> > >> > >> > >> > Tel : 415.380.6220 >> > >> > >> > >> > Cell: 925.784.0812 > --------------------------------------------------------------------- > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] -- ===================================================================== Jeffrey D. Brekke [EMAIL PROTECTED] Wisconsin, USA [EMAIL PROTECTED] [EMAIL PROTECTED] --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
