Re: enhancements needed for the OS/400 parser?
On Thursday 07 December 2006 20:44, Daniel Manley wrote: Hi Rory, I don't know much about OS/400, OS/390 or MVS. I would feel more comfortable submitting a patch to the OS/400 parser and let you guys figure out if this is applicable to OS/390 or MVS. I think OS/400 should still have its own parser, but you can evt use the MVS parser as a start. Henrik - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: enhancements needed for the OS/400 parser?
On Tuesday 05 December 2006 22:06, Daniel Manley wrote: super. that seems very comprehensive. thanks I see though that the default parser factory still scans for SYST_OS400. The system line for my server is: OS/400 is the remote operating system. The TCP/IP version is V5R2M0. So, this won't use the MVS parser in this case, right? Yes that is correct. I am not familiar with OS/400, are the file patterns similar to OS/390 ? Henrik - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: enhancements needed for the OS/400 parser?
On Tuesday 05 December 2006 21:52, Rory Winston wrote: Hi Henrik Actually the parser is under the 2.0 branch here: http://svn.apache.org/viewvc/jakarta/commons/proper/net/branches/JDK_1_5_BR ANCH/src/main/java/org/apache/commons/net/ftp/parser/MVSFTPEntryParser.java? revision=439598view=markup And the test is here: http://svn.apache.org/viewvc/jakarta/commons/proper/net/branches/JDK_1_5_BR ANCH/src/test/java/org/apache/commons/net/ftp/parser/MVSFTPEntryParserTest.j ava?revision=439615view=markup Cheers Rory thanks for the new links. I just looked at the links and some of the comments have been reformatted in way that makes them had to read, for example: /** * Matches these entries: Volume Unit Referred Ext Used Recfm Lrecl BlkSz * Dsorg Dsname B10142 3390 2006/03/20 2 31 F 80 80 PS MDI.OKL.WORK * */ should really have been: /** * Matches these entries: * Volume Unit Referred Ext Used Recfm Lrecl BlkSz Dsorg Dsname * B10142 3390 2006/03/20 2 31 F 8080PSMDI.OKL.WORK * */ it might seem a bit weird, but the data is really formatted that way. Which javadoc tricks can be used to achieve this ? I don't mind go through all the comments again and fix it. Henrik - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: enhancements needed for the OS/400 parser?
On Tuesday 05 December 2006 16:37, Daniel Manley wrote: To solve my problem without hacking/customizing the commons-net 1.4.1 jar, I subclassed the OS/400 parser to try parsing with two REGEX values. you can see how it is done for a OS/390 parser. There I also had to create different REGEXes. The changes are in the CVS for the upcomming commons-net 2.0 Code: http://svn.apache.org/viewvc/jakarta/commons/proper/net/trunk/src/java/org/apache/commons/net/ftp/parser/MVSFTPEntryParser.java Test cases: http://svn.apache.org/viewvc/jakarta/commons/proper/net/trunk/src/test/org/apache/commons/net/ftp/parser/MVSFTPEntryParserTest.java good luck. Henrik - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
[jira] Updated: (NET-99) [net] MVSFTPEntryParser.java only halfway implemented
[ http://issues.apache.org/jira/browse/NET-99?page=all ] henrik sorensen updated NET-99: --- Attachment: FTPParseTestFramework.java MVSFTPEntryParser.java MVSFTPEntryParserTest.java The MVSFTPEntryParser now supports JES Interface Level 1 and 2. The JUNIT test cases have been updated accordingly. Have fun Henrik [net] MVSFTPEntryParser.java only halfway implemented - Key: NET-99 URL: http://issues.apache.org/jira/browse/NET-99 Project: Commons Net Issue Type: Bug Environment: Operating System: other Platform: Other Reporter: henrik Attachments: FTPParseTestFramework.java, FTPParseTestFramework.java, MVSFTPEntryParser.java, MVSFTPEntryParser.java, MVSFTPEntryParser.java, MVSFTPEntryParser.java, MVSFTPEntryParser.java, MVSFTPEntryParser.java, MVSFTPEntryParser.java, MVSFTPEntryParserTest.java, MVSFTPEntryParserTest.java, RegexFTPFileEntryParserImpl.java I have been chasing some bugs when using the ant ftp task, and some problems stem from the parsing of the result of the ftp list command. The file structure of MVS is not a hierarchical filesystem, but it has a directory concept known as partition organized datasets. These can be compared to a single level directory. The current parser does not take these into account. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Fwd: Re: [Fwd: Re: apache bugzilla 39109 ... assignee very quite ...]
-- Forwarded Message --
Subject: Re: [Fwd: Re: apache bugzilla 39109 ... assignee very quite ...]
Date: Wednesday 30 August 2006 20:23
From: [EMAIL PROTECTED]
To: Rory Winston [EMAIL PROTECTED]
Yes you are right,
I made the test case pass here by adding the header as written eairler, and
by changing the goodSamples processing as well.
What would it take to have a getGoodSamplesList, that should return a List
where each element is an array to test ?
It is not clear to me what the purpose of
/* (non-Javadoc)
* @see
org.apache.commons.net.ftp.parser.FTPParseTestFramework#testParseFieldsOnFile
() */
public void testParseFieldsOnFile() throws Exception {
FTPFile file = getParser().parseFTPEntry(Migrated
file1.I);
assertNotNull(Could not parse entry., file);
assertTrue(Should have been a file., file.isFile());
assertEquals(file1.I, file.getName());
FTPFile file2 = getParser().parseFTPEntry(PSMLC1 3390 2005/04/04
1 1 VB 27994 27998 PS file2.I);
assertNotNull(Could not parse entry., file2);
assertTrue(Should have been a file., file2.isFile());
assertEquals(file2.I, file2.getName());
}
is this some sort of test that the parsed content is what is expected ?
Here is the same problem, that a complete list must be available, and the
preParse must be called.
With the new parser it is even possible to do the
public void testParseFieldsOnDirectory() throws Exception
{
}
because the PO dataset organisations are now supported.
Give me a day or two, and I will make this work.
One thing about the latest Parser, it contains some less tested code, the JES
parser. It is something I would like to add as well, but I need some more
time to finish it off. When is your latest 'merge' window ?
Henrik
On Wednesday 30 August 2006 08:19, you wrote:
Henrik
I think I see the problem with the current setup.
1. Firstly, the parser needs to initialize itself via an FTP listing
header line in preParse(), in order to set the isType flag and the
parent class regex. As long as we have a way of explicitly setting this,
we can get the class to work in a unit test.
2.Secondly, the current parser will never be able to parse a line such as:
Migratedfile1.I
Because as far as the parser is concerned, this is not a valid line, it is
incomplete. The old parser used a much simpler regex that was able to
handle lines like this, but the old parser simply extracted the last token
(the filename) and used that. Your parser is able to extract more
information - when it exists. If it is possible for a line such as the
above to occur in practise (which I am assuming it is) then we need to be
able to handle it as well.
Also, the method parseFileList() expects to find a valid value for the
DSORG header field. If it does not, it returns fakse from
parseFileList(), faiing the parse. This will not work for lines where a
DSORG value may not be present, so we would need a way of handling this
as well.
I think we are nearly there -if we can get these issues sorted, then we can
look at getting this in to the next release.
Thanks
Rory
---
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
[jira] Updated: (NET-99) [net] MVSFTPEntryParser.java only halfway implemented
[ http://issues.apache.org/jira/browse/NET-99?page=all ] henrik sorensen updated NET-99: --- Attachment: FTPParseTestFramework.java MVSFTPEntryParserTest.java With these two changes the JUNIT tests now completes. I had to hack a bit in the FTPParseTestFramework.java, to make it work with multiple good samples testsets. [net] MVSFTPEntryParser.java only halfway implemented - Key: NET-99 URL: http://issues.apache.org/jira/browse/NET-99 Project: Commons Net Issue Type: Bug Environment: Operating System: other Platform: Other Reporter: henrik Attachments: FTPParseTestFramework.java, MVSFTPEntryParser.java, MVSFTPEntryParser.java, MVSFTPEntryParser.java, MVSFTPEntryParser.java, MVSFTPEntryParser.java, MVSFTPEntryParser.java, MVSFTPEntryParserTest.java, RegexFTPFileEntryParserImpl.java I have been chasing some bugs when using the ant ftp task, and some problems stem from the parsing of the result of the ftp list command. The file structure of MVS is not a hierarchical filesystem, but it has a directory concept known as partition organized datasets. These can be compared to a single level directory. The current parser does not take these into account. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
[jira] Updated: (NET-99) [net] MVSFTPEntryParser.java only halfway implemented
[ http://issues.apache.org/jira/browse/NET-99?page=all ] henrik sorensen updated NET-99: --- Attachment: MVSFTPEntryParser.java RegexFTPFileEntryParserImpl.java Latest version of the MVS Ftp parser. This version supports parsing of the following dataset organisations: PS: sequential files PO: partitioned files (single level direcory) PO-E:partitioned files (single level direcory) and the following record formats beginning with V: Variable length records F: Fixed length records The files system is described in details within the source code. Further a parser for the JES subsystem level one has been added. (JES: Job entry Subsystem (scheduler)) [net] MVSFTPEntryParser.java only halfway implemented - Key: NET-99 URL: http://issues.apache.org/jira/browse/NET-99 Project: Commons Net Issue Type: Bug Environment: Operating System: other Platform: Other Reporter: henrik Attachments: MVSFTPEntryParser.java, MVSFTPEntryParser.java, MVSFTPEntryParser.java, MVSFTPEntryParser.java, MVSFTPEntryParser.java, MVSFTPEntryParser.java, RegexFTPFileEntryParserImpl.java I have been chasing some bugs when using the ant ftp task, and some problems stem from the parsing of the result of the ftp list command. The file structure of MVS is not a hierarchical filesystem, but it has a directory concept known as partition organized datasets. These can be compared to a single level directory. The current parser does not take these into account. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
apache bugzilla 39109 ... assignee very quite ...
Hello Steven, ** PING ** Are you still assigned to this bug ? http://issues.apache.org/bugzilla/show_bug.cgi?id=39109 I have been waiting patiently more than one month now, is there any outstanding issues, beside the junit tests, that needs to be solved ? Henrik - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: apache bugzilla 39109 ... assignee very quite ...
On Tuesday 30 May 2006 18:17, Sandy McArthur wrote: On 5/30/06, [EMAIL PROTECTED] [EMAIL PROTECTED] wrote: http://issues.apache.org/bugzilla/show_bug.cgi?id=39109 I don't know about the issue but you should probably be tracking issues in Jira now: http://issues.apache.org/jira/browse/NET-99 thanks for the tip when look at the jira-zilla, I see there is nobody assigned to this error. Is this part of a conversion, or is it really so ? Henrik - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
[net] time for a new MVSFTPFileNameParser.java ?
Hi List,
This is my first post to this list ...
I have been chasing some problems when using the optional ant task FTP to
access a MVS type host.
Part of the problem is the MVSFTPEntryParser.java seems to be only halfway
implemented.
Here is a updated version of
org/apache/commons/net/ftp/parser/MVSFTPEntryParser.java
for review, note there is a lot of debug info, and further note, I am not
really a java programmer but are willing to learn, so just shoot on whatever
is moving ...
looking forward to hear your comments.
Henrik
/*
* Copyright 2005 The Apache Software Foundation
*
* Licensed under the Apache License, Version 2.0 (the License);
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an AS IS BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.commons.net.ftp.parser;
import java.util.List;
import java.util.StringTokenizer;
import org.apache.commons.net.ftp.FTPClientConfig;
import org.apache.commons.net.ftp.FTPFile;
/**
* Implementation of FTPFileEntryParser and FTPFileListParser for IBM MVS Systems.
*
* @author a href=[EMAIL PROTECTED]Jeff Nadler/a
* @author a href=[EMAIL PROTECTED]William Noto/a
* @version $Id$
* @see org.apache.commons.net.ftp.FTPFileEntryParser FTPFileEntryParser (for usage instructions)
*/
public class MVSFTPEntryParser extends ConfigurableFTPFileEntryParserImpl
{
/**
* This is the regular expression used by this parser.
*/
private static final String REGEX = (.*)\\s+([^\\s]+)\\s*;
/**
* Although this parser is now ignoring dates, someone may someday
* figure out a way to accomodate this and this appears to be the
* format used. For now, it won't be used.
* SMC 2005/04/08
*/
static final String DEFAULT_DATE_FORMAT
= /MM/dd; // 2001/11/09
// This is not at all the tightest possible regexp for MVS LIST
// output, but I'm not a mainframe guru so I have little idea what the
// range of valid values are. I just needed to get the filename (Dsname);
// note that no other FTPFile fields can be filled in with the results of
// a LIST on MVS. The 'Referred' date seems to be 'last accessed date'
// and not 'last modified date' so I didn't bother parsing it.
//
// Of course it works perfectly as-is and it distinguishes header lines from
// file results so that's the important thing.
//
// This parser should be used when SYST returns:
// 'MVS is the operating system of this server. FTP Server is running on z/OS.'
//
// Also note that there is no concept of directories in MVS, just datasets,
// which have names composed of four dot separated names of up to 8 chars.
// As a result, FTPFile.FILE_TYPE is always used. -JN 6/2004 jnadleratsrcgincdotcom
// Sample LIST results from MVS:
//
//Volume UnitReferred Ext Used Recfm Lrecl BlkSz Dsorg Dsname
//FPFS42 3390 2004/06/23 11 FB 128 6144 PS INCOMING.RPTBM023.D061704
//FPFS41 3390 2004/06/23 11 FB 128 6144 PS INCOMING.RPTBM056.D061704
//FPFS25 3390 2004/06/23 11 FB 128 6144 PS INCOMING.WTM204.D061704
/* Format of ZOS files:
*
Volume UnitReferred Ext Used Recfm Lrecl BlkSz Dsorg Dsname
B10142 3390 2006/03/20 2 31 F 8080 PS CMN.CREF.WORK
ARCIVE Not Direct Access Device NI.NI6860.ERROR.PL.UNITTEST
B1N231 3390 2006/03/20 1 15 VB 256 27998 PO PLU
B1N192 3390 2006/03/03 1 10 U13680 13680 PS RZ1.A74A7198.BACKUP
B11298 3390 2006/03/20 11 FB 150 6000 PS RZ1.BRODCAST.DATA
B11199 3390 2006/03/03 1 10 U13680 13680 PS RZ1.ISR8062.BACKUP
ARCIVE Not Direct Access Device SQLXPLR.CNTL
B11256 3390 2006/03/17 1 62 FB 80 27920 PO TBOX.TABLES
ARCIVE Not Direct Access Device WOK.NI.NI6860.EXCEL.DIFFER
*
*/
// last two tokens describe file:
// 'PS': sequential file
// 'PO': PDS
// 'PO-E': PDS Library
// ' ': unknown, probably archived.
/*
* Format of a PDS
*0 12 34 5 6 7 8
* Name VV.MM Created Changed Size Init Mod Id
TBSHELF 01.03 2002/09/12 2002/10/11 09:371111 0 A338692
TBTOOL01.12 2002/09/12 2004/11/26 19:545128 0 A338692
*
*/
/**
* The sole constructor for a MVSFTPEntryParser object.
*
* @exception IllegalArgumentException
* Thrown if the regular expression is unparseable. Should not be seen
* under normal
