Noel J. Bergman wrote:
I don't disagree with the need for real-world client compliance tests. For
example, I can tell you that we have an issue with Yahoo! web mail. It uses
an obsolete (deprecated) POP command. My point is that each test should
have a defined role within the test plan. The RFCs are one coverage area
that must be addressed, and real-world applications are another. And I
disagree with the idea of generating test script for RFC dictated behavior
by recording a live session. That just documents how that particular
implementation implemented the RFC.
+1. Clearly both RFC compliance and real-world compliance need to be
integrated into into the more formal testing framework that you propose.
Each James handler logs all protocol traffic at the DEBUG log level. That
is how I know about the problem with Yahoo! web mail, although I'm not
really keen to implement an obsolete command just for them. The place for
using a packet sniffer or proxy would be where we want to understand some
extension, such as one that might be found between Outlook and Exchange. A
modified version of tcpdump that can be set to filter on a particular
IP:port would be a good solution on supported platforms, since it would
require no changes of any kind. Another proxy option would be the TCP GUI
monitor program that is packaged with Axis. It acts as a local proxy,
recording the traffic and displaying each side of the protocol in a split
window. It requires configuring the local side of the protocol to work with
the proxy, bit it does work on any Java enabled platform, and with any
application.
Didn't know about the Axis option, though I've used a similar two-sided
GUI.
Hopefully I am expressing myself clearly.
--- Noel
-----Original Message-----
From: Serge Knystautas [mailto:sergek@;lokitech.com]
Sent: Saturday, October 19, 2002 1:05
To: James Developers List
Subject: Re: Testing: Plans, Scripts and Tools
Well, James can just log all the communication (you shouldn't need some
separate TCP analysis). If a transcript works, you extract the session and
add it as a test to always check moving forward. If it fais, well, we've
got something to fix in James and then set it as a test. :)
For my money, I think the hard part is not POP3 and IMAP... there is a
relatively small number of possible clients that you need to handle. SMTP
has a lot more implementation variations... again though, just keep your
James server at debug level and grab those sessions for test suites.
Serge Knystautas
Loki Technologies
http://www.lokitech.com/
----- Original Message -----
From: "Nathan Carter" <[EMAIL PROTECTED]>
To: "James Developers List" <[EMAIL PROTECTED]>
Sent: Saturday, October 19, 2002 12:39 AM
Subject: Re: Testing: Plans, Scripts and Tools
Noel,
In *addition to* the "RFC-compliance" tests, it seems necessary to me
(especially looking forward to IMAP, where real-world client compliance
may not be as strict as with POP3) to have some sort of
"session-recorder" that would make recording new scripts from real-world
clients almost trivial. I've been looking at building such a recorder
using a modified packet sniffer:
1) tcpflow is an extension to tcpdump that writes data portion of all
filtered packets to file. (*nix only)
2) jpcap is a java wrapper for libpcap/tcpdump that would allow us to
capture sessions from both Windows and *nix machines.
I'm thinking that I could use 2) to build a recorder to create the (C:
blah S:blah) test scripts that we're currently using. Does this sound
helpful to others?
Jmeter has a similar "recording" feature built on a simple local proxy,
but it's a bit painful to turn the proxy on/off and change your browser
settings every time you want to record. Thoughts?
Nathan
Noel J. Bergman wrote:
I have looked at some of the recent posts, and get the sense that the te
st
scripts are ad hoc. It seems to me that we ought to adopt a more formal
methodology, develop a test strategy, and base the initial majority of
scripts on testing for compliance with the RFCs. Additional scripts can
be
based upon specific needs, e.g., accommodating real-world compatibility
with
specific applications. We should have documented scripts that explain
what
the standard is, what any special deviation(s) might be, what is being
tested, and why.
For these "telnet-based" protocols, I believe that we can (and should)
leverage regular expressions, rather than just plain canned strings for
comparing. That way we can better handle response strings, and use data
from matched groups within our following commands. I am also of a mind
that
we should look at using BSF as a key component of our testing framework.
That would let us leverage simple templates as well as more complex
scripts,
including those adapted from other suites. Harmeet mentioned Python,
which
can be used with BSF: http://www.lonsteins.com/articles/jython-bsf.html.
Done properly, this becomes a very useful general purpose test bed,
customized for a given protocol by additional beans and scripts.
Lastly, we needn't have an NIH approach to this sort of external
testing. I
am sure that we can find other tests to supplement those that we build.
Thoughts?
--- Noel
--
To unsubscribe, e-mail: <mailto:james-dev-unsubscribe@;jakarta.apache.org>
For additional commands, e-mail: <mailto:james-dev-help@;jakarta.apache.org>
--
To unsubscribe, e-mail: <mailto:james-dev-unsubscribe@;jakarta.apache.org>
For additional commands, e-mail: <mailto:james-dev-help@;jakarta.apache.org>
-Nathan
--
To unsubscribe, e-mail: <mailto:james-dev-unsubscribe@;jakarta.apache.org>
For additional commands, e-mail: <mailto:james-dev-help@;jakarta.apache.org>
