Re: [Dovecot] script to test CATENATE

2013-08-05 Thread Mike Abbott
 x append inbox catenate (url ;invalid; url {5}
 
 Dovecot replies with + OK because it wants to read all the URLs into memory 
 before parsing them, while catenate.pl expects an error message immediately.

I see that Example 4 in Appendix A of RFC 4469 explicitly allows both models.  
Here's a patch to catenate.pl to expect dovecot-2.2's behavior.



catenate-patch
Description: Binary data


Re: [Dovecot] script to test CATENATE

2013-08-04 Thread Timo Sirainen
On 23.7.2013, at 1.17, Mike Abbott michael.abb...@apple.com wrote:

 Attached please find a perl script which tests the CATENATE support in 
 dovecot.  I used this to test my CATENATE implementation a few years ago and 
 it runs fine against dovecot in OS X Server.  When run against dovecot-2.2.4 
 though it always fails or hangs, which in some cases means we interpreted 
 RFCs differently and in other cases means it's finding bugs; both conditions 
 are worthy of scrutiny.  (It's random-number driven so every run is 
 different.)  A couple months ago I reported a few simple bugs which this 
 script found and you fixed them; thanks.  Then it started finding problems 
 for which it's harder to isolate simple reproducible test cases.  Vacations 
 and other work interceded but now Apple is pleased to give you the script 
 itself to allow you to iterate faster.

Thanks. I finally tested this and fixed various bugs: 
http://hg.dovecot.org/dovecot-2.2/rev/3c2e1879fdf6

There is still one difference between what the script expects and what Dovecot 
does:

x append inbox catenate (url ;invalid; url {5}

Dovecot replies with + OK because it wants to read all the URLs into memory 
before parsing them, while catenate.pl expects an error message immediately. It 
doesn't look like this could be easily changed in Dovecot.



Re: [Dovecot] script to test CATENATE

2013-07-23 Thread Ed W

On 22/07/2013 23:17, Mike Abbott wrote:

Attached please find a perl script which tests the CATENATE support in dovecot. 
 I used this to test my CATENATE implementation a few years ago and it runs 
fine against dovecot in OS X Server.


Hi Mike

Do you think you might re-submit the matching BURL support to Postfix?  
It seems like it accidentally fell on the floor due to arriving at a bad 
moment some years back?


Cheers

Ed W


Re: [Dovecot] script to test CATENATE

2013-07-23 Thread Mike Abbott
 Do you think you might re-submit the matching BURL support to Postfix?

I don't think re-submitting is a good idea unless Wietse  co. request it, 
which I doubt will happen.


Re: [Dovecot] script to test CATENATE

2013-07-23 Thread Ed W

On 23/07/2013 14:30, Mike Abbott wrote:

Do you think you might re-submit the matching BURL support to Postfix?

I don't think re-submitting is a good idea unless Wietse  co. request it, 
which I doubt will happen.


My reading of it at the time was something like There are no clients 
that support this.  We don't understand the need


Now that there is at least one large client (I'm presuming that IOS does 
support it?) I think the world has changed and of course the patch has 
now had large scale testing (since I'm presuming again that it's 
included in the Apple distributed Postfix version?)


I personally think that the idea is perfect and I would like to see it 
break into mainstream use and from there I think we will possibly see 
support added to additional clients (I think this is how to break the 
chicken/egg cycle).  The idea that you can use IMAP commands to 
construct a message server side from bits of other messages, and then 
post it out server side is fantastic.


Please consider having at least one more go.  I think there is likely to 
be much better reception now that clients exist, the patch is well 
tested and Dovecot at least supports the IMAP side. Please...?


Cheers

Ed W


[Dovecot] script to test CATENATE

2013-07-22 Thread Mike Abbott
Attached please find a perl script which tests the CATENATE support in dovecot. 
 I used this to test my CATENATE implementation a few years ago and it runs 
fine against dovecot in OS X Server.  When run against dovecot-2.2.4 though it 
always fails or hangs, which in some cases means we interpreted RFCs 
differently and in other cases means it's finding bugs; both conditions are 
worthy of scrutiny.  (It's random-number driven so every run is different.)  A 
couple months ago I reported a few simple bugs which this script found and you 
fixed them; thanks.  Then it started finding problems for which it's harder to 
isolate simple reproducible test cases.  Vacations and other work interceded 
but now Apple is pleased to give you the script itself to allow you to iterate 
faster.

Here are some examples of it running.  One time only, pass the --init argument 
to store some template messages used by the real tests:
$ ./catenate.pl --host your.test.server --user testuser --password 1234 --init
connecting (imaps)...
capability...
logging in...
deleting old templates mailbox...
creating templates mailbox...
append1...
Append succeeded
append2...
Append succeeded
logout...
success
$ ./catenate.pl --host your.test.server --user testuser --password 1234
connecting (imaps)...
capability...
logging in...
append1...
Append failed as it should have (bad url): append1 NO [BADURL ;invalid;] 
Invalid IMAP URL: Unexpected IMAP URL path segment: `;invalid;'.
append2...
Append failed as it should have (bad url): append2 NO [BADURL 
/inbox/;uid=] Message not found.
append3...
Append succeeded
[...]
All tests passed.
$ 

If it concludes with anything other than All tests passed or hangs then it 
found something that it didn't expect and that should be examined.  Use the 
--verbose option to see the entire client-server conversation.

We hope that you find this script helpful to harden your CATENATE code.  I will 
be happy to answer any questions.

#!/usr/bin/perl

# Test CATENATE support in dovecot.
# Use --init to initialize the template files in the mail store.

# Copyright (c) 2013 Apple Inc.  All Rights Reserved.
# 
# @APPLE_LICENSE_HEADER_START@
# 
# This file contains Original Code and/or Modifications of Original Code
# as defined in and that are subject to the Apple Public Source License
# Version 2.0 (the 'License').  You may not use this file except in
# compliance with the License.  Please obtain a copy of the License at
# http://www.opensource.apple.com/apsl/ and read it before using this
# file.
# 
# The Original Code and all software distributed under the License are
# distributed on an 'AS IS' basis, WITHOUT WARRANTY OF ANY KIND, EITHER
# EXPRESS OR IMPLIED, AND APPLE HEREBY DISCLAIMS ALL SUCH WARRANTIES,
# INCLUDING WITHOUT LIMITATION, ANY WARRANTIES OF MERCHANTABILITY, FITNESS
# FOR A PARTICULAR PURPOSE, QUIET ENJOYMENT OR NON-INFRINGEMENT.  Please
# see the License for the specific language governing rights and
# limitations under the License.
# 
# @APPLE_LICENSE_HEADER_END@

use strict;
use IO::Socket::INET;
use Getopt::Long;
use IPC::Open3;
use Digest::HMAC_MD5;
use APR::Base64;
use feature 'state';

sub usage
{
	die EOT;
Usage: $0 --host imap-server --user name --password pw
Options:
	--appends n	number of APPENDs
	--bufsiz n	output buffer size
	--buftag	tag output buffer flushes
	--init		initialize template messages
	--mailbox box	append messages to this mailbox
	--messages n	messages per MULTIAPPEND
	--parts n	parts per message
	--quiet
	--select box	select this mailbox before appending
	--select-size n	virtual size of message with UID=1 in selected mailbox
	--store path	path to mail store for the user
	--verbose
EOT
}

my %opts;
GetOptions(\%opts,
'appends=i',
'bufsiz=i',
'buftag',
'host=s',
'init',
'mailbox=s',
'messages=i',
'parts=i',
'password=s',
'quiet',
'select=s',
'select-size=i',
'store=s',
'user=s',
'verbose',
) || usage();

$opts{appends} = 1000 unless defined($opts{appends});
usage() unless $opts{host};
$opts{mailbox} = INBOX unless defined($opts{mailbox});
usage() unless $opts{user};
usage() unless $opts{password};
if (defined($opts{select})  !defined($opts{'select-size'})) {
	print STDERR --select needs --select-size\n;
	usage();
}

my $checksizes = 0;
if (defined($opts{store})) {
	my $myhost = `hostname`;
	chomp $myhost;
	if ($opts{host} eq 'localhost' or $opts{host} eq $myhost) {
		$checksizes = 1;
	} else {
		die must run on server to check sizes\n;
	}
}

$| = 1;

# These must match the constants in cmd-append.c (dovecot  2.2).
# Change both copies (cmd-append.c and here) for better testing coverage.
# Not sure if these apply to dovecot = 2.2.
my $MAX_URL_LITERAL_SIZE = 2048;	# for testing: 20
my $MAX_CATENATE_MSG_SIZE = 4294967295;	# for testing: 800
my $MAX_CATENATE_PARTS = 50;		# for testing: 6

my $smallbody ='EOT';
Subject: small test message
Date: Thu, 03 Sep 2009 21:37:10 -0500 (CDT)
From: